Channel state information prediction method and device based on large language model

Abstract

The invention discloses a channel state information prediction method and device based on a large language model, and the method comprises the steps: obtaining and preprocessing a historical channel state information (CSI) sequence, and obtaining a CSI time feature; through a cross attention mechanism, embedding the CSI time features (as queries) and pre-training words of LLM into a dictionary (as keys and values) to carry out cross-modal semantic alignment, mapping the CSI features into a semantic space of the LLM, and fusing the CSI features with original features to generate fused channel features; and finally, inputting the fused features into LLM for inference, and generating final predicted CSI through a projection layer. According to the method, a modal gap is effectively bridged, the powerful sequence modeling capability of a large language model can be fully utilized, the accuracy and generalization capability of channel prediction are remarkably improved, and an effective technical approach is provided for overcoming the channel aging problem in wireless communication.

Description

Technical Field

[0001] This invention relates to the field of wireless communication technology, and more specifically, to a technique for predicting Channel State Information (CSI) using an artificial intelligence model, particularly a method and apparatus for predicting CSI based on a Large Language Model (LLM) and incorporating cross-modal semantic alignment. Background Technology

[0002] Massive Multiple-Input Multiple-Output (MIMO) technology is a core technology for improving the spectrum efficiency and connection reliability of modern wireless communication systems, such as 5G and future 6G. The performance of MIMO systems largely depends on the effectiveness of precoding techniques, and accurate precoding is highly dependent on real-time and accurate downlink CSI acquired by the base station.

[0003] In Frequency Division Duplex (FDD) systems, because the uplink and downlink operate on different frequency bands, the channel lacks reciprocity. Downlink CSI (Continuous Channel Indicator) measurement requires the user terminal to perform the measurement and feedback to the base station. This feedback delay introduces a delay that causes the base station to use outdated CSI, a problem known as "channel aging." In Time Division Duplex (TDD) systems, while channel reciprocity can be utilized to estimate downlink CSI through uplink probing, reducing feedback overhead, the time-varying nature of the channel, especially in scenarios with high-speed user movement (such as high-speed rail and vehicle-to-everything (V2X)), still results in processing delays between channel estimation and actual data transmission, making the channel aging problem severe. Channel aging leads to precoding beamforming mismatch, preventing signal energy from being accurately focused on the target user, thus causing significant performance degradation.

[0004] To address this challenge, channel prediction technology has emerged. Its core idea is to utilize the temporal correlation of historical CSI sequences to predict future CSI moments, allowing base stations to use the predicted future CSI for precoding in advance to compensate for system latency. Traditional channel prediction methods, such as those based on Kalman filtering or autoregressive models, have relatively simple models and struggle to capture high-dimensional, nonlinear channel dynamics.

[0005] In recent years, with the development of deep learning, methods such as Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN) and their variants Long Short-Term Memory Networks (LSTM), Gated Recurrent Units (GRU), and Transformers have been applied to CSI prediction and have demonstrated performance superior to traditional methods. These methods can automatically extract spatiotemporal features from CSI data. However, they typically require a large amount of labeled data for end-to-end training, and their generalization ability remains limited for unknown channel environments outside the training scenario.

[0006] Large Language Models (LLMs), such as the GPT and BERT series, have mastered powerful sequence modeling, pattern recognition, and contextual reasoning capabilities through large-scale self-supervised pre-training on massive amounts of text data. These capabilities make them highly promising for tasks involving sequential and structured data. Existing research has begun to explore applying LLMs to CSI prediction, for example, by designing specific embedding layers to feed CSI data into a fixed LLM backbone network, or by analogizing CSI prediction tasks to the "next word" generation task of LLMs. However, these existing attempts face a fundamental technical challenge: CSI is a high-dimensional complex numerical sequence representing physical channel characteristics; while the internal knowledge system of LLMs is built upon the semantic space of natural language. A significant "modal gap" exists between the two. Existing methods have failed to effectively address this modal mismatch, preventing the powerful pre-trained knowledge of LLMs from being fully and efficiently transferred and utilized in CSI prediction tasks, thus limiting further improvements in prediction performance. Summary of the Invention

[0007] This invention primarily addresses the technical problem of insufficient CSI prediction performance in existing technologies by providing a channel state information prediction method and apparatus based on a large language model. The aim is to solve the mode gap problem existing when directly applying LLM for CSI prediction in existing technologies, thereby improving the accuracy and generalization ability of CSI prediction.

[0008] The present invention addresses the aforementioned technical problems primarily through the following technical solution: a channel state information (CSI) prediction method based on a large-scale language model, applied in a wireless communication system, comprising: S1: Obtain the sequence of historical channel state information of the wireless channel over the past T time steps, where T is a preset time length; S2: Preprocess the historical channel state information sequence to obtain the temporal characteristics of the channel state information to be processed; S3: Align the temporal features of the channel state information with the pre-trained word embedding dictionary of a large language model across modalities in the semantic space to generate fused channel features, wherein the cross-modal alignment includes: S31: Through a cross-attention mechanism, the temporal features of the channel state information are mapped to the semantic space defined by the pre-trained word embedding dictionary to generate a text space representation aligned with the temporal features of the channel state information; wherein, the temporal features of the channel state information are used to generate a query matrix, and the pre-trained word embedding dictionary or its reduced-dimensional representation is used to generate a key matrix and a value matrix. S32: Adaptively fuse the text space representation with the channel state information time features to generate the fused channel features; In step S1, using the CSI time feature as the query can be understood as asking a question using the current channel state; using the LLM word embedding dictionary as the key and value can be understood as retrieving the answer from the entire semantic knowledge base of the LLM. The calculation process of cross-attention involves finding the semantic concept (key) most relevant to the current channel state (query) and weighting and summing its corresponding semantic representation (value) to obtain a vector representation (Y) in the text semantic space. t ij,cross This step is the core of "modal translation". The subsequent step S32 performs adaptive fusion to prevent the loss of the inherent and important original information of CSI during modal conversion. Through gating and other mechanisms, the model can dynamically decide whether to retain more of the original CSI numerical features or adopt more of the translated text semantic features in the final fused features, thereby achieving information complementarity and optimal performance.

[0009] S4: Input the fused channel features into the large language model for processing, so as to use the sequence modeling capability of the large language model to infer the future channel state information features; S5: Based on the output of the large language model, predictive channel state information for future moments is generated through a projection layer.

[0010] This invention constructs a complete processing pipeline from raw CSI data to the final prediction result. This scheme does not treat LLM as a simple sequence processor, but creatively introduces the core idea of ​​cross-modal alignment in step S3. The purpose of this step is to attempt to map numerical, physically driven CSI features to the abstract semantic space constructed from natural language, which LLM understands. In this way, the subsequent step S4 no longer involves LLM processing purely numerical sequences that it is unfamiliar with, but rather processing features that already carry "language-like" semantic information, which LLM is better at understanding. This fundamentally solves the "modal gap" problem mentioned in the background technology. The resulting technical effect is that LLM's powerful pre-trained knowledge (such as its ability to capture sequence dependencies and latent patterns) can be effectively activated and transferred to the CSI prediction task, thereby achieving more accurate predictions than existing technologies. The projection layer in step S5 is a necessary decoding step, mapping the inference results of LLM in the high-dimensional feature space back to the actual physical dimension of CSI, thereby improving the precoding performance of the wireless communication system and improving the overall system efficiency.

[0011] The pre-trained word embedding dictionary is acquired through self-supervised learning on massive natural language text corpora, in conjunction with the large-scale language model. This dictionary contains complex linguistic patterns, contextual relationships, and world knowledge captured through large-scale self-supervised learning. The rationale for choosing this dictionary is that complex channel dynamics exhibit a high-order isomorphism or similarity to certain abstract structural patterns in natural language sequences. Using this dictionary, rich in structural knowledge, as an alignment target, compared to other simpler knowledge bases, is more likely to find semantic representations that match the CSI dynamic patterns, thus producing superior prediction results.

[0012] Preferably, step S31 specifically includes: Before the cross-attention mechanism is executed, Principal Component Analysis (PCA) is first applied along the vocabulary dimension to reduce the dimensionality of the pre-trained word embedding dictionary to obtain a dimensionality-reduced representation, thereby reducing the computational complexity of cross-attention. Wherein, the text space represents Y t ij,cross Calculate using the following formula: Y t ij,cross =Softmax((QK T ) / sqrt(c))·V, where Q is the query matrix generated from the temporal features of the channel state information, K and V are the key matrix and value matrix generated from the dimensionality reduction representation, respectively, and c is a scalar used for scaling, whose value is the dimension of each attention head. sqrt(·) is the non-negative square root function.

[0013] Large language models typically have enormous vocabularies (e.g., tens of thousands), resulting in extremely high-dimensional word embedding dictionaries. Directly performing cross-attention calculations with such a high-dimensional dictionary incurs huge computational overhead and memory consumption, making the solution difficult to train and deploy in practice. The purpose of this invention is to propose a computationally feasible method. By using PCA for dimensionality reduction, the massive vocabulary can be compressed into a much smaller "core concept" dictionary. Although this dimensionality-reduced dictionary loses some information, it retains the most important semantic directions and structures in the original semantic space. Thus, without sacrificing the core semantic alignment capability, the computational complexity of cross-attention is reduced by several orders of magnitude, making the entire technical solution highly efficient and practical. The PCA step is only performed once before training, and its overhead is negligible during the entire training process.

[0014] Preferably, the adaptive fusion in step S32 is implemented through a gating mechanism, and the fused channel feature Y t ij,text Calculate using the following formula: Y t ij,text =Y t ij,cross ⊙+Y t ij,time ⊙(1-g), where Y t ij,cross For text space representation, Y t ij,time The channel state information time characteristics are represented by ⊙, which indicates element-wise multiplication, and g is a gate signal, which is generated by Y. t ij,cross and Y t ij,time After splicing, it is passed through a gated projection matrix W T gate The gating unit is generated using the Sigmoid activation function, and the formula is: g = Sigmoid(W T gate ([Y t ij,cross ;Y t ij,time ]), W T gate The gated projection matrix is ​​obtained through learning.

[0015] Compared to simple linear addition or feature concatenation, the gating mechanism endows the model with stronger non-linear expressive power. The gating signal g takes a value between 0 and 1, and can be understood as a soft switch or attention weight. When g is close to 1, the fused features are more biased towards the text space representation Y. t ij,crossWhen g approaches 0, it leans more towards the original CSI time characteristic Y. t ij,time Since g itself is determined by two features, the model can learn how to optimally balance the information of these two modes under different channel states, thereby achieving better performance than fixed-weight fusion.

[0016] Preferably, before inputting the fused channel features into the large language model, the method further includes: S33: Based on the fused channel features, retrieve several semantic prompts that are most relevant to the fused channel features from a set of preset semantic anchors; S34: The one or more semantic prompts are used as prefixes and concatenated with the fused channel features to form an enhanced input sequence; In step S4, the input to the large language model is the enhanced input sequence.

[0017] This approach introduces semantic prompting to further leverage the characteristics of LLMs to guide them towards more accurate predictions. LLMs typically perform better when processing inputs with explicit context or task instructions (i.e., prompts). Semantic prompts play this role. Instead of human-designed natural language instructions, they are semantic instructions retrieved from a compact semantic anchor space that best match the current channel state. In simpler terms, before telling the LLM "this is the current channel characteristic," it assigns a label, such as "this might be a channel pattern in a high-speed mobile scenario" or "this is similar to a specific multipath fading pattern" (these semantic labels are represented by learnable anchor vectors e'). This input sequence with contextual prompts helps the LLM focus on relevant knowledge and patterns more quickly, leading to more accurate inferences and further improving prediction performance.

[0018] Preferably, in step S33, the relevance is determined by calculating the cosine similarity between the fused channel features and each anchor point in the set of preset semantic anchor points; one or more semantic anchor points with the highest cosine similarity are selected as the semantic cues. Wherein, the cosine similarity γ(Y) t ij,text ,e') is calculated using the following formula: γ(Y t ij,text ,e')=(Y t ij,text ·e') / (||Y t ij,text||·||e'||), where Y t ij,text Here, e' represents the fused channel features, and e' is the semantic anchor point.

[0019] The reason for choosing cosine similarity is that it measures the similarity of two vectors in direction, without concern for their absolute magnitude. This is a common and effective measure of semantic similarity in high-dimensional feature spaces. By finding the anchor point with the highest cosine similarity, we can efficiently find the cue that is semantically closest to the current channel state.

[0020] Preferably, the preprocessing in step S2 specifically includes: S21: Separate the complex values ​​of the historical channel state information sequence into real and imaginary parts, and concatenate them into a real tensor; S22: Normalize the real tensor; S23: In order to capture local time features and reduce computational complexity, the historical channel state information sequence is divided into blocks along the time dimension to obtain several data blocks.

[0021] Separating the real and imaginary parts is necessary because neural networks typically handle real-valued inputs. Normalization (e.g., Z-score normalization) scales the data to a suitable range, preventing vanishing or exploding gradients and accelerating model convergence. Block processing aims to capture local temporal dependencies while reducing the computational complexity of subsequent processing modules (such as attention mechanisms). Since the computational cost of attention mechanisms is proportional to the square of the sequence length, dividing long sequences into shorter blocks effectively controls computational costs.

[0022] Preferably, step S2 further includes: S24: Attention is calculated between the spatial dimensional features within each data block using the Cross Spatial Self-Attention (CSSA) module, thereby enhancing the spatial correlation expression of channel features and obtaining spatially enhanced data blocks; The cross-space self-attention module includes a backbone path and a residual connection; the backbone path includes, in sequence: A first convolutional layer and a second convolutional layer, wherein the first convolutional layer receives the original input and outputs to the second convolutional layer and the residual connection; A channel attention unit is used to generate channel attention weights based on the output of the second convolutional layer and to weight the output features of the second convolutional layer; it receives the output of the second convolutional layer (conv(2)) and generates a channel attention weight through a pooling layer, two fully connected layers (FC(1), FC(2)) and a sigmoid activation function; the weight is applied to the output of the second convolutional layer through channel-wise multiplication to reweight the feature channels of the backbone path; A multi-head self-attention layer is used to process the weighted features; The residual connection adds the output of the first convolutional layer to the output of the multi-head self-attention layer to obtain the output of the cross-space self-attention module.

[0023] The CSSA module effectively extracts the complex spatiotemporal correlations inherent in CSI data. CSI data exhibits complex coupling relationships both spatially (between different antennas / subcarriers) and temporally. The brilliance of the CSSA module lies in: 1) convolutional layers capturing local spatial patterns; 2) channel attention units (such as SE-Block) adaptively learning the importance of different feature channels and enhancing key features; 3) multi-head self-attention layers capturing global, long-range spatial dependencies; and 4) residual connections avoiding the vanishing gradient problem during deep network training, ensuring effective information flow. Combining these components allows for a more comprehensive and efficient extraction of meaningful deep features from raw CSI compared to a single network structure, providing higher-quality input for subsequent cross-modal alignment and thus improving overall prediction performance.

[0024] Preferably, step S2 further includes: S25: After reshaping the spatially enhanced data block, process it using a multi-head self-attention mechanism (MHSA) to capture the time-series dependencies of the data block and generate the CSI time feature Y to be processed. t ij,time ; Wherein, the CSI time feature Y t ij,time Calculate using the following formula: Y t ij,time =MHSA(Y t ij In the formula, Y t ij This is the reshaped representation of the data block that has undergone spatial feature enhancement.

[0025] Reshaping refers to performing a dimensional transformation operation on the data block that has undergone spatial feature enhancement and has spatial dimension, block number dimension, and intra-block time step dimension, in order to reconstruct it into a two-dimensional sequence tensor. One dimension of the two-dimensional sequence tensor is the time series dimension formed by merging the block number dimension and the intra-block time step dimension, and the other dimension is the feature dimension formed by the spatial dimension.

[0026] CSSA primarily handles feature extraction in the spatial dimension, while MHSA here specifically captures long-range dependencies in the temporal dimension. After CSSA processing, the data block first undergoes dimensional transformation (reshaping) to transform it from a multi-dimensional format (e.g., [spatial dimension, number of blocks, intra-block time]) into a standard two-dimensional sequence format ([total time steps, feature dimension]). Then, MHSA performs self-attention computation on this time series, effectively modeling the relationship between any two time steps in the sequence, which is crucial for predicting the long-term evolution trend of the channel. This step ensures that the CSI temporal features fed into the cross-modal alignment module simultaneously contain rich spatial and temporal dependency information.

[0027] The large-scale language model used in this scheme is a pre-trained language model that utilizes Low-Rank Adaptation (LoRA) technology for efficient parameter fine-tuning.

[0028] A channel state information prediction device based on a large language model, comprising: The acquisition module is used to acquire the historical channel state information sequence of the wireless channel over the past T time steps, where T is a preset time length; The preprocessing module is used to preprocess the historical channel state information sequence to obtain the temporal characteristics of the channel state information to be processed. A cross-modal alignment module is used to perform cross-modal alignment of the channel state information temporal features with a pre-trained word embedding dictionary of a large language model in the semantic space to generate fused channel features. The cross-modal alignment module is configured as follows: Through a cross-attention mechanism, the temporal features of the channel state information are mapped to the semantic space defined by the pre-trained word embedding dictionary, generating a text space representation aligned with the temporal features of the channel state information. The temporal features of the channel state information are used to generate a query matrix, and the pre-trained word embedding dictionary or its reduced-dimensional representation is used to generate a key matrix and a value matrix. The text spatial representation is adaptively fused with the temporal features of the channel state information to generate the fused channel features; The prediction processing module is used to input the fused channel features into the large language model for processing, so as to use the sequence modeling capability of the large language model to infer the future channel state information features; The result generation module is used to generate predicted channel state information for future times through a projection layer based on the output of the large language model.

[0029] Preferably, the channel state information prediction device based on a large language model further includes: The semantic prompt generation module is used to retrieve one or more of the most relevant semantic prompts from a set of preset semantic anchors based on the fused channel features before the prediction processing module processes the fused channel features. An input sequence construction module is used to concatenate the one or more semantic prompts as prefixes with the fused channel features to form an enhanced input sequence; The prediction processing module is configured to input the enhanced input sequence into the large language model.

[0030] The substantial effects of this invention are: 1. Significantly Improves Prediction Accuracy and Generalization Ability, Effectively Overcoming Channel Aging: The core advantage of this invention lies in its innovative cross-modal semantic alignment mechanism, which successfully and efficiently transfers the powerful sequence modeling and pattern reasoning capabilities gained from pre-training large-scale language models on massive text data to physical channel prediction tasks. As shown in the experimental data of subsequent embodiments of this invention, whether in conventional communication scenarios, data-sparse few-shot learning scenarios, or zero-shot generalization scenarios facing unknown channel environments, the prediction error (e.g., Normalized Mean Square Error, NMSE) of the proposed solution is significantly lower than that of traditional deep learning methods (such as CNN, LSTM, etc.) and other baseline methods that directly apply LLM. This high-precision prediction capability enables communication systems to obtain more accurate future channel states, thereby enabling more effective precoding and beamforming, directly combating the negative impact of channel aging, and ultimately resulting in higher system throughput and more reliable communication connections.

[0031] 2. Pioneeringly Solving the "Modality Gap" and Achieving Efficient Cross-Domain Knowledge Transfer: The substantial contribution of this invention lies in not simply treating LLM as a black-box tool, but designing a complete and feasible "translation" mechanism (cross-attention alignment and semantic cues) that maps abstract, continuous CSI numerical features to a discretized semantic space that LLM can understand. This fundamentally solves the modality mismatch problem encountered when applying pre-trained language models to physical layer signal processing. This mechanism not only enables the effective utilization of LLM knowledge, but more importantly, it provides a novel and universally applicable technical path for using prior knowledge acquired in domain A (such as natural language) to solve problems in domain B (such as wireless communication), demonstrating unexpected technical insights.

[0032] 3. Balancing high performance and computational feasibility, with practical application value: Addressing the significant computational challenges posed by directly interacting high-dimensional CSI features with the massive word embedding dictionary of an LLM, this invention proposes a strategy for dimensionality reduction of the dictionary based on Principal Component Analysis (PCA), a highly efficient and ingenious engineering solution. This strategy reduces the computational complexity of key steps by several orders of magnitude while preserving core semantic information, enabling the entire high-performance prediction model to be trained and deployed with reasonable hardware resources and time costs. This overcomes the drawback of many advanced algorithms being "theoretically feasible but prohibitively expensive in practice," giving this invention real potential for application in future 5G-Advanced and 6G communication systems. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of a channel state information prediction method based on a large language model according to the present invention; Figure 2 This is a schematic diagram of a cross-space attention module structure according to the present invention; Figure 3 This is a schematic diagram of a cross-modal alignment process according to the present invention; Figure 4 This describes the performance of various solutions under a TDD system at different user speeds. Figure 5 This shows the performance of different schemes at different speeds under few-shot learning. Figure 6 This represents the performance of different approaches under transfer learning; Figure 7 These are the results of an ablation experiment; Figures 4-7 In this embodiment, CSI-ALM is the channel state information prediction method based on a large language model. Detailed Implementation

[0034] The technical solution of the present invention will be further described in detail below through embodiments and in conjunction with the accompanying drawings.

[0035] Example: A pipeline for a channel state information prediction method based on a large language model proposed in this invention is as follows: Figure 1 As shown, the entire device consists of a CSI data preprocessing module with semantic alignment mechanism, a large language model backbone fine-tuned using LoRA technology, and a final output projection module.

[0036] The input uplink CSI sequence first undergoes a series of preprocessing steps. However, directly inputting high-dimensional sequences into the network leads to significant computational complexity and long training time, especially when the number of antennas and subcarriers is large. To effectively address this issue, this invention proposes a parallel antenna processing method, which processes each transmit / receive antenna pair (… i,j The CSI of each ) is predicted separately, where i and j These represent the indices of the transmitting and receiving antennas, respectively. Specifically, the input sequence H... t,T u,ij ∈C F×T Based on the current t The first moment i The transmitting antenna and the first j Between the receiving antennas in the past T CSI sequences within each time step: H t,T u,ij =[H t-1 u,ij H t-2 u,ij ,…,H t-T u,ij ]; Where H t-p u,ij The transmitting antenna is at time tp. i and receiving antenna j CSI, F This indicates the number of subcarriers in each antenna pair. It should be noted that... T It is the time length of the historical CSI used for prediction.

[0037] Since neural networks typically deal with real numbers, this invention separates and concatenates the real and imaginary parts of H. t,T u,ij Convert to real tensor X t ij ∈R 2×F×T And further normalization processing: X` t ij =(X tij -μ) / σ; Where μ and σ represent the mean and standard deviation of the real-valued sequence within the training batch, respectively.

[0038] Then, X` t ij Remodeled into a 2-dimensional structure F × T The tensor. To capture local temporal features and reduce computational complexity, this invention applies block operations along the time dimension, specifically as follows: Figure 1 As shown. X` t ij Divided into sizes of N The non-overlapping blocks yield tensor Y'. t ij ∈R 2F×N×T' Where T' = ceil(T / N) represents the number of blocks. If the last block is not completely filled, zero-filling is performed.

[0039] After serializing the CSI through block operations, deep features are derived from the latent channel features output by the embedding module using the Cross Spatial Self-Attention (CSSA) mechanism. The specific structure is as follows: Figure 2 As shown: Y t ij =Reshape(CSSA(Y' t ij )); Multi-Head SelfAttention (MHSA) is applied to the latent channel features output by the embedding module to derive the projected time series: Y t ij,time =MHSA(Y t ij ); Then, Y t ij,time With the word embedding dictionary D∈R of a pre-trained large language model |A|×D' Alignment is performed, where |A| is the vocabulary size and D' is the embedding length. However, |A| is typically very large (e.g., the vocabulary size of GPT-2 is 50257), making direct cross-attention alignment computationally difficult. To address this issue, this invention applies Principal Component Analysis (PCA) along the vocabulary dimension to reduce its dimensionality. This results in a compact dictionary composed of linearly combined embeddings, rather than a complete set, thus significantly reducing the number of entries. D^=PCA(D), d≪|A|; where d is a predefined reduced dictionary size. It's worth noting that the PCA step is performed only once before training, and its overhead during optimization is negligible.

[0040] Subsequently, this invention applies multi-head cross-attention, where D^ is used to generate the key and value matrices, and Y... t ij,time Used to generate a query matrix. This aligns the subject embedding with the temporal latent CSI representation to obtain the aligned text representation Y. t ij,cross ∈R D×T : Y t ij,cros =softmax((QK T ) / sqrt(c))·V; Where Q=Y t ij,time ·W Q K=D^·W K V=D^·W V , respectively, are the projection matrices of the query (Q), key (K), and value (V) matrices, W. Q W K and W V This is a learnable weight matrix used to project Q, K, and V into the corresponding space. The scalar c is a scaling factor, taking values ​​corresponding to the dimension of each attention head. Then, the aligned text representation and the temporal representation are adaptively combined to generate the final representation Y. t ij,text : g = Sigmoid(W T gate ([Y t ij,cross ;Y t ij,time ]), Y t ij,text =Y t ij,cross ⊙+Y t ij,time ⊙(1-g); Among them W T gate This is the learned gated projection matrix, where ⊙ denotes element-wise multiplication. The cross-modal alignment process is as follows: Figure 3 As shown.

[0041] After cross-modal alignment, the text-based CSI representation Y t ij,text With semantic hints eSimilarly, this invention transforms the pre-trained word embedding dictionary into a compact set of semantic anchors D'^, where... Unlike previous methods employing PCA, this invention uses a learnable linear layer for this transformation: D'^=Linear(D^); To enrich CSI embeddings, this invention applies cosine similarity-based score matching to retrieve the most relevant semantic anchors. e ′, which means maximizing Y t ij,text and e Cosine similarity of ′: Maxγ(Y t ij,text ,e')=(Y t ij,text ⋅e') / (||Y t ij,text ||·||e'||); Then, the present invention selects the top 4 embedding vectors with the highest scores as semantic cues. e 1:K′ The semantic prompt length can be adjusted to suit different task requirements. These selected prompts are used as prefixes with Y. t ij,text The data is spliced ​​together to form an enhanced representation Z. t ij Then it is fed into the backbone of a large language model.

[0042] A large language model maps optimized latent CSI sequences to predictions of current CSI features, which are then passed through a projection layer to generate the final CSI prediction. H t d,ij =Proj(LLM(Z t ij )); Where H t d,ij This represents the final predicted downlink CSI at time t. Proj is a linear projection layer used to adjust the sequence length of the actual output, and the output dimension can be adjusted according to specific needs.

[0043] To demonstrate the performance of this invention compared to current deep learning methods, the following experiments were conducted on a time-varying CSI dataset conforming to the 3GPP standard generated by the channel generator QuaDRiGa: Experimental environment: Linux operating system 20.04.5; deep learning framework PyTorch 1.10, Python version 3.8; loss function is the NMSE of predicted CSI and true CSI, optimized using the Adam optimizer. This embodiment's evaluation criteria are based on a comparison with advanced methods in channel prediction, which are: CNN: W. Xia, G. Zheng, Y. Zhu, J. Zhang, J. Wang, and AP Petropulu, "A deep learning framework for optimization of MISO downlink beam-forming," IEEE Trans. Commun., vol. 68, no. 3, pp. 1866–1880, Mar.2020. RNN: W. Jiang and HD Schotten, “Neural network-based fading channel prediction: A comprehensive overview,” IEEE Access, vol. 7, pp. 112–118 124, 2019. LSTM: W. Jiang and HD Schotten, “Deep Learning for Fading Channel Prediction,” IEEE Open J. Commun. Soc., vol. 1, pp. 320–332, 2020. GRU: I. Helmy, P. Tarafder, and W. Choi, “LSTM-GRU model-basedchannel prediction for one-bit massive MIMO system,” IEEE Trans. Veh. Tech.,vol. 72, no. 8, pp.11 053–11 057, Aug. 2023. Transformer: H. Jiang, M. Cui, DWK Ng, and L. Dai, “Accuratechannel prediction based on transformer: Making mobility negligible,” IEEE J.Sel. Areas Commun., vol. 40, no. 9, pp. 2717–2732, Sep. 2022. LLM4CP: B. Liu, X. Liu, S. Gao, X. Cheng, and L. Yang, “LLM4CP: Adapting large language models for channel prediction,” J.Commun. Inf. Netw.,vol. 9, no. 2, pp. 113–125, 2024. like Figure 4 As shown, the results demonstrate that the proposed method further improves performance by introducing a modality alignment mechanism. This mechanism effectively bridges the gap between CSI features and linguistic knowledge, exhibiting superior feature extraction and representation capabilities, significantly outperforming traditional deep learning methods.

[0044] like Figure 5 As shown, the results in the few-shot scenario demonstrate that the channel prediction capability of all methods decreases under limited data conditions. However, the method proposed in this invention—utilizing rich pre-trained knowledge—achieves significantly better performance under these constraints. Furthermore, the zero-shot generalization capability of all models is evaluated by training all models in the UMa scenario and directly applying them to the UMi scenario while keeping all other settings unchanged. Figure 6 The results show that when applied to channel prediction, the LLM-based method retains the strong generalization ability common in NLP tasks.

[0045] Compared to traditional methods, the fundamental advantage of LLM-based CSI prediction lies in its ability to leverage pre-trained knowledge. To further analyze this aspect, this invention compares three different initialization strategies: (1) random initialization, (2) initialization from a pre-trained base model, and (3) removal of the LLM backbone. Figure 7As shown, initialization using a pre-trained base model yields the best performance, significantly outperforming the other two methods. Randomly initialized models lack access to pre-trained knowledge, rendering the modality alignment mechanism ineffective. Therefore, their performance is comparable to models without an LLM backbone. These results further confirm the crucial role of pre-trained knowledge in improving the effectiveness of channel prediction.

[0046] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

[0047] Although this document uses various terms extensively, the possibility of using other terms is not excluded. These terms are used merely for the convenience of describing and explaining the essence of the invention; interpreting them as any additional limitation would contradict the spirit of the invention.

Claims

1. A channel state information prediction method based on a large-scale language model, characterized in that, Include: S1: Obtain the sequence of historical channel state information of the wireless channel over the past T time steps, where T is a preset time length; S2: Preprocess the historical channel state information sequence to obtain the temporal characteristics of the channel state information to be processed; S3: Align the temporal features of the channel state information with the pre-trained word embedding dictionary of a large language model across modalities in the semantic space to generate fused channel features, wherein the cross-modal alignment includes: S31: Through a cross-attention mechanism, the temporal features of the channel state information are mapped to the semantic space defined by the pre-trained word embedding dictionary to generate a text space representation aligned with the temporal features of the channel state information; wherein, the temporal features of the channel state information are used to generate a query matrix, and the pre-trained word embedding dictionary or its reduced-dimensional representation is used to generate a key matrix and a value matrix. S32: Adaptively fuse the text space representation with the channel state information time features to generate the fused channel features; S4: Input the fused channel features into the large language model for processing, so as to use the sequence modeling capability of the large language model to infer the future channel state information features; S5: Based on the output of the large language model, predictive channel state information for future moments is generated through a projection layer.

2. The channel state information prediction method based on a large-scale language model according to claim 1, characterized in that, Step S31 specifically includes: Before the cross-attention mechanism is executed, principal component analysis (PCA) is first applied along the vocabulary dimension to reduce the dimensionality of the pre-trained word embedding dictionary to obtain a dimensionality-reduced representation. Wherein, the text space represents Y t ij,cross Calculate using the following formula: Y t ij,cross =Softmax((QK T ) / sqrt(c))·V, where Q is the query matrix generated from the time features of the channel state information, K and V are the key matrix and value matrix generated from the dimensionality reduction representation, respectively, and c is a scalar used for scaling, the value of which is the dimension of each attention head.

3. The channel state information prediction method based on a large language model according to claim 2, characterized in that, The adaptive fusion in step S32 is achieved through a gating mechanism, and the fused channel feature Y t ij,text Calculate using the following formula: Y t ij,text =Y t ij,cross ⊙+Y t ij,time ⊙(1-g), where Y t ij,cross For text space representation, Y t ij,time The channel state information time characteristics are represented by ⊙, which indicates element-wise multiplication, and g is a gate signal, which is generated by Y. t ij,cross and Y t ij,time After splicing, it is passed through a gated projection matrix W T gate The gating unit is generated using the Sigmoid activation function, and the formula is: g = Sigmoid(W T gate ([Y t ij,cross ;Y t ij,time ]), W T gate The gated projection matrix is ​​obtained through learning.

4. A channel state information prediction method based on a large language model according to any one of claims 1 to 3, characterized in that, Before inputting the fused channel features into the large language model, the process also includes: S33: Based on the fused channel features, retrieve several semantic prompts that are most relevant to the fused channel features from a set of preset semantic anchors; S34: The one or more semantic prompts are used as prefixes and concatenated with the fused channel features to form an enhanced input sequence; In step S4, the input to the large language model is the enhanced input sequence.

5. The channel state information prediction method based on a large language model according to claim 4, characterized in that, In step S33, the relevance is determined by calculating the cosine similarity between the fused channel features and each anchor point in the set of preset semantic anchor points; one or more semantic anchor points with the highest cosine similarity are selected as the semantic cue. Wherein, the cosine similarity γ(Y) t ij,text ,e') is calculated using the following formula: γ(Y t ij,text ,e')=(Y t ij,text ·e') / (||Y t ij,text ||·||e'||), where Y t ij,text Here, e' represents the fused channel features, and e' is the semantic anchor point.

6. A channel state information prediction method based on a large language model according to any one of claims 1 to 5, characterized in that, The preprocessing in step S2 specifically includes: S21: Separate the complex values ​​of the historical channel state information sequence into real and imaginary parts, and concatenate them into a real tensor; S22: Normalize the real tensor; S23: In order to capture local time features and reduce computational complexity, the historical channel state information sequence is divided into blocks along the time dimension to obtain several data blocks.

7. The channel state information prediction method based on a large language model according to claim 6, characterized in that, Step S2 further includes: S24: Attention is calculated between the spatial dimensional features within each data block using a cross-spatial self-attention module to obtain spatially enhanced data blocks; The cross-space self-attention module includes a main path and a residual connection; The main pathway includes, in sequence: A first convolutional layer and a second convolutional layer, wherein the first convolutional layer receives the original input and outputs to the second convolutional layer and the residual connection; A channel attention unit is used to generate channel attention weights based on the output of the second convolutional layer and to weight the output features of the second convolutional layer. A multi-head self-attention layer is used to process the weighted features; The residual connection adds the output of the first convolutional layer to the output of the multi-head self-attention layer to obtain the output of the cross-space self-attention module.

8. The channel state information prediction method based on a large language model according to claim 7, characterized in that, Step S2 further includes: S25: After reshaping the spatially enhanced data block, it is processed through a multi-head self-attention mechanism to capture the time-series dependencies of the data block and generate the CSI time feature Y to be processed. t ij,time ; Wherein, the CSI time feature Y t ij,time Calculate using the following formula: Y t ij,time =MHSA(Y t ij In the formula, Y t ij This is the reshaped representation of the data block that has undergone spatial feature enhancement.

9. A channel state information prediction device based on a large-scale language model, characterized in that, Include: The acquisition module is used to acquire the historical channel state information sequence of the wireless channel over the past T time steps, where T is a preset time length; The preprocessing module is used to preprocess the historical channel state information sequence to obtain the temporal characteristics of the channel state information to be processed. A cross-modal alignment module is used to perform cross-modal alignment of the channel state information temporal features with a pre-trained word embedding dictionary of a large language model in the semantic space to generate fused channel features. The cross-modal alignment module is configured as follows: Through a cross-attention mechanism, the temporal features of the channel state information are mapped to the semantic space defined by the pre-trained word embedding dictionary, generating a text space representation aligned with the temporal features of the channel state information. The temporal features of the channel state information are used to generate a query matrix, and the pre-trained word embedding dictionary or its reduced-dimensional representation is used to generate a key matrix and a value matrix. The text space representation is adaptively fused with the channel state information temporal features to generate the fused channel features; The prediction processing module is used to input the fused channel features into the large language model for processing, so as to use the sequence modeling capability of the large language model to infer the future channel state information features; The result generation module is used to generate predicted channel state information for future times through a projection layer based on the output of the large language model.

10. A channel state information prediction device based on a large-scale language model according to claim 9, characterized in that, Also includes: The semantic prompt generation module is used to retrieve one or more of the most relevant semantic prompts from a set of preset semantic anchors based on the fused channel features before the prediction processing module processes the fused channel features. An input sequence construction module is used to concatenate the one or more semantic prompts as prefixes with the fused channel features to form an enhanced input sequence; The prediction processing module is configured to input the enhanced input sequence into the large language model.

Images