Design and training method of wireless base large model

Abstract

The invention discloses a design and training method of a wireless pedestal large model, a network architecture of the designed pedestal large model and a self-supervised training method based on a mask auto-encoder, and the self-supervised training method can effectively capture a wireless space-time-frequency three-dimensional relation of channel state information of a wireless communication system. The pre-trained wireless base large model is used for reasoning a channel estimation task, a time domain channel prediction task and a frequency domain channel prediction task of a wireless communication system, the accuracy of channel estimation and prediction can be improved, and the working cost is reduced.

Description

Technical Field

[0001] The present invention belongs to the field of wireless communication technology, and specifically relates to a design and training method for a wireless base large model. The present invention is a design scheme for a base large model based on a masked autoencoder applied to a MIMO-OFDM wireless communication system. The constructed base large model is self-supervised pre-trained on a large-scale CSI data set, thereby achieving high-precision and strong generalization channel estimation and channel prediction. Background Art

[0002] Massive Multiple-Input Multiple-Output (mMIMO) technology and Orthogonal frequency-division multiplexing (OFDM) technology are two basic technologies widely used in the fifth generation mobile communication system (5G). In MIMO-OFDM systems, accurate channel state information (CSI) is crucial for tasks including precoding, beamforming, power allocation, modulation selection, and transmit antenna selection. Channel estimation and channel prediction are two basic ways to obtain CSI. The former generally estimates CSI through received pilot signals, while the latter generally predicts unknown CSI in the time domain and frequency domain based on known partial CSI.

[0003] Due to its strong nonlinear fitting ability, deep learning technology has been widely used in the field of channel estimation and channel prediction, such as convolutional neural networks (CNN) and Transformer networks. However, there are some defects in the application of existing deep learning technology in channel estimation and prediction tasks. 1) Due to the limitation of model scale, the estimation accuracy and prediction accuracy are insufficient; 2) Training is performed on a single data set with a fixed configuration, and retraining is required when the data distribution or system parameters change; 3) Each task is designed separately, and multiple models need to be deployed simultaneously on the base station side, which greatly increases the storage and computing overhead of the model.

[0004] The Foundation Model has achieved great success in fields such as natural language processing. That is, after self-supervised pre-training on large-scale data sets, neural networks with large-scale parameters can demonstrate excellent few-sample learning capabilities on downstream tasks and surpass specially designed small models. The Foundation Model provides a new solution to the problems of existing deep learning technologies in channel estimation and prediction tasks. Summary of the invention

[0005] The present invention proposes a design and training method for a large wireless base model, which is a design scheme for a large wireless base model based on masked autoencoders (MAE). The network architecture of the large base model and a self-supervised training method based on MAE are designed. The pre-trained large wireless base model can be directly used for inference on the channel estimation tasks, time domain and frequency domain channel prediction tasks of the wireless communication system, thereby improving the accuracy of channel estimation and prediction and reducing work costs.

[0006] The network architecture of the base large model proposed in the present invention can adapt to CSI data of different sizes. The self-supervised training method proposed in the present invention can effectively capture the three-dimensional relationship of space-time-frequency of the channel state information of the wireless communication system. The pre-trained base large model can be directly applied to channel estimation and channel prediction tasks.

[0007] To achieve the above objectives, the present invention designs a network architecture based on a masked autoencoder MAE, including modules such as an encoder and a decoder. At the same time, a variety of pre-training tasks such as mask reconstruction are designed, so that the network can perform effective self-supervised learning (pre-training) to obtain a pre-trained base model. Finally, the pre-trained base model is directly applied to channel estimation and channel prediction tasks, and CSI data of different sizes can be processed.

[0008] The design scheme of the base large model based on the masked autoencoder of the present invention comprises the following steps:

[0009] 1) Build a large-scale wireless base station model network architecture based on MAE

[0010] The network architecture includes an embedding module, a mask module, an encoder module, a decoder module, and an output module. The CSI data is first converted into a series of tokens through the embedding module, and then the mask module selectively retains a portion of the tokens and inputs them into the encoder module. The output of the encoder module is sequentially concatenated with some learnable mask features and input into the decoder. The decoder output is restored to complete CSI data through the output module.

[0011] 2) Designed a variety of pre-training tasks such as mask reconstruction and interpolation denoising to pre-train the large wireless base model;

[0012] Two types of pre-training tasks are designed. The first type is mask reconstruction, which is to mask some tokens in a certain method and proportion in the mask module, and perform training by restoring and reconstructing the complete CSI. The second type is interpolation denoising, which is to downsample the original complete CSI and then interpolate to obtain a rough estimate, and then estimate the original accurate CSI through the base large model.

[0013] 3) Apply the pre-trained base model to channel estimation and channel prediction tasks

[0014] For the channel estimation task, it can be converted into an interpolation denoising task for processing. For the time domain and frequency domain channel prediction tasks, they can be converted into time domain and frequency domain mask recovery tasks for processing respectively.

[0015] Compared with the prior art, the present invention has the following beneficial effects:

[0016] The proposed wireless base station large model design scheme based on masked autoencoder can perform efficient self-supervised pre-training on large-scale data sets with inconsistent CSI sizes, and can be directly applied to channel estimation and prediction tasks after training. It can greatly improve the accuracy of the model in channel estimation and channel prediction tasks, and has strong generalization ability, and can process CSI data of different sizes and distributions.

[0017] The large base model design scheme based on masked autoencoder proposed in the present invention has the following technical advantages:

[0018] (I) Improved estimation and prediction accuracy: The base model has acquired strong CSI characterization capabilities through pre-training on massive data sets, which can significantly improve the accuracy of channel estimation and prediction.

[0019] (ii) Enhanced generalization capability: The base large model is able to achieve strong performance on CSI distributions not seen in the training set without the need for retraining.

[0020] (iii) Reduced model deployment overhead: The large base model can simultaneously process CSI data of different configurations and different types of channel estimation and prediction tasks, greatly reducing the number of models required on the base station side, thereby reducing model deployment overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a structural block diagram of a large wireless base model based on a masked autoencoder constructed by the present invention. DETAILED DESCRIPTION

[0022] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] The present invention provides a design and training method for a large model of a wireless base, designs a network architecture of the large model of the base and a self-supervised training method based on MAE, the self-supervised training method can effectively capture the three-dimensional relationship of space-time-frequency, and the pre-trained large model of the wireless base can directly perform reasoning on the channel estimation task, time domain and frequency domain channel prediction tasks of the wireless communication system, which can improve the accuracy of channel estimation and prediction and reduce work costs.

[0024] In specific implementation, the present invention is applied to a MIMO-OFDM system, in which the base station side is equipped with a planar array multi-antenna, and the user side is equipped with a single antenna. The proposed base station large model is deployed on the base station side, and can simultaneously process the channel estimation and channel prediction tasks of the "time-frequency-space" 3D CSI. Among them, channel estimation refers to the CSI estimation value of the known partial position, and the complete CSI is restored. The channel prediction task includes time domain channel prediction and frequency domain channel prediction, which refers to predicting the unknown CSI through partial CSI in the time and frequency directions respectively.

[0025] The present invention includes three steps: network construction, pre-training task design and deployment application. The specific steps are as follows:

[0026] S10: Construct a large-scale wireless base model network architecture based on MAE, including an embedding module, a mask module, an encoder module, a decoder module and an output module.

[0027] S20: We design a variety of self-supervised pre-training tasks such as mask reconstruction and interpolation denoising. The proposed large base model can obtain strong CSI representation capabilities after pre-training.

[0028] S30: Apply the pre-trained wireless base station large model to channel estimation and channel prediction tasks to obtain better estimation and prediction accuracy with strong generalization ability.

[0029] In step S10: Figure 1 The figure shows the network structure of the constructed MAE-based wireless base large model. In specific implementation, the present invention obtains a three-dimensional CSI data set of a wireless communication system by simulation or actual measurement. The three-dimensional CSI samples of the data set are sequentially passed through an embedding module, a mask module, an encoder module, a decoder module and an output module, and finally the reconstructed CSI is output. This step includes the following processes S11 to S15:

[0030] S11: The embedding module is used to process the input “time-space-frequency” three-dimensional CSI sample into a series of tokens. Assume that the three-dimensional CSI sample is H∈C T×N×K, where C represents a complex set, T, N and K represent the number of time sampling points, the number of antennas and the number of subcarriers of the CSI sample respectively. First, the real and imaginary parts of the three-dimensional CSI sample H are separated and converted into a real tensor H in ∈C 2×T×N×K ,Right now

[0031] H in [1,:,:,:]=Re{H}

[0032] H in [2,:,:,:]=Im{H}

[0033] Among them, H in [1,:,:,:] and H in [2,:,:,:] represent H in In the sub-tensors corresponding to the first dimension index 1 and 2, Re and Im represent the real and imaginary parts of the tensor respectively.

[0034] We use 3D convolution to implement the block and embedding operations, that is, divide the 3D samples into cubes of size (t, n, k) and embed them into the feature space to obtain a series of 1D tokens. t, n, and k represent the time, space, and frequency sampling lengths of each cube, respectively. in The three-dimensional token tensor obtained after the three-dimensional convolution is H conv , which can be expressed as

[0035]

[0036] Conv3d represents a three-dimensional convolution operation, the number of input channels and the number of output channels are 2 and D respectively, and the convolution kernel size and step size are both (t,n,k). conv The last three dimensions of the CSI token tensor H are obtained by merging the dimensions emb ∈R D×L ,in That is, the total number of tokens. The resulting CSI token tensor H emb This is the output of the embedding module.

[0037] S12: The CSI token tensor H obtained by embedding the module emb Perform mask operation on the input H emb Perform mask processing according to the mask rules, that is, keep the original H according to the mask rules. emb The specific masking rule depends on the pre-training task performed (see steps S21 to S24). The tensor composed of the tokens retained after the masking operation is H mask , and H mask ∈R D×W ,Right now

[0038] H mask =MASK{H emb}

[0039] Wherein, MASK represents a mask operation, and each pre-training task corresponds to a different mask operation, as shown in step S20 for details.

[0040] S13: H mask Enter the encoder module. First, add the encoder position encoding for the masked W tokens. Assume the encoder position encoding is P e ∈R D×W , then the token after adding the position encoding is

[0041] H pos =H mask +P e ∈R D×W

[0042] The position encoding of each token depends on the token dimension before merging in H conv The three-dimensional position in H mask The ith token in , assuming its three-dimensional position is (Pt i ,Pn i ,Pk i ), whose corresponding position code is P ei =P e [:,i]∈R D×1 .P ei Encoded by time position Spatial Position Encoding and frequency position encoding Among them, Indicates the rounding down operation. All three use the SinCos absolute position encoding commonly used in transformers.

[0043] like Figure 1 As shown in Figure 1, the main part of the encoder after adding the positional encoding is composed of M transformer blocks. Each transformer block uses the same block structure as the classic vision transformer architecture (ViT). After these transformer blocks, the output token tensor of the overall encoder module is H enc , and H enc ∈R D×W .

[0044] S14: The encoder result H enc The input decoder module is processed. First, use LW mask tokens M∈R D×(L-W)And the output of the encoder is spliced ​​to obtain the spliced ​​tensor H con =[H enc ,M]∈R D×L . Each mask token is a learnable variable. Then H con Add the decoder position code, and the token after adding the position code is

[0045] H dec_pos =H con +P d ∈R D×L

[0046] The decoder position coding is set the same as the encoder position coding in step S13, and both perform SinCos absolute position coding based on the three-dimensional position of each token.

[0047] The position-encoded token is fed into the main body of the decoder, which consists of N ViT transformer blocks. The output of the decoder module is H dec , and H dec ∈R D×L .

[0048] S15: The output token H of the decoder dec The final reconstructed CSI is obtained through the output module. First, H dec The fully connected layer of the input-output module obtains the final output H pred ∈R (2tnk)×L . pred After dimension rearrangement, it becomes H pred1 ∈ Then further re-arrange the dimensions into a 7-dimensional tensor Then for the 7-dimensional tensor H pred2 The 2nd and 3rd dimensions, the 4th and 5th dimensions, the 6th and 7th dimensions are respectively merged to obtain the tensor H after the dimension is merged pred3 , and H pred3 ∈R 2×T×N×K . Among them, H pred3 The first dimension (size is 2) represents the real and imaginary parts of the final reconstructed CSI. The final reconstructed CSI is denoted as H final , and H final ∈C T×N×K , then the following relationship is satisfied:

[0049] H final =H pred3 [1,:,:,:]+1j×H pred3 [2,:,:,:]

[0050] Among them, 1h represents the imaginary unit. pred3[1,:,:,:] and H pred3 [2,:,:,:] represent the tensor H pred3 The subtensors corresponding to indices 1 and 2 in dimension 1.

[0051] In step S20: multiple self-supervised pre-training tasks such as mask reconstruction and interpolation denoising are designed, that is, pre-training is performed based on the base model constructed in S10. S21 to S25 respectively introduce five pre-training tasks: random mask reconstruction, time domain mask reconstruction, spatial domain mask reconstruction, frequency domain mask reconstruction and interpolation denoising:

[0052] S21: Pre-training task based on random mask reconstruction. Based on the wireless base station large model constructed in S10, input the three-dimensional CSI sample H∈C T×N×K The final output of the wireless base large model network is H final , the loss function of network model training is output H final The mean square error (MSE) between and input H is

[0053]

[0054] The mask module adopts a random masking method. That is, in step S12, H is randomly selected. emb ∈R D×L W tokens out of L block tokens are reserved as tokens H mask ∈R D×W .

[0055] S22: Pre-training task based on time domain mask reconstruction. Its training process is the same as S21, except that the mask module uses the time domain mask method. emb The dimensions are rearranged to in and Represents the number of blocks in the three dimensions of time, space and frequency. Then keep all the time dimension coordinates less than Token, assuming the result after retention is H maskres ,but

[0056]

[0057] Then for H maskres Rearrange the dimensions to obtain the time domain mask result H mask ∈R D×W ,in

[0058] S23: Pre-training task based on frequency-domain mask reconstruction. Its training process is the same as that of S21, except that the mask module adopts the frequency-domain mask method. That is, all tokens with frequency dimension coordinates less than are retained. Assuming the result after retention is H maskres , then

[0059]

[0060] Then, perform dimension rearrangement on H maskres to obtain the result of the frequency-domain mask H mask ∈R D×W , where

[0061] S24: Pre-training task based on spatial-domain mask reconstruction. Its training process is the same as that of S21, except that the mask module adopts the spatial-domain mask method. Different from the frequency-domain mask and the time-domain mask, the spatial-domain mask randomly retains some tokens according to a certain ratio Q (0 < Q < 1) according to the spatial coordinates, and the result after retention is where,

[0062] Then, perform dimension rearrangement on H maskres to obtain the result of the spatial-domain mask H mask ∈R D×W , where

[0063] S25: Pre-training task based on interpolation denoising. Based on the base large model constructed in S10, input the three-dimensional CSI sample H ∈ C T×N×K , and first perform downsampling at intervals of it, in, and ik in the time domain, spatial domain, and frequency domain. Assuming the result after downsampling is H ds , then

[0064] H ds = H[1:it:T, 1:in:N, 1:ik:K]

[0065] Perform linear interpolation on H ds to obtain H us ∈ C T×N×K , that is, the same size as H. Input H us into the network, where the mask module in the network does not perform masking, that is

[0066] H mask = H emb

[0067] The final output of the network is H final , and the loss function for network training is the MSE between the output H final and the input H, that is

[0068]

[0069] In step S30: the pre-trained base model is applied to channel estimation and channel prediction tasks. In steps S31 to S33, the method of applying the pre-trained network to channel estimation, time domain channel prediction and frequency domain channel prediction is described respectively:

[0070] S31: Use the pre-trained wireless base station large model for channel estimation tasks. Channel estimation refers to restoring the complete CSI when the CSI estimation value of the partial position is known. First, linear interpolation is performed on the CSI estimation value of the known partial position to obtain the interpolated CSI, that is, H∈C T×N×K , and then input into the network. The final output result is the CSI estimation result.

[0071] S32: Use the pre-trained wireless base station large model for the time domain channel prediction task. Time domain prediction refers to predicting the unknown CSI through partial CSI in time. Assume that the known CSI is H t0 ∈C T0×N×K , the CSI to be predicted is H t1 ∈C (T -T0)×N×K First, H t0 By adding 0, the overall CSI is obtained as H∈C T×N×K , and then input into the network. The mask module adopts the time domain masking method, that is, retaining the tokens with time domain coordinates of 1:T0. Finally, the output result H final The time domain prediction part of is taken as the prediction result, that is,

[0072] H t1 =H final [T0+1:T,:,:]

[0073] S33: Use the pre-trained wireless base station large model for frequency domain channel prediction tasks. Frequency domain prediction refers to predicting unknown CSI based on partial CSI in frequency. Assume that the known CSI is H k0 ∈C T×N×K0 , the CSI to be predicted is H k1 ∈C T ×N×(K-K0) First, H k0 By adding 0, the overall CSI is obtained as H∈C T×N×K , and then input into the network. The mask module adopts the frequency domain masking method, that is, retaining the tokens with the frequency domain coordinates of 1:K0. Finally, the output result H final The frequency domain prediction part of f1 =H final [:,:,K0+1:K].

[0074] Through the above steps, the present invention designs the network architecture of the base large model and the self-supervised training method based on MAE. The self-supervised training method can effectively capture the three-dimensional relationship of space-time-frequency. The pre-trained wireless base large model can directly perform reasoning on the channel estimation task, time domain and frequency domain channel prediction tasks of the wireless communication system, which can improve the accuracy of channel estimation and prediction and reduce work costs.

[0075] It should be noted that the purpose of publishing the embodiments is to help further understand the present invention, but those skilled in the art can understand that various substitutions and modifications are possible without departing from the scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the contents disclosed in the embodiments, and the scope of protection claimed by the present invention shall be subject to the scope defined in the claims.

Claims

1. A design and training method for a large wireless base model, characterized in that: Design a network architecture based on masked autoencoder (MAE) for channel estimation and channel prediction tasks; design a mask reconstruction pre-training task so that the network can perform effective self-supervised learning and obtain a pre-trained base model; The pre-trained base model is used to process channel state information (CSI) data of different sizes to achieve channel estimation and channel prediction; including: 1) Construct a large-scale wireless base station model network architecture based on MAE, including an embedding module, a mask module, an encoder module, a decoder module, and an output module; Acquire the time-space-frequency three-dimensional CSI data set of the wireless communication system; convert the CSI data into a series of tokens through the embedding module; then selectively retain a part of the tokens through the mask module and input them into the encoder module; the output of the encoder module sequentially concatenates the learnable mask features and inputs them into the decoder module; the output of the decoder module is restored to the complete CSI data through the output module; 2) Design a variety of self-supervised pre-training tasks to pre-train the wireless base station large model; the self-supervised pre-training tasks include: mask reconstruction and interpolation denoising; mask reconstruction further includes random mask reconstruction, time domain mask reconstruction, spatial domain mask reconstruction, and frequency domain mask reconstruction; For the pre-training task of mask reconstruction, the 3D CSI samples are input into the constructed wireless base station large model, and the loss function of the network model training is the mean square error between the output and the input; the mask module adopts random masking, time domain masking, frequency domain masking, and spatial domain masking respectively; For the pre-training task of interpolation denoising, the three-dimensional CSI samples are input into the constructed wireless base large model, and downsampled at intervals in the time domain, spatial domain and frequency domain to obtain the downsampled results; the downsampled results are obtained by linear interpolation; the results obtained by linear interpolation are then input into the wireless base large model network, where the mask module in the network is not masked; the loss function of the network training is the mean square error between the final output and the input; 3) Apply the pre-trained base model to channel estimation and channel prediction tasks; The channel estimation task is converted into an interpolation denoising task for processing; the time domain and frequency domain channel prediction tasks are converted into time domain and frequency domain mask recovery tasks for processing respectively; Through the above steps, channel estimation and channel prediction can be achieved for the wireless communication system.

2. The design and training method of the wireless base large model as claimed in claim 1, characterized in that: Specifically, a time-space-frequency three-dimensional CSI data set is obtained through simulation or actual measurement.

3. The design and training method of the wireless base large model as claimed in claim 1, characterized in that: The time-space-frequency three-dimensional CSI dataset samples are denoted as H∈C T×N×K , where C represents a set of complex numbers, and T, N, and K represent the number of time sampling points, the number of antennas, and the number of subcarriers of the CSI samples, respectively.

4. The design and training method of the wireless base large model as claimed in claim 3, characterized in that: The MAE-based wireless base large model network architecture constructed in step 1) specifically includes: S11. The embedding module separates the real and imaginary parts of the CSI data sample and converts them into real tensors; uses three-dimensional convolution operations to implement block division and embedding operations, and embeds them into the feature space to obtain a three-dimensional token tensor; merges the last three dimensions of the obtained three-dimensional token tensor, and the obtained CSI token tensor is the output of the embedding module; S12: Mask the CSI token tensor obtained by the embedding module, retain a portion of the tokens, and obtain a masked tensor; each pre-training task corresponds to a different mask processing operation; S13: inputting the tokens masked by the mask module into the encoder module; adding encoder position codes to the masked tokens; the position code of each token depends on the three-dimensional position of the token in the three-dimensional token tensor before the token dimension is merged; inputting the tokens after adding the position code into the encoder module to obtain the output of the encoder module; S14: Inputting the result output by the encoder module into the decoder module for processing, including: using a part of the mask tokens and the output of the encoder to concatenate to obtain a concatenated tensor; wherein each mask token is a learnable variable; and then adding a decoder position code to the concatenated tensor to obtain a token after adding the position code; wherein the decoder position code and the encoder position code are set to be the same; Input the position-encoded token into the decoder module to obtain the output result of the decoder module, i.e., the output token; S15: The output of the decoder module is passed through the output module to obtain the final reconstructed CSI; The output token of the decoder module is input into the fully connected layer to obtain the output token H pred ; First, H pred After dimension rearrangement, it becomes H pred1 , and then rearrange the dimensions to H pred2 ; Then for H pred2 The second and third dimensions, the fourth and fifth dimensions, and the sixth and seventh dimensions of are merged to obtain H pred3 ; where H pred3 The first dimension represents the real part of the reconstructed CSI, and the second dimension represents the imaginary part of the reconstructed CSI; the real part and the imaginary part are converted into corresponding complex numbers to obtain the final reconstructed CSI.

5. The design and training method of the wireless base large model as claimed in claim 4, characterized in that: In S11, the embedded module specifically includes: First, the real and imaginary parts of the three-dimensional CSI sample are separated and converted into a real tensor H in ∈C 2×T×N×K , expressed as: A in [1,:,:,:]=Re{H} H in [2,:,:,:]=In{H} Among them, H in [1,:,:,:] and H in [2,:,:,:] represent H in In the sub-tensors corresponding to the first dimension index 1 and 2, Re and Im represent the real and imaginary parts of the tensor respectively; The three-dimensional convolution operation is used to implement the block and embedding operation, that is, the three-dimensional samples are divided into blocks according to the size of (t, n, k) cubes and embedded into the feature space to obtain a series of one-dimensional tokens; where t, n and k represent the time, space and frequency sampling length of each cube respectively; The result after three-dimensional convolution is H conv , expressed as: Among them, Conv3d represents a three-dimensional convolution operation, the number of input channels and the number of output channels are 2 and D respectively, and the convolution kernel size and step size are both (t,n,k); Put H conv The last three dimensions of H are combined to obtain emb ∈R D×L ,in That is, the total number of blocks; the obtained H emb This is the output of the embedded module; In S12, the embedded token is masked, specifically, the input H emb Perform mask processing, that is, keep the original H according to the mask rules emb W tokens in the L block of tokens; let the masked tensor be H mask ∈R D×W ,Right now H mask =MASK{H emb } Wherein MASK represents the mask operation.

6. The design and training method of the wireless base large model as claimed in claim 5, characterized in that: In S13, the encoder position encoding is added to the masked W token, specifically: Assume the encoder position code is P e ∈R D×W , then the token after adding the position encoding is represented as: H pos =H mask +P e ∈R D×W For H mask The 3D position of the ith token in (Pt i ,Pn i ,Pk i ), whose corresponding position code is P ei =P e [:,i]∈R D×1 ; P ei Encoded by time position Spatial Position Encoding and frequency position encoding spliced ​​together; among them, Indicates rounding down operation; the position encoding adopts the SinCos absolute position encoding commonly used in transformer.

7. The design and training method of the wireless base large model as claimed in claim 6, characterized in that: The main body of the encoder consists of M transformer blocks; each transformer block adopts the block structure of the vision transformer architecture (visiontransformer, ViT).

8. The design and training method of the wireless base large model as claimed in claim 6, characterized in that: In S14, the decoder module first uses LW mask tokens M∈R D×(L-W) And the output of the encoder is spliced ​​to obtain the spliced ​​tensor H con =[H enc ,M]∈R D×L ; where each mask token is a learnable variable; then H con Add the decoder position code to get the token after adding the position code, expressed as: H dec_pos =H con +P d ∈R D×L The main body of the decoder consists of N ViT transformer blocks; In S15, specifically, H dec Input the fully connected layer to obtain the final output H pred ∈R (2tnk)×L ; including: firstly, after dimension rearrangement, it is transformed into Then the dimensions are rearranged into Then combine the first and third dimensions, the third and fourth dimensions, and the fifth and sixth dimensions to obtain H pred1 ∈R 2×T×N×K ; where H pred1 The first dimension represents the real part of the reconstructed CSI, and the second dimension represents the imaginary part of the reconstructed CSI; the final reconstructed CSI is H final ∈C T×N×K , expressed as: H final =H pred1 [1,:,:,:]+1j×H pred1 [2,:,:,:]. Where 1j represents an imaginary unit. pred3 [1,:,:,:] and H pred3 [2,:,:,:] represent the tensor H pred3 The subtensors corresponding to indices 1 and 2 in dimension 1.

9. The design and training method of the wireless base large model as claimed in claim 8, characterized in that: In step 2), for the pre-training task of random mask reconstruction, the loss function of network training is expressed as: For the pre-training task of time domain mask reconstruction, firstly H emb The dimensions are rearranged to in and Represents the number of blocks in the three dimensions of time, space and frequency respectively; then retain all time dimension coordinates less than The token of the reserved result H maskres It is expressed as: Then for H maskres Rearrange the dimensions to obtain the time domain mask result H mask ∈R D×W ,in For the pre-training task of frequency domain mask reconstruction, the retained result H maskres It is expressed as: Then for H maskres Rearrange the dimensions to obtain the frequency domain mask result H mask ∈R D×W ,in For the pre-training task of spatial mask reconstruction, the retained result is in, Then for H maskres Rearrange the dimensions to get the result H of the spatial mask mask ∈R D×W ,in For the pre-training task of interpolation denoising, the down-sampled result H ds It is expressed as: H ds =H[1:it:T,1:in:N,1:ik:K] For H ds H is obtained by linear interpolation us ∈C T×N×K , which is the same size as H; us Input network, where the mask module in the network does not perform masking, that is, H mask =H emb .

10. The design and training method of the wireless base large model as claimed in claim 9, characterized in that: In step 3), the pre-trained network is used for channel estimation, time domain channel prediction and frequency domain channel prediction, specifically: S31: Use the pre-trained wireless base station large model for channel estimation tasks: First, the CSI estimation value of the known partial position is linearly interpolated to obtain the interpolated CSI, that is, H∈C T ×N×K , and then input the pre-trained wireless base station large model, and the final output result is the CSI estimation result; S32: Use the pre-trained wireless base station large model for the time domain channel prediction task: First, the known CSI is padded with 0 to obtain the overall CSI, denoted as H∈C T×N×K , and then input the pre-trained wireless base large model; the mask module adopts the time domain mask method, that is, retains the token with the time domain coordinate of 1:T0; finally, the time domain prediction part of the output result is taken as the predicted CSI result, that is, H t1 =H final [T0+1:T,:,:]; S33: Use the pre-trained wireless base station large model for the frequency domain channel prediction task. The process is as follows: First, the known CSI is padded with 0 to obtain the overall CSI, denoted as H∈C T×N×K , and then input the pre-trained wireless base large model; the mask module adopts the frequency domain masking method, that is, retaining the tokens with frequency domain coordinates of 1:K0; finally, the frequency domain prediction part of the output result is taken as the prediction result, that is, H f1 =H final [:,:,K0+1:K].

Images