A model alignment based beam prediction method and device

Abstract

The application discloses a kind of based on model alignment's beam prediction method and device, significantly improve the intelligent level and deployment efficiency of beam management.The method comprises the following steps: base station configures candidate service beam codebook;Base station determines probe beam according to candidate service beam codebook;Base station obtains probe beam receiving power;System carries out normalization processing to probe beam receiving power;System generates signal word embedding through signal modal aligner;System combines signal word embedding with the text word embedding corresponding to prompt text after inputting pre-training large language model;System calculates the prediction result of candidate service beam through classification head;Base station selects service beam according to the prediction result of candidate service beam.The application solves the cross-scene generalization bottleneck problem of high-frequency band wireless communication system beam prediction, data dependence and deployment cost problem and signal and language semantic gap problem.

Description

Technical Field

[0001] This invention relates to the fields of mobile communication and artificial intelligence technologies, and in particular to a beam prediction method and apparatus based on model alignment. Background Technology

[0002] Millimeter-wave communication technology is a key technology for enhancing fifth-generation mobile communication systems and for future sixth-generation mobile communication systems. By providing ultra-large continuous bandwidth in the high-frequency band, it can achieve extremely high peak data rates. However, millimeter-wave signals have extremely high path loss, which needs to be compensated for by generating highly directional narrow beams through large-scale multiple-input multiple-output antenna arrays.

[0003] In practical systems, base stations typically select the optimal serving beam from a codebook containing dozens of candidate beams. Traditional exhaustive or hierarchical search methods incur high training overhead and long processing times. In recent years, data-driven methods have effectively reduced training overhead by measuring the received power of a small number of probe beams and using machine learning models to directly predict the optimal beam.

[0004] The first category of existing technical solutions is the traditional deep learning method, which uses architectures such as multilayer perceptrons, convolutional neural networks, and recurrent neural networks to directly establish the mapping from detection power to the optimal beam. The generalization ability of this type of method is heavily dependent on the distribution of training data. Its performance drops sharply in scenarios with distribution offsets, such as across base stations, across frequency bands, or in the presence of environmental occlusion. Furthermore, each deployment to a new scenario requires the collection of a large amount of labeled data for retraining, resulting in high data collection and labeling costs.

[0005] The second category is multimodal fusion methods, which attempt to integrate data from multiple sources such as cameras, global navigation satellite systems, and lidar to assist beam prediction. These methods require the deployment of additional sensor equipment, increasing system cost and complexity. Furthermore, in scenarios such as indoor environments, urban canyons, areas with severe obstruction, or where satellite positioning signals are interfered with, some auxiliary modes may not be acquired stably. The data acquisition process is complex and may raise concerns about user privacy.

[0006] The third category consists of methods based on large language models, which utilize the generalization capabilities of pre-trained large language models for beam prediction. Typical methods include converting probe power into text descriptions or discretizing continuous power values ​​into text prototypes based on preset rules. These methods suffer from a signal-to-language semantic gap; continuous probe power needs to undergo rule-based, discretized, or textual processing before being input into the language model. This process can easily lead to the loss of accurate numerical information and beam spatial structure information.

[0007] Furthermore, most methods require historical beam tags, complete channel state information, or external sensor data as additional inputs, cannot make predictions based solely on probe power, and lack systematic generalization studies for distribution offsets across scenarios. Summary of the Invention

[0008] This invention addresses the shortcomings of existing technologies by proposing a model-aligned beam prediction method to solve the cross-scenario generalization bottleneck problem of beam prediction in high-frequency wireless communication systems. This method enables the model to maintain high prediction accuracy even when there are significant distribution differences between training and testing scenarios. Furthermore, this invention addresses data dependency and deployment cost issues, reducing reliance on large amounts of labeled data and achieving effective beam prediction in zero-sample or low-sample scenarios. In addition, this invention addresses the signal-to-language semantic gap by establishing a direct mapping between physical layer measurement signals and the semantic space of a large language model, avoiding information loss caused by discretization or textualization intermediate steps. To address these issues, this invention aims to achieve a lightweight, high-frequency multi-antenna system beam prediction method with strong generalization and high data efficiency, significantly improving the intelligence level and deployment efficiency of beam management.

[0009] To achieve the above objectives, the present invention provides the following technical solution:

[0010] In a first aspect, the present invention provides a beam prediction method based on model alignment, comprising: a base station configuring a candidate service beam codebook; the base station determining a probe beam based on the candidate service beam codebook; the base station acquiring the received power of the probe beam; the system normalizing the received power of the probe beam; the system generating signal word embeddings through a signal mode aligner; the system combining the signal word embeddings with the text word embeddings corresponding to the prompt text and inputting them into a pre-trained large language model; the system calculating the prediction results of the candidate service beams through a classification head; and the base station selecting a service beam based on the prediction results of the candidate service beams.

[0011] Furthermore, the method includes a system configuration and model building phase, a model training phase, and an online beam prediction phase, with the specific implementation steps as follows:

[0012] S1. Obtain the number of candidate service beams Calculate the number of detection beams The base station depends on the number of detection beams. From the candidate service beam codebook Select a detection beam and obtain the detection beam index set. According to the candidate service beam codebook The index order is used to perform linearly equally spaced sampling between the first and last candidate service beam codewords to obtain... One sampling location;

[0013] S2. Construct a beam prediction model that includes a signal modality aligner, a lexical encoder, a word embedding layer, a pre-trained large language model, a classification head, and a low-rank adaptation module.

[0014] S3. During the model training phase, the base station or training equipment constructs a training sample set. The system obtains the number of samples. Training sample set include The training sample is obtained; The probe beam power vector of each training sample and optimal service beam tag ,in, ;No. The probe beam power vector of each training sample Depend on Each detection beam power value Composition, the first The power value of each detection beam is denoted as , ;

[0015] S4. Obtain the input probe beam power vector For the input probe beam power vector A logarithmic transformation is performed, and the logarithmic results of the received power of all probe beams are centered to obtain the normalized probe beam power vector. During the model training phase, the input probe beam power vector is... For the first The probe beam power vector of each training sample During the online beam prediction phase, the input probe beam power vector is... The probe beam power vector reported by the target user;

[0016] S5. Normalize the probe beam power vector The input signal mode aligner is used to calculate the expanded signal embedding vector. Expand the signal embedding vector According to the number of signal words Hidden Dimensions of Large Language Models Divided into A length of The subvector, and the Arrange the subvectors row by row to obtain the signal word embedding matrix. ;

[0017] S6. Construct text including prefixes and suffix text The prompt text, prefix text Includes beam prediction task description content, suffix text Includes predicted tokens [PRED]; employs a pre-trained large language model. The built-in lexical encoder and word embedding layer are used for prefix text. and suffix text Encode the prefix text token embedding matrix to obtain the prefix text token embedding matrix. and suffix text lexical embedding matrix The prefix text is processed by the lexical encoder. and suffix text The number of prefix text tokens is obtained after encoding. and the number of suffix text morphemes Obtain the predicted token position number Embed the prefix text tokens into the matrix Signal word embedding matrix and suffix text lexical embedding matrix Concatenate them sequentially to obtain the large language model input embedding matrix. and the length of the input sequence of the large language model Length of input sequence for large language models Number of prefix text terms Number of signal words and the number of suffix text morphemes sum;

[0018] S7. Input the large language model into the embedding matrix. Input pre-trained large language model The hidden state matrix of the large language model is obtained. Based on the number of prefix text lexical units Number of signal words and prediction marker lexical position number Determine the predicted position index of the predicted token in the input sequence of the large language model. Output the hidden state matrix from the large language model. Extract the predicted location index The corresponding predicted location hidden feature vector ;

[0019] S8. Hide the feature vector of the predicted location. Input the classification header to obtain the candidate service beam logical score vector. Based on the candidate service beam logical score vector Calculate the candidate service beam prediction probability vector ;

[0020] S9. Predict the probability vector based on the candidate service beam. Calculate the training loss function using the optimal service beam label. The optimal service beam label is obtained through full candidate service beam scanning, historical network measurement records, or offline channel simulation data.

[0021] S10. Execute the first training phase: Modal warm-up phase. In the first training phase, the system freezes the pre-trained large language model. All backbone parameters are updated, only the first weight matrix in the signal mode aligner is updated. First bias vector Second weight matrix Second bias vector and the class head weight matrix and classification head bias vector Before the start of each training round in the first training phase, the base station or training device performs training on the training sample set. The training samples are randomly shuffled and arranged according to the training batch size of the first training phase. The shuffled training samples are divided into multiple training batches; in each parameter update during the first training phase, the base station or training device selects a training batch and calculates the training loss function according to steps S4 to S9. According to the training loss function Calculate the gradient and update the parameters of the signal mode aligner and the classification head; the first training phase is complete. After one training round, the system enters the second training phase;

[0022] S11. Execute the second training phase: end-to-end low-rank adaptation fine-tuning phase. Before the start of the second training phase, the system pre-trains the large language model. Introduce a low-rank adaptation matrix set into at least one attention projection matrix or at least one feedforward network projection matrix; for any original projection matrix By pre-trained large language model The pre-trained weights are obtained and kept frozen; the original projection matrix input dimension is obtained. Original projection matrix output dimension and the input feature vector of the projection layer Set low-rank dimension Low-rank scaling factor Low-rank dimensionality reduction matrix and low-rank increasing-dimensional matrix According to the original projection matrix Low-rank dimensionality reduction matrix and low-rank increasing-dimensional matrix Calculate the projection matrix after low-rank adaptation correction. And use the projection matrix after low-rank adaptation correction. For the input feature vector Perform projection calculations; the training loss function will continue to be used in the second training phase. Training is performed, and the signal mode aligner, classification head, and low-rank dimensionality reduction matrix are jointly updated. and low-rank increasing-dimensional matrix While maintaining the original projection matrix and pre-trained large language models Other backbone parameters are frozen; before the start of each training round in the second training phase, the base station or training device adjusts the training sample set. The training samples in the dataset are randomly shuffled and processed according to the training batch size of the second training phase. The shuffled training samples are divided into multiple training batches; in each parameter update during the second training phase, the base station or training device selects a training batch and calculates the training loss function according to steps S4 to S9. According to the training loss function Calculate the gradient and update the parameters of the signal mode aligner, classification head, and low-rank adaptation matrix set; the second training phase is complete. After one training round, the system determines that the model training is complete and saves the parameters of the signal mode aligner, classification head, and low-rank adaptation matrix group.

[0023] S12, In the online beam prediction phase, the base station sequentially uses the probe beam index set. The corresponding probe beam sends a reference signal to the target user equipment. The target user equipment measures the received power value corresponding to each probe beam and reports the measurement results to the base station. The base station obtains the target user's online probe beam power vector. Online detection beam power vector of the target user include The detection beam power value, the target user for the first Online detection beam power values ​​measured by each detection beam , The base station will transmit the online detection beam power vector of the target user. Input the trained beam prediction model and calculate the candidate serving beam prediction probability vector according to steps S4 to S8. ;

[0024] S13, The base station predicts the probability vector based on the candidate serving beam. Select Prediction Service Beam Index Predictive Service Beam Index Predicting probability vectors for candidate service beams The index corresponding to the largest element in the middle is used by the base station based on the predicted serving beam index. Select candidate service beamcodeword The service beam for the target users.

[0025] Furthermore, in step S1, the number of detection beams... The calculation formula is:

[0026]

[0027] in, For the number of candidate service beams, , To detect the scaling factor.

[0028] Further, in step S2, the signal mode aligner includes: a first weight matrix. First bias vector Second weight matrix Second bias vector and activation function Activation function Use either the GELU or ReLU function; the classification head includes: a classification head weight matrix. Classification head bias vector Pre-trained large language models An encoder-type pre-trained language model or a causal pre-trained language model can be used.

[0029] Furthermore, in step S3, the first The probe beam power vector of each training sample Data is collected in the following manner: The base station sequentially uses the probe beam index set. The corresponding probe beam transmits a reference signal, and the user equipment measures the received power value corresponding to each probe beam and reports the measurement results to the base station or training equipment; Optimal service beam label for each training sample Obtained through full candidate service beam scanning, historical network measurement records, or offline channel simulation data; each training sample includes at least the [number]th [sample]. The probe beam power vector of each training sample and the Optimal service beam label for each training sample , among which, the The probe beam power vector of each training sample , No. Optimal service beam label for each training sample are integers and satisfy .

[0030] Further, in step S4, the calculation of the first... Normalized probe beam power value The formula is as follows:

[0031]

[0032] in, For the first Each detection beam power value, It is the numerical stability constant. Let be the power value of the j-th probe beam.

[0033] Further, in step S5, the signal embedding vector is expanded. Calculate using the following formula:

[0034]

[0035] in, This is the first bias vector. This is the second bias vector. This is the first weight matrix. This is the second weight matrix.

[0036] Furthermore, in step S8, the candidate service beam logical score vector Composed of candidate service beam logic scores, the first The logical score of each candidate service beam is denoted as: , :

[0037] ,

[0038] in, This is the class head weight matrix. This is the classifier head bias vector;

[0039] Candidate service beam prediction probability vector Composed of candidate service beam prediction probability values, the first The candidate service beam prediction probability value corresponding to each candidate service beam is denoted as . Candidate service beam prediction probability value Calculate using the following formula:

[0040]

[0041] in, Let be the logical score value of the candidate service beam corresponding to the j-th candidate service beam.

[0042] Furthermore, in step S9, the loss function is trained. The calculation formula is:

[0043]

[0044] in, Indicates the first in the training batch The optimal serving beam label for each training sample in this sample The predicted probability value of the candidate service beam at the location. This represents the current training batch size.

[0045] Furthermore, in step S11, the projection matrix after low-rank adaptation correction... for:

[0046]

[0047] in, It is a low-rank dimension. This is the low-rank scaling factor.

[0048] Secondly, the present invention provides a beam prediction device based on model alignment, comprising the following modules for implementing the method described in any of the above-mentioned embodiments:

[0049] The probe beam configuration module is used to configure the candidate service beam codebook. and detection ratio coefficient Candidate service beam codebook This includes the number of candidate service beams. The system will use the candidate service beam codeword. The candidate service beam codewords are denoted as follows: ,in, Detection ratio coefficient The range of values ​​is to ;

[0050] The power acquisition module sequentially uses the probe beam index set. The corresponding probe beam transmits a reference signal, and the user equipment measures the received power value corresponding to each probe beam and reports the measurement results to the base station or training equipment; Optimal service beam label for each training sample Obtained through full candidate service beam scanning, historical network measurement records, or offline channel simulation data; each training sample includes at least the [number]th [sample]. The probe beam power vector of each training sample and the Optimal service beam label for each training sample , among which, the The probe beam power vector of each training sample , No. Optimal service beam label for each training sample are integers and satisfy ;

[0051] The normalization module is used to normalize the input probe beam power vector. Logarithmic transformation and centering are performed to obtain the normalized probe beam power vector. Input probe beam power vector Composed of the detection beam power value;

[0052] The signal mode alignment module is used to align the normalized probe beam power vector. Calculate the expanded signal embedding vector The signal mode alignment module includes: a first weight matrix. First bias vector Second weight matrix Second bias vector and activation function Activation function Use the GELU function or the ReLU function;

[0053] The text encoding module is used to construct text including prefixes. and suffix text The prompt text, and the prefix text and suffix text Encode the prefix text token embedding matrix to obtain the prefix text token embedding matrix. Suffix text lexical embedding matrix Number of prefix text terms and the number of suffix text morphemes Obtain the predicted token position number Embed the prefix text tokens into the matrix Signal word embedding matrix and suffix text lexical embedding matrix Concatenate them sequentially to obtain the large language model input embedding matrix. and the length of the input sequence of the large language model ;

[0054] The large language model inference module is used to input the large language model into the embedding matrix. Output hidden state matrix The large language model inference module adopts either an encoder-type pre-trained language model or a causal pre-trained language model.

[0055] The classification prediction module is used to hide the feature vector of the predicted location. Output candidate service beam logical score vector The classification prediction module includes: a classifier head weight matrix. Classification head bias vector and activation function Activation function Use the GELU function or the ReLU function;

[0056] The training control module is used to execute the first and second training phases. The first training phase uses the AdamW optimizer, and the learning rate for the first training phase is... The range of values ​​is to Training batch size in the first training phase The value ranges from 64 to 512; the first training phase consists of training rounds. The value ranges from 1 to 20; the weight decay coefficient typically ranges from 0 to 0.1; the low-rank dimension in the first training phase. The value ranges from 1 to 64; low-rank scaling factor The value ranges from 1 to 64; low-rank dimensionality reduction matrix. The elements are randomly initialized by the system; low-rank, high-dimensional matrix. The elements are initialized to zero or randomly by the system; the learning rate in the second training phase. The range of values ​​is to Second training phase training batch size The value range is from 8 to 128; the second training phase training rounds The value ranges from 1 to 10; the low-rank adaptation random inactivation rate ranges from 0 to 0.2.

[0057] Thirdly, the present invention provides an electronic device, including a processor and a memory, wherein the memory stores program instructions, characterized in that the program instructions, when executed by the processor, implement the method described in any of the above-mentioned embodiments.

[0058] Fourthly, the present invention provides a computer-readable storage medium storing program instructions, characterized in that the program instructions, when executed by a processor, implement the method described in any of the preceding claims.

[0059] This invention uses the received power of the probe beam in a high-frequency multi-antenna system as the physical layer input. A signal mode aligner converts continuous power measurements into signal word embeddings that can be processed by a pre-trained large language model. These signal word embeddings, along with the text word embeddings corresponding to the prompt text, are then input into the pre-trained large language model. A classification head then outputs the prediction results of candidate service beams to determine the service beam for the target user. This application also employs a two-stage training method of mode warm-up and low-rank adaptation fine-tuning, training the signal mode aligner, classification head, and low-rank adaptation matrix while keeping the backbone parameters of the pre-trained large language model frozen. Compared with existing technologies, this invention has the following advantages:

[0060] 1. This invention directly converts the detected beam received power into signal token embeddings and inputs them into a pre-trained large language model, eliminating the need to discretize continuous power values ​​into text tags or rewrite the power values ​​into natural language descriptions. Therefore, this invention can preserve the continuous numerical information and beam spatial structure information in the detected beam received power, reducing information loss caused by manual rule conversion.

[0061] 2. In the online beam prediction stage, this invention uses only the target user's detected beam received power as input, without requiring data from external sensors such as cameras, GPS, or lidar, nor does it require complete channel state information as online input. Therefore, this invention can reduce hardware deployment complexity, data acquisition costs, and user privacy risks.

[0062] 3. This invention obtains the received power by using a preset number of probe beams and predicts the serving beam based on this received power, without needing to traverse all candidate serving beams during the online prediction phase. Therefore, this invention can reduce beam training latency and signaling overhead, and is suitable for scenarios in high-frequency multi-antenna systems that require rapid beam selection.

[0063] 4. This invention establishes a correspondence between physical layer measurement signals and the input space of a pre-trained large language model through a signal mode aligner, enabling the pre-trained large language model to participate in beam prediction tasks. Compared to relying solely on traditional deep learning models to directly learn the mapping relationship between probe power and serving beam, this invention is beneficial for improving the model's adaptability in cross-base station, cross-frequency band, and obstruction scenarios.

[0064] 5. This invention employs a two-stage training approach. In the first training stage, the backbone parameters of the pre-trained large language model are frozen, and the signal modality aligner and classification head are trained. In the second training stage, a low-rank adaptation matrix set is introduced into the pre-trained large language model and joint training is performed. This training method allows for the alignment between the physical layer signal and the language model input space first, followed by fine-tuning of beam prediction task-related parameters, thereby reducing the training instability risk associated with direct end-to-end training.

[0065] 6. In the second training phase, this invention keeps the backbone parameters of the pre-trained large language model frozen and updates the parameters related to the beam prediction task through a low-rank adaptation matrix set. Compared with the full parameter fine-tuning method, this invention can reduce the number of parameters that need to be updated and stored during training, reduce training computational overhead and model adaptation costs, and reduce the risk of damaging the original pre-training capabilities.

[0066] 7. The beam prediction model of this invention consists of a signal mode aligner, a lexical encoder, a word embedding layer, a pre-trained large language model, a classification head, and a low-rank adaptation module, with clear interfaces between the modules. This structure can adapt to pre-trained large language models of different sizes or architectures, facilitating migration and expansion in different base stations, different frequency bands, or different deployment environments. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0068] Figure 1 The overall flowchart of the beam prediction method based on model alignment provided in the embodiments of the present invention is shown.

[0069] Figure 2 This is a schematic diagram of the module composition of the beam prediction model provided in an embodiment of the present invention. Detailed Implementation

[0070] To better understand this technical solution, the method of the present invention will be described in detail below with reference to the accompanying drawings.

[0071] This invention provides a beam prediction method based on model alignment, including a system configuration and model building phase, a model training phase, and an online beam prediction phase. For example... Figure 1 As shown, the specific implementation steps are as follows:

[0072] Step 1:

[0073] The base station pre-configures a candidate serving beam codebook, denoted as: Candidate service beam codebook Pre-set by the communication system. The system acquires the number of candidate serving beams, denoted as: Number of candidate service beams satisfy Candidate service beam codebook This includes the number of candidate service beams. The first candidate service beam codeword. The system will... The candidate service beam codewords are denoted as follows: ,in, The system has a pre-set detection ratio coefficient, denoted as: Detection ratio The typical value range is: to The system calculates the number of detection beams according to the following formula, denoted as: :

[0074]

[0075] Base stations based on the number of detection beams From the candidate service beam codebook Select a probe beam and obtain the probe beam index set, denoted as: Detection beam index set include The system will use the first detection beam index. Each detection beam in the candidate service beam codebook The probe beam index in the data is denoted as: ,in, The system follows the candidate service beam codebook. The index order is used to perform linearly equally spaced sampling between the first and last candidate service beam codewords to obtain... There are several sampling positions; when the sampling position is not an integer index, the system rounds the sampling position down.

[0076] Step 2:

[0077] Construct a beam prediction model. The beam prediction model includes a signal mode aligner, a lexical encoder, a word embedding layer, a pre-trained large language model, a classification head, and a low-rank adaptation module. For example... Figure 2 As shown, the system is configured with a pre-trained large language model, denoted as: The system sets the number of signal tokens, denoted as: Number of signal words The value typically ranges from 1 to 8. The system obtains the hidden dimension of the large language model, denoted as: Hidden Dimensions of Large Language Models From pre-trained large language models The word embedding dimension is determined. The system sets the hidden dimension of the signal modality aligner, denoted as: Signal mode aligner hides dimensions The typical value range is from 64 to 1024.

[0078] The signal mode aligner includes: a first weight matrix, denoted as: The first bias vector is denoted as: The second weight matrix is ​​denoted as: The second bias vector is denoted as: The activation function is denoted as: The classification head includes: the classification head weight matrix, denoted as: The classifier head bias vector is denoted as: Activation function Use GELU or ReLU functions; pre-train a large language model. An encoder-type pre-trained language model or a causal pre-trained language model can be used.

[0079] Step 3:

[0080] During the model training phase, the base station or training device constructs a training sample set, denoted as: The system obtains the number of samples, denoted as: Sample size Determined by the size of the training data. Training sample set. include The training sample. For the first training sample. The system obtains the training sample. The detector beam power vector of each training sample is denoted as: The system obtains the first The optimal serving beam label for each training sample is denoted as: ,in, . No. The probe beam power vector of each training sample Depend on Composed of the power values ​​of the detection beams, the first The power value of each detection beam is denoted as: , .

[0081] No. The probe beam power vector of each training sample Data is collected in the following manner: The base station sequentially uses the probe beam index set. The corresponding probe beam transmits a reference signal, and the user equipment measures the received power value corresponding to each probe beam and reports the measurement results to the base station or training equipment. Optimal service beam label for each training sample Obtained through full candidate service beam scanning, historical network measurement records, or offline channel simulation data. Each training sample includes at least the first... The probe beam power vector of each training sample and the Optimal service beam label for each training sample , among which, the The probe beam power vector of each training sample , No. Optimal service beam label for each training sample are integers and satisfy .

[0082] Step 4:

[0083] Obtain the input probe beam power vector, denoted as: Input probe beam power vector Composed of the detection beam power value, the first The power value of each detection beam is denoted as: , During the model training phase, the input probe beam power vector is... For the first The probe beam power vector of each training sample During the online beam prediction phase, the input probe beam power vector is... The probe beam power vector reported by the target user. The system sets a numerical stability constant, denoted as: Numerical stability constant The typical value range is: to The system processes the input probe beam power vector. After performing logarithmic transformation and centering, the normalized probe beam power vector is obtained, denoted as: The system calculates the first... The normalized probe beam power value is denoted as: Normalized detection beam power value Calculate using the following formula:

[0084]

[0085] Normalized probe beam power vector Depend on It consists of a normalized probe beam power value.

[0086] Step 5:

[0087] Normalized probe beam power vector The input signal mode aligner is used to calculate the expanded signal embedding vector, denoted as: Expand the signal embedding vector Calculate using the following formula:

[0088]

[0089] Expand the signal embedding vector According to the number of signal words Hidden Dimensions of Large Language Models Divided into A length of The subvector, and the Arrange the subvectors in rows to obtain the signal word embedding matrix, denoted as: Signal word embedding matrix The dimension is .

[0090] Step 6:

[0091] Construct the prompt text. The prompt text includes a prefix text and a suffix text. The system sets the prefix text, denoted as: Prefix text This includes the description of the beam prediction task. The system sets the suffix text, denoted as: , suffix text Includes predicted tokens [PRED]. For example, prefix text. Set to "As a communication base station, please predict the optimal serving beam based on the following detected beam's received power characteristics:", suffix text Set the optimal beam index to: [PRED]

[0092] Employing pre-trained large language models The built-in lexical encoder and word embedding layer are used for prefix text. Encoding is performed to obtain the prefix text lexical embedding matrix, denoted as: ; For suffix text Encoding is performed to obtain the suffix text lexical embedding matrix, denoted as: The system obtains the number of prefix text tokens, denoted as: The system obtains the number of suffix text tokens, denoted as: Number of prefix text terms and the number of suffix text morphemes The prefix text is processed by the lexical encoder. and suffix text After encoding, the system obtains the predicted token position number, denoted as: Predicting the position number of the token. This indicates the position of the predicted marker [PRED] in the suffix text lexical sequence, and satisfies... .

[0093] Embedding prefix text terms into a matrix Signal word embedding matrix and suffix text lexical embedding matrix Concatenate them sequentially to obtain the large language model input embedding matrix, denoted as: :

[0094]

[0095] Wherein, the symbol [;] indicates concatenation along the lexical sequence dimension. The system obtains the length of the input sequence of the large language model, denoted as: Length of input sequence for large language models Number of prefix text terms Number of signal words and the number of suffix text morphemes sum.

[0096] Step 7:

[0097] Input the large language model into the embedding matrix Input pre-trained large language model The hidden state matrix output by the large language model is obtained, denoted as: The system determines the number of prefix text tokens based on the number of prefix text tokens. Number of signal words and prediction marker lexical position number Determine the predicted position index of the predicted token in the input sequence of the large language model, denoted as: The system outputs the hidden state matrix from the large language model. Extract the predicted location index The corresponding hidden feature vector at the predicted location is denoted as: .

[0098] Step 8:

[0099] Hide the feature vector at the predicted location Input the classification header to obtain the candidate service beam logical score vector, denoted as: Candidate service beam logical score vector Composed of candidate service beam logic scores, the first The candidate service beam logical score value corresponding to each candidate service beam is denoted as: , :

[0100]

[0101] Based on the candidate service beam logical score vector Calculate the candidate serving beam prediction probability vector, denoted as: Candidate service beam prediction probability vector Composed of candidate service beam prediction probability values, the first The candidate service beam prediction probability value corresponding to each candidate service beam is denoted as: Candidate service beam prediction probability value Calculate using the following formula:

[0102]

[0103] Step 9:

[0104] Based on the candidate service beam prediction probability vector The training loss function is calculated using the optimal service beam label and denoted as: In calculating the training loss function At that time, the current training batch size is denoted as: The training samples in the training batch are arranged according to Number. Training loss function for:

[0105]

[0106] in, Indicates the first in the training batch The optimal serving beam label for each training sample in this sample The predicted probability value of the candidate service beam at the location.

[0107] Step 10:

[0108] The first training phase is performed: the modality warm-up phase. In this phase, the system freezes the pre-trained large language model. All backbone parameters are updated, only the first weight matrix in the signal mode aligner is updated. First bias vector Second weight matrix Second bias vector and the class head weight matrix and classification head bias vector The AdamW optimizer is used in the first training phase. The system sets the learning rate for the first training phase, denoted as: Learning rate in the first training phase The typical value range is: to The system sets the training batch size for the first training phase, denoted as: Training batch size in the first training phase The typical value range is 64 to 512. The system sets the first training phase training rounds, denoted as: The first training phase training rounds The value typically ranges from 1 to 20. The system sets a weight decay coefficient, which typically ranges from 0 to 0.1. Before each training round in the first training phase, the base station or training device processes the training sample set... The training samples are randomly shuffled and arranged according to the training batch size of the first training phase. The shuffled training samples are divided into multiple training batches. During each parameter update in the first training phase, the base station or training device selects one training batch and calculates the training loss function according to steps 4 through 9. According to the training loss function Calculate the gradient and update the parameters of the signal mode aligner and the classification head. The first training phase is complete. After one training round, the system enters the second training phase.

[0109] Step 11:

[0110] The second training phase is then executed: the end-to-end low-rank adaptation fine-tuning phase. Before the second training phase begins, the system pre-trains a large language model. A low-rank adaptation matrix set is introduced into at least one attention projection matrix or at least one feedforward network projection matrix. For any original projection matrix, denoted as: Original projection matrix From pre-trained large language models The pre-trained weights are obtained and kept frozen. The system obtains the original projection matrix input dimension, denoted as: The system obtains the original projection matrix output dimension, denoted as: The system obtains the input feature vector of this projection layer, denoted as: The system sets the low-rank dimension, denoted as: low-rank dimension The value typically ranges from 1 to 64. The system sets a low-rank scaling factor, denoted as: Low-rank scaling factor The values ​​typically range from 1 to 64. The system sets a low-rank dimensionality reduction matrix, denoted as: Its elements are randomly initialized by the system; the system sets a low-rank, high-dimensional matrix, denoted as: Its elements are initialized to zero or randomly by the system. The system is based on the original projection matrix. Low-rank dimensionality reduction matrix and low-rank increasing-dimensional matrix Calculate the projection matrix after low-rank adaptation correction, denoted as: And use the projection matrix after low-rank adaptation correction. For the input feature vector Perform projection calculations. The projection matrix after low-rank adaptation correction. for:

[0111]

[0112] The second training phase continues to use the training loss function. Training is performed, and the signal mode aligner, classification head, and low-rank dimensionality reduction matrix are jointly updated. and low-rank increasing-dimensional matrix While maintaining the original projection matrix and pre-trained large language models Other core parameters are frozen. The system sets the learning rate for the second training phase, denoted as: Learning rate in the second training phase The typical value range is: to The system sets the training batch size for the second training phase, denoted as: The training batch size in the second training phase The typical value range is 8 to 128. The system sets the second training phase training rounds, denoted as: Second training phase training rounds The value typically ranges from 1 to 10. The system sets a low-rank adaptation random inactivation rate, which typically ranges from 0 to 0.2. Before the start of each training round in the second training phase, the base station or training device processes the training sample set... The training samples in the dataset are randomly shuffled and processed according to the training batch size of the second training phase. The shuffled training samples are divided into multiple training batches. During each parameter update in the second training phase, the base station or training device selects a training batch and calculates the training loss function according to steps 4 to 9. According to the training loss function Calculate the gradient and update the parameters of the signal mode aligner, classification head, and low-rank adaptation matrix set. The second training phase is then complete. After one training round, the system determines that the model training is complete and saves the parameters of the signal mode aligner, classification head, and low-rank adaptation matrix group.

[0113] Step 12:

[0114] During the online beam prediction phase, the base station sequentially uses the probe beam index set. The corresponding probe beam sends a reference signal to the target user equipment. The target user equipment measures the received power value corresponding to each probe beam and reports the measurement results to the base station. The base station obtains the target user's online probe beam power vector, denoted as: The target user's online detection beam power vector include The detection beam power value. Target user for the first... The online detection beam power value measured by each detection beam is denoted as: , The base station will use the online detection beam power vector of the target user. Input the trained beam prediction model and calculate the candidate service beam prediction probability vector according to steps 4 to 8, denoted as: .

[0115] Step 13:

[0116] The base station predicts the probability vector based on the candidate serving beam. Select the prediction service beam index, denoted as: Predictive Service Beam Index Predicting probability vectors for candidate service beams The index of the largest element in the middle:

[0117]

[0118] Base station based on predicted service beam index Select candidate service beamcodeword The service beam for the target users.

[0119] Secondly, the present invention provides a beam prediction device based on model alignment, comprising the following modules for implementing the method described in any of the above-mentioned embodiments:

[0120] The probe beam configuration module is used to configure the candidate service beam codebook. and detection ratio coefficient Candidate service beam codebook This includes the number of candidate service beams. The system will use the candidate service beam codeword. The candidate service beam codewords are denoted as follows: ,in, Detection ratio coefficient The range of values ​​is to ;

[0121] The power acquisition module sequentially uses the probe beam index set. The corresponding probe beam transmits a reference signal, and the user equipment measures the received power value corresponding to each probe beam and reports the measurement results to the base station or training equipment; Optimal service beam label for each training sample Obtained through full candidate service beam scanning, historical network measurement records, or offline channel simulation data; each training sample includes at least the [number]th [sample]. The probe beam power vector of each training sample and the Optimal service beam label for each training sample , among which, the The probe beam power vector of each training sample , No. Optimal service beam label for each training sample are integers and satisfy ;

[0122] The normalization module is used to normalize the input probe beam power vector. Logarithmic transformation and centering are performed to obtain the normalized probe beam power vector. Input probe beam power vector Composed of the detection beam power value;

[0123] The signal mode alignment module (signal mode aligner) is used to normalize the probe beam power vector. Calculate the expanded signal embedding vector The signal mode alignment module includes: a first weight matrix. First bias vector Second weight matrix Second bias vector and activation function Activation function Use the GELU function or the ReLU function;

[0124] The text encoding module, including a word embedding layer and a lexical encoder, is used to construct text including prefixes. and suffix text The prompt text, and the prefix text and suffix text Encode the prefix text token embedding matrix to obtain the prefix text token embedding matrix. Suffix text lexical embedding matrix Number of prefix text terms and the number of suffix text morphemes Obtain the predicted token position number Embed the prefix text tokens into the matrix Signal word embedding matrix and suffix text lexical embedding matrix Concatenate them sequentially to obtain the large language model input embedding matrix. and the length of the input sequence of the large language model ;

[0125] The large language model inference module is used to input the large language model into the embedding matrix. Output hidden state matrix The large language model inference module adopts either an encoder-type pre-trained language model or a causal pre-trained language model.

[0126] The classification prediction module (classification head) is used to hide the feature vector of the predicted location. Output candidate service beam logical score vector The classification prediction module includes: a classifier head weight matrix. Classification head bias vector and activation function Activation function Use the GELU function or the ReLU function;

[0127] The training control module is used to execute the first training phase (modal warm-up) and the second training phase (low-rank adaptation fine-tuning). The first training phase uses the AdamW optimizer, and the learning rate for the first training phase is... The range of values ​​is to Training batch size in the first training phase The value ranges from 64 to 512; the first training phase consists of training rounds. The value ranges from 1 to 20; the weight decay coefficient typically ranges from 0 to 0.1; the low-rank dimension in the first training phase. The value ranges from 1 to 64; low-rank scaling factor The value ranges from 1 to 64; low-rank dimensionality reduction matrix. The elements are randomly initialized by the system; low-rank, high-dimensional matrix. The elements are initialized to zero or randomly by the system; the learning rate in the second training phase. The range of values ​​is to Second training phase training batch size The value ranges from 8 to 128; the second training phase consists of training rounds. The value ranges from 1 to 10; the low-rank adaptation random inactivation rate ranges from 0 to 0.2.

[0128] Thirdly, the present invention provides an electronic device, including a processor and a memory, wherein the memory stores program instructions, characterized in that the program instructions, when executed by the processor, implement the method described in any of the above-mentioned embodiments.

[0129] Fourthly, the present invention provides a computer-readable storage medium storing program instructions, characterized in that the program instructions, when executed by a processor, implement the method described in any of the preceding claims.

[0130] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. However, these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A beam prediction method based on model alignment, characterized in that, The steps include: the base station configures a candidate service beam codebook; the base station determines the detection beam based on the candidate service beam codebook; The base station acquires the received power of the probe beam; the system normalizes the received power of the probe beam; the system generates signal word embeddings through a signal mode aligner; the system combines the signal word embeddings with the text word embeddings corresponding to the prompt text and inputs them into a pre-trained large language model; the system calculates the prediction results of candidate service beams through a classification head; the base station selects a service beam based on the prediction results of the candidate service beams.

2. The beam prediction method based on model alignment according to claim 1, characterized in that, The system includes three phases: system configuration and model building, model training, and online beam prediction. The specific implementation steps are as follows: S1. Obtain the number of candidate service beams Calculate the number of detection beams The base station depends on the number of detection beams. From the candidate service beam codebook Select a detection beam and obtain the detection beam index set. According to the candidate service beam codebook The index order is used to perform linearly equally spaced sampling between the first and last candidate service beam codewords to obtain... One sampling location; S2. Construct a beam prediction model that includes a signal modality aligner, a lexical encoder, a word embedding layer, a pre-trained large language model, a classification head, and a low-rank adaptation module. S3. During the model training phase, the base station or training equipment constructs a training sample set. The system obtains the number of samples. Training sample set include One training sample; Get the The probe beam power vector of each training sample and optimal service beam tag ,in, ;No. The probe beam power vector of each training sample Depend on Power value of each detection beam Composition, the first The power value of each detection beam is denoted as , ; S4. Obtain the input probe beam power vector For the input probe beam power vector A logarithmic transformation is performed, and the logarithmic results of the received power of all probe beams are centered to obtain the normalized probe beam power vector. During the model training phase, the input probe beam power vector is... For the first The probe beam power vector of each training sample During the online beam prediction phase, the input probe beam power vector is... The probe beam power vector reported by the target user; S5. Normalize the probe beam power vector The input signal mode aligner is used to calculate the expanded signal embedding vector. Expand the signal embedding vector According to the number of signal words Hidden Dimensions of Large Language Models Divided into A length of The subvector, and the Arrange the subvectors row by row to obtain the signal word embedding matrix. ; S6. Construct text including prefixes and suffix text The prompt text, prefix text Includes beam prediction task description content, suffix text Includes predicted tokens [PRED]; employs a pre-trained large language model. The built-in lexical encoder and word embedding layer are used for prefix text. and suffix text Encode the prefix text token embedding matrix to obtain the prefix text token embedding matrix. and suffix text lexical embedding matrix The prefix text is processed by the lexical encoder. and suffix text The number of prefix text tokens is obtained after encoding. and the number of suffix text morphemes Obtain the predicted token position number Embed the prefix text tokens into the matrix Signal word embedding matrix and suffix text lexical embedding matrix Concatenate them sequentially to obtain the large language model input embedding matrix. and the length of the input sequence of the large language model Length of input sequence for large language models Number of prefix text terms Number of signal words and the number of suffix text morphemes sum; S7. Input the large language model into the embedding matrix. Input pre-trained large language model The hidden state matrix of the large language model is obtained. Based on the number of prefix text lexical units Number of signal words and prediction marker lexical position number Determine the predicted position index of the predicted token in the input sequence of the large language model. Output the hidden state matrix from the large language model. Extract the predicted location index The corresponding predicted location hidden feature vector ; S8. Hide the feature vector of the predicted location. Input the classification header to obtain the candidate service beam logical score vector. Based on the candidate service beam logical score vector Calculate the candidate service beam prediction probability vector ; S9. Predict the probability vector based on the candidate service beam. Calculate the training loss function using the optimal service beam label. The optimal service beam label is obtained through full candidate service beam scanning, historical network measurement records, or offline channel simulation data. S10. Execute the first training phase: Modal warm-up phase. In the first training phase, the system freezes the pre-trained large language model. All backbone parameters are updated, only the first weight matrix in the signal mode aligner is updated. First bias vector Second weight matrix Second bias vector and the class head weight matrix and classification head bias vector Before the start of each training round in the first training phase, the base station or training device performs training on the training sample set. The training samples are randomly shuffled and arranged according to the training batch size of the first training phase. The shuffled training samples are divided into multiple training batches; in each parameter update during the first training phase, the base station or training device selects a training batch and calculates the training loss function according to steps S4 to S9. According to the training loss function Calculate the gradient and update the parameters of the signal mode aligner and the classification head; the first training phase is complete. After one training round, the system enters the second training phase; S11. Execute the second training phase: end-to-end low-rank adaptation fine-tuning phase. Before the start of the second training phase, the pre-trained large language model... Introduce a low-rank adaptation matrix set into at least one attention projection matrix or at least one feedforward network projection matrix; for any original projection matrix By pre-trained large language model The pre-trained weights are obtained and kept frozen; the original projection matrix input dimension is obtained. Original projection matrix output dimension and the input feature vector of the projection layer Set low-rank dimension Low-rank scaling factor Low-rank dimensionality reduction matrix and low-rank increasing-dimensional matrix According to the original projection matrix Low-rank dimensionality reduction matrix and low-rank increasing-dimensional matrix Calculate the projection matrix after low-rank adaptation correction. And use the projection matrix after low-rank adaptation correction. For the input feature vector Perform projection calculations; the training loss function will continue to be used in the second training phase. Training is performed, and the signal mode aligner, classification head, and low-rank dimensionality reduction matrix are jointly updated. and low-rank increasing-dimensional matrix While maintaining the original projection matrix and pre-trained large language models Other backbone parameters are frozen; before the start of each training round in the second training phase, the base station or training device adjusts the training sample set. The training samples in the dataset are randomly shuffled and processed according to the training batch size of the second training phase. The shuffled training samples are divided into multiple training batches; in each parameter update of the second training phase, the base station or training device selects a training batch and calculates the training loss function according to steps S4 to S9. According to the training loss function Calculate the gradient and update the parameters of the signal mode aligner, classification head, and low-rank adaptation matrix set; the second training phase is complete. After one training round, the system determines that the model training is complete and saves the parameters of the signal mode aligner, classification head, and low-rank adaptation matrix group. S12, In the online beam prediction phase, the base station sequentially uses the probe beam index set. The corresponding probe beam sends a reference signal to the target user equipment. The target user equipment measures the received power value corresponding to each probe beam and reports the measurement results to the base station. The base station obtains the target user's online probe beam power vector. Online detection beam power vector of the target user include The detection beam power value, the target user for the first Online detection beam power values ​​measured by each detection beam , The base station will transmit the online detection beam power vector of the target user. Input the trained beam prediction model and calculate the candidate serving beam prediction probability vector according to steps S4 to S8. ; S13, The base station predicts the probability vector based on the candidate serving beam. Select Prediction Service Beam Index Predictive Service Beam Index Predicting probability vectors for candidate service beams The index corresponding to the largest element in the middle is used by the base station based on the predicted serving beam index. Select candidate service beamcodeword The service beam for the target users.

3. The beam prediction method based on model alignment according to claim 1, characterized in that, In step S1, the number of detection beams is... The calculation formula is: ， in, For the number of candidate service beams, , To detect the scaling factor.

4. The beam prediction method based on model alignment according to claim 1, characterized in that, In step S3, the first The probe beam power vector of each training sample Data is collected in the following manner: The base station sequentially uses the probe beam index set. The corresponding probe beam transmits a reference signal, and the user equipment measures the received power value corresponding to each probe beam and reports the measurement results to the base station or training equipment; Optimal service beam label for each training sample Obtained through full candidate service beam scanning, historical network measurement records, or offline channel simulation data; each training sample includes at least the [number]th [sample]. The probe beam power vector of each training sample and the Optimal service beam label for each training sample , among which, the The probe beam power vector of each training sample , No. Optimal service beam label for each training sample are integers and satisfy .

5. The beam prediction method based on model alignment according to claim 1, characterized in that, In step S4, calculate the first... Normalized probe beam power value The formula is as follows: in, For the first Each detection beam power value, It is the numerical stability constant. Let be the power value of the j-th detection beam.

6. The beam prediction method based on model alignment according to claim 1, characterized in that, In step S5, the signal embedding vector is expanded. Calculate using the following formula: ， in, This is the first bias vector. This is the second bias vector. This is the first weight matrix. This is the second weight matrix.

7. The beam prediction method based on model alignment according to claim 1, characterized in that, In step S8, the candidate service beam logical score vector Composed of candidate service beam logic scores, the first The logical score of each candidate service beam is denoted as: , : ， in, This is the class head weight matrix. This is the classifier head bias vector; Candidate service beam prediction probability vector Composed of candidate service beam prediction probability values, the first The candidate service beam prediction probability value corresponding to each candidate service beam is denoted as . Candidate service beam prediction probability value Calculate using the following formula: ， in, Let be the logical score value of the candidate service beam corresponding to the j-th candidate service beam.

8. The beam prediction method based on model alignment according to claim 1, characterized in that, In step S9, the loss function is trained. The calculation formula is: ， in, Indicates the first in the training batch The optimal serving beam label for each training sample in this sample The predicted probability value of the candidate service beam at the location. This represents the current training batch size.

9. The beam prediction method based on model alignment according to claim 1, characterized in that, In step S11, the projection matrix after low-rank adaptation correction for: ， in, It is a low-rank dimension. This is the low-rank scaling factor.

10. A beam prediction device based on model alignment, characterized in that, The following modules are included for implementing the method according to any one of claims 1-9: The probe beam configuration module is used to configure the candidate service beam codebook. and detection ratio coefficient ; Candidate service beam codebook This includes the number of candidate service beams. The system will use the candidate service beam codeword. The candidate service beam codewords are denoted as follows: ,in, Detection ratio coefficient The range of values ​​is to ; The power acquisition module sequentially uses the probe beam index set. The corresponding probe beam transmits a reference signal, and the user equipment measures the received power value corresponding to each probe beam and reports the measurement results to the base station or training equipment; Optimal service beam label for each training sample Obtained through full candidate service beam scanning, historical network measurement records, or offline channel simulation data; each training sample includes at least the [number]th [sample]. The probe beam power vector of each training sample and the Optimal service beam label for each training sample , among which, the The probe beam power vector of each training sample , No. Optimal service beam label for each training sample are integers and satisfy ; The normalization module is used to normalize the input probe beam power vector. Logarithmic transformation and centering are performed to obtain the normalized probe beam power vector. Input probe beam power vector Composed of the detection beam power value; The signal mode alignment module is used to align the normalized probe beam power vector. Calculate the expanded signal embedding vector The signal mode alignment module includes: a first weight matrix. First bias vector Second weight matrix Second bias vector and activation function Activation function Use the GELU function or the ReLU function; The text encoding module is used to construct text including prefixes. and suffix text The prompt text, and the prefix text and suffix text Encode the prefix text token embedding matrix to obtain the prefix text token embedding matrix. Suffix text lexical embedding matrix Number of prefix text terms and the number of suffix text morphemes Obtain the predicted token position number Embed the prefix text tokens into the matrix Signal word embedding matrix and suffix text lexical embedding matrix Concatenate them sequentially to obtain the large language model input embedding matrix. and the length of the input sequence of the large language model ; The large language model inference module is used to input the large language model into the embedding matrix. Output hidden state matrix The large language model inference module adopts either an encoder-type pre-trained language model or a causal pre-trained language model. The classification prediction module is used to hide the feature vector of the predicted location. Output candidate service beam logical score vector The classification prediction module includes: a classifier head weight matrix. Classification head bias vector and activation function Activation function Use the GELU function or the ReLU function; The training control module is used to execute the first and second training phases. The first training phase uses the AdamW optimizer, and the learning rate for the first training phase is... The range of values ​​is to Training batch size in the first training phase The value ranges from 64 to 512; the first training phase consists of training rounds. The value ranges from 1 to 20; the weight decay coefficient typically ranges from 0 to 0.1; the low-rank dimension in the first training phase. The value ranges from 1 to 64; low-rank scaling factor The value ranges from 1 to 64; low-rank dimensionality reduction matrix. The elements are randomly initialized by the system; low-rank, high-dimensional matrix. The elements are initialized to zero or randomly by the system; the learning rate in the second training phase. The range of values ​​is to Second training phase training batch size The value ranges from 8 to 128; the second training phase consists of training rounds. The value ranges from 1 to 10; the low-rank adaptation random inactivation rate ranges from 0 to 0.2.

Images