Beam control algorithm for wireless communication

Abstract

The invention relates to the technical field of communication beam control, in particular to a beam control algorithm for wireless communication, which comprises the following steps of: selecting an initial beam direction set through a preset beam mode library; constructing a channel state feature map through a deep learning model, and predicting a future channel change trend of the user; according to the future channel change trend of the channel state feature map, preliminarily adjusting the beam direction set to generate a plurality of candidate beam directions; the self-adaptive tuning module based on reinforcement learning adjusts the beam width, so that the beam is concentrated in the target user direction and covers a potential multi-path reflection path; and continuously updating the beam pattern library and the deep learning model. According to the invention, the beam direction can be dynamically adjusted, so that the effectiveness and stability of signal coverage are improved, and the communication quality in a complex propagation environment is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of communication beam control, and in particular to a beam control algorithm for wireless communication. Background Art

[0002] In wireless communication systems, as users' demand for high speed and high capacity increases, traditional fixed beam steering technology can no longer meet the requirements of complex communication environments. Especially in 5G and future communication systems, the application of millimeter wave communication technology makes signal propagation more susceptible to environmental obstacles and multipath effects, resulting in signal attenuation and decreased communication performance. Therefore, how to improve channel utilization and coverage quality through more intelligent beam steering algorithms has become the focus of current research.

[0003] Most existing beam steering algorithms are based on real-time channel state information (CSI), and have poor adaptability to changes in user positions and environmental characteristics. Especially when users move at high speed or network load fluctuates greatly, it is difficult for the system to respond and adjust the beam direction quickly, resulting in unstable signal transmission. In addition, traditional beam steering often uses a fixed beam width, which cannot flexibly adapt to complex propagation environments and is difficult to effectively cover multipath reflection paths, further reducing the robustness of communications. Summary of the invention

[0004] The present invention provides a beam control algorithm for wireless communication.

[0005] A beam steering algorithm for wireless communication, comprising:

[0006] S1: Initialize the beam set and select the initial beam direction set θ through the preset beam pattern library init , used to cover a preset spatial area;

[0007] S2: Obtain the historical channel state information (CSI) of the user terminal in real time and construct the channel state feature map F through the deep learning model CSI , channel state feature map F CSI The construction is based on the user's mobility history, environmental reflection characteristics and multipath effects, and is used to predict the user's future channel change trend;

[0008] S3: According to the channel state characteristic graph F CSI Based on the future channel change trend, the beam direction set is preliminarily adjusted. The adjustment process adopts a multi-objective optimization algorithm, comprehensively considering the channel gain, interference signal strength and system power consumption, and generates multiple candidate beam directions;

[0009] S4: For each candidate beam direction, the adaptive tuning module based on reinforcement learning adjusts the beam width so that the beam is not only focused on the target user direction but also covers the potential multipath reflection path to improve the robustness of signal transmission;

[0010] S5: The beam control algorithm is self-iteratively optimized according to the changes in user locations and the real-time situation of network load. By continuously updating the beam pattern library and deep learning model, the beam control can better adapt to the complex wireless communication environment in the future.

[0011] Optionally, the initialization beam set in S1 specifically includes:

[0012] S11: pre-constructing a plurality of beam pattern libraries according to historical data and user distribution characteristics in the wireless communication system, wherein the beam pattern library includes a plurality of preset beam pattern sets with different widths, directions and coverage ranges;

[0013] S12: matching a beam pattern set from a beam pattern library according to the current user's geographical location, target spatial area requirements, and coverage targets;

[0014] S13: Based on the shape and size of the target space area, an initial beam direction set that can cover the target space area is selected from the matching beam pattern set.

[0015] Optionally, the S2 specifically includes:

[0016] S21: receiving historical channel state information of a user terminal in real time through a base station or access point in a wireless communication system, wherein the historical channel state information includes channel gain, signal strength, and channel attenuation information of the user terminal at different time points;

[0017] S22: construct a channel state feature matrix H(t), where the elements in the matrix represent the channel state of the user at different locations and time points. The channel state feature matrix H(t) is constructed based on the user's movement trajectory and the channel features at different time points, and each element represents the channel state at different time and location;

[0018] S23: Input the channel state feature matrix H(t) into a pre-trained deep learning model for feature extraction to extract spatial and temporal features of the channel state;

[0019] S24: Processing the extracted channel state features through a deep learning model to generate a channel state feature map, where the channel state feature map reflects the influence of user movement history, environmental reflection characteristics, and multipath effects on the channel state;

[0020] S25: Using the generated channel state characteristic graph, predict the user's future channel change trend for dynamic adjustment and optimization of the beam direction.

[0021] Optionally, the channel state is also affected by a multipath propagation effect, which is modeled by: Among them, α k is the attenuation coefficient of the kth path, f k is the frequency of the kth path, t is the time, and K represents the total number of multipath paths.

[0022] Optionally, the deep learning model combines a convolutional neural network (CNN) and a recurrent neural network (RNN) to extract spatial and temporal features, wherein the convolutional layer of the convolutional neural network is used to extract spatial features and output a feature map F conv , the recurrent neural network is used to capture time dependencies and output the hidden state feature h at time t t ;

[0023] Channel state feature map F CSI The generation of F is expressed as: CSI =f(F conv ,h t ), where f represents the deep learning model for the feature map F conv and hidden state feature h t The fusion operation, F CSI is the final generated channel state feature map;

[0024] Using the channel state feature map F CSI , by learning historical data, predicting the future channel state, the prediction model adopts the recurrent neural network model, and the prediction is expressed as:

[0025] in, is the predicted channel state at the next time point t+1, including the time variation of channel gain, interference signal strength and multipath propagation effect, and f is the prediction function of the recurrent neural network model.

[0026] Optionally, the S3 specifically includes:

[0027] S31: From the channel state feature map F CSI extracting future channel variation trends from the channel, wherein the trends include time variations of channel gain, interference signal strength, and multipath effects;

[0028] S32: According to the future channel change trend, the initial beam direction set θ is adjusted using a multi-objective optimization algorithm init , the optimization objectives include:

[0029] Channel gain maximization: The optimization objective function is: Among them, G i (θ i ) represents the beam direction θ i The corresponding channel gain, N is the number of beam directions;

[0030] Minimize the interference signal strength: The optimization objective function is: Among them, I i (θ i ) represents the beam direction θ i Interference signal strength at

[0031] Minimize system power consumption: The optimization objective function is: Among them, P i (θ i ) is the beam direction θ i Corresponding system power consumption;

[0032] S33: Through the multi-objective optimization algorithm, the weights of channel gain, interference signal strength and system power consumption are comprehensively considered to generate multiple candidate beam directions θ cand , to meet the optimization needs of future channel state change trends and serve as the basis for the next step of beam direction adjustment.

[0033] Optionally, the S4 specifically includes:

[0034] S41: Set each candidate beam direction θ cand As state input in a reinforcement learning environment, the state includes current channel state information, user location, and multipath reflection path information;

[0035] S42: Define the action space of reinforcement learning as the adjustment range of beam width Δθ, action a t Indicates the reduction or expansion of the current beam width, with the goal of optimizing channel gain and coverage;

[0036] S43: Define reward function R t : is a function that comprehensively considers channel gain, user direction signal quality and multipath reflection path coverage effect, expressed as:

[0037] R t =w4·C(θ cand ,Δθ)+w5·C(θ cand ,Δθ), where C(θ cand , Δθ) represents the beam direction θ cand and the channel gain under beamwidth Δθ, C(θ cand , Δθ) represents the ability of the beam to cover the potential multipath reflection path, w4 and w5 are the weight coefficients of the channel gain and multipath coverage effect;

[0038] S44: Based on the reinforcement learning algorithm, the beam width is adjusted through multiple trainings so that the beam can be focused in the direction of the target user, while covering the multipath reflection path to optimize the signal quality and coverage, and the optimized beam width Δθ is output. opt , used to adjust the beam direction during subsequent communication.

[0039] Optionally, the reinforcement learning algorithm includes defining the state in the reinforcement learning as the current beam direction, beam width, channel state information and user location information; the action is the adjustment of the beam width (the beam can be enlarged or reduced); the reward measures the effect after the beam adjustment, including channel gain, user direction signal quality and coverage of multipath reflection paths;

[0040] The reinforcement learning algorithm starts training, initializes the Q table or, at each time step, selects an action based on the current state (i.e., adjusts the beam width), and executes the action. After executing the action, it updates to the next state and updates the Q table based on the feedback calculated by the reward function.

[0041] Through repeated iterations, the reinforcement learning algorithm gradually learns how to select the best beam width adjustment strategy and finally outputs the optimized beam width.

[0042] Optionally, the S5 specifically includes:

[0043] S51: Acquire the location information of the user terminal and the load of the network in real time, where the network load includes the number of connections of the current base station, the channel occupancy rate and the signal interference intensity;

[0044] S52: inputting the user location change information and the network load data into the deep learning model, and the deep learning model identifies the user location movement pattern and the load distribution characteristics through continuous online training and learning;

[0045] S53: According to the output results of the deep learning model, the beam direction and beam width are dynamically adjusted to adapt to the changes in the user's position and ensure the effectiveness of signal coverage;

[0046] S54: self-update the beam pattern library, optimize the beam direction, beam width and coverage in the beam pattern library based on the actual distribution of users and network load conditions, and generate a new beam pattern suitable for the current network status;

[0047] S55: Through error feedback, the parameters of the deep learning model are updated. The convolutional neural network part of the deep learning model updates the weights of the spatial feature extraction layer, and the recurrent neural network part of the deep learning model updates the weights of the time series prediction layer. The updated model will be able to better adapt to changes in user locations and dynamic changes in network load.

[0048] Optionally, the user location change data is obtained by collecting the user location (x u ,y u ) changes, calculate the position difference Δd u , i.e. the displacement of the user at two moments before and after; the network load data is obtained by collecting the real-time connection number N of the base station conn , channel occupancy U channel , interference signal strength I intf , real-time calculation of network load L net ;

[0049] The deep learning model uses a convolutional neural network to process changes in user locations and a recurrent neural network to process changes in network load over time;

[0050] Input data: X input =(Δd u ,L net ), where Δd u represents the user's mobile displacement, L net Indicates the real-time network load situation;

[0051] Output data: (Δθ pred ,ΔW pred ) = f DL (X input ), where Δθ pred : Predicted beam direction adjustment, ΔW pred : predicted beam width adjustment;

[0052] According to the real-time situation of user location changes and network load data, the deep learning model performs continuous online training and learning updates. At each time step t, the new user location and network load data are input into the model to generate beam adjustment suggestions. Combined with the actual beam effect, the error function is used to calculate the beam adjustment suggestions. Calculate the difference between the model's prediction and the actual value:

[0053] Among them, θ actual and W actual is the actual beam direction and width, θ i and W i The beam direction and width predicted by the model are updated by back propagation through error feedback. The convolutional neural network part updates the weights of the spatial feature extraction layer, and the recurrent neural network part updates the weights of the time series prediction layer.

[0054] Beneficial effects of the present invention:

[0055] The present invention, by introducing a channel state feature map construction method based on deep learning, can accurately predict the user's future channel change trend. Compared with the traditional algorithm that relies on real-time channel state information (CSI), it not only considers the user's historical movement trajectory, but also combines the environmental reflection characteristics and multipath effects, which greatly improves the prediction accuracy of the channel state. Based on these characteristics, the beam direction can be dynamically adjusted to improve the effectiveness and stability of signal coverage and ensure the communication quality in complex propagation environments.

[0056] The present invention, by combining a deep learning model, realizes adaptive dynamic beam control based on user position changes and network load. By extracting spatial features through a convolutional neural network (CNN) and capturing temporal features through a recurrent neural network (RNN), it can analyze the mobility trend of user terminals and changes in network load in real time, accurately predict the adjustment of beam direction and beam width, and more flexibly optimize channel gain in a complex and changeable wireless communication environment, effectively avoid signal interference and coverage blind spots, thereby improving the overall communication quality of the system.

[0057] The present invention comprehensively considers factors such as channel gain, interference signal strength and system power consumption through a multi-objective optimization algorithm, dynamically optimizes the beam direction and width, and can balance signal coverage quality and energy consumption under different user scenarios and communication conditions through an adaptive tuning module of reinforcement learning, thereby achieving precise adjustment of the beam control strategy. It shows significant advantages in high-density user and high-load network environments, and can ensure that the communication system maintains a high energy efficiency ratio and stable network performance without sacrificing user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0059] Figure 1 Schematic diagram of algorithm steps of an embodiment of the present invention;

[0060] Figure 2 A schematic diagram of constructing a channel state characteristic diagram according to an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. At the same time, it is explained here that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments, and those skilled in the art may also adopt other alternatives to implement some known technologies; and the accompanying drawings are only for more specific description of the embodiments, and are not intended to specifically limit the present invention.

[0062] It should be noted that the references to "one embodiment", "an embodiment", "an exemplary embodiment", "some embodiments" and the like in the specification indicate that the embodiments described may include specific features, structures or characteristics, but not every embodiment may include the specific features, structures or characteristics. In addition, when a specific feature, structure or characteristic is described in conjunction with an embodiment, it should be within the knowledge of a person skilled in the art to implement such feature, structure or characteristic in conjunction with other embodiments (whether or not explicitly described).

[0063] In general, a term can be understood, at least in part, from its use in context. For example, depending, at least in part, on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in the singular sense, or can be used to describe a combination of features, structures, or characteristics in the plural sense. Additionally, the term "based on" can be understood as not necessarily intended to convey an exclusive set of factors, but can instead, depending, at least in part, on the context, allow for the presence of other factors that are not necessarily explicitly described.

[0064] like Figure 1-Figure 2 As shown, a beam steering algorithm for wireless communication includes:

[0065] S1: Initialize the beam set and select the initial beam direction set θ through the preset beam pattern library init , used to cover a preset spatial area;

[0066] S2: Obtain the historical channel state information (CSI) of the user terminal in real time and construct the channel state feature map F through the deep learning model CSI , channel state feature map F CSI The construction is based on the user's mobility history, environmental reflection characteristics and multipath effects, and is used to predict the user's future channel change trend;

[0067] S3: According to the channel state characteristic graph F CSI Based on the future channel change trend, the beam direction set is preliminarily adjusted. The adjustment process adopts a multi-objective optimization algorithm, comprehensively considering the channel gain, interference signal strength and system power consumption, and generates multiple candidate beam directions;

[0068] S4: For each candidate beam direction, the adaptive tuning module based on reinforcement learning adjusts the beam width so that the beam is not only focused on the target user direction but also covers the potential multipath reflection path to improve the robustness of signal transmission;

[0069] S5: The beam control algorithm is self-iteratively optimized according to the changes in user locations and the real-time situation of network load. By continuously updating the beam pattern library and deep learning model, the beam control can better adapt to the complex wireless communication environment in the future.

[0070] The initialization beam set in S1 specifically includes:

[0071] S11: pre-build multiple beam pattern libraries according to historical data and user distribution characteristics in the wireless communication system, the beam pattern library including a plurality of preset beam pattern sets with different widths, directions and coverage ranges;

[0072] The beam pattern library is constructed based on historical data and user distribution characteristics in the wireless communication system. Multiple preset beam pattern sets are defined, each with the following parameters:

[0073] Beam direction angle θ: beam propagation direction, range [0°, 360°];

[0074] Beam width Δθ: the coverage angle range of the beam, in degrees;

[0075] Coverage radius R: the signal coverage range of the beam, in meters;

[0076] The beam pattern library presets beam patterns of different directions, widths, and coverage ranges for subsequent selection.

[0077] S12: matching a beam pattern set from a beam pattern library according to the current user's geographical location, target spatial area requirements, and coverage targets;

[0078] The matching is based on whether the user's geographical location is within the coverage area of ​​the beam and whether the direction and width of the beam are suitable for covering the target spatial area.

[0079] The conditions for the matching process are:

[0080]

[0081] Among them, (x u ,y u ) is the current user’s geographic location, A beam is the coverage area of ​​each beam, determined by its direction θ, width Δθ and radius R, A tar getis the area of ​​the target space region; when these conditions are met, the matching is successful, and a beam pattern set that meets the conditions is selected from the beam pattern library.

[0082] S13: Based on the shape and size of the target space area, an initial beam direction set that can cover the target space area is selected from the matching beam pattern set.

[0083] Based on the shape and size of the target spatial area, an initial set of beam directions is selected from the set of matching beam patterns. The goal of the selection is to ensure that the beam direction set can cover the largest part of the target area. The conditions for selecting the beam direction are:

[0084] The shape of the target area is defined as a rectangular or polygonal area with width and height W. target and H target , the beam coverage condition is: Δθ m is the beam width in the beam pattern, R m is the coverage radius in the beam pattern.

[0085] This formula ensures that the angle of the beam covers the width of the target area.

[0086] From the set of matching beam patterns, select the beam direction θ that satisfies the condition m , so that it covers the entire A tar get , the final initial beam direction set is Θ init ,in:

[0087] Θ init ={θ1,θ2,..,θ n} makes

[0088] The beam pattern set is a part of beam patterns selected from the beam pattern library according to specific conditions (such as user location and coverage requirements). The beam pattern set is a subset of beam patterns that meet the requirements matched from the beam pattern library for the current communication scenario.

[0089] The beam direction set is a specific beam direction set selected from the matching beam patterns in the beam pattern set according to the shape and size of the target spatial area. The beam direction set is the actual beam direction configuration ultimately used to cover the target area.

[0090] The beam pattern library is a collection of all beam patterns.

[0091] The beam pattern set is a subset that meets the conditions and is selected from the beam pattern library according to requirements.

[0092] The beam direction set selects the specific beam direction configuration that can cover the target area from the beam pattern set.

[0093] S2 specifically includes:

[0094] S21: receiving historical channel state information of the user terminal in real time through a base station or access point in the wireless communication system, where the historical channel state information includes channel gain, signal strength and channel attenuation information of the user terminal at different time points;

[0095] S22: Construct a channel state feature matrix H(t). The elements in the matrix represent the channel state of the user at different locations and time points. The channel state feature matrix H(t) is constructed based on the user's movement trajectory and the channel characteristics at different time points. Each element represents the channel state at different time and location, which is expressed as:

[0096] in;

[0097] h ij (t) represents the channel state information from base station i to user position j at time t, including channel gain, signal strength and attenuation characteristics. i is the number of base stations, j is the number of sampling points at the user terminal, and t represents time. This matrix records the channel state of the user at different times and locations, and is used to describe the channel characteristics between the user and the base station.

[0098] S23: Input the channel state feature matrix H(t) into a pre-trained deep learning model for feature extraction to extract the spatial and temporal features of the channel state;

[0099] S24: Process the extracted channel state features through a deep learning model to generate a channel state feature map, which reflects the influence of user movement history, environmental reflection characteristics and multipath effects on the channel state;

[0100] S25: Using the generated channel state characteristic graph, predict the user's future channel change trend for dynamic adjustment and optimization of the beam direction.

[0101] The channel state is also affected by the multipath propagation effect, which is modeled by: Among them, α k is the attenuation coefficient of the kth path, f k is the frequency of the kth path, t is the time, and K is the total number of multipath paths. It represents the superposition of multiple path signals from the base station to the user terminal, which can reflect the impact of multipath propagation in the channel.

[0102] The deep learning model combines convolutional neural network (CNN) and recurrent neural network (RNN) to extract spatial and temporal features. The convolutional layer of the convolutional neural network is used to extract spatial features and output feature maps Fconv , the recurrent neural network is used to capture time dependencies and output the hidden state feature h at time t t ;

[0103] The convolutional layer is represented as: F conv =ReLU(W conv *H(t)+b conv ), where F conv is the feature map output by the convolutional layer, W conv is the convolution kernel weight matrix* represents the convolution operation, b conv is the bias term, ReLU is the activation function;

[0104] The recurrent neural network processes time series features, which can be expressed as:

[0105] h t =σ(W h ·x t +U h ·h t-1 +b h );h t is the hidden state at time t, x t is the current input feature, h t-1 is the hidden state at the previous time point, W h and U h is the weight matrix, b h is the bias term, σ is the activation function;

[0106] Channel state feature map F CSI The generation of F is expressed as: CSI =f(F conv ,h t ), where f represents the deep learning model for the feature map F conv and hidden state feature h t The fusion operation, F CSI is the final generated channel state feature map;

[0107] Using the channel state feature map F CSI , by learning historical data, predicting the future channel state, the prediction model adopts the recurrent neural network model, and the prediction is expressed as:

[0108] in, is the predicted channel state at the next time point t+1, including the time variation of channel gain, interference signal strength and multipath propagation effect, and g is the prediction function of the recurrent neural network model.

[0109] S3 specifically includes:

[0110] S31: From the channel state feature map FCSI Extract the future channel change trend, including the time change of channel gain, interference signal strength and multipath effect;

[0111] S32: According to the future channel change trend, the initial beam direction set θ is adjusted using a multi-objective optimization algorithm init , the optimization objectives include:

[0112] Channel gain maximization: The optimization objective function is: Among them, G i (θ i ) represents the beam direction θ i The corresponding channel gain, N is the number of beam directions;

[0113] Minimize the interference signal strength: The optimization objective function is: Among them, I i (θ i ) represents the beam direction θ i Interference signal strength at

[0114] Minimize system power consumption: The optimization objective function is: Among them, P i (θ i ) is the beam direction θ i Corresponding system power consumption;

[0115] S33: Through the multi-objective optimization algorithm, the weights of channel gain, interference signal strength and system power consumption are comprehensively considered to generate multiple candidate beam directions θ cand , to meet the optimization needs of future channel state change trends and serve as the basis for the next step of beam direction adjustment.

[0116] The steps of the multi-objective optimization algorithm are as follows:

[0117] Step 1: Determine the objective function. In order to solve this optimization problem, three objective functions are defined:

[0118] Maximizing channel gain: f1(θ i )=G i (θ i ), where G i (θ i ) is the beam direction θ i The corresponding channel gain.

[0119] Minimize the interference signal strength: f2(θ i )=I i (θ i ), where I i (θ i ) is the beam direction θ i The interference signal strength at the location.

[0120] Minimize system power consumption: f3(θ i )=P i (θ i ), where P i (θ i ) is the beam direction θ i The corresponding system power consumption.

[0121] Step 2: Determine the weights and assign weights w1, w2, and w3 to each objective function, which respectively represent the importance of channel gain, interference signal strength, and system power consumption in the optimization process. The weights satisfy the following conditions: w1+w2+w3=1. The weight values ​​are set according to the priority in the actual application. For example, if you want to give priority to channel gain, you can set w1 to be larger.

[0122] Step 3: Construct a comprehensive objective function and use the weighted sum method to combine the three objective functions into a single objective function:

[0123] F(θ i )=w1·f1(θ i )-w2·f2(θ i )-w3·f3(θ i ), f1(θ i ) is the channel gain that needs to be maximized, so the positive weight w1,f2(θ i ) and f3(θ i ) needs to be minimized, hence the negative weights w2 and w3.

[0124] Step 4: Initialize the candidate set of beam directions, starting from the initial beam direction set θ init One or more initial candidate beam directions are selected as the initial points for optimization.

[0125] Step 5: Iterative optimization, through the optimization algorithm in the comprehensive objective function F(θ i ) to find the optimal beam direction, taking the gradient descent method as an example:

[0126] 1. Calculate the gradient: For the comprehensive objective function F(θ i ) Calculate the beam direction θ i The gradient is:

[0127] 2. Update beam direction: Update beam direction θ according to the negative gradient direction i :

[0128] Where η is the learning rate, which determines the step size.

[0129] 3. Convergence condition: When the gradient changes When it is less than the set threshold ∈ or reaches the maximum number of iterations, the iteration stops.

[0130] Step 6: Generate multiple candidate beam directions. During the iterative optimization process, record all beam directions that meet the optimization objectives and generate multiple candidate beam direction sets θ cand . The candidate set contains multiple optimization results.

[0131] Step 7: Based on the weight allocation and the prediction of future channel states, the optimal beam direction or combination is selected from the candidate beam direction set.

[0132] S4 specifically includes:

[0133] S41: Set each candidate beam direction θ cand As the state input in the reinforcement learning environment, the state includes the current channel state information, user location and multipath reflection path information;

[0134] S42: Define the action space of reinforcement learning as the adjustment range of beam width Δθ, action a t Indicates the reduction or expansion of the current beam width, with the goal of optimizing channel gain and coverage;

[0135] S43: Define reward function R t : is a function that comprehensively considers channel gain, user direction signal quality and multipath reflection path coverage effect, expressed as:

[0136] R t =w4·G(θ cand ,Δθ)+w5·C(θ cand ,Δθ), where G(θ cand , Δθ) represents the beam direction θ cand and the channel gain under beamwidth Δθ, C(θ cand , Δθ) represents the ability of the beam to cover the potential multipath reflection path, w4 and w5 are the weight coefficients of the channel gain and multipath coverage effect;

[0137] S44: Based on the reinforcement learning algorithm, the beam width is adjusted through multiple trainings so that the beam can be focused in the direction of the target user, while covering the multipath reflection path to optimize the signal quality and coverage, and the optimized beam width Δθ is output. opt , used to adjust the beam direction during subsequent communication.

[0138] The reinforcement learning algorithm includes defining the state in reinforcement learning as the current beam direction, beam width, channel state information and user location information; the action is the adjustment of the beam width (the beam can be enlarged or reduced); the reward measures the effect of adjusting the beam, including channel gain, signal quality in the user direction and coverage of multipath reflection paths;

[0139] The reinforcement learning algorithm starts training, initializes the Q table or, at each time step, selects an action based on the current state (i.e., adjusts the beam width), and executes the action. After executing the action, it updates to the next state and updates the Q table based on the feedback calculated by the reward function.

[0140] Through repeated iterations, the reinforcement learning algorithm gradually learns how to select the best beam width adjustment strategy and finally outputs the optimized beam width.

[0141] The specific steps of the reinforcement learning algorithm are as follows:

[0142] 1. Define Reinforcement Learning:

[0143] Status t , at time step t, state s t Including the current beam direction θ cand , beam width Δθ, channel state information, and user location information, where the state represents the current configuration of the beam and environmental conditions;

[0144] Action a t : The action space includes the adjustment of the beam width Δθ, action a t Defined as increasing or decreasing the beam width, the actions include: a t ∈{Δθ+δ,Δθ-δ}, where δ is the step size of beam width adjustment;

[0145] Reward R t : The reward function measures the quality of each action. The function should take into account the channel gain, the quality of the signal in the user direction, and the coverage of the multipath reflection path. It is defined as follows:

[0146] R t =w4·G(θ cand ,Δθ)+w5·C(θ cand ,Δθ);

[0147] 2. Training steps of reinforcement learning algorithm:

[0148] 2.1, Initialize a Q table Q(s t , a t ), record the reward value of the state-action pair.

[0149] 2.2 Execute actions and obtain feedback: At each time step t, according to the current state s tSelect action a via the ∈-greedy strategy t ,Right now:

[0150]

[0151] In the early stages of training, the environment is explored with high probability ∈, and as training progresses, exploration is gradually reduced, increasing the weight of exploiting existing policies.

[0152] 2.3 Execute actions and update status: Execute action a t Adjust the beam width Δθ and update to the next state s t+1 , and get reward R t , the reward is calculated by the aforementioned reward function.

[0153] 2.4 Update Q value or DQN model: Update the Q table using the following formula: Q(s t , a t )←Q(s t , a t )+ɑ[R t +γmax a′ Q(s t+1 , a′)-Q(s t , a t )], where α is the learning rate and γ is the discount factor.

[0154] Repeat the above steps. After multiple trainings, the model will gradually learn to select the optimal beam width adjustment strategy to achieve the best results in terms of channel gain and multipath reflection coverage.

[0155] S5 specifically includes:

[0156] S51: Acquire the location information of the user terminal and the load of the network in real time, where the network load includes the number of connections of the current base station, the channel occupancy rate and the signal interference intensity;

[0157] S52: inputting the user location change information and the network load data into the deep learning model, and the deep learning model identifies the user location movement pattern and the load distribution characteristics through continuous online training and learning;

[0158] S53: According to the output results of the deep learning model, the beam direction and beam width are dynamically adjusted to adapt to the changes in the user's position and ensure the effectiveness of signal coverage;

[0159] S54: self-update the beam pattern library, optimize the beam direction, beam width and coverage in the beam pattern library based on the actual distribution of users and network load conditions, and generate a new beam pattern suitable for the current network status;

[0160] S55: Through error feedback, the parameters of the deep learning model are updated. The convolutional neural network part of the deep learning model updates the weights of the spatial feature extraction layer, and the recurrent neural network part of the deep learning model updates the weights of the time series prediction layer. The updated model will be able to better adapt to changes in user locations and dynamic changes in network load.

[0161] The user location change data is collected by collecting the user location (x u ,y u ) changes, calculate the position difference Δd u , that is, the displacement of the user at two moments before and after; the network load data is collected by collecting the real-time connection number N of the base station conn , channel occupancy U channel , interference signal strength I intf , real-time calculation of network load L net ;

[0162] The deep learning model uses a convolutional neural network to handle changes in user location and a recurrent neural network to handle temporal changes in network load;

[0163] Input data: X input =(Δd u ,L net ), where Δd u represents the user's mobile displacement, L net Indicates the real-time network load situation;

[0164] Output data: (Δθ pred ,ΔW pred ) = f DL (X input ), where Δθ pred : Predicted beam direction adjustment, ΔW pred : predicted beam width adjustment;

[0165] According to the real-time situation of user location changes and network load data, the deep learning model performs continuous online training and learning updates. At each time step t, the new user location and network load data are input into the model to generate beam adjustment suggestions. Combined with the actual beam effect, the error function is used to calculate the beam adjustment suggestions. Calculate the difference between the model's prediction and the actual value:

[0166] Among them, θ actual and W actual is the actual beam direction and width, θ i and W iThe beam direction and width predicted by the model are updated by back propagation through error feedback. The convolutional neural network part updates the weights of the spatial feature extraction layer, and the recurrent neural network part updates the weights of the time series prediction layer.

[0167] After each adjustment of the beam direction and width according to the deep learning model, the beam pattern library also needs to be updated accordingly to maintain its adaptability. The conditions for updating the beam pattern library are:

[0168] When the user position change Δdu is large or the network load fluctuates violently, the beam direction and width need to be adjusted. Therefore, the beam pattern in the beam pattern library needs to be dynamically updated. pred , ΔW pred : Applied to the beam pattern library adjustment for adjustment.

[0169] The present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention. In order to make the public have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention, but those skilled in the art can fully understand the present invention without the description of these details. In addition, in order to avoid unnecessary confusion about the essence of the present invention, well-known methods, processes, procedures, components and circuits are not described in detail.

[0170] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A beam steering algorithm for wireless communication, characterized in that: include: S1: Initialize the beam set and select the initial beam direction set θ through the preset beam pattern library init , used to cover a preset spatial area; S2: Obtain historical channel state information of user terminals in real time and construct a channel state feature map F through a deep learning model CSI , channel state feature map F CSI The construction is based on the user's mobility history, environmental reflection characteristics and multipath effects, and is used to predict the user's future channel change trend; S3: According to the channel state characteristic graph F CSI Based on the future channel change trend, the beam direction set is preliminarily adjusted. The adjustment process adopts a multi-objective optimization algorithm, comprehensively considering the channel gain, interference signal strength and system power consumption, and generates multiple candidate beam directions; S4: For each candidate beam direction, the adaptive tuning module based on reinforcement learning adjusts the beam width so that the beam is not only focused on the target user direction but also covers the potential multipath reflection path; S5: Based on the changes in user locations and the real-time situation of network load, the beam steering algorithm is self-iteratively optimized by continuously updating the beam pattern library and deep learning model.

2. A beam steering algorithm for wireless communication according to claim 1, characterized in that: The initialization beam set in S1 specifically includes: S11: pre-constructing a plurality of beam pattern libraries according to historical data and user distribution characteristics in the wireless communication system, wherein the beam pattern library includes a plurality of preset beam pattern sets with different widths, directions and coverage ranges; S12: matching a beam pattern set from a beam pattern library according to the current user's geographical location, target spatial area requirements, and coverage targets; S13: Based on the shape and size of the target space area, an initial beam direction set that can cover the target space area is selected from the matching beam pattern set.

3. A beam steering algorithm for wireless communication according to claim 1, characterized in that: The S2 specifically includes: S21: receiving historical channel state information of a user terminal in real time through a base station or access point in a wireless communication system, wherein the historical channel state information includes channel gain, signal strength, and channel attenuation information of the user terminal at different time points; S22: construct a channel state feature matrix H(t), where the elements in the matrix represent the channel state of the user at different locations and time points. The channel state feature matrix H(t) is constructed based on the user's movement trajectory and the channel features at different time points, and each element represents the channel state at different time and location; S23: Input the channel state feature matrix H(t) into a pre-trained deep learning model for feature extraction to extract spatial and temporal features of the channel state; S24: Processing the extracted channel state features through a deep learning model to generate a channel state feature map, where the channel state feature map reflects the influence of user movement history, environmental reflection characteristics, and multipath effects on the channel state; S25: Using the generated channel state characteristic graph, predict the user's future channel change trend for dynamic adjustment and optimization of the beam direction.

4. A beam steering algorithm for wireless communication according to claim 3, characterized in that: The channel state is also affected by multipath propagation effects, which are modeled by: Among them, α k is the attenuation coefficient of the kth path, f k is the frequency of the kth path, t is the time, and K represents the total number of multipath paths.

5. A beam steering algorithm for wireless communication according to claim 4, characterized in that: The deep learning model combines convolutional neural network and recurrent neural network to extract spatial and temporal features, wherein the convolutional layer of the convolutional neural network is used to extract spatial features and output feature map F conv , the recurrent neural network is used to capture time dependencies and output the hidden state feature h at time t t ; Channel state feature map F CSI The generation of F is expressed as: CSI =f(F conv ,h t ), where f represents the deep learning model for the feature map F conv and hidden state feature h t The fusion operation, F CSI is the final generated channel state feature map; Using the channel state feature map F CSI , by learning historical data, predicting the future channel state, the prediction model adopts the recurrent neural network model, and the prediction is expressed as: in, is the predicted channel state at the next time point t+1, including the time variation of channel gain, interference signal strength and multipath propagation effect, and g is the prediction function of the recurrent neural network model.

6. A beam steering algorithm for wireless communication according to claim 1, characterized in that: The S3 specifically includes: S31: From the channel state feature map F CSI extracting future channel variation trends from the channel, wherein the trends include time variations of channel gain, interference signal strength, and multipath effects; S32: According to the future channel change trend, the initial beam direction set θ is adjusted using a multi-objective optimization algorithm init , the optimization objectives include: Channel gain maximization: The optimization objective function is: Among them, G i (θ i ) represents the beam direction θ i The corresponding channel gain, N is the number of beam directions; Minimize the interference signal strength: The optimization objective function is: Among them, I i (θ i ) represents the beam direction θ i Interference signal strength at Minimize system power consumption: The optimization objective function is: Among them, P i (θ i ) is the beam direction θ i Corresponding system power consumption; S33: Through the multi-objective optimization algorithm, the weights of channel gain, interference signal strength and system power consumption are comprehensively considered to generate multiple candidate beam directions θ cand , to meet the optimization needs of future channel state change trends.

7. A beam steering algorithm for wireless communication according to claim 6, characterized in that: The S4 specifically includes: S41: Set each candidate beam direction θ cand As state input in a reinforcement learning environment, the state includes current channel state information, user location, and multipath reflection path information; S42: Define the action space of reinforcement learning as the adjustment range of beam width Δθ, action a t Indicates the reduction or expansion of the current beam width, with the goal of optimizing channel gain and coverage; S43: Define reward function R t : is a function that comprehensively considers channel gain, user direction signal quality and multipath reflection path coverage effect, expressed as: R t =w4·G(θ cand ,Δθ)+w5C(θ cand ,Δθ), where G(θ cand , Δθ) represents the beam direction θ cand and the channel gain under beamwidth Δθ, C(θ cand ,Δθ) represents the ability of the beam to cover the potential multipath reflection path, w4 and w5 are the weight coefficients of the channel gain and multipath coverage effect; S44: Based on the reinforcement learning algorithm, the beam width is adjusted through multiple trainings so that the beam can be focused in the direction of the target user, while covering the multipath reflection path to optimize the signal quality and coverage, and the optimized beam width Δθ is output. opt .

8. A beam steering algorithm for wireless communication according to claim 7, characterized in that: The reinforcement learning algorithm includes defining the state in the reinforcement learning as the current beam direction, beam width, channel state information and user location information; the action is the adjustment of the beam width; the reward measures the effect after the beam adjustment, including channel gain, user direction signal quality and coverage of multipath reflection paths; The reinforcement learning algorithm starts training, initializes the Q table or, at each time step, selects an action based on the current state and executes the action. After executing the action, it updates to the next state and updates the Q table based on the feedback calculated by the reward function. Through repeated iterations, the reinforcement learning algorithm gradually learns how to select the best beam width adjustment strategy and finally outputs the optimized beam width.

9. The beam steering algorithm for wireless communication according to claim 1, characterized in that: The S5 specifically includes: S51: Acquire the location information of the user terminal and the load of the network in real time, where the network load includes the number of connections of the current base station, the channel occupancy rate and the signal interference intensity; S52: inputting the user location change information and the network load data into the deep learning model, and the deep learning model identifies the user location movement pattern and the load distribution characteristics through continuous online training and learning; S53: According to the output results of the deep learning model, the beam direction and beam width are dynamically adjusted to adapt to the changes in the user's position and ensure the effectiveness of signal coverage; S54: self-update the beam pattern library, optimize the beam direction, beam width and coverage in the beam pattern library based on the actual distribution of users and network load conditions, and generate a new beam pattern suitable for the current network status; S55: Through error feedback, the parameters of the deep learning model are updated. The convolutional neural network part of the deep learning model updates the weights of the spatial feature extraction layer, and the recurrent neural network part of the deep learning model updates the weights of the time series prediction layer.

10. A beam steering algorithm for wireless communication according to claim 9, characterized in that: The user location change data is obtained by collecting the user location (x u ,y u ) changes, calculate the position difference Δd u , that is, the displacement of the user at two moments before and after; The network load data is obtained by collecting the real-time connection number N of the base station. conn , channel occupancy U channel , interference signal strength I intf , real-time calculation of network load L net ; The deep learning model uses a convolutional neural network to process changes in user locations and a recurrent neural network to process changes in network load over time; Input data: X input =(Δd u ,L net ), where Δd u represents the user's mobile displacement, L net Indicates the real-time network load situation; Output data: (Δθ pred ,ΔW pred ) = f DL (X input ), where Δθ pred : Predicted beam direction adjustment, ΔW pred : predicted beam width adjustment; According to the real-time situation of user location changes and network load data, the deep learning model performs continuous online training and learning updates. At each time step t, the new user location and network load data are input into the model to generate beam adjustment suggestions. Combined with the actual beam effect, the error function is used to calculate the beam adjustment suggestions. Calculate the difference between the model's prediction and the actual value: Among them, θ actual and W actual is the actual beam direction and width, θ i and W i The beam direction and width predicted by the model are updated by back propagation through error feedback. The convolutional neural network part updates the weights of the spatial feature extraction layer, and the recurrent neural network part updates the weights of the time series prediction layer.

Images