Unlock instant, AI-driven research and patent intelligence for your innovation.

Millimeter wave communication beam training method based on deep reinforcement learning

A technology of beam training and reinforcement learning, which is applied in the field of millimeter wave wireless communication, can solve the problems of high training overhead, hardware complexity, and high power consumption, and achieve the effect of reducing overhead and hardware complexity

Active Publication Date: 2021-09-17
SOUTHEAST UNIV
View PDF4 Cites 10 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] In view of this, the object of the present invention is to provide a millimeter wave communication beam training method based on deep reinforcement learning, which introduces a framework of reinforcement learning into beam training, so that the trained beam can be adjusted in time as the channel changes. The state of the channel is tracked, which effectively reduces the cost of beam training and ensures the performance of beam training. It solves the technical problems of high training cost, hardware complexity and power consumption of the existing beam training method. At the same time, it supports both Communication scenarios with multiple antennas

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Millimeter wave communication beam training method based on deep reinforcement learning
  • Millimeter wave communication beam training method based on deep reinforcement learning
  • Millimeter wave communication beam training method based on deep reinforcement learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0072] see Figure 1-Figure 5 , this embodiment provides a millimeter wave communication beam training method based on deep reinforcement learning, which specifically includes:

[0073] Consider a mmWave massive MIMO system for a single user with N r Antennas, N at the base station t The antennas are arranged in the form of a uniform linear array (Uniform Linear Array, ULA). According to the widely used Saleh-Valenzuela model, the downlink mmWave channel can be modeled as:

[0074]

[0075] Among them, L, α l , θ l Respectively represent the number of paths, the channel gain of the lth path, the angle of arrival of the channel and the angle of departure of the channel. Usually the path with l=1 is the LOS path, and the other paths are the NLOS paths. definition Θ l and Ψ l is the angle of arrival and angle of departure in the spatial domain, both of which obey the uniform distribution in [0, π]. d t and d r represent the distance between the array antennas at...

Embodiment 2

[0149] On the basis of Embodiment 1, this embodiment provides a millimeter-wave communication beam training device based on deep reinforcement learning, which includes:

[0150] Beam selection module, according to the action A performed at time t t The set of beam combinations used for testing at the next moment is obtained as where I denotes the total number of beamcombinations used for training, Indicates the i-th transmit and receive beam combination.

[0151] The channel sample generation module generates several time-varying channel matrices H according to the random change of the channel steering angle t , use beam scanning method to determine each channel matrix H t Corresponding optimal transmit and receive beam combination

[0152] Receive signal matrix module, use set B t+1 The beams in are tested sequentially to get each beam combination Corresponding received signal strength z t+1 , and the received signals corresponding to other untested beam combinati...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a millimeter wave communication beam training method based on deep reinforcement learning. According to the method, the method comprises the steps of: tracking a millimeter wave channel by defining specific representation of elements such as a state, a target and an award in a reinforcement learning model in a practical problem of beam training; defining a state as an image form, using a convolutional neural network to approximate a value function in reinforcement learning, and defining an action as a triple form based on a moving direction, a distance and a beam coverage range of an optimal beam combination of a channel at a previous moment; when a reward function is designed, using the effective data reachable rate in a time slice as a target value; in the training process of the neural network, using a Q learning method to update network parameters; and using the trained deep Q network for prediction, the action with the maximum Q value is selected, wherein the action corresponds to the beam combination needing to be tested at the next moment.

Description

technical field [0001] The present invention relates to the technical field of millimeter wave wireless communication, in particular to a beam training method for millimeter wave communication based on deep reinforcement learning. Background technique [0002] With the continuous development of wireless communication technology, almost all the spectrum resources in some lower frequency bands have been occupied. In order to meet the requirements of communication performance and obtain more spectrum resources, people's attention has shifted to a higher frequency band, that is, the millimeter wave frequency band. This frequency band refers to a frequency band with a frequency in the range of 30-300 GHz. Spectrum resources in the frequency band are abundant and transmission rates are high, which can meet the needs of some applications with high bandwidth requirements. However, due to the propagation characteristics of mmWave signals, the path loss of mmWave channels is higher t...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): H04B7/06H04B7/08G06N3/04
CPCH04B7/0617H04B7/086G06N3/045
Inventor 戚晨皓姜国力王宇杰
Owner SOUTHEAST UNIV