Millimeter wave beam alignment method supporting identification of multiple candidate users

Abstract

The invention relates to a millimeter wave beam alignment method supporting multi-candidate user identification, and belongs to the field of wireless communication. The method comprises the following steps: acquiring model parameters of a multi-sensor millimeter wave communication system, constructing a multi-modal perception-assisted wireless communication beam alignment problem model, and converting a beam alignment problem in a multi-candidate scene into an optimization problem based on machine learning; constructing a beam alignment neural network model in multiple candidate scenes and training the beam alignment neural network model; and obtaining sensing data of different modes, inputting the sensing data into the trained beam alignment neural network model, identifying bounding box information of a user, and performing beam alignment by using the identified bounding box information of the user. According to the invention, the precision and the stability of base station user identification are improved, the directivity precision and the signal receiving quality of millimeter wave communication are improved, and the communication delay and the calculation overhead of a millimeter wave system are reduced.

Description

Technical Field

[0001] The present invention belongs to the field of wireless communications and relates to a millimeter wave beam alignment method supporting multiple candidate user identification. Background Art

[0002] With the rapid development of artificial intelligence and communication technologies, next-generation mobile communication networks are gradually adopting millimeter wave and higher frequency band communication technologies to meet the needs of high-speed, low-latency communication. However, millimeter wave signals face severe path loss and penetration attenuation during propagation, and rely heavily on large-scale antenna arrays and precise beam alignment techniques to ensure communication quality. Traditional beam scanning-based alignment methods, while reliable, require traversing all possible beam direction combinations, which incurs significant training overhead and latency in large-scale antenna array scenarios. Therefore, there is an urgent need to explore new millimeter wave beam alignment methods that can significantly reduce training overhead while maintaining beam alignment accuracy, thereby reducing the latency and consumption of millimeter wave communication systems.

[0003] Since it does not rely on channel state information, vision-assisted machine learning methods can effectively reduce the overhead of beam management and channel estimation in millimeter wave communication systems. This method uses a machine learning model to process the visual image information collected by the video sensor installed on the base station to perform millimeter wave beam prediction and alignment. At present, most vision-assisted millimeter wave beam prediction and alignment methods are mainly aimed at scenarios with a single candidate user, in which the system always serves this user. However, in actual communication systems, there are often multiple candidate users, which causes most existing methods to face the problem of user identification ambiguity in this scenario, thereby limiting their applicability for beam prediction and alignment in scenarios with multiple candidate users. Therefore, how to invent a millimeter wave beam alignment method that supports multiple candidate user identification, uses multimodal data for user identification, and provides accurate and stable beam alignment has become an important challenge currently faced. Summary of the Invention

[0004] In view of this, the object of the present invention is to provide a millimeter wave beam alignment method that supports multi-candidate user identification, utilizes data from multiple modalities for user identification, and provides more accurate and stable beam alignment.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A millimeter wave beam alignment method supporting multiple candidate user identification, the method comprising:

[0007] Obtain the model parameters of a multi-sensor millimeter-wave communication system, build a multimodal sensing-assisted wireless communication beam alignment problem model, and transform the beam alignment problem in multiple candidate scenarios into a machine learning-based optimization problem.

[0008] Build and train a beam alignment neural network model for multiple candidate scenarios;

[0009] The perception data of different modalities are input into the trained beam alignment neural network model to identify the user's bounding box information, and the identified user bounding box information is used for beam alignment.

[0010] Furthermore, the beam alignment problem model is constructed as follows:

[0011] In a vision-assisted millimeter-wave communication system model with multiple candidate users, the base station includes an antenna, a lidar sensor, and a visual data sensor; the base station uses a predefined beamforming codebook Q is the total number of beamforming vectors in the codebook, is the beamforming vector in the codebook;

[0012] At time step t, if each mmWave service user u is served by the base station using the beamforming vector f q [t] service, channel is The downlink from the base station to the user is expressed as:

[0013]

[0014] Where, P T is the average transmit power, To transmit complex symbols, (·) H represents the conjugate transpose, n[t] represents a noise sample extracted from a complex Gaussian distribution, and k is the index of the subcarrier;

[0015] Channel h k [t] uses the geometric physical channel model and is expressed as:

[0016]

[0017] Where L represents the total number of channels; α l , τ l ,θ l and φ l represent path gain, delay, arrival azimuth and arrival elevation respectively; D c is the cyclic prefix length, K is the total number of subcarriers, T s is the sampling time, p(·) is the pulse shaping function, and a is the array steering vector;

[0018] In the case of multiple candidate users, the multimodal user recognition task is performed using visual and location information. Based on the available perception data at a given time t, the base station aims to lock the unique user in the image. The goal of this user recognition task is to use the image information V[t] and location information G[t] to use the prediction function f Θ Identify the only user who needs service from U candidate users, as shown in the following formula:

[0019]

[0020] Where b u is the true label, For the predicted results, is the conditional probability;

[0021] After completing user identification, the beam alignment is performed on the user who needs service. The beamforming task is to select the user from the predefined codebook. Select the index as The optimal beamforming vector To maximize the received signal power; therefore, the beam alignment optimization problem is expressed as:

[0022]

[0023] Where f is the beamforming vector.

[0024] Furthermore, the beam alignment problem in multiple candidate scenarios is transformed into an optimization problem based on machine learning. Based on the characteristic that the signal is concentrated in a certain direction in space during beam alignment, the spatial dimension of the scene is divided into multiple sectors using beam vectors, and a unique value is assigned to each sector.

[0025] The machine learning task is a classification task using a predetermined codebook, using the bounding box information of the target user in the image and using a prediction function to predict the optimal beam index of the target user.

[0026] Furthermore, a beam alignment neural network model is constructed under multiple candidate scenarios. The model identifies the coordinates of the user's bounding box center point based on the acquired perception data of different modalities, and uses the identified coordinates of the user's bounding box center point to perform beam alignment.

[0027] Among them, the beam alignment neural network model includes a feature extraction module, a user identification module and a beam alignment module; the feature extraction module processes the input image data and position data, extracts the bounding box and bounding box center point coordinates of the detected object according to the image data, extracts position features according to the position data, and splices the visual features and position features into multimodal information; the user identification module performs user identification based on the multimodal information; the beam alignment module outputs the optimal beam index according to the bounding box center point coordinates corresponding to the identified user.

[0028] Furthermore, the feature extraction module uses YOLOv3 to process the input visual image and generate bounding boxes for all detected objects in the image at time step t, which is expressed as V[t] = [x c ,y c ], x c 、y c The horizontal and vertical coordinates of the center point of the bounding box of the detected object are respectively; the position data is the latitude and longitude information of the user, which is normalized to obtain the position feature Concatenate V[t] and G[t] into a four-dimensional vector

[0029] The user identification module uses a multi-layer perceptron based on residual connections to process multimodal information X[t] and combine it with the Sigmoid function to achieve user identification;

[0030] The beam alignment module performs classification prediction based on the coordinates of the center point of the bounding box corresponding to the identified user through a fully connected network and outputs the optimal beam index.

[0031] Furthermore, the beam alignment neural network model is trained, including using a binary cross entropy loss function to optimize the user identification problem. The binary cross entropy loss function is expressed as:

[0032]

[0033] Where y and are the true label and the probability that the model predicts it to be a real user, respectively. N represents the number of samples in each batch;

[0034] Use the cross entropy loss function to optimize the model's optimal beam index prediction problem;

[0035] The model parameters are updated through binary cross entropy loss and cross entropy loss until the model training reaches the termination condition.

[0036] The beneficial effects of the present invention are:

[0037] (1) To address the recognition ambiguity problem caused by the coexistence of multiple candidate users in actual scenarios, the present invention constructs multimodal fusion information by fusing visual images and user location information. Through the collaborative processing of the coordinates of the center point of the bounding box and the position features by the neural network, the accuracy and robustness of user identity discrimination can be enhanced, multi-user interference is eliminated, the accuracy and stability of base station user identification are improved, a reliable foundation is laid for subsequent beam alignment, and the directional accuracy and signal reception quality of millimeter wave communication are greatly improved.

[0038] (2) Traditional beam scanning requires traversing a large number of beam combinations, while this invention transforms user identification and beam selection into an end-to-end machine learning optimization problem. Based on the spatial mapping characteristics of the bounding box center point coordinates, the model can directly predict the optimal beam index, eliminating the complex channel state information acquisition and iterative scanning process, reducing the communication latency and computational overhead of the millimeter wave system, and can be deployed in large-scale millimeter wave communication systems.

[0039] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:

[0041] Figure 1 Schematic diagram of the millimeter wave communication system model for visual assistance;

[0042] Figure 2 This is a schematic diagram of the millimeter wave beam alignment neural network model that supports multiple candidate user identification proposed in the present invention;

[0043] Figure 3 This is a flow chart of the millimeter wave beam alignment that supports multiple candidate user identification proposed by the present invention. DETAILED DESCRIPTION

[0044] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0045] Among them, the accompanying drawings are only for illustrative purposes and represent only schematic diagrams rather than actual pictures, and should not be understood as limiting the present invention. In order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the dimensions of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions may be omitted in the accompanying drawings.

[0046] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "back", etc. indicating directions or positional relationships, they are based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operate in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0047] An embodiment of the present invention provides a millimeter wave beam alignment method that supports multi-candidate user identification. In a visually assisted millimeter wave communication system model, various key parameters in the system are obtained, and perception data of different modalities is obtained from multiple sensors equipped in the system. By constructing a multimodal data fusion neural network model for millimeter wave communication user identification and introducing residual connection training neural network, the model stability is optimized and the convergence speed is improved. The identified user boundary box information is used for beam alignment to achieve real-time prediction of the optimal beam at the future moment, thereby ensuring millimeter wave communication efficiency while reducing latency and computational overhead.

[0048] The method is as follows:

[0049] 1. Obtain the model parameters of the multi-sensor millimeter-wave communication system, build a multimodal perception-assisted wireless communication beam alignment problem model, and transform the beam alignment problem in multiple candidate scenarios into a machine learning-based optimization problem.

[0050] like Figure 1 As shown in the figure, in a model of a visually assisted millimeter wave communication system with multiple candidate users, a base station serves a mobile user device. The base station contains N antennas, a lidar sensor, and an RGB camera (visual data sensor) to provide awareness of the surrounding environment. The base station uses a predefined beamforming codebook. Q is the total number of beamforming vectors in the codebook, is the beamforming vector in the codebook. At time step t, if each mmWave service user u is served by the base station using the beamforming vector f q[t] service, channel is The downlink from the base station to the user is expressed as:

[0051]

[0052] in, P T is the average transmit power, To transmit complex symbols, (·) H represents the conjugate transpose, n[t] represents a noise sample extracted from a complex Gaussian distribution, k is the index of the subcarrier, and the channel h k [t] adopts a geometric physical channel model, which is expressed as:

[0053]

[0054] Where L represents the total number of channels; α l , τ l ,θ l and φ l Denote path gain, delay, azimuth angle of arrival and elevation angle of arrival respectively. c is the cyclic prefix length, K is the total number of subcarriers, T s is the sampling time, p(·) is the pulse shaping function, and a is the array steering vector.

[0055] In the case of multiple candidate users, the multimodal user recognition task is performed using visual and location information. Based on the available perception data at a given time t, the base station aims to lock the unique user in the image. The goal of this task is to use the image information V[t] and location information G[t] to use a prediction function f Θ Identify the only user who needs service from U candidate users. This task can be expressed as:

[0056]

[0057] Where b u is the true label, For the predicted results, is the conditional probability;

[0058] After completing user identification, beam alignment is performed on the user who needs service. The beamforming task is to select the Select the index as The optimal beamforming vector To maximize the received signal power. Therefore, the beam alignment optimization problem can be expressed as:

[0059]

[0060] Where f is the beamforming vector.

[0061] From a predetermined codebook The best beam index is selected in the CNN, and the signal is concentrated in a certain direction in space; the spatial dimension of the scene is divided into many possibly overlapping sectors by the beam vector, and a unique value is assigned to each sector.

[0062] Therefore, the machine learning task can be viewed as a classification task using a pre-determined codebook, where the user’s real-time position in the visual scene determines which beam index in the codebook to assign. Through a prediction function g φ Directly predict the optimal beam index, which can be expressed as:

[0063]

[0064] 2. Build a beam alignment neural network model for multiple candidate scenarios, obtain perception data of different modalities from the system's sensors, identify the user's bounding box information, and use the identified user bounding box information for beam alignment.

[0065] Among them, the beam alignment neural network model is as follows Figure 2 As shown in Figure 3, the model mainly consists of three parts: feature extraction module, user identification module and beam alignment module.

[0066] The feature extraction block processes the raw input data. Its primary goal is to identify and capture essential features that are crucial for subsequent user identification and beam alignment. The feature extraction block uses an object detection algorithm to detect objects in the image, generating bounding box information. It then normalizes the user's GPS data and concatenates the two as input to the user identification block.

[0067] The user identification module is composed of a multi-layer perceptron with residual connections, which aims to improve model stability and accelerate convergence. The user identification module identifies users based on the spliced ​​information from the feature extraction module.

[0068] In the beam alignment module, the coordinates of the center point of the bounding box corresponding to the identified user are passed through multiple fully connected layers and finally input into the fully connected network for classification prediction. The classifier is responsible for predicting the optimal beam in the future, that is, outputting the optimal beam index.

[0069] The processing in the beam alignment neural network model is as follows:

[0070] (1) The original input data is processed using the feature extraction module; the feature extraction module simplifies the original data by extracting key information, thereby minimizing the feature space.

[0071] For the original image data, the feature extraction module uses object detection to identify and annotate key objects in the image. Specifically, YOLOv3 is used to process the input visual image and generate bounding boxes for all detected objects in the image at time step t, which is expressed as V[t] = [x c ,y c ], Among them, x c 、y c The horizontal and vertical coordinates represent the center point of the bounding box of the detected object, respectively. Therefore, the bounding box obtained from the original image is used as the key feature for user identification and beam alignment tasks.

[0072] The location data is the latitude and longitude information provided by the user's GPS. It is normalized to reduce the impact of its error and is expressed as

[0073] For visual features and position features, concatenate them into a four-dimensional vector As the input of the user recognition module; the coordinates of the center point of the user's bounding box Serves as input to the beam alignment module.

[0074] (2) The data processed by the feature extraction module enters the user identification module. The user identification module adopts a multi-layer perceptron based on residual connection. By processing the multimodal information X[t] after the fusion of vision and position, combined with the Sigmoid function, it realizes fast and efficient user identification and promotes accurate prediction of the beam.

[0075] (3) After user identification is completed, the coordinates of the center point of the bounding box corresponding to the identified user are used as the input of the beam alignment module; the coordinates of the center point of the bounding box are passed through multiple fully connected layers and finally input into the fully connected network for classification prediction; the classifier is responsible for predicting the future optimal beam, that is, outputting the optimal beam index.

[0076] 3. Introduce residual block training beam alignment neural network model to accelerate model convergence and enhance model stability.

[0077] Residual connections are introduced to improve the convergence speed and stability of the model, and the binary classification results are input through the Sigmoid function to simplify the user identification task, thereby improving the user identification accuracy of the millimeter wave communication system and improving the beam alignment accuracy. Specifically, the binary cross entropy loss function is used to optimize the user identification module, which is defined as follows:

[0078]

[0079] Among them, y and are the true label and the probability that the model predicts it to be a real user, respectively, where The range is (0, 1). N represents the number of samples in each batch.

[0080] In addition, during the model training process, the cross-entropy loss function is also used to optimize the beam alignment module to improve the model's optimal beam index prediction accuracy.

[0081] The model parameters are updated through the two losses mentioned above until the model training reaches the termination condition.

[0082] like Figure 3 FIG. 1 shows a millimeter wave beam alignment method supporting multiple candidate user identification provided by an embodiment of the present invention, the method comprising the following steps:

[0083] V1-V4: Acquire sensor data from the vision-assisted millimeter-wave communication system, build a millimeter-wave beam alignment problem model for multiple candidate users, and transform the user identification and beam alignment problems into a machine learning-based optimization problem.

[0084] V5-V15: Construct a millimeter-wave beam alignment neural network model for multiple candidate user scenarios. Feature extraction is performed on the acquired multimodal data. Image and position features are paired into positive and negative sample pairs. User matching is performed using the user identification module. Model training and parameter optimization are performed using the binary cross-entropy loss function. The coordinates of the user's bounding box center point are extracted. The beam alignment module selects the optimal beam, and model training and parameter optimization are performed using the cross-entropy loss function.

[0085] V16~V18: After the training termination conditions are met, the neural network parameters are generated, and the visually assisted millimeter wave communication system performs beam alignment based on the fully trained beam alignment network under multiple candidate scenarios.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. A millimeter wave beam alignment method supporting multiple candidate user identification, characterized in that: The method includes: Obtain the model parameters of a multi-sensor millimeter-wave communication system, build a multimodal sensing-assisted wireless communication beam alignment problem model, and transform the beam alignment problem in multiple candidate scenarios into a machine learning-based optimization problem. Build and train a beam alignment neural network model for multiple candidate scenarios; The perception data of different modalities are input into the trained beam alignment neural network model to identify the user's bounding box information, and the identified user bounding box information is used for beam alignment.

2. The method according to claim 1, characterized in that In a vision-assisted millimeter-wave communication system model with multiple candidate users, the base station includes an antenna, a lidar sensor, and a visual data sensor; the base station uses a predefined beamforming codebook Q is the total number of beamforming vectors in the codebook, is the beamforming vector in the codebook; At time step t, if each mmWave service user u is served by the base station using the beamforming vector f q [t] service, channel is The downlink from the base station to the user is expressed as: Where, P T is the average transmit power, To transmit complex symbols, (·) H represents the conjugate transpose, n[t] represents a noise sample extracted from a complex Gaussian distribution, and k is the index of the subcarrier; Channel h k [t] uses the geometric physical channel model and is expressed as: Where L represents the total number of channels; α l , τ l ,θ l and φ l represent path gain, delay, arrival azimuth and arrival elevation respectively; D c is the cyclic prefix length, K is the total number of subcarriers, T s is the sampling time, p(·) is the pulse shaping function, and a is the array steering vector; Multimodal user recognition task using visual and location information in the case of multiple candidate users. Based on the available perception data at a given time t, the base station aims to lock the unique user in the image; The goal of the user recognition task is to use the image information V[t] and location information G[t] to use the prediction function f Θ Identify the only user who needs service from U candidate users, as shown in the following formula: Where b u is the true label, For the predicted results, is the conditional probability; After completing user identification, the beam alignment is performed on the user who needs service. The beamforming task is to select the user from the predefined codebook. Select the index as The optimal beamforming vector To maximize the received signal power; therefore, the beam alignment optimization problem is expressed as: Where f is the beamforming vector.

3. The method according to claim 2, characterized in that The beam alignment problem in multiple candidate scenarios is transformed into a machine learning-based optimization problem. Based on the characteristic that the signal is concentrated in a certain direction in space during beam alignment, the spatial dimension of the scene is divided into multiple sectors using beam vectors, and a unique value is assigned to each sector. The machine learning task is a classification task using a predetermined codebook, using the bounding box information of the target user in the image and using a prediction function to predict the optimal beam index of the target user.

4. The method according to claim 3, characterized in that A beam alignment neural network model is constructed in multiple candidate scenarios. The model identifies the coordinates of the center point of the user's bounding box based on the acquired perception data of different modalities, and uses the identified coordinates of the center point of the user's bounding box to perform beam alignment.

5. The method according to claim 4, characterized in that The beam alignment neural network model includes a feature extraction module, a user identification module, and a beam alignment module. The feature extraction module processes the input image data and position data, extracts the bounding box and the coordinates of the bounding box center point of the detected object based on the image data, extracts position features based on the position data, and splices the visual features and position features into multimodal information. The user identification module performs user identification based on multimodal information; the beam alignment module outputs the optimal beam index based on the coordinates of the center point of the bounding box corresponding to the identified user.

6. The method according to claim 5, characterized in that The feature extraction module uses YOLOv3 to process the input visual image and generate bounding boxes for all detected objects in the image at time step t, which is expressed as V[t]=[x c ,y c ], x c 、y c are the horizontal and vertical coordinates of the center point of the bounding box of the detected object; The location data is the user's latitude and longitude information, which is normalized to obtain location features. Concatenate V[t] and G[t] into a four-dimensional vector The user identification module uses a multi-layer perceptron based on residual connections to process multimodal information X[t] and combine it with the Sigmoid function to achieve user identification; The beam alignment module performs classification prediction based on the coordinates of the center point of the bounding box corresponding to the identified user through a fully connected network and outputs the optimal beam index.

7. The method according to claim 4, characterized in that The beam alignment neural network model is trained, including using a binary cross entropy loss function to optimize the user identification problem. The binary cross entropy loss function is expressed as: Where y and are the true label and the probability that the model predicts it to be a real user, respectively. N represents the number of samples in each batch; Use the cross entropy loss function to optimize the model's optimal beam index prediction problem; The model parameters are updated through binary cross entropy loss and cross entropy loss until the model training reaches the termination condition.

Images