Unmanned aerial vehicle beam prediction method and device based on multi-modal information

Abstract

The application discloses a UAV beam prediction method and device based on a multi-modal information intelligent screening mechanism, and the method comprises the following steps: 1) collecting multi-modal information in a target airspace; 2) pre-processing the information obtained in step 1), performing a normalization operation and splicing to form an initial multi-modal information-optimal beamforming vector dataset; 3) performing feature selection on the dataset in step 2), eliminating features with a contribution to beam prediction accuracy less than a set value under the current environment, screening out the features with the largest contribution as key information and constructing a key multi-modal information-optimal beamforming vector dataset; 4) using the dataset in step 3) to construct and train a beam prediction neural network model; 5) in each time slot, the UAV obtains key multi-modal information and transmits the key multi-modal information to a base station, and the base station performs beam prediction using the beam prediction neural network model to obtain the optimal beamforming vector in the current time slot.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of wireless communication, relates to the field of millimeter wave beam prediction in the unmanned aerial vehicle scene, and in particular to an unmanned aerial vehicle beam prediction method and device based on a multi-modal information intelligent screening mechanism. BACKGROUND

[0002] Unmanned aerial vehicles (UAVs) have high mobility, rapid deployment capability, and low cost, and are widely used in military and civilian fields, including reconnaissance, search and rescue, transportation, agricultural irrigation, and aviation. During the execution of tasks, UAVs often need to transmit information related to tasks, such as high-resolution images and sensor data, which requires high data transmission rates. To meet these requirements, UAVs are usually equipped with millimeter wave or terahertz (THz) transceivers, deploy large-scale antenna arrays, and use narrow beam technology.

[0003] However, due to the rapid changes in the wireless channel in the UAV communication scene, the accurate adjustment of the optimal narrow beam at the receiving and transmitting ends to maintain alignment will result in huge beam training overhead and delay, making it difficult to support high mobility and delay-sensitive application requirements. Data-driven methods can achieve effective beam prediction, and existing methods mostly rely on single-modal perception information, such as location, vision, and radar data. However, these solutions fail to provide comprehensive environmental features, limiting the improvement of communication system performance. Moreover, each type of perception information has its inherent limitations. For example, the error of GPS positioning sensors will reduce the positioning accuracy, and single-point data is usually insufficient for accurate beam prediction, while vision data is easily disturbed by noise and occlusion. In addition, long-distance communication between UAVs often reduces the resolution and reliability of sensors such as vision and radar, making beam prediction in long-distance situations even more complex. Notably, key data such as UAV location, height, and speed can be obtained at any time and are lightweight text-based information. Developing lightweight beam prediction methods can effectively reduce the bandwidth and resource requirements of beamforming and improve real-time transmission efficiency. Therefore, how to integrate these lightweight multi-modal data sources to provide a more robust and feasible solution for accurate beam prediction in actual UAV applications is the focus of current research. Based on this, the present application proposes an unmanned aerial vehicle beam prediction method and device based on a multi-modal information intelligent screening mechanism. SUMMARY

[0004] In view of the above, the present application proposes an unmanned aerial vehicle beam prediction method and device based on a multi-modal information intelligent screening mechanism. The present application can effectively cope with the high degree of freedom of motion of unmanned aerial vehicles, improve the accuracy of beam prediction, ensure the establishment of more stable communication links, and improve the achievable millimeter wave communication rate between unmanned aerial vehicles.

[0005] In order to achieve the above object, the present application is realized by adopting the following technical scheme:

[0006] A UAV beam prediction method based on a multi-modal information intelligent screening mechanism, comprising the following steps:

[0007] 1) Training phase: collecting multi-modal information in the target airspace;

[0008] 2) Preprocessing the information obtained in step 1), performing normalization operation and splicing to form an initial multi-modal information-optimal beamforming vector dataset;

[0009] 3) Feature selection in step 2), eliminating features with small contribution (less than a set value) to beam prediction accuracy in the current environment, and screening out the largest contribution features as key information and constructing a key multi-modal information-optimal beamforming vector dataset;

[0010] 4) Using the dataset in 3) to build and train a beam prediction neural network model;

[0011] 5) Prediction phase: at each time slot, the UAV end obtains the key multi-modal information and transmits it to the base station, and the base station uses the beam prediction neural network model to perform beam prediction process, thereby obtaining the optimal beamforming vector of the current time slot.

[0012] Preferably, in step 1), the multi-modal information includes the position, attitude, speed, rate, height and distance of the UAV end. The multi-modal information is represented as:

[0013] M[t]={g[t],a[t],v[t],s[t],h[t],d[t]}

[0014] Wherein, g[t] represents the position of the UAV at time t, including latitude and longitude; a[t] represents the attitude of the UAV at time t, including pitch angle and roll angle; v[t] represents the speed of the UAV at time t, including the speed in x, y and z directions; s[t] represents the speed of the UAV at time t; h[t] represents the height of the UAV at time t; d[t] represents the distance between the UAV and the base station at time t.

[0015] Preferably, in step 2), the position, height, distance, rate and attitude information are normalized by Min-Max normalization to the range of [0, 1].

[0016] Preferably, in step 2), the optimal beam index f[t] corresponding to each set of multi-modal information is extracted from the received power data, which is expressed as: f[t] ∈ {1, 2, …, N}, where N is the total number of beams in the base station antenna array. Through the collected multi-modal information M[t] and the optimal beam index f[t] extracted at the base station end, an initial multi-modal information-optimal beam index dataset is generated:

[0017]

[0018] wherein M u represents the multi-modal information set of the u-th sample, corresponding to M[t] at time t; f u represents the optimal beam index corresponding to the multi-modal information M u ; U is the total number of samples.

[0019] Preferably, in step 3), the dataset of step 2) is subjected to feature selection by recursive feature elimination (RFE) method. In the scenario of the present application, the RFE method has the following advantages: first, there may be non-linear correlation between the features of multi-modal information, and RFE can better mine the key features by gradually eliminating features and combining regression models for evaluation; second, considering the accuracy requirements and computational complexity in practical applications, the RFE method performs best in balancing performance and efficiency.

[0020] Preferably, in step 3), in the recursive feature elimination (RFE) method, an MLP containing two 128-node hidden layers is used as a base estimator, the least contributing features are recursively removed, and a 5-fold stratified cross-validation is used to evaluate the model performance under each feature quantity; in each iteration, one feature is removed, the base estimator is retrained, and the average accuracy of the model is calculated through cross-validation; finally, by comparing the cross-validation results of different feature quantities, the optimal feature combination is selected, thereby obtaining the screened key multi-modal information

[0021] Preferably, in step 3), the key multi-modal information depending on the contribution of the features to the model performance in the current scenario, which may include but is not limited to position, height, and speed, thereby obtaining the key multi-modal information-optimal beam index dataset for constructing and training the beam prediction neural network model:

[0022]

[0023] wherein M u * represents the key multi-modal information set of the u-th sample screened by the RFE method, which only includes the features with the largest contribution to beam prediction.u * representing the set of key multi-modal information M u * corresponding optimal beam index. Preferably, in step 4), the constructed beam prediction neural network model adopts a three-layer fully connected neural network to predict the optimal beam index; the dimension of the input layer is adjusted according to the actual dimension of the key information; the hidden layer contains three layers, each layer has 256 hidden units; the output layer generates the probability distribution of the beam index through the softmax function.

[0024] Preferably, in step 4), the cross-entropy loss function is used in the model training process, and the Adam optimizer is used for optimization.

[0025] Preferably, in step 5), the unmanned aerial vehicle transmits the acquired key multi-modal information to the base station through the millimeter wave transmitter.

[0026] Through the above steps, the unmanned aerial vehicle beam prediction based on the multi-modal information intelligent screening mechanism is realized, and the millimeter wave communication in the flight process of the unmanned aerial vehicle is completed.

[0027] The application also provides an unmanned aerial vehicle beam prediction device based on a multi-modal information intelligent screening mechanism, which is used to execute the above method and includes an unmanned aerial vehicle and a base station; a perception module (including a GPS receiver and an inertial measurement unit IMU) and a millimeter wave transmitter are installed on the unmanned aerial vehicle, and a GPS receiver, a millimeter wave receiver, a data preprocessing module, a feature selection module and a beam prediction neural network module are installed on the base station.

[0028] In the training phase, the unmanned aerial vehicle flies in the target airspace, and the GPS receiver and the inertial measurement unit IMU in the perception module collect position, height, distance, rate, speed and attitude data at each time slot, and the base station end collects the corresponding optimal beamforming vector. The unmanned aerial vehicle transmits the acquired multi-modal information to the base station; the data preprocessing module of the base station preprocesses the acquired multi-modal information, performs normalization operation and splicing to form an initial multi-modal information-optimal beamforming vector dataset; the feature selection module of the base station selects the features of the initial multi-modal information-optimal beamforming vector dataset, eliminates the features with a contribution to the beam prediction accuracy less than a set value in the current environment, selects the features with the largest contribution as key information and constructs a key multi-modal information-optimal beamforming vector dataset, and uses the dataset to construct and train a beam prediction neural network model;

[0029] In the prediction stage, the key multi-modal information acquired by the unmanned aerial vehicle in each time slot is transmitted to the base station through the millimeter wave transmitter, the millimeter wave receiver of the base station receives the key multi-modal information, and the beam prediction neural network module predicts the optimal beam of the current time slot by using the beam prediction neural network model to establish a stable and reliable millimeter wave communication link.

[0030] Compared with the prior art, the present application has the following technical effects:

[0031] The present application can effectively cope with the high degree of freedom of the unmanned aerial vehicle, improve the beam prediction accuracy, ensure the establishment of a more stable communication link, and improve the achievable millimeter wave communication rate between unmanned aerial vehicles. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 A schematic diagram of the unmanned aerial vehicle beam prediction scene based on the multi-modal information intelligent screening mechanism of the present application.

[0033] Figure 2 A flowchart of the unmanned aerial vehicle beam prediction based on the multi-modal information intelligent screening mechanism provided by the preferred embodiment of the present application.

[0034] Figure 3 A schematic diagram of the specific process of feature selection by the recursive feature elimination method used in the preferred embodiment of the present application.

[0035] Figure 4 The beam prediction neural network model constructed in the preferred embodiment of the present application adopts a three-layer fully connected neural network structure diagram.

[0036] Figure 5 A performance comparison diagram provided by the preferred embodiment of the present application.

[0037] Figure 6 A schematic diagram of the relationship between the unmanned aerial vehicle beam prediction accuracy and the position noise level and sample ratio provided by the preferred embodiment of the present application. DETAILED DESCRIPTION

[0038] The technical solutions of the present application will be specifically described below in combination with the drawings and preferred embodiments.

[0039] Referring to Figure 1 The present embodiment relates to an unmanned aerial vehicle beam prediction device based on a multi-modal information intelligent screening mechanism, comprising an unmanned aerial vehicle and a base station.

[0040] Referring to Figure 2As shown, in this embodiment, the UAV is equipped with a sensing module (including a GPS receiver and an inertial measurement unit, IMU) and a millimeter wave transmitter, and the base station is equipped with a GPS receiver and a millimeter wave receiver. The spatial geometric information and environmental information relative to the base station are obtained by the sensing module on the UAV, and then the key information selection and beam prediction process are performed by the data preprocessing module, the feature selection module and the beam prediction neural network module of the base station.

[0041] Next, an outdoor millimeter wave communication scenario is determined as an example of a practical scenario. The scenario is a public rectangular park in a certain area, with a length of 205 meters and a width of 152 meters.

[0042] As shown in FIG. 1, a base station equipped with a global positioning system (GPS) receiver and a millimeter wave receiver communicates with another UAV equipped with a global positioning system (GPS) receiver and an inertial measurement unit (IMU). The GPS receiver provides real-time position information, including the latitude, longitude, and altitude of the UAV. The IMU provides real-time speed, velocity, and attitude information. Figure 1

[0043] The UAV also includes a single-antenna transmitter, i.e., a millimeter wave transmitter, while the base station is equipped with an M-element uniform linear array (ULA).

[0044] The communication system uses orthogonal frequency division multiplexing (OFDM) technology, with K subcarriers and a cyclic prefix of length D. The OFDM system allows the UAV to efficiently transmit data by dividing the signal into multiple subcarriers, while the cyclic prefix helps to prevent inter-symbol interference due to multipath propagation.

[0045] The base station uses a predefined codebook wherein, represents the nth beamforming vector, represents a column vector in the complex domain, where M represents the number of antenna array elements and N represents the number of beamforming vectors.

[0046] In this scenario, the base station uses a 16-element antenna array operating in the 60 GHz frequency band, and a 64-predefined beam oversampling codebook is used to receive the transmitted signal. The codebook is used to select the optimal beam to maximize the signal strength.

[0047] In the downlink scenario, if the wireless channel between the UAV and the base station at time t and the kth subcarrier is represented as then the signal received by the base station can be represented as:

[0048]

[0049] wherein h k ​H [t] represents the channel vector h k the complex conjugate transpose of [t], which is used to receive the matched filter to maximize the signal strength after channel and beam direction coupling, f n [t] represents the beamforming vector at time t, z k [t]· represents the noise component, the beamforming vector selected at time t For maximizing the average SNR received, it is defined as follows:

[0050]

[0051] Where K represents the number of subcarriers received at time t.

[0052] The goal of this communication system is to select the optimal receive beamforming vector f * [t] from the codebook The transmitted symbol x must satisfy Where E[] represents the mathematical expectation, P is the average power of each symbol, and the goal is to maximize P based on formula (2).

[0053] The specific steps of the UAV beam prediction process based on the multi-modal information intelligent screening mechanism are as follows:

[0054] S1, training phase: the UAV flies in this park airspace, and the UAV end GPS and IMU acquire original multi-modal information at each time slot including position (latitude and longitude), height speed (including x, y and z directions), attitude (pitch angle and roll angle), distance and rate The UAV transmits the acquired multi-modal information to the base station.

[0055] The base station receives the signal using a predefined codebook containing 64 beamforming vectors, selects the beam index with the maximum receive power as the optimal beamforming vector at the current time by calculating the receive power of each beamforming vector f n

[0056] ​S2, the base station end data preprocessing module normalizes the obtained original multi-modal information. For GPS coordinates (longitude and latitude), height, distance, speed, and attitude information (roll angle, yaw angle), Min-Max normalization is adopted to scale them to the range of [0, 1] to maintain the relative change trend of the data and improve the comparability. For speed (x, y, and z directions), since the change range is large and the data mean and standard deviation of each direction are different, the Z-score standardization method is adopted in this embodiment. Subsequently, a vector data set composed of multi-modal information-optimal beam index is generated:

[0057]

[0058] wherein U is the total number of samples.

[0059] S3, considering that in actual scenarios, the contribution of these information to beam prediction is different, this embodiment introduces RFE to screen key information for constructing the final beam prediction neural network model. Specifically, the feature selection module at the base station end uses MLP containing two 128-node hidden layers as the base estimator in RFE. Feature selection starts from the complete feature set containing position, height, speed, distance, rate, and attitude, recursively removes the least contributing features, and uses 5-fold stratified cross-validation to evaluate the model performance under each feature quantity. As shown in Figure 3 In each iteration, the number of features is gradually reduced from the initial 6 features to 1 feature by evaluating the impact of features on the model accuracy. When the number of features is 3 (position, height, and speed are retained), the Top-1 accuracy of cross-validation reaches 67.17%, close to the performance of the complete feature set (68.83%). After removing more features, the accuracy starts to decrease significantly and cannot be further simplified. Therefore, the remaining 3 features (position, height, and speed) are selected as the optimal feature combination to balance the relationship between feature simplification and model performance.

[0060] In this embodiment, the key multi-modal information selected by RFE includes position (longitude and latitude), height speed (including x, y, and z directions), thereby obtaining a key multi-modal information-optimal beam index data set for constructing and training the beam prediction neural network model:

[0061]

[0062] S4, the beam prediction neural network model constructed in this embodiment adopts a three-layer fully connected neural network (as shown in Figure 4The input layer has a dimension of 6, which is adjusted according to the actual dimension of the key information. The hidden layer includes three layers, each having 256 hidden units, and uses a ReLU activation function to capture complex patterns in the data. The output layer generates a probability distribution of beam indices by using a softmax function, which converts the prediction task into a classification problem. In this embodiment, the output layer has a dimension of 64. The model training uses a batch gradient descent method, and each iteration uses a batch size of 128 samples to ensure a balance between memory usage and training speed. The loss function uses a cross-entropy loss function, which is defined as follows:

[0063]

[0064] where N represents the number of batch samples, C represents the total number of classes, y ij represents the true label of sample i belonging to class j, represents the probability value of class j predicted by the model. The optimization of the loss function uses an Adam optimizer, which has the characteristics of adaptive learning rate adjustment. In the early stage of training, it can quickly converge, and in the later stage, it can more finely adjust the weight parameters. During the training process, in order to avoid the model falling into a local optimum and speed up the convergence, the initial learning rate is set to 0.01, and a learning rate decay mechanism is introduced. Specifically, at the 15th, 25th and 40th training epochs, the learning rate is reduced by a decay factor of 0.1, thereby improving the stability and accuracy of the later training. The total number of training epochs of the preferred embodiment of the present application is 60, and in each training epoch, the model will completely traverse the entire training data set and optimize the weights through batch gradient descent. Table I lists the hyperparameter configurations during the training process, including batch size, learning rate, learning rate reduction factor and total training epochs. Through the above optimization strategies and parameter configurations, the model can accurately predict the beam index, has strong generalization ability and robustness.

[0065] Table I

[0066] Hyperparameter name Value Batch size 128 Learning rate 0.01 Learning rate decay points 15th, 25th, and 40th epochs Learning rate decay factor 0.1 Total training epochs 60

[0067] The key evaluation indicator provided by the present application is Top-k accuracy. Top-k accuracy refers to the percentage of test samples that the actual beam occupies in the top k predicted beams.

[0068] S5, base station end, after the beam prediction neural network model is constructed and trained, in the prediction stage, the unmanned aerial vehicle obtains the position, height and speed information from the perception module every time slot, and transmits to the base station through the millimeter wave transmitter, and directly inputs the beam prediction neural network module through the data preprocessing module, the optimal beam of the current time slot is predicted by using the beam prediction neural network model, the optimal beam forming vector of the current time slot is obtained, and a stable and reliable millimeter wave communication link is established.

[0069] Figure 5 It is shown that the performance of the beam prediction neural network model of the preferred embodiment of the application is significantly better than the model using only position information (position-height-speed in the figure is the application), and is comparable to the upper limit model using all information.

[0070] Figure 6 The beam prediction top-1 accuracy of the beam prediction neural network model of the preferred embodiment of the application under different position noise levels and sample proportions is shown, which shows its advantages in noise resistance and sample sensitivity of the data set.

[0071] The above is only the preferred embodiment of the application, it should be pointed out that for those skilled in the art, without departing from the technical principles of the application, a number of improvements and modifications can be made, these improvements and modifications should also be considered as the protection scope of the application.

Claims

1. A UAV beam prediction method based on a multi-modal information intelligent screening mechanism, characterized by Comprising the following steps: 1) Collecting multi-modal information in the target airspace; 2) Preprocessing the multi-modal information obtained in step 1), performing normalization operation and splicing to form an initial multi-modal information-optimal beamforming vector dataset; 3) Feature selection on the dataset of step 2), eliminating features with a contribution to beam prediction accuracy less than a set value in the current environment, and selecting the most contributing features as key information to construct a key multi-modal information-optimal beamforming vector dataset; 4) Using the dataset of step 3) to construct and train a beam prediction neural network model; 5) In each time slot, the UAV obtains the key multi-modal information and transmits it to the base station, and the base station uses the beam prediction neural network model to perform beam prediction to obtain the optimal beamforming vector in the current time slot; In step 3), the dataset of step 2) is subjected to recursive feature elimination (RFE) for feature selection. In the recursive feature elimination (RFE) method, an MLP containing two 128-node hidden layers is used as a base estimator, the least contributing features are recursively removed, and the model performance under each feature quantity is evaluated using 5-fold stratified cross-validation; In step 3), the key multi-modal information selected by the recursive feature elimination (RFE) method includes position, height, and speed, thereby obtaining the key multi-modal information-optimal beam index dataset used to construct and train the beam prediction neural network model: wherein M u * denotes the key multi-modal information set of the u-th sample after screening by the RFE method, only including the features with the largest contribution to beam prediction; f u * denotes the key multi-modal information set M u * corresponding to the optimal beam index; In step 4), the constructed beam prediction neural network model uses a three-layer fully connected neural network to predict the optimal beam index; the dimension of the input layer is adjusted according to the actual dimension of the key information; the hidden layer contains three layers, each with 256 hidden units; the output layer generates a probability distribution of the beam index through a softmax function; In step 4), the cross-entropy loss function is used in the model training process, and the Adam optimizer is used for optimization. 2.The UAV beam prediction method based on the multi-modal information intelligent screening mechanism of claim 1, characterized in that: In step 1), the multi-modal information includes the position, attitude, speed, rate, height, and distance of the UAV; the multi-modal information is represented as: M[t]={g[t],a[t],v[t],s[t],h[t],d[t]} wherein, represents the position of the UAV at time t, including latitude and longitude; represents the attitude of the UAV at time t, including pitch and roll angles; represents the velocity of the UAV at time t, including velocity in x, y, z directions; represents the velocity of the UAV at time t; represents the height of the UAV at time t; represents the distance of the UAV from the base station at time t. 3.The UAV beam prediction method based on the multi-modal information intelligent screening mechanism of claim 2, characterized in that: In step 2), Min-Max normalization is used for position, attitude, speed, rate, height, and distance information, scaled to the range of [0, 1]. 4.The method of claim 2 or 3, characterized in that: In step 2), the optimal beam index f[t] corresponding to each group of multi-modal information is extracted from the received power data, represented as: f[t]∈{1,2,…,N}, where N is the total number of beams in the base station antenna array; the initial multi-modal information-optimal beam index dataset is generated from the collected multi-modal information M[t] and the extracted optimal beam index f[t] at the base station end: Wherein, M u M [t] represents a multimodal information set of the u-th sample, corresponding to M[t] of the t-th moment; f u M [t] represents a multimodal information set of the u-th sample, corresponding to M[t] of the t-th moment; f u Corresponding optimal beam index; U is the total number of samples.

5. The unmanned aerial vehicle beam prediction method based on the multi-modal information intelligent screening mechanism according to any one of claims 1-3, characterized in that: The UAV transmits the obtained key multi-modal information to the base station through the millimeter wave transmitter.

6. The UAV beam prediction apparatus based on the intelligent screening mechanism of multi-modal information, for executing the method as claimed in any one of claims 1-5, characterized in that, The UAV beam prediction device includes a UAV and a base station; the UAV is equipped with a perception module and a millimeter wave transmitter, and the base station is equipped with a millimeter wave receiver, a data preprocessing module, a feature selection module, and a beam prediction neural network module; The unmanned aerial vehicle flies in the target airspace, and a multi-modal information is collected by a sensing module, and a corresponding optimal beamforming vector is collected by a base station; the unmanned aerial vehicle transmits the acquired multi-modal information to the base station; The data preprocessing module of the base station pre-processes the acquired multi-modal information, performs a normalization operation and splicing to form an initial multi-modal information-optimal beamforming vector data set; The feature selection module of the base station performs feature selection on the initial multi-modal information-optimal beamforming vector data set, eliminates features with a contribution to beam prediction accuracy less than a set value in the current environment, selects the features with the largest contribution as key information, and constructs a key multi-modal information-optimal beamforming vector data set, and uses the data set to construct and train a beam prediction neural network model; The unmanned aerial vehicle transmits the acquired key multi-modal information to the base station through a millimeter wave transmitter at each time slot, the millimeter wave receiver of the base station receives the key multi-modal information, and the beam prediction neural network module predicts the optimal beam of the current time slot by using the beam prediction neural network model.

Images