Beamforming method, apparatus and system for mmWave UAV MIMO system based on multi-armed gambling machine

Abstract

The application discloses a beamforming method, device and system of a mmWave UAV MIMO system based on a multi-armed bandit, the method comprising: constructing a two-stage beamforming problem model and establishing a pre-beamforming candidate configuration set, each configuration being an arm in the multi-armed bandit, and initializing; based on historical feedback information, calculating a benefit-cost ratio index of each candidate arm, combining a current remaining resource budget, screening an executable high-performance super arm combination, and executing the selected configuration in a communication time slot to obtain reduced dimension channel state information and feedback rewards; using the obtained reduced dimension channel information to construct an equivalent channel model, and based on singular value decomposition, generating a digital beamforming matrix, combining a simulation pre-beamformer to form an optimal beamformer; and according to actual energy consumption and reward feedback, dynamically updating the cost estimation of each arm and the remaining energy of the system. The application effectively balances the contradiction between pilot overhead and the number of communication time slots.

Description

Technical Field

[0001] This invention belongs to the field of beamforming design technology for mmWave UAV MIMO systems, specifically relating to a beamforming method, apparatus, and system for an mmWave UAV MIMO system based on a multi-arm gambling machine. Background Technology

[0002] Applying mmWave (Millimeter Wave) communication to unmanned aerial vehicle (UAV) assisted communication systems can significantly meet the demands for high throughput and low latency. The mmWave band offers a vast and continuous available bandwidth, effectively alleviating the spectrum resource shortage faced by microwave bands below 6 GHz. Simultaneously, UAVs can more easily establish line-of-sight (LoS) links with ground users or base stations. The existence of such LoS links is more advantageous for mmWave communication because non-line-of-sight (NLoS) paths typically suffer severe attenuation. Furthermore, UAVs face strict limitations in volume and weight due to their small size and limited load-bearing capacity. Thanks to the short wavelength characteristics of mmWave signals, a large number of antennas can be integrated within the small space of the UAV platform, and spatial multiplexing gain and enhanced directional array gain can be obtained through flexible beamforming technology. The sparsity of the mmWave channel and the directional characteristics of the beam significantly reduce interference in UAV communication, improving system spectral efficiency. Advances in beamforming technology depend on the acquisition of channel state information.

[0003] Channel State Information (CSI) accurately describes the propagation path of a wireless signal from the transmitter to the receiver at a specific carrier frequency, leveraging multipath propagation characteristics. It can be obtained through channel estimation. However, due to the significantly increased number of antennas in massive MIMO systems, the pilot overhead associated with CSI estimation becomes increasingly substantial. Excessive pilot overhead reduces the system's spectral efficiency.

[0004] The multi-armed bandit (MAB) problem, as a mainstream theoretical model for the exploration-exploitation tradeoff, is widely used in the field of learning, from medical experiments and communication networks to web search and advertising. Combinatorial MAB with a budget is a variant of MAB. In this variant, the algorithm incurs additional costs when choosing an arm given an unknown reward distribution. MAB uses historical data to learn the reward distribution and make decisions without relying on additional information. Using MAB to design beamforming can reduce the need for statistical CSI and further reduce pilot overhead. Researchers have conducted extensive experiments to evaluate MAB-based beamforming designs and compare them with traditional estimation algorithms. Experimental results show that MAB-based algorithms can achieve higher spectral efficiency under certain conditions. However, problems such as high pilot overhead and low spectral and energy efficiency still exist. Summary of the Invention

[0005] To address the aforementioned issues, this application provides a beamforming method, apparatus, and system for a multi-armed gambling machine-based mmWave UAV MIMO system, aiming to achieve higher system spectral efficiency with less pilot overhead than traditional channel estimation algorithms, thereby overcoming or at least partially overcoming the shortcomings of the prior art.

[0006] The embodiments of this application adopt the following technical solutions: In a first aspect, this application provides a beamforming method for an mmWave UAV MIMO system based on a multi-armed gambling machine. The method is applied to a millimeter-wave unmanned aerial vehicle (UAV) assisted multiple-input multiple-output (MIMO) downlink communication system. The system includes a UAV base station equipped with a cylindrical conformal array and ground user equipment equipped with a uniform linear array. The method includes: Initialization and modeling phase: Construct a two-stage beamforming problem model and establish a pre-beamforming candidate configuration set, with each configuration serving as one arm in a multi-armed gambling machine, and initialize system state parameters, energy budget, reward estimation, and selection strategy; Online learning and decision-making phase: Based on historical feedback information, calculate the cost-benefit ratio of each candidate arm, combine it with the current remaining resource budget, screen the executable high-performance superarm combination, and execute the selected configuration in the communication time slot to obtain dimensionality reduction channel state information and feedback rewards; Beamforming construction stage: The obtained dimensionality reduction channel information is used to construct an equivalent channel model, and a digital beamforming matrix is ​​generated based on singular value decomposition. The optimal beamformer is formed by combining the simulated pre-beamformer. Iterative update and control phase: Based on actual energy consumption and reward feedback, dynamically update the cost estimates of each arm and the remaining energy of the system.

[0007] Secondly, this application also provides a beamforming device for an mmWave UAV MIMO system based on a multi-armed gambling machine, the device comprising: Initialization and Modeling Unit: Used to construct a two-stage beamforming problem model, establish a pre-beamforming candidate configuration set, with each configuration serving as one arm in a multi-armed gambling machine, and initialize system state parameters, energy budget, reward estimation, and selection strategy; Online learning and decision-making unit: Based on historical feedback information, it calculates the cost-benefit ratio of each candidate arm, combines the current remaining resource budget, selects executable high-performance superarm combinations, and executes the selected configuration in the communication time slot to obtain dimensionality reduction channel state information and feedback rewards. Beamforming building unit: used to construct an equivalent channel model using the obtained dimensionality-reduced channel information, and generate a digital beamforming matrix based on singular value decomposition, which is then combined with an analog pre-beamformer to form an optimal beamformer; Iterative update and control unit: used to dynamically update the cost estimates of each arm and the remaining energy of the system based on actual energy consumption and reward feedback.

[0008] Thirdly, this application also provides a millimeter-wave unmanned aerial vehicle (UAV) assisted multiple-input multiple-output (MIMO) downlink communication system, the system including a UAV base station equipped with a cylindrical conformal array and a ground user equipment equipped with a uniform linear array, the system being deployed with the aforementioned beamforming device of the multi-arm gambling machine-based mmWave UAV MIMO system.

[0009] Fourthly, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of a beamforming method for a multi-armed gambling machine-based mmWave UAV MIMO system.

[0010] Fifthly, this application also provides a computer-readable storage medium storing a computer program that, when instructed by a processor, implements the steps of a beamforming method for a multi-armed gambling machine-based mmWave UAV MIMO system.

[0011] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects: This application constructs a two-stage beamforming problem model and introduces a multi-arm gambling machine mechanism to effectively balance the contradiction between pilot overhead and the number of communication slots. In the initialization and modeling phase, pre-beamforming candidate configurations are abstracted as "arms," ​​and an optimization framework is established by combining energy budget and reward estimation, providing a theoretical basis for subsequent decision-making. In the online learning and decision-making phase, a cost-benefit ratio index is used to screen high-performance super-arm combinations, prioritizing configurations with higher energy consumption returns under limited resources, significantly reducing pilot overhead during channel detection, and extending effective communication duration. In the beamforming construction phase, an optimal beamformer is generated based on dimensionality-reduced channel information, ensuring signal transmission quality while reducing the number of activated RF chains. The iterative update and control phase dynamically adjusts cost estimation and energy state to achieve adaptive resource allocation during long-term communication. In summary, this method, while satisfying the energy constraints of the UAV, reduces the pilot symbol length required for channel estimation and improves spectral efficiency per unit energy, showing significant improvements in both spectral utilization and energy efficiency compared to traditional fixed beamforming algorithms. Attached Figure Description

[0012] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A schematic flowchart of a beamforming method for a multi-armed gambling machine-based mmWave UAV MIMO system according to an embodiment of this application is shown. Figure 2 A scene diagram of MmWave UAV MIMO according to an embodiment of this application is shown; Figure 3 A schematic diagram of an antenna activation mode according to an embodiment of this application is shown; Figure 4 The reward curve obtained for each time slot according to one embodiment of this application is shown; Figure 5 A graph showing the variation of spectral efficiency with signal-to-noise ratio according to an embodiment of this application is provided. Figure 6 A schematic diagram of the beamforming apparatus of a multi-armed gambling machine-based mmWave UAV MIMO system according to an embodiment of this application is shown. Figure 7 A schematic diagram of the structure of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] To make the content of this application easier to understand, the technical solution of this application will be described in detail below with reference to specific embodiments and accompanying drawings. Figure 1 This paper illustrates a flowchart of a beamforming method for a multi-armed multi-input multiple-output (mmWave UAV) MIMO system according to an embodiment of this application. The method is applied to a millimeter-wave unmanned aerial vehicle (UAV) assisted multiple-input multiple-output (mmWave UAV) MIMO system, which includes a UAV base station equipped with a cylindrical conformal array (CCA) and a ground user equipment (UE) equipped with a uniform linear array (ULA). The method employs a two-stage collaborative design architecture, achieving joint optimization of pilot overhead, energy consumption, and system performance through a budgeted multi-armed multi-input multi-output mechanism while satisfying the UAV's energy budget constraints. Figure 1 The illustrated embodiment includes the following steps S110 to S140: Initialization and Modeling Phase S110: Construct a two-stage beamforming problem model and establish a pre-beamforming candidate configuration set, with each configuration serving as an "arm" in a multi-armed gambling machine, and initialize system state parameters, energy budget, reward estimation, and selection strategy.

[0015] (I) Problem Modeling Consider an mmWave UAV MIMO downlink communication system. The UAV deploys a CCA, and a single user deploys a uniform linear array. Assume it is equipped with... A base station equipped with a root antenna serves a The user of the root antenna. Assuming the drone is equipped with... Root radio frequency link. Let the azimuth and elevation angles be denoted as... and The radius of the cylindrical antenna array is denoted as . The steering vector can then be expressed as: ; in: ; in, Indicates the carrier wavelength. Indicates the spacing between components along the z-axis.

[0016] exist The angular spacing between components on the plane is The angular position of the nth element is determined by the formula express.

[0017] Please refer to Figure 2 , Figure 2 This illustration shows a scenario diagram of MmWave UAV MIMO according to an embodiment of this application, assuming the user is equipped with... Root RF link. The steering vector at this point can be expressed as: ; This application employs an adaptive partially connected analog beamforming architecture. This architecture dynamically adjusts the connection method between the RF chain and antenna elements according to channel changes, thereby forming a subarray. Within a fixed time slot, the beamforming architecture maintains a traditional partially connected structure, while the connection method is adjusted according to channel quality changes caused by movement within different time slots. The channel expression between the UAV and the user can be represented as: ; in, Representing a path The complex channel gain has a path loss exponent of . The distance between the drone and the user is The free space path loss is ,in These correspond to elevation and azimuth angles, respectively. The angle of elevation.

[0018] Given the UAV prebeamforming matrix (dimension is) ), beamforming matrix (dimension is) Combination matrix of users (dimension is) The received signal is: ; in To transmit signals, satisfy , For the number of data streams, This refers to the transmission power. It is a dimension of The noise vector, where each element is independent and identically distributed, and follows a normal distribution. The signal-to-noise ratio between the drone and the user is: ; Therefore, the system spectral efficiency is: ; in, The pilot length, The length of the symbol.

[0019] When the receiver has perfect channel state information, the user can directly calculate the combiner based on the channel matrix. First, the singular value decomposition of the channel is represented as... ,in It is a left singular vector. It is a right singular vector. Matrix It is a diagonal matrix whose non-zero elements are singular values ​​arranged in descending order. The final combiner can be represented as: ; in, ; The remaining energy E_t at time slot t can be expressed as: ; in, This indicates the energy required to charge the solar panel. This refers to the power consumption required by the battery to maintain flight. Power is consumed for transmitting signals. Let... For indicator functions: .

[0020] To reduce pilot overhead during ICSI channel acquisition, this application employs a two-stage beamforming (TSB) scheme for time-slot transmission. The first stage designs a pre-beamformer to reduce pilot overhead, while the second stage designs a beamformer to improve system performance.

[0021] In the first stage, to improve the spectral efficiency of the CCA mmWave UAV system, it is necessary to determine the optimal beamforming and combining vectors. Specifically, since UAV energy is limited, activating all antennas simultaneously would result in power waste. Therefore, the activation subarrays and beam angles need to be optimized. Based on this, this application constructs a model for maximizing spectral efficiency: ; in, In order to receive energy, This indicates the activation mode of the CCA. The activated antenna array. Careful selection is required, as it has a significant impact on beam gain.

[0022] In the second phase, the obtained dimensionality-reduced ICSI will be used to design beamformers. Then data transmission occurs. The problem model is constructed as follows: .

[0023] (II) Pre-beamforming design Consider a hierarchical codebook, represented as: ; in, Indicates when When using different codebooks, the activated antenna is 1. To achieve full spatial coverage, the quantization level is determined by the beamwidth in the azimuth and elevation planes. and Decide.

[0024] First, let's address the problem. The problem is modeled as a combined multi-armed gambling machine problem with a budget, and then a pre-beamforming design scheme is proposed based on a greedy strategy.

[0025] Construction of the combined multi-armed gambling machine problem: In the model of this application, the budget represents the total available resources, and the cost of each arm corresponds to its resource consumption in each time slot. The combinatorial multi-armed gambling machine problem with budget consists of four key elements: arms, rewards, budget, and cost.

[0026] Arm: Assumption The discrete order of magnitude is , The discrete two-level is At the same time, activate all antennas. There are three types. When When the antenna array is activated, the lower half is activated; when When the antenna array is activated, the upper half is activated; when At that time, the middle part of the antenna array is activated, such as Figure 3 As shown, Figure 3 A schematic diagram of an antenna activation mode according to an embodiment of this application is shown. Activation module for Each one Defined as an arm Therefore, it can be seen that the arm has a total of Each time slot. Only one option is available. One mode. Each time slot Multiple arms are selected as a combined arm. It can be seen that the arm represents a pre-beamforming configuration unit, including beam pointing in the azimuth and elevation directions and the activation region of the antenna subarray; the activation region is divided into at least one of an upper half, a lower half, or a middle annular region, and each combination of spatial pointing and activation mode constitutes an independent candidate action.

[0027] Reward: Super Arm The reward is defined as the total energy collected within a single time slot. Note that... Equals all selected arms The total amount of energy received. Arm c The reward is defined as .therefore, .

[0028] Budget: In this application, budget refers to the available energy of the battery carried by the drone. Therefore, time slot Budget Set as .

[0029] Cost: The beamformer is connected to the pre-beamformer via an RF link. Therefore, the energy consumption of the corresponding RF link is defined as the cost.

[0030] In the multi-armed gambling machine problem, the upper confidence limit for each arm is used to evaluate its potential to earn rewards. Unlike the traditional MAB model, the multi-armed gambling machine problem with budget combinatorial features aims to maximize the total cumulative reward with a limited budget. To this end, the "bangperbuck ratio" is introduced, defined as: [the ratio of the ratio of the number of arms in a time slot to the number of arms in a time slot]. ,arm The bangperbuck ratio is: ; in, Indicates arm In the time slot The UCB value, Representative selection arm In the time slot The estimated cost.

[0031] The UCB value is positively correlated with the reward per slot, and therefore positively correlated with the expected reward. Higher costs result in a lower number of communication slots, leading to a reduction in overall reward. Therefore, introducing the boomerbuck ratio helps to strike a trade-off between the reward per slot and the sum of accumulated rewards. In the time slot The UCB value is given by the following formula: ; in, Indicates the first Arm in each time slot The expected value of the reward, This indicates that the arm is in front. The total number of times selected within a time slot. and The calculation formula is as follows: ; ; In the time slot Select Arm The estimated cost is given by the following formula: .

[0032] Finally, parameter initialization is performed, including but not limited to... , , , Then proceed to each time slot. The cycle.

[0033] Online learning and decision-making phase S120: Based on historical feedback information, calculate the cost-benefit ratio of each candidate arm, combine it with the current remaining resource budget, select executable high-performance superarm combinations, and execute the selected configuration in the communication time slot to obtain dimensionality reduction channel state information and feedback rewards.

[0034] In each time slot First, the first step, through Calculate the bangperbuck ratio for all arms.

[0035] The second step is to sort the arms in descending order by the Bangperbuck ratio, and then obtain all the arms. The new index, that is, satisfying .

[0036] The third step is to calculate the number of available time slots for each arm within the budget. For example, arm The budget is The formula for calculating the number of selectable time slots is as follows: and update the remaining budget in the same way. ; The fourth step is to filter the basic sub-arms in sequence and select those that meet the requirements. , and ,(in Because when At that time, the rewards they received were negligible compared to the energy consumption of the radio frequency link.

[0037] Step 5: Execute the superarm and update each UCB value.

[0038] Step 6, Update and .

[0039] In the first stage, the pre-beamforming design problem is modeled as a combined budgetary beamforming (MAB) problem to maximize received energy. This MAB problem involves a trade-off: maximizing energy per time slot versus maximizing the number of communication time slots. To achieve a good trade-off, UAV energy consumption and received energy are jointly considered, and a greedy algorithm based on the boomer-buck ratio is proposed. The proposed algorithm makes decisions by learning from historical data online, while reducing the dimensionality of instantaneous channel state information, thereby reducing pilot overhead. Furthermore, the proposed algorithm reduces the number of activated radio links, improving energy efficiency. In the second stage, a new equivalent channel matrix is ​​defined, and the beamforming matrix is ​​obtained using SVD decomposition. Simulation results verify the superiority of the proposed algorithm over traditional channel estimation algorithms.

[0040] Beamforming construction stage S130: Construct an equivalent channel model using the obtained dimensionality-reduced channel information, generate a digital beamforming matrix based on singular value decomposition, and combine it with an analog pre-beamformer to form an optimal beamformer.

[0041] make ,in This is a dummy variable, which will then be substituted into the aforementioned... .

[0042] because , Can be converted to: ; in, .

[0043] The corresponding closed-form solution is: ; in Through from Select the first from the right singular vector matrix Column-based.

[0044] When the optimal Then, the optimal beamformer can be derived: .

[0045] In the iterative update and control phase S140, the cost estimates of each arm and the remaining energy of the system are dynamically updated based on actual energy consumption and reward feedback.

[0046] After completing the beamformer calculations, update the estimated costs for each sub-arm. ; and through Update the remaining budget.

[0047] The method of this application was simulated and tested, and the simulation results are as follows: A CCA was deployed in the UAV, assuming that the UAV service equipped with a root antenna is equipped with a single UE with a root antenna. Figure 4 The reward curve obtained for each time slot according to one embodiment of this application is shown. Figure 4 As can be seen from the data, after a certain time interval, the distribution of rewards in this application tends to be stable, which shows the convergence of the proposed scheme. Figure 5 A graph showing the spectral efficiency as a function of signal-to-noise ratio according to an embodiment of this application is illustrated. Figure 5 This demonstrates that the application still exhibits superior performance under different signal-to-noise ratio environments.

[0048] In summary, this application effectively balances the contradiction between pilot overhead and the number of communication slots by constructing a two-stage beamforming problem model and introducing a multi-arm gambling machine mechanism. In the initialization and modeling stage, pre-beamforming candidate configurations are abstracted as "arms," ​​and an optimization framework is established by combining energy budget and reward estimation, providing a theoretical basis for subsequent decision-making. In the online learning and decision-making stage, a cost-benefit ratio index is used to screen high-performance super-arm combinations, prioritizing configurations with higher energy consumption returns under limited resources, significantly reducing pilot overhead during channel detection, and extending effective communication duration. In the beamforming construction stage, an optimal beamformer is generated based on dimensionality-reduced channel information, ensuring signal transmission quality while reducing the number of activated RF chains. The iterative update and control stage dynamically adjusts cost estimation and energy state to achieve adaptive resource allocation during long-term communication. Overall, this method, while satisfying the energy constraints of the UAV, reduces the pilot symbol length required for channel estimation and improves spectral efficiency per unit energy, showing significant improvements in both spectral utilization and energy efficiency compared to traditional fixed beamforming algorithms.

[0049] Figure 6 A schematic diagram of the beamforming apparatus of an mmWave UAV MIMO system based on a multi-armed gambling machine according to an embodiment of this application is shown. Figure 6 As can be seen, the beamforming device 600 of the mmWave UAV MIMO system based on a multi-armed gambling machine includes: Initialization and Modeling Unit 610: Used to construct a two-stage beamforming problem model, establish a pre-beamforming candidate configuration set, with each configuration serving as one arm in a multi-armed gambling machine, and initialize system state parameters, energy budget, reward estimation, and selection strategy; Online learning and decision-making unit 620: Based on historical feedback information, it calculates the cost-benefit ratio of each candidate arm, combines the current remaining resource budget, selects executable high-performance superarm combinations, and executes the selected configuration in the communication time slot to obtain dimensionality reduction channel state information and feedback rewards. Beamforming construction unit 630: used to construct an equivalent channel model using the obtained dimensionality reduction channel information, and generate a digital beamforming matrix based on singular value decomposition, and combine it with an analog pre-beamformer to form an optimal beamformer; Iterative update and control unit 640: used to dynamically update the cost estimate of each arm and the remaining energy of the system based on actual energy consumption and reward feedback.

[0050] In some embodiments of this application, the above-described apparatus employs two-stage beamforming for time-slot transmission, wherein the pre-beamformer of the first stage is represented as: ; in, In order to receive energy, Indicates the activation mode of CCA; The second-stage prebeamformer is represented as follows: .

[0051] In some embodiments of this application, in the above-described apparatus, the initialization and modeling unit 610 is used to convert the pre-beamformer of the first stage into a budgeted combined multi-arm gambling machine problem, the budgeted combined multi-arm gambling machine problem including the following key elements: arm, reward, budget and cost; Furthermore, the combined multi-armed gambling machine problem with budget also introduces the bangperbuck ratio.

[0052] In some embodiments of this application, in the above-described apparatus, the arm represents a pre-beamforming configuration unit, including beam pointing in the azimuth and elevation directions and an activation region of the antenna subarray; the activation region is divided into at least one of an upper half, a lower half, or a middle annular region, and each combination of spatial pointing and activation mode constitutes an independent candidate action. The reward is the total energy collected within a single time slot. The budget refers to the available energy of the drone's battery. The cost is defined as the energy consumption of the radio frequency link.

[0053] In some embodiments of this application, in the above-described apparatus, during time slots ,arm The bangperbuck ratio is: ; in, Indicates arm In the time slot The UCB value, Representative selection arm In the time slot Estimated costs; arm In the time slot The UCB value is: ; in, Indicates the first Arm in each time slot The expected value of the reward, This indicates that the arm is in front. The total number of times selected within each time slot; in, and The calculation formula is as follows: ; ; In the time slot Select Arm The estimated cost is: .

[0054] In some embodiments of this application, in the above-described apparatus, the online learning and decision-making unit 620 is used in each time slot. The first step is to... Calculate the bangperbuck ratio for all arms; The second step is to sort the arms in descending order by the Bangperbuck ratio, and then obtain all the arms. The new index; The third step is to calculate the number of available time slots for each arm within the budget. and update the remaining budget in the same way; The fourth step is to sequentially filter the basic sub-arms, and the basic sub-arms must meet the following conditions. , and ,in ; Step 5: Execute the superarm and update each UCB value.

[0055] Step 6, Update and .

[0056] In some embodiments of this application, in the above-described apparatus, the beamforming construction unit 630 is used to: ,in This will be used as a dummy variable, which will then be substituted into the pre-beamformer of the second stage. ; because ,Will Convert to: ; in, .

[0057] in, The corresponding closed-form solution is: ; in Through from Select the first from the right singular vector matrix Column-based; When the optimal Then, the optimal beamformer was derived: .

[0058] In some embodiments of this application, in the above-described apparatus, the iterative update and control unit 640 is used to update the estimated cost of each sub-arm. ; and through Update the remaining budget.

[0059] It should be noted that the beamforming device of the mmWave UAV MIMO system based on the multi-armed gambling machine can realize the aforementioned beamforming method of the mmWave UAV MIMO system based on the multi-armed gambling machine, and will not be described in detail here.

[0060] Figure 7 This invention illustrates a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 7 As shown, the electronic device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used for communication with external devices via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a near-field beam training method with an integrated user scheduling mechanism.

[0061] In one embodiment, the electronic device provided in this application includes a memory and a processor. The memory stores a database and a computer program that can run on the processor. When the processor executes the computer program, it implements the steps of a beamforming method for a multi-armed gambling machine-based mmWave UAV MIMO system.

[0062] The above is as stated in this application. Figure 6 The beamforming device of the mmWave UAV MIMO system based on a multi-armed gambling machine disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The steps of the method disclosed in the embodiments of this application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0063] In one embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of a beamforming method for a multi-armed gambling machine-based mmWave UAV MIMO system.

[0064] It should be noted that the functions or steps that the above-mentioned electronic devices or computer-readable storage media can achieve can be referred to the relevant descriptions in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0065] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0066] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0067] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A beamforming method for an mmWave UAV MIMO system based on a multi-armed gambling machine, characterized in that, The method is applied to a millimeter-wave unmanned aerial vehicle (UAV) assisted multiple-input multiple-output (MIMO) downlink communication system. The system includes a UAV base station equipped with a cylindrical conformal array and ground user equipment equipped with a uniform linear array. The method includes: Initialization and Modeling Phase: A two-stage beamforming problem model is constructed, and a set of pre-beamforming candidate configurations is established. Each configuration is considered an arm in a multi-armed gambling machine. System state parameters, energy budget, reward estimation, and selection strategy are initialized. Each arm represents a pre-beamforming configuration unit, including beam pointing in the azimuth and elevation directions and the active region of the antenna subarray. The active region is divided into at least one of an upper half, a lower half, or a middle annular region. Each combination of spatial pointing and activation mode constitutes an independent candidate action. The initialization and modeling phase specifically includes: Two-stage beamforming is used for time-slot transmission. The pre-beamformer in the first stage is represented as follows: ； in, In order to receive energy, Indicates the activation mode of CCA. For the activated antenna array, For prebeamforming matrix, The total number of radio frequency links equipped for drone base stations; The second-stage prebeamformer is represented as follows: ； in, For digital beamforming matrix, This refers to the transmission power. The prebeamformer in the first stage is transformed into a budgeted combined multi-arm gambling machine problem, which includes the following key elements: arm, reward, budget, and cost; and the budgeted combined multi-arm gambling machine problem also introduces the boomerbuck ratio; The reward is the total energy collected within a single time slot, the budget is the available energy of the battery carried by the drone, and the cost is the energy consumption of the radio frequency link; In the time slot ,arm The bangperbuck ratio is: ； in, Indicates arm In the time slot The UCB value, Representative selection arm In the time slot Estimated costs; arm In the time slot The UCB value is: ； in, Indicates the first Arm in each time slot The expected value of the reward, This indicates that the arm is in front. The total number of times selected within each time slot; in, and The calculation formula is as follows: ； ； In the time slot Select Arm Estimated cost: ； Online learning and decision-making phase: Based on historical feedback information, the benefit-cost ratio of each candidate arm is calculated. Combined with the current remaining resource budget, a high-performance, cost-effective superarm combination is selected, and the selected configuration is executed in the communication time slot to obtain reduced-dimensional channel state information and feedback rewards. The online learning and decision-making phase specifically includes: In each time slot The first step is to... Calculate the bangperbuck ratio for all arms; The second step is to sort the arms in descending order by the Bangperbuck ratio, and then obtain all the arms. The new index; The third step is to calculate the number of available time slots for each arm within the budget. and update the remaining budget in the same way; The fourth step is to sequentially filter the basic sub-arms, and the basic sub-arms must meet the following conditions. , and ,in , For the number of data streams, Indicates superarm; Step 5: Execute the superarm and update each UCB value; Step 6, Update and ; Beamforming construction stage: An equivalent channel model is constructed using the obtained dimensionality-reduced channel information, and a digital beamforming matrix is ​​generated based on singular value decomposition. This matrix is ​​then combined with an analog pre-beamformer to form the optimal beamformer. The beamforming construction stage specifically includes: make ,in This will be used as a dummy variable, which will then be substituted into the pre-beamformer of the second stage. ; because ,Will Convert to: ； in, , For the user-side combination matrix, The pilot length, For the symbol length, For noise variance, The channel matrix between the drone and the user; in, The corresponding closed-form solution is: ； in, Through from Select the first from the right singular vector matrix Column-based; When the optimal Then, the optimal beamformer was derived: ； Iterative update and control phase: Based on actual energy consumption and reward feedback, dynamically update the cost estimates of each arm and the remaining energy of the system.

2. The method according to claim 1, characterized in that, The iterative update and control phase specifically includes: Update the estimated costs for each sub-arm ; and through Update the remaining budget.

3. A beamforming device for an mmWave UAV MIMO system based on a multi-armed gambling machine, characterized in that, The device includes: Initialization and Modeling Unit: Used to construct a two-stage beamforming problem model and establish a set of pre-beamforming candidate configurations. Each configuration is considered an arm in a multi-armed gambling machine. The unit initializes system state parameters, energy budget, reward estimation, and selection strategy. Each arm represents a pre-beamforming configuration unit, including beam pointing in the azimuth and elevation directions and the active region of the antenna subarray. The active region is divided into at least one of an upper half, a lower half, or a middle annular region. Each combination of spatial pointing and activation mode constitutes an independent candidate action. The initialization and modeling phase specifically includes: Two-stage beamforming is used for time-slot transmission. The pre-beamformer in the first stage is represented as follows: ； in, In order to receive energy, Indicates the activation mode of CCA. For the activated antenna array, For prebeamforming matrix, The total number of radio frequency links equipped for drone base stations; The second-stage prebeamformer is represented as follows: ； in, For digital beamforming matrix, This refers to the transmission power. The prebeamformer in the first stage is transformed into a budgeted combined multi-arm gambling machine problem, which includes the following key elements: arm, reward, budget, and cost; and the budgeted combined multi-arm gambling machine problem also introduces the boomerbuck ratio; The reward is the total energy collected within a single time slot, the budget is the available energy of the battery carried by the drone, and the cost is the energy consumption of the radio frequency link; In the time slot ,arm The bangperbuck ratio is: ； in, Indicates arm In the time slot The UCB value, Representative selection arm In the time slot Estimated costs; arm In the time slot The UCB value is: ； in, Indicates the first Arm in each time slot The expected value of the reward, This indicates that the arm is in front. The total number of times selected within each time slot; in, and The calculation formula is as follows: ； ； In the time slot Select Arm Estimated cost: The online learning and decision-making unit is used to calculate the cost-benefit ratio of each candidate arm based on historical feedback information, combine it with the current remaining resource budget, select executable high-performance superarm combinations, and execute the selected configuration in the communication time slot to obtain dimensionality-reduced channel state information and feedback rewards. The online learning and decision-making phase specifically includes: In each time slot The first step is to... Calculate the bangperbuck ratio for all arms; The second step is to sort the arms in descending order by the Bangperbuck ratio, and then obtain all the arms. The new index; The third step is to calculate the number of available time slots for each arm within the budget. and update the remaining budget in the same way; The fourth step is to sequentially filter the basic sub-arms, and the basic sub-arms must meet the following conditions. , and ,in , For the number of data streams, Indicates superarm; Step 5: Execute the superarm and update each UCB value; Step 6, Update and Beamforming building unit: Used to construct an equivalent channel model using the obtained dimensionality-reduced channel information, and generate a digital beamforming matrix based on singular value decomposition, which is then combined with an analog pre-beamformer to form an optimal beamformer; the beamforming building unit is specifically used for: make ,in This will be used as a dummy variable, which will then be substituted into the pre-beamformer of the second stage. ; because ,Will Convert to: ； in, , For the user-side combination matrix, The pilot length, For the symbol length, For noise variance, The channel matrix between the drone and the user; in, The corresponding closed-form solution is: ； in, Through from Select the first from the right singular vector matrix Column-based; When the optimal Then, the optimal beamformer was derived: ； Iterative update and control unit: used to dynamically update the cost estimates of each arm and the remaining energy of the system based on actual energy consumption and reward feedback.

4. A millimeter-wave unmanned aerial vehicle (UAV) assisted multiple-input multiple-output downlink communication system, characterized in that, The system includes a UAV base station equipped with a cylindrical conformal array and a ground user equipment equipped with a uniform linear array. The system is equipped with a beamforming device for the mmWave UAV MIMO system based on a multi-arm gambling machine as described in claim 3.

Images