IoT device polarization optimization method based on multi-armed bandit algorithm

Abstract

The application discloses an IoT device polarization optimization method based on a multi-arm slot machine algorithm, which is performed according to the following steps: 1) establishing a wireless transmission frame structure; 2) based on the wireless frame structure, constructing a system transmission rate maximization problem, and decomposing the problem into a frame-level polarization optimization problem and a coherent interval-level joint optimization problem of AP receiving beams and RIS phases; 3) for the optimization problem at the coherent interval level, the AO algorithm is used to iteratively solve the BS receiving beam forming vector and the RIS phase configuration; 4) for the frame-level polarization rotation direction optimization problem, the problem is reconstructed into an MAB problem, and an iterative learning algorithm is used to find the best arm as the optimal polarization rotation direction. Under the condition that the spectrum resource and transmission power are limited, the method reduces the polarization loss by rotating the polarization direction, thereby significantly improving the system transmission performance.

Description

Technical Field

[0001] This invention belongs to the field of Internet of Things (IoT), specifically, it relates to a polarization optimization method for IoT devices based on a multi-armed slot machine algorithm. Background Technology

[0002] With the rapid development of Internet of Things (IoT) technology, low-power, low-cost IoT devices are being widely deployed. However, IoT devices are often limited by transmit power, antenna size, and hardware cost, and their communication quality is prone to degradation in complex propagation environments or when direct links are blocked. Reconfigurable smart surfaces (RIS) enhance the propagation channel between access points (APs) and IoT devices by adjusting their own phase configuration, and have been extensively studied.

[0003] However, in practical IoT systems, even with a strong reflection link provided by the RIS (Reflection Link System), polarization loss still limits communication performance. Many low-cost IoT devices typically use single-polarized antennas, while the AP (Access Point) antenna polarization direction is often fixed. When the IoT device's orientation changes, its polarization direction may become inconsistent with the AP's polarization direction, leading to polarization mismatch, polarization loss, and reduced received power and transmission rate.

[0004] Existing RIS-assisted communication methods mostly focus on RIS phase design and AP coding design, typically assuming polarization direction matching of the transmit and receive antennas, without fully considering the polarization mismatch problem caused by polarization direction changes in IoT devices. Therefore, how to jointly optimize RIS polarization and RIS phase configuration in RIS-assisted IoT networks to improve system transmission rate is an urgent problem to be solved. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a polarization optimization method for IoT devices based on a multi-armed slot machine algorithm, which is used to improve the system transmission rate of IoT networks by adaptively selecting the polarization rotation direction and combining RIS phase configuration and AP coding matrix design when polarization mismatch exists in IoT devices.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] I. Explanation of Symbols

[0008] In the notation, matrices and vectors are represented by bold uppercase letters and bold lowercase letters, respectively. and Representing matrices respectively The transpose and conjugate transpose of . Symbols , and Representing the Kronecker product, Hadama product, and Norm operators. This represents the modulo operation. (Operator) Indicates the vector Diagonalization is the primary method for taking diagonal elements from... The matrix. Finally, the complex Gaussian vector is denoted as... The mean vector is The covariance matrix is .

[0009] II. System Model

[0010] In the system model, the IoT network uplink transmission scenario includes a device equipped with... An AP with one antenna, equipped with RIS of individual units and A single-antenna IoT device, wherein the AP and RIS respectively adopt a size of [missing information]. and Uniform planar array (UPA), subscript and Let represent the row dimension and column dimension of UPA, respectively. Furthermore, it is assumed that the channel state information (CSI) and the polarization direction of the IoT device are known and can be obtained through pilot estimation within the channel coherence interval.

[0011] The system model includes two scenarios: In the first scenario, it is assumed that the IoT device is stationary, that is, the position and polarization direction of the device remain unchanged; in the second scenario, the IoT device has low-speed movement characteristics and the device's posture changes slowly, that is, the position and polarization direction of the device are constantly changing.

[0012] III. Polarization Model

[0013] In the polarization model, it is assumed that the AP, RIS, and IoT devices all use linearly polarized antennas. The normalized Jones vector of the AP antenna and RIS element is defined as... This indicates that they can only transmit and receive vertically polarized electromagnetic wave signals; for the first An IoT device, whose normalized Jones vector is ,in This indicates the angle between the polarization direction and the horizontal polarization direction of the device.

[0014] In addition, using Indicates that by AP and the first Amplitude loss caused by polarization mismatch between devices. The normalized receiver power loss (polarization loss) is... Given about Symmetry, dividing the interval Cases merged into intervals Thus obtain .

[0015] Furthermore, the polarization rotator enables polarization rotation that maximizes received power. The metasurface of this rotator is typically deployed between the device and the AP and can operate in either reflection or projection mode. Deploying this metasurface on the RIS in reflection mode allows the RIS to perform both polarization rotation and signal reflection. Therefore, the complete Jones matrix of the RIS can be represented as...

[0016] ,

[0017] in Indicates the inherent phase difference of incident polarization. This indicates the overall transmission phase introduced by the birefringent metasurface. is the polarization rotation angle on the RIS.

[0018] In addition, equipment The normalized received signal at the AP is

[0019] ,

[0020] and the received power loss is

[0021] .

[0022] IV. Channel Model

[0023] In the channel model, define respectively and For the channel and slave device from RIS to AP The channel to RIS. Furthermore, the definition... This is the RIS phase vector. (Device) The cascaded channel to the AP is

[0024] ,

[0025] in This represents the RIS phase diagonal matrix.

[0026] All channels are modeled as Ricean channels, therefore and The specific structure is as follows:

[0027] ,

[0028] in and The non-line-of-sight (NLOS) components of the channel are represented by their elements, which follow a distribution. , and They represent and Large-scale decline. and Represents Rice factor, The steering vector representing the angle of arrival (AOA) of the access point (AP). The steering vector representing the RIS departure angle (AOD). Indicates that it comes from the device The RIS-AOA steering vector.

[0029] V. Signal Model

[0030] In the signal model, the signal received by the AP is

[0031] ,

[0032] in Indicates equipment The transmitted data symbols, This indicates the data transmission power of each device. This indicates that the mean is 0 and the variance is 0. Additive white Gaussian noise (AWGN).

[0033] Applying equalization techniques to the AP receiver yields...

[0034] ,

[0035] in Indicates equipment The equalizer.

[0036] Therefore, equipment The uplink reachability rate is defined as

[0037] ,

[0038] Among the equipment SINR is defined as

[0039] ,

[0040] in This represents the normalized transmit power.

[0041] VI. Specific Steps

[0042] Specifically, the design steps of this method are as follows:

[0043] Step 1: Establish the wireless transmission frame structure;

[0044] Step 2: Based on the wireless frame structure, construct the system transmission rate maximization problem, and decompose it into a frame-level polarization optimization problem and a coherent interval-level AP receiving beam and RIS phase joint optimization problem.

[0045] Step 3: For the optimization problem at the coherent interval level, the alternating optimization (AO) algorithm is used to iteratively solve the AP receiving beamforming vector and RIS phase configuration;

[0046] Step 4: For the frame-level polarization rotation direction optimization problem, it is reconstructed into a multi-armed slot machine (MAB) problem, and the best arm is found as the optimal polarization rotation direction through an iterative learning algorithm.

[0047] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects:

[0048] This invention solves the uplink system rate maximization problem in IoT networks. This joint optimization problem involves phase optimization and polarization rotation angle optimization at the RIS (Receiving Arm) end, and receive beam matrix optimization at the AP (Access Point) end. First, based on the wireless frame structure, the original optimization problem is decomposed at different time scales. The polarization rotation angle is optimized at the frame level, while the AP receive beam and RIS phase configuration are optimized at the channel coherence interval level. This avoids simultaneously optimizing the RIS phase, polarization rotation angle, and AP receive beam, reducing the difficulty of solving the joint optimization problem. Furthermore, this invention reconstructs the polarization rotation direction optimization problem into a MAB (Multi-Arm Absorption Optimization) problem. Based on relevant statistical information learned through an iterative learning algorithm, a superarm is selected as the polarization rotation direction for the current frame in each iteration. This achieves polarization resource configuration without frequent global searches, effectively reducing polarization loss and thus improving the system transmission rate. Attached Figure Description

[0049] Figure 1 This is a model diagram of the IoT system of the present invention;

[0050] Figure 2 This is a schematic diagram of the method flow of the present invention;

[0051] Figure 3 This is a frame structure diagram of the transmission protocol considered in this invention. Detailed Implementation

[0052] To further understand the content of this invention, a detailed description of the invention will be provided in conjunction with the accompanying drawings and specific embodiments. It should be understood that the embodiments are merely illustrative and not limiting of the invention.

[0053] In one embodiment, the system model is as follows: Figure 1 As shown, in the IoT network uplink transmission scenario, there is a device equipped with An AP with one antenna, equipped with RIS of individual units and A single-antenna IoT device. The system model includes two scenarios: the first scenario is a static scenario, assuming the IoT device is stationary, that is, the device's position and polarization direction remain unchanged; the second scenario is a dynamic scenario, where the IoT device has low-speed movement characteristics, that is, the device's position changes continuously, and the polarization direction of the device's antenna undergoes a sudden change at regular intervals.

[0054] In one embodiment, the method flowchart is as follows: Figure 2 As shown, a polarization optimization method for IoT devices based on a multi-armed slot machine algorithm is provided. The method includes the following steps:

[0055] Step 1: Establish the wireless transmission frame structure;

[0056] Step 2: Based on the wireless frame structure, construct the system transmission rate maximization problem, and decompose it into a frame-level polarization optimization problem and a coherent interval-level AP receiving beam and RIS phase joint optimization problem.

[0057] Step 3: For the optimization problem at the coherent interval level, the alternating optimization (AO) algorithm is used to iteratively solve the AP receiving beamforming vector and RIS phase configuration;

[0058] Step 4: For the frame-level polarization rotation direction optimization problem, it is reconstructed into a multi-armed slot machine (MAB) problem, and the best arm is found as the optimal polarization rotation direction through an iterative learning algorithm.

[0059] The specific details of each step are as follows.

[0060] I. Specific details of Step One

[0061] In one embodiment, the frame structure designed in step one is as follows: Figure 3 As shown, each duration in the frame structure is... The frames are divided into Each channel has a coherent interval. Within each coherent interval, each IoT device sends a message to the AP. Data symbols.

[0062] We assume that the CSI remains constant within a channel coherence interval. Furthermore, in dynamic scenarios, we assume that the position and antenna orientation of the IoT device remain constant within a frame, meaning that the line-of-sight (LOS) component of the channel and the device polarization direction remain constant within a frame.

[0063] II. Specific details of step two

[0064] In one embodiment, step two is implemented as follows:

[0065] (1) Problem stereotypes

[0066] The problem of maximizing the system's transmission rate is as follows:

[0067] ,

[0068] in Indicates the system transmission rate. It is the first The transmission rate of an IoT device This represents the AP receive beamforming matrix. Indicates IoT devices equalizer, This represents the RIS phase diagonal matrix. Representation matrix The Middle Line 1 Column elements, This represents the polarization rotation angle on the RIS.

[0069] (2) Joint optimization design

[0070] The above problems are addressed by jointly optimizing the polarization rotation angle, RIS phase configuration, and AP receiver beamforming matrix, thus decomposing the problem on a time scale. Specifically, These are long-term control variables and should be optimized at the frame level; while and Then optimization is performed at the coherence interval level. First, at the beginning of each frame, determine... Then, within each coherent time slot of the current frame, based on the selected optimization and .

[0071] (3) Subproblem 1 - Joint Optimization and

[0072] Within each channel coherence interval, based on a given ,optimization and The subproblem is

[0073] .

[0074] (4) Subproblem 2 - Optimization

[0075] At the beginning of each frame, optimize The subproblem is

[0076] .

[0077] III. Specific details of step three

[0078] In one embodiment, step three is implemented as follows:

[0079] Using the AO algorithm, Decompose into two sub-problems: optimization Subproblems and optimization The two subproblems are then optimized alternately until the problem converges. The specific optimization methods for the two subproblems are as follows.

[0080] (1) Optimization subproblems

[0081] optimal solution Obtained by a minimum mean square error (LMMSE) equalizer, i.e.

[0082] ,

[0083] in Indicates length is unit array, IoT devices The optimal solution for the equalizer.

[0084] (2) Optimization subproblems

[0085] Note This is a classic multi-ratio fractional programming (FP) problem, therefore we employ the Lagrange dual transformation to... Rewritten as:

[0086] ,

[0087] in It is a set of auxiliary variables and has an optimal solution. .

[0088] After introducing auxiliary variables, the RIS phase optimization is transformed into a phase vector optimization problem under constant modulus constraints, and the successive convex approximation (SCA) method is used to iteratively update the RIS phase vector. .

[0089] IV. Specific details of step four

[0090] In one embodiment, step four is implemented as follows:

[0091] (1) Problem Restructuring

[0092] The MAB problem is modeled as follows:

[0093] ,

[0094] in Indicates a time interval The number of internal radio frames, Indicates the first The polarization rotation angle selected during each frame. The polarization rotation angle is discretized and used... express A set of candidate polarization rotation angles. The reward value is... The definition is as follows:

[0095] ,

[0096] Operators This represents the expectation operation. Indicates the first During the first frame period The system transmission rate of each channel coherent interval.

[0097] In the MAB problem In the game, AP is considered the player. Indicates the first One arm, As a collection of arms, It is the number of arms, and the first The arm selected during each frame This is called a superarm. Furthermore, only one arm is selected per frame.

[0098] Next, we will give the definition of the superarm in static and dynamic scenarios respectively.

[0099] 1. Static Scene

[0100] In a static scenario, the expected reward value of each arm remains constant, and the optimal superarm is...

[0101] ,

[0102] in Indicates a time interval The arm with the highest expected reward value. For the first Individual arm reward value The expected reward value.

[0103] 2. Dynamic Scenes

[0104] In dynamic scenarios, the expected reward value of each arm changes over time. Therefore, in the... During each frame, the optimal superarm and the highest expected reward value Defined respectively

[0105] ,

[0106] and

[0107] .

[0108] (2) Polarization rotation optimization

[0109] Iterative learning algorithms are used to solve The algorithm iterates for a total of Each round corresponds to one wireless frame. In static scenarios, the iterative learning algorithm uses the upper confidence bound (UCB) algorithm; while in dynamic scenarios, the iterative learning algorithm uses the discounted upper confidence bound (D-UCB) algorithm. Specifically, the iterative learning algorithm executes according to the following steps:

[0110] Step a: Define the metric value for each arm, which is calculated based on the arm's historical reward value and the number of times the arm was selected.

[0111] Specifically, the metric values ​​for the two algorithms are defined as follows:

[0112] 1. UCB Algorithm

[0113] In a static scenario, the update rule for the metric value of each arm in the UCB algorithm is as follows:

[0114] ,

[0115] in Indicates the first Before the nth iteration The number of times each arm is selected is defined as:

[0116] ,

[0117] and Indicates the first Before the next iteration, select the... The average reward value of each arm is defined as follows:

[0118] ,

[0119] in Indicates the first Choose the arm in the next iteration. The rewards received. It is an indicator function, if the first... The round selected is the first If there are one arm, the function takes the value 1; otherwise, it takes the value 0. These are the coefficients used in the UCB algorithm to balance exploration and exploitation. Furthermore, it is stipulated that... .

[0120] 2. D-UCB Algorithm

[0121] In dynamic scenarios, the update rule for the metric values ​​of each arm in the D-UCB algorithm is as follows:

[0122] ,

[0123] in Indicates the first Before the next iteration, select the... The average reward value of each arm after decay weighting is defined as follows:

[0124] ,

[0125] For the first The weighted total number of selections for all arms before the next iteration is defined as follows:

[0126] ,

[0127] and Indicates the first Before the nth iteration The decay-weighted selection number of each arm is defined as follows:

[0128] ,

[0129] in and These are all coefficients used in the D-UCB algorithm to balance exploration and exploitation.

[0130] Step b: In the front In the next iteration, each arm is selected in turn to initialize the reward values ​​for all arms.

[0131] Step c: in the At the start of each iteration, the index value for each arm is calculated, and the arm with the largest index value is selected as the superarm.

[0132] Step d: in the At the end of the iteration, record the number of iterations. The superarm and its reward value are selected in the next iteration, and the selection count of that arm is incremented by 1.

[0133] Step e: Repeat steps c and d until... .

[0134] The present invention and its embodiments have been described above illustratively. This description is not restrictive, and the figures shown are only one embodiment of the present invention; the actual structure is not limited thereto. Therefore, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the present invention, such designs should fall within the protection scope of the present invention.

Claims

1. A polarization optimization method for IoT devices based on a multi-armed slot machine algorithm, characterized in that, The specific steps of this method are as follows: Step 1: Establish the wireless transmission frame structure; Step 2: Based on the wireless frame structure, construct the system transmission rate maximization problem, and decompose it into a joint optimization problem of AP receiving beam and RIS phase at the coherent interval level, and a frame-level polarization optimization problem. Step 3: For the optimization problem at the coherent interval level, the AO algorithm is used to iteratively solve the AP receiving beamforming vector and RIS phase configuration; Step four: For the frame-level polarization rotation direction optimization problem, it is reconstructed into a MAB problem, and the best arm is found as the optimal polarization rotation direction through an iterative learning algorithm.

2. The IoT device polarization optimization method based on a multi-armed slot machine algorithm according to claim 1, characterized in that, In IoT network uplink transmission scenarios, there is a device equipped with An AP with one antenna, equipped with RIS of individual units and A single-antenna IoT device.

3. The IoT device polarization optimization method based on the multi-armed slot machine algorithm according to claim 1, characterized in that, The specific frame structure of the transmission protocol considered in step one is as follows: Each duration in the frame structure is The frames are divided into Each channel has a coherent interval, and in each coherent interval, each IoT device sends a message to the AP. Data symbols.

4. The IoT device polarization optimization method based on the multi-armed slot machine algorithm according to claim 1, characterized in that, The system transmission rate maximization problem in step two is modeled as follows: ， in Indicates the system transmission rate. It is the first The transmission rate of an IoT device This represents the AP receive beamforming matrix. Indicates IoT devices equalizer, This represents the RIS phase diagonal matrix. Representation matrix The Middle Line 1 Column elements, Represents the polarization rotation angle on RIS, operator Represents the modulo operation, operator Represents the 2-norm operation.

5. The IoT device polarization optimization method based on the multi-armed slot machine algorithm according to claim 1, characterized in that, The joint optimization problem of AP receiving beam and RIS phase at the coherent spacing level in step two is modeled as follows: 。 6. The IoT device polarization optimization method based on the multi-armed slot machine algorithm according to claim 1, characterized in that, The frame-level polarization optimization problem in step two is modeled as follows: 。 7. The IoT device polarization optimization method based on a multi-armed slot machine algorithm according to claim 1, characterized in that, The iterative algorithm in step three is used to solve... The algorithm comprises the following sub-steps: Step a: Initialization and ; Step b: Fix ,renew ; Step c: Fix ,renew ; Step d: Alternate between steps b and c until the algorithm converges.

8. The IoT device polarization optimization method based on the multi-armed slot machine algorithm according to claim 1, characterized in that, The MAB problem model in step four is as follows: ， in Indicates a time interval The number of internal radio frames, Indicates the first The polarization rotation angle selected during each frame is discretized and then used as... express A set of candidate polarization rotation angles, where the reward value The definition is as follows: ， Operators This represents the expectation operation. Indicates the first During the first frame period The system transmission rate of each channel coherent interval; In the MAB problem In the game, AP is considered the player. Indicates the first One arm, As a collection of arms, It is the number of arms, and the first The arm selected during each frame It is called a superarm; in addition, only one arm is selected per frame. definition As the first Individual arm reward value Given the expected reward value, the optimal superarm is: ， in Indicates a time interval The arm with the highest expected reward value.

9. The IoT device polarization optimization method based on the multi-armed slot machine algorithm according to claim 1, characterized in that, The iterative learning algorithm in step four is used to solve... The algorithm iterates for a total of Each round corresponds to one wireless frame. Specifically, the iterative learning algorithm executes according to the following steps: Step a: Define the metric value for each arm, which is calculated based on the arm's historical reward value and the number of times the arm was selected; Step b: In the front In each iteration, each arm is selected sequentially to complete the initialization of all arms; Step c: In the first At the beginning of the next iteration, the index value of each arm is calculated, and the arm with the largest index value is selected as the superarm. Step d: in the At the end of the iteration, record the number of iterations. The superarm selected in the next iteration and its reward value are incremented by 1. Step e: Repeat steps c and d until... .

Images