Low-altitude Internet of Things management system

Abstract

The invention discloses a low-altitude Internet of Things management system, and belongs to the technical field of Internet of Things. The system comprises a grouping module, a channel interference generation module, a first channel division module and a first reinforcement learning module, and the grouping module groups low-altitude operation targets and divides the low-altitude operation targets into a plurality of low-altitude operation target groups; the first channel division module divides uplink channel resources of the base station into a plurality of discontinuous uplink channels, and the channel interference generation module is used for interfering each uplink channel of the base station i; the first reinforcement learning module enables the low-altitude operation target of the group x to communicate through the uplink channel of the base station according to the uplink channel management strategy; obtaining an award according to a comparison result of the data rates of the adjacent time of the low-altitude operation target; and constructing a first action-state value function according to the reward, and obtaining an optimal uplink channel management strategy according to the second action-state value function. According to the invention, the uplink channel and the downlink channel of the base station in the upper and low altitude Internet of Things can be dynamically optimized, so that the total speed data rate is maximized.

Description

Technical Field

[0001] This invention relates to a low-altitude Internet of Things (IoT) management system, belonging to the field of IoT technology. Background Technology

[0002] Drones have become a significant driver of wireless networks due to their flexibility and ubiquity. The widespread adoption of 5G-related services has enabled the large-scale deployment of IoT devices. As an efficient multiple access technology, Non-Orthogonal Multiple Access (NOMA) has emerged as a promising solution for IoT networks. However, issues such as time-varying connectivity requirements, network dynamics, and resource management need to be addressed. In real-time environments, model-driven methods struggle to capture the inherent dynamic characteristics of communication networks. Summary of the Invention

[0003] To overcome the shortcomings of the prior art, the purpose of this invention is to provide an airborne Internet of Things (IoT) management system that can dynamically optimize the uplink and downlink channels of base stations in low-altitude IoT.

[0004] To achieve the aforementioned objective, this invention provides an airborne IoT management system, comprising: a grouping module, a channel interference generation module, a first channel partitioning module, and a first reinforcement learning module. The grouping module divides low-altitude operating targets into X low-altitude operating target groups, where any group is represented as... , ,Group Any low-altitude target in the middle is represented by y. , Belongs to group The number of low-altitude operating targets; the first channel allocation module divides the uplink channel resources of base station i into There are several discontinuous uplink channels. The channel interference generation module is used to interfere with each uplink channel of base station i. The first reinforcement learning module is based on the uplink channel management strategy. Make group low-altitude operation target Communication is conducted via the uplink channel w of base station i; based on the low-altitude operation target. Rewards are awarded based on the comparison of data rates at adjacent time points. According to the reward Construct the first action-state value function According to the first action-state value function Obtain the optimal uplink channel allocation scheme: , In the formula, , , , ; , , They represent the groups respectively. low-altitude operation target And the gain of the uplink channel w connected to base station i, the connectivity with base station i through the uplink channel w, and the power transmitted to base station i through the uplink channel w; for The set of all possible solutions.

[0005] Compared with the prior art, the low-altitude Internet of Things (IoT) management system provided by the present invention has the following beneficial effects: it can dynamically optimize the uplink and downlink channels of base stations in low-altitude IoT. Attached Figure Description

[0006] Figure 1 This is a block diagram of the low-altitude narrowband Internet of Things management system provided by the present invention. Detailed Implementation

[0007] It should be noted that, below, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The advantages and features of the present invention, as well as the methods for achieving these advantages and features, will become clear from the accompanying drawings and the detailed embodiments described below.

[0008] However, the present invention is not limited to the embodiments disclosed below, and can be implemented in many different forms. This embodiment is only used to make the disclosure of the present invention more complete and to fully inform those skilled in the art of the present invention of the scope of the invention. The present invention is defined only by the scope of the claims.

[0009] While terms such as "first," "second," etc., are used to describe various elements, components, and / or parts, these elements, components, and / or parts are not limited by these terms. These terms are used only to distinguish one element, component, or part from other elements, components, or parts. Therefore, it is apparent that, within the technical spirit of this disclosure, the first element, first component, or first part mentioned below may also be a second element, second component, or second part, and the terminology used in this specification is for describing embodiments only and is not intended to limit this disclosure.

[0010] In this specification, unless otherwise specified in the text, the singular includes the plural. The use of "comprising" and / or "consisting of" in this specification does not exclude the presence or addition of one or more other structural elements, steps, actions, and / or components mentioned.

[0011] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Furthermore, terms defined in commonly used dictionaries shall not be interpreted ideally or excessively unless explicitly and specifically defined.

[0012] First Embodiment

[0013] The low-altitude Internet of Things (IoT) provided in the first embodiment of the present invention can be a wireless communication network such as LoRa or Turmass. Although the wireless communication network is illustrated herein as LoRa or Turmass, the wireless communication network 100 can also employ technologies from 3G to 5G, 6G, LTE, WCDMA, GSM / EDGE, WiMAX, UMB, GSM, or any other similar network or system. The wireless communication network can also employ ultra-dense network (UDN) technology, which can transmit, for example, over millimeter waves (mmW).

[0014] The first embodiment of the present invention provides a low-altitude Internet of Things (IoT) comprising M base stations, wherein any base station is denoted as i. Each base station can serve at least one airspace to provide relay for low-altitude operational targets within that airspace, including unmanned aerial vehicles (UAVs). Each of the M base stations can be any type of base station capable of communicating with low-altitude operational targets in the low-altitude Internet of Things (IoT), including fixed-location base stations and mobile base stations, which can be UAV base stations. Low-altitude operational targets can be configured to transmit data to the base stations via air or radio interface in the uplink channel, and the base stations can transmit data to the low-altitude operational targets via air or radio interface in the downlink channel.

[0015] In the first embodiment of this invention, the base station is configured with a single omnidirectional antenna and performs data acquisition using interfering discontinuous channel resource blocks on a specified time-segmented trajectory (containing T time slots). Each time slot of the trajectory contains multiple uplink cycles and multiple downlink cycles. Each low-altitude operating target is assigned to a group, and the low-altitude operating targets in each group communicate with the base station via non-orthogonal multiple access (NOMA) technology, with a specified decoding and encoding order. For any given NOMA group, after determining the low-altitude IoT decoding order based on channel gain, continuous interference cancellation (SIC) is performed, allowing multiple low-altitude operating targets in the same uplink channel to be sorted from largest to smallest transmit signal gain.

[0016] In the first embodiment of the present invention, once the NOMA uplink is established, base station i receives the QPSK modulated superimposed signal: ,

[0017] In the formula, For base station i, low-altitude targets from group x can be detected via uplink channel w. Gain; For base station i and low-altitude operating targets belonging to group x The distance; Indicates low-altitude operating target The power of its transmission; Indicates path loss; This represents uplink channel white noise; External interference to the uplink channel includes interference with the communication signals of other low-altitude targets operating on the same uplink channel of the same base station, interference with the communication signals of other low-altitude targets operating on other uplink channels of the same base station, interference with the communication signals of other low-altitude targets operating on all uplink channels of different base stations, and artificially set interference. For low-altitude operation targets The connection coefficient of the uplink channel w with base station i is 1 if connected, and 0 otherwise; For low-altitude operation targets The transmitted signal.

[0018] The signal received by base station i from the low-altitude target receiving base station is: , In the formula, For low-altitude operating targets belonging to group x The perceived gain of downlink channel j from base station i; For base station i and low-altitude operating targets belonging to group x distance; This indicates that base station i communicates with low-altitude targets belonging to group x via downlink channel j. Distributed power; Indicates path loss; This represents white noise in the downlink channel; External interference to the downlink channel includes interference with the communication signals of other low-altitude targets operating on the same downlink channel of the same base station, interference with the communication signals of other low-altitude targets operating on other downlink channels of the same base station, and interference with the communication signals of other low-altitude targets operating on all downlink channels of different base stations. Let be the downlink channel j coefficient of base station i, 1 for connectivity and 0 otherwise; For base station i to communicate with low-altitude targets belonging to group x via downlink channel j The transmitted signal.

[0019] Figure 1 This is a block diagram of the low-altitude narrowband Internet of Things management system provided in the first embodiment of the present invention, as shown below. Figure 1As shown, the low-altitude narrowband IoT management system provided by the present invention includes: a grouping module, a channel interference generation module, a first channel partitioning module, and a first reinforcement learning module. The grouping module divides the low-altitude operating targets into X low-altitude operating target groups, where any group is represented as... , ,Group Any low-altitude target in the middle is represented by y. , Belongs to group The number of low-altitude operating targets; the first channel allocation module divides the uplink channel resources of base station i into There are several discontinuous uplink channels. The channel interference generation module is used to interfere with each uplink channel of base station i. The first reinforcement learning module is based on the uplink channel management strategy. Make group low-altitude operation target Communication is conducted via the uplink channel w of base station i; based on the low-altitude operation target. Rewards are awarded based on the comparison of data rates at adjacent time points. According to the reward Construct the first action-state value function According to the first action-state value function Obtain the optimal uplink channel allocation scheme: , In the formula, , , , ; , , They represent the groups respectively. low-altitude operation target And the gain of the uplink channel w connected to base station i, the connectivity with base station i through the uplink channel w, and the power transmitted to base station i through the uplink channel w; for The set of all possible solutions. In the first embodiment, It belongs to both action and state.

[0020] In this invention, , In the formula, , These represent low-altitude operating targets. The data rate of the uplink channel w connected to base station i at times t and t-1.

[0021] In the first embodiment of the present invention, the reinforcement learning module calculates the low-altitude operating target according to the following formula. Uplink data rate : , In the formula, For low-altitude operation targets The signal-to-noise ratio of the uplink channel w connected to base station i.

[0022] In the first embodiment of the present invention, , In the formula, , This indicates a low-altitude operating target belonging to group x and connected to base station i via uplink channel w. Different low-altitude targets; , , Low-altitude targets The gain of the uplink channel w connected to base station i, the connectivity with base station i via uplink channel w, and the power transmitted from base station i to i via channel w.

[0023] , In the formula, This indicates a low-altitude operational target that belongs to the low-altitude operational target group u and is connected to the uplink channel v of base station i; , , Let V represent the gain of the uplink channel v that belongs to the low-altitude target group u and is connected to the base station i, the connectivity with the base station i through the uplink channel v, and the power transmitted to the base station i through the uplink channel v, respectively. The number of low-altitude targets in group u; Let be the number of uplink channels for base station i.

[0024] , This indicates a low-altitude operational target connected to the uplink channel w of a base station m that is different from base station i; , , These represent low-altitude operating targets. The uplink channel w connected to base station m, the connectivity with base station m via uplink channel w, and the power transmitted to base station m via uplink channel w; The number of low-altitude targets in group z; Let m be the number of uplink channels for base station m.

[0025] For low-altitude operation targets The noise power of the uplink channel w connected to base station i.

[0026] The interference power generated by the channel interference generation module on channel w.

[0027] In the first embodiment of the present invention, , In the formula, The number of low-altitude targets in group x; Let be the number of uplink channels for base station i.

[0028] The low-altitude IoT management system provided in the first embodiment of the present invention further includes a second channel partitioning module and a second reinforcement learning module. The second channel partitioning module partitions the downlink channel resources of base station i into J discontinuous downlink channels; the second reinforcement learning module, according to the downlink channel management strategy... Make low-altitude targets Communicating via downlink channel j of base station i and obtaining power from base station i, based on low-altitude operation target. The second reward is obtained by comparing the data rates of adjacent time periods. According to the second reward Constructing the second action-state value function According to the second action-state value function Obtain the optimal downlink channel allocation scheme: , In the formula, , , , , This indicates that group x is a low-altitude operating target connected to downlink channel j of base station i; , , These represent the low-altitude targets in group x. The gain of downlink channel j connected to base station i at time t, the connectivity with base station i through downlink channel j, and the power obtained from base station i through downlink channel j; for The set of all possible solutions.

[0029] In the first embodiment of the present invention, , In the formula, , These represent low-altitude operating targets. The downlink data rate of downlink channel j connected to base station i at times t and t-1.

[0030] In the first embodiment of the present invention, the reinforcement learning module calculates the low-altitude operating target according to the following formula. downlink data rate : , In the formula, For low-altitude operation targets The signal-to-noise ratio of downlink channel j connected to base station i at time t.

[0031] In the first embodiment of the present invention, , In the formula, This indicates interference caused by different low-altitude targets on the same downlink channel from the same base station. , This indicates that the downlink channel j connected to base station i is related to the low-altitude operating target. Different low-altitude targets; , , Low-altitude targets The downlink channel j gain connected to base station i at time t, the connectivity with base station i via downlink channel j, and the power obtained from base station i via downlink channel j.

[0032] Interference caused by different low-altitude targets on different downlink channels of the same base station is represented as follows: , This indicates a low-altitude operational target connected to a downlink channel k that is different from downlink channel j at base station i; , , These represent low-altitude operating targets. The downlink channel k gain connected to base station i at time t, the connectivity between base station i and downlink channel k, and the distance from base station i to the low-altitude target via downlink channel k. Transmitted power; Let be the number of downlink channels for base station i.

[0033] , This indicates a low-altitude operational target connected to downlink channel j of a base station m that is different from base station i; , , These represent low-altitude operating targets. The downlink channel j gain connected to base station m, the connectivity with base station m via downlink channel j, and the distance to low-altitude targets via downlink channel j. Transmitting power; The number of low-altitude targets in group z; Let m be the number of downlink channels for base station m.

[0034] For low-altitude operation targets The noise power of the downlink channel j connected to base station i.

[0035] In the first embodiment of the present invention,

[0036] The first embodiment of the present invention can dynamically optimize the uplink and downlink channels of base stations in the low-altitude Internet of Things to maximize the total data rate through the above technical solution.

[0037] Second Embodiment

[0038] The second embodiment of the present invention only describes the contents that are different from those of the first embodiment; the contents that are the same will not be described again.

[0039] The optimal management strategy for the uplink channel is obtained through the following steps: S1-01: According to the uplink channel management policy Select an uplink channel connectivity and receive power scheme. Obtain channel gain and receive power allocation results. ; S1-02: If If it is a termination result, then Otherwise, store the uplink channel connectivity and receive power scheme: This indicates that an uplink channel management strategy is adopted. When the channel gain and received power allocation result is At that time, the uplink channel connectivity and receive power scheme are as follows: ; S1-03: , This is the time when the estimated value is updated; n is the number of steps; S1-4: If The cumulative return is calculated according to the following formula: , In the formula, It is the discount factor; S1-5: If Update the accumulated reward according to the following formula and the estimated value update time First State-Value Function : , , In the formula Step size; S1-6: Obtained according to the following formula , In the formula, for The set of all possible solutions.

[0040] The optimal downlink channel management strategy is obtained through the following steps: S2-01: According to the downlink channel management policy Select a downlink channel connectivity and transmit power scheme Obtain downlink channel gain and transmit power allocation results ; S2-02: If If it is a termination result, then Otherwise, store the downlink channel connectivity and transmit power scheme: , It indicates that a downlink channel management strategy is adopted. When the channel gain and transmit power allocation result is At that time, the downlink channel connectivity and base station transmit power scheme are as follows: ; S2-03: , This is the time when the estimated value is updated; n is the number of steps; S2-4: If The cumulative return is calculated according to the following formula: , In the formula, It is the discount factor; S2-5: If Update the accumulated reward according to the following formula and the estimated value update time Second State-Value Function : , , In the formula Step size; S2-6: Obtained according to the following formula , In the formula, For what The set of all possible solutions.

[0041] The second embodiment achieves the following beneficial effects through the above technical solution: It utilizes the cumulative reward over n steps, rather than relying solely on single-step rewards, allowing each update step to contain richer long-term reward information, thereby improving sample efficiency; simultaneously, it reduces the update frequency, lowers the fluctuation of value function estimation, and enhances the stability of the learning process; by adjusting the hyperparameter n, the algorithm can balance bias and variance: utilizing more future reward information reduces bias and may lead to more stable policy improvements; directly approximating long-term cumulative rewards reduces errors, thereby accelerating convergence; and it makes it more robust to randomness or delayed rewards in the environment, making it suitable for non-stationary environments.

[0042] It should be noted that the preferred embodiments of the present invention disclosed are only for the purpose of illustrating the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to specific implementation methods. Obviously, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A low-altitude Internet of Things (IoT) management system, characterized in that, include: The system comprises a grouping module, a channel interference generation module, a first channel partitioning module, and a first reinforcement learning module. The grouping module divides low-altitude targets into X groups, where any group is represented as... , ,Group Any low-altitude target in the middle is represented by y. , Belongs to group The number of low-altitude operating targets; the first channel allocation module divides the uplink channel resources of base station i into There are several discontinuous uplink channels. The channel interference generation module is used to interfere with each uplink channel of base station i. The first reinforcement learning module is based on the uplink channel management strategy. Make group low-altitude operation target Communication is conducted via the uplink channel w of base station i; based on the low-altitude operation target. Rewards are awarded based on the comparison of data rates at adjacent time points. According to the reward Construct the first action-state value function According to the first action-state value function Obtain the optimal uplink channel allocation scheme: ， In the formula, , , , ; , , They represent the groups respectively. low-altitude operation target And the gain of the uplink channel w connected to base station i, the connectivity with base station i through the uplink channel w, and the power transmitted to base station i through the uplink channel w; for The set of all possible solutions.

2. The low-altitude IoT management system according to claim 1, characterized in that, ， In the formula, , These represent low-altitude operating targets. The data rate of the uplink channel w connected to base station i at times t and t-1.

3. The low-altitude IoT management system according to claim 2, characterized in that, The reinforcement learning module calculates the low-altitude flight target according to the following formula. Uplink data rate : ， In the formula, For low-altitude operation targets The signal-to-noise ratio of the uplink channel w connected to base station i.

4. The low-altitude IoT management system according to claim 3, characterized in that, ， In the formula, , This indicates a low-altitude operating target belonging to group x and connected to base station i via uplink channel w. Different low-altitude targets; , , Low-altitude targets The uplink channel w gain connected to base station i, the connectivity with base station i via uplink channel w, and the power transmitted from base station i to i via channel w; , In the formula, This indicates a low-altitude operational target that belongs to the low-altitude operational target group u and is connected to the uplink channel v of base station i; , , Let V represent the gain of the uplink channel v, the connectivity with base station i via uplink channel v, and the power transmitted to base station i via uplink channel v, respectively, belonging to the low-altitude target group u and connected to base station i. The number of low-altitude targets in group u; Let be the number of uplink channels for base station i; , This represents a low-altitude operational target connected to channel w of a base station m that is different from base station i. , , These represent low-altitude operating targets. The uplink channel w connected to base station m, the connectivity with base station m via uplink channel w, and the power transmitted to base station m via uplink channel w; The number of low-altitude targets in group z; Let m be the number of uplink channels for base station m. For low-altitude operation targets The noise power of the uplink channel w connected to base station i; The interference power generated by the channel interference generation module on channel w.

5. The low-altitude IoT management system according to claim 4, characterized in that, , In the formula, The number of low-altitude targets in group x; Let be the number of uplink channels for base station i.

6. The low-altitude IoT management system according to claim 5, characterized in that, It also includes a second channel partitioning module and a second reinforcement learning module. The second channel partitioning module partitions the downlink channel resources of base station i into J discontinuous downlink channels; the second reinforcement learning module, based on the downlink channel management strategy... Make low-altitude targets Communicating via downlink channel j of base station i and obtaining power from base station i, based on low-altitude operation target. The second reward is obtained by comparing the data rates of adjacent time periods. According to the second reward Constructing the second action-state value function According to the second action-state value function Obtain the optimal downlink channel allocation scheme: ， In the formula, , , , , This indicates that group x is a low-altitude operating target connected to downlink channel j of base station i; , , These represent the low-altitude targets in group x. The gain of downlink channel j connected to base station i at time t, the connectivity with base station i through downlink channel j, and the power obtained from base station i through downlink channel j; for The set of all possible solutions.

7. The low-altitude IoT management system according to claim 6, characterized in that, ， In the formula, , These represent low-altitude operating targets. The downlink data rate of downlink channel j connected to base station i at times t and t-1.

8. The low-altitude IoT management system according to claim 7, characterized in that, The reinforcement learning module calculates the low-altitude flight target according to the following formula. downlink data rate : ， In the formula, For low-altitude operation targets The signal-to-noise ratio of downlink channel j connected to base station i at time t.

9. The low-altitude IoT management system according to claim 7, characterized in that, , In the formula, , This indicates that the downlink channel j connected to base station i is related to the low-altitude operating target. Different low-altitude targets; , , Low-altitude targets The downlink channel j gain connected to base station i at time t, the connectivity with base station i via downlink channel j, and the power obtained from base station i via downlink channel j; The number of low-altitude targets in group x; , This indicates a low-altitude operational target connected to a downlink channel k that is different from downlink channel j at base station i; , , These represent low-altitude operating targets. The downlink channel k gain connected to base station i at time t, the connectivity between base station i and downlink channel k, and the distance from base station i to the low-altitude target via downlink channel k. Transmitted power; The number of low-altitude targets in group u; Let be the number of downlink channels for base station i; , This indicates a low-altitude operational target connected to downlink channel j of a base station m that is different from base station i; , , These represent low-altitude operating targets. The downlink channel j gain connected to base station m, the connectivity with base station m via downlink channel j, and the distance to low-altitude targets via downlink channel j. Transmitting power; The number of low-altitude targets in group z; Let m be the number of downlink channels for base station m; For low-altitude operation targets The noise power of the downlink channel j connected to base station i.

10. The low-altitude Internet of Things management system according to claim 9, characterized in that, , In the formula, The number of low-altitude targets in group x; Let be the number of downlink channels for base station i.

Images