Wireless sensor network resource allocation method and apparatus, and electronic device

Abstract

The application discloses a wireless sensor network resource allocation method and device and electronic equipment. It is related to the field of artificial intelligence, and the method comprises the following steps: acquiring position information corresponding to a plurality of terminals included in a wireless sensor network; determining the data transmission rate between the plurality of terminals and corresponding radio remote units; acquiring an initial resource allocation strategy of the wireless sensor network; based on the position information corresponding to the plurality of terminals and the data transmission rate between the plurality of terminals and corresponding radio remote units, taking the maximum system throughput in the wireless sensor network as the optimization target, optimizing the initial resource allocation strategy to obtain the target resource allocation strategy of the wireless sensor network. The application solves the technical problems of low network resource allocation efficiency and low accuracy in the related art wireless sensor network resource allocation method.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and more specifically, to a method, apparatus, and electronic device for allocating resources in a wireless sensor network. Background Technology

[0002] With the deepening of the construction of the power Internet of Things (IoT) and the continuous increase in the number of wired channels in recent years, the demand for monitoring these channels has become increasingly urgent. There is a pressing need for low-cost, high-efficiency digital operation and maintenance methods to support this demand. Deploying low-cost wireless sensor network (WSN) systems to cover monitoring sensors is one of the important ways to realize channel communication systems in related technologies.

[0003] However, there is no mature resource allocation solution for wireless sensor networks with wired channels in the relevant technologies. The resource allocation strategy of wireless sensor networks is mainly determined manually, which not only has a strong subjectivity, but also suffers from low resource allocation efficiency and poor accuracy.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a method, apparatus, and electronic device for allocating resources in a wireless sensor network, thereby at least addressing the technical problems of low efficiency and low accuracy in resource allocation in related wireless sensor network resource allocation methods.

[0006] According to one aspect of the present invention, a method for allocating resources in a wireless sensor network is provided, comprising: acquiring location information corresponding to a plurality of terminals included in the wireless sensor network; determining the data transmission rate between the plurality of terminals and their corresponding radio frequency remote units; acquiring an initial resource allocation strategy for the wireless sensor network, wherein the initial resource allocation strategy is used to indicate the deployment location and initial bandwidth resource allocation strategy corresponding to the plurality of radio frequency remote units included in the wireless sensor network; and optimizing the initial resource allocation strategy based on the location information corresponding to the plurality of terminals and the data transmission rate between the plurality of terminals and their corresponding radio frequency remote units, with the goal of maximizing the total system throughput within the wireless sensor network, to obtain a target resource allocation strategy for the wireless sensor network.

[0007] According to another aspect of the present invention, a wireless sensor network resource allocation apparatus is also provided, comprising: a location acquisition module, configured to acquire location information corresponding to a plurality of terminals included in the wireless sensor network; a transmission rate determination module, configured to determine the data transmission rate between the plurality of terminals and their corresponding radio frequency remote units; an initial strategy determination module, configured to acquire an initial resource allocation strategy for the wireless sensor network, wherein the initial resource allocation strategy is used to indicate the deployment locations and initial bandwidth resource allocation strategies of the plurality of radio frequency remote units included in the wireless sensor network; and a strategy optimization module, configured to optimize the initial resource allocation strategy based on the location information corresponding to the plurality of terminals and the data transmission rate between the plurality of terminals and their corresponding radio frequency remote units, with the goal of maximizing the total system throughput within the wireless sensor network, to obtain a target resource allocation strategy for the wireless sensor network.

[0008] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement any one of the wireless sensor network resource allocation methods.

[0009] In this embodiment of the invention, location information corresponding to multiple terminals included in a wireless sensor network is obtained; the data transmission rate between the multiple terminals and their corresponding remote radio units is determined; an initial resource allocation strategy for the wireless sensor network is obtained, wherein the initial resource allocation strategy is used to indicate the deployment location and initial bandwidth resource allocation strategy corresponding to the multiple remote radio units included in the wireless sensor network; based on the location information corresponding to the multiple terminals and the data transmission rate between the multiple terminals and their corresponding remote radio units, the initial resource allocation strategy is optimized with maximizing the total system throughput within the wireless sensor network as the optimization objective, thereby obtaining the target resource allocation strategy for the wireless sensor network. This achieves the goal of optimizing the resource allocation strategy of the wireless sensor network with maximizing the total system throughput within the wireless sensor network as the optimization objective, thus realizing the technical effect of improving the efficiency and accuracy of wireless sensor network resource allocation, and solving the technical problems of low network resource allocation efficiency and low accuracy in related wireless sensor network resource allocation methods. Attached Figure Description

[0010] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0011] Figure 1 This is a flowchart of a wireless sensor network resource allocation method according to an embodiment of the present invention;

[0012] Figure 2 This is a comparison chart of the cumulative rewards of an optional agent throughout the training process according to an embodiment of the present invention;

[0013] Figure 3 This is a comparison chart of the total system throughput as a function of the number of radio frequency remote units under different optional algorithms according to an embodiment of the present invention;

[0014] Figure 4 This is a schematic diagram of a wireless sensor network resource allocation device according to an embodiment of the present invention. Detailed Implementation

[0015] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0016] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0017] Currently, the electricity load in large cities is constantly increasing. To alleviate the contradiction between the limited capacity of transmission corridors and the growth in load, beautify the urban landscape, ensure urban safety, and mitigate the impact of natural disasters such as typhoons on urban power systems, cable channels are being widely promoted and their proportion is constantly rising. However, the overall construction of cable channels is still in its initial stage. The inspection and fault diagnosis of cable channels mainly rely on manual labor, and there is a lack of mature, efficient, low-cost, and comprehensive solutions for monitoring the operating environment and condition of cable channels and for automatic early warning.

[0018] In recent years, with the deepening construction of the power Internet of Things (IoT) and the continuous increase in the number of wired channels, the demand for monitoring cable channels has become increasingly urgent. There is a pressing need for low-cost, high-efficiency digital operation and maintenance methods to support this demand. Deploying low-cost wireless sensor network (WSN) systems to cover monitoring sensors is one of the important ways to realize channel communication systems in related technologies. Furthermore, with the development of low-power wireless sensor network technology, its power supply requirements are becoming increasingly lower, making it one of the most economical and applicable methods for cable channel coverage.

[0019] However, there is no mature resource allocation solution for wireless sensor networks with wired channels in the relevant technologies. The resource allocation strategy of wireless sensor networks is mainly determined manually, which not only has a strong subjectivity, but also suffers from low resource allocation efficiency and poor accuracy.

[0020] According to an embodiment of the present invention, a method for allocating resources in a wireless sensor network is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0021] Figure 1 This is a flowchart of a wireless sensor network resource allocation method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0022] Step S102: Obtain the location information corresponding to the multiple terminals included in the wireless sensor network;

[0023] Optionally, the wireless sensor network can be a wireless sensor network within a cable tunnel, consisting of a aggregation node, multiple (e.g., K) remote radio units (RRUs), and multiple (e.g., K×N) terminals, where each RRU corresponds to N terminals. For example, the set of RRUs can be denoted as R = {R1, ..., R...} k ,...,R KLet D be the set of terminals served by K RRUs. 1 ,...,D k ,...,D K Each RRU serves N terminals, that is D = D 1 ∪D 2 ∪...∪D K This represents the total set of terminals. Optionally, these multiple terminals can be multiple sensors. By acquiring the location information corresponding to each of the multiple terminals, the location distribution of the multiple terminals in the wireless sensor network can be obtained.

[0024] Optionally, a convergence node refers to a node in a wireless sensor network responsible for collecting and processing data from multiple terminals and multiple radio frequency remote units, and transmitting the obtained data to a server or other upper-layer system. This convergence node can be implemented using a specially designed hardware device, which may include, but is not limited to, components such as a microprocessor, communication module, and storage device. The convergence node is primarily used to coordinate and manage the entire wireless sensor network, ensuring timely data transmission and processing.

[0025] In this implementation, cable channels refer to underground cable lines, mainly used for power transmission; while wireless sensor networks serve to monitor the operating environment and status of cable channels, using various IoT sensors to monitor the water level, gas, temperature, humidity, partial discharge, and other conditions within the channels; by combining cable channels with wireless sensor networks, the efficiency of wireless sensor networks within cable channels can be improved, and low-cost, high-efficiency digital cable channel operation and maintenance methods can be developed.

[0026] Step S104: Determine the data transmission rate between the multiple terminals and their corresponding radio frequency remote units;

[0027] Optionally, this data transmission rate is used to indicate the rate at which the terminal transmits data to the corresponding remote radio unit. By determining the data transmission rate of each of the multiple terminals to the corresponding remote radio unit, the efficiency and speed of each terminal transmitting data to the corresponding remote radio unit can be measured.

[0028] Step S106: Obtain the initial resource allocation strategy of the wireless sensor network, wherein the initial resource allocation strategy is used to indicate the deployment location and initial bandwidth resource allocation strategy of the multiple radio frequency remote units included in the wireless sensor network.

[0029] Optionally, a resource allocation strategy can be initialized for the wireless sensor network. The initial resource allocation strategy includes the deployment locations of multiple radio frequency remote units included in the wireless sensor network and an initial bandwidth resource allocation strategy, wherein the initial bandwidth resource allocation strategy is used to indicate the bandwidth resources initially allocated by the corresponding radio frequency remote unit to the corresponding terminal.

[0030] Step S108: Based on the location information corresponding to multiple terminals and the data transmission rate between multiple terminals and their corresponding radio frequency remote units, the initial resource allocation strategy is optimized with the goal of maximizing the total system throughput within the wireless sensor network, thus obtaining the target resource allocation strategy for the wireless sensor network.

[0031] It is understandable that by acquiring the location information of multiple terminals, the location distribution of multiple terminals in the wireless sensor network can be obtained; by measuring the data transmission rate between each terminal and its corresponding remote radio unit, the efficiency and speed of each terminal transmitting data to its corresponding remote radio unit can be measured; based on the terminal location information and terminal data transmission rate, and with the goal of maximizing the total system throughput within the wireless sensor network, the resource allocation of the wireless sensor network is optimized. The resulting target resource allocation strategy is the optimal resource allocation strategy that maximizes system throughput while considering terminal location information and terminal data transmission rate.

[0032] Optionally, the system throughput can be obtained by combining the data transmission rates of multiple terminals, for example, from the terminal To the radio frequency remote unit R k Data transmission rate TR n,k The system throughput of the corresponding wireless sensor network can be expressed as:

[0033] In one optional embodiment, based on the location information corresponding to multiple terminals and the data transmission rate between the multiple terminals and their corresponding remote radio units, the initial resource allocation strategy is optimized with the goal of maximizing the total system throughput within the wireless sensor network, resulting in a target resource allocation strategy for the wireless sensor network. This includes: constructing a state space based on the location information corresponding to multiple terminals and the data transmission rate between the multiple terminals and their corresponding remote radio units; constructing an action space based on multiple candidate resource allocation strategies; constructing a reward function based on the total system throughput; optimizing the parameters of the initial resource allocation model based on the state space and action space, with the goal of maximizing the function value of the reward function, to obtain a target resource allocation model; and using the target resource allocation model to obtain the target resource allocation strategy.

[0034] In the above approach, by constructing a state space based on terminal location information and data transmission rate, and an action space based on candidate resource allocation strategies, various possible states and selectable actions in the wireless sensor network system can be clearly described, providing a comprehensive consideration for subsequent resource allocation. By constructing a reward function based on the total system throughput, the advantages and disadvantages of different resource allocation strategies can be quantified, providing a clear objective for parameter optimization and selecting the optimal strategy. Optimizing the parameters of the initial resource allocation model to obtain the target resource allocation model can improve system performance and efficiency, making resource allocation more rational and effective. The target resource allocation strategy obtained using the target resource allocation model can guide the rational allocation of resources in the wireless sensor system, improving overall system performance and user experience.

[0035] Optionally, the state space S represents the location information and data transmission rate of each terminal, expressed by the formula: In action space A, the deployment locations and bandwidth resource allocation of all possible RRUs are represented by the formula: The reward function r(S,A) can be designed to maximize the total system throughput, as expressed by the formula:

[0036] In one optional embodiment, the initial resource allocation model is optimized based on the state space and action space, with the goal of maximizing the function value of the reward function, to obtain the target resource allocation model. This includes: optimizing the parameters of the initial resource allocation model using a deep deterministic policy gradient algorithm, with the goal of maximizing the function value of the reward function, based on the state space and action space, to obtain the target resource allocation model.

[0037] As can be understood, the Deep Deterministic Policy Gradient (DDPG) algorithm is a policy optimization algorithm for continuous action spaces. It combines deterministic policy gradient methods with deep learning techniques, primarily addressing the continuous action space problem in reinforcement learning by learning a deterministic policy function to maximize cumulative reward. By optimizing the parameters of the initial resource allocation model using DDPG, the model can allocate resources more effectively given a state and action space, maximizing the reward function's value. The optimized target resource allocation model can better adapt to environmental and demand changes, improving the resource utilization efficiency of wireless communication networks and reducing resource waste.

[0038] In one optional embodiment, the initial resource allocation model includes a policy network, a critic network, a target policy network, and a target critic network. Based on the state space and action space, and with the objective of maximizing the reward function value, a deep deterministic policy gradient algorithm is used to optimize the parameters of the initial resource allocation model to obtain a target resource allocation model. This includes: inputting the state space into the policy network to obtain a first action determined from the action space; evaluating the first action using the critic network to obtain a first action value function estimate; inputting the state space into the target policy network to obtain a second action determined from the action space; evaluating the second action using the target critic network to obtain a second action value function estimate; obtaining a first loss based on the first action value function estimate; obtaining a second loss based on the second action value function estimate; optimizing the parameters of the policy network, critic network, target policy network, and target critic network based on the first and second losses; repeating the above operations until a predetermined termination condition is reached; and constructing the target resource allocation model based on the updated parameters of the policy network, critic network, target policy network, and target critic network obtained at the predetermined termination condition.

[0039] Optionally, the target resource allocation model is set up by optimizing the parameters of four deep neural networks (DNNs): a policy network μ(S,ω), a critic network Q(S,A,λ), a target policy network μ′(S,ω′), and a target critic network Q′(S,A,λ′). Here, ω, λ, ω′, and λ′ represent the parameters to be optimized for each DNN. The policy network, based on the current state space, outputs an action, determining the resource allocation strategy to be adopted in the current state. In other words, the policy network is responsible for outputting actions to guide optimal decision-making in the current state, achieving optimal allocation of wireless communication network resources. The critic network evaluates the value of the actions output by the policy network, estimating the value function of the action based on the combination of action and state. In other words, the critic network provides feedback to the policy network, guiding it to adjust towards higher-value actions, thereby improving the efficiency of resource allocation. The target policy network is a "target version" of the policy network used to estimate the target action; it is used to compute the target action. Setting a target policy network reduces instability during training, and updating the target network makes training more stable and converges faster. The target critic network is a "target version" of the critic network used to estimate the target action value function. It is used to compute the value function of the target action. By setting up the target critic network, instability during training can be reduced. By updating the target network, training can be made more stable and converge faster.

[0040] In the above approach, the optimization problem of continuous action space is solved through interactive learning between the policy network and the critic network, improving the efficiency and accuracy of the resource allocation model. By continuously optimizing the network parameters of each network, the model can more accurately predict value functions and actions, improving the accuracy and efficiency of resource allocation. Through the updating and training iteration of the target network (i.e., the target policy network and the target critic network), the model becomes more stable and converges faster, reducing instability during the training process.

[0041] Optionally, the state space can be input into the policy network and the target policy network respectively, yielding their respective outputs: the first action and the second action. Further, a critic network and a target critic network are used to evaluate the first and second actions, obtaining their corresponding outputs: estimates of the first and second action value functions. Loss is calculated based on these estimates. Using the obtained first and second losses, the parameters of each network are optimized with the goal of minimizing the loss, iterating continuously until a predetermined condition is met (e.g., both the first and second losses are minimized).

[0042] In one optional embodiment, obtaining a second loss based on a second action value function estimate includes: obtaining a target action value function based on a reward function, a second action, and a second action value function estimate; and obtaining a second loss based on the mean squared error between the target action value function and the second action value function estimate.

[0043] Optionally, in the Deep Deterministic Policy Gradient (DDPG) algorithm, the target action-value function is a function used to estimate the long-term reward obtained by performing an action in the current state. It can be expressed by the formula as Where i represents the current round, i+1 represents the next round, and μ′(S i+1 ,ω′) represents the output of the target policy network in the current round (i.e., the second action), Q′(S i+1 ,μ′(S i+1 ,ω′),λ′) represents the output of the target policy network as μ′(S i+1 In the case of ω′), the output of the target commentator network (i.e., the estimate of the second action value function), S i+1 This indicates the state of the next round after the current round, and γ represents the discount factor.

[0044] Optionally, the first loss is used to update the parameters of the policy network, and the second loss is used to update the parameters of the critic network. This can be achieved by minimizing the target action-value function. The mean squared error between the network output Q(S,A,λ) and the commenter network output is expressed as the loss function. This loss function can be used to calculate the second loss, Loss(λ), where S i This represents the state of the current round. The above methods can adjust the parameters of the critic network to better fit the real action-value function, helping to improve the accuracy of the critic network's estimation of action values, thus more accurately guiding the agent's decision-making. Simultaneously, optimizing the critic network can improve the training effect of deep reinforcement learning algorithms, enabling the agent to better learn the optimal policy.

[0045] Optionally, the loss function of the policy network, i.e., the first loss Loss(ω), can be obtained as follows: Minimize the loss function of the policy network to obtain the policy gradient of the sampling policy. in, This represents the gradient value of Loss(ω); This represents the gradient value of the critic network's output Q(S,A,λ) under the influence of the parameter ω to be optimized in the policy network; This represents the gradient value of the critic network's output Q(S,A,λ) under the influence of the action space A; This represents the gradient value of the policy network's output Q(S,A,λ) under the influence of the parameter ω to be optimized.

[0046] Optionally, the parameter updates of the two target networks (i.e., the target policy network and the target critic network) are achieved through soft updates, λ”←τλ+(1-τ)λ′, ω”←τω+(1-τ)ω′, where λ” represents the updated target policy network parameters; ω” represents the updated target critic network parameters; and τ is the soft update coefficient.

[0047] In one optional embodiment, based on the state space and action space, the initial resource allocation model is optimized with the goal of maximizing the function value of the reward function to obtain a target resource allocation model. This includes: determining constraints, wherein the constraints include at least bandwidth resource allocation constraints, data transmission power constraints, and data transmission rate constraints; and optimizing the parameters of the initial resource allocation model based on the state space, action space, and constraints with the goal of maximizing the function value of the reward function to obtain the target resource allocation model.

[0048] Optionally, during the parameter optimization of the initial resource allocation model, by constraining the allocation of bandwidth resources, we can ensure the rational use of resources, avoid over-allocation or waste of resources, and improve network performance and efficiency; by constraining data transmission power, we can limit data transmission power, reduce energy consumption, reduce interference and uneven signal strength, and ensure communication quality and equipment lifespan; by constraining data transmission rate, we can balance the competition between different terminals or tasks, and ensure the fairness and stability of data transmission.

[0049] Optionally, the bandwidth resource allocation constraint is used to indicate that the total amount of bandwidth resources allocated by multiple radio frequency remote units to their respective terminals is less than or equal to a predetermined amount of resources. The bandwidth resource constraint can be set in the following form:

[0050]

[0051] in, B represents the total amount of bandwidth resources allocated by multiple radio frequency remote units to the corresponding terminal. T This represents the total system bandwidth (i.e., the amount of reserved resources). This represents the bandwidth corresponding to any terminal.

[0052] Optionally, the data transmission power constraint is used to indicate that the data transmission power of any terminal among multiple terminals is within a predetermined transmission power range. The data transmission power constraint can be set in the following form:

[0053] 0≤P n,k ≤P max

[0054] Among them, P n,k P represents the data transmission power corresponding to any terminal. max This indicates the preset maximum transmission power.

[0055] Optionally, the data transmission rate constraint is used to indicate that the data transmission rate between any of the multiple terminals and the corresponding radio frequency remote unit is less than a predetermined transmission rate. The data transmission rate constraint can be set in the following form:

[0056] TR n,k ≤TR * n,k

[0057] Among them, TR * n,k This indicates the predetermined transmission rate for any given terminal.

[0058] In one alternative embodiment, the data transmission rate constraint is obtained as follows:

[0059]

[0060] Among them, TR * n,k This indicates the predetermined transmission rate for any given terminal. This represents the bandwidth resources allocated by any radio frequency remote unit to any terminal; P n,k This represents the data transmission power from any terminal to any radio frequency remote unit; h n,k σ represents the channel gain between any RF remote unit and any terminal; 2 This indicates the preset noise power.

[0061] Through the above steps S102 to S108, the resource allocation strategy of the wireless sensor network can be optimized with the goal of maximizing the total system throughput within the wireless sensor network. This achieves the technical effect of improving the efficiency and accuracy of wireless sensor network resource allocation, thereby solving the technical problems of low network resource allocation efficiency and low accuracy in related wireless sensor network resource allocation methods.

[0062] Based on the above embodiments and optional embodiments, the present invention proposes an implementation method for an optional wireless sensor network resource allocation method, the method comprising:

[0063] S1. Model Establishment: A system model of the wireless sensor network within the cable tunnel is established, and the total throughput of the cable tunnel wireless sensor network system is derived. This includes the following steps:

[0064] S11, consider a wireless sensor network in a cable tunnel, which consists of a convergence node, K radio remote units (RRUs) and multiple terminals;

[0065] S12, denote the RRU set as R = {R1, ..., R}. k ,...,R K Let D be the set of terminals served by K RRUs. 1 ,...,D k ,...,D K Each RRU serves N terminals, that is D = D 1 ∪D 2 ∪...∪D K Represents the total set of terminals;

[0066] S13, Let the x-coordinates of the RRU and the terminal in the channel be q respectively. k and in For a known fixed variable, q k The variables to be determined;

[0067] S14 uses the Channel Impulse Response (CI) model to describe the path loss between the RRU and the terminal, i.e., path loss. In the formula, f is the carrier frequency, c is the speed of light, ρ is the path loss exponent, d0 = 1m is the reference distance, and d n,k For the terminal To RRUR k The distance;

[0068] S15, and R k The channel gain between them is expressed as

[0069] S16, assuming from To R k The data transmission power is P n,k , To be assigned to the terminal bandwidth, σ 2 For noise power, then from the terminal To the radio frequency remote unit R k Data transmission rate TR n,k Constraints should be satisfied

[0070] S17, apply a data transmission power upper limit constraint to the terminal of 0 ≤ P n,k ≤P max , where P max This indicates the preset maximum transmission power;

[0071] S18, assuming the total system bandwidth is B T Apply bandwidth resource allocation constraints as and

[0072] S19, expressing the total system throughput of the cable channel wireless sensor network as

[0073] S2 employs the DDPG algorithm from deep reinforcement learning to solve the joint optimization problem of radio frequency remote unit deployment and bandwidth resource allocation, specifically including the following steps:

[0074] S21, to increase the system throughput of the wireless sensor network within the cable tunnel Maximize the horizontal coordinate q of the RRU. k and bandwidth resources As an optimization variable, let Represented as q, Represented as B;

[0075] S22, the optimization problem is formulated as follows: That is, the system's total throughput is at its maximum;

[0076] S23, describing the process from terminal D n k To RRU R k Maximum reachable rate constraint 1:

[0077] S24 describes the terminal transmission power upper limit constraint 2: 0≤P n,k ≤P max ;

[0078] S25 describes the system's allocable bandwidth resource constraint 3: and constraint 4:

[0079] S3 employs Deep Deterministic Policy Gradient (DDPG) from deep reinforcement learning to solve the joint optimization problem of radio frequency remote unit deployment and bandwidth resource allocation, specifically including the following steps:

[0080] S31, model the optimization problem as a Markov process, and then define the state space, action space and reward function;

[0081] S32, in the state space S, represents the location information and data transmission rate of each terminal, expressed by the formula:

[0082] S33, in action space A, represents the possible deployment locations of the left and right RRUs and the allocation of bandwidth resources, expressed by the formula:

[0083] S34, the reward function r(S,A) can be designed to maximize the total system throughput, expressed by the formula:

[0084] S35, Define historical state H t For H t ={S1,S2,...,S t Let's introduce a discount factor γ, representing the total reward r. t sum For r t sum =r t+1 +γr t+2 +...+γ T-t-1 r T , of which S a (a = 1, 2, ..., t) represents any state before historical time t, r b(b=1,2…,T) represents the reward at any point in the current round, where the discount factor γ is different for each point in time (distinguished by different superscripts);

[0085] S36 indicates that the expected payoff for state S is the state value function V. π (S)=E[r t sum |S t =S]=E[r t+1 +…+γ T-t-1 r T |S t =S];

[0086] The state-value function represents the expected cumulative reward that the agent can obtain in the current state. It measures the agent's long-term expected reward in a given state, that is, the expected cumulative reward obtained by adopting the optimal strategy in the current state.

[0087] S37 indicates that the expected reward of taking action A in state S is the action-value function Q(S,A)=E[r t sum |S t =S,A t =A]=E[r t+1 +...+γ T-t-1 r T |S t =S,A t =A];

[0088] The action-value function represents the expected cumulative reward an agent can obtain by taking a specific action in the current state. The action-value function measures the expected cumulative reward that can be obtained by taking a specific action in a given state.

[0089] S38, describing the state value function V π The conversion between (S) and the action value function Q(S,A) is expressed by the following formula:

[0090] The agent's policy can be optimized through the transformation between state-value functions and action-value functions. This mutual transformation helps the agent better evaluate the value of different states and actions, thus guiding it to learn the optimal policy. The transformation function is designed to achieve effective transformation between state-value functions and action-value functions in reinforcement learning, thereby improving the agent's learning efficiency and performance.

[0091] S39 defines four deep neural networks (DNNs): policy network μ(S,ω), critic network Q(S,A,λ), target policy network μ′(S,ω′) and target critic network Q′(S,A,λ′), where ω, λ, ω′, and λ′ represent the parameters of each DNN;

[0092] S310 indicates state S t Action A in t It can be A t =μ(S) t ,ω)+N t , where N t To detect noise, appropriate noise is added to prevent getting trapped in local optima;

[0093] S311 represents the target action value function. for Where i represents the current round, i+1 represents the next round, and μ′(S i+1 ,ω′) represents the output of the target policy network in the current round, Q′(S i+1 ,μ′(S i+1 ,ω′),λ′) represents the output of the target policy network as μ′(S i+1 The output of the target critic network in the case of ω′);

[0094] S312, using gradient descent to minimize the target action-value function. The mean squared error between the network output Q(S,A,λ) and the commenter network output is expressed as the loss function. Where M is the sampling batch size, i.e. the number of rounds sampled;

[0095] S313, the loss function Loss(ω) of the policy network is expressed as:

[0096] S314, minimize the loss function of the policy network to obtain the policy gradient of the sampling policy.

[0097] S315, update the parameters of the two target networks (i.e., the target policy network and the target critic network): λ”←τλ+(1-τ)λ′ and ω”←τω+(1-τ)ω′, where τ is the soft update coefficient.

[0098] It should be noted that the cable tunnel radio frequency remote unit deployment and bandwidth resource allocation optimization method based on deep reinforcement learning provided in this embodiment of the invention utilizes machine learning to solve the deployment and bandwidth resource allocation issues. The approach involves abstracting unit deployment and resource allocation into an optimization problem through system modeling, treating the location deployment of radio frequency remote units and bandwidth resource allocation as independent variables, and the total system throughput as the dependent variable. Then, a Markov model is applied to the problem, enabling the use of deep reinforcement learning to solve it. By defining the observation space, action space, and reward function, after a certain training period, the problem can be directly solved based on the observed data during the execution phase. Therefore, the cable tunnel radio frequency remote unit deployment and bandwidth resource optimization method based on machine learning proposed in this embodiment of the invention has advantages such as good optimization performance and high reliability. Furthermore, the deep reinforcement learning method DDPG used in this embodiment of the invention is a relatively novel and high-performance algorithm in machine learning, which not only overcomes the shortcomings of traditional optimization algorithms that optimize each variable individually, but also improves the total system throughput.

[0099] Based on the above embodiments and optional embodiments, the present invention proposes another optional implementation of the wireless sensor network resource allocation method, the method comprising:

[0100] S01: Initialize DNN parameters ω, λ, ω′, λ′, and round number ep max Number of steps T, learning rate α, detector noise N t Discount rate γ, soft update coefficient τ, replay buffer capacity M (i.e., sampling batch size), batch size m.

[0101] S02: Initialize channel state and resource requirements.

[0102] S03: Get state S t And select action A accordingly. t =μ(S) t ω)+N t .

[0103] S04: According to action A t Determine the RRU placement location v and bandwidth resource allocation B.

[0104] S05: Execute the corresponding strategy and obtain the next state S t+1 and reward r t .

[0105] S06: Store the conversion in the replay buffer (S t A t ,r t ,S t+1 And randomly sample m batches for conversion, where S t+1This represents the state at the next time step after time t.

[0106] S07: Determine the target action value function And minimize the loss function to update the critical network parameter λ.

[0107] S08: Update the actor network parameter ω, and simultaneously update the target actor network parameter ω′ and the target critic network parameter λ′.

[0108] S09: Output the fully trained neural network parameters ω,λ,ω′,λ′.

[0109] S010: Set simulation parameters: all RRUs and terminals are randomly distributed within a 1000m × 500m area; the number of terminals is 50; the number of RRUs is 2–8; carrier frequency f = 0.1 GHz; path loss index ρ = 3.27; noise power σ 2 = -122dBm; Total allocable system bandwidth resources B T =1MHz; all terminals are randomly generated within the specified area.

[0110] S011: Comparison with benchmark algorithms, including Uniform RRU Deployment (URD): All RRU units are uniformly distributed, but bandwidth resources are randomly allocated; Average Bandwidth Resource Allocation (ABRA): Bandwidth resources are uniformly distributed among terminals, but all RRU units are randomly allocated; Deep Q Network (DQN): The Deep Q Network algorithm is used to optimize location deployment and bandwidth resource allocation.

[0111] Figure 2 This is a graph showing the cumulative reward comparison results of an optional agent throughout the training process according to an embodiment of the present invention. The horizontal axis represents the number of episodes, and the vertical axis represents the total reward for each episode. It can be observed that the total reward gradually increases with training time, indicating that the actions taken by the agent receive higher rewards. The convergence reward value of the DDPG method is 8.45% higher than that of the DQN method. Furthermore, the convergence time of the DDPG algorithm is approximately 200 episodes, while the convergence time of the DQN algorithm is approximately 300 episodes.

[0112] Figure 3This is a comparison chart showing the change in total system throughput with the number of RRUs under different optional algorithms according to an embodiment of the present invention. It can be observed that the DDPG and DQN algorithms are significantly superior to the ABRA and URD algorithms. Specifically, when the number of remote radio units is 5, the total system transmission rate of the DDPG method in this embodiment of the invention is improved by 12.04%, 60.64%, and 132.21% compared to DQN, ABRA, and URD, respectively. As the number of RRUs increases, the total system throughput increases accordingly, eventually converging. This is because, with the increase in the number of RRUs, there are more optimal deployment locations to serve multiple users. However, when the number of RRUs reaches a certain level, due to the limited total bandwidth resources, the growth rate of the total system throughput will decrease, which limits the bandwidth available to users.

[0113] This embodiment also provides a wireless sensor network resource allocation device, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the terms "module" and "device" can refer to a combination of software and / or hardware that performs a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, hardware implementations, or a combination of software and hardware, are also possible and contemplated.

[0114] According to embodiments of the present invention, an apparatus embodiment for implementing the above-described wireless sensor network resource allocation method is also provided. Figure 4 This is a schematic diagram of the structure of a wireless sensor network resource allocation device according to an embodiment of the present invention, as shown below. Figure 4 As shown, the aforementioned wireless sensor network resource allocation device includes: a location acquisition module 400, a transmission rate determination module 402, an initial strategy determination module 404, and a strategy optimization module 406, wherein:

[0115] The location acquisition module 400 is used to acquire the location information of multiple terminals included in the wireless sensor network.

[0116] The transmission rate determination module 402 is connected to the location acquisition module 400 and is used to determine the data transmission rate between multiple terminals and their respective radio frequency remote units.

[0117] The initial strategy determination module 404 is connected to the transmission rate determination module 402 and is used to obtain the initial resource allocation strategy of the wireless sensor network. The initial resource allocation strategy is used to indicate the deployment location and initial bandwidth resource allocation strategy of the multiple radio frequency remote units included in the wireless sensor network.

[0118] The strategy optimization module 406, connected to the initial strategy determination module 404, is used to optimize the initial resource allocation strategy based on the location information corresponding to multiple terminals and the data transmission rate between the multiple terminals and their corresponding radio frequency remote units, with the goal of maximizing the total system throughput within the wireless sensor network, to obtain the target resource allocation strategy for the wireless sensor network.

[0119] In this embodiment of the invention, a location acquisition module 400 is configured to acquire location information corresponding to multiple terminals included in the wireless sensor network; a transmission rate determination module 402, connected to the location acquisition module 400, is configured to determine the data transmission rate between the multiple terminals and their corresponding radio frequency remote units; an initial strategy determination module 404, connected to the transmission rate determination module 402, is configured to acquire an initial resource allocation strategy for the wireless sensor network, wherein the initial resource allocation strategy indicates the deployment location and initial bandwidth resource allocation strategy corresponding to the multiple radio frequency remote units included in the wireless sensor network; and a strategy optimization module 406 is connected to the initial strategy determination module 400. 4. Based on the location information corresponding to multiple terminals and the data transmission rate between the multiple terminals and their corresponding radio frequency remote units, the initial resource allocation strategy is optimized with the goal of maximizing the total system throughput within the wireless sensor network. This yields the target resource allocation strategy for the wireless sensor network, achieving the objective of optimizing the resource allocation strategy with the goal of maximizing the total system throughput within the wireless sensor network. This improves the efficiency and accuracy of resource allocation in the wireless sensor network, thereby solving the technical problems of low efficiency and low accuracy in resource allocation methods for wireless sensor networks in related technologies.

[0120] It should be noted that the above modules can be implemented by software or hardware. For example, for the latter, it can be implemented in the following ways: the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0121] It should be noted that the location acquisition module 400, transmission rate determination module 402, initial strategy determination module 404, and strategy optimization module 406 mentioned above correspond to steps S102 to S108 in the embodiments. The instances and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules, as part of the device, can run in a computer terminal.

[0122] It should be noted that the optional or preferred implementation methods of this embodiment can be found in the relevant descriptions in the embodiments, and will not be repeated here.

[0123] The aforementioned wireless sensor network resource allocation device may also include a processor and a memory. The aforementioned location acquisition module 400, transmission rate determination module 402, initial strategy determination module 404, strategy optimization module 406, etc., are all stored in the memory as program modules, and the processor executes the aforementioned program modules stored in the memory to realize the corresponding functions.

[0124] The processor contains a core that retrieves the corresponding program modules from memory. One or more cores may be configured. Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory includes at least one memory chip.

[0125] According to an embodiment of this application, an embodiment of a non-volatile storage medium is also provided. Optionally, in this embodiment, the non-volatile storage medium includes a stored program, wherein, when the program runs, it controls the device containing the non-volatile storage medium to execute any of the aforementioned wireless sensor network resource allocation methods.

[0126] Optionally, in this embodiment, the non-volatile storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals, and the non-volatile storage medium includes stored programs.

[0127] Optionally, during program execution, the device containing the non-volatile storage medium performs the following functions: acquiring location information corresponding to multiple terminals included in the wireless sensor network; determining the data transmission rate between each terminal and its corresponding remote radio unit; acquiring the initial resource allocation strategy of the wireless sensor network, wherein the initial resource allocation strategy is used to indicate the deployment location and initial bandwidth resource allocation strategy of each of the multiple remote radio units included in the wireless sensor network; and optimizing the initial resource allocation strategy based on the location information corresponding to each terminal and the data transmission rate between each terminal and its corresponding remote radio unit, with the goal of maximizing the total system throughput within the wireless sensor network, to obtain the target resource allocation strategy of the wireless sensor network.

[0128] According to an embodiment of this application, an embodiment of a processor is also provided. Optionally, in this embodiment, the processor is used to run a program, wherein the program executes any of the wireless sensor network resource allocation methods described above.

[0129] According to an embodiment of this application, an embodiment of a computer program product is also provided, which, when executed on a data processing device, is adapted to execute a program that initializes the wireless sensor network resource allocation method steps described above.

[0130] Optionally, when the aforementioned computer program product is executed on a data processing device, it is suitable to execute an initialization program with the following method steps: acquiring location information corresponding to multiple terminals included in the wireless sensor network; determining the data transmission rate between the multiple terminals and their corresponding remote radio units; acquiring an initial resource allocation strategy for the wireless sensor network, wherein the initial resource allocation strategy is used to indicate the deployment location and initial bandwidth resource allocation strategy corresponding to the multiple remote radio units included in the wireless sensor network; and optimizing the initial resource allocation strategy based on the location information corresponding to the multiple terminals and the data transmission rate between the multiple terminals and their corresponding remote radio units, with the goal of maximizing the total system throughput within the wireless sensor network, to obtain a target resource allocation strategy for the wireless sensor network.

[0131] This invention provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps: acquiring location information corresponding to multiple terminals included in a wireless sensor network; determining the data transmission rate between each terminal and its corresponding remote radio unit; acquiring an initial resource allocation strategy for the wireless sensor network, wherein the initial resource allocation strategy indicates the deployment location of each of the multiple remote radio units included in the wireless sensor network and an initial bandwidth resource allocation strategy; and optimizing the initial resource allocation strategy based on the location information corresponding to each of the multiple terminals and the data transmission rate between each terminal and its corresponding remote radio unit, with the goal of maximizing the total system throughput within the wireless sensor network, to obtain a target resource allocation strategy for the wireless sensor network.

[0132] The order of the above embodiments of the present invention is merely for description and does not represent the superiority or inferiority of the embodiments.

[0133] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0134] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of modules described above can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between modules, and may be electrical or other forms.

[0135] The modules described above as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0136] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0137] If the aforementioned integrated modules are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable non-volatile storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a non-volatile storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned non-volatile storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0138] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for allocating resources in a wireless sensor network, characterized in that, include: Acquire the location information of multiple terminals included in the wireless sensor network; Determine the data transmission rate between the plurality of terminals and their respective radio frequency remote units; Obtain the initial resource allocation strategy of the wireless sensor network, wherein the initial resource allocation strategy is used to indicate the deployment location and initial bandwidth resource allocation strategy of the multiple radio frequency remote units included in the wireless sensor network. Based on the location information corresponding to the multiple terminals and the data transmission rate between the multiple terminals and their corresponding remote radio units, the initial resource allocation strategy is optimized with the goal of maximizing the total system throughput within the wireless sensor network. This optimization yields the target resource allocation strategy for the wireless sensor network, including: constructing a state space based on the location information corresponding to the multiple terminals and the data transmission rate between the multiple terminals and their corresponding remote radio units; constructing an action space based on multiple candidate resource allocation strategies; constructing a reward function based on the total system throughput; and, in the case where the initial resource allocation model includes a policy network, a critic network, a target policy network, and a target critic network, based on the state space and the action space, and with the goal of maximizing the function value of the reward function, the initial resource allocation model is optimized using a deep deterministic policy gradient algorithm in the following manner to obtain the target resource allocation model: inputting the state space into the policy network... The network obtains a first action determined from the action space; evaluates the first action using the critic network to obtain a first action value function estimate; inputs the state space into the target policy network to obtain a second action determined from the action space; evaluates the second action using the target critic network to obtain a second action value function estimate; obtains a first loss based on the first action value function estimate; obtains a second loss based on the second action value function estimate; optimizes the parameters of the policy network, the critic network, the target policy network, and the target critic network based on the first loss and the second loss; repeats the above operations until a predetermined termination condition is reached; constructs the target resource allocation model based on the updated policy network, the updated critic network, the updated target policy network, and the updated target critic network obtained at the predetermined termination condition; and obtains the target resource allocation strategy using the target resource allocation model.

2. The method according to claim 1, characterized in that, The process of obtaining the second loss based on the estimated value of the second action value function includes: Based on the reward function, the second action, and the estimated value of the second action value function, the target action value function is obtained; The second loss is obtained based on the mean square error between the target action value function and the estimated value of the second action value function.

3. The method according to claim 1, characterized in that, The step of optimizing the parameters of the initial resource allocation model based on the state space and the action space, with the objective of maximizing the function value of the reward function, to obtain the target resource allocation model includes: Define the constraints, wherein the constraints include at least bandwidth resource allocation constraints, data transmission power constraints, and data transmission rate constraints: Based on the state space, the action space, and the constraints, the parameters of the initial resource allocation model are optimized with the goal of maximizing the function value of the reward function, to obtain the target resource allocation model.

4. The method according to claim 3, characterized in that, The bandwidth resource allocation constraint is used to indicate that the total amount of bandwidth resources allocated by the plurality of radio frequency remote units to the corresponding terminals is less than or equal to a predetermined amount of resources. The data transmission power constraint is used to indicate that the data transmission power of any one of the plurality of terminals is within a predetermined data transmission power range. The data transmission rate constraint is used to indicate that the data transmission rate between any of the plurality of terminals and the corresponding radio frequency remote unit is less than a predetermined transmission rate.

5. The method according to claim 4, characterized in that, The data transmission rate constraint is obtained in the following manner: ； in, This represents the predetermined transmission rate corresponding to any of the terminals; This indicates the bandwidth resources allocated by any radio frequency remote unit to any terminal. This indicates the data transmission power of any terminal transmitting data to any radio frequency remote unit; This represents the channel gain between any of the radio frequency remote units and any of the terminals; This indicates the preset noise power.

6. A wireless sensor network resource allocation device, characterized in that, include: The location acquisition module is used to acquire the location information of multiple terminals included in the wireless sensor network. A transmission rate determination module is used to determine the data transmission rate between the plurality of terminals and their respective radio frequency remote units; An initial strategy determination module is used to obtain the initial resource allocation strategy of the wireless sensor network, wherein the initial resource allocation strategy is used to indicate the deployment location and initial bandwidth resource allocation strategy of the multiple radio frequency remote units included in the wireless sensor network. The strategy optimization module is used to optimize the initial resource allocation strategy based on the location information corresponding to the multiple terminals and the data transmission rate between the multiple terminals and their corresponding remote radio units, with the goal of maximizing the total system throughput within the wireless sensor network, to obtain the target resource allocation strategy for the wireless sensor network. This includes: constructing a state space based on the location information corresponding to the multiple terminals and the data transmission rate between the multiple terminals and their corresponding remote radio units; constructing an action space based on multiple candidate resource allocation strategies; constructing a reward function based on the total system throughput; and, in the case where the initial resource allocation model includes a policy network, a critic network, a target policy network, and a target critic network, optimizing the parameters of the initial resource allocation model using a deep deterministic policy gradient algorithm with the goal of maximizing the function value of the reward function, based on the state space and the action space, to obtain the target resource allocation model: inputting the state space into the... The policy network is used to obtain a first action determined from the action space; the critic network is used to evaluate the first action to obtain a first action value function estimate; the state space is input into the target policy network to obtain a second action determined from the action space; the target critic network is used to evaluate the second action to obtain a second action value function estimate; a first loss is obtained based on the first action value function estimate; a second loss is obtained based on the second action value function estimate; the parameters of the policy network, the critic network, the target policy network, and the target critic network are optimized based on the first loss and the second loss; the above operations are repeated until a predetermined termination condition is reached; the target resource allocation model is constructed based on the updated policy network, the updated critic network, the updated target policy network, and the updated target critic network obtained at the predetermined termination condition; the target resource allocation strategy is obtained using the target resource allocation model.

7. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the wireless sensor network resource allocation method according to any one of claims 1 to 5.

Images