Joint optimization method for link scheduling and power control in underwater visible light sensor network

Abstract

The invention discloses a link scheduling and power control joint optimization method in an underwater visible light sensor network. The method comprises the following steps: (1) modeling an underwater visible light channel; (2) modeling a multi-objective optimization problem of the underwater visible light sensor network; and (3) link scheduling and power control joint optimization based on multi-agent reinforcement learning. According to the method, the network performance optimization problem in the multi-hop underwater visible light sensor network is considered, in the method, the underwater visible light channel influenced by path loss, underwater turbulence effect and pointing error is considered, and underwater visible light signal transmission is represented by a more accurate model. Besides, from the perspective of link scheduling and power adjustment, a multi-constraint and multi-target optimization problem is proposed for the requirements of network freshness, fairness and energy consumption performance. And finally, aiming at the optimization model, providing a multi-target multi-agent algorithm with parameterized actions to solve the optimization problem, optimizing the operation of the network through distributed chain wheel scheduling and power control, and improving the comprehensive performance of the underwater visible light sensor network.

Description

Technical Field

[0001] The invention belongs to the technical field of radio networks, and in particular relates to a joint optimization method for link scheduling and power control in an underwater visible light sensor network. Background Art

[0002] Underwater Optical Wireless Sensor Networks (UOWSNs) were proposed to solve the problem of insufficient bandwidth and high latency in underwater acoustic sensor networks. For applications such as underwater disaster warning and real-time monitoring, the data bandwidth requirements are large and the information freshness requirements are high. UOWSNs can better meet the requirements. However, due to the absorption and scattering of light signals by water media and the influence of underwater turbulence on the propagation of light signals, the underwater visible light signal coverage is poor and the actual communication range is limited, which makes UOWSNs usually rely on multi-hop transmission to operate. In multi-hop networks, efficient scheduling is required from the perspective of multi-link selection to meet the application requirements of information freshness. The link scheduling methods of multi-hop sensor networks are mainly divided into centralized scheduling and distributed scheduling. Compared with centralized scheduling, the flexible characteristics of distributed scheduling are more suitable for the dynamic topology of underwater networks and have lower requirements for hardware devices.

[0003] Information freshness is measured by the age of information (AoI). Since information age overemphasizes the efficiency of information transmission, it usually causes uneven allocation of channel resources among network nodes, which in turn causes the premature death of some nodes in the sensor network. In order to avoid a short network life, fairness between nodes should be considered while considering the freshness of information. In addition, both indicators are affected by the transmission results, and power control in the specific transmission process can further optimize the power consumption of nodes. Therefore, combining power control with link scheduling is a more practical and effective direction.

[0004] Multi-agent reinforcement learning is often used to solve cooperation problems in reality. Agents must cooperate with other agents by considering the state of their environment and the behavior of other agents. At the same time, multi-objective reinforcement learning expands the single-objective reinforcement learning method so that it can handle two or more objectives. When the agent faces a mixed action space, that is, the agent needs to select a discrete action and a continuous parameter associated with it at each time step, parameterized action can solve such problems. The present invention combines the reinforcement learning of multi-agent multi-objective parameterized actions with the design in response to the application requirements of UOWSNs link scheduling and power control under multiple objectives.

[0005] At present, the relevant research papers on the joint optimization problem of link scheduling and power control in underwater visible light sensor networks are as follows:

[0006] 1. Zhengxin Fan et al. proposed a method for the maximum link scheduling problem in UOWSNs in the article "A Link Scheduling Algorithm for Underwater Optical Wireless Networks" published at the 2020 IFIP Networking Conference. The goal is to maximize the number of links scheduled in a time slot and improve the channel utilization of the network. However, the method does not consider the effects of underwater turbulence when establishing a physical interference model, and still uses the greedy idea for link scheduling, which limits the versatility of this method.

[0007] 2. Yu Tian et al. proposed a scheme to minimize the network AoI in UOWSNs using a greedy link scheduling strategy in the article “Minimizing Age of Information for Underwater Optical Wireless Sensor Networks” published in IEEE INFOCOM in 2023. However, this method assumes that all sensor nodes are located on the water surface and information only flows in a two-dimensional plane underwater, which limits the versatility of the collision and interference models considered by the scheme. In addition, since the scheme uses a greedy strategy for link scheduling, the method only focuses on the freshness requirements, resulting in the neglect of node energy consumption and network life, and the inability to optimize network performance at multiple levels. Therefore, this method is only suitable for some application scenarios where information freshness is extremely important.

[0008] 3. Tong Zhang et al. proposed a method for joint optimization of link scheduling and power allocation in underwater acoustic sensor networks with limited node energy in the article "Joint Link Scheduling and Power Allocation in Imperfect and Energy-Constrained Underwater Wireless Sensor Networks" published in the 2024 IEEE Transactions on Mobile Computing, and used multi-agent reinforcement learning to solve this optimization problem. However, since this method is designed for underwater acoustic networks, it is not suitable for the directional transmission form of UOWSNs. In addition, the link scheduling of this method is implicit in the power allocation, and the agent only focuses on its transmission power adjustment, which is also not applicable to multi-hop networks. Summary of the invention

[0009] To address the above problems, a joint optimization method for link scheduling and power control in underwater visible light sensor networks was invented. For multi-hop sensor networks, the path loss, underwater turbulence effect and pointing error characteristics experienced by underwater optical signals were taken into account. Link scheduling and power control were performed based on the UOWSNs multi-objective optimization problem that comprehensively considered information age, fairness and energy consumption performance. This problem was solved based on multi-objective multi-agent reinforcement learning in parameterized action space.

[0010] To achieve the above technical objectives and the above technical effects, the present invention is implemented through the following technical solutions:

[0011] A method for joint optimization of link scheduling and power control in an underwater visible light sensor network comprises the following steps:

[0012] (1) Underwater visible light link channel modeling

[0013] The underwater visible light link channel model is established by comprehensively considering the underwater optical signal path loss, underwater turbulence effect and pointing error at the receiving end. The propagation of underwater optical signals must be processed by this channel gain model. The optical signal is described as light intensity, which represents the optical power per unit area. The signal is transmitted from the transmitting node, passes through the multiplicative channel gain and additive environmental noise, and is received at the receiving node. The channel model acts on the signal in the form of channel gain.

[0014] (2) Modeling of multi-objective optimization problems for underwater visible light sensor networks

[0015] From the perspective of link scheduling and power control, the information age, fairness and energy consumption performance of underwater visible light sensor networks are comprehensively optimized. The constraint conditions of link scheduling and power adjustment are established using the underwater visible light link channel model, and a multi-constraint and multi-objective optimization problem is proposed.

[0016] (3) Joint optimization of link scheduling and power control based on multi-agent reinforcement learning

[0017] Distributed control treats each underwater node as a multi-agent, and seeks the Pareto solution for the multi-objective optimization problem in step (2). Link scheduling and power control are respectively the discrete and continuous actions of the agent, forming a hybrid action space decision. In each time slot, the output is whether the link between each node is scheduled and the specific transmission power, and the Pareto optimal solution is obtained.

[0018] Preferably, the specific settings of underwater optical signal path loss, underwater turbulence effect and receiving point pointing error in the above step (1) are:

[0019] The path loss is described by a negative exponential function of the propagation distance, and the attenuation amplitude is affected by absorption and scattering factors that are related to the wavelength;

[0020] The turbulence effect is described by a mixed exponential generalized gamma distribution. The pointing error starts from the radial displacement of the receiving beam center and the receiving point center, which satisfies the Rayleigh distribution. Since the two are independent and both act on the signal in the form of multiplicative channel gain, the two random distributions are combined, and their probability density function is described as follows:

[0021]

[0022] Where Г(x) is the gamma function, variable h is the channel gain, ω is the mixed weight of the exponential distribution and the generalized gamma distribution, with a value of 0<ω<1, λ is the parameter of the exponential distribution, a and c are the shape parameters of the generalized gamma distribution, and b is the scale parameter of the generalized gamma distribution. By controlling the values ​​of parameters ω, λ, a, b and c, the mixed exponential generalized gamma distribution is changed to adapt to different underwater turbulent environments. γ is the ratio of the equivalent beam radius of the receiver to the standard deviation of the pointing error, which reflects the degree of the pointing error. In order to quantitatively analyze the impact of the joint random channel gain, its cumulative distribution function is further integrated to obtain the following:

[0023]

[0024] Where Г(s,x) is the incomplete gamma function, and the cumulative distribution function is used to quantitatively calculate the channel outage probability.

[0025] Preferably, the multiple objectives in the above step (2) are specifically set as:

[0026] The information age is described by the time from data generation to dissemination;

[0027] The fairness is described by the allocation status of channel transmission resources, and the average number of channel transmission data packets is combined with the alpha fairness index, which balances network efficiency and fairness among users by adjusting parameter α.

[0028] The purpose of the optimization is to improve the data freshness of the underwater visible light sensor network through link scheduling, while ensuring fair allocation of resources for each channel, and on this basis optimize the transmission power of specific transmission to avoid energy waste and increase network life.

[0029] Preferably, the multiple constraints in the above step (2) are specifically set as four cases:

[0030] (i.) The node remaining energy limit requires that the energy consumption of the signal to be transmitted by the node in the channel must not cause the node to die, and puts forward requirements on the transmission power;

[0031] (ii.) Transmission collision restriction requires that there should be no collision caused by many-to-one transmission in multi-hop transmission;

[0032] (iii.) There may be interference at the receiving end of the signal, and the signal needs to meet the interruption probability requirement after the channel gain;

[0033] (iv.) In a full-duplex system, a node cannot transmit and receive simultaneously.

[0034] Preferably, the multi-constraint multi-objective optimization problem in the above step (2) is specifically set as:

[0035]

[0036] st E rest -Pη≤E th

[0037]

[0038] P out (SINR <SINR th )>0.5

[0039]

[0040] Where N is the number of network nodes, f α (x (i) ) is the alpha fairness index, whose variable x (i) is the average number of data packets transmitted, A is the information age, and the constraints correspond to the four cases described in claim 4: (i) E rest is the node residual energy, P is the transmission power, η is the energy conversion efficiency, E th is the minimum threshold of node energy; in (ii), link(n a ,n b ) represents node n a to n b Link scheduling status, N b-1 is node n b The previous hop node set; in (iii), P out (x) is the outage probability, SINR is the interference signal-to-noise ratio; in (iv), N b+1 is node n b The set of next hop nodes.

[0041] Preferably, the multi-agent reinforcement learning algorithm in the above step (3) is specifically set as follows:

[0042] The link scheduling and power control in the underwater visible light sensor network are regarded as partially observable Markov processes. The multi-agent reinforcement learning algorithm adopts a paradigm of centralized training and distributed execution. The mixed action space of the agent is designed for the two processes of link scheduling and power control. According to the state space obtained by the node after partial observation, the node as an agent simultaneously decides whether to send and the specific sending power.

[0043] The multi-agent reinforcement learning algorithm is aimed at evaluating state-action pairs using the Pareto frontier to obtain the Pareto optimal solution of the multi-objective optimization problem.

[0044] Preferably, the above hybrid action space is specifically set as:

[0045] The hybrid action space is described by parameterized action A composed of discrete action link scheduling and continuous action transmit power:

[0046] A={(k,x k )|k∈(k1,k2,...,k n ),x k ∈[0,P max ]}

[0047] In the formula, the value of k is the selected link transmission, and the transmission power x k At the maximum value P max Select any value in .

[0048] Preferably, the above state space is specifically set as:

[0049] The state and observation space contains all environmental feedback information: (1) the remaining energy of the sending node; (2) the location information of the receiving node; (3) the location information of the nodes around the sending node; and (4) the receiving result of the receiving node.

[0050] Preferably, the above multi-agent reinforcement learning algorithm structure is specifically set as follows:

[0051] The network maintained by the agent adopts a hierarchical structure. Each agent needs two layers of Q network to make decisions: the high-level decision-making discrete actions and the low-level processing continuous actions. When the high-level decision is not sent, the low-level decision is skipped by default;

[0052] The optimization objectives information age and fairness mainly depend on the link scheduling status, so the Pareto front evaluation is only introduced into the high-level network.

[0053] Beneficial effects of the present invention:

[0054] The method of the present invention comprehensively considers the underwater visible light channel of path loss, underwater turbulence effect and pointing error, and summarizes a more complete channel model of underwater optical signals; according to the propagation characteristics of underwater optical signals and the performance requirements of multi-hop networks, a comprehensive performance optimization problem of underwater visible light sensor networks with multiple constraints and multiple objectives is proposed from the perspectives of link scheduling and power control.

[0055] Based on the proposed multi-objective optimization problem, starting from the characteristics of distributed link scheduling, link scheduling and power control, a multi-objective multi-agent reinforcement learning algorithm with parameterized action space is adopted to solve the optimization problem. Ultimately, fairness is guaranteed under the requirement of network information freshness, and the network life is maximized. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 Flowchart for joint optimization of link scheduling and power control in underwater visible light sensor networks;

[0057] Figure 2 Structure diagram of the multi-objective multi-agent reinforcement learning algorithm with parameterized actions. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0059] The application principle of the present invention is described in detail below in conjunction with the accompanying drawings.

[0060] The present invention proposes a joint optimization method for link scheduling and power control in an underwater visible light sensor network, taking into account the network performance optimization problem in a multi-hop underwater visible light sensor network. In this method, the underwater visible light channel affected by path loss, underwater turbulence effect and pointing error is considered, and a more accurate model is used to characterize underwater visible light signal transmission. In addition, from the perspective of link scheduling and power adjustment, a multi-constraint multi-objective optimization problem is proposed for network freshness, fairness and energy consumption performance requirements; finally, for the optimization problem, a multi-objective multi-agent algorithm with parameterized actions is proposed to solve the optimization problem, and the network operation is optimized in a distributed scheduling and control manner to improve performance.

[0061] like Figure 1 As shown, the present invention specifically comprises the following steps:

[0062] Step 1: Underwater visible light link channel modeling, specifically including the following steps:

[0063] (1.1) In underwater visible light communication, the light beam emitted by the sender passes through the ocean medium. After considering the ocean turbulence loss, the absorption and scattering loss of the light signal by the seawater, and the geometric loss of the light beam divergence, the light beam reaches the receiver. In addition, the pointing error loss between the center of the light beam and the receiving center is added. If the light signal received by the receiver meets the communication threshold, an end-to-end communication is completed. In actual operation, it is also necessary to avoid interference between different light signals. In optical signals, light intensity is usually used to measure the signal strength. It is defined as the light power per unit area. The light intensity of the received signal can be expressed as:

[0064] I r =I t h+E n

[0065] Where h is the multiplicative channel gain, E n It is the additive noise of underwater environment.

[0066] (1.2) In the above steps, the multiplicative underwater visible light channel gain is divided into three parts: path loss, underwater turbulence effect and pointing error. Among them, the path loss is certain, which is affected by absorption and scattering in the propagation path; while the underwater turbulence effect and pointing error are random and need to be considered from the perspective of random distribution. The channel propagation attenuation model caused by these factors is calculated as:

[0067] h=h l h t h p

[0068] Among them, h l is the determined path loss gain, which is a negative exponential function of the propagation distance L:

[0069] h l =exp(-c(λ)L)

[0070] Wherein, c(λ) is the attenuation coefficient that varies with the wavelength λ of the underwater optical signal, which includes the result of the absorption and scattering of photons in the water medium.

[0071] The present invention uses a mixed exponential generalized gamma distribution to simulate the gain h of the optical signal caused by the underwater turbulence effect. t The mixed exponential generalized gamma distribution is composed of the mixed weighted sum of the exponential distribution and the generalized gamma distribution. It takes into account the bubble and temperature characteristics of the water medium and can accurately characterize the gain of the turbulence in the underwater environment on the optical signal. The probability density function of the distribution is expressed as:

[0072]

[0073] Among them, Г(x) is the gamma function. Other parameters take different values ​​for different water qualities, such as fresh water or sea water, and the different conditions of bubbles and temperature in the corresponding water medium.

[0074] Finally, the pointing error r between the beam center and the receiving center at the receiving end is characterized by Rayleigh distribution. The probability distribution function of the pointing error attenuation can be obtained using the Rayleigh distribution:

[0075]

[0076] Among them, the parameter γ reflects the degree of pointing error, and its value depends on the accuracy of the alignment technology. Because the signal gain caused by underwater turbulence effect and pointing error both act on the signal strength in a multiplicative manner, the random distributions describing the multiplicative channel gain caused by underwater turbulence effect and pointing error are combined to finally obtain the probability density function of the channel gain of the random part in signal propagation:

[0077]

[0078] Where Г(x) is the gamma function, variable h is the channel gain, ω is the mixed weight of the exponential distribution and the generalized gamma distribution, with a value of 0<ω<1, λ is the parameter of the exponential distribution, a and c are the shape parameters of the generalized gamma distribution, and b is the scale parameter of the generalized gamma distribution. By controlling the values ​​of parameters ω, λ, a, b and c, the mixed exponential generalized gamma distribution is changed to adapt to different underwater turbulent environments. γ is the ratio of the equivalent beam radius of the receiver to the standard deviation of the pointing error, which reflects the degree of the pointing error. In order to quantitatively analyze the impact of the joint random channel gain, its cumulative distribution function is further integrated to obtain the following:

[0079]

[0080] (1.3) The present invention considers the signal interference model, including the interference of other underwater optical signals and environmental noise. For a given initial emission light intensity I t , when the signal-to-noise ratio (SINR) at the receiving end is greater than the threshold, the signal is received successfully. The interruption probability of the underwater optical link can be obtained from the cumulative distribution function of step 1.2:

[0081]

[0082] Among them, I in The light intensity of the interference signal is represented, and the channel gain h of the received signal is not completely the same as that of the interference signal. After the above steps, the present invention proposes a unified underwater optical signal channel model.

[0083] Step 2: Modeling of multi-objective optimization problem of underwater visible light sensor network, including the following implementation steps:

[0084] (2.1) Due to the limited coverage of underwater visible light signals, underwater visible light sensor networks usually rely on multi-hop transmission to operate. In view of this network characteristic, in order to achieve efficient information transmission while ensuring the fairness of network resource allocation and improving the life of the network, it is necessary to optimize the information age, energy consumption, and fairness, which are essentially conflicting goals.

[0085] The information age well reflects the timeliness of the node receiving the data. It is measured by the time after the data packet is generated. The system of the present invention works in a time slot mode, so the definition of the information age at a certain node is as follows:

[0086]

[0087] Among them, τ is the timestamp when the data packet is generated, s(t) represents the reception status of the node, and when the node does not receive a new data packet, it increases with the number of time slots.

[0088] The fairness index uses the α-fairness index, which can change the value of α to measure the allocation of network resources under different fairness requirements. Fairness requirements can reduce the over-reliance on a certain path in the network and improve the network life. Because the focus of the present invention is on the use of node channels, the successful transmission rate x of R data packets transmitted in the past T time slots is used. (i) =R / T combined with α-fairness index to calculate fairness:

[0089]

[0090] The energy consumption of a node is directly related to the transmission power, and it is more appropriate to control it within the constraints.

[0091] (2.2) The specific method for setting the constraints of the optimization problem in step 2 is as follows:

[0092] The first is the node remaining energy limit. The transmission should ensure that the node does not die. When the node dies, it directly affects the life of the network:

[0093] E rest -Pη≤E th

[0094] Direct collisions should be prevented at the receiving end, that is, the number of links scheduled to the same destination node should not exceed 1:

[0095]

[0096] Interference between beam signals should be prevented, because the beam has a certain divergence angle. Even if the destination nodes are different, there will be interference with each other. The signal should be sent only when the interruption probability at the receiving end is greater than 0.5. According to the underwater visible light channel model results of step 1, the following constraints are proposed:

[0097] P out (SINR <SINR th )>0.5

[0098] Due to hardware limitations, the system should be half-duplex, that is, nodes cannot send and receive at the same time;

[0099]

[0100] Among the above constraints, one and three are proposed from the perspective of power control, and two and four are proposed from the perspective of link scheduling.

[0101] (2.3)In summary, the optimization problem can be summarized as:

[0102]

[0103] st E rest -Pη≤E th

[0104]

[0105] P out (SINR <SINR th )>0.5

[0106]

[0107] Fairness and information age are goals with opposite optimization directions, so the reciprocal of the fairness index is taken to seek the Pareto optimal solution while satisfying the following four constraints.

[0108] Step 3: Joint optimization of link scheduling and power control based on multi-agent reinforcement learning, including the following implementation steps:

[0109] (3.1) First, the system model, optimization problem and reinforcement learning framework are matched. The intelligent agent of reinforcement learning is the underwater sensor node. The present invention adopts distributed link scheduling, and each node participates in the scheduling decision, thus forming a multi-agent system.

[0110] In addition to selecting to execute discrete actions, each action in the parameterized action space also needs to specify some continuous parameters to determine the specific implementation method of the action. The present invention combines link scheduling and power control, so its action space conforms to the characteristics of parameterized actions, where link scheduling is a discrete value, which plays the role of channel allocation and selects the next hop for transmission in directional transmission, while the transmission power is a continuous value, which reflects the specific communication implementation method.

[0111] In the present invention, the discrete action k indicates that the link sent to the next hop node is scheduled, and its continuous parameter value xk Represents the transmit power of the link when it is scheduled;

[0112] A={(k,x k )|k∈(k1,k2,...,k n ),x k ∈[0,P max ]}

[0113] The state and observation space in a multi-agent system refers to all feedback information in the environment. At the same time, since the agents in a real underwater environment cannot obtain global information, it is a partially observable Markov decision process. The agents can only make local observations based on the limited environment, and the global information can only be estimated and inferred through partial observation information. Specifically, the information in the observation space includes: (i) the remaining energy of the sending node; (ii) the location information of the receiving node; (iii) the location information of the nodes around the sending node; and (iv) the receiving result of the receiving node.

[0114]

[0115] The task in the present invention requires all agents to work closely together, so the same reward is provided to all agents, positive rewards are given according to the target value of the optimization problem, and penalties are given according to the constraints.

[0116] R t =[-R1,-R2]-λζ

[0117] Among them, R1 and R2 correspond to two separate objective functions, and λ indicates whether all constraints in the optimization problem are satisfied. There is no penalty when all constraints are satisfied.

[0118] (3.2) Figure 2 As shown, the multi-agent reinforcement learning framework of the present invention adopts the paradigm of centralized training and distributed execution (CTDE). The specific steps are that when selecting a hybrid action, the agent first selects a discrete action through the high-level policy network, and then determines the corresponding continuous parameters based on the given discrete action and individual observation through the low-level network.

[0119] In each training step, the continuous parameters x associated with the high-level actions k of each agent are first calculated using the current low-level network strategy to improve the exploration efficiency of the underlying network. They are then combined with all the observation information and input into the hypernetwork to generate the weights Q of the hybrid network. tot , in order to coordinate the joint learning of discrete actions among agents.

[0120] For the underlying network part, each agent selects continuous parameters in the policy network based on its new observations and the discrete actions of the high-level network. Its observations actually include both environmental observations and discrete actions:

[0121] o x =(o t ,k)

[0122] After the underlying policy network outputs continuous actions, the Q value of the mixed action is output in the underlying Q network.

[0123] Since each agent needs two layers of networks to process discrete actions and continuous actions respectively, and the selection of actions in Pareto optimization is based on the Q value vector Q determined by multiple objectives set , which results in the agent needing to update two layers of networks and two-dimensional Q value vectors, greatly increasing the amount of calculation. However, considering the two goals of information age and fairness of the present invention, since they are affected by the link scheduling results and have nothing to do with the specific transmission power during the transmission process, it is only necessary to introduce Pareto optimization into the high-level network. Specifically, the agent in the upper-level network selects a point on the Pareto frontier as the target, and calculates the Q value corresponding to each action at each time step. set , find Q that contains the target point set , execute the corresponding action, Q set The update method is as follows:

[0124]

[0125] Where R is the average immediate reward, ND t is the set of non-dominated future rewards, ⊕ is a vector sum operation. There are only two optimization objectives in this paper, so both are 2-dimensional vectors.

[0126] The high-level decision result k of the agent directly affects the low-level decision x, and only one discrete action with optimal continuous parameters needs to be calculated for each agent. It is not necessary to calculate the continuous parameters for each possible discrete action. When the discrete action is chosen not to be sent, the low-level decision can be skipped directly to help save energy.

[0127] In summary:

[0128] The present invention discloses a joint optimization method for link scheduling and power control in an underwater visible light sensor network, which takes into account the network performance optimization problem in a multi-hop underwater visible light sensor network. In this method, the underwater visible light channel affected by path loss, underwater turbulence effect and pointing error is considered, and a more accurate model is used to characterize the underwater visible light signal transmission. In addition, from the perspective of link scheduling and power adjustment, a multi-constraint and multi-objective optimization problem is proposed with respect to network freshness, fairness and energy consumption performance requirements. Finally, with respect to the optimization model, a multi-objective and multi-agent algorithm with parameterized actions is proposed to solve the optimization problem, and the network operation is optimized by distributed sprocket scheduling and power control, thereby improving the comprehensive performance of the underwater visible light sensor network.

[0129] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A joint optimization method for link scheduling and power control in an underwater visible light sensor network, characterized in that The following steps are involved: (1) Underwater visible light link channel modeling The underwater visible light link channel model is established by comprehensively considering the underwater optical signal path loss, underwater turbulence effect and pointing error at the receiving end. The propagation of underwater optical signals must be processed by this channel gain model. The optical signal is described as light intensity, which represents the optical power per unit area. The signal is transmitted from the transmitting node, passes through the multiplicative channel gain and additive environmental noise, and is received at the receiving node. The channel model acts on the signal in the form of channel gain. (2) Modeling of multi-objective optimization problems for underwater visible light sensor networks From the perspective of link scheduling and power control, the information age, fairness and energy consumption performance of underwater visible light sensor networks are comprehensively optimized. The constraint conditions of link scheduling and power adjustment are established using the underwater visible light link channel model, and a multi-constraint and multi-objective optimization problem is proposed. (3) Joint optimization of link scheduling and power control based on multi-agent reinforcement learning Distributed control treats each underwater node as a multi-agent, and seeks the Pareto solution for the multi-objective optimization problem in step (2). Link scheduling and power control are respectively the discrete and continuous actions of the agent, forming a hybrid action space decision. In each time slot, the output is whether the link between each node is scheduled and the specific transmission power, and the Pareto optimal solution is obtained.

2. The method for joint optimization of link scheduling and power control in an underwater visible light sensor network according to claim 1, characterized in that The specific settings of underwater optical signal path loss, underwater turbulence effect and receiving point pointing error in step (1) are: The path loss is described by a negative exponential function of the propagation distance, and the attenuation amplitude is affected by absorption and scattering factors that are related to the wavelength; The turbulence effect is described by a mixed exponential generalized gamma distribution. The pointing error starts from the radial displacement of the receiving beam center and the receiving point center, which satisfies the Rayleigh distribution. Since the two are independent and both act on the signal in the form of multiplicative channel gain, the two random distributions are combined, and their probability density function is described as follows: Where Г(x) is the gamma function, variable h is the channel gain, ω is the mixed weight of the exponential distribution and the generalized gamma distribution, with a value of 0<ω<1, λ is the parameter of the exponential distribution, a and c are the shape parameters of the generalized gamma distribution, and b is the scale parameter of the generalized gamma distribution. By controlling the values ​​of parameters ω, λ, a, b and c, the mixed exponential generalized gamma distribution is changed to adapt to different underwater turbulent environments. γ is the ratio of the equivalent beam radius of the receiver to the standard deviation of the pointing error, which reflects the degree of the pointing error. In order to quantitatively analyze the impact of the joint random channel gain, its cumulative distribution function is further integrated to obtain the following: Where Г(s,x) is the incomplete gamma function, and the cumulative distribution function is used to quantitatively calculate the channel outage probability.

3. The method for joint optimization of link scheduling and power control in an underwater visible light sensor network according to claim 1, characterized in that The multiple objectives in step (2) are specifically set as follows: The information age is described by the time from data generation to dissemination; The fairness is described by the allocation status of channel transmission resources, and the average number of channel transmission data packets is combined with the alpha fairness index, which balances network efficiency and fairness among users by adjusting parameter α. The purpose of the optimization is to improve the data freshness of the underwater visible light sensor network through link scheduling, while ensuring fair allocation of resources for each channel, and on this basis optimize the transmission power of specific transmission to avoid energy waste and increase network life.

4. The method for joint optimization of link scheduling and power control in an underwater visible light sensor network according to claim 1, characterized in that The multiple constraints in step (2) are specifically set as four situations: (i.) The node remaining energy limit requires that the energy consumption of the signal to be transmitted by the node in the channel must not cause the node to die, and puts forward requirements on the transmission power; (ii.) Transmission collision restriction requires that there should be no collision caused by many-to-one transmission in multi-hop transmission; (iii.) There may be interference at the receiving end of the signal, and the signal needs to meet the interruption probability requirement after the channel gain; (iv.) In a full-duplex system, a node cannot transmit and receive simultaneously.

5. The method for joint optimization of link scheduling and power control in an underwater visible light sensor network according to claim 4, characterized in that The multi-constraint multi-objective optimization problem in step (2) is specifically set as: s.t E rest -Pη≤E th P out (SINR <SINR th )>0.5 Where N is the number of network nodes, f α (x (i) is the alpha fairness index, whose variable x (i) is the average number of data packets transmitted, A is the information age, and the constraints correspond to the four cases described in claim 4: (i) E rest is the node residual energy, P is the transmission power, η is the energy conversion efficiency, E th is the minimum threshold of node energy; in (ii), link(n a ,n b ) represents node n a to n b Link scheduling status, N b-1 is node n b The previous hop node set; in (iii), P out (x) is the outage probability, SINR is the interference signal-to-noise ratio; in (iv), N b+1 is node n b The set of next hop nodes.

6. The method for joint optimization of link scheduling and power control in an underwater visible light sensor network according to claim 1, characterized in that The specific setting of the multi-agent reinforcement learning algorithm in step (3) is: The link scheduling and power control in the underwater visible light sensor network are regarded as partially observable Markov processes. The multi-agent reinforcement learning algorithm adopts a paradigm of centralized training and distributed execution. The mixed action space of the agent is designed for the two processes of link scheduling and power control. According to the state space obtained by the node after partial observation, the node as an agent simultaneously decides whether to send and the specific sending power. The multi-agent reinforcement learning algorithm is aimed at evaluating state-action pairs using the Pareto frontier to obtain the Pareto optimal solution of the multi-objective optimization problem.

7. The method for joint optimization of link scheduling and power control in an underwater visible light sensor network according to claim 6, characterized in that The specific settings of the hybrid action space are: The hybrid action space is described by parameterized action A composed of discrete action link scheduling and continuous action transmit power: A={(k,x k )|k∈(k1,k2,...,k n ),x k ∈[0,P max ]} In the formula, the value of k is the selected link transmission, and the transmission power x k At the maximum value P max Select any value in .

8. The method for joint optimization of link scheduling and power control in an underwater visible light sensor network according to claim 6, characterized in that The state space is specifically set as: The state and observation space contains all environmental feedback information: (1) the remaining energy of the sending node; (2) the location information of the receiving node; (3) the location information of the nodes around the sending node; and (4) the receiving result of the receiving node.

9. The method for joint optimization of link scheduling and power control in an underwater visible light sensor network according to claim 6, characterized in that The specific structure of the multi-agent reinforcement learning algorithm is: The network maintained by the agent adopts a hierarchical structure. Each agent needs two layers of Q network to make decisions: the high-level decision-making discrete actions and the low-level processing continuous actions. When the high-level decision is not sent, the low-level decision is skipped by default; The optimization objectives information age and fairness mainly depend on the link scheduling status, so the Pareto front evaluation is only introduced into the high-level network.

Images