Underwater acoustic network adaptive time slot adjustment method and system based on cross-layer joint optimization

Abstract

The invention belongs to the technical field of communication, and relates to an underwater acoustic network adaptive time slot adjustment method and system based on cross-layer joint optimization. The method comprises the following steps: firstly, obtaining a physical layer channel state, a link layer queue length and an application layer information age in a cross-layer manner, and constructing a system state space; and then, establishing an optimization model based on a semi-Markov decision process, calculating an average reward by defining an information value function and combining throughput, meanwhile, constraining energy consumption and queue length under a constrained Markov decision framework, and solving an optimal time slot length by using a Bellman equation. And finally, applying the optimal time slot according to a solving result. According to the invention, through cross-layer information fusion and constraint optimization, challenges of high time delay, time-varying channel and resource limitation of the underwater acoustic network are effectively dealt with, and the network performance is improved.

Description

Technical Field

[0001] This invention belongs to the field of communication technology and relates to an adaptive time slot adjustment method and system for underwater acoustic networks based on cross-layer joint optimization. Background Technology

[0002] Underwater communication networks have important applications in tasks such as marine environmental monitoring, resource exploration, disaster early warning, and military communications. Due to the low propagation speed, limited bandwidth, and strong time-varying nature of underwater acoustic channels, how to efficiently utilize limited underwater acoustic resources has always been a key research focus. Currently, in multi-node cooperative underwater wireless sensor networks, commonly used medium access control methods mainly include the CSMA protocol, the TDMA protocol, and the CTS / RTS handshake protocol. Among these, the TDMA protocol, compared to the other two, can avoid data collisions and has lower power consumption, making it more widely used in underwater networking.

[0003] Traditional TDMA protocols typically allocate channel resources using fixed time slot lengths, but this approach has significant shortcomings in real-world underwater communication environments. Too short a time slot leads to frequent node wake-ups, significantly increasing energy consumption, high control and synchronization overhead, and reduced effective data transmission rate; too long a time slot increases end-to-end latency, causes untimely data updates, and results in excessively long retransmission times and reduced reliability when transmission fails.

[0004] To address these issues, Sergiu et al. proposed a scheme for dynamically adjusting the time slot length based on changes in local traffic load. Zhang et al. proposed a scheme for dynamically adjusting the time slot length based on node queue conditions. Existing improvement methods mostly focus on optimizing single metrics, failing to consider the complexity of the underwater environment. Improving a single metric may lead to a decrease in the performance of other metrics, and the rewards are instantaneous, failing to account for changes in rewards over time slots, resulting in limited overall network performance improvement. Summary of the Invention

[0005] The purpose of this invention is to propose an adaptive time slot adjustment method for underwater acoustic networks based on cross-layer joint optimization, in order to overcome the shortcomings of existing technologies.

[0006] To achieve the above-mentioned objectives, the present invention employs the following technical solution: an adaptive time slot adjustment method for underwater acoustic networks based on cross-layer joint optimization, comprising the following steps: S1: Obtain physical layer channel state information across layers, and construct the system state space by combining the queue length of the data link layer and the information age of the application layer; at the beginning of each frame, by observing the queue length, information age, and channel signal-to-noise ratio, the current network state is input into the decision model for analysis and processing according to the signal quality and delay requirements of each time slot. S2: An optimization model is established based on a semi-Markov decision framework. An information value function is constructed based on the time delay. The average reward within the time slot length is calculated by combining the throughput. Based on a constrained Markov decision framework, constraints are imposed on energy consumption and waiting queues. The optimal time slot length in the preset candidate time slot set is obtained using the Bellman equation. S3: Based on the optimization results, output and apply the optimal timeslot length for this frame for scheduling and transmission within the frame.

[0007] Furthermore, the information value function constructed in step S2 is: ; in, For the delay size, This is the value decay coefficient; By using the value information function and taking time delay as a moderating factor, the reward function obtained is: ; in, In the channel state and the selected action is At that time, the predicted value that can successfully transmit data packets; The selected action is The length of the time slot; It is the number of bits contained in a data packet.

[0008] Furthermore, a constrained Markov decision process is used to constrain the queue length and energy consumption, and the candidate time slot set is screened. Under the premise of satisfying all cost constraints, the long-term average tracking reward is maximized to obtain the optimal strategy, that is, the discrete time slot length that is closest to the theoretical optimal value.

[0009] Furthermore, the queue length and energy consumption constraints specifically include: First, the instantaneous energy cost at each moment is defined as equal to the energy consumed in that time slot: ; for Total energy consumption during this period; Define the instantaneous queue cost as an indicator function to provide an indication when a risk occurs: ; in, This is an indicator function; it is 1 when the condition is true and 0 otherwise. The length of the queue to transition to the next state after an action is taken; The set queue length warning threshold; The long-term average queue overflow risk constraint is: ; in, A strategy is a mapping rule from a state to an action; The probability of the maximum acceptable queue overflow; This represents the expected value of energy consumed within this time slot. The long-run average energy consumption constraint is: ; in, This represents the maximum acceptable average energy consumption value.

[0010] Furthermore, Lagrange multiplier vectors are used. The instantaneous Lagrange reward rate is calculated as follows: ; in, These are the penalty coefficients for queue length overflow and energy consumption, respectively. The long-term average reward is: ; The long-term average Lagrange reward is: .

[0011] Furthermore, for fixed Optimal strategy The value of is defined by the Bellman equation, the solution of which is the optimal long-run average Lagrange reward. The Bellman equation is: ; For value assessment function; The optimal strategy is: .

[0012] Furthermore, step S3 also includes: recording the actual performance of the network in the previous time slot, using this indicator to calculate the optimal length to be selected in the next time slot, and calculating the real reward and cost of this decision, constructing an experience tuple, storing it in the experience replay pool, and iteratively training and correcting the model.

[0013] The present invention also provides an adaptive time slot adjustment system for underwater acoustic networks based on cross-layer joint optimization. The system includes a computer-readable storage medium and a computer program stored in the computer-readable storage medium. When the computer program is read, it executes the steps of the above method.

[0014] Compared with existing technologies, this invention has the following advantages: By cleverly combining the information age of the application layer, the queue state of the data link layer, and the channel quality of the physical layer, and using a constrained semi-Markov decision process (C-SMDP) for modeling and solving, this invention provides a systematic solution to the inherent challenges of underwater acoustic communication. The core advantage of this invention lies in shifting from a "passive adaptation" environment to "active prediction and collaborative optimization," effectively addressing the core challenges of high latency, time-varying channels, and resource constraints in underwater acoustic networks. Attached Figure Description

[0015] Figure 1 This is a schematic diagram illustrating an implementation scenario of the present invention; Figure 2 The reinforcement learning process diagram of this invention; Figure 3 This is a flowchart of the process for determining the time slot length according to the present invention; Figure 4 This is a time slot structure diagram of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described more clearly and completely below through a preferred embodiment and in conjunction with the accompanying drawings. It should be understood that the embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0017] refer to Figure 1-4 The present invention provides an adaptive time slot adjustment method for underwater acoustic networks based on cross-layer joint optimization, comprising the following steps: S1: Obtain physical layer channel state information across layers, and construct the system state space by combining the queue length of the data link layer and the information age of the application layer. Information age refers to the time elapsed from the generation or update of information to its use, reflecting the freshness and timeliness of a piece of information; the signal-to-noise ratio is chosen to represent the channel state. At the beginning of each frame, the current state of nodes such as queue length and channel quality is observed. Based on the signal quality and delay requirements of each time slot, the collected node information is used as the model's dataset. The number of elements and their values ​​in the selected time slot set are determined according to the scale of the underwater sensor network and the number of underwater sensor nodes. The data is then processed and input into the model. Specifically, this includes: S1-1: Constructing the system state space: At the beginning of each frame period, key information of the nodes in the underwater wireless sensor network is collected, including parameters such as queue length, information age, and channel signal-to-noise ratio. The system state vector defined accordingly is: ; in, This indicates the queue length of each node, which is the number of data packets in the node's sequence. Indicates the information age of each node; It represents the signal-to-noise ratio of each node and is used to describe the current channel state; This indicates the number of nodes in the network.

[0018] By obtaining the physical layer channel state across layers, we can effectively avoid invalid transmissions caused by allocating long time slots for communication when the channel state is poor. We can allocate longer time slots when the channel quality is good and allocate shorter time slots or keep the channel silent when the channel quality is poor, thereby avoiding waste of resources.

[0019] The action involves selecting the length of the next time slot. In this model, the output is a continuous value. However, considering that random values ​​in the TDMA protocol can lead to inconsistent synchronization times, requiring frequent switching of synchronization times, which consumes a lot of system computing resources and incurs high overhead, the time slot length is limited to a certain set, discretized into a finite set. Based on the solved optimal time slot length, the closest value in the set is selected as the length of the next time slot. The action space is: ; in, It is the length of the next selectable time slot. It represents the number of selectable time slot lengths in the set, and the specific value can be set according to different environments.

[0020] action When selecting an action And stay After (time slot length), the system transitions from state... Transition to state The state transition probability is: ; in These represent the current system state and the system state for the next frame, respectively.

[0021] S1-2: State Normalization and Input Processing To avoid the unbalanced influence of states with different dimensions on the model, the above state parameters are normalized: ; in, These are the minimum and maximum values ​​of the variable within the statistical interval, respectively.

[0022] Normalized state vector As input to subsequent decision-making models, it helps improve the stability of modeling and computation.

[0023] S1-3: Initialization of the decision model: The normalized state vector Input it into the decision model.

[0024] S2: An optimization model is established based on a semi-Markov decision framework. Delay is used as a adjustment factor and set as the information value function. The reward value is calculated jointly based on the throughput predicted from the physical channel state. Considering engineering implementation limitations, the actual time slots are selected from a pre-set finite candidate set. A constrained Markov decision framework is introduced to establish constraints on queue length and energy consumption. The candidate set is then filtered to maximize the long-term average tracking reward while satisfying all cost constraints, thus obtaining the optimal strategy, i.e., the discrete time slot length closest to the theoretical optimum.

[0025] A reward function is constructed based on throughput and latency, and the average reward within the time slot length is calculated. Simultaneously, to ensure system performance stability, CMDP (Constraint-Based Dynamics) is used for constraints, considering energy consumption and queue length constraints. Finally, the optimal policy is solved using the Bellman equation to obtain the optimal time slot. Details are as follows: S2-1: Reward Function Design In SMDP, the reward function is the state. Take action Then, the accumulated reward between the next decision and the time interval is divided by the duration. In this system, the objective is to maximize network throughput, minimize latency, and ensure that energy consumption is not significantly increased.

[0026] To jointly optimize throughput and latency, considering the time-sensitivity of information (as latency increases, information becomes outdated and its value decreases significantly), and given that latency has a smaller impact on rewards when it is low, maximizing throughput is more important. Conversely, when latency is high, minimizing latency is crucial to complete data transmission. Therefore, the information value function is defined as follows: ; in, For the delay size, This is the value decay coefficient.

[0027] By using the value information function and taking time delay as a moderating factor, the reward function obtained is: ; in, In the channel state and the selected action is At that time, the predicted value that can successfully transmit data packets; The selected action is The length of the time slot; It is the number of bits contained in a data packet.

[0028] S2-2: Constraint Model While considering optimization of throughput and latency, a constrained Markov decision process is used to constrain queue length and energy consumption, so as to avoid significantly increasing sensor energy consumption, increasing the length of queues waiting to send messages, or even causing queue overflow while improving throughput and reducing latency.

[0029] To measure long-term costs, we first define the instantaneous cost at each moment. The instantaneous energy cost is equal to the energy consumed in that time slot: ; for Total energy consumption during this period.

[0030] Instantaneous queue cost is an indicative signal defined in this invention, providing an indication when a risk occurs: ; in, This is an indicator function; it is 1 when the condition is true and 0 otherwise. The length of the queue to transition to the next state after an action is taken; The set queue length warning threshold.

[0031] The constraints are for long-term behavior, therefore it is necessary to convert instantaneous costs into long-term average costs, resulting in long-term average queue overflow risk constraints: ; in, A strategy is a mapping rule from a state to an action; The probability of the maximum acceptable queue overflow; This represents the expected energy consumed within this time slot.

[0032] Long-term average energy consumption constraint: ; in, This represents the maximum acceptable average energy consumption value.

[0033] To transform the constrained problem into an unconstrained problem, Lagrange multiplier vectors are introduced. The instantaneous Lagrange reward rate is: ; in, These are the penalty coefficients for queue length overflow and energy consumption, respectively.

[0034] The long-term average reward is: ; The long-term average Lagrange reward is: .

[0035] For fixed Optimal strategy The value of is defined by the Bellman equation, and the solution is the optimal long-run average Lagrange reward. The Bellman equation is: ; For the value assessment function, it represents the penalty coefficient. and state The next score is used to evaluate the state. Starting with the optimal strategy, we can expect long-term returns over an infinitely long period of time.

[0036] The optimal strategy is: .

[0037] According to this strategy, in the state... Under these conditions, the action that maximizes the average reward can be obtained, thus achieving the optimal time slot length.

[0038] S3: Based on the results of constraint optimization, output and apply the optimal timeslot length for this frame for intra-frame scheduling and transmission. After the frame ends, record the actual throughput, latency, and other metrics, and use these as feedback input to the state modeling stage of the next frame, thereby achieving online iteration and adaptive adjustment to continuously optimize system performance. Specifically, this includes: S3-1: Decision Outputs and Applications: Based on the approximate optimal strategy learned from model training, the optimal time slot length is obtained, and this time slot length is set as the next time slot length.

[0039] S3-2: Performance Feedback and Iterative Adjustment Record the network's actual performance in the previous time slot, use this metric to calculate the optimal length to be selected in the next time slot, and calculate the real reward and cost of this decision. Construct an experience tuple and store it in the experience replay pool to iteratively train and correct the model.

Claims

1. An adaptive time slot adjustment method for underwater acoustic networks based on cross-layer joint optimization, characterized in that, Includes the following steps: S1: Obtain physical layer channel state information across layers, and construct the system state space by combining the queue length of the data link layer and the information age of the application layer; at the beginning of each frame, by observing the queue length, information age, and channel signal-to-noise ratio, the current network state is input into the decision model for analysis and processing according to the signal quality and delay requirements of each time slot. S2: An optimization model is established based on a semi-Markov decision framework. An information value function is constructed based on the time delay. The average reward within the time slot length is calculated by combining the throughput. Based on a constrained Markov decision framework, constraints are imposed on energy consumption and waiting queues. The optimal time slot length in the preset candidate time slot set is obtained using the Bellman equation. S3: Based on the optimization results, output and apply the optimal timeslot length for this frame for scheduling and transmission within the frame.

2. The adaptive time slot adjustment method for underwater acoustic networks based on cross-layer joint optimization according to claim 1, characterized in that, The information value function constructed in step S2 is: ； in, For the delay size, This is the value decay coefficient; By using the value information function and taking time delay as a moderating factor, the reward function obtained is: ； in, In the channel state and the selected action is At that time, the predicted value that can successfully transmit data packets; The selected action is The length of the time slot; It is the number of bits contained in a data packet.

3. The adaptive time slot adjustment method for underwater acoustic networks based on cross-layer joint optimization according to claim 1, characterized in that, By using a constrained Markov decision process to constrain queue length and energy consumption, and filtering the candidate time slot set, the optimal strategy is obtained by maximizing the long-term average tracking reward while satisfying all cost constraints. This strategy is the discrete time slot length that is closest to the theoretical optimal value.

4. The adaptive time slot adjustment method for underwater acoustic networks based on cross-layer joint optimization according to claim 3, characterized in that, The queue length and energy consumption constraints specifically include: First, the instantaneous energy cost at each moment is defined as equal to the energy consumed in that time slot: ； for Total energy consumption during this period; Define the instantaneous queue cost as an indicator function to provide an indication when a risk occurs: ； in, This is an indicator function; it is 1 when the condition is true and 0 otherwise. The length of the queue to transition to the next state after an action is taken; The set queue length warning threshold; The long-term average queue overflow risk constraint is: ； in, A strategy is a mapping rule from a state to an action; The probability of the maximum acceptable queue overflow; The long-run average energy consumption constraint is: ； in, This represents the maximum acceptable average energy consumption value.

5. The adaptive time slot adjustment method for underwater acoustic networks based on cross-layer joint optimization according to claim 4, characterized in that, Using Lagrange multiplier vectors The instantaneous Lagrange reward rate is calculated as follows: ； in, These are the penalty coefficients for queue length overflow and energy consumption, respectively. The long-term average reward is: ； The long-term average Lagrange reward is: 。 6. The adaptive time slot adjustment method for underwater acoustic networks based on cross-layer joint optimization according to claim 5, characterized in that, For fixed Optimal strategy The value of is defined by the Bellman equation, the solution of which is the optimal long-run average Lagrange reward. The Bellman equation is: ； For value assessment function; The optimal strategy is: 。 7. The adaptive time slot adjustment method for underwater acoustic networks based on cross-layer joint optimization according to any one of claims 1-6, characterized in that, Step S3 further includes: recording the actual performance of the network in the previous time slot, using this indicator to calculate the optimal length to be selected in the next time slot, and calculating the real reward and cost of this decision, constructing an experience tuple, storing it in the experience replay pool, and iteratively training and correcting the model.

8. An adaptive time-slot adjustment system for underwater acoustic networks based on cross-layer joint optimization, characterized in that, The system includes a computer-readable storage medium and a computer program stored in the computer-readable storage medium, which, when read, performs the steps of the method as described in any one of claims 1-6.

Images