Data transmission method, device and equipment of underwater wireless sensor network and medium

Abstract

The invention discloses a data transmission method and device of an underwater wireless sensor network, equipment and a medium, and relates to the technical field of data transmission, an MAC protocol and protocol basic information are configured, a node state calculation formula, a node action calculation formula and a reward function are defined, and a decision network is added for a time slot model; calculating an initial node state and an initial node action, and if the initial node action is to send data, performing data communication with other sending nodes; after acknowledgement characters of other sending nodes are obtained, a reward value is calculated, and the state of the initial node is updated; updating the decision network by using the initial node state, the initial node action, the reward value and the updated node state; and inputting the current node data into the updated decision network, outputting a node data transmission strategy, and carrying out data transmission between the receiving nodes in the local and underwater wireless sensor networks according to the node data transmission strategy, thereby maximizing the network throughput and improving the data transmission efficiency of the underwater wireless sensor networks.

Description

Technical Field

[0001] The present invention relates to the technical field of data transmission, and in particular to a data transmission method, device, equipment and medium of an underwater wireless sensor network. Background Art

[0002] There are significant differences between the communication environment of underwater acoustic sensor networks and terrestrial sensor networks. The underwater acoustic channel has the characteristics of high latency, low bandwidth, high bit error rate, multipath effect and Doppler frequency shift; secondly, underwater nodes are susceptible to passive movement due to water flow or underwater organisms; in addition, underwater nodes are powered by batteries, and battery replacement and maintenance are costly and difficult to implement. The above characteristics make terrestrial wireless sensor network protocols not directly applicable to underwater, and new protocols that adapt to the characteristics of underwater networks need to be studied. The underwater sensor network protocol stack can be divided into the physical layer, data link layer, network layer, transport layer and application layer. Among them, the data link layer mainly studies and solves the problem of medium access control, namely the MAC (Medium Access Control) protocol. The long propagation delay characteristics underwater bring challenges to the research of underwater MAC protocols, increase the complexity of channel competition and the probability of conflict, and lead to data packet collisions. Traditional underwater MAC protocols can generally be divided into two types: contention-based and fixed allocation-based. The need for handshake information exchange or random time retransmission leads to additional communication overhead, and the fixed strategy makes it highly dependent on the environment and lacks adaptability.

[0003] From the above, it can be seen that how to comprehensively consider the completeness of node observation and the overall energy consumption of the underwater network, maximize network throughput and save energy of underwater sensor nodes, solve the conflicts caused by the long propagation delay characteristics underwater, and improve the efficiency of data transmission in underwater wireless sensor networks are problems to be solved in this field. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a data transmission method, device, equipment and medium for an underwater wireless sensor network, which can fully consider the completeness of node observation and the overall energy consumption of the underwater network, maximize network throughput and save energy of underwater sensor nodes, solve the conflict caused by the long propagation delay characteristics underwater, and improve the efficiency of data transmission in underwater wireless sensor networks. The specific scheme is as follows:

[0005] In a first aspect, the present application discloses a data transmission method for an underwater wireless sensor network, which is applied to any sending node in a distributed underwater wireless sensor network, comprising:

[0006] Configure the MAC protocol and basic information of the protocol, build a time slot model, define a node state calculation formula, a node action calculation formula and a reward function based on the MAC protocol and the basic information of the protocol, and add a decision network to the time slot model;

[0007] Acquire initial node data, calculate the initial node data using the node state calculation formula and the node action calculation formula to obtain an initial node state and an initial node action, and if the initial node action is to send data, perform data communication with other sending nodes except the local node;

[0008] After obtaining the confirmation character fed back by the other sending nodes, the reward value is calculated based on the confirmation character and using the reward function, and the initial node state is updated according to the reward value to obtain an updated node state;

[0009] Updating the decision network in the time slot model using the initial node state, the initial node action, the reward value, and the updated node state to obtain the updated decision network;

[0010] The current node data is obtained and input into the updated decision network to output a node data transmission strategy, and data transmission between local and distributed receiving nodes in the underwater wireless sensor network is performed according to the node data transmission strategy.

[0011] Optionally, the constructing of the time slot model includes:

[0012] Build the initial time slot model and determine the task objectives based on business needs;

[0013] The local data transmission work is used as a training parameter, and the initial time slot model is trained in combination with the task goal to obtain the time slot model.

[0014] Optionally, the node status calculation formula is:

[0015] ;

[0016] ;

[0017] in, The action of node j in time slot t observed by node i in the underwater sensor, is the action selected by node j at time slot t, AP is the receiving node, Whether the AP observes node i to return a confirmation character to node j in time slot t, is the action chosen by node j at time slot t.

[0018] Optionally, the node action calculation formula is:

[0019] ;

[0020] in, is the action selected by node i at time slot t, p is the transmit power level, p={1,2,…,p-1,p}.

[0021] Optionally, the reward function is:

[0022] ;

[0023] in, is the reward value obtained by node i in time slot t, p is the transmission power level, p={1,2,…,p-1,p}, For reward and penalty items.

[0024] Optionally, adding a decision network to the time slot model includes:

[0025] A MAAC algorithm based on a policy-value architecture is added to the time slot model; the MAAC algorithm includes a decision network; the decision network includes a policy network and a value network.

[0026] Optionally, the updating the decision network in the time slot model by using the initial node state, the initial node action, the reward value and the updated node state includes:

[0027] Generate update information based on the initial node state, the initial node action, the reward value, and the updated node state, and store the update information in a local buffer;

[0028] The decision network in the time slot model is updated using the update information corresponding to each time step in the buffer.

[0029] In a second aspect, the present application discloses a data transmission device for an underwater wireless sensor network, which is applied to any sending node in a distributed underwater wireless sensor network, comprising:

[0030] A construction and definition module is used to configure the MAC protocol and basic information of the protocol, build a time slot model, define a node state calculation formula, a node action calculation formula and a reward function based on the MAC protocol and the basic information of the protocol, and add a decision network to the time slot model;

[0031] A calculation module, used to obtain initial node data, calculate the initial node data using the node state calculation formula and the node action calculation formula to obtain the initial node state and the initial node action, and if the initial node action is to send data, perform data communication with other sending nodes except the local node;

[0032] A node state updating module, configured to, after obtaining the confirmation character fed back by the other sending nodes, calculate a reward value based on the confirmation character and using the reward function, and update the initial node state according to the reward value to obtain an updated node state;

[0033] A decision network updating module, used to update the decision network in the time slot model using the initial node state, the initial node action, the reward value and the updated node state to obtain the updated decision network;

[0034] The data transmission module is used to obtain current node data and input the current node data into the updated decision network to output a node data transmission strategy, and perform data transmission between local and distributed receiving nodes in the underwater wireless sensor network according to the node data transmission strategy.

[0035] In a third aspect, the present application discloses an electronic device, comprising:

[0036] Memory, used to store computer programs;

[0037] The processor is used to execute the computer program to implement the aforementioned data transmission method for the underwater wireless sensor network.

[0038] In a fourth aspect, the present application discloses a computer storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the steps of the aforementioned disclosed method for transmitting data in an underwater wireless sensor network are implemented.

[0039] It can be seen that the present application provides a data transmission method for an underwater wireless sensor network, including configuring a MAC protocol and basic protocol information, constructing a time slot model, defining a node state calculation formula, a node action calculation formula and a reward function based on the MAC protocol and the basic protocol information, and adding a decision network to the time slot model; obtaining initial node data, calculating the initial node data using the node state calculation formula and the node action calculation formula to obtain an initial node state and an initial node action, and if the initial node action is to send data, performing data communication with other sending nodes other than the local node; when feedback from the other sending nodes is obtained After receiving the confirmation character, the reward value is calculated based on the confirmation character and using the reward function, and the initial node state is updated according to the reward value to obtain the updated node state; the decision network in the time slot model is updated using the initial node state, the initial node action, the reward value and the updated node state to obtain the updated decision network; the current node data is obtained, and the current node data is input into the updated decision network to output a node data transmission strategy, and data transmission between the local and distributed receiving nodes in the underwater wireless sensor network is performed according to the node data transmission strategy. The present application is applied to any sending node in a distributed underwater wireless sensor network, configures the MAC protocol and basic protocol information, builds a time slot model, and does not need to cooperate with the traditional MAC protocol to maximize network throughput. It defines the node state calculation formula, node action calculation formula and reward function, and adds a decision network to the time slot model. It can be applied to various network topologies, takes into account the completeness of node observation and the overall energy consumption of the underwater network, and more effectively maximizes network throughput and saves energy of underwater sensor nodes. It calculates the initial node state and initial node action. If the initial node action is to send data, it communicates with other sending nodes except the local node for data transmission. According to communication, when the confirmation character fed back by other sending nodes is obtained, the reward value is calculated, and the initial node state is updated, so as to update the decision network in the time slot model to obtain an updated decision network, and the current node data is input into the updated decision network to output the node data transmission strategy, and the data transmission between the receiving nodes in the local and distributed underwater wireless sensor network is carried out according to the node data transmission strategy, taking into account the conflict problem caused by the long underwater propagation delay characteristics, and solving the conflict caused by the long underwater propagation delay characteristics. At the same time, there is no need to cooperate with the traditional MAC protocol, and it has high adaptability, which improves the efficiency of data transmission in the underwater wireless sensor network. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0041] Figure 1 A flow chart of a data transmission method for an underwater wireless sensor network disclosed in this application;

[0042] Figure 2 A diagram showing the time slot length of a data packet disclosed in this application;

[0043] Figure 3 A flowchart for obtaining an initial node data set and calculating updates disclosed in this application;

[0044] Figure 4 A data transmission flow chart of a distributed semi-cooperative underwater acoustic network MAC protocol based on deep reinforcement learning disclosed in this application;

[0045] Figure 5 This is a schematic diagram of the structure of a data transmission device for an underwater wireless sensor network disclosed in this application;

[0046] Figure 6 A structural diagram of an electronic device provided for this application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] There are significant differences between the communication environment of underwater acoustic sensor networks and terrestrial sensor networks. The underwater acoustic channel has the characteristics of high latency, low bandwidth, high bit error rate, multipath effect and Doppler frequency shift; secondly, underwater nodes are susceptible to passive movement due to water flow or underwater organisms; in addition, underwater nodes are powered by batteries, and battery replacement and maintenance are costly and difficult to implement. The above characteristics make it impossible to directly apply terrestrial wireless sensor network protocols to underwater, and it is necessary to study new protocols that adapt to the characteristics of underwater networks. The underwater sensor network protocol stack can be divided into the physical layer, data link layer, network layer, transport layer and application layer. Among them, the data link layer mainly studies and solves the problem of medium access control, namely MAC protocol. The long propagation delay characteristics underwater bring challenges to the research of underwater MAC protocol, increase the complexity of channel competition and the probability of conflict, and lead to data packet collision. Traditional underwater MAC protocols can generally be divided into two types: competition-based and fixed allocation-based. The need for handshake information exchange or random time retransmission leads to additional communication overhead, and the fixed strategy makes it highly dependent on the environment and lacks adaptability. From the above, it can be seen that how to comprehensively consider the completeness of node observation and the overall energy consumption of the underwater network, maximize network throughput and save energy of underwater sensor nodes, solve the conflicts caused by the long propagation delay characteristics underwater, and improve the efficiency of data transmission in underwater wireless sensor networks are problems to be solved in this field.

[0049] See also Figure 1 As shown, the embodiment of the present invention discloses a data transmission method of an underwater wireless sensor network, which is applied to any sending node in a distributed underwater wireless sensor network and may specifically include:

[0050] Step S11: configure the MAC protocol and basic information of the protocol, build a time slot model, define a node state calculation formula, a node action calculation formula and a reward function based on the MAC protocol and the basic information of the protocol, and add a decision network to the time slot model.

[0051] In this embodiment, the MAC protocol and basic protocol information are configured, an initial time slot model is constructed, and a task objective is determined according to business needs; local data transmission work is used as a training parameter, and the initial time slot model is trained in combination with the task objective to obtain the time slot model, a node state calculation formula, a node action calculation formula and a reward function are defined based on the MAC protocol and the basic protocol information, and a MAAC algorithm based on a strategy-value architecture is added to the time slot model; the MAAC algorithm includes a decision network; the decision network includes a policy network and a value network.

[0052] Specifically, the present application is applied to any sending node in a distributed underwater wireless sensor network, and a MAC protocol is configured in the sending node. For all sending nodes, the sending nodes in N underwater wireless sensor networks are configured with the same MAC protocol, and data packets are transmitted to the AP central receiving node in a time slot manner, and the basic protocol information is configured. Then, an initial time slot model is constructed. The task goal is to train a conflict-free data transmission strategy based on a conventional time slot model to maximize the throughput of the underwater wireless sensor network. Based on this goal, the data transmission power of the underwater wireless sensor node is used as a training parameter of the model to achieve the management and allocation of energy for the underwater sensor nodes. In the time slot model, it is assumed that the data packets of any underwater wireless sensor data sending node have the same packet length. , and the ACK (Acknowledge character) packet from the AP center receiving node to each data sending node also has the same packet length , the packet time slot length is Figure 2 As shown, the length of each time slot Corresponds to the duration of the node sending the data packet .

[0053] The MAC protocol takes into account the long propagation delay characteristics underwater. The central receiving node AP can only receive one data packet at a time. If two or more data packets sent from different underwater wireless sensor nodes arrive at the AP at the same time, a data reception conflict occurs at the receiving node. The data transmission and reception processes of each underwater wireless sensor node are mutually exclusive, that is, if a data packet from another node or a feedback ACK from a receiving node arrives at the current node at the same time and the current node is sending data, a receive-transmit conflict occurs. Each underwater wireless sensor node, like the receiving node, can only receive one data packet at a time.

[0054] In this embodiment, the node status calculation formula is:

[0055] ;

[0056] ;

[0057] in, The action of node j in time slot t observed by node i in the underwater sensor, is the action selected by node j at time slot t, AP is the receiving node, Whether the AP observes node i to return a confirmation character to node j in time slot t, is the action selected by node j at time slot t;

[0058] The node action calculation formula is:

[0059] ;

[0060] in, is the action selected by node i at time slot t, p is the transmission power level, p={1,2,…,p-1,p};

[0061] The reward function is:

[0062] ;

[0063] in, is the reward value obtained by node i in time slot t, p is the transmission power level, p={1,2,…,p-1,p}, For reward and penalty items.

[0064] The specific principles for defining the node state calculation formula, node action calculation formula, and reward function are as follows: (1) Define the node state calculation formula, where the state s can be regarded as an observation information parameter ,The state of each underwater wireless sensor node is divided into two parts, namely whether the node transmits data and the reception of the node feedback information ACK. is the observation information of underwater sensor node i at time slot t, The underwater sensor node i observes the action of node j in time slot t, which is based on the strategy get, is the sensor node i observing whether the AP returns ACK to node j in time slot t. Ideally, any node i can get In fact, due to the uncertainty of the underwater environment and the collision of data packets transmitted by different nodes in the underwater wireless sensor network, some information may not be obtained correctly, so the above node state calculation formula is defined. In reality, the underwater environment changes dynamically. In order to avoid invalid data caused by a single observation abnormality in a certain time slot, the observation of the actual input model is defined. , which not only includes the observation of underwater sensor node i itself at the current time slot t , the state information observed by node i in the past M time slots should also be considered comprehensively. ;

[0065] (2) Define the node action calculation formula, assuming is the action selected by node i at time slot t, according to the strategy Calculated. When energy consumption is not considered, the action space size is only 2, that is, sending or refusing to send a data packet, represented by 1 and 0 respectively; if energy consumption is considered, the successful communication between underwater wireless sensor nodes depends on sufficient communication signal strength to ensure that the signal can be reliably transmitted within a predetermined distance and overcome environmental interference and noise. The communication signal strength is described by the SINR (Signal to Interference plus Noise Ratio) at the receiving node:

[0066] ;

[0067] in, is the node receiving power, For interference, The node receiving power is obtained by the node transmitting power after losing part of the energy through underwater propagation.

[0068] ;

[0069] ;

[0070] ;

[0071] Among them, spre is the underwater geometric propagation loss, abso is the absorption loss, is the Thorp formula, f is the signal frequency, and is usually measured in Hertz.

[0072] ;

[0073] Let the transmission power level be defined as p={1,2,...,p-1,p}, where p can be adjusted according to the network topology, each level corresponds to a transmission power, and define the node action calculation formula;

[0074] (3) Define the reward function, let is the ACK situation where sensor node i receives AP feedback in time slot t, and its definition is as follows:

[0075] ;

[0076] Feedback information is divided into two types, and the node's probability estimate of successful transmission at the current power consumption level is dynamically changed according to different transmission results. The formula defines that when the underwater wireless sensor node does not transmit data or the data packet transmission fails, the feedback value obtained is 0; the feedback value obtained when the data is successfully transmitted is 1. A successful data transmission process can be described as node i at time slot t using the lowest power level p that satisfies packet transmission to send a data packet and the data packet is successfully received, and the receiving node AP feeds back ACK to node i to confirm , which indicates whether the data sent by node i at time slot t is successfully transmitted to the receiving node.

[0077] definition is the reward obtained by node i in time slot t. According to the goal of the project, it is necessary to minimize the energy consumption of sensor nodes and maximize the throughput while the underwater sensor nodes successfully communicate. Define that if a node does not send a data packet, after the corresponding time slot ends, it will be given ; If the sent data packet successfully receives ACK, give it ; If the node sending the data packet does not receive the returned ACK, When a node sends a data packet, it will choose a certain gear in p={1,2,…,p-1,p}. Different transmission powers correspond to different node energy consumption. For node energy loss, a reward and penalty term is defined. Each time a data packet is sent, the energy consumption dEng is calculated and the penalty is included in the reward to define the reward function.

[0078] When a time slot ends, the information of the time slot is combined into an experience sample. Store it in the training data space for future training. Suppose node i receives the data packet from node j at time slot l, then consider is the sum of the data packet propagation time and the data packet sending time between two nodes, Divide the time slot length and round it to get the number of interval time slots , then actually The observation information should be updated. If all observation information is updated before executing the next step, the idle waiting time cannot be avoided. Therefore, the model does not guarantee that all observation information Timely update: the initial small number of data samples are partially missing information, which can be restored to normal after a certain period of time, and does not affect model training.

[0079] In this embodiment, the principle of adding a decision network to the time slot model is as follows: the model in the MAC protocol specifically adopts the MAAC algorithm based on the Actor-Critic (policy-value) architecture. The Critic network adopts a Value-based approach to evaluate the value of action selection, while the Actor network adopts a Policy-based approach to predict the probability distribution of actions through input states, and then generate action selections. Each underwater wireless sensor node is equipped with an Actor network and a Critic network that shares global information. The Actor network inputs the local observation of node i based on the Critic evaluation results and selects the best action, with the goal of maximizing the Q value evaluated by the Critic network. The Critic network uses the temporal difference error to update its network weights, with the goal of minimizing the TD error. The loss function is as follows:

[0080] ;

[0081] in, For reward, is the discount factor, is the entropy coefficient, which controls the balance between exploration and utilization. , and its calculation formula is as follows:

[0082] ;

[0083] in, To contribute to the value of node i itself, the node needs to input the local observation values ​​of all underwater sensor nodes And action set Calculate the value of the underwater sensor nodes other than node i. is the weighted sum of each node’s value:

[0084] .

[0085] Step S12: Obtain initial node data, and use the node state calculation formula and the node action calculation formula to calculate the initial node data to obtain the initial node state and the initial node action. If the initial node action is to send data, data communication is performed with other sending nodes except the local node.

[0086] Step S13: After obtaining the confirmation character fed back by the other sending nodes, the reward value is calculated based on the confirmation character and using the reward function, and the initial node state is updated according to the reward value to obtain an updated node state.

[0087] Step S14: updating the decision network in the time slot model using the initial node state, the initial node action, the reward value and the updated node state to obtain the updated decision network.

[0088] In this embodiment, update information is generated based on the initial node state, initial node action, reward value and updated node state, and the update information is stored in a local buffer; the decision network in the time slot model is updated using the update information corresponding to each time step in the buffer to obtain the updated decision network.

[0089] In this embodiment, the process of obtaining the initial node data and calculating the update is as follows: Figure 3 As shown in the figure: Each underwater wireless sensor node initializes the state s and converts it into the state input of the model. Each underwater node selects one of the two actions of waiting or sending data through the model according to the current state. In the initial training stage of the model, the algorithm adopts Greedy strategy, with probability Randomly select actions to explore with probability 1- Select the best action under the current model's known strategy, as training progresses Gradually decreases, and the model gradually transitions from the exploration stage to the optimization stage. When each underwater wireless sensor node determines the action and starts to execute, the node communication is carried out considering the long propagation delay characteristics underwater, and the AP receiving node obtains the confirmation information ACK. The confirmation information is input into the reward function to obtain the reward value of the current underwater node in the current time slot communication. If the node does not transmit data, the reward value is zero. When the above process is completed, the state of each underwater node in the current time slot ,action , Reward Value and the updated status All the data are stored in the experience replay buffer and are waiting to be updated. The model will update the data based on the experience stored in the experience replay buffer at each time step. Update the policy network, gradually adjust the action selection strategy of the node in different states, and finally obtain the appropriate node sending strategy. Update the model network during the training phase. Initialize the parameters of the Actor and Critic networks of node i at the beginning of the model operation. and After that, each time slot continuously carries out interactive communication between underwater wireless sensor nodes and other nodes in the network in the underwater environment, and the experience of each time step is Store it in the experience replay buffer for later use. When updating the Critic network, sample a batch of data from the experience replay buffer and calculate the target Q value:

[0090] ;

[0091] Then calculate the current Q value , and minimize the TD error To update the parameters of the Critic network , while evaluating the action selected by the Actor network output to calculate the policy gradient, maximizing the Q value evaluated by the Critic network to update the parameters of the Actor network , the Critic network parameters are periodically adjusted during the retraining phase Copy to target network parameters ,After the training is completed, the state is input to the target network, and the ,corresponding data transmission strategy for underwater wireless sensor nodes can be ,derived.

[0092] Step S15: Acquire current node data, and input the current node data into the updated decision network to output a node data transmission strategy, and perform data transmission between local and distributed receiving nodes in the underwater wireless sensor network according to the node data transmission strategy.

[0093] This application proposes a data transmission method of a distributed semi-cooperative MAC protocol based on deep reinforcement learning. The specific process is as follows Figure 4 As shown, first, the MAC protocol and basic protocol information are configured to build a time slot model, then the node state calculation formula, node action calculation formula and reward function are defined, and then a decision network is added to the time slot model. The node state calculation formula and the node action calculation formula are used to calculate the initial node data to obtain the initial node state and the initial node action. If the initial node action is to send data, data communication is performed with other sending nodes other than the local node. After obtaining the confirmation character fed back by other sending nodes, the reward value is calculated based on the confirmation character and the reward function, and the initial node state is updated according to the reward value to obtain the updated node state. The decision network in the time slot model is updated using the initial node state, the initial node action, the reward value and the updated node state to obtain the updated decision network. Finally, the current node data is obtained, and the current node data is input into the updated decision network to output the node data transmission strategy. According to the node data transmission strategy, data transmission is performed between the sending node and the receiving node in the distributed underwater wireless sensor network.

[0094] This application can maximize network throughput without the need to cooperate with traditional MAC protocols, and this application focuses on the energy consumption of communication between underwater network nodes, and comprehensively considers the conflicts in various underwater communication processes caused by the long underwater propagation delay characteristics, so that the research work has high fidelity. The traditional MAC protocol requires handshake information exchange or random time retransmission, which leads to additional communication overhead, and the fixed strategy makes it highly dependent on the environment and lacks adaptability. Applying reinforcement learning to MAC protocol design enables continuous interaction training between sensor nodes and the environment to derive transmission strategies, which can be applied to various network topologies. This application takes into account the completeness of node observations and the overall energy consumption of the underwater network, more effectively realizes the maximization of network throughput and saves energy of underwater sensor nodes, comprehensively considers the conflict problems caused by the long underwater propagation delay characteristics, improves the detection of conflict situations, and at the same time, has high adaptability.

[0095] In this embodiment, the MAC protocol and basic protocol information are configured, a time slot model is constructed, a node state calculation formula, a node action calculation formula and a reward function are defined based on the MAC protocol and the basic protocol information, and a decision network is added to the time slot model; initial node data is obtained, and the initial node data is calculated using the node state calculation formula and the node action calculation formula to obtain the initial node state and the initial node action. If the initial node action is to send data, data communication is performed with other sending nodes other than the local node; after the confirmation character fed back by the other sending node is obtained, the reward value is calculated based on the confirmation character and using the reward function, and the initial node state is updated according to the reward value to obtain an updated node state; the decision network in the time slot model is updated using the initial node state, the initial node action, the reward value and the updated node state to obtain the updated decision network; the current node data is obtained, and the current node data is input into the updated decision network to output a node data transmission strategy, and data transmission between the local and distributed receiving nodes in the underwater wireless sensor network is performed according to the node data transmission strategy. The present application is applied to any sending node in a distributed underwater wireless sensor network, configures the MAC protocol and basic protocol information, builds a time slot model, and does not need to cooperate with the traditional MAC protocol to maximize network throughput. It defines the node state calculation formula, node action calculation formula and reward function, and adds a decision network to the time slot model. It can be applied to various network topologies, takes into account the completeness of node observation and the overall energy consumption of the underwater network, and more effectively maximizes network throughput and saves energy of underwater sensor nodes. It calculates the initial node state and initial node action. If the initial node action is to send data, it communicates with other sending nodes except the local node for data transmission. According to communication, when the confirmation character fed back by other sending nodes is obtained, the reward value is calculated, and the initial node state is updated, so as to update the decision network in the time slot model to obtain an updated decision network, and the current node data is input into the updated decision network to output the node data transmission strategy, and the data transmission between the receiving nodes in the local and distributed underwater wireless sensor network is carried out according to the node data transmission strategy, taking into account the conflict problem caused by the long underwater propagation delay characteristics, and solving the conflict caused by the long underwater propagation delay characteristics. At the same time, there is no need to cooperate with the traditional MAC protocol, and it has high adaptability, which improves the efficiency of data transmission in the underwater wireless sensor network.

[0096] See also Figure 5 As shown, the embodiment of the present invention discloses a data transmission device of an underwater wireless sensor network, which is applied to any sending node in a distributed underwater wireless sensor network and may specifically include:

[0097] A construction and definition module 11 is used to configure the MAC protocol and basic information of the protocol, construct a time slot model, define a node state calculation formula, a node action calculation formula and a reward function based on the MAC protocol and the basic information of the protocol, and add a decision network to the time slot model;

[0098] The calculation module 12 is used to obtain the initial node data, calculate the initial node data using the node state calculation formula and the node action calculation formula to obtain the initial node state and the initial node action, and if the initial node action is to send data, perform data communication with other sending nodes except the local node;

[0099] The node state updating module 13 is used for, after obtaining the confirmation character fed back by the other sending nodes, calculating the reward value based on the confirmation character and using the reward function, and updating the initial node state according to the reward value to obtain the updated node state;

[0100] A decision network updating module 14, configured to update the decision network in the time slot model using the initial node state, the initial node action, the reward value and the updated node state to obtain the updated decision network;

[0101] The data transmission module 15 is used to obtain current node data and input the current node data into the updated decision network to output a node data transmission strategy, and perform data transmission between local and distributed receiving nodes in the underwater wireless sensor network according to the node data transmission strategy.

[0102] In this embodiment, the MAC protocol and basic protocol information are configured, a time slot model is constructed, a node state calculation formula, a node action calculation formula and a reward function are defined based on the MAC protocol and the basic protocol information, and a decision network is added to the time slot model; initial node data is obtained, and the initial node data is calculated using the node state calculation formula and the node action calculation formula to obtain the initial node state and the initial node action. If the initial node action is to send data, data communication is performed with other sending nodes other than the local node; after the confirmation character fed back by the other sending node is obtained, the reward value is calculated based on the confirmation character and using the reward function, and the initial node state is updated according to the reward value to obtain an updated node state; the decision network in the time slot model is updated using the initial node state, the initial node action, the reward value and the updated node state to obtain the updated decision network; the current node data is obtained, and the current node data is input into the updated decision network to output a node data transmission strategy, and data transmission between the local and distributed receiving nodes in the underwater wireless sensor network is performed according to the node data transmission strategy. The present application is applied to any sending node in a distributed underwater wireless sensor network, configures the MAC protocol and basic protocol information, builds a time slot model, and does not need to cooperate with the traditional MAC protocol to maximize network throughput. It defines the node state calculation formula, node action calculation formula and reward function, and adds a decision network to the time slot model. It can be applied to various network topologies, takes into account the completeness of node observation and the overall energy consumption of the underwater network, and more effectively maximizes network throughput and saves energy of underwater sensor nodes. It calculates the initial node state and initial node action. If the initial node action is to send data, it communicates with other sending nodes except the local node for data transmission. According to communication, when the confirmation character fed back by other sending nodes is obtained, the reward value is calculated, and the initial node state is updated, so as to update the decision network in the time slot model to obtain an updated decision network, and the current node data is input into the updated decision network to output the node data transmission strategy, and the data transmission between the receiving nodes in the local and distributed underwater wireless sensor network is carried out according to the node data transmission strategy, taking into account the conflict problem caused by the long underwater propagation delay characteristics, and solving the conflict caused by the long underwater propagation delay characteristics. At the same time, there is no need to cooperate with the traditional MAC protocol, and it has high adaptability, which improves the efficiency of data transmission in the underwater wireless sensor network.

[0103] In some specific embodiments, the construction and definition module 11 may specifically include:

[0104] The initial model building module is used to build the initial time slot model and determine the task objectives according to business needs;

[0105] The model training module is used to use the local data transmission work as a training parameter and train the initial time slot model in combination with the task goal to obtain the time slot model.

[0106] In some specific embodiments, the node status calculation formula is:

[0107] ;

[0108] ;

[0109] in, The action of node j in time slot t observed by node i in the underwater sensor, is the action selected by node j at time slot t, AP is the receiving node, Whether the AP observes node i to return a confirmation character to node j in time slot t, is the action chosen by node j at time slot t.

[0110] In some specific embodiments, the node action calculation formula is:

[0111] ;

[0112] in, is the action selected by node i at time slot t, p is the transmit power level, p={1,2,…,p-1,p}.

[0113] In some specific embodiments, the reward function is:

[0114] ;

[0115] in, is the reward value obtained by node i in time slot t, p is the transmission power level, p={1,2,…,p-1,p}, For reward and penalty items.

[0116] In some specific embodiments, the construction and definition module 11 may specifically include:

[0117] The algorithm adding module is used to add a MAAC algorithm based on a strategy-value architecture to the time slot model; the MAAC algorithm includes a decision network; the decision network includes a strategy network and a value network.

[0118] In some specific embodiments, the decision network update module 14 may specifically include:

[0119] An update information generation module, used to generate update information based on the initial node state, the initial node action, the reward value and the updated node state, and store the update information in a local buffer;

[0120] An updating module is used to update the decision network in the time slot model using the update information corresponding to each time step in the buffer.

[0121] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the data transmission method of the underwater wireless sensor network performed by the electronic device disclosed in any of the aforementioned embodiments.

[0122] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0123] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon include an operating system 221, a computer program 222 and data 223, etc. The storage method can be temporary storage or permanent storage.

[0124] Among them, the operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20, so as to realize the operation and processing of the data 223 in the memory 22 by the processor 21, which can be Windows, Unix, Linux, etc. In addition to including a computer program that can be used to complete the data transmission method of the underwater wireless sensor network performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can also further include a computer program that can be used to complete other specific tasks. In addition to including data transmitted from an external device received by the data transmission device of the underwater wireless sensor network, the data 223 can also include data collected by its own input and output interface 25, etc.

[0125] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0126] Furthermore, an embodiment of the present application also discloses a computer-readable storage medium, in which a computer program is stored. When the computer program is loaded and executed by a processor, the steps of the data transmission method of the underwater wireless sensor network disclosed in any of the aforementioned embodiments are implemented.

[0127] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0128] The above is a detailed introduction to the data transmission method, device, equipment and storage medium of an underwater wireless sensor network provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A data transmission method for an underwater wireless sensor network, characterized in that: Applicable to any sending node in a distributed underwater wireless sensor network, including: Configure the MAC protocol and basic information of the protocol, build a time slot model, define a node state calculation formula, a node action calculation formula and a reward function based on the MAC protocol and the basic information of the protocol, and add a decision network to the time slot model; Acquire initial node data, calculate the initial node data using the node state calculation formula and the node action calculation formula to obtain an initial node state and an initial node action, and if the initial node action is to send data, perform data communication with other sending nodes except the local node; After obtaining the confirmation character fed back by the other sending nodes, the reward value is calculated based on the confirmation character and using the reward function, and the initial node state is updated according to the reward value to obtain an updated node state; Updating the decision network in the time slot model using the initial node state, the initial node action, the reward value, and the updated node state to obtain the updated decision network; The current node data is obtained and input into the updated decision network to output a node data transmission strategy, and data transmission between local and distributed receiving nodes in the underwater wireless sensor network is performed according to the node data transmission strategy.

2. The data transmission method of underwater wireless sensor network according to claim 1, characterized in that: The constructing of the time slot model comprises: Build the initial time slot model and determine the task objectives based on business needs; The local data transmission work is used as a training parameter, and the initial time slot model is trained in combination with the task goal to obtain the time slot model.

3. The data transmission method of underwater wireless sensor network according to claim 1, characterized in that: The node status calculation formula is: ； ； in, The action of node j in time slot t observed by node i in the underwater sensor, is the action selected by node j at time slot t, AP is the receiving node, Whether the AP observes node i to return a confirmation character to node j in time slot t, is the action chosen by node j at time slot t.

4. The data transmission method of underwater wireless sensor network according to claim 1, characterized in that: The node action calculation formula is: ； in, is the action selected by node i at time slot t, p is the transmit power level, p={1,2,…,p-1,p}.

5. The data transmission method of underwater wireless sensor network according to claim 1, characterized in that: The reward function is: ； in, is the reward value obtained by node i in time slot t, p is the transmission power level, p={1,2,…,p-1,p}, For reward and punishment items.

6. The data transmission method of underwater wireless sensor network according to claim 1, characterized in that: The adding a decision network to the time slot model comprises: A MAAC algorithm based on a policy-value architecture is added to the time slot model; the MAAC algorithm includes a decision network; the decision network includes a policy network and a value network.

7. The data transmission method of an underwater wireless sensor network according to any one of claims 1 to 6, characterized in that: The updating of the decision network in the time slot model by using the initial node state, the initial node action, the reward value and the updated node state comprises: Generate update information based on the initial node state, the initial node action, the reward value, and the updated node state, and store the update information in a local buffer; The decision network in the time slot model is updated using the update information corresponding to each time step in the buffer.

8. A data transmission device for an underwater wireless sensor network, characterized in that: Applicable to any sending node in a distributed underwater wireless sensor network, including: A construction and definition module is used to configure the MAC protocol and basic information of the protocol, build a time slot model, define a node state calculation formula, a node action calculation formula and a reward function based on the MAC protocol and the basic information of the protocol, and add a decision network to the time slot model; A calculation module, used to obtain initial node data, calculate the initial node data using the node state calculation formula and the node action calculation formula to obtain the initial node state and the initial node action, and if the initial node action is to send data, perform data communication with other sending nodes except the local node; A node state updating module, configured to, after obtaining the confirmation character fed back by the other sending nodes, calculate a reward value based on the confirmation character and using the reward function, and update the initial node state according to the reward value to obtain an updated node state; A decision network updating module, used to update the decision network in the time slot model using the initial node state, the initial node action, the reward value and the updated node state to obtain the updated decision network; The data transmission module is used to obtain current node data and input the current node data into the updated decision network to output a node data transmission strategy, and perform data transmission between local and distributed receiving nodes in the underwater wireless sensor network according to the node data transmission strategy.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the data transmission method for an underwater wireless sensor network as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store computer programs; wherein, when the computer program is executed by a processor, the data transmission method of the underwater wireless sensor network as described in any one of claims 1 to 7 is implemented.

Images