Evaporative waveguide unmanned ship buffer relay cooperative control method and system

Abstract

The present application relates to the technical fields of maritime wireless communication, autonomous navigation of unmanned surface vehicle and multi-agent cooperative control, and discloses a kind of evaporation waveguide unmanned vehicle buffer relay cooperative control method and system.The method comprises the position of transmitting end and the position of receiving end are jointly inquired to the link gain of the second unmanned surface vehicle to the first unmanned surface vehicle, the first unmanned surface vehicle to the link gain of shore base station;According to the link rate, the local data queue of the second unmanned surface vehicle, the relay data queue and the local data queue of the first unmanned surface vehicle are updated, so that the relay out queue quantity is constrained by the received data quantity.Two shipborne strategy networks generate heading control commands according to the state of the ship, the geometry between ships, the link state and the queue summary, respectively, to drive the navigation mechanism and the wireless transceiver to operate in a time slot closed loop;The strategy network is centrally trained by using the global state of the central evaluation network, and does not need the central evaluation network when deployed.The present application completes double-hop data delivery and two-ship arrival task while meeting the navigation safety constraint.

Description

Technical Field

[0001] This invention relates to the fields of maritime wireless communication, autonomous navigation of unmanned surface vessels (USVs), and multi-agent cooperative control. In particular, it relates to a method and system for achieving double-hop data transmission and closed-loop navigation control of USVs by utilizing evaporative waveguide position-dependent link gain, buffer relay queues, and a centralized training-distributed execution strategy. In this document, "unmanned surface vessel" refers to any unmanned surface vessel. Background Technology

[0002] Evaporated waveguides are unique propagation environments near the sea surface where the atmospheric refractive index varies with altitude. These environments confine some radio wave energy between the sea surface and the waveguide layer, thus altering the path loss of long-distance maritime links. Large-scale channel gain in the ocean exhibits significant spatial non-uniformity due to the influence of evaporated waveguide height, frequency, antenna height, meteorological refractive index profile, and sea surface conditions.

[0003] By conducting on-sea measurements, simulating electromagnetic propagation using parabolic equations, ray tracing, or data-driven reconstruction, a channel gain map or link gain query model can be pre-established, and the link status can be queried according to the location of the communication end during navigation, thus avoiding the need to recalculate highly complex electromagnetic propagation for each time slot.

[0004] Existing technology CN120074715A discloses a multi-objective ship communication path planning method based on marine waveguide channel maps. Its controlled object is the direct link between a single ship and a fixed shore-based base station. It employs an offline population iterative algorithm that integrates NSGA-II and PSO to optimize transmission and arrival times for multiple objectives. This scheme does not involve links between two simultaneously moving communication terminals, nor does it address data causality in mobile relay buffers.

[0005] Existing technology CN121809801A discloses a ship multi-waypoint path planning method for evaporative waveguide channel maps. This method focuses on a single ship navigating along multiple waypoints and directly transmitting data to a shore-based base station, employing a single agent PPO for offline training and online inference. The segmented data volume in this scheme is used to constrain the completion of each segment's data before the next waypoint, and is not equivalent to the relay enqueueing and dequeueing causality in a source-relay-shore base station two-hop link.

[0006] In a collaborative data transmission mission between unmanned surface vessels (USVs), a second USV located far from a shore-based base station can first send data to a first USV, which then forwards it to the shore-based base station. The first USV may also have its own data to be uploaded. In this scenario, the mobile-to-mobile link between the second and first USVs, the mobile-to-fixed link between the first and first USVs, the local services and relay services of the first USV, and the navigation safety of both vessels are interconnected.

[0007] If a channel map centered on a fixed base station and indexed only by the location of a single mobile terminal is still used, the link state of the first link cannot be directly expressed as the relative positions of the two boats change together; if only the cumulative transmission volume is constrained without maintaining the relay queue, the relay boat may forward data that has not yet been received, which is an unrealistic result.

[0008] Furthermore, the course decisions of the two unmanned surface vessels (USVs) simultaneously alter the relative geometry between them, the gains of the two links, the platoon evolution, and the arrival time. Independent planning or training makes the environment faced by each USV non-stationary due to the changes in the other USV's strategy, while online centralized solutions for non-convex trajectory optimization increase the onboard computational load and communication burden.

[0009] Therefore, a technical solution is needed that combines link gain query based on the combination of transmitter and receiver positions, relay causality of three types of buffer queues, navigation control of two boats, and distributed shipborne inference into a unified closed loop. Summary of the Invention

[0010] I. Technical problems to be solved The purpose of this invention is to address the problems of existing single-vessel unmanned surface vessel (USV) having significant spatial non-uniformity in large-scale channel gain in the sea area, the inability of existing single-vessel direct communication path planning to characterize mobile-mobile relay links, the inability to guarantee reception before forwarding during state evolution, the inability to form relative geometric coordination between two vessels, and the large overhead of online centralized computing. This invention provides an evaporating waveguide unmanned surface vessel buffer relay cooperative control method and system.

[0011] II. Technical Solution This invention applies to a maritime communication system comprising a shore-based base station, a first unmanned surface vessel (USV), and a second USV. The first USV acts as a mobile relay, configured with a first local data queue and a relay data queue; the second USV is configured with a second local data queue. The first USV receives data from the second USV via a first wireless transceiver and transmits the relay data and the first USV's local data to the shore-based base station.

[0012] This invention employs an evaporating waveguide link gain query module that uses the combination of transmitter and receiver locations as query variables. The first link gain is queried based on the locations of the second and first unmanned surface vessels (USVs), and the second link gain is queried based on the locations of the first USV and the shore-based base station, thereby simultaneously characterizing both mobile-to-mobile and mobile-to-fixed links.

[0013] This invention calculates the effective transmission volume of two links according to discrete time slots and updates the second local data queue, the relay data queue, and the first local data queue. The amount of relay data dequeued within a time slot does not exceed the amount of existing data in the relay queue at the beginning of that time slot, thereby embedding relay causality into the queue state recursion.

[0014] This invention deploys a first policy network and a second policy network in the shipboard controllers of two unmanned surface vessels (USVs). Each policy network independently generates a heading adjustment amount based on its own sensor status, inter-vessel relative geometry, link status, its own queuing status, and a received fixed-dimensional task status summary, and converts the heading adjustment amount into actual heading control commands.

[0015] The first and second policy networks were obtained through centralized training and distributed execution during the offline phase. During training, the central evaluation network used the global state, which was composed of or encoded from the local states of the two ships, to perform value estimation and credit allocation. During deployment, the corresponding policy networks were run only on the two ships, the central evaluation network was not run, and online parameter synchronization or centralized non-convex optimization was not performed.

[0016] The two links can be configured using frequency division, time division, or full-duplex resource allocation with self-interference calibration. The effective rate includes the actual resource coefficient to ensure that queue updates correspond to the specific wireless transmission timing.

[0017] The present invention also provides an operating system, offline training device, computer program product and computer-readable storage medium corresponding to the above method.

[0018] An evaporating waveguide unmanned surface vessel (USV) buffer relay cooperative control method is applied to a maritime communication system. The maritime communication system includes a shore-based base station, a first USV, and a second USV. The first USV acts as a mobile relay between the second USV and the shore-based base station. The first USV is configured with a first local data queue and a relay data queue, while the second USV is configured with a second local data queue. The following... , and These represent the first unmanned surface vessel, the second unmanned surface vessel, and the shore-based base station, respectively. , and Representing the first local data queue, relay data queue, and second local data queue respectively, the method includes: S1, obtain the real-time position, heading, destination status, data queue status and navigation safety status of the first and second unmanned surface vessels; S2, using the evaporating waveguide link gain query module with the combination of the transmitter position and the receiver position as the query variable, the first link gain of the first link from the second unmanned surface vessel to the first unmanned surface vessel and the second link gain of the second link from the first unmanned surface vessel to the shore base station are obtained. S3, respectively form a first local state vector and a second local state vector corresponding to the first unmanned surface vessel and the second unmanned surface vessel, input the first local state vector into the first strategy network to generate a first heading adjustment amount, input the second local state vector into the second strategy network to generate a second heading adjustment amount, and convert the corresponding heading adjustment amount into a heading control command and send it to the navigation control mechanism of the corresponding unmanned surface vessel; S4. Determine the effective transmission volume of the first link and the effective transmission volume of the second link based on the first link gain, the second link gain and the wireless resource configuration. Control the wireless transceiver of the second unmanned surface vessel to transmit data to the wireless transceiver of the first unmanned surface vessel that does not exceed the existing data volume of the second local data queue. Control the wireless transceiver of the first unmanned surface vessel to transmit data in the first local data queue and data in the relay data queue to the shore-based base station. Update the three types of data queues according to the measured transmission volume. The outgoing volume of the relay data queue in any time slot does not exceed the existing data volume of the relay data queue at the beginning of that time slot. S5, under the constraints of ocean current, turning, navigation area, obstacles and safe distance between boats, execute the course control command and update the position of the two unmanned surface vessels, return to step S1 to perform the next time slot control, until the two unmanned surface vessels have reached their respective destinations and all three types of data queues are cleared; The first local state vector and the second local state vector each include at least the motion state of the vessel, the navigation state related to the destination, the relative geometric state of the two vessels, the link state of at least one of the two links, the data queue state that the vessel can obtain, and the received fixed-dimensional task state summary; the first policy network and the second policy network are centrally trained by a central evaluation network using global state, and the central evaluation network does not participate in heading control during deployment.

[0019] Preferably, the evaporation waveguide link gain query module is represented as follows: .

[0020] in, For the three-dimensional coordinates of the transmitter, For the receiver's three-dimensional coordinates, The environmental parameter vector includes at least one of the following: evaporation waveguide characteristic height, carrier frequency, transmit antenna height, receive antenna height, meteorological refractive index profile, or sea surface state. The link gain query model stores the link gain using a transmitter-receiver location pair grid or a relative location grid, and obtains the link gain at any location through neighboring grid query, bilinear interpolation, or trilinear interpolation.

[0021] Preferably, the first and second links operate using different frequency bands, different time slots, or full-duplex resources calibrated by self-interference, so that the effective transmission volume of the two links corresponds to the actual wireless resource configuration; for the link index Link Index In the time slot The effective transmission rate is determined according to the following formula: .

[0022] in, Link Index The wireless resource coefficient within the time slot, For link bandwidth, For transmission power, and These are the transmit antenna gain and the receive antenna gain, respectively. For link gain, This represents the noise power spectral density.

[0023] Preferred, time slot Effective transmission volume of the first internal link Total effective transmission volume of the second link Each satisfies , Under the relay priority service rule, , And update the queue according to the following formula: , , .

[0024] in, and The first and second links are respectively in time slots The effective transmission rate, and The second link in time slots Of the total effective transmission volume, the portion allocated to the relay data queue and the first local data queue. For the second local data queue, For relay data queue, For the first local data queue, This represents the time slot length.

[0025] Preferably, the total effective transmission volume of the second link is allocated between the relay data queue and the first local data queue according to the weighted queue service rule. The weighted queue service rule determines the relay service weight and the local service weight based on at least two of the following: queue occupancy ratio, remaining task time slots, first link rate, second link rate, destination distance, and service priority. Unused allocated capacity of one queue is reallocated to another non-empty queue, while ensuring that the outbound volume of each queue does not exceed the existing data volume of that queue at the beginning of the time slot.

[0026] Preferred, let For the first Unmanned surface vessels according to time slots Initial heading The actual course executed within the time slot, after correction of the preset course range, the maximum turning angle of adjacent time slots, and navigation safety constraints, and the corresponding course adjustment amount, is then the first... The position of the unmanned surface vessel is updated according to the following formula: .

[0027] And order ,in, , For cruising speed, The current velocity component is used; heading corrections are performed via motion projection, safety shields, or navigation controllers to meet navigation area constraints, obstacle safety distance constraints, and minimum safe distance constraints between two vessels.

[0028] Preferably, the first local state vector includes the position, heading, distance to the first destination, distance between the two vessels, first link status, second link status, first local data queue status, relay data queue status, second local data queue summary, heading error towards the first destination, normalized mission time, and obstacle characteristics of the first unmanned surface vessel; the second local state vector includes the position, heading, distance to the second destination, distance between the two vessels, first link status, second link status, second local data queue status, relay data queue summary, first local data queue summary, heading error towards the second destination, normalized mission time, and obstacle characteristics of the second unmanned surface vessel; wherein, the second local data queue summary, which the first unmanned surface vessel cannot directly obtain, as well as the relay data queue summary, the first local data queue summary, and the second link status summary, which the second unmanned surface vessel cannot directly obtain, are all obtained through mission status communication links broadcast in a fixed dimension; the mission status communication links are respectively connected to the first shipborne controller and the second shipborne controller.

[0029] Preferably, the task cost is constructed using the completion time slots of the first and second unmanned surface vessels entering the destination tolerance range, the number of time slots with effective transmission on the first link, and the number of time slots with effective transmission on the second link, and both unmanned surface vessels entering the corresponding destination tolerance range are considered as having completed the task. , and Setting all zeros as the condition for task success.

[0030] An evaporating waveguide unmanned surface vessel buffer relay collaborative control system includes a shore-based base station, a first unmanned surface vessel, a second unmanned surface vessel, an evaporating waveguide link gain query module, and a mission status communication link; A shore-based base station includes a shore-based wireless transceiver; The first unmanned surface vessel includes a first wireless transceiver, a first memory, a first shipborne controller, a first navigation mechanism, and a first state perception unit. The first memory stores a first local data queue and a relay data queue, and the first shipborne controller stores a first policy network. The first shipborne controller is connected to the first wireless transceiver, the first memory, the first navigation mechanism, and the first state perception unit, respectively. The second unmanned surface vessel includes a second wireless transceiver, a second memory, a second shipboard controller, a second navigation mechanism, and a second state awareness unit. The second memory stores a second local data queue, and the second shipboard controller stores a second policy network. The second shipboard controller is connected to the second wireless transceiver, the second memory, the second navigation mechanism, and the second state awareness unit. The second wireless transceiver establishes a first link with the first wireless transceiver, and the first wireless transceiver establishes a second link with the shore-based wireless transceiver; the main transmission direction of task data on the first link is from the second wireless transceiver to the first wireless transceiver, and the main transmission direction of task data on the second link is from the first wireless transceiver to the shore-based wireless transceiver. The mission status communication link is connected to the first shipborne controller and the second shipborne controller respectively, and is used to transmit fixed-dimensional mission status summaries bidirectionally between the two unmanned surface vessels. The evaporating waveguide link gain query module is connected to the first shipborne controller and the second shipborne controller respectively, and interacts with the first shipborne controller and the second shipborne controller respectively to query data, and outputs the first link gain and the second link gain according to the combination of the transmitter position and the receiver position. The first shipborne controller and the second shipborne controller respectively form a first local state vector and a second local state vector based on the motion state of the ship itself, the destination state, the relative geometric state of the two ships, the link state, the data queue state and the received task state summary provided by the corresponding state perception unit, and generate a first heading control command and a second heading control command through the first policy network and the second policy network respectively. The first and second shipborne controllers respectively control the corresponding navigation mechanisms to execute course control commands, and recursively update the three types of data queues according to the actual transmission volume until both unmanned surface vessels have reached their respective destinations and all three types of data queues are cleared.

[0031] Preferably, it also includes an offline training device, which is connected to the first policy network and the second policy network respectively; the offline training device includes a digital twin environment simulator, a trajectory cache, a central evaluation network and a policy update module; The digital twin environment simulator is used to update the position, link status, and three types of data queues of the two unmanned surface vessels based on the motion status of the first and second unmanned surface vessels, the first and second link gains output by the evaporating waveguide link gain query module, the status of the first local data queue, the relay data queue, and the second local data queue, as well as navigation safety constraints, and to generate training transfers. The trajectory cache is connected to the digital twin environment simulator and the policy update module respectively, and is used to store training transitions. The training transitions include a first local state vector, a second local state vector, a global state vector composed of the first local state vector and the second local state vector, a first action output by the first policy network, a second action output by the second policy network, a shared reward, and the first local state vector, the second local state vector, and the global state vector of the next time slot. The policy update module is connected to the central evaluation network. The central evaluation network takes the global state vector as input and outputs the corresponding state value. It is only used for value estimation and credit allocation during the offline training phase. The policy update module is connected to the trajectory cache, the central evaluation network, the first policy network, and the second policy network, respectively. It is used to calculate the advantage estimate and the reward target based on the training transition in the trajectory cache, update the first policy network and the second policy network respectively by using the near-end policy probability ratio to prune the target, and update the central evaluation network according to the error between the state value and the reward target. The policy update module is also used to write the updated first policy network parameters and second policy network parameters into the first shipboard controller and the second shipboard controller respectively after training convergence. The central evaluation network is not run during the online operation phase.

[0032] Beneficial effects To address the issues of spatially non-uniform distribution of link gain in the marine area caused by evaporating waveguides, and the simultaneous influence of transmitter and receiver positions on mobile-to-mobile links, this invention employs an evaporating waveguide link gain query module. This module uses the position combination of the second and first unmanned surface vessels (USVs) as the query variable for the first link, and the position combination of the first USV and a shore-based base station as the query variable for the second link. In each time slot, the module obtains the first and second link gains corresponding to the current transmitter-receiver position combination, and uses these link gains for local state construction, effective transmission volume calculation, and heading control. This allows the two USVs to adjust their relative geometry and navigation direction based on the actual propagation conditions corresponding to their current positions, identify and utilize favorable propagation areas of the evaporating waveguide, reduce link state mismatch caused by treating the marine channel as uniformly distributed or estimating only geometric distance, and improve the adaptability of two-hop transmission decisions to spatially non-uniform channels.

[0033] By recursively updating the second local data queue, the first local data queue, and the relay data queue, the forwarding volume of the relay boat is constrained by the amount of data already received, thus avoiding transmission schemes that do not meet the physical communication timing requirements.

[0034] By jointly determining the course of the two vessels through two independent shipborne strategy networks, the strategy can proactively maintain a favorable relative geometry during the critical relay transmission phase and turn to a more direct destination route after the relay mission is completed.

[0035] Offline centralized training via a central evaluation network can mitigate the non-stationarity caused by simultaneous updates of two policy networks; online distributed execution can concentrate the main computational burden in the offline phase, reducing the overhead of shipboard real-time computing and inter-ship communication.

[0036] By incorporating navigation area, obstacles, safe distance between vessels, turning ability, and ocean current effects into a closed-loop control system, the feasibility of collaborative missions and navigation safety can be improved.

[0037] By setting "both boats reach their respective destinations and all three queues are cleared" as the mission completion condition, the completion of the navigation mission is kept consistent with the completion of double-hop data delivery. This avoids data non-delivery caused by pursuing only the shortest path, and also avoids excessive detours caused by pursuing only instantaneous high-channel areas. Attached Figure Description

[0038] Figure 1 This is a structural diagram of the marine dual-boat buffer relay cooperative control system of the present invention.

[0039] Figure 2 This is a schematic diagram of the link gain query based on the combination of transmitter and receiver positions and the causal data flow of three types of data queues according to the present invention.

[0040] Figure 3 This is a schematic diagram of the centralized training and distributed execution architecture of this invention.

[0041] Figure 4 This is a flowchart of the online time-slot closed-loop control of the present invention.

[0042] Reference numerals: 100 – Maritime communication system; 110 – Shore-based base station; 111 – Shore-based radio transceiver; 120 – First unmanned surface vessel; 121 – First radio transceiver; 122 – First local data queue; 123 – Relay data queue; 124 – First shipborne controller; 125 – First strategy network; 126 – First navigation mechanism; 127 – First memory; 128 – First state awareness unit; 130 – Second unmanned surface vessel; 131 – Second radio transceiver; 132 – Second local data queue; 133 – Second memory; 134 – Second shipborne controller; 135 – Second strategy network; 136 – Second navigation mechanism; 138 – Second state awareness unit; 140 – Evaporated waveguide link gain query module; 150 – Mission status communication link; 160 – Offline training equipment; 161 – Central evaluation network; 162 – Digital twin environment simulator; 163 – Trajectory cache. Detailed Implementation

[0043] Example 1: System Composition and Physical Closed Loop like Figure 1 As shown, an evaporating waveguide unmanned surface vessel (UUV) buffer relay cooperative control system includes a shore-based base station 110, a first UUV 120, a second UUV 130, an evaporating waveguide link gain query module 140, a mission status communication link 150, and an offline training device 160. The shore-based base station 110 is equipped with a shore-based wireless transceiver 111. The first UUV 120 and the second UUV 130 are respectively equipped with a wireless transceiver, a memory, a shipborne controller, a policy network, a navigation mechanism, and a status awareness unit.

[0044] The first unmanned surface vessel 120 includes a first wireless transceiver 121, a first memory 127, a first shipborne controller 124, a first policy network 125, a first navigation mechanism 126, and a first state awareness unit 128. The first memory 127 stores a first local data queue 122 and a relay data queue 123. The first shipborne controller 124 is connected to the first wireless transceiver 121, the first memory 127, the first navigation mechanism 126, and the first state awareness unit 128. The first shipborne controller 124 reads the states of the first local data queue 122 and the relay data queue 123 from the first memory 127 and writes the updated queue states back to the first memory 127. The first shipborne controller 124 sends heading control commands to the first navigation mechanism 126 and receives command execution status feedback from the first navigation mechanism 126. The first state awareness unit 128 provides the first shipborne controller 124 with the position, heading, and speed status of the first unmanned surface vessel 120.

[0045] The second unmanned surface vessel 130 includes a second wireless transceiver 131, a second memory 133, a second shipborne controller 134, a second policy network 135, a second navigation mechanism 136, and a second state awareness unit 138. The second memory 133 stores a second local data queue 132. The second shipborne controller 134 is connected to the second wireless transceiver 131, the second memory 133, the second navigation mechanism 136, and the second state awareness unit 138. The second shipborne controller 134 reads the state of the second local data queue 132 from the second memory 133 and writes the updated queue state back to the second memory 133; the second shipborne controller 134 sends heading control commands to the second navigation mechanism 136 and receives command execution status feedback from the second navigation mechanism 136; the second state awareness unit 138 provides the second shipborne controller 134 with the position, heading, and speed status of the second unmanned surface vessel 130.

[0046] The second wireless transceiver 131 establishes a first link with the first wireless transceiver 121, and the first wireless transceiver 121 establishes a second link with the shore-based wireless transceiver 111. Both the first and second links support bidirectional wireless signal transmission. Specifically, the primary transmission direction of task data on the first link is from the second wireless transceiver 131 to the first wireless transceiver 121, and the primary transmission direction of task data on the second link is from the first wireless transceiver 121 to the shore-based wireless transceiver 111; the reverse transmission is used to carry acknowledgment information, retransmission requests, link control information, or task configuration information.

[0047] The mission status communication link 150 is communicatively connected to the first shipborne controller 124 and the second shipborne controller 134, respectively, for bidirectional transmission of fixed-dimensional mission status summaries between the two unmanned surface vessels. The mission status communication link 150 is not directly connected to the first navigation mechanism 126 or the second navigation mechanism 136. The evaporating waveguide link gain query module 140 is connected to the first shipborne controller 124 and the second shipborne controller 134, respectively, and performs query data interaction with them, receiving the transmitter position, receiver position, and environmental parameters, and returning the link gain under the corresponding position combination. After training, the offline training device 160 sends the network parameters of the first policy network 125 and the second policy network 135 to the first shipborne controller 124 and the second shipborne controller 134, respectively.

[0048] The offline training device 160 is connected to the first policy network 125 and the second policy network 135 respectively; the offline training device 160 includes a digital twin environment simulator 162, a trajectory cache 163, a central evaluation network 161 and a policy update module 164.

[0049] The digital twin environment simulator 162 is used to update the position, link status and three types of data queues of the two unmanned surface vessels based on the motion status of the first unmanned surface vessel 120 and the second unmanned surface vessel 130, the first link gain and the second link gain output by the evaporating waveguide link gain query module 140, the status of the first local data queue 122, the relay data queue 123 and the second local data queue 132, and navigation safety constraints, and to generate training transfer.

[0050] The trajectory cache 163 is connected to the digital twin environment simulator 162 and the policy update module 164 respectively, and is used to store training transitions. The training transitions include a first local state vector, a second local state vector, a global state vector composed of the first local state vector and the second local state vector, a first action output by the first policy network, a second action output by the second policy network, a shared reward, and the first local state vector, the second local state vector, and the global state vector of the next time slot.

[0051] The policy update module 164 is connected to the central evaluation network 161. The central evaluation network 161 takes the global state vector as input and outputs the corresponding state value, and is only used for value estimation and credit allocation in the offline training phase. The policy update module 164 is connected to the trajectory cache 163, the central evaluation network 161, the first policy network 125, and the second policy network 135, respectively. It is used to calculate the advantage estimate and reward target based on the training transition in the trajectory cache 163, update the first policy network 125 and the second policy network 135 respectively by pruning the target using the near-end policy probability ratio, and update the central evaluation network 161 according to the error between the state value and the reward target.

[0052] The policy update module 164 is also used to write the updated parameters of the first policy network 125 and the second policy network 135 into the first shipboard controller 124 and the second shipboard controller 134 respectively after training convergence, and the central evaluation network 161 is not run during the online operation phase.

[0053] Example 2: Symbol Conventions and Link Gain Query Based on Transmitter-Receiver Location Pairs To avoid ambiguity in symbol usage, the following conventions are adopted in this specification: Indicates the first unmanned surface vessel. Indicates the second unmanned surface vessel. Indicates shore-based base station; For indexing unmanned surface vessels, For link index, For task slot index, For sampling index within the trajectory segment, Indexing obstacles For summation offset index; bold lowercase letters indicate vectors, the upper right corner letter of a vector or matrix. Indicates transpose. The square brackets represent the Euclidean norm, and they indicate vector components or vector concatenation. , , and Use its usual mathematical meaning; without parameter subscripts Representing pi, with network parameter subscripts. unmanned surface vessel The policy network; the tilde-marked quantity represents a fixed-dimensional summary that has been normalized, quantized, and broadcast via the task state communication link. and They jointly stated that unmanned surface vessels Corresponding policy network in time slots Output the shape parameter pairs of the Beta distribution.

[0054] Let the three-dimensional coordinates of the shore-based base station 110 be... Unmanned surface vessel In the time slot The horizontal position is The corresponding three-dimensional coordinates of the antenna are ,in For unmanned surface vessels Antenna height.

[0055] Unmanned Surface Vessel The initial position and destination are denoted as follows: and The first and second unmanned surface vessels (USVs) can have different initial positions, destinations, navigation areas, and navigation parameters.

[0056] The link gain query module is denoted as ,in and These are the coordinates of the transmitter and receiver, respectively. This is a vector of environmental parameters. Environmental parameters may include the evaporation waveguide height. carrier frequency Transmit and receive antenna height, refractive index profile, sea surface roughness, and meteorological time stamps.

[0057] Link gain data can be stored using two preferred methods. The first method involves gridding the coordinates of the transmitter and receiver separately to form a position-paired grid; the second method stores the data in a relatively level and stable sea area using relative displacement. The transmit antenna height and receive antenna height form a relative position grid. During the query, neighboring grids are selected and bilinear or trilinear interpolation is performed.

[0058] Link gain data can be generated using a parabolic equation solver. Specifically, the propagation field is solved based on a given refractive index profile, sea surface boundary conditions, carrier frequency, and antenna configuration, converted into a large-scale power gain, and written into the position pair grid; the grid can also be calibrated using marine measurements or ray tracing results.

[0059] In the time slot The gains of the first link and the second link are respectively: (1).

[0060] (2).

[0061] For link index The effective transmission rate consistent with the wireless resource configuration can be calculated using the following formula: (3).

[0062] In equation (3), Link Index In the time slot The resource coefficient in For link bandwidth, For transmission power, and These are the transmit antenna gain and the receive antenna gain, respectively. For the link gain obtained from equation (1) or equation (2), This represents the noise power spectral density. For time-division multiplexing, For frequency division multiplexing, the two links use different... For full-duplex mode, the measured residual self-interference is further included in the noise or interference term. The link indices for the first and second links are different, but the same general link index ℓ is used to represent the two different links. When At that time, the link is the first link from the second unmanned surface vessel to the first unmanned surface vessel; when At that time, the link is the second link from the first unmanned surface vessel to the shore-based base station. Both links use the same effective transmission rate calculation method, but the resource coefficient, bandwidth, transmit power, antenna gain, and link gain of each link are taken according to the corresponding link.

[0063] When using formula (3), Using Hertz, Using watts, Using watts per hertz, and employing linear values ​​for antenna gain and link power gain, then... The unit is bits per second; engineering parameters given in dBm, dBi, or dB should first be converted to the corresponding linear quantity.

[0064] In practical systems, Alternatively, the effective payload rate can be obtained by modulation and coding scheme, packet loss rate, protocol overhead, and retransmission calculation. As long as the queue update uses the same amount of effective data as the actual transmission of the wireless transceiver, the relay causal loop of this invention remains unchanged.

[0065] Example 3: Three Types of Queues and Relay Causality like Figure 2 As shown, in the time slot At the beginning, This indicates the amount of local data that the first unmanned surface vessel has not yet uploaded to the shore-based base station. This indicates the amount of local data that the second unmanned surface vessel has not yet sent to the first unmanned surface vessel. This indicates the amount of relay data that the first unmanned surface vessel has received from the second unmanned surface vessel but has not yet forwarded to the shore-based base station.

[0066] Unless otherwise stated, , , , , The transmission volume and data arrival volume in each time slot should use the same data unit. It is preferable to uniformly convert to bits. It can be directly compared with the amount of data in the queue; when using bytes, MB or GB as the unit of measurement, it should be uniformly converted to bits before queue recursion.

[0067] If both vessels have completed their data collection for this round before the mission begins, the initial queue can be set to... , , , among which and These represent the mission data volume for the first and second unmanned surface vessels, respectively.

[0068] The first link is in the time slot The actual amount of task data transmitted internally is: (4).

[0069] The second link is in the time slot The total amount of task data that can be transferred internally is: (5).

[0070] In the relay-priority service mode, the total task data volume of the second link is broken down as follows: (6).

[0071] (7).

[0072] and These are the portions of the total task data volume of the second link used for the relay data queue and the first local data queue, respectively.

[0073] The three types of queues are updated according to the following formula: (8).

[0074] (9).

[0075] (10).

[0076] For the second local data queue, For relay data queue, For the first local data queue, This represents the time slot length.

[0077] From equation (6), we can see that Not exceeding the start of the time slot Therefore, the data newly sent by the second unmanned surface vessel to the first unmanned surface vessel in the current time slot will participate in the forwarding in the next time slot at the earliest, automatically satisfying the principle of receiving before forwarding, without the need to set up a causal inequality for cumulative information.

[0078] Another implementation uses a weighted queue service. Let the relay service weights be... and local service weight Both are non-negative and satisfy The weights are calculated based on queue occupancy percentage, remaining task time slots, two link speeds, destination distance, or service priority. First, they are allocated according to their respective weights. If a queue is insufficient to use its allocated capacity, the remaining capacity is reallocated to another non-empty queue; the amount of data dequeued shall not exceed the amount of data in the corresponding queue at the start of the time slot.

[0079] In continuous sensing scenarios, time slots can be further configured. Data arrivals not included in the local queues of the first and second unmanned surface vessels and Rewrite equations (8) and (10) as follows: and This extension does not change the relay queue's constraint of receiving before forwarding. and They are time slots The amount of data newly generated by the mission payloads or sensing devices of the first and second unmanned surface vessels and written to the corresponding local data queue at the end of the time slot; the amount of data is obtained by counting the data packets of the mission payload, statistics of the data acquisition module, or the product of the application layer data generation rate and the time slot length, and uses the same data measurement unit as each data queue.

[0080] Example 4: Motion Model and Safety Constraints Discretize the task duration into lengths of... The maximum number of time slots is Unmanned surface vessels are deployed. The cruising speed is , For time slots The initial heading angle, the policy network outputs the heading adjustment amount in that time slot. The actual course executed within this time slot, after correction by motion projection, safety shield, or underlying navigation controller, is recorded as follows: The ocean current velocity vector is The position is then updated to: (11).

[0081] In equation (11), the location, destination distance, obstacle distance, and safety distance all use the same unit of length. and Use the corresponding length per time unit. Using time units, thus position increments and The dimensions are consistent.

[0082] Actual course Restricted to Inside, among them, and These are the lower and upper limits of the preset heading angle representation range, respectively, which are predetermined by the heading angle definition method of the mission coordinate system and the heading controller interface. For example, in a... In the heading angle representation where the positive direction of the axis is zero heading and the counterclockwise direction is positive heading, one of the following can be taken: The upper and lower limits of the heading angle are used to define a unique range for the heading angle. The actual turning capability of the unmanned surface vessel is determined by the time slot length. Maximum allowable heading adjustment Limited. And satisfying. ,in Time slot length The corresponding maximum permissible steering distance; at the end of the time slot, let Therefore, time slots The resulting heading action directly affects the displacement update in the same time slot and does not introduce additional control time slot delay.

[0083] The first and second unmanned surface vessels can each establish independent navigation areas. (Regarding obstacles...) Settings Center and safe distance ,Require A minimum safe distance should be set between the two boats. ,Require .

[0084] The aforementioned constraints can be enforced by action projection, safety shields, model prediction corrections, or the underlying navigation controller. If the policy output violates a hard constraint, the safety module projects it to the nearest feasible course; safety penalties in the training rewards are used to reduce the frequent triggering of safety corrections.

[0085] When the unmanned surface vessel enters the destination Centered on, with radius Once the tolerance zone is reached, it is determined that the navigation mission has been completed and can remain stationary, stationary at low speed, or cruise safely within the tolerance zone.

[0086] Example 5: Local State, Distributed Inference, and Online Process The first local state vector can be represented as: (12).

[0087] The second local state vector can be represented as: (13).

[0088] In equations (12) and (13), and These are the navigation and relative geometric sub-vectors of the two boats, respectively; and These are the link state sub-vectors; and These are the queue state sub-vectors. This is the distance from the vessel to its destination. The distance between boats For the current course of this vessel to satisfy the direction of its destination The minimum heading error, To normalize task time, For fixed-dimensional obstacle features. Queue size marked with a tilde and... These represent queues or link summaries broadcast via the task status communication link, respectively. For the first The destination of the unmanned surface vessel. This represents the distance from the first unmanned surface vessel to its destination. This represents the distance from the second unmanned surface vessel to its destination. The distance between the two boats; This indicates that the variables within the parentheses are concatenated column by column to form a vector. , , as well as These represent the queue summary or link status summary broadcast via the task status communication link after normalization or quantization, respectively.

[0089] The first unmanned surface vessel can be obtained directly. , The second unmanned surface vessel (USV) receives the second local data queue digest directly through two link statuses and via mission status communication link 150. It receives the relay queue, local queue, and second link state summary sent by the first unmanned surface vessel (USV) via the mission state communication link 150. Thus, the "local state" is composed of the USV's sensor state and a fixed-dimensional broadcast summary, and does not require online aggregation of the complete global state.

[0090] During the training phase, the global state vector of the central evaluation network is formed by concatenating the first local state vector and the second local state vector. in, This indicates that the vectors within the parentheses are concatenated column by column; and The first and second unmanned surface vessels were in time slots respectively. The local state vector, For the central evaluation network in time slots The global state vector used. During the online deployment phase, no central evaluation network is run, therefore there is no need to construct a global state vector.

[0091] like Figure 4 As shown, each online time slot executes the following sequentially: reading sensor status and queue status; querying... and ;form and The first strategy network 125 and the second strategy network 135 output heading adjustment amounts respectively; the safety module generates heading control commands and sends them to the navigation mechanism; double-hop wireless transmission is performed within the time slot; the three types of queues are updated according to the actual confirmed payload data; the position is updated and the next time slot is entered.

[0092] The wireless transceiver can count the amount of payload successfully transmitted in the current time slot through link layer acknowledgments, sequence numbers, or cumulative acknowledgment messages. It is preferable to update the queue with the measured amount of successful payload; when simulation or the link layer does not provide acknowledgments, the effective transmission amount estimated according to equation (3) can be used.

[0093] The mission success condition is that both unmanned surface vessels enter the corresponding destination tolerance zone and , and All are zero. If it reaches... If the success conditions are still not met, it will be recorded as a timeout failure.

[0094] Define unmanned surface vessel The completion time slot is: (14).

[0095] in, For indexing unmanned surface vessels, subscript " "" is an abbreviation for destination; Indicates the first An unmanned surface vessel in a time slot The Euclidean distance between it and its destination is defined as: .

[0096] in, For the first An unmanned surface vessel in a time slot The position is the first The destination of the unmanned surface vessel. To reach the tolerance radius, This represents the maximum number of task time slots. Indicates the first The time slot when the unmanned surface vessel first enters its destination tolerance zone. If the above set is empty, i.e., the [missing information]... If an unmanned surface vessel fails to enter its destination tolerance zone within the maximum mission time domain, then: .

[0097] The number of time slots for effective transmission of the first and second links are defined as follows: (15).

[0098] (16).

[0099] in This is an indicator function.

[0100] The following task cost can be constructed: (17).

[0101] in This represents the task cost weight.

[0102] Example 6: Centralized Training and Policy Network Structure This embodiment provides an offline centralized training method for obtaining a first policy network 125 and a second policy network 135, including the following steps: T1, construct a digital twin environment simulator 162, which includes motion models of two unmanned surface vessels, an evaporating waveguide link gain query module 140, three types of data queues, and navigation safety constraints; T2, set up a first strategy network 125 and a second strategy network 135 for the first unmanned surface vessel 120 and the second unmanned surface vessel 130 respectively, and set up a central evaluation network 161 with global state vector as input; T3, in each sampling time slot, the first policy network 125 and the second policy network 135 sample normalized actions according to the corresponding local state vectors respectively. The digital twin environment simulator 162 updates the position, link status and three types of data queues of the two unmanned surface vessels according to the actions, calculates the shared reward, and writes the local state, global state, joint action, shared reward and the next time slot state into the trajectory cache 163. T4, the policy update module 164 calculates the advantage estimate and reward target based on the sampled data in the trajectory cache 163, updates the first policy network 125 and the second policy network 135 respectively using the near-end policy probability ratio to prune the target, and updates the central evaluation network 161 using the global state value reward error. T5. Repeat steps T3 and T4 until training converges. The trained first policy network 125 and second policy network 135 are distributed to the first shipborne controller 124 and the second shipborne controller 134, respectively. The central evaluation network 161 is not distributed to the online operating system.

[0103] like Figure 3 As shown, the offline training device 160 includes a digital twin environment simulator 162, a trajectory cache 163, a first policy network 125, a second policy network 135, and a central evaluation network 161. The digital twin environment simulator 162 implements the link query, motion update, queue update, state construction, and security constraints of equations (1) to (17).

[0104] In a feasible network architecture, the first and second policy networks are both multilayer perceptrons. The input layer dimension of each policy network is equal to the dimension of the corresponding local state vector, followed by two fully connected hidden layers, each with 256 neurons and activated using Tanh or ReLU. (For unmanned surface vessel indexing...) With unmanned surface vessels The corresponding policy network in the time slot Output a pair of Beta distribution shape parameters and The Beta distribution is defined on the interval [0, 1]. Above, a continuous probability distribution is determined by two shape parameters that are greater than zero. and The first and second policy networks together constitute a pair of shape parameters output in the same time slot, thus each outputs its own pair of Beta-distributed shape parameters. The action distribution parameter output layer of the policy network can be adjusted using the Softplus function and a positive bias. ,in, .

[0105] From the parameter is and Sampling normalization action in the Beta distribution: , And the normalized motion is mapped to a heading adjustment: (18).

[0106] in, For indexing unmanned surface vessels, For time slot index, Unmanned surface vessel The normalization action of the corresponding policy network sampling, For the corresponding heading adjustment, The time slot length is The maximum allowable heading adjustment at that time. Therefore, .

[0107] The input to the central evaluation network 161 is the global state. Output scalar state value ,in These are the parameters for the central evaluation network. The central evaluation network is only used for value estimation and credit assignment during the training phase and does not directly generate any shipboard heading actions.

[0108] The shared reward consists of four parts: data progress, navigation progress, safety, and termination. (19).

[0109] In equation (19), For time slots Shared rewards , , and These are data progress rewards, navigation progress rewards, safety rewards, and termination rewards.

[0110] Data progress rewards both relay provisioning and final delivery, for example: (20).

[0111] in The total amount of data for the task. For dimensionless data progress reward weights, To prevent the denominator from being zero and... Stability constants using the same data units.

[0112] The navigation progress bonus is constructed based on the total reduction in distance to their respective destinations, course consistency, turning radius, and unit time slot cost. The safety bonus is an immediate negative bonus applied when the navigation area is crossed, the safe distance to obstacles is insufficient, or the safe distance between the vessels is insufficient.

[0113] The termination reward is given when two boats arrive and all three types of queues are cleared, and early completion items can be added based on the time slot of completion; negative rewards are given when the time is exceeded, the boats do not arrive, or the queues are not cleared, and the negative rewards can be proportional to the total number of remaining queues.

[0114] In each sampling time slot, the two policy networks respectively... and The sampling action, the digital twin environment simulator 162 returns the shared reward and the next state, and transfers the tuple. Write track buffer 163. Among them, and The first policy network and the second policy network are respectively located in time slots. Normalized actions sampled from their respective Beta action distributions, with values ​​ranging from [value range]. And according to Equation (18), they are mapped to the heading adjustment amount of the first unmanned surface vessel and the second unmanned surface vessel respectively; The scalar shared instant reward calculated by the digital twin environment simulator 162 after two unmanned surface vessels perform two actions consists of a data progress reward, a navigation progress reward, a safety reward, and a termination reward.

[0115] For containing Each state transition and The trajectory segment of each state, with This represents the state transition index within a trajectory segment. To avoid confusion with the complete task time slot index... Confusion, in the following formula, the first segment within the trajectory segment The global state and shared reward of each sample are rewritten as follows: and Define time-series difference residuals and use generalized dominance estimation: (twenty one).

[0116] (twenty two).

[0117] (twenty three).

[0118] In equations (21) to (23), and The first segment within the trajectory segment The global state and shared reward of each sample. For time-series difference residuals, As a discount factor, For parameters of generalized dominance estimation, The length of the trajectory segment. To sum the offset index, For the sake of advantage estimation, For the purpose of reward; when At the corresponding termination time slot, the next state value term is set to zero. The central evaluation network minimizes... The mean squared error is updated. For each policy network, the probability ratio of the new and old policies is calculated, and the PPO pruning agent objective is maximized. The pruning interval is determined by... This is confirmed, and an entropy regularization term can be added to maintain the exploration.

[0119] The training process includes: initializing two policy networks and a central evaluation network; resetting the initial positions, queues, and environmental parameters of the two submarines; and collecting data. The joint trajectory of each time slot is calculated; the advantage estimate and reward objective are computed; the central evaluation network and the two policy networks are updated in mini-batch mode for multiple training epochs; sampling and updating are repeated until the reward, task success rate, and queue completion metrics converge. During training, the old policy parameters are only used to calculate probability ratios and are not reinitialized with each update.

[0120] An example training configuration is: discount factor GAE parameters PPO trimming parameters trajectory length The learning rates for both the policy network and the central evaluation network are... Each trajectory segment is updated 10 times, with a mini-batch size of 128 and an initial entropy coefficient of 0.005. The above parameters are for an implementable example and do not limit the scope of protection.

[0121] After training convergence, only the first policy network 125, the second policy network 135, and their corresponding normalized parameters are sent to the first shipborne controller 124 and the second shipborne controller 134. During the online phase, each vessel performs only one policy network forward operation, link gain query, queue update, and security correction per time slot.

[0122] Example 7: Simulation Verification In one example, the shore-based base station is located at The initial positions of the first and second unmanned surface vessels are respectively and The destinations are respectively and The respective navigation radius of the two boats is Seven obstacles were placed in the navigation area, with a safe distance between them. .

[0123] Example parameters include: evaporation waveguide height Unmanned surface vessel speed Time slot length Maximum number of time slots: 500; ocean currents direction and The directional components are respectively and Channel map horizontal grid width Vertical grid height carrier frequency ,bandwidth Task data volume .

[0124] Two-stage cooperative behavior can be observed after training convergence. While relaying data from the second unmanned surface vessel to the first unmanned surface vessel remains critical, the two vessels maintain a relatively favorable relative geometry along the coordination curve; once most of the relay data is completed, the two vessels gradually turn to a more direct destination route.

[0125] In a set of simulation statistics, the effective transmission time slots for the first and second links are approximately 29.1 and 57, respectively, while the navigation time slots for the two ships are approximately 101.7 and 104.8, respectively. These non-integer values ​​are the result of multiple evaluations or statistical averages and do not indicate the existence of a fractional number of time slots in a single discrete task.

[0126] The shortest feasible path baseline has fewer sailing time slots, but upon arrival, only about 49.7% of the data delivery for the first unmanned surface vessel (USV) and about 86% for the second USV are completed. The greedy baseline for instantaneous communication can improve the transmission of the current time slot, but it causes a significant detour. The reward fluctuations for independent PPO training are more pronounced. These results indicate that long-term joint consideration of link position correlation, three types of queues, and the relative geometry of the two USVs is beneficial for balancing complete delivery and sailing costs.

[0127] The results of reward ablation also indicate that removing data progress, navigation progress, or security rewards reduces mission completion stability or increases slot costs.

[0128] Example 8: Extension Method This invention can be extended to 1 unmanned surface vessel and one or more mobile relays, wherein the number of unmanned surface vessels is [number missing]. .

[0129] Mobile relay roles can be fixed or selected before the mission begins based on the initial queue, link coverage, and energy status. If a relay role is switched during the mission, the relay queue should be migrated or redefined simultaneously, and the outbound volume of each relay data stream should be constrained by the corresponding received volume.

[0130] The link gain query model can be further supplemented with time-stamped values ​​of the evaporation waveguide height over time, gain confidence intervals, or multiple meteorological scenarios. During training, environmental parameters can be randomized, and a robust strategy to waveguide uncertainties can be obtained through risk penalties.

[0131] The underlying navigation control can employ rudder angle control, differential propulsion, or a course autopilot. The outputs of the first and second strategy networks are not limited to course increments; they can also output target course or desired angular velocity, as long as they are converted into actual motion by the navigation control mechanism and meet steering and safety constraints.

[0132] Without departing from the core concept of this invention, the link resource method, queue service rules, hidden layer width of the policy network, security constraint execution method, and reward weight can be replaced with equivalent substitutions. The core combination of this invention lies in: characterizing two evaporating waveguide links by the combination of transmitter and receiver positions, maintaining mobile relay causality with three types of queues, and having two shipborne policy networks operate collaboratively in closed-loop wireless transmission and navigation control.

Claims

1. A method for coordinated control of an evaporating waveguide unmanned surface vessel (USV) with buffer relay, applied to a maritime communication system, wherein the maritime communication system includes a shore-based base station, a first USV, and a second USV, the first USV serving as a mobile relay from the second USV to the shore-based base station, the first USV being configured with a first local data queue and a relay data queue, and the second USV being configured with a second local data queue; (The following is a simplified version of the original text.) , and These respectively represent the first unmanned surface vessel, the second unmanned surface vessel, and the shore-based base station, in order to , and Representing the first local data queue, the relay data queue, and the second local data queue respectively, characterized in that, The method includes: S1, obtain the real-time position, heading, destination status, data queue status and navigation safety status of the first unmanned surface vessel and the second unmanned surface vessel; S2, using an evaporating waveguide link gain query module with the combination of the transmitter position and the receiver position as the query variable, obtain the first link gain of the first link from the second unmanned surface vessel to the first unmanned surface vessel and the second link gain of the second link from the first unmanned surface vessel to the shore-based base station. S3, respectively form a first local state vector and a second local state vector corresponding to the first unmanned surface vessel and the second unmanned surface vessel, input the first local state vector into the first strategy network to generate a first heading adjustment amount, input the second local state vector into the second strategy network to generate a second heading adjustment amount, and convert the corresponding heading adjustment amount into a heading control command and send it to the navigation control mechanism of the corresponding unmanned surface vessel; S4. Based on the first link gain, the second link gain, and the wireless resource configuration, determine the effective transmission volume of the first link and the effective transmission volume of the second link. Control the wireless transceiver of the second unmanned surface vessel to transmit data to the wireless transceiver of the first unmanned surface vessel that does not exceed the existing data volume of the second local data queue. Control the wireless transceiver of the first unmanned surface vessel to transmit data in the first local data queue and data in the relay data queue to the shore-based base station. Update the three types of data queues according to the measured transmission volume. In any time slot, the outbound volume of the relay data queue does not exceed the existing data volume of the relay data queue at the beginning of that time slot. S5, under the constraints of ocean current, turning, navigation area, obstacles and safe distance between boats, execute the course control command and update the position of the two unmanned surface vessels, return to step S1 to perform the next time slot control, until the two unmanned surface vessels have reached their respective destinations and all three types of data queues are cleared; The first local state vector and the second local state vector each include at least the motion state of the vessel, the navigation state related to the destination, the relative geometric state of the two vessels, the link state of at least one of the two links, the data queue state that the vessel can obtain, and the received fixed-dimensional task state summary; the first policy network and the second policy network are centrally trained by a central evaluation network using global state, and the central evaluation network does not participate in heading control during deployment.

2. The method for coordinated control of evaporating waveguide unmanned surface vessels with buffer relay according to claim 1, characterized in that, The evaporating waveguide link gain query module is represented as follows: ， in, For the three-dimensional coordinates of the transmitter, For the receiver's three-dimensional coordinates, The environmental parameter vector includes at least one of the following: evaporation waveguide characteristic height, carrier frequency, transmit antenna height, receive antenna height, meteorological refractive index profile, or sea surface state. The link gain query model stores the link gain using a transmitter-receiver position pair grid or a relative position grid, and obtains the link gain at any position through neighboring grid query, bilinear interpolation, or trilinear interpolation.

3. The method for coordinated control of evaporating waveguide unmanned surface vessels with buffer relay according to claim 1, characterized in that, The first and second links operate using different frequency bands, different time slots, or full-duplex resources calibrated by self-interference, ensuring that the effective transmission volume of the two links corresponds to the actual wireless resource configuration; for the link index... Link Index In the time slot Effective transmission rate according to It is confirmed that, among them, Link Index The radio resource coefficient within the time slot, For link bandwidth, For transmission power, and These are the transmit antenna gain and the receive antenna gain, respectively. For link gain, This represents the noise power spectral density.

4. The method for coordinated control of evaporating waveguide unmanned surface vessel buffer relay according to claim 3, characterized in that, Time slot Effective transmission volume of the first internal link Total effective transmission volume of the second link Each satisfies , Under the relay priority service rule, , and in accordance with ， Update the queue, where, and The first and second links are respectively in time slots The effective transmission rate, and The second link in time slots Of the total effective transmission volume, the portion allocated to the relay data queue and the first local data queue; This is the second local data queue. For the relay data queue, This is the first local data queue. This represents the time slot length.

5. The method for coordinated control of evaporating waveguide unmanned surface vessels with buffer relay according to claim 1, characterized in that, The total effective transmission volume of the second link is allocated between the relay data queue and the first local data queue according to the weighted queue service rule. The weighted queue service rule determines the relay service weight and the local service weight based on at least two of the following: queue occupancy ratio, remaining task time slots, first link rate, second link rate, destination distance, and service priority. Unused allocated capacity of one queue is reallocated to another non-empty queue, while ensuring that the outbound volume of each queue does not exceed the existing data volume of that queue at the beginning of the time slot.

6. The method for coordinated control of evaporating waveguide unmanned surface vessel buffer relay according to claim 4, characterized in that, make For the first Unmanned surface vessels according to time slots Initial heading And the corresponding heading adjustment amount, after being corrected for the preset heading range, the maximum turning angle of adjacent time slots, and navigation safety constraints, the actual heading executed in the time slot, then the first The location of the unmanned surface vessel according to Update, and make ,in, , For cruising speed, The current velocity component is used; the course correction is performed through motion projection, safety shield, or navigation controller to meet the constraints of navigation area, obstacle safety distance, and minimum safety distance between the two vessels.

7. The method for coordinated control of evaporating waveguide unmanned surface vessels with buffer relay according to claim 1, characterized in that, The first local state vector includes the position, heading, distance to the first destination, distance between the two vessels, first link status, second link status, first local data queue status, relay data queue status, second local data queue summary, heading error toward the first destination, normalized mission time, and obstacle features; The second local state vector includes the position, heading, distance to the second destination, distance between the two vessels, first link status, second link status, second local data queue status, relay data queue summary, first local data queue summary, heading error toward the second destination, normalized mission time, and obstacle characteristics. Among them, the second local data queue summary, which cannot be directly obtained by the first unmanned surface vessel, as well as the relay data queue summary, the first local data queue summary, and the second link status summary, which cannot be directly obtained by the second unmanned surface vessel, are all obtained by broadcasting in a fixed dimension through the mission status communication link. The mission status communication link is connected to the first shipborne controller and the second shipborne controller, respectively.

8. The method for coordinated control of evaporating waveguide unmanned surface vessels with buffer relay according to claim 1, characterized in that, The task cost is constructed using the completion time slots of the first and second unmanned surface vessels entering the destination tolerance range, the number of time slots with effective transmission on the first link, and the number of time slots with effective transmission on the second link. This is done while ensuring that both unmanned surface vessels enter the corresponding destination tolerance range. , and Setting all zeros as the condition for task success.

9. A buffer relay cooperative control system for evaporating waveguide unmanned surface vessels, characterized in that, It includes a shore-based base station, a first unmanned surface vessel, a second unmanned surface vessel, an evaporating waveguide link gain query module, and a mission status communication link; The shore-based base station includes a shore-based wireless transceiver; The first unmanned surface vessel includes a first wireless transceiver, a first memory, a first shipborne controller, a first navigation mechanism, and a first state perception unit. The first memory stores a first local data queue and a relay data queue, and the first shipborne controller stores a first policy network. The first shipborne controller is connected to the first wireless transceiver, the first memory, the first navigation mechanism, and the first state perception unit. The second unmanned surface vessel includes a second wireless transceiver, a second memory, a second shipboard controller, a second navigation mechanism, and a second state awareness unit. The second memory stores a second local data queue, and the second shipboard controller stores a second policy network. The second shipboard controller is connected to the second wireless transceiver, the second memory, the second navigation mechanism, and the second state awareness unit. The second wireless transceiver establishes a first link with the first wireless transceiver, and the first wireless transceiver establishes a second link with the shore-based wireless transceiver; The main transmission direction of task data for the first link is from the second wireless transceiver to the first wireless transceiver, and the main transmission direction of task data for the second link is from the first wireless transceiver to the shore-based wireless transceiver. The mission status communication link is connected to the first shipborne controller and the second shipborne controller respectively, and is used to transmit fixed-dimensional mission status summaries bidirectionally between the two unmanned surface vessels. The evaporating waveguide link gain query module is connected to the first shipborne controller and the second shipborne controller respectively, and performs query data interaction with the first shipborne controller and the second shipborne controller respectively, and outputs the first link gain and the second link gain according to the combination of the transmitter position and the receiver position. The first shipborne controller and the second shipborne controller respectively form a first local state vector and a second local state vector based on the motion state of the ship itself, the destination state, the relative geometric state of the two ships, the link state, the data queue state and the received task state summary provided by the corresponding state perception unit, and respectively generate a first heading control command and a second heading control command through the first policy network and the second policy network. The first shipborne controller and the second shipborne controller respectively control the corresponding navigation mechanism to execute course control commands, and recursively update the three types of data queues according to the actual transmission volume until both unmanned surface vessels have reached their respective destinations and the three types of data queues are cleared.

10. The evaporation waveguide unmanned surface vessel buffer relay cooperative control system according to claim 9, characterized in that, It also includes offline training equipment, which is connected to the first policy network and the second policy network respectively; the offline training equipment includes a digital twin environment simulator, a trajectory cache, a central evaluation network and a policy update module; The digital twin environment simulator is used to update the position, link status, and three types of data queues of the two unmanned surface vessels based on the motion status of the first and second unmanned surface vessels, the first and second link gains output by the evaporating waveguide link gain query module, the status of the first local data queue, the relay data queue, and the second local data queue, as well as navigation safety constraints, and to generate training transfer. The trajectory cache is connected to the digital twin environment simulator and the policy update module respectively, and is used to store the training transition. The training transition includes a first local state vector, a second local state vector, a global state vector composed of the first local state vector and the second local state vector, a first action output by the first policy network, a second action output by the second policy network, a shared reward, and the first local state vector, the second local state vector, and the global state vector of the next time slot. The policy update module is connected to the central evaluation network. The central evaluation network takes the global state vector as input and outputs the corresponding state value, and is only used for value estimation and credit allocation during the offline training phase. The policy update module is connected to the trajectory cache, the central evaluation network, the first policy network, and the second policy network, respectively. It is used to calculate the advantage estimate and reward target based on the training transition calculation in the trajectory cache, update the first policy network and the second policy network respectively using the near-end policy probability ratio pruning target, and update the central evaluation network based on the error between the state value and the reward target. The policy update module is also used to write the updated first policy network parameters and second policy network parameters into the first shipborne controller and the second shipborne controller respectively after training convergence, and the central evaluation network is not run during the online operation phase.

Images