A method and apparatus for persistent coverage observation by a swarm of underwater gliders

Abstract

This invention discloses a method and apparatus for continuous coverage observation of a cluster of underwater gliders, comprising: acquiring a cluster of underwater gliders, the cluster including several individual underwater gliders, and constructing motion models for each individual underwater glider based on a preset communication cycle; acquiring an observation area, determining the coverage area of ​​the cluster of underwater gliders in each communication cycle based on the observation area and the motion models of each individual underwater glider, and constructing a revisit period map and a priority coverage map based on the coverage area; constructing an agent for the underwater gliders based on preset underwater conditions, a target reinforcement learning algorithm, the revisit period map, and the priority coverage map, and performing continuous coverage observation of the observation area based on the agent of the underwater gliders; during the continuous coverage observation process, acquiring the state information of the cluster of underwater gliders, evaluating communication conditions based on the state information, and dynamically matching a target reinforcement learning algorithm based on the communication conditions.

Description

Technical Field

[0001] This invention relates to the field of underwater observation technology, and in particular to a method and apparatus for continuous coverage observation using a cluster of underwater gliders. Background Technology

[0002] Continuous coverage observation refers to the continuous monitoring of a specific target area to ensure that any location within the area can be re-covered within a specified maximum time threshold, thereby minimizing the longest revisit time for any area. Current technical methods mainly fall into the following categories: First, methods based on abstract path graphs. This method abstracts the continuous environment into a feasible path graph with several vertices and performs path planning for swarm robots based on this graph. By transforming the problem of continuous coverage into a problem of continuously visiting multiple vertices along feasible paths, the complexity of path planning in continuous space is simplified. Second, methods based on fixed region partitioning. This method first pre-divides the target area into several non-overlapping sub-regions, and then assigns a specific sub-region to each robot, enabling each robot to independently and continuously cover its assigned area. Region partitioning is usually statically based on factors such as environmental characteristics, coverage requirements, or robot performance. Third, methods based on dynamic region partitioning. This method dynamically partitions the observation area according to the robot's initial position and real-time state, and uses an iterative algorithm to allocate and adjust regions among robots, then constructs an optimized coverage path for each sub-region.

[0003] Existing continuous coverage observation methods for underwater gliders, based on abstract path graphs, introduce accuracy loss during discretization. Their coverage quality is highly dependent on the expressive power of the graph model and requires complete environmental information to be acquired in advance, making them difficult to adapt to unknown or large-scale complex scenarios. While fixed region partitioning strategies are simple and feasible, their coverage efficiency decreases due to heterogeneous robot performance or dynamic environmental changes. Furthermore, global partitioning must be redone every time the number of robots or task requirements change, failing to fully leverage the advantages of multi-robot collaboration. Although dynamic region partitioning methods are adaptive, frequent redistribution of responsibility areas can cause task switching jitter, increasing system computation and communication overhead. They also heavily rely on real-time communication, and message loss can lead to coverage failure and partition inconsistencies, thereby reducing the overall system stability and efficiency. Summary of the Invention

[0004] This invention provides a method and apparatus for continuous coverage observation by a cluster of underwater gliders, in order to improve the efficiency and adaptability of underwater gliders in continuous coverage observation missions in complex marine environments.

[0005] To address the aforementioned technical problems, this invention provides a method for continuous coverage observation using a cluster of underwater gliders, comprising:

[0006] Acquire a cluster of underwater gliders, which includes several individual underwater gliders, and construct a motion model for each individual underwater glider based on a preset communication cycle;

[0007] The observation area is obtained, and the coverage area of ​​the cluster of underwater gliders in each communication cycle is determined based on the observation area and the motion model of each individual underwater glider. A revisit cycle map and a priority coverage map are constructed based on the coverage area.

[0008] The agent of the underwater glider is constructed based on the preset underwater conditions, target reinforcement learning algorithm, revisit periodic map and priority coverage map, and the agent of the underwater glider performs continuous coverage observation of the observation area.

[0009] During continuous coverage observation, the status information of the cluster of underwater gliders is acquired, the communication conditions are evaluated based on the status information, and the target reinforcement learning algorithm is dynamically matched based on the communication conditions.

[0010] This invention generates motion models for each underwater glider based on a preset communication cycle and dynamically determines the coverage area within each communication cycle, thereby achieving refined allocation of coverage tasks. By introducing a revisit cycle map and a priority coverage map, coverage scheduling can take into account both regional revisit requirements and priority ranking, thus optimizing observation efficiency while ensuring continuous coverage. At the same time, by evaluating communication conditions in real time, reinforcement learning algorithms are dynamically matched to construct intelligent agents for continuous observation. This not only improves the autonomous scheduling capability of the glider swarm in complex underwater environments but also enhances the adaptability of the swarm of underwater gliders in changing communication conditions.

[0011] Furthermore, the communication conditions include primary communication, secondary communication, and tertiary communication; the step of acquiring the status information of the cluster of underwater gliders during continuous coverage observation, evaluating the communication conditions based on the status information, and dynamically matching the target reinforcement learning algorithm based on the communication conditions includes:

[0012] During continuous coverage observation, the status information of the cluster of underwater gliders is acquired, including the surface synchronization rate.

[0013] When the water surface synchronization rate is higher than or equal to a preset first threshold, the communication condition is level one communication;

[0014] When the water surface synchronization rate is lower than a preset first threshold and higher than a preset second threshold, the communication condition is secondary communication;

[0015] When the water surface synchronization rate is lower than or equal to a preset second threshold, the communication condition is level three communication;

[0016] A dynamic target reinforcement learning algorithm is used to match the communication conditions.

[0017] This invention defines three levels of communication conditions corresponding to surface synchronization rate and threshold, and accurately evaluates the communication level based on real-time status information. This enables adaptive adjustment of the glider swarm's cooperative mode during observation. The distinction between different communication conditions allows the system to dynamically match target reinforcement learning algorithms based on communication quality, ensuring basic information synchronization and task execution even when communication is limited. This effectively balances communication overhead and the continuity of coverage tasks, improving stability under varying hydrological conditions.

[0018] Furthermore, the reinforcement learning algorithm for dynamically matching targets based on the communication conditions includes:

[0019] When the communication condition is Level 1 communication, the target reinforcement learning algorithm is the PPO algorithm;

[0020] When the communication condition is two-level communication, the target reinforcement learning algorithm is the MAPPO algorithm;

[0021] When the communication condition is three-level communication, the target reinforcement learning algorithm is the IPPO algorithm.

[0022] This invention dynamically matches three target reinforcement learning algorithms—PPO, MAPPO, and IPPO—to different communication conditions, enabling glider swarms to finely adapt to communication resources and collaboration requirements. For Level 1 communication, a single-agent PPO algorithm is used to enhance individual decision-making by utilizing ample communication bandwidth. For Level 2 communication, a centrally trained MAPPO algorithm is applied to improve multi-aircraft collaboration efficiency through a global value function. For Level 3 communication, the system switches to an independently trained IPPO algorithm to ensure basic coverage is maintained even under extreme communication constraints.

[0023] Furthermore, the acquisition of the cluster of underwater gliders, which includes several individual underwater gliders, involves constructing motion models for each individual underwater glider based on a preset communication cycle, including:

[0024] An inertial coordinate system is constructed based on the horizontal plane and the preset coordinate axis directions;

[0025] Obtain a cluster of underwater gliders, and for any single underwater glider in the cluster, construct a sub-model of the single underwater glider's sailing speed, a sub-model of its heading angle, a sub-model of its ocean current speed, and a sub-model of its position change in each gliding cycle.

[0026] The motion model of the underwater glider is constructed based on the sailing speed sub-model, the heading angle sub-model, the ocean current speed component sub-model, and the position change sub-model.

[0027] This invention constructs an inertial coordinate system on a horizontal plane and in a preset coordinate system, and uses a sub-model of sailing speed, heading angle, ocean current speed component and position change as a basis to accurately describe the motion trajectory of a single glider in each gliding cycle.

[0028] Furthermore, the acquisition of the observation area, based on the observation area and the motion model of each individual underwater glider, determines the coverage area of ​​the cluster of underwater gliders in each communication cycle, and constructs a revisit cycle map and a priority coverage map based on the coverage area, including:

[0029] Obtain the observation area and its boundary size information, and discretize the observation area into several grid cells based on the boundary size information, the preset cell size and the Cartesian grid method;

[0030] Establish a two-dimensional index for each grid cell, and generate an observation grid model based on the grid cells and the two-dimensional index;

[0031] Based on the observation area and the motion model of each individual underwater glider, the coverage area of ​​the cluster of underwater gliders in each communication cycle is determined, and a revisit cycle map and a priority coverage map are constructed based on the coverage area.

[0032] This invention discretizes the observation area using Cartesian grids and determines the coverage cells using a single-machine motion model, thereby establishing a two-dimensional index and generating an observation grid model. This allows the determination of the coverage area, revisiting, and priority map construction to be performed quickly on the discrete cells.

[0033] Furthermore, the coverage area includes the grid cells covered in each communication cycle; determining the coverage area of ​​the cluster of underwater gliders in each communication cycle based on the observation area and the motion model of each individual underwater glider, and constructing a revisit cycle map and a priority coverage map based on the coverage area, includes:

[0034] Obtain a position change sub-model for each single underwater glider in each gliding cycle, and determine the first dive point and first surface point of the single underwater glider in the current communication cycle based on the position change sub-model.

[0035] Based on the first dive point and the first surfacing point, the coverage sub-region of the single underwater glider is determined, and the coverage sub-regions of all single underwater gliders are integrated to obtain the coverage area of ​​the cluster of underwater gliders in each communication cycle.

[0036] Based on the coverage area, a revisit cycle map and a priority coverage map are constructed.

[0037] This invention extracts the first dive point and first surfacing point of each glider within the current communication cycle, further subdivides the coverage sub-regions of each glider, and integrates them into a cluster. This accurately reflects the spatial range covered by the actual underwater movement trajectory of each glider. It improves the spatiotemporal accuracy of coverage area calculation and solves the problem of deviation between the coverage area and the actual trajectory in traditional coverage algorithms, thus ensuring that the construction of revisit period maps and priority coverage maps more closely matches real observation needs.

[0038] Furthermore, the construction of the revisit cycle map and the priority coverage map based on the coverage area includes:

[0039] Initialize a two-dimensional revisit periodogram, wherein the revisit periodogram includes the revisit values ​​of each grid cell of the observation grid model;

[0040] At the end of each communication cycle, all grid cells of the observation grid model are traversed. If the current grid cell is contained within the covered area, the revisit value of the grid cell is set to 1; otherwise, the revisit value of the grid cell is calculated based on a preset first attenuation factor.

[0041] When it is detected that any revisit value in the revisit periodogram decays to 0 or the cumulative communication cycle reaches a preset first revisit threshold, the revisit periodogram construction is completed.

[0042] A priority coverage map is constructed based on the revisit periodic map and a preset second revisit threshold.

[0043] This invention achieves real-time monitoring and attenuation processing of the revisit status of each grid cell by initializing and iteratively updating the revisit periodicity map based on the coverage area. Setting revisit value reset and attenuation factor not only dynamically reflects the coverage frequency, but also triggers the reconstruction of the revisit periodicity map when necessary, ensuring that the system can promptly schedule supplementary coverage for areas that have not been visited for a long time.

[0044] Furthermore, the step of constructing a priority coverage map based on the revisit periodicity map and a preset second revisit threshold includes:

[0045] Initialize a two-dimensional priority coverage map, which includes the priority coverage value of each grid cell of the observation grid model;

[0046] At the end of each communication cycle, the revisit cycle graph is traversed. If the revisit value of the current grid cell is less than the preset second revisit threshold, the priority coverage value of the grid cell is set to 1; otherwise, it is set to 0.

[0047] This invention quantifies and sorts the priority of each grid cell by using a revisit periodicity map and a second revisit threshold priority coverage map. By combining threshold determination with attenuation calculation, it can quickly identify low-revisit value cells that require urgent revisiting, while maintaining a balanced scheduling of priority coverage needs for the entire area.

[0048] Furthermore, the underwater conditions include still water conditions and ocean current conditions; the construction of the underwater glider's agent based on the preset underwater conditions, the target reinforcement learning algorithm, the revisit periodic map, and the priority coverage map, and the continuous coverage observation of the observation area based on the underwater glider's agent, includes:

[0049] In the inertial coordinate system, the state space is defined by the position coordinates of the cluster of underwater gliders, the current heading angle of the cluster of underwater gliders, the revisit period map, the priority coverage map, and the ocean current velocity component; when the underwater conditions are still water conditions, the ocean current velocity component is set to 0.

[0050] The observation space is defined by the position coordinates of a single underwater glider, the current heading angle of the single underwater glider, the revisit period map, the priority coverage map, and the ocean current velocity component; when the underwater conditions are still water conditions, the ocean current velocity component is set to 0.

[0051] The heading angle variation step size is preset based on the maneuverability of a single underwater glider, and the action space is constructed based on the heading angle variation step size. The action space includes all heading angles that the underwater glider can execute.

[0052] A reward function is constructed based on a pre-defined differential reward function and a boundary penalty function;

[0053] The intelligent agent of the underwater glider is constructed based on the state space, observation space, action space, reward function, and target reinforcement learning algorithm.

[0054] The intelligent agent performs continuous coverage observation of the observation area.

[0055] This invention defines the state and observation space by combining the glider's position, heading, graphical model, and ocean current velocity components, and constructs the action space by combining the heading angle step size preset with maneuverability performance. This provides a comprehensive and accurate basis for reinforcement learning agents to perceive the environment and make action decisions. By introducing differential reward and boundary penalty functions, the glider can be effectively guided to prioritize covering high-priority cells and avoid going out of bounds during training.

[0056] In a second aspect, the present invention provides a continuous coverage observation device for clustered underwater gliders, comprising: a motion model construction module, a revisit periodic map construction module, a continuous coverage observation module, and an algorithm adjustment module;

[0057] The motion model construction module is used to acquire a cluster of underwater gliders, which includes several individual underwater gliders, and to construct motion models for each individual underwater glider based on a preset communication cycle.

[0058] The revisit period map construction module is used to obtain the observation area, determine the coverage area of ​​the cluster of underwater gliders in each communication cycle based on the observation area and the motion model of each individual underwater glider, and construct a revisit period map and a priority coverage map based on the coverage area.

[0059] The continuous coverage observation module is used to construct the agent of the underwater glider based on preset underwater conditions, target reinforcement learning algorithm, the revisit periodic map and the priority coverage map, and to perform continuous coverage observation of the observation area based on the agent of the underwater glider.

[0060] The algorithm adjustment module is used to acquire the status information of the cluster of underwater gliders during continuous coverage observation, evaluate the communication conditions based on the status information, and dynamically match the target reinforcement learning algorithm based on the communication conditions. Attached Figure Description

[0061] Figure 1 A flowchart illustrating a method for continuous coverage observation using clustered underwater gliders, provided in an embodiment of the present invention;

[0062] Figure 2 A schematic diagram of the communication cycle of a cluster of underwater gliders in a discrete observation area, provided in an embodiment of the present invention;

[0063] Figure 3 A schematic diagram of the coverage area of ​​a cluster of underwater gliders in a discrete observation area, provided in an embodiment of the present invention;

[0064] Figure 4 This is a schematic diagram showing the decomposition of ocean current velocity in the direction of the single underwater glider's flight speed according to an embodiment of the present invention. Detailed Implementation

[0065] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0066] The terms "first" and "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0067] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0068] Example 1

[0069] See Figure 1 , Figure 1 This is a flowchart illustrating a method for continuous coverage observation using swarm underwater gliders, provided in an embodiment of the present invention. The method includes steps 101 to 104, as detailed below:

[0070] Step 101: Obtain a cluster of underwater gliders, which includes several individual underwater gliders. Construct motion models for each individual underwater glider based on a preset communication cycle.

[0071] In this embodiment, the acquisition of a cluster of underwater gliders, which includes several individual underwater gliders, involves constructing a motion model for each individual underwater glider based on a preset communication cycle, including:

[0072] An inertial coordinate system is constructed based on the horizontal plane and the preset coordinate axis directions;

[0073] Obtain a cluster of underwater gliders, and for any single underwater glider in the cluster, construct a sub-model of the single underwater glider's sailing speed, a sub-model of its heading angle, a sub-model of its ocean current speed, and a sub-model of its position change in each gliding cycle.

[0074] The motion model of the underwater glider is constructed based on the sailing speed sub-model, the heading angle sub-model, the ocean current speed component sub-model, and the position change sub-model.

[0075] In this embodiment, the cluster of underwater gliders includes several individual underwater gliders. Each individual underwater glider is simplified as a two-dimensional point mass model, moving at a constant speed v. glider Navigation. A gliding cycle consists of a complete descent and ascent. Three consecutive gliding cycles are defined as one communication cycle T. com .

[0076] In this embodiment, an inertial coordinate system O-xy is established in the horizontal plane, with the x-axis pointing east and the y-axis pointing north.

[0077] In this embodiment, the position of the underwater glider in the inertial coordinate system at time t is represented by a vector. Indicates. In position At this point, the motion of the underwater glider is determined by its own velocity vector. and ocean current velocity vector Joint decision, combined speed Represented as:

[0078]

[0079] The resultant velocity vector In the inertial coordinate system O-xy, it is decomposed into velocity components v in the x and y directions. x and v y ,Right now

[0080] v x =v glider ·cos(φ)+v cx (2)

[0081] v y =v glider sin(φ)+v cy (3)

[0082] Among them, v glider The speed of the underwater glider in still water is given by φ, the heading angle is given by v. cx and v cy Let T represent the velocity components of the ocean current along the x and y axes in an inertial coordinate system. After one communication cycle T... com Subsequently, the underwater glider's location was updated to:

[0083]

[0084] In this embodiment, the system first constructs an inertial coordinate system O-xy in the horizontal plane with the eastward direction as the x-axis and the northward direction as the y-axis, and then acquires the cluster of underwater gliders, which includes several individual underwater gliders. For each individual underwater glider, a preset communication period T is established. comUsing three consecutive gliding cycles as a communication cycle, at a constant still water speed v glider Establish a sub-model for navigation speed, and a sub-model for navigation angle using the navigation angle φ, and a sub-model for the velocity components of the ocean current in the inertial coordinate system. Establish a quantum model of ocean current velocity components, and use the resultant velocity as the basis for the model. The system constructs a sub-model of position changes, along with its components. These four sub-models work together to form a two-dimensional point mass motion model of a single glider. By constructing this motion model for each individual glider within the cluster, the system can accurately predict the position distribution of each glider at the end of each communication cycle, providing a reliable kinematic basis for subsequent coverage area planning and graph model construction.

[0085] In this embodiment, a gliding cycle for each single underwater glider includes a complete dive and ascent process, and three consecutive gliding cycles constitute a communication cycle. This division makes periodic task management possible, facilitating the segmentation of the entire observation mission into several relatively independent cycles for control and evaluation. Within a communication cycle, data can be aggregated and strategies adjusted based on the cumulative effects of multiple gliding cycles, improving the stability of the coverage mission.

[0086] Step 102: Obtain the observation area, determine the coverage area of ​​the cluster of underwater gliders in each communication cycle based on the observation area and the motion model of each individual underwater glider, and construct a revisit cycle map and a priority coverage map based on the coverage area;

[0087] In this embodiment, the step of acquiring the observation area, determining the coverage area of ​​the cluster of underwater gliders in each communication cycle based on the observation area and the motion model of each individual underwater glider, and constructing a revisit cycle map and a priority coverage map based on the coverage area, includes:

[0088] Obtain the observation area and its boundary size information, and discretize the observation area into several grid cells based on the boundary size information, the preset cell size and the Cartesian grid method;

[0089] Establish a two-dimensional index for each grid cell, and generate an observation grid model based on the grid cells and the two-dimensional index;

[0090] Based on the observation area and the motion model of each individual underwater glider, the coverage area of ​​the cluster of underwater gliders in each communication cycle is determined, and a revisit cycle map and a priority coverage map are constructed based on the coverage area.

[0091] In this embodiment, the observation area and its boundary size information are obtained. The boundary size information of the observation area includes an L×L rectangular observation area. A Cartesian grid-based method is used to model the observation area. The L×L rectangular observation area is discretized into m×m grid cells using the Cartesian grid method. r cell Define the cell size. Assign a unique two-dimensional index γ to each grid cell. i,j ,i,j∈{1,…,m}, are used to locate the positions of grid cells.

[0092] In this embodiment, the Cartesian grid method is used to accurately model the observation area, which helps to discretize the continuous space into manageable grid cells, facilitating subsequent path planning and coverage monitoring. This improves the applicability and computational efficiency of the algorithm. Furthermore, by clearly defining the boundary dimensions and cell dimensions of the observation area, standardized discretization of the area is achieved, ensuring the accuracy and consistency of grid division and providing a reliable foundation for subsequent path planning and coverage monitoring.

[0093] In this embodiment, the coverage area includes the grid cells covered in each communication cycle; determining the coverage area of ​​the cluster of underwater gliders in each communication cycle based on the observation area and the motion model of each individual underwater glider, and constructing a revisit cycle map and a priority coverage map based on the coverage area, includes:

[0094] Obtain a position change sub-model for each single underwater glider in each gliding cycle, and determine the first dive point and first surface point of the single underwater glider in the current communication cycle based on the position change sub-model.

[0095] Based on the first dive point and the first surfacing point, the coverage sub-region of the single underwater glider is determined, and the coverage sub-regions of all single underwater gliders are integrated to obtain the coverage area of ​​the cluster of underwater gliders in each communication cycle.

[0096] Based on the coverage area, a revisit cycle map and a priority coverage map are constructed.

[0097] In this embodiment, a position change sub-model of a single underwater glider in each gliding cycle is obtained, and the first dive point and the first surfacing point of the underwater glider in the current communication cycle are determined based on the position change sub-model; the coordinate positions of the first dive point and the first surfacing point are obtained, and the coverage area is determined based on the coordinate positions.

[0098] Please refer to Figure 2 and Figure 3 , Figure 2 A schematic diagram of the communication cycle of a cluster of underwater gliders in a discrete observation area, provided in an embodiment of the present invention; Figure 3This is a schematic diagram of the coverage area of ​​a cluster of underwater gliders in a discrete observation area, provided in an embodiment of the present invention.

[0099] In this embodiment, within each communication cycle, a single underwater glider executes a series of consecutive gliding cycles according to preset parameters. Each gliding cycle includes a complete dive and ascent process. The system acquires real-time position change data of the single underwater glider during each gliding cycle through underwater positioning devices and sensors. Specifically, using a pre-established motion model, the position of the single underwater glider at various moments in the inertial coordinate system (e.g., X-axis pointing east, Y-axis pointing north) is determined, and its initial and final position changes within each gliding cycle are recorded.

[0100] In this embodiment, based on the aforementioned location information, the embodiment utilizes position change data within continuous gliding cycles to select two key points within the communication cycle: The first dive point: refers to the position when the single underwater glider first enters a dive state within the communication cycle; this position typically represents the initial dive boundary for the entire cycle. The first ascent point: refers to the final position reached by the single underwater glider upon completing its ascent within the communication cycle; this position reflects the ascent completion state within the current cycle. Through comprehensive analysis of displacement data in each gliding cycle, the specific coordinate values ​​of these two points can be determined. These key points provide stable geometric boundaries for the subsequent construction of the coverage area.

[0101] In this embodiment, after obtaining the first dive point and the first surfacing point, the next step is to determine the coverage area on the horizontal plane. Specifically, based on the coordinate positions of the first dive point and the first surfacing point, their positions in the inertial coordinate system are projected. To ensure that the coverage area can fully represent the trajectory of the single underwater glider throughout the entire communication cycle, a geometric shape (e.g., a minimum rectangle) formed by connecting the two points is typically used as the initial coverage area.

[0102] In this embodiment, the coverage area is the set of all projected areas from the first dive point to the first surfacing point. The boundary of this area can be determined based on the extreme values ​​of the first dive point and the first surfacing point in the horizontal plane, thus considering that the trajectory of the glider falls within this area throughout the entire communication cycle.

[0103] In this embodiment, to facilitate subsequent observation data statistics and coverage status assessment, the determined coverage area is further discretized using a pre-established observation grid model: the coverage area is mapped into several discrete grid cells using an observation grid model constructed based on the Cartesian grid method. Each grid cell within the coverage area is considered an independent coverage cell, and all grid cells falling within the coverage area constitute a set of coverage cells.

[0104] In this embodiment, by accurately calculating the positions of the dive point and the surfacing point, the coverage area is determined, ensuring the comprehensiveness and effectiveness of the observation mission and improving the quality and reliability of the observation data.

[0105] In this embodiment, the observation area of ​​a single underwater glider is determined within each communication cycle. The single underwater glider departs from diving point A at time t and travels through one communication cycle T. com The glider then reaches surface point B. During this period, the coverage sub-region of the underwater glider is defined as the largest rectangular traversal area between point A and point B for a single underwater glider. The grid cells within the coverage sub-region constitute the coverage cell set K. n ={γ i,j}, where i∈{k,…,l}, j∈{g,…,f}, and the row and column range (k,l,g,f) of the rectangular region is uniquely determined by the positions of points A and B. The coverage sub-region K of each underwater glider... n This constitutes the communication cycle T of the cluster of underwater gliders. com Coverage area K = K 1 ∪K 2 …∪K n .

[0106] In this embodiment, constructing a revisit cycle map and a priority coverage map based on the coverage area includes:

[0107] Initialize a two-dimensional revisit periodogram, wherein the revisit periodogram includes the revisit values ​​of each grid cell of the observation grid model;

[0108] At the end of each communication cycle, all grid cells of the observation grid model are traversed. If the current grid cell is contained within the covered area, the revisit value of the grid cell is set to 1; otherwise, the revisit value of the grid cell is calculated based on a preset first attenuation factor.

[0109] When it is detected that any revisit value in the revisit periodogram decays to 0 or the cumulative communication cycle reaches a preset first revisit threshold, the revisit periodogram construction is completed.

[0110] A priority coverage map is constructed based on the revisit periodic map and a preset second revisit threshold.

[0111] In this embodiment, the set of covered sub-regions of the observation grid model for all single underwater gliders in each communication cycle is statistically analyzed to form a covered region, and a revisit periodicity map of the observation region is constructed based on the covered region.

[0112] In this embodiment, a revisit periodogram M is established to quantify the persistent coverage of the observation area. Each element M(γ) in the revisit periodogram M... i,j ) represents the mesh element γ i,jThe revisit situation, including γ i,j This represents the grid cell located in the i-th row and j-th column of the observation region. In each communication cycle T... com Within, for any mesh element γ i,j If it is covered by an underwater glider, then M(γ) i,j M(γ) is reset to 1; otherwise, M(γ) is reset to 1. i,j The value of ) will decrease by a decay factor δ. Accordingly, the revisit periodogram M is calculated in each communication period T. com At the end, update according to the following equation:

[0113]

[0114] In this embodiment, during the system initialization phase, a two-dimensional revisit periodicity diagram corresponding to the observation area is established based on the previously constructed observation grid model using the Cartesian grid method. Each element in the revisit periodicity diagram corresponds one-to-one with a corresponding grid cell in the observation grid model. Each element records the revisit value of that grid cell, i.e., the state maintenance time index of that cell after its last coverage within a continuous communication cycle. The revisit periodicity diagram can be stored in the form of a two-dimensional array, with each array element initialized to a predefined initial value (e.g., "1" or other suitable value), indicating that each grid cell is initially in a "just covered" state. This initial value helps distinguish cells that have not been covered or have been covered for too long during subsequent updates.

[0115] In this embodiment, after each preset communication cycle, the system traverses all grid cells in the entire observation grid model, comparing the area covered by the underwater glider in the current cycle with the distribution of discrete grid cells, thereby updating the value of each grid cell in the revisit period map. During the traversal, for each grid cell, it checks whether it is included in the coverage area (covered cell set) formed by the underwater glider in the current cycle. If the grid cell is covered in the current communication cycle, it means that its latest "revisit" status has been refreshed, and the revisit value corresponding to the grid cell is reset to "1".

[0116] If a grid cell is not covered, the current revisit value is updated according to a preset attenuation factor. For example, the revisit value of the grid cell can be gradually reduced by multiplying by an attenuation coefficient or subtracting a fixed attenuation amount until a certain threshold is reached.

[0117] In this embodiment, constructing a priority coverage map based on the revisit periodicity map and a preset second revisit threshold includes:

[0118] Initialize a two-dimensional priority coverage map, which includes the priority coverage value of each grid cell of the observation grid model;

[0119] At the end of each communication cycle, the revisit cycle graph is traversed. If the revisit value of the current grid cell is less than the preset second revisit threshold, the priority coverage value of the grid cell is set to 1; otherwise, it is set to 0.

[0120] In this embodiment, in order to effectively identify the local areas that urgently need priority coverage during the continuous coverage observation of the cluster of underwater gliders, the system is based on the constructed revisit period map M(γ) i,j And a priority coverage graph is constructed using the preset second revisit threshold ξ. First, a two-dimensional priority coverage map G is initialized. This map corresponds one-to-one with the observation grid model, recording the priority coverage value of each grid cell. Then, in each communication cycle T... com After completion, the system iterates through the revisit period values ​​corresponding to all grid cells. For a given grid cell γ... i,j If its current revisit value If the value is less than the set priority coverage threshold ξ, the cell is considered to have been uncovered for a long time and should be assigned the highest priority. In this case, the priority coverage value of the cell is set to 1. Otherwise, it is set to 0 to reflect the increasing priority trend caused by coverage lag in that area. In this way, by combining revisit frequency information and coverage attenuation strategy, the priority coverage map can dynamically reflect the coverage urgency of each grid cell, providing an accurate reference for coverage priority in subsequent reinforcement learning strategy action selection, enhancing the system's ability to focus on key areas, and thus improving the rationality and scientific nature of the overall observation coverage.

[0121] In this embodiment, the termination conditions for a single underwater glider's continuous coverage mission are defined. Mission termination is determined in the following two cases: when any grid cell γ... i,j When the revisit period value decays to 0, it indicates that the underwater glider failed to revisit any location within the specified maximum time threshold, meaning that the time interval between two consecutive revisits in a local area exceeds the specified maximum time threshold, which is considered a mission failure and terminates. When the communication period reaches the preset maximum value T... max When the time is right, the task is considered successful and terminated.

[0122] Step 103: Construct the agent of the underwater glider based on the preset underwater conditions, target reinforcement learning algorithm, the revisit periodic map and the priority coverage map, and perform continuous coverage observation of the observation area based on the agent of the underwater glider;

[0123] In this embodiment, the underwater conditions include still water conditions and ocean current conditions; the construction of the underwater glider agent based on the preset underwater conditions, the target reinforcement learning algorithm, the revisit periodic map, and the priority coverage map, and the continuous coverage observation of the observation area based on the underwater glider agent, includes:

[0124] In the inertial coordinate system, the state space is defined by the position coordinates of the cluster of underwater gliders, the current heading angle of the cluster of underwater gliders, the revisit period map, the priority coverage map, and the ocean current velocity component; when the underwater conditions are still water conditions, the ocean current velocity component is set to 0.

[0125] The observation space is defined by the position coordinates of a single underwater glider, the current heading angle of the single underwater glider, the revisit period map, the priority coverage map, and the ocean current velocity component; when the underwater conditions are still water conditions, the ocean current velocity component is set to 0.

[0126] The heading angle variation step size is preset based on the maneuverability of a single underwater glider, and the action space is constructed based on the heading angle variation step size. The action space includes all heading angles that the underwater glider can execute.

[0127] A reward function is constructed based on a pre-defined differential reward function and a boundary penalty function;

[0128] The intelligent agent of the underwater glider is constructed based on the state space, observation space, action space, reward function, and target reinforcement learning algorithm.

[0129] The intelligent agent performs continuous coverage observation of the observation area.

[0130] In this embodiment, a state space is created. The underwater gliders are mapped to reinforcement learning agents, and the total number of underwater gliders in the swarm is N. State space. It must include the two-dimensional ocean current field within the observation area. The ocean current velocity component v at each location in the m×m grid is... cx and v cy After flattening, we get v fcx ,v fcy This is then added to the state space. At time t, the state is s. t =(x 1 ,y 1 ,φ 1 …x n ,y n ,φ n M f G f ,v fcx ,v fcy In still water, vfcx ,v fcy =0.

[0131] In this embodiment, to comprehensively reflect the operational status of the underwater glider in a complex marine environment, the real-time position coordinates of the underwater glider (e.g., represented by eastward and northward components) are obtained in an inertial coordinate system, along with its current heading angle, to ensure the direction of motion is indicated. Furthermore, a vector composed of the revisit values ​​of each grid cell in the revisit periodogram reflects the cumulative time since the most recent coverage of each location within the observation area. Combined with the velocity components of the ocean currents along the x and y axes, a multi-dimensional state description is formed, encompassing static position, dynamic trajectory, environmental coverage status, and external disturbances. This allows the agent to consider uneven coverage and the influence of dynamic ocean currents during decision-making, thereby improving the global optimization performance and adaptability of the strategy.

[0132] In this embodiment, the observation space is defined. state space It must include the two-dimensional ocean current field within the observation area. The ocean current velocity component v at each location in the m×m grid is... cx and v cy After flattening, you get v fcx ,v fcy To join the observation space, i.e. o t =(x n ,y n ,φ n M f G f ,v fcx ,v fcy In still water, v fcx ,v fcy =0.

[0133] In this embodiment, a discrete action space is defined. The action space is constituted by the discretized heading angle φ of the underwater glider. Based on the maneuverability of the underwater glider, the heading angle is discretized at equal intervals of i° within the range of [0°, 359°] to construct the action space, i.e.

[0134] In this embodiment, the control actions of the underwater glider are limited to adjusting the heading angle. To adapt to actual underwater maneuverability, a heading angle adjustment step size is preset (e.g., 5° or 10° each time), thereby discretizing the continuous heading angle changes into a finite set of actions. This discretization method reduces the dimensionality of the action space, facilitating the training of reinforcement learning algorithms and policy search, while ensuring that the underwater glider maintains sufficient control accuracy when performing actions, thus achieving a more stable and controllable coverage strategy.

[0135] In this embodiment, the reward function r o Differential reward r d and boundary penalty r p It consists of two parts.

[0136] In this embodiment, different target reinforcement learning algorithms all employ a shared reward function mechanism, meaning that each underwater glider receives the same reward. The reward function r... o It can be represented as:

[0137] r o =r d +r p (7)

[0138] r d =k1·∑(1-M) t (γ i,j )), γ i,j ∈K (8)

[0139] Among them, the differential reward r d The boundary penalty r is calculated based on the change in the revisit periodicity graph M before and after the operation of the clustered underwater gliders. p Adaptive penalty mechanism:

[0140]

[0141] in, and represents the horizontal and vertical distances of the underwater glider n from the boundary of the observation area, respectively; L is the length of the observation area; and k1 and k2 are weighting coefficients.

[0142] In this embodiment, the differential reward is calculated based on the changes in the revisit period map within the current communication cycle. If the underwater glider's actions cause the revisit value of some areas to rise from a smaller old value to a higher value, it indicates that the area has been covered in a timely manner, and a positive reward is given; conversely, if the change in the area's revisit value is small, the reward is lower.

[0143] In this embodiment, to prevent the underwater glider from deviating from the observation area, a penalty function based on the distance of the underwater glider from the preset observation area boundary is designed. Once the underwater glider deviates from the area, corresponding penalties will be triggered according to its horizontal and vertical distance from the boundary, ensuring that the agent prioritizes the coverage effect within the area when selecting actions.

[0144] In this embodiment, the reward function is constructed by differential reward and boundary penalty, so that the reinforcement learning agent can obtain clear positive and negative feedback during the training process, thereby continuously correcting the strategy and tending towards the optimal coverage scheme.

[0145] In this embodiment, by introducing a reinforcement learning framework, the underwater glider can autonomously learn the optimal coverage strategy, improve its adaptability and decision-making ability in complex environments, and achieve efficient and continuous coverage observation.

[0146] In this embodiment, the state space, discrete action space, and reward function are used as the input and feedback of the reinforcement learning model. An agent is constructed using a reinforcement learning framework (such as a policy gradient-based method or the PPO algorithm). Specific steps include: defining a policy network and a value network, and using model structures such as neural networks to approximate the optimal policy. In each training cycle, the agent takes actions in the environment according to the current policy and collects state transition data and reward feedback; by calculating the advantage estimate and continuously updating the policy and value function using the policy gradient algorithm, the model gradually converges to the optimal policy. During training, the agent achieves adaptive learning to environmental changes by iteratively optimizing the objective function.

[0147] In this embodiment, the agent of the underwater glider is constructed based on the state space, observation space, action space, reward function, and target reinforcement learning algorithm. Specifically, this includes constructing a hybrid neural network, initializing the policy network parameters and value network parameters of the hybrid neural network, and training the hybrid neural network based on the agent's state space, observation space, action space, and reward function. In each training cycle, the agent is controlled to continuously cover observation actions, record the agent's state transition data, add the state transition data to the experience replay pool, randomly sample some data from the experience replay pool to calculate the advantage estimate, and update the policy network parameters and value network parameters based on the advantage estimate until a preset condition is reached. The optimal policy network parameters and value network parameters are then saved, and the agent of the cluster underwater glider is output.

[0148] In this embodiment, a hybrid neural network is established to process state or observation information generated by various algorithms. The hybrid neural network consists of three main modules: a convolutional feature extractor, an embedding module, and a fusion module. Each module performs a different function in the entire network, working together to achieve feature extraction from the input state information, data fusion, and the final output of policy and value estimation.

[0149] In this embodiment, the convolutional feature extractor uses the revisit periodic map M and the priority coverage map G as inputs (which can be expanded to multi-channel input, such as 4 channels, in ocean current environments) to capture the coverage duration and dynamic change features of each grid cell within the observation area. It comprises a three-layer convolutional network: the first convolutional layer has 2 input channels (4 input channels in ocean current environments), 8 output channels, a 3×3 kernel size, and a stride of 2. After convolution, it is activated by the ReLU function and then fed into the second layer; the second convolutional layer has 8 input channels, 16 output channels, a 3×3 kernel size, and a stride of 2. After convolution, it is activated by the ReLU function and then fed into the third layer; the third convolutional layer has 16 input channels, 32 output channels, a 3×3 kernel size, and a stride of 2. After convolution, it is activated by the ReLU function, and the result is flattened into a one-dimensional vector.

[0150] In this embodiment, a three-layer convolutional network is used to extract features from the revisit periodic map M step by step, which helps to capture the spatial distribution features of the coverage state within the region and provides rich visual information for subsequent decision-making.

[0151] In this embodiment, the embedded module g e A single-layer fully connected network is used, with 3 input dimensions and 64 output dimensions.

[0152] In this embodiment, the fusion module g f Convolutional feature extractor g m and embedded module g e After the outputs are merged, they are processed through a three-layer fully connected network: the first fully connected layer has an output dimension of 64, and then passes through the Tanh activation function; the second fully connected layer has an output dimension of 64, and then passes through the Tanh activation function; for the policy network, the output dimension of the third fully connected layer is equal to the action space dimension; for the value network, the output dimension of the third fully connected layer is equal to 1.

[0153] In this embodiment, an experience replay pool is created. The experience replay pool is initialized to store data on the agent's interactions with the environment, including observation data, action data, logarithms of action probabilities, reward data, termination flags, and state values. Data is sampled in mini-batch mode for network parameter updates. After the data is used up, the replay pool is cleared and data is collected again.

[0154] In this embodiment, data is randomly sampled from the experience replay pool using a mini-batch method for updating network parameters. After each training round, the experience data is utilized, the replay pool is cleared after the update is completed, and data is re-collected for the next round to ensure data freshness and diversity.

[0155] Step 104: During continuous coverage observation, acquire the status information of the cluster of underwater gliders, evaluate the communication conditions based on the status information, and dynamically match the target reinforcement learning algorithm based on the communication conditions.

[0156] The communication conditions include primary, secondary, and tertiary communication; the process of acquiring the status information of the cluster of underwater gliders during continuous coverage observation, evaluating the communication conditions based on the status information, and dynamically matching the target reinforcement learning algorithm based on the communication conditions includes:

[0157] During continuous coverage observation, the status information of the cluster of underwater gliders is acquired, including the surface synchronization rate.

[0158] When the water surface synchronization rate is higher than or equal to a preset first threshold, the communication condition is level one communication;

[0159] When the water surface synchronization rate is lower than a preset first threshold and higher than a preset second threshold, the communication condition is secondary communication;

[0160] When the water surface synchronization rate is lower than or equal to a preset second threshold, the communication condition is level three communication;

[0161] A dynamic target reinforcement learning algorithm is used to match the communication conditions.

[0162] In this embodiment, the reinforcement learning algorithm for dynamically matching targets based on the communication conditions includes:

[0163] When the communication condition is Level 1 communication, the target reinforcement learning algorithm is the PPO algorithm;

[0164] When the communication condition is two-level communication, the target reinforcement learning algorithm is the MAPPO algorithm;

[0165] When the communication condition is three-level communication, the target reinforcement learning algorithm is the IPPO algorithm.

[0166] In this embodiment, during continuous coverage observation, the status information of the underwater gliders in the cluster (especially the surface synchronization rate) is acquired in real time, and the communication condition level is dynamically evaluated based on this status information: when the surface synchronization rate is ≥ the first threshold, it is determined to be Level 1 communication (good condition), at which time the cluster can maintain synchronous ascent within the time window and achieve real-time sharing of full status information; when the surface synchronization rate is between the second threshold and the first threshold, it is determined to be Level 2 communication (partially restricted condition), at which time synchronous communication can be maintained during the training phase, but only limited shared information can be acquired asynchronously during the execution phase; when the surface synchronization rate is ≤ the second threshold, it is determined to be Level 3 communication (strictly restricted condition), and both the training and execution phases can only rely on individual local observation information. The reinforcement learning algorithm is dynamically matched based on the evaluation results: Level 1 communication uses the PPO (Proximal Policy Optimization) algorithm to achieve centralized decision-making; Level 2 communication uses the MAPPO (Multi-Agent PPO) algorithm to support collaborative learning with partial information sharing; Level 3 communication switches to the IPPO (Independent PPO) algorithm to achieve fully distributed autonomous decision-making. This dynamic matching mechanism significantly improves the adaptability of cluster collaborative observation under different communication constraints, and in particular ensures the robustness of the algorithm when communication conditions deteriorate.

[0167] In this embodiment, the surface synchronization rate refers to whether multiple gliders can surface simultaneously within the same time window to exchange information effectively. The surface synchronization rate directly determines the synchronicity and timeliness of information acquisition within the cluster and is a key indicator for evaluating the quality of communication conditions.

[0168] In this embodiment, different reinforcement learning algorithms are matched based on communication conditions for continuous coverage task training. The total number of training steps is set to T. During each training cycle, the underwater glider interacts with the environment to obtain state information s. t The discrete action probabilities are obtained by inputting the policy network, and the action a is obtained by sampling. t After performing the action, a new state s is obtained. t+1 Determine if the round end condition is met, and store all status, action, and reward information into the experience replay pool. Retrieve data from the experience pool in small batches (b). mini The policy network and value network are updated with sampled data, and the experience pool is cleared after the update is complete. The environment exploration and network update process is repeated until the algorithm converges to a stable state.

[0169] In this embodiment, the total number of training steps is set to T. During each training cycle, the underwater glider interacts with the environment: in the current state, the state data is input into the policy network, and a discrete action probability distribution is output; a specific action is sampled according to the probability distribution and executed; based on the execution result, new state information and immediate rewards are obtained, and it is determined whether the round termination condition is met; all state, action, action probability, and reward data are stored in the experience replay pool.

[0170] In this embodiment, small batches of data are sampled from the experience replay pool to calculate the advantage estimate. The parameters of the policy network and value network are then updated using the policy gradient update formula. This process of environment exploration and network update is repeated until the algorithm converges to a stable state. Once the preset convergence condition is met, the optimal policy network parameters are saved, and the trained agent is output.

[0171] In this embodiment, after the intelligent agent based on the underwater glider performs continuous coverage observation of the observation area, the method further includes:

[0172] The effectiveness of the continuous coverage observation is evaluated based on preset evaluation indicators, including the first full coverage performance indicator, the longest revisit period indicator, the global coverage uniformity indicator, and the ocean current utilization efficiency indicator.

[0173] This invention evaluates the observation results through multi-dimensional performance indicators, comprehensively measures the coverage efficiency and quality of the algorithm, provides data support for subsequent optimization, and ensures the successful implementation of the observation task.

[0174] In this embodiment, an initial full coverage performance index, PM1, is defined. This index represents the number of communication cycles required for the underwater glider to achieve initial full coverage of the observation area, and is used to evaluate the initial coverage efficiency. The smaller the PM1 value, the shorter the time required for the underwater glider to achieve initial full coverage, and the higher the coverage efficiency.

[0175] In this embodiment, the longest revisit period index PM2 is defined. This index represents the revisit period of the area with the lowest visit frequency within the observation area after the underwater glider enters a stable and continuous coverage phase, and is used to assess coverage uniformity. The higher the PM2 value, the more frequently the underwater glider can revisit all areas, and the better the coverage adequacy.

[0176] In this embodiment, a global coverage uniformity index is defined. The average revisit period (Avg) and standard deviation (Sd) are used to evaluate the overall performance of sustained coverage. A higher average value indicates a higher overall revisit frequency in the observed area; a smaller standard deviation indicates smaller differences in revisit periods across areas, resulting in more uniform coverage.

[0177] In this embodiment, the ocean current utilization index PM3 is defined. In the inertial coordinate system, the ocean current velocity vector... It can be decomposed into components v in the x and y directions. cx and v cy ,like Figure 4 As shown, Further decomposed into components v along the velocity direction of the underwater glider and perpendicular to the velocity direction of the underwater glider. c1 and v c2 α represents the direction of ocean current. With the direction of the underwater glider The angle between them. Where, v c1 It can directly contribute to the speed of the underwater glider, while v c2 This will cause the underwater glider's flight path to deviate. When (v c1 -v c2 When PM3 ≥ 0, it indicates that the ocean current has a positive effect on the motion of the underwater glider. The formula for calculating PM3 is as follows:

[0178]

[0179] Where, N T The communication cycle completed during a continuous coverage mission for an underwater glider, where N is the total number of underwater gliders. A higher PM3 value means that, on average, during the entire continuous coverage process, the acceleration effect of the ocean current on the underwater glider's motion exceeds its offset effect. This indicates that the planned path can effectively utilize the ocean current to accelerate forward while minimizing lateral offset caused by the ocean current, thereby improving the underwater glider's motion efficiency.

[0180] In this embodiment of the invention, a terminal device is also provided, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the above-described method for continuous coverage observation of clustered underwater gliders.

[0181] This invention also provides a continuous coverage observation device for clustered underwater gliders, comprising: a motion model construction module, a revisit periodic map construction module, a continuous coverage observation module, and an algorithm adjustment module;

[0182] The motion model construction module is used to acquire a cluster of underwater gliders, which includes several individual underwater gliders, and to construct motion models for each individual underwater glider based on a preset communication cycle.

[0183] The revisit period map construction module is used to obtain the observation area, determine the coverage area of ​​the cluster of underwater gliders in each communication cycle based on the observation area and the motion model of each individual underwater glider, and construct a revisit period map and a priority coverage map based on the coverage area.

[0184] The continuous coverage observation module is used to construct the agent of the underwater glider based on preset underwater conditions, target reinforcement learning algorithm, the revisit periodic map and the priority coverage map, and to perform continuous coverage observation of the observation area based on the agent of the underwater glider.

[0185] The algorithm adjustment module is used to acquire the status information of the cluster of underwater gliders during continuous coverage observation, evaluate the communication conditions based on the status information, and dynamically match the target reinforcement learning algorithm based on the communication conditions.

[0186] In this embodiment of the invention, a computer-readable storage medium is also provided, which includes a stored computer program, wherein the computer program controls the device where the computer-readable storage medium is located to execute the above-described cluster underwater glider continuous coverage observation method when it is running.

[0187] For example, a computer program can be divided into one or more modules, one or more of which are stored in memory and executed by a processor to perform the present invention. The one or more modules can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in a terminal device.

[0188] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor, memory, and display. Those skilled in the art will understand that the above components are merely examples of terminal devices and do not constitute a limitation on the terminal device. It may include more or fewer components, or combinations of certain components, or different components. For example, the terminal device may also include input / output devices, network access devices, buses, etc.

[0189] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device through various interfaces and lines.

[0190] Memory can be used to store computer programs and / or modules. The processor implements various functions of the terminal device by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory. Memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for at least one function (such as sound playback, text conversion, etc.), etc.; the data storage area can store data created based on the use of the mobile phone (such as audio data, text message data, etc.). In addition, memory can include high-speed random access memory, and can also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital cards (SD cards), flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0191] The module for continuous coverage observation of clustered underwater gliders, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. Those skilled in the art can understand and implement this without any inventive effort.

[0192] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A method for persistent coverage observation by a swarm of underwater gliders, characterized in that, include: Acquire a cluster of underwater gliders, which includes several individual underwater gliders, and construct a motion model for each individual underwater glider based on a preset communication cycle; The observation area is obtained, and the coverage area of ​​the cluster of underwater gliders in each communication cycle is determined based on the observation area and the motion model of each individual underwater glider. A revisit cycle map and a priority coverage map are constructed based on the coverage area. Based on preset underwater conditions, a target reinforcement learning algorithm, the revisit periodic map, and the priority coverage map, an agent for the underwater glider is constructed. This agent performs continuous coverage observation of the observation area, including: defining the state space in the inertial coordinate system using the position coordinates of the swarm of underwater gliders, the current heading angle of the swarm, the revisit periodic map, the priority coverage map, and the ocean current velocity component; when the underwater conditions are still water conditions, the ocean current velocity component is set to 0; and using the position coordinates of a single underwater glider and the current heading angle of a single underwater glider... The observation space is defined by the heading angle, revisit periodic map, priority coverage map, and ocean current velocity component. A heading angle variation step size is preset based on the maneuverability of a single underwater glider, and an action space is constructed based on this step size, including all executable heading angles of the underwater glider. A reward function is constructed based on a preset differential reward function and a boundary penalty function. An agent for the underwater glider is constructed based on the state space, observation space, action space, reward function, and the target reinforcement learning algorithm. The agent performs continuous coverage observation of the observation area. During continuous coverage observation, the status information of the cluster of underwater gliders is acquired, the communication conditions are evaluated based on the status information, and the target reinforcement learning algorithm is dynamically matched based on the communication conditions.

2. A method of persistent coverage observation by a swarm of underwater gliders as claimed in claim 1, wherein, The communication conditions include primary communication, secondary communication, and tertiary communication; the process of acquiring the status information of the cluster of underwater gliders during continuous coverage observation, evaluating the communication conditions based on the status information, and dynamically matching the target reinforcement learning algorithm based on the communication conditions includes: During continuous coverage observation, the status information of the cluster of underwater gliders is acquired, including the surface synchronization rate. When the water surface synchronization rate is higher than or equal to a preset first threshold, the communication condition is level one communication; When the water surface synchronization rate is lower than a preset first threshold and higher than a preset second threshold, the communication condition is secondary communication; When the water surface synchronization rate is lower than or equal to a preset second threshold, the communication condition is level three communication; A dynamic target reinforcement learning algorithm is used to match the communication conditions.

3. A method of persistent coverage observation by a swarm of underwater gliders as claimed in claim 2, wherein, The dynamic target matching reinforcement learning algorithm based on the communication conditions includes: When the communication condition is Level 1 communication, the target reinforcement learning algorithm is a near-end policy optimization algorithm; When the communication condition is two-level communication, the target reinforcement learning algorithm is a multi-agent proximal policy optimization algorithm; When the communication condition is three-level communication, the target reinforcement learning algorithm is an independent near-end policy optimization algorithm.

4. A method of persistent coverage observation by a swarm of underwater gliders as claimed in claim 1, wherein, The acquisition of a cluster of underwater gliders, comprising several individual underwater gliders, involves constructing motion models for each individual underwater glider based on a preset communication cycle, including: An inertial coordinate system is constructed based on the horizontal plane and the preset coordinate axis directions; Obtain a cluster of underwater gliders, and for any single underwater glider in the cluster, construct a sub-model of the single underwater glider's sailing speed, a sub-model of its heading angle, a sub-model of its ocean current speed, and a sub-model of its position change in each gliding cycle. The motion model of the underwater glider is constructed based on the sailing speed sub-model, the heading angle sub-model, the ocean current speed component sub-model, and the position change sub-model.

5. The method for continuous coverage observation using clustered underwater gliders as described in claim 4, characterized in that, The process of acquiring the observation area involves determining the coverage area of ​​the cluster of underwater gliders in each communication cycle based on the observation area and the motion model of each individual underwater glider, and constructing a revisit cycle map and a priority coverage map based on the coverage area, including: Obtain the observation area and its boundary size information, and discretize the observation area into several grid cells based on the boundary size information, the preset cell size and the Cartesian grid method; Establish a two-dimensional index for each grid cell, and generate an observation grid model based on the grid cells and the two-dimensional index; Based on the observation area and the motion model of each individual underwater glider, the coverage area of ​​the cluster of underwater gliders in each communication cycle is determined, and a revisit cycle map and a priority coverage map are constructed based on the coverage area.

6. The method for continuous coverage observation using clustered underwater gliders as described in claim 5, characterized in that, The coverage area includes the grid cells covered in each communication cycle; the determination of the coverage area of ​​the cluster of underwater gliders in each communication cycle based on the observation area and the motion model of each individual underwater glider, and the construction of a revisit cycle map and a priority coverage map based on the coverage area, includes: Obtain a position change sub-model for each single underwater glider in each gliding cycle, and determine the first dive point and first surface point of the single underwater glider in the current communication cycle based on the position change sub-model. Based on the first dive point and the first surfacing point, the coverage sub-region of the single underwater glider is determined, and the coverage sub-regions of all single underwater gliders are integrated to obtain the coverage area of ​​the cluster of underwater gliders in each communication cycle. Based on the coverage area, a revisit cycle map and a priority coverage map are constructed.

7. A method of persistent coverage observation by a swarm of underwater gliders as claimed in claim 6, wherein, The construction of the revisit cycle map and priority coverage map based on the coverage area includes: Initialize a two-dimensional revisit periodogram, wherein the revisit periodogram includes the revisit values ​​of each grid cell of the observation grid model; At the end of each communication cycle, all grid cells of the observation grid model are traversed. If the current grid cell is contained within the covered area, the revisit value of the grid cell is set to 1; otherwise, the revisit value of the grid cell is calculated based on a preset first attenuation factor. When it is detected that any revisit value in the revisit periodogram decays to 0 or the cumulative communication cycle reaches a preset first revisit threshold, the revisit periodogram construction is completed. A priority coverage map is constructed based on the revisit periodic map and a preset second revisit threshold.

8. A method of persistent coverage observation by a swarm of underwater gliders as claimed in claim 7, wherein, The construction of the priority coverage map based on the revisit periodicity map and the preset second revisit threshold includes: Initialize a two-dimensional priority coverage map, which includes the priority coverage value of each grid cell of the observation grid model; At the end of each communication cycle, the revisit cycle graph is traversed. If the revisit value of the current grid cell is less than the preset second revisit threshold, the priority coverage value of the grid cell is set to 1; otherwise, it is set to 0, until all grid cells have been traversed, and the priority coverage graph construction is completed.

9. A swarm of underwater gliders for persistent coverage observation, characterized in that, include: The module includes a motion model construction module, a revisit periodicity graph construction module, a continuous coverage observation module, and an algorithm tuning module. The motion model construction module is used to acquire a cluster of underwater gliders, which includes several individual underwater gliders, and to construct motion models for each individual underwater glider based on a preset communication cycle. The revisit period map construction module is used to obtain the observation area, determine the coverage area of ​​the cluster of underwater gliders in each communication cycle based on the observation area and the motion model of each individual underwater glider, and construct a revisit period map and a priority coverage map based on the coverage area. The continuous coverage observation module is used to construct an agent for the underwater glider based on preset underwater conditions, a target reinforcement learning algorithm, the revisit periodic map, and the priority coverage map. Based on this agent, the module performs continuous coverage observation of the observation area, including: defining a state space in the inertial coordinate system using the position coordinates of the swarm of underwater gliders, the current heading angle of the swarm of underwater gliders, the revisit periodic map, the priority coverage map, and the ocean current velocity component; when the underwater conditions are still water conditions, the ocean current velocity component is set to 0; and using the position coordinates of a single underwater glider and the current velocity component of a single underwater glider... The observation space is defined by the glider's current heading angle, revisit period map, priority coverage map, and ocean current velocity components. A heading angle variation step size is preset based on the maneuverability of a single underwater glider, and an action space is constructed based on this step size, including all executable heading angles of the underwater glider. A reward function is constructed based on a preset differential reward function and a boundary penalty function. An agent for the underwater glider is constructed based on the state space, observation space, action space, reward function, and the target reinforcement learning algorithm. The agent performs continuous coverage observation of the observation area. The algorithm adjustment module is used to acquire the status information of the cluster of underwater gliders during continuous coverage observation, evaluate the communication conditions based on the status information, and dynamically match the target reinforcement learning algorithm based on the communication conditions.

Images