Cooperative control method and system for seabed intelligent underwater vehicle and mother boat

Abstract

The invention discloses a cooperative control method and system for a seabed intelligent underwater vehicle and a mother boat, and relates to the technical field of underwater intelligent control and multi-boat cooperative navigation, and the method comprises the steps: collecting multi-source marine environment parameters to construct a dynamic environment perception model, and building a propulsion energy consumption estimation map according to three-dimensional space nodes. And constructing a multi-agent state-action structure, and executing path selection and cooperative route planning of the mother boat-underwater vehicle through reinforcement learning. And error judgment is executed according to the propulsion energy consumption data returned by the underwater vehicle, and an online increment mechanism is called to dynamically update the energy consumption model and the path diagram structure. According to the method disclosed by the invention, an underwater cluster control whole-process system from sensing to planning to feedback optimization is completed. Each step has independent value and supports each other, finally, a high-adaptability, high-efficiency and intelligent cooperative seabed operation control system is constructed, the dependence of the prior art on communication, a high-precision model and cluster control capability is broken through, and the method is suitable for deep sea complex application scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater intelligent control and multi-vessel collaborative navigation, and specifically to a method and system for collaborative control of a seabed intelligent submersible and a mother vessel. Background Art

[0002] In recent years, with the increasing development of marine resources and deep-sea exploration missions, autonomous underwater vehicles (AUVs) have been widely studied and applied as autonomous underwater platforms. In particular, in scenarios such as ocean mapping, hydrological monitoring, communication relay, and rescue exploration, AUV deployment efficiency and mission accuracy have become important indicators of deep-sea operational capabilities. At the same time, to expand mission coverage and cope with complex sea conditions, collaborative control technology between AUVs and their mother vessels has become a current research hotspot. Current control strategies primarily focus on path planning, autonomous navigation, and energy optimization. Intelligent optimization methods such as artificial intelligence and reinforcement learning have been initially applied to local path search and environmental modeling. However, faced with the challenges of multi-vessel distributed operations, limited underwater communication bandwidth, and energy management, existing methods still face technical bottlenecks in global collaborative planning and real-time model adaptation.

[0003] Current mainstream control methods for submarine submersibles primarily rely on autonomous control of individual submersibles. While these methods offer some capabilities in local environmental perception and path generation, they lack a coordinated scheduling mechanism with the mothership. This leads to frequent path conflicts, task duplication, and uneven energy distribution in multi-submersible joint operations. Some research has attempted to achieve multi-submersible control through centralized path planning. However, these methods rely on high-frequency communication links and continuous scheduling from a central node, making them inadequate for underwater communication interruptions and high channel latency. Furthermore, existing technologies generally employ static energy estimation models for path cost assessment, failing to dynamically adjust based on real-time feedback. This results in reduced energy estimation accuracy and limited mission completion efficiency. In the area of ​​reinforcement learning path planning, some methods neglect state coordination among multiple submersibles and control of path overlap, failing to effectively avoid task path conflicts. Furthermore, the lack of a local error feedback mechanism during propulsion makes it difficult for the system to adaptively adjust the path graph structure to sudden environmental changes. In summary, existing technologies are unable to achieve collaborative path optimization and dynamic updating of energy consumption models based on intelligent perception between the mother ship and multiple submersibles. They also lack a highly robust self-learning feedback closed-loop system, making it difficult to meet the comprehensive requirements of global coordination, path independence and system adaptability in future complex deep-sea missions. Summary of the Invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problems solved by the present invention are: the existing methods for cooperative control of submarine submersibles lack a joint state modeling mechanism between the mothership and the submersible, low path coordination efficiency and lack of a conflict control mechanism, and the inability to dynamically adjust the propulsion energy consumption evaluation model; and how to realize adaptive path optimization control based on energy consumption prediction and reinforcement learning in a complex environment.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solutions: A method for collaborative control of an intelligent submarine and a mothership, comprising collecting multi-source ocean environmental parameters to construct a dynamic environmental perception model, and establishing a propulsion energy consumption estimation map based on three-dimensional spatial nodes. A multi-agent state-action structure is constructed, and path selection and mothership-submarine collaborative trajectory planning are performed through reinforcement learning. Error judgment is performed based on the propulsion energy consumption data returned by the submarine, and an online incremental mechanism is called to dynamically update the energy consumption model and path map structure. Constructing the multi-agent state-action structure includes setting a local state information vector inside each submarine, including the current position coordinates, target point coordinates, current environmental state vector, remaining power level, and mission urgency level. After the mothership collects the status of each submarine, a multi-agent joint state table is formed. Each submersible selects its next forward direction through a discrete action set based on the state information table. The action set consists of a set of nodes adjacent to the current position. The energy consumption cost value and task priority scheduling factor of each action are calculated by calling the strategy mapping function to form the final action selection result. The coordination between the state-actions of multiple submersibles is dynamically balanced and scheduled by the mother ship using a unified path intersection scoring rule.

[0007] As a preferred embodiment of the method for collaborative control of an intelligent submarine and a mothership described in the present invention, the method of collecting multi-source ocean environment parameters to construct a dynamic environment perception model includes obtaining raw data from electromagnetic current meters, micro-CTDs, and conductivity sampling components deployed on the submarine and the mothership, timestamping each data item and binding it to spatial coordinates to form a set of raw ocean environment data vectors. The raw data vectors are input into the dynamic environment perception model, which uses a perception processing unit based on a time series structure, performs feature fusion operations in the depth extraction layer, and outputs a multidimensional environmental field estimation tensor with spatial coordinates as the primary index. Each spatial location point contains a fused expression of the water flow vector field, thermal field, and salinity field, forming a dynamic perception environment map.

[0008] As a preferred embodiment of the method for collaborative control of an underwater intelligent submersible and a mothership described in the present invention, the method includes establishing a propulsion energy consumption estimation map based on three-dimensional spatial nodes, wherein each three-dimensional position node generated by the dynamic environment perception model is used as a path candidate point in the graph structure, the weight of the connecting edge between nodes is composed of the comprehensive propulsion energy consumption of the connecting path, and the propulsion energy consumption is calculated by weighting the square of the water flow intensity, the temperature deviation term, and the salinity gradient term at each position point. According to the actual operating sea area grid division standard, a three-dimensional hexahedral grid graph structure is constructed, and each node in the graph records the environmental perception state vector, the historical path usage frequency, and the propulsion energy consumption estimate. All nodes and connecting edges constitute a statically initialized propulsion energy consumption map.

[0009] As a preferred embodiment of the collaborative control method between submarine intelligent submersibles and a mothership, the multi-agent state-action structure is constructed by setting an independent local state vector for each submersible. The local state vector contains the current position coordinates, target point coordinates, current environment state vector, remaining battery level, and mission urgency level, representing the complete contextual state information required for decision-making. The mothership receives the local state vectors of all submersibles and constructs a global joint state table based on a set data reception period. The state table uses a spatial hash index structure to store the current states of all submersibles. Each submersible uses the joint state table, synchronized with the mothership's local state vectors, to construct a discrete action set. The action set consists of traversable nodes adjacent to the current path node. When selecting an action, the submersible scores each candidate action path based on a path intersection scoring function provided by the mothership. The path intersection scoring function calculates and quantitatively evaluates the degree of spatial overlap between the action path and the predetermined paths of other submersibles. The submersible ranks the candidate action paths according to their comprehensive scores, selects the best-scoring path, and transmits the action execution results back to the mothership in real time. The mothership iteratively updates the joint state table based on all the feedback information.

[0010] As a preferred embodiment of the collaborative control method between an underwater intelligent submersible and a mothership described in the present invention, the method of executing path selection and mothership-submersible collaborative trajectory planning through reinforcement learning includes training the path strategy network using a policy gradient reinforcement learning method. The objective function is the weighted total cost of the total propulsion energy consumption of multiple submersibles and the path overlap. The path overlap is calculated by the intersection-and-union ratio of each submersible's path node set. The mothership issues scheduling adjustment instructions to individual submersibles based on the path overlap risk. During each training cycle, after the path is executed, the mothership updates the collaborative scheduling feedback value based on the actual propulsion consumption and path deviation. The feedback value serves as a training sample to adjust the local path strategy of the submersible.

[0011] As a preferred embodiment of the collaborative control method between submarine intelligent submersibles and a mothership described in the present invention, the error determination based on propulsion energy consumption data transmitted back by the submersibles includes: during path propulsion, each submersible continuously measures the actual energy consumption required for propulsion per unit path segment, and compares the difference with the predicted value in the propulsion energy consumption map. If the difference exceeds a preset threshold for more than three consecutive cycles, an energy consumption error alarm is generated. The error alarm is sent to the mothership, where it is received and processed by a central scheduling program. The mothership records the error time series of each submersible and performs trend fitting. If the error trend continues to increase, the current path segment is marked as a high energy consumption fluctuation area, and a correction mechanism is initiated.

[0012] As a preferred embodiment of the collaborative control method between submarine intelligent submersibles and a mothership described in the present invention, the method involves dynamically updating the energy consumption model and path graph structure by invoking an online incremental mechanism. The mothership then performs a small-batch gradient update on the feature weights in the dynamic environment perception model based on the energy consumption errors and current path point information transmitted by each submersible, correcting the local deviations of the energy consumption estimation function. Path nodes with significant errors are marked as areas to be replanned. The mothership then triggers a local graph reconstruction process, removes the current high-error path edges, and reconstructs the local path edge weights using the updated energy consumption estimation function, completing the dynamic adaptive update of the path graph. The updated results are then broadcast and synchronized to all submersibles.

[0013] Another object of the present invention is to provide a cooperative control system for an intelligent submarine and a mothership, which can construct a multi-agent state-action structure through a reinforcement learning path cooperative optimization module, and perform path selection and mothership-submarine cooperative trajectory planning through reinforcement learning, thereby solving the problems of the existing submarine cooperative control method, such as the lack of a mothership-submarine joint state modeling mechanism, low path coordination efficiency and lack of conflict control mechanism, and the inability to dynamically adjust the propulsion energy consumption evaluation model, as well as the problem of how to realize adaptive path optimization control based on energy consumption prediction and reinforcement learning in a complex environment.

[0014] As a preferred solution of the cooperative control system of the submarine intelligent submersible and the mother ship described in the present invention, it includes: establishing an environmental perception and energy consumption estimation model module, a reinforcement learning path collaborative optimization module, and a feedback correction module.

[0015] The module for establishing an environmental perception and energy consumption estimation model is used to collect multi-source ocean environmental parameters to build a dynamic environmental perception model, and to establish a propulsion energy consumption estimation map based on three-dimensional space nodes.

[0016] The reinforcement learning path collaborative optimization module is used to construct a multi-agent state-action structure, and perform path selection and mothership-submarine collaborative trajectory planning through reinforcement learning.

[0017] The feedback correction module is used to perform error judgment based on the propulsion energy consumption data sent back by the submersible, and call the online incremental mechanism to dynamically update the energy consumption model and path diagram structure.

[0018] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement a method for collaborative control of an underwater intelligent submersible and a mother ship.

[0019] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for collaborative control of an underwater intelligent submersible and a mother vessel.

[0020] Beneficial effects of the present invention: The method for collaborative control of an intelligent submarine and a mother ship provided by the present invention collects multi-source ocean environment parameters to construct a dynamic environment perception model, and establishes a propulsion energy consumption estimation map, providing data support for the subsequent intelligent path selection and collaborative control of the submarine, improving the rationality of the path and the energy utilization efficiency of the mission, while enhancing the system's adaptability to the changing ocean environment.

[0021] By building a multi-agent state-action structure and executing path selection and collaborative trajectory planning between the mothership and the submersible through reinforcement learning, the path conflict rate and redundant propulsion energy consumption between multi-vessel missions are significantly reduced, the efficiency of cluster missions is improved, and the system has the ability to adapt and update strategies, so that the system can still maintain its collaborative capabilities when the mission scope and complexity increase.

[0022] Error judgment is performed based on the propulsion energy consumption data sent back by the submersible, and the online incremental mechanism is called to dynamically update the energy consumption model and path diagram structure, thereby enhancing the system robustness and improving the long-term effectiveness and accuracy of the path strategy, solving the problem of long-term error accumulation caused by static energy consumption estimation in existing technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1 This is an overall flow chart of a method for collaborative control of an underwater intelligent submersible and a mother vessel provided in the first embodiment of the present invention.

[0025] Figure 2 This is an overall flow chart of a cooperative control system between an underwater intelligent submersible and a mother vessel provided in the third embodiment of the present invention. DETAILED DESCRIPTION

[0026] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, the following detailed description of the specific embodiments of the present invention is given in conjunction with the accompanying drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in this field without creative work should fall within the scope of protection of the present invention.

[0027] Example 1, reference Figure 1 , which is an embodiment of the present invention, provides a method for cooperative control of a submarine intelligent submersible and a mothership, comprising:

[0028] S1: Collect multi-source ocean environmental parameters to build a dynamic environmental perception model, and establish a propulsion energy consumption estimation map based on three-dimensional space nodes.

[0029] Raw data is collected from electromagnetic current meters, microCTDs, and conductivity sampling components deployed on the submersible and the mothership. Each data item is timestamped and bound to spatial coordinates to form a raw marine environmental data vector set. This raw data vector set is then fed into a dynamic environmental perception model. The model employs a perception processing unit based on a time series structure, performs feature fusion operations in the depth extraction layer, and outputs a multidimensional environmental field estimation tensor indexed primarily by spatial coordinates. Each spatial location contains a fused representation of the water flow vector field, thermal field, and salinity field, forming a dynamic perception environment map.

[0030] A preferred solution for the dynamic environmental perception model specifically includes the following: the environmental perception model consists of a perception processing structure built based on a deep neural network, and its input is a group of raw data vectors organized in chronological order. The time series processing unit integrated in the model is used to learn temporal dynamic characteristics, while the multi-layer perception extraction unit is used to capture parameter correlations in the spatial dimension. During the feature fusion process, the model jointly models factors such as water flow, water temperature, and salinity. The final output result is a multidimensional environmental tensor structure with three-dimensional spatial positions as index points. This tensor makes an overall estimate of the hydrodynamic environment, thermal environment, and salinity changes of the entire target operation area, forming a dynamic perception environment map.

[0031] Each three-dimensional location node generated by the dynamic environmental perception model is considered a candidate path point in the graph structure. The weight of the edges connecting the nodes is composed of the comprehensive propulsion energy consumption of the connecting path. The propulsion energy consumption is calculated by weighting the squared current intensity, temperature deviation, and salinity gradient at each location. Based on the actual operating sea area grid division standard, a three-dimensional hexahedral grid graph structure is constructed. Each node in the graph records the environmental perception state vector, historical path usage frequency, and propulsion energy consumption estimate. All nodes and connecting edges form a statically initialized propulsion energy consumption map.

[0032] A preferred approach to estimating propulsion energy consumption specifically includes the following: the energy consumption estimate is not static, but is instead estimated based on changes in environmental parameters along the path connecting nodes. The energy consumption of each path edge is determined by the spatial variations in water flow intensity, water temperature, and salinity between the starting and ending nodes. For example, if a path segment contains strong reverse currents, high salinity gradients, or sudden temperature changes, propulsion energy consumption will increase significantly. Based on the numerical relationships between these changes, the model uses a weighted judgment mechanism to output the estimated propulsion cost for that path segment. The propulsion energy consumption map serves as the graph structure input for reinforcement learning path planning, enabling the submersible to evaluate path costs and select actions within the policy network.

[0033] It should be noted that S1 collects ocean environment parameters through multi-source sensors, constructs a spatial perception model and propulsion energy consumption map, enables the submersible to dynamically understand the complex sea environment and estimate the path energy consumption, improves the accuracy of path planning and the rationality of energy distribution, and provides basic support for subsequent intelligent control.

[0034] S2: Build a multi-agent state-action structure and perform path selection and collaborative trajectory planning between the mothership and the submersible through reinforcement learning.

[0035] An independent local state vector is set in each submersible. The local state vector includes the current position coordinates, target point coordinates, current environment state vector, remaining power level and mission urgency level, representing the complete contextual state information required for decision-making.

[0036] A preferred solution for the local state vector specifically includes constructing an independent local state vector structure for all submarine intelligent submersibles participating in the mission. Each submersible is numbered U k , where represents the kth submersible. For each submersible, at any time τ, the local state vector is expressed as:

[0037]

[0038] in, represents the local state vector of the kth underwater vehicle at time τ. Indicates the spatial coordinates of the current position. G (k) Indicates the spatial coordinates of the current target point. Indicates the water flow intensity, water temperature and salinity at the corresponding location. Indicates the current remaining power ratio, with a value range of [0,1]. Indicates the urgency level of the task. A larger value indicates a higher priority. The range is the integer set {1, 2, 3, 4, 5}. The urgency level of the task is calculated by weighting the remaining time to complete the task, the resource consumption of the task, and the percentage of the task completed.

[0039] The mother ship receives the local state vectors of all submersibles and builds a global joint state table based on the set data reception cycle. The state table uses a spatial hash index structure to store the current states of all submersibles.

[0040] A preferred solution of the joint state table specifically includes that the mother ship collects all the submersible state vectors at fixed time intervals and constructs a three-dimensional hash table H locally. τ , the hash table uses the vehicle number and spatial position coordinates as keys to map the state of each vehicle. The joint state table is represented as:

[0041]

[0042] Among them, H τ (U k ) represents the hash structure of the joint state table of the kth submarine at time τ, which is used to store the submarine U k Current status information. Represents a mapping.

[0043] Each submersible constructs a discrete action set through a joint state table synchronized with the mothership through its local state vector. The action set consists of the traversable nodes adjacent to the current path node. When selecting an action, the submersible scores each candidate action path based on the path intersection scoring function provided by the mothership. The path intersection scoring function calculates the degree of spatial overlap between the action path and the predetermined paths of other submersibles and performs a quantitative evaluation.

[0044] A preferred solution for constructing a discrete action set specifically includes: for each submersible U k , whose current optional path direction is represented as a set of discrete actions:

[0045]

[0046] in, Represents a set of discrete actions. M represents the feasible path of the submarine from the current node to the mth adjacent node. k represents the path set of the kth underwater vehicle. The propulsion energy consumption and path conflict score of each action path can be calculated.

[0047] A preferred solution for the path intersection scoring function specifically includes: to achieve multi-vessel coordination, the mother vessel sets a path intersection scoring function to measure the degree of spatial overlap between the candidate motion path of the submersible and the established paths of other submersibles. The scoring function is defined as follows:

[0048]

[0049] in, represents the path intersection score of the k-th underwater vehicle action m. Indicates the action path The set of nodes covered. represents the set of nodes on the path that the lth submersible is scheduled to take in the current cycle. |·| represents the size of the set. N represents the total number of submersibles.

[0050] The higher the intersection score, the more serious the potential path conflict. The mothership can issue recommended adjustment instructions to the submersible based on the score.

[0051] The submersibles sort candidate action paths by comprehensive scores, select the path with the best score, and propel forward. The results of the action execution are transmitted back to the mother ship in real time, and the mother ship iteratively updates the joint state table based on all the feedback information. The path policy network is trained using a policy gradient reinforcement learning method. The objective function is the weighted total cost of the total propulsion energy consumption of multiple submersibles and the path overlap. The path overlap is calculated as the intersection-and-union ratio of each submersible's path node set. The mother ship issues scheduling adjustment instructions to individual submersibles based on the path overlap risk. During each training cycle, the mother ship updates the collaborative scheduling feedback value based on the actual propulsion consumption and path deviation after the path execution. The feedback value serves as a training sample to adjust the submersible's local path strategy.

[0052] The submersibles sort the candidate action paths according to the comprehensive scores and select the path with the best score for advancement. A preferred solution specifically includes: each submersible sorts the candidate paths according to the following comprehensive cost values ​​and finally selects the one with the lowest cost for advancement:

[0053]

[0054] in, Represents the comprehensive path cost of action path m. represents propulsion energy consumption. α and β represent the global weighting factors set by the mother ship. Select the smallest The corresponding actions are executed. The path costs are normalized, the propulsion energy consumption is normalized to the maximum energy consumption value, and the path conflict score is scaled to the [0, 1] interval according to the intersection-to-union ratio coefficient.

[0055] A preferred solution for training the path policy network using the policy gradient reinforcement learning method specifically includes training the submersible path policy function using the policy gradient method and defining the following expected objective function:

[0056]

[0057] Among them, θ represents the policy network parameters. Represents the expected total cost under the current task when the policy network parameters are θ. θ Represents a policy function that takes state input and outputs action probability. Indicates taking the strategy based on π θ (i.e. the current policy network) sample path. τ' represents the stage index of the current submersible in the path, counting from the starting point of the path. T k represents the kth submersible mission cycle. γ represents the discount factor. represents the predicted propulsion energy consumption value of the kth submersible at stage τ'. It represents the spatial overlap between the path selected by the kth submersible in stage τ' and the paths of other submersibles.

[0058] It should be noted that S2 achieves state coordination, path allocation, and conflict avoidance between submersibles and the mothership by constructing a multi-agent state-action structure. The design introduces a joint state table, a path intersection scoring function, and a comprehensive path cost function, allowing each submersible to not only make decisions based on its own state, but also dynamically respond to the global coordination information provided by the mothership, achieving active avoidance between paths and reasonable resource allocation. Compared to the existing method of independently planning submersible paths and the mothership only relaying information, this step implements a global feedback optimization mechanism based on reinforcement learning, achieving the dual goals of minimizing path energy consumption and minimizing conflicts in a dynamic environment, improving the efficiency of group collaborative decision-making and the overall intelligence of the system.

[0059] S3: Perform error judgment based on the propulsion energy consumption data sent back by the submersible, and call the online incremental mechanism to dynamically update the energy consumption model and path diagram structure.

[0060] As each submersible executes its propulsion, it continuously measures its actual energy consumption per unit path segment and compares the difference with the predicted value from the propulsion energy consumption map. If the difference exceeds a preset threshold for more than three consecutive cycles, an energy consumption error alarm is generated. This error alarm is sent to the mothership, where it is received and processed by the central dispatch program. The mothership records the error time series of each submersible and performs trend analysis. If the error trend continues to increase, the current path segment is marked as a high energy consumption fluctuation area, and a correction mechanism is initiated.

[0061] A preferred solution for continuously measuring the actual energy consumption required for propulsion of the current unit path segment and comparing the difference with the predicted value in the propulsion energy consumption map specifically includes continuously measuring the actual propulsion energy consumption of the current path segment while the submersible is performing the path propulsion mission. This energy consumption value is collected by an energy consumption monitoring unit onboard the submersible and is measured in joules. After each completed unit path segment (i.e., propulsion from a three-dimensional path map node to its adjacent node), the propulsion energy consumption value for that segment is compared with the predicted propulsion energy consumption value recorded in the propulsion energy consumption map, and the energy consumption error between the two is calculated. Energy consumption error = actual propulsion energy consumption value - predicted propulsion energy consumption value. To ensure statistical significance of the error, a continuous time window mechanism is introduced. In the present invention, a detection is performed to determine whether the energy consumption error of three consecutive path segments exceeds a threshold of 20%. If the energy consumption error of a submersible exceeds 20% in three consecutive path propulsion cycles, an energy consumption error alarm is automatically generated. This information includes the start and end node numbers of the alarm path segment, the error magnitude, the error duration, and the acquisition timestamp.

[0062] The mother ship performs small-batch gradient updates on the feature weights in the dynamic environment perception model based on the energy consumption errors and current path point information sent back by each submersible, and corrects the local deviation of the energy consumption estimation function.

[0063] Furthermore, the mother ship uses the environmental state characteristics of the current path segment and the returned energy consumption error as training samples, and uses a small batch incremental training method to correct the dynamic environment perception model. Its update operation includes extracting the input vector corresponding to the path segment (such as water flow, temperature, salinity, etc.). The sampling error is used as the target value difference for gradient back propagation. A micro-step optimizer (the present invention uses Adam, and the learning rate is set to 0.0005) is used to perform a round of iterative adjustment on the parameters of the dynamic environment perception model. The current dynamic environment perception model snapshot is retained to support the rollback mechanism and ensure the stability of the dynamic environment perception model. This process is a local fine-tuning, and the model is only modified for the error area to avoid degradation of the overall prediction performance.

[0064] Path nodes with significant errors are marked as areas to be replanned. The mother ship triggers the local graph reconstruction process, removes the current high-error path edges, and reconstructs the local path edge weights through the updated energy consumption estimation function, completing the dynamic adaptive update of the path map and broadcasting the updated results to all submersibles.

[0065] After the model update is completed, the following structural reconstruction operations are performed on the path edges with significant errors:

[0066] Edges between error nodes in the path graph are set to non-selectable or low priority to prevent them from being prioritized in subsequent path selection. The edge weights of surrounding nodes are recalculated using the newly revised energy consumption estimation model. The reconstructed edge structure is reinjected into the local path graph to form an updated regional subgraph. This local update result is synchronized to the local path graph copies of all submersibles via the mothership communication broadcast mechanism. After receiving the updated data, all submersibles will use the revised subgraph to perform path search and cost judgment in the next path selection cycle, ensuring that the path selection logic matches environmental changes in real time.

[0067] It should be noted that S3 uses energy consumption error monitoring and feedback to achieve local online model correction and adaptive path map structure updates, addressing the problem of propulsion energy consumption prediction deviations caused by environmental changes. This mechanism improves the dynamic adaptability and robustness of path planning, ensuring the submersible can efficiently and stably execute its mission in complex waters.

[0068] Example 2 is an embodiment of the present invention, which provides a method for coordinated control of an underwater intelligent submersible and a mother ship. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0069] This embodiment uses a deep-sea mapping operation area as the experimental background, and sets up a comparative experiment to evaluate the actual effect of the multi-submersible intelligent collaborative path planning method proposed in the invention. The experimental area is divided into a 12 square kilometer grid area, the average water depth of the sea is 800 meters, and there is a complex temperature-salinity gradient and directional water flow environment. The experiment uses two groups of 6 submarine intelligent submersibles, of which the experimental group (numbered A1A3) adopts the mothership-submersible joint state scheduling and reinforcement learning path control method proposed in the present invention. The control strategy includes environmental dynamic perception modeling, strategy optimization based on propulsion energy consumption and path conflict scoring, and path feedback error driven map update mechanism. The control group (numbered B1B3) uses the traditional single-boat independent path planning method based on Dijkstra graph structure. The path is statically generated and the mothership does not participate in real-time scheduling.

[0070] Before the experiment, all submersibles were initialized to a fully charged state and configured with a unified mapping mission target and operating area. After the experiment began, the submersibles deployed from the mother vessel in sequence according to the mission schedule, autonomously planned a path to the target, completed the mission, and returned. During the experiment, data was collected for each submersible's propulsion energy consumption, path overlap score, execution time, number of path conflicts, and mission completion rate. To eliminate interference from environmental disturbances, the experiment was repeated three times over a three-day intertidal period. The data shown is the average of these three rounds. Changes in various indicators were recorded throughout the experiment, and actual propulsion behavior was compared and modeled to ensure data reproducibility. The experimental data is shown in Table 1.

[0071] Table 1 Experimental data table

[0072]

[0073]

[0074] As can be seen from the tabular data, in the experimental group (A1B3) using the method of the present invention, the average propulsion energy consumption was reduced by about 23%, the path overlap score was reduced by more than 70%, the number of path conflicts was almost zero, and the task completion rate was maintained at more than 98%, indicating that path optimization and collaborative control achieved efficient energy management while ensuring mission integrity.

[0075] In terms of path conflict control, this invention incorporates a scoring function for the synchronization of the mother-vessel joint state table and the degree of path intersection, effectively predicting areas of multi-vessel path overlap and dynamically avoiding them. The experimental group experienced virtually zero path conflicts, while the control group experienced frequent path intersections (an average of over three per vessel). These events led to resource competition and physical obstructions in densely populated areas, resulting in reduced execution efficiency.

[0076] In terms of propulsion energy consumption, the experimental group's energy consumption profile could be dynamically adjusted in real time, allowing the submersible to avoid countercurrents and high-density obstacle paths. The propulsion cost per unit distance was significantly lower than the control group (average 15.26 J / m vs. 19.87 J / m). This demonstrates the adaptability of the energy consumption prediction and feedback mechanism designed in this invention, resolving the issues of low static cost modeling accuracy and large long-term distortion accumulation errors associated with traditional path generation methods.

[0077] In terms of task completion efficiency, although the experimental and control groups started from the same state, the experimental group, relying on a reinforcement learning path strategy, effectively converged to the optimal strategy during training, keeping the average completion time within 47 minutes, saving over 20% compared to the traditional strategy. Furthermore, the task completion rate in the experimental group remained high at 100% to 98%, while the control group, affected by path conflicts and inaccurate energy consumption control, had a minimum completion rate of only 82%.

[0078] Example 3, reference Figure 2 , which is an embodiment of the present invention, provides a cooperative control system for an underwater intelligent submersible and a mother ship, including an environment perception and energy consumption estimation model establishment module 100, a reinforcement learning path collaborative optimization module 200, and a feedback correction module 300.

[0079] S4: Establishing an environmental perception and energy consumption estimation model module 100 is used to collect multi-source ocean environmental parameters to build a dynamic environmental perception model, and establish a propulsion energy consumption estimation map based on three-dimensional space nodes.

[0080] The environment perception and energy consumption estimation model building module 100 includes an environment data collection submodule 101 and an energy consumption map building submodule 102 .

[0081] Furthermore, the environmental data acquisition submodule 101 is used to obtain parameters such as water flow velocity, temperature, salinity, etc. from the electromagnetic current meters, CTD sensors, and conductivity acquisition devices deployed on the mother ship and the submersible, and to label and organize the collected environmental data according to time and spatial position, and to construct a multi-source environmental vector group as the input basis of the perception model.

[0082] The energy consumption map construction submodule 102 is used to divide the three-dimensional operating area into hexahedral grid nodes. It calculates propulsion energy consumption indicators based on the changing relationships between the environmental parameters corresponding to each node. Using the nodes as the basis for the graph structure, it constructs a propulsion energy consumption estimation map with edge weights. This map is used to quantify the propulsion cost of any path segment and provides cost function input for the path optimization stage.

[0083] It should be noted that the environmental data acquisition submodule 101 is the starting point for establishing the perception model, and the accuracy of the data it collects directly affects the accuracy of the map construction. The energy consumption map construction submodule 102 is an important input source for the reinforcement learning path collaborative optimization module 200, providing fundamental support for path energy efficiency control.

[0084] S5: The reinforcement learning path collaborative optimization module 200 is used to construct a multi-agent state-action structure, and perform path selection and mothership-submarine collaborative trajectory planning through reinforcement learning.

[0085] The reinforcement learning path collaborative optimization module 200 includes a joint state construction submodule 201 and a path strategy generation submodule 202 .

[0086] Furthermore, the joint state construction submodule 201 is used to construct a local state vector for each submersible containing information such as the current position, target point, environmental status, power, mission level, etc., and summarize it on the mother ship side to form a joint state table for unified management and scheduling of multi-ship status information, which is used as state input when the submersible path action is generated.

[0087] The path strategy generation submodule 202 trains the path strategy network using a policy gradient reinforcement learning method based on the joint state table. It uses propulsion energy consumption and path overlap as a joint cost function to output the optimal path action. During each path execution cycle, the submersible selects a propulsion direction based on the comprehensive scores of the candidate action paths and reports the execution results to the mothership for subsequent feedback and optimization.

[0088] It should be noted that the joint state construction submodule 201 is the input source for path strategy generation, ensuring that the submersible's path movement has a global collaborative perspective. The path strategy generation submodule 202 constitutes the core of the system's intelligent control and directly determines the path efficiency and the rationality of task distribution.

[0089] S6: The feedback correction module 300 is used to perform error judgment based on the propulsion energy consumption data sent back by the submersible, and call the online incremental mechanism to dynamically update the energy consumption model and path diagram structure.

[0090] The feedback correction module 300 includes an error detection submodule 301 and a graph structure updating submodule 302 .

[0091] Furthermore, the error detection submodule 301 collects the actual propulsion energy consumption per unit path segment during the propulsion process and compares it in real time with the predicted value in the corresponding map. If the energy consumption error exceeds a set threshold (e.g., 20%) for three consecutive cycles, an error alarm is generated and uploaded to the mothership. The mothership then performs trend fitting on the error sequence and marks the abnormal area.

[0092] The graph structure update submodule 302 is used to trigger the online incremental model update mechanism based on the error alarm, perform micro-step adjustment on the feature weights in the perception model, and locally reconstruct the graph structure corresponding to the error path segment, including edge weight correction and path disable setting, and finally form an updated regional path subgraph, which is synchronized to all submersible path graph copies through the mother ship broadcast.

[0093] It should be noted that the error detection submodule 301 constitutes the triggering premise for the model's adaptive adjustment, ensuring the accuracy of the path data. The graph structure update submodule 302 implements dynamic graph correction to enhance the system's ability to quickly respond to complex ocean disturbances.

[0094] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0095] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0096] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0097] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having logic gate circuits for implementing logical functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc. It should be noted that the above embodiments are merely illustrative of the technical solutions of the present invention and are not intended to be limiting. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced with equivalents without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications should be encompassed by the claims of the present invention.

Claims

1. A method for cooperative control of an underwater intelligent submersible and a mother ship, characterized in that: include: Collect multi-source ocean environmental parameters to build a dynamic environmental perception model, and establish a propulsion energy consumption estimation map based on three-dimensional spatial nodes; Construct a multi-agent state-action structure and perform path selection and collaborative trajectory planning between the mothership and the submersible through reinforcement learning; According to the propulsion energy consumption data sent back by the submersible, error judgment is performed and the energy consumption model and path diagram structure are dynamically updated by calling the online incremental mechanism; Constructing a multi-agent state-action structure includes setting a local state information vector inside each submersible, including the current position coordinates, target point coordinates, current environment state vector, remaining power level and mission urgency level. The mother ship collects the status of each submersible to form a multi-agent joint state table; each submersible selects the next forward direction through a discrete action set based on the state information table. The action set consists of a set of nodes adjacent to the current position. The energy consumption cost value and task priority scheduling factor of each action are calculated by calling the strategy mapping function to form the final action selection result. The coordination between the state-actions of multiple submersibles is dynamically balanced and scheduled by the mother ship using a unified path intersection scoring rule.

2. The method for cooperative control of an underwater intelligent submersible and a mothership according to claim 1, wherein: The said collecting multi-source ocean environment parameters to construct a dynamic environment perception model includes: Obtain raw data from the electromagnetic current meter, micro CTD, and conductivity sampling components deployed on the submersible and the mother ship, timestamp each data item and bind it to spatial coordinates to form a vector group of raw marine environmental data; The original data vector group is input into the dynamic environmental perception model. The model adopts a perception processing unit based on the time series structure, performs feature fusion operations in the deep extraction layer, and outputs a multidimensional environmental field estimation tensor with spatial coordinates as the main index. Each spatial position point contains a fused expression of the water flow vector field, thermal field, and salinity field to form a dynamic perception environment map.

3. The method for cooperative control of a submarine intelligent submersible and a mother vessel according to claim 1 or 2, characterized in that: The establishing of a propulsion energy consumption estimation map based on three-dimensional space nodes includes: Each 3D location node generated by the dynamic environment perception model is used as a path candidate point in the graph structure. The weight of the connecting edge between nodes is composed of the comprehensive propulsion energy consumption of the connection path. The propulsion energy consumption is calculated by weighting the square of the water flow intensity, the temperature deviation term, and the salinity gradient term at each location point. According to the actual operating sea area grid division standard, a three-dimensional hexahedral grid graph structure is constructed. Each node in the graph records the environmental perception state vector, historical path usage frequency and propulsion energy consumption estimation. All nodes and connecting edges constitute a statically initialized propulsion energy consumption map.

4. The method for cooperative control of a submarine intelligent submersible and a mothership according to claim 3, wherein: The construction of the multi-agent state-action structure includes: An independent local state vector is set in each submersible. The local state vector contains the current position coordinates, target point coordinates, current environment state vector, remaining power level and mission urgency level, representing the complete contextual state information required for decision-making; The mother ship receives the local state vectors of all submersibles and builds a global joint state table based on the set data reception cycle. The state table uses a spatial hash index structure to store the current states of all submersibles. Each submersible constructs a discrete action set through a joint state table synchronized with the mothership through its local state vector. The action set consists of traversable nodes adjacent to the current path node. When selecting an action, the submersible scores each candidate action path based on the path intersection scoring function provided by the mothership. The path intersection scoring function calculates the degree of spatial overlap between the action path and the paths planned by other submersibles and performs a quantitative evaluation. The submersible sorts the candidate action paths according to the comprehensive scores, selects the path with the best score to advance, and transmits the action execution results back to the mother ship in real time. The mother ship iteratively updates the joint state table based on all the returned information.

5. The method for cooperative control of a submarine intelligent submersible and a mother vessel according to any one of claims 1, 2 or 4, characterized in that: The execution of path selection and mothership-submarine collaborative trajectory planning through reinforcement learning includes: The path strategy network is trained using a policy gradient reinforcement learning method. The objective function is the weighted total cost of the total propulsion energy consumption of multiple submarines and the path overlap. The path overlap is calculated by the intersection-and-union ratio of each submarine's path node set. The mother ship issues scheduling adjustment instructions to individual submersibles based on the path overlap risk. In each training cycle, the mother ship updates the collaborative scheduling feedback value based on the actual propulsion consumption and path deviation after the path is executed. The feedback value is used as a training sample to adjust the local path strategy of the submersible.

6. The method for cooperative control of an underwater intelligent submersible and a mother vessel according to claim 5, characterized in that: The error judgment performed based on the propulsion energy consumption data sent back by the submersible includes: During the execution of path propulsion, each submersible continuously measures the actual energy consumption required for propulsion of the current unit path segment and compares the difference with the predicted value in the propulsion energy consumption map. When the difference exceeds the preset threshold for more than three consecutive cycles, an energy consumption error alarm is generated; The error alarm is sent to the mother ship and received and processed by the central scheduling program. The mother ship records the error time series of each submersible and performs trend fitting judgment. If the error trend continues to grow, the current path segment is marked as a high energy consumption fluctuation area and the correction mechanism is activated.

7. The method for cooperative control of a submarine intelligent submersible and a mother vessel according to any one of claims 1, 2, 4 or 6, characterized in that: The calling of the online incremental mechanism to dynamically update the energy consumption model and the path graph structure includes: The mother ship performs small-batch gradient updates on the feature weights in the dynamic environment perception model based on the energy consumption errors and current path point information sent back by each submersible, correcting the local deviation of the energy consumption estimation function. Path nodes with significant errors are marked as areas to be replanned. The mother ship triggers the local graph reconstruction process, removes the current high-error path edges, and reconstructs the local path edge weights through the updated energy consumption estimation function, completing the dynamic adaptive update of the path map and broadcasting the updated results to all submersibles.

8. A cooperative control system for an underwater intelligent submersible and a mothership, characterized by: It includes an environment perception and energy consumption estimation model building module (100), a reinforcement learning path collaborative optimization module (200), and a feedback correction module (300); The environment perception and energy consumption estimation model building module (100) is used to collect multi-source ocean environment parameters to build a dynamic environment perception model, and to build a propulsion energy consumption estimation map based on three-dimensional space nodes; The reinforcement learning path collaborative optimization module (200) is used to construct a multi-agent state-action structure, and perform path selection and mothership-submarine collaborative track planning through reinforcement learning; The feedback correction module (300) is used to perform error judgment based on propulsion energy consumption data transmitted back by the submersible, and to call an online increment mechanism to dynamically update the energy consumption model and the path map structure.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for collaborative control of an underwater intelligent submersible and a mother vessel as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for collaborative control of an underwater intelligent submersible and a mother vessel according to any one of claims 1 to 7 are implemented.

Images