Multi-agent collaborative underwater logistics system and method based on reinforcement learning

Abstract

This invention provides a multi-agent cooperative underwater logistics system and method based on reinforcement learning, applicable to the fields of underwater robot cooperative control and intelligent logistics technology. The system integrates six modules: task management, multimodal communication, cooperative localization, path planning, energy management, and fault handling. Through a dynamic switching mechanism in multimodal communication, it adaptively selects between acoustic, optical, and magnetic induction modes to ensure communication reliability in underwater environments. A distributed cooperative localization method based on dynamic weights is employed, optimizing weight allocation by considering communication quality, battery power, and computing power to achieve cooperative localization. A multi-agent reinforcement learning controller with a multi-branch attention mechanism is introduced to achieve real-time obstacle avoidance, formation maintenance, and task-oriented cooperative path planning. An intelligent charging strategy based on multi-level battery thresholds and task urgency supports continuous system operation. A sliding window fault detection and adaptive task reassignment mechanism enhance the system's fault tolerance and robustness.

Description

Technical Field

[0001] This invention relates to the field of underwater robot cooperative control and intelligent logistics technology, and more specifically to a multi-agent cooperative underwater logistics system and method based on reinforcement learning. Background Technology

[0002] With the continuous development of marine resources, marine scientific research, and underwater engineering construction, the demand for underwater logistics systems is increasing. Traditional underwater logistics operations mainly rely on a single autonomous underwater vehicle (AUV), but due to its limited payload capacity, low mission efficiency, and inability to handle complex handling tasks, it is difficult to meet the needs of large-scale, collaborative, and intelligent underwater operations.

[0003] In recent years, multi-AUV cooperative systems have gradually become a research hotspot, as multi-AUV collaboration can improve mission execution capabilities and system robustness. However, in real-world complex and dynamic underwater environments, multi-AUV cooperative logistics systems still face many technical challenges:

[0004] Task allocation and collaborative decision-making are challenging: Underwater logistics tasks often involve cargo of different specifications, requiring the dynamic organization of AUVs of varying numbers and capabilities for collaborative handling. Existing methods mostly employ static task allocation or centralized scheduling, which are difficult to adapt to dynamic environments and the heterogeneous characteristics of multiple AUVs, and are prone to single points of failure.

[0005] Underwater communication is limited and unstable: underwater acoustic communication has a long range but low bandwidth and high latency; optical communication has high bandwidth but is subject to stringent requirements for water quality and attitude; a single communication method cannot cover all operational scenarios. The lack of a multimodal communication adaptive switching mechanism often leads to problems such as interruption of collaborative control and data asynchrony.

[0006] Insufficient cooperative positioning accuracy: GPS cannot be used underwater, and a single AUV relying on inertial navigation and acoustic positioning is prone to cumulative errors. The lack of an efficient cooperative positioning mechanism among multiple AUVs leads to a decrease in the accuracy of formation control and cooperative transport.

[0007] Path planning and formation control are complex: the underwater environment is subject to interference from static obstacles, ocean current disturbances, and dynamic organisms. Multiple AUVs need to maintain formation, avoid obstacles, and optimize the global path simultaneously during transport. Traditional planning methods are difficult to achieve real-time and safe collaborative control under dynamic constraints.

[0008] Energy management and system sustainability challenges: AUVs have limited energy, and traditional charging strategies lack dynamic scheduling mechanisms that coordinate with tasks, affecting the system's long-term operational capability.

[0009] Inadequate system failure and anomaly handling mechanisms: Underwater equipment is susceptible to environmental influences and failures. Existing systems often lack rapid fault detection, severity assessment, and task reconfiguration capabilities, resulting in insufficient system robustness.

[0010] To address the aforementioned issues, current technologies have not yet developed a complete, efficient, and adaptive multi-AUV collaborative logistics system capable of operating in complex underwater environments. Therefore, there is an urgent need for a collaborative control system that integrates multimodal communication, high-precision collaborative positioning, intelligent path planning, dynamic energy management, and fault self-recovery to achieve efficient, reliable, and autonomous execution of underwater logistics tasks. Summary of the Invention

[0011] (a) Technical problems to be solved

[0012] To address at least one of the aforementioned problems, this invention provides a multi-agent collaborative underwater logistics system and method based on reinforcement learning. By integrating multimodal dynamic communication, dynamic weighted collaborative localization, multi-branch attention reinforcement learning control, intelligent energy management, and fault adaptive recovery mechanisms, it achieves efficient task allocation, reliable collaborative localization, intelligent path planning, and continuous operation capabilities for autonomous underwater vehicle (AUV) swarms in complex environments, significantly improving the autonomy, robustness, and overall operational efficiency of the underwater logistics system.

[0013] (II) Technical Solution

[0014] To address the aforementioned technical problems, embodiments of the present invention propose a multi-agent cooperative underwater logistics system and method based on reinforcement learning.

[0015] According to a first aspect of the present invention, a multi-agent collaborative underwater logistics system based on reinforcement learning is provided, comprising: a task management module, a multimodal dynamic communication module, a dynamic weighted collaborative positioning module, a reinforcement learning path planning and control module, an intelligent energy management module, and a fault detection and processing module; wherein, the multimodal dynamic communication module supports three communication methods: acoustic, optical, and magnetic induction, and can adaptively switch communication modes based on environmental perception information and task status prediction; the dynamic weighted collaborative positioning module dynamically allocates weights based on the local positioning of each underwater vehicle, integrating the communication quality, battery status, computing power, and historical positioning accuracy of each vehicle, and uses a distributed consensus algorithm to achieve collaborative positioning; the reinforcement learning path planning and control module uses a multi-agent reinforcement learning algorithm with a multi-branch attention mechanism, and generates collaborative control commands by combining environmental maps, obstacle information, companion status, and task objectives; the intelligent energy management module triggers charging decisions in a graded manner according to the vehicle's battery status and the urgency of the task, and performs charging scheduling based on the comprehensive score of the charging station; the fault detection and processing module monitors key operating indicators through a sliding window to achieve fault level assessment and adaptive task reassignment.

[0016] In some exemplary embodiments, the task management module uses a consensus-based auction algorithm to allocate tasks. The value of a task is calculated by weighting the task urgency, capability matching degree, execution efficiency assessment and estimated cost. The task urgency is calculated by combining the normalized result of the original urgency with the time urgency factor, and a penalty coefficient is applied to overdue tasks.

[0017] In some exemplary embodiments, the multimodal dynamic communication module includes a performance evaluation unit and a switching decision unit; the performance evaluation unit is used to evaluate the channel quality of acoustic, optical and magnetic induction communication in the current and predicted environment; the switching decision unit triggers communication mode switching when the predicted utility of the current communication mode is lower than a threshold and there is a better candidate mode.

[0018] In some exemplary embodiments, the dynamic weights assigned to each autonomous underwater vehicle by the dynamic weight cooperative positioning module are jointly determined by a communication quality assessment factor, a battery status assessment factor, a computing power assessment factor, and a historical positioning accuracy assessment factor. Among them, the communication quality assessment factor is used to reflect the stability and latency of the communication link; the battery status assessment factor reflects the current power level of the vehicle; the computing power assessment factor reflects the performance and load status of the vehicle's processor; and the historical positioning accuracy assessment factor reflects the reliability of the vehicle's past positioning estimates.

[0019] In some exemplary embodiments, the multi-branch attention mechanism includes: a local attention branch for calculating obstacle threat level and companion threat level and normalizing them into weights; a global attention branch for calculating path relevance and boundary distance relevance and normalizing them into weights; a temporal attention branch for calculating the temporal change rate of each state feature group and normalizing it into weights; and the weights of the local attention branch, global attention branch, and temporal attention branch are fused according to a preset ratio to form the final attention weight for state feature weighting.

[0020] According to a second aspect of the present invention, a multi-agent cooperative underwater logistics method based on reinforcement learning is provided, comprising: receiving task information, assessing task urgency, and completing initial task allocation based on a consensus auction algorithm; perceiving the underwater communication environment, assessing the current and predicted performance of each communication mode, dynamically switching communication modes based on utility assessment and interruption probability prediction, and maintaining acoustic control link redundancy during switching; each vehicle performing local positioning and mapping, and when observing companions or common features, calculating dynamic weights based on communication quality, battery status, computing power, and historical accuracy, and performing cooperative positioning calculation using a weighted consensus algorithm; constructing and updating the environmental map, setting a cooperative formation according to cargo type, inputting the environment and task status into a multi-agent reinforcement learning controller with a multi-branch attention mechanism, outputting control commands and driving the formation to perform transport; monitoring vehicle battery power, triggering charging decisions according to multi-level battery thresholds and task urgency, selecting and scheduling charging stations based on comprehensive charging station scores; monitoring key indicators through a sliding window, detecting anomalies and assessing fault levels, executing corresponding handling measures, and triggering task reassignment when necessary.

[0021] In some exemplary embodiments, dynamically switching communication modes includes: evaluating the total utility of each communication mode, where the total utility is a weighted fusion result of the current utility and the predicted utility; and triggering a switch to the optimal candidate mode when the predicted utility of the current mode is lower than a set threshold and there are other modes whose predicted utility is higher than the threshold plus a set margin.

[0022] In some exemplary embodiments, the cooperative positioning solution using a weighted consensus algorithm includes: calculating dynamic weights for each vehicle, the dynamic weights being composed of a quality factor reflecting communication reliability, a battery factor reflecting energy state, a computation factor reflecting processing capability, and a historical factor reflecting past positioning accuracy; within a communication window, each vehicle exchanges its local positioning estimate and corresponding dynamic weights; based on the received estimates and weights from neighboring vehicles, the weighted consensus algorithm is used to interpolate and update the results in the manifold space, and the cooperative positioning result is output after iterative convergence.

[0023] In some exemplary embodiments, the multi-branch attention fusion mechanism includes: calculating local attention weights through local attention branches based on the proximity of obstacles and companions; calculating global attention weights through global attention branches based on path progress and boundary distance; calculating temporal attention weights through temporal attention branches based on the changing trend of state features; and fusing local attention weights, global attention weights, and temporal attention weights according to a preset ratio to perform weighted processing on the input state features.

[0024] In some exemplary embodiments, triggering charging decisions based on multi-level power thresholds and task urgency levels includes: interrupting the task and performing emergency charging when the power level is below the emergency threshold; reserving a charging station when the power level is below the low power threshold and the current task is a non-urgent task; and planning charging during task intervals when the power level is below the medium threshold. Wherein, the emergency threshold is lower than the low power threshold, and the low power threshold is lower than the medium threshold.

[0025] (III) Beneficial Effects

[0026] As can be seen from the above technical solutions, the multi-agent cooperative underwater logistics system and method based on reinforcement learning provided by the embodiments of the present invention has at least the following beneficial effects:

[0027] (1) Through the task allocation mechanism that combines consensus auction algorithm with reinforcement learning value assessment, the system can realize dynamic optimization allocation and rapid reallocation of multi-machine collaborative tasks according to the real-time status of AUV, the urgency of tasks and environmental changes, thereby improving task execution efficiency and system response flexibility.

[0028] (2) By adopting the fusion of acoustic, optical and magnetic induction multimodal communication and prediction-based dynamic switching strategy, the system can adaptively select the optimal communication mode according to environmental parameters such as water quality, distance and noise, and maintain control continuity through acoustic link redundancy, which significantly improves the reliability of communication link and data transmission stability.

[0029] (3) Based on the distributed collaborative positioning mechanism of dynamic weight, the weighted consensus optimization algorithm on the manifold space is adopted to effectively suppress the accumulation of single AUV positioning error and realize high-precision and high-reliability group collaborative positioning by taking into account factors such as communication quality, power consumption, computing power and historical accuracy.

[0030] (4) By introducing a multi-agent reinforcement learning controller with a multi-branch attention mechanism, the system can simultaneously focus on local obstacle avoidance, global task orientation and dynamic change trends, realize real-time path planning and formation maintenance of multi-AUV collaborative transport in complex environments, and take into account safety, efficiency and energy consumption optimization.

[0031] (5) Based on the multi-level power threshold and task urgency, the hierarchical charging strategy, combined with the comprehensive scoring and scheduling mechanism of charging stations, realizes dynamic coordination between energy supply and task execution, extends the system operation cycle, and improves resource utilization efficiency.

[0032] (6) Through sliding window fault detection, confidence assessment and graded response mechanism, the system can quickly identify anomalies and execute an adaptive recovery process from module restart to task reassignment, ensuring that the overall task can continue to be executed in the event of a partial fault, which significantly enhances the fault tolerance and job continuity of the system. Attached Figure Description

[0033] The above-described features, other objects, and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0034] Figure 1 The diagram illustrates a block diagram of a multi-agent cooperative underwater logistics system based on reinforcement learning according to an embodiment of the present invention.

[0035] Figure 2 This illustration schematically shows a general architecture diagram of a multi-agent cooperative underwater logistics system based on reinforcement learning according to an embodiment of the present invention.

[0036] Figure 3 This illustration schematically shows a task allocation flowchart of a task management module according to an embodiment of the present invention;

[0037] Figure 4 A schematic diagram illustrating the dynamic communication switching process according to an embodiment of the present invention is shown.

[0038] Figure 5 A schematic diagram illustrating the positioning collaboration process according to an embodiment of the present invention is shown.

[0039] Figure 6 This illustration schematically shows a flowchart of path planning and formation control of a reinforcement learning path planning control module according to an embodiment of the present invention;

[0040] Figure 7 The schematic diagram illustrates a local map and a global map according to an embodiment of the present invention;

[0041] Figure 8 This diagram schematically illustrates the structure of a reinforcement learning path planning control module according to an embodiment of the present invention.

[0042] Figure 9 The schematic diagram illustrates the workflow of an intelligent energy management module according to an embodiment of the present invention;

[0043] Figure 10 This schematically illustrates a flowchart of a fault detection and processing module performing fault detection and processing according to an embodiment of the present invention;

[0044] Figure 11 A flowchart illustrating a multi-agent cooperative underwater logistics method based on reinforcement learning according to an embodiment of the present invention is shown. Detailed Implementation

[0045] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0046] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0047] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0048] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0049] Figure 1 A block diagram of a multi-agent cooperative underwater logistics system 100 based on reinforcement learning according to an embodiment of the present invention is shown schematically.

[0050] like Figure 1As shown, a multi-agent collaborative underwater logistics system 100 based on reinforcement learning according to an embodiment of the present invention includes: a task management module 110, a multimodal dynamic communication module 120, a dynamic weighted collaborative localization module 130, a reinforcement learning path planning and control module 140, an intelligent energy management module 150, and a fault detection and processing module 160; wherein, the multimodal dynamic communication module 120 supports three communication methods: acoustic, optical, and magnetic induction, and can adaptively switch communication modes according to environmental perception information and task status prediction; the dynamic weighted collaborative localization module 130, based on the local localization of each underwater vehicle, integrates the local localization of each vehicle... The system dynamically assigns weights to the communication quality, battery status, computing power, and historical positioning accuracy of the vehicle, and uses a distributed consensus algorithm to achieve collaborative positioning. The reinforcement learning path planning and control module 140 uses a multi-agent reinforcement learning algorithm with a multi-branch attention mechanism to generate collaborative control commands by combining environmental maps, obstacle information, companion status, and task objectives. The intelligent energy management module 150 triggers charging decisions in a graded manner based on the vehicle's battery status and the urgency of the task, and schedules charging based on the comprehensive score of the charging station. The fault detection and handling module 160 monitors key operating indicators through a sliding window to achieve fault level assessment and adaptive task reassignment.

[0051] Figure 2 The schematic diagram illustrates the overall architecture of a multi-agent cooperative underwater logistics system 100 based on reinforcement learning according to an embodiment of the present invention.

[0052] like Figure 2As shown, according to an embodiment of the present invention, a multi-agent cooperative underwater logistics system 100 based on reinforcement learning consists of several AUVs and a shore-based / mother ship platform. The platform is responsible for task issuance and overall monitoring. Each AUV is equipped with local perception, SLAM, propulsion, communication, and charging interfaces. The overall system flow is as follows: the platform issues tasks, and the task information includes cargo type, start and end locations, deadline, and original urgency. Cargo is divided into three types: small, medium, and large. Small cargo: small in size and light in weight, requiring one AUV to handle; medium cargo: medium in size and medium in weight, requiring three AUVs to handle (triangular formation); large cargo: large in size and heavy in weight, requiring four AUVs to handle (two AUVs on each side). The task management module 110 assesses the urgency and value of tasks, and completes task allocation based on the task value assessment using a consensus-based auction algorithm. The multimodal dynamic communication module 120, during operation, actively detects communication channels and dynamically switches communication modes based on the AUV's built-in sensors and environmental assessment, completing positioning consensus, task coordination, and data synchronization within the communication window. The dynamic weighted collaborative positioning module 130, based on local SLAM, constructs shared variables and dynamically calculates weights within the communication window based on communication quality, power consumption, computing power, and historical estimation accuracy. It updates shared variables using weighted ADMM consensus and uses the consensus as a priori for local optimization, thereby achieving high-precision distributed positioning. The AUV receives and allocates tasks and executes transport tasks based on path planning using a reinforcement learning algorithm. The intelligent energy management module 150 continuously monitors power consumption and executes charging decisions and scheduling based on thresholds. The fault detection and handling module 160 monitors various indicators and triggers task reassignment or emergency procedures in case of faults or fault warnings.

[0053] In some exemplary embodiments, the task management module uses a consensus-based auction algorithm to allocate tasks. The value of a task is calculated by weighting the task urgency, capability matching degree, execution efficiency assessment and estimated cost. The task urgency is calculated by combining the normalized result of the original urgency with the time urgency factor, and a penalty coefficient is applied to overdue tasks.

[0054] For example, task urgency assessment:

[0055] Original urgency (Values ​​range 1–10) Normalized to:

[0056] ∈[0,1]

[0057] Calculate the time urgency factor

[0058]

[0059] in, Indicates the task deadline. Indicates the task creation time. To represent a very small positive value, avoid division by zero. Indicates the current time.

[0060] Overall urgency ,in, ( , (Adjustable weights).

[0061] Overdue processing: If ,but:

[0062]

[0063] in, This indicates the penalty coefficient for exceeding the time limit.

[0064] The task management module 110 employs a consensus-based auction algorithm (CBAA) to achieve adaptive, load-balanced task allocation and redistribution among multiple AUV groups. This method treats each task as a "commodity" to be auctioned, with AUVs acting as "bidders." Through iterative bidding and consensus processes, it ensures that each task is undertaken by the most suitable AUV or combination of AUVs. The task allocation process is as follows: Figure 3 As shown. The AUV capability assessment includes evaluating whether an AUV can handle cargo handling tasks based on cargo type and AUV performance: small cargo: any AUV can handle it; medium cargo: requires at least a medium-performance AUV; large cargo: requires a high-performance AUV. The AUVs in this system have different computing capabilities, memory sizes, and performance.

[0065] Each Calculate the task value of undertaking the task based on its own state and the characteristics of task j. As a bid, each AUV node maintains clock consistency via a time synchronization protocol and freezes the time parameters at the start of the auction round. Until the allocation is complete.

[0066] Task value The general structure of a function is as follows:

[0067]

[0068] All of these values ​​are normalized values ​​(0–1). is a positive integer greater than or equal to 0, used to represent the normalized weight of the corresponding term. + =1. This indicates the overall urgency level (utility item). Calculated by the task management module, it reflects the time urgency of the task. This indicates the degree of matching ability (utility). The degree to which the load capacity, mechanical interface, etc., are matched with the requirements of task j. This represents an efficiency assessment (utility item). It comprehensively considers path efficiency (distance to the starting point, obstacle risk) and energy efficiency (based on path length, speed, and remaining battery power). This represents the task cost (penalty term), a normalized positive value. Consider... The current load factor Li (to avoid overload) and expected travel time of task j are calculated.

[0069] For tasks requiring multi-AUV collaborative handling (such as medium to large cargo), the allocation process involves AUV grouping. For example, medium cargo requires three AUVs with similar performance forming a triangle formation, and the bidding calculation is based on the collective value assessment of the task by the AUV grouping. Large cargo requires four high-performance AUVs, divided into left and right groups, and the bidding calculation is based on the collective value assessment of the task by the AUV grouping.

[0070] The consensus-based auction algorithm (CBAA) process is as follows:

[0071] Allocation is achieved through iterative communication and consensus among AUV groups:

[0072] Initialization: All AUVs maintain a local task allocation table. ,express The optimal allocation as perceived.

[0073] Each Calculate its value for all unassigned tasks value and the highest bid Broadcast to the neighbors.

[0074] Maintain a record of each task Best bidder and best offer The local view.

[0075] Received from neighbor After receiving the information, update the local view:

[0076]

[0077]

[0078] Through iterative communication, the AUV community gradually reaches a consensus on which task should be bid on by which AUV (or AUV combination) with the highest value. It tends to be consistent across all AUVs.

[0079] Once consensus converges (or the maximum number of iterations is reached). Ultimately, they accepted the task with the highest bid, which they eventually won.

[0080] This distributed, consensus-driven allocation method improves the robustness and real-time performance of allocation, and avoids the risk of failure of the central node.

[0081] The system continuously monitors the AUV's status (battery level, fault warnings) and mission progress.

[0082] Triggering conditions: When the AUV malfunctions (determined by the fault detection module) or the system load is unbalanced, task redistribution is triggered.

[0083] Reassignment: The affected tasks are marked as unassigned and re-enter the CBAA process described above. However, the task value will be dynamically adjusted based on factors such as task urgency, AUV's current location, and battery level. The weights are determined to prioritize fault recovery and system efficiency.

[0084] In some exemplary embodiments, the multimodal dynamic communication module 120 includes a performance evaluation unit and a switching decision unit; the performance evaluation unit is used to evaluate the channel quality of acoustic, optical and magnetic induction communication in the current and predicted environment; the switching decision unit triggers communication mode switching when the predicted utility of the current communication mode is lower than a threshold and there is a better candidate mode.

[0085] For example, the multimodal dynamic communication module 120 is used to establish reliable multimodal communication links between multiple AUVs and between AUVs and shore-based / mother ships, supporting three communication modes: acoustic communication, optical communication, and magnetic induction communication. Under hardware-feasible conditions, this module integrates environmental perception information, mission urgency, AUV attitude status, and energy status to perform real-time modeling, evaluation, and prediction of the availability and communication quality of different communication modes, and dynamically switches between communication modes, providing underlying information support for mission allocation, distributed positioning, cooperative control, and charging docking.

[0086] Dynamic communication switching process as follows Figure 4 As shown, the specific steps are as follows:

[0087] Step 1: Environmental Sensing and Status Acquisition: Each AUV periodically collects the following information: Environmental parameters: Water turbidity is collected via a turbidity sensor. Underwater noise signals are collected using acoustic sensors, and then the power spectral density is obtained through hardware / software analysis. Motion status: Distance is measured using sonar (or blue-green laser or magnetic induction). Position is obtained by combining inertial navigation system (INS) with Doppler velocimetry (DVL) data. Speed ​​was measured using a Doppler velocimeter (DVL). Attitude is acquired through an inertial measurement unit (IMU). (Pitch, roll, yaw). Mission status: Urgency. Energy demand Data rate requirements Output: Current state vector .

[0088] Step 2: Communication Mode Performance Evaluation: The real-time performance of each communication mode is evaluated based on the following model: Acoustic Communication (medium to long distance, tolerating certain delays, insensitive to turbidity): Acoustic communication uses sonar as the communication device. The acoustic channel is mainly used for medium to long distance communication and is the basic control communication link of the system. In the underwater environment, the propagation loss of acoustic signals is mainly determined by the spread loss and absorption loss. Its propagation loss model can be expressed as:

[0089] in, It is the geometric expansion factor. Suitable for surface acoustic channels and deep-sea acoustic channels, cylindrical wave propagation; Applicable to open water (free field), spherical wave propagation, where d represents the communication distance (m). This indicates the absorption rate of seawater (dB / km).

[0090] Based on the above propagation loss model, the received signal-to-noise ratio (SNR) of acoustic communication (…) This can be represented as:

[0091]

[0092] in, This indicates the sound source level at the communication transmitter (corresponding to the transmission intensity of the communication signal). Indicates the ambient noise level; This indicates the directivity index of the receiving transducer (which can reduce noise).

[0093] SNR prediction (linear approximation) can be expressed as:

[0094]

[0095] in, It can be estimated using moving regression or Kalman filtering; τ represents the prediction time window (s); This represents the acoustic signal-to-noise ratio at the current time t. This represents the signal-to-noise ratio at the predicted future time t+τ. This represents the rate of change of the signal-to-noise ratio at the current moment (unit: change of signal-to-noise ratio per second).

[0096] The communication module measures the signal-to-noise ratio in real time using sonar, and indirectly derives the packet loss rate through a simple communication protocol design. This rate is then used as a quantitative indicator of the current availability of acoustic communication for subsequent communication utility evaluation and mode switching decisions.

[0097] Optical communication is characterized by short range, clear water conditions, and the requirement for precise attitude alignment. The optical communication submodule uses blue-green LEDs or laser diodes as transmitters and avalanche photodiodes as receivers. FPGAs or MCUs are used for signal encoding (such as Manchester encoding, 8b / 10b) and decoding, as well as processing communication protocols.

[0098] Optical link received power The formula can be expressed as:

[0099]

[0100] in, Represents receive / transmit power; Indicates optical transmit / receive efficiency; Indicates the beam attenuation coefficient (absorption) +scattering , ); Indicates transmission distance; The aperture area of ​​the receiver; Indicates the angle between the transmitter beam and the receiver beam; This indicates the divergence angle of the transmitter.

[0101] The rotation matrices of the receiver / transmitter AUVs are respectively / The displacement vector of an AUV in its own body coordinate system can be expressed as: The rotation matrix of the AUV is in These represent the AUV's yaw angle about the Z-axis, pitch angle about the Y-axis, and roll angle about the X-axis, respectively; reception angle. The formula is: ,in This represents the displacement vector from the transmitter to the receiver in the receiver's coordinate system. This is the unit direction vector representing the receiver's attitude. (Emitting angle) The formula used to evaluate the pointing accuracy of the emitted light signal from the AUV at the transmitting end is: ,in, This represents the displacement vector from the transmitter to the receiver in the transmitter's coordinate system. This is the unit direction vector of the transmitter's attitude.

[0102] The communication module estimates the received light intensity in real time through the optical receiver and, in conjunction with the water turbidity parameters provided by the environmental sensing module, assesses the instantaneous availability of optical communication. When the predicted quality of the optical communication link is higher than that of acoustic communication and the communication distance meets the constraints, the system prioritizes the optical communication mode.

[0103] Magnetic induction communication is characterized by its extremely short range and suitability for turbid waters or areas with dense metal equipment. Each AUV in the magnetic induction communication submodule is equipped with a triaxial orthogonal coil antenna system wound on a ferrite core to ensure omnidirectional magnetic field coverage. The circuit operates in resonant mode and is equipped with tuning capacitors for impedance matching. It is primarily used for short-range communication in heavily obstructed environments, utilizing the high penetration of magnetic fields into non-conductive media (mud, sand, water) to overcome dead zones in underwater communication.

[0104] Assuming the coils are coaxially aligned, the formula for calculating the mutual inductance M between the transmitter and receiver is: The magnetic induction communication system is modeled as a transformer-coupled circuit. According to Kirchhoff's Voltage Law (KVL), the voltage-current relationship between the transmitting circuit (primary side) and the receiving circuit (secondary side) is as follows: Among them, the total impedance of the circuit and Defined as:

[0105]

[0106]

[0107] Emission current and receiving induced current :

[0108]

[0109]

[0110] Transmit power ( ) and received power ( ):

[0111]

[0112]

[0113] Path loss ( The attenuation of a communication link is defined as the logarithm of the ratio of received power to transmitted power.

[0114]

[0115] in, Indicates the magnetic permeability of the underwater medium. Indicates the number of turns in the AUV transmitting coil. Indicates the number of turns in the AUV receiver coil. Indicates the radius of the AUV transmitting coil. d represents the radius of the AUV receiving coil, and d represents the spacing between the cooperating AUVs. Indicates signal frequency (Hz). This represents the peak current (A) in the transmitting coil. Indicates angular frequency; ; This represents the total impedance of the transmit / receive circuit; This indicates the load resistance at the receiving end; Indicates the internal resistance of the coil; Indicates coil inductance; This indicates a tuning capacitor.

[0116] Step 3: Actively detect and predict communication channels, and assess the probability of interruption. Underwater channels are subject to severe fluctuations, and communication often only passively switches after the link is broken, causing disruptions to logistics control. This module utilizes the AUV's built-in sensors (sonar, camera, magnetometer) to actively detect communication channels while performing transport tasks. A turbidity sensor detects water turbidity to help determine the attenuation coefficient of the optical communication link, and a Doppler velocimeter estimates relative velocity to correct frequency compensation for acoustic communication. This achieves a closed loop where "communication serves perception, and perception reacts to communication," significantly reducing the "blind spot time" of communication switching. Simultaneously, kinematic and environmental models are used to predict future states.

[0117] The kinematic predictions are as follows:

[0118]

[0119]

[0120] in, α represents the angular velocity vector (rad / s), and α represents the angular acceleration vector (rad / s²). This represents the position vector of the AUV. This represents the velocity vector of the AUV.

[0121] The environmental change predictions are as follows:

[0122]

[0123] Indicates the predicted time The turbidity of the water body; Indicates the turbidity at the current moment; It represents the gradient of turbidity in the spatial direction x (representing the rate of spatial change, such as the trend of turbidity increase or decrease along the direction of AUV travel). This represents the velocity component (m / s) of the AUV along the x-direction.

[0124]

[0125] The ambient noise power spectral density (W / Hz) represents the predicted time. : Noise power spectral density at the current moment; The rate of change of noise over time (i.e., the trend of noise increase or decrease).

[0126] Calculate the interruption probability components for each communication mode:

[0127] Each indicator is first converted into a probability using a mapping function:

[0128]

[0129] When SNR is close to or below the threshold At that time, communication reliability dropped sharply. Approaching 1.

[0130]

[0131] Optical communication is very sensitive to attitude; when the AUV deflects too much, the link is quickly interrupted.

[0132]

[0133] When the water becomes turbid or absorption is enhanced, K(c) increases, and communication attenuation becomes significant. Close to 1.

[0134]

[0135] This is a human-weighted item, derived from experience. The more critical the task, the more "sensitive" the system is to the risk of interruption, and the higher the probability of switching over in advance.

[0136] The total interruption probability is obtained by fusion. as follows:

[0137]

[0138] This represents the interruption probability based on the signal-to-noise ratio. Indicates the signal-to-noise ratio threshold. The signal-to-noise ratio sensitivity coefficient (dB) - ¹), This represents the probability of interruption based on alignment error. This represents the attitude angle difference between two AUVs. This indicates the attitude error threshold (the communication link will be significantly weakened after exceeding this angle). Represents the attitude sensitivity coefficient (rad) - ¹), This represents the probability of interruption based on environmental changes. The total environmental attenuation coefficient, m - ¹, K(c) = a(c) + b(c) is determined by the turbidity c. Indicates the communication attenuation threshold, m - ¹, This represents the environmental sensitivity coefficient. This represents the probability of interruption risk based on task urgency. The more urgent the task, the lower the system's tolerance for communication interruptions. This parameter represents the urgency of the task, with a value range of [0,1]. 0 indicates a normal task, and 1 indicates an urgent task (such as cargo handling control). Indicates the urgency sensitivity coefficient. , This represents the corresponding weight coefficient, which satisfies .

[0139] according to Determine the warning level (green / yellow / orange / red).

[0140] Step 4: Communication mode utility evaluation. Based on the above three types of communication models, the communication module evaluates the utility of different communication methods. Construct a unified communication utility function, current utility It can be represented as:

[0141]

[0142] Where m represents the communication mode (acoustic, optical, magnetic induction). This represents the data rate matching function, which measures the rate of pattern m. With task requirements The degree of satisfaction; The function represents the link reliability. The power consumption function is given, considering the power consumption of mode m. Energy demand / remaining power Relationship; The link availability function represents the availability of mode m under the current attitude ϕ. This represents the cost of switching between different communication modes; The available data rate (bps) represents the current / predicted pattern m. , This indicates the current / predicted link reliability metric (which can be used as a complement to success probability, packet loss rate, etc.), and is dimensionless or a percentage. The power consumption metric (W) for mode m is used to represent the power consumption of mode m. calculate; Indicates the energy requirements of the task / system or the relevant indicators of the node's remaining energy (which can be a percentage of electricity or an estimate of remaining energy, in units of J or %). The availability metric for mode m (e.g., antenna orientation reachability, link establishment probability) is dimensionless. , Represents the current / predicted attitude vector (pitch, roll, yaw), in rad or deg; Indicates the data rate (Rate) weight. Indicates the link reliability weight. Indicates power consumption / energy (Power) weight. Indicates availability / alignment weights. The weight representing the switching cost (used to deduct switching overhead); , , , , This represents the corresponding dynamic weighting coefficient.

[0143] Predicted utility It can be represented as:

[0144]

[0145] Fusion effect It can be represented as:

[0146]

[0147] in, (Based on confidence-adaptive). Control the relative weights of current utility and predicted utility. , This represents the uncertainty (variance) between the current utility estimate and the predicted utility estimate. Unit: Variance units of U (dimensionless if U is dimensionless). Note: These variances can be obtained from filters (such as Kalman) or historical residual estimates.

[0148] Based on task urgency Dynamically adjust weights:

[0149]

[0150] in, This represents the urgency index of the task, with a value range of [0,1] (1 being the most urgent). This indicates the urgency threshold (when U exceeds this value, it enters emergency mode).

[0151] The communication control unit performs real-time selection and smooth switching of communication modes based on the utility function results.

[0152] Step 5: Switch Decisions ; .

[0153] in, The utility threshold is represented; m′ represents the candidate target communication mode (acoustic / optical / magnetic induction). This indicates the switching lag amount to prevent mode oscillation. This indicates that once a mode is switched to, the user must remain in that mode for at least that duration before being allowed to switch again, to prevent jitter. The unit is seconds (s). This sets the minimum dwell time. Prevent frequent oscillations.

[0154] Step 6: Seamless Execution Switch

[0155] exist Under the specified conditions, handover is performed, and the acoustic channel is maintained as the control link during handover to ensure uninterrupted command transmission. Before handover, the transmission rate is proactively reduced via the flow control protocol, and priority pruning is performed on the data queue, retaining only high-priority control commands in the acoustic link buffer.

[0156] Data packets are synchronized with cache and timestamps to achieve a smooth transition; probability of successful handover:

[0157]

[0158] in, This represents the probability of a successful switch on the i-th attempt; Indicates the time required for the switchover; Indicates the lead time for prediction. (How far in advance the system predicts / assesses that an outage will occur), in seconds. This indicates time estimation (predicted time). With switching time The uncertainty (standard deviation) of ). Unit: seconds. It is the standard normal distribution function, used to convert time difference into probability confidence level.

[0159] Step 7: Feedback and Learning. Record the communication performance, utility accuracy, and interruption events of this switchover, and update the weights. Prediction window Model parameters, adaptively adjusting the prediction time window:

[0160]

[0161] in, Indicates the uncertainty of prediction. , Indicates the minimum and maximum forecast times. Represents the rate of change of state. This represents the adjustment coefficient.

[0162] In some exemplary embodiments, the dynamic weighted cooperative positioning module 130 assigns dynamic weights to each autonomous underwater vehicle (AUV) by means of a communication quality assessment factor, a battery status assessment factor, a computing power assessment factor, and a historical positioning accuracy assessment factor. The communication quality assessment factor reflects the stability and latency of the communication link; the battery status assessment factor reflects the current battery level of the vehicle; the computing power assessment factor reflects the performance and load status of the vehicle's processor; and the historical positioning accuracy assessment factor reflects the reliability of the vehicle's past positioning estimates.

[0163] For example, the dynamic weighted cooperative localization module 130 is responsible for estimating the six-degree-of-freedom (6-DoF) pose of the AUV using multi-source sensor data in underwater communication-constrained environments. Considering the nonlinear characteristics of AUV attitude, this module abandons the traditional Euclidean space linear averaging method and adopts a method based on... The weighted Manifold Edge-based Separable ADMM algorithm for Lie group manifolds, combined with the positioning confidence weights of each AUV, achieves globally consistent high-precision positioning.

[0164] Positioning system process as follows Figure 5 As shown, it includes the following steps:

[0165] Step 1: Hardware Configuration and Underlying Sensing Mechanism. The positioning system relies on the physical hardware mounted on the AUV to perceive its environment and its own state, providing raw data support. Specifically, the Inertial Measurement Unit (IMU) and depth gauge use a three-axis accelerometer and gyroscope to sense the AUV's acceleration and angular velocity, combined with a hydrostatic depth sensor to obtain vertical depth information. This hardware data forms the basis for local state estimation. The Doppler velocimeter (DVL) utilizes the acoustic Doppler effect to sense the AUV's three-dimensional velocity relative to the bottom through bottom track velocity measurement. Acoustic transducers and positioning arrays are used for relative ranging and baseline positioning between AUVs. When the acoustic hardware of AUV-i detects AUV-j, it calculates its relative position in the local coordinate system as a physical trigger condition for creating shared variables. Computational and storage hardware, including embedded processors (CPU / GPU) and high-bandwidth memory, is used to maintain the factor graph structure in real time and perform high-frequency matrix operations and ADMM iterations.

[0166] Step 2: State Definition and Space Construction: The State of the AUV Defined in special Euclidean groups Above. Based on motion data acquired by hardware, the state of the AUV is obtained. Its state is represented as follows: ,in For rotation matrix, Let the position vector be denoted. Define a generalized error function on the manifold. Using Lie algebra Logarithmic mapping on the surface calculates state differences, avoiding gimbal lock-up and large-angle linearization errors: .

[0167] Step 3: Weight Calculation: The localization module is used to create shared variables when acoustic or optical sensing hardware observes other AUVs or common landmarks, based on local SLAM (IMU / DVL data) running on the AUV. This integrates the communication quality of the AUVs. Battery status Computational ability and historical estimation accuracy ,pass Calculate the overall weight, and then... Normalized weights.

[0168] Communication quality assessment, monitoring packet loss rate :

[0169]

[0170] Measuring communication delay :

[0171]

[0172] Calculate the communication quality weighting factor :

[0173]

[0174] in, and It is an adjustment parameter used to balance the influence of different factors.

[0175] Battery status assessment, penalizing only when the battery level is too low, using the following formula:

[0176]

[0177] in, This is the current battery level. This represents the battery capacity threshold.

[0178] Computational capability assessment, evaluating processor performance, memory size, etc.; This is a computational capability score (a comprehensive score based on processor speed, memory size, etc.), where N is the total number of AUVs in the network, and the computational capability weighting factor is:

[0179]

[0180] Historical estimation accuracy assessment:

[0181] Collect the deviation between historical estimates and consensus values:

[0182]

[0183] Calculate the historical estimation accuracy weighting factor:

[0184]

[0185] in, It is the true value of the shared variable at the k-th consensus (if there is an external reference). yes The estimation of shared variables at the k-th consensus. It is the size of the history window.

[0186] Step 4: Distributed Weighted ADMM consensus update, when and When establishing communication, perform ADMM consensus update:

[0187]

[0188] in, It is a local cost function (e.g.) ), It is the manifold error function. yes and Marginal consensus variables, weights (and ).

[0189] ADMM Iteration Steps (the first step) (nth iteration)

[0190] Local primitive variable update (PrimalUpdate / x-update):

[0191] A factor graph is constructed based on local sensor measurements, and the marginal consensus variables calculated in the previous round are used. As a "biased prior factor," it is incorporated into the optimization objective to solve for the local optimum estimate. :

[0192]

[0193] in, It is a penalty parameter. These are dual variables. The local estimate is swapped with the weights, when... and During communication: they exchange their local estimates of the shared variable. and Exchange their respective weights: and Global consensus variable update (ConsensusUpdate / z-update): The MESA algorithm uses "edge pairwise interpolation". After introducing weights, weighted interpolation is used to calculate dynamic weight coefficients. :

[0194]

[0195] Update edge consensus variables (Use manifold interpolation functions such as Split or SLERP):

[0196]

[0197] Update dual variables: using manifold error function Update the Lagrange multipliers:

[0198]

[0199] Local SLAM optimization:

[0200] Adding a "biasedprior" factor to the local factor graph will pull the local estimates of shared variables toward the consensus value. :

[0201]

[0202] Step 5: Convergence and Final Output: After multiple iterations, when the manifold constraint error... The algorithm converges when the value is below the threshold. The final output is... This means that the local high-precision inertial navigation system (high weighting) is automatically fused. ( ) and the global optimal estimate of group consensus information.

[0203] Step 6: Iterative Optimization: The entire localization process is an iterative optimization loop.

[0204] Continuous operation: Each AUV continuously runs local SLAM, constantly updating its state estimate.

[0205] Periodic communication: AUVs communicate with other AUVs according to a preset communication cycle or based on event triggers (such as the detection of new shared variables).

[0206] Weight update: Before each communication, AUV recalculates its dynamic weight.

[0207] Consensus Update: Update the consensus value of shared variables using the ADMM algorithm.

[0208] Local optimization: Use consensus values ​​as prior information to optimize the local SLAM system.

[0209] Result evaluation: Evaluate the positioning accuracy. If the termination condition is met, stop; otherwise, continue iterating.

[0210] The positioning module finally outputs the converged value. Pose matrix ,Will The position part in the middle Projected into Euclidean space, it is used to calculate the Euclidean distance, thereby achieving metric decoupling between state estimation and task planning.

[0211] In some exemplary embodiments, the multi-branch attention mechanism includes: a local attention branch for calculating obstacle threat level and companion threat level and normalizing them into weights; a global attention branch for calculating path relevance and boundary distance relevance and normalizing them into weights; a temporal attention branch for calculating the temporal change rate of each state feature group and normalizing it into weights; and the weights of the local attention branch, global attention branch, and temporal attention branch are fused according to a preset ratio to form the final attention weight for state feature weighting.

[0212] Figure 6 The flowchart illustrating the path planning and formation control of the reinforcement learning path planning control module according to an embodiment of the present invention is shown.

[0213] like Figure 6 As shown, the specific steps of the path planning and formation control process of the reinforcement learning path planning control module according to an embodiment of the present invention are as follows:

[0214] Step 1: Problem modeling and mathematical representation of the environment, including probabilistic modeling of the 3D underwater environment. In the process of probabilistic modeling of the 3D underwater environment, a global and local two-layer map is defined.

[0215] Global map ( ):

[0216] in , The length is in the horizontal direction. The depth dimension is set according to the actual task, and must cover the entire path from the starting point to the target point, with a 5m safety zone reserved.

[0217] Representing the entire underwater operation area, it is generated by probabilistic fusion of local maps of all AUVs, and is defined as follows: ,in It is the first Each AUV constructs a subset of the global map based on its own observations. The global map is discretized using a 3D voxel grid, with a voxel resolution set to [value missing]. (Balancing spatial precision and computational complexity). The global voxel index number is:

[0218]

[0219] Each voxel The state is represented by the occupancy probability. describe, (0 indicates free space, 1 indicates space occupied by an obstacle).

[0220] Partial map ( ) The real-time environmental characterization of an AUV within a 50m observation radius, with the mathematical boundary as follows:

[0221]

[0222]

[0223] in yes The real-time position coordinates are obtained by fusing inertial navigation and sonar positioning. This represents the voxel-occupied state. A schematic diagram of the local and global maps is shown below. Figure 7 As shown.

[0224] Real-time updating of local map occupancy probability based on Bayes' theorem, assuming... time The sensor observation sequence is (Sonar echo intensity, image features, etc.), pose sequence is (Position + Attitude) Time voxels The posterior occupancy probability is :

[0225]

[0226] in: These are normalization coefficients to ensure that the sum of probabilities is 1; It is a sensor likelihood model; if voxels are observed... If an obstacle exists, the likelihood value is set to 0.9; otherwise, it is set to 0.1. yes The prior occupancy probability at a given time.

[0227] To simplify the calculation, a log-odds transformation is introduced:

[0228]

[0229] The update formula can then be linearized:

[0230]

[0231] The global map is generated by weighted probabilistic fusion of local maps from all AUVs, transmitting only the coordinates of detected obstacles (sparse features) to avoid system paralysis due to communication congestion. The core principle is to integrate multi-source observations based on Bayes' theorem, with weights positively correlated with the observation distance from the AUV to the voxel (the closer the observation, the greater the weight, improving fusion accuracy).

[0232] The specific assembly process is as follows:

[0233] ①: Local map preprocessing

[0234] Each AUV at time step After completing the local map update, coordinate transformation is performed: the relative coordinates of the local map (centered on the AUV) are converted to the absolute coordinates of the global map (based on the real-time pose of the AUV). Local voxel centers In the i-th AUV body coordinate system, after attitude With position Transform to global coordinates:

[0235]

[0236] The corresponding global voxel index is:

[0237] in, .like Within the global map scope, voxels are mapped and participate in fusion.

[0238] ②: Weighted fusion calculation

[0239] First, calculate the original weights of each AUV observation.

[0240] in AUV to voxel Euclidean distance (unit: m). To avoid a denominator of 0, weight normalization is then performed. Ensure the sum of the weights is 1:

[0241]

[0242] Global Map time The update formula is:

[0243]

[0244] Inverse transformation: Convert the fused logarithmic odds value back to the occupied probability to obtain the global voxel probability. :

[0245]

[0246] Update frequency: Synchronized with the local map, every Fusion once; Consistency check: If the occupancy probability fluctuation of a voxel is ≤0.1 for three consecutive time steps, it is marked as a "stable voxel"; if the fluctuation is >0.3, the AUV covering the voxel is re-observed to avoid error accumulation.

[0247] The mathematical constraints of the collaborative transport task, where the task objective is for N AUVs to collaboratively transport goods from the starting point. To the target point The formation is selected based on the cargo size (equilateral triangle formation for medium-sized cargo, square formation for large cargo). When the formation constraint model is an equilateral triangle formation (3 AUVs), the center point of the cargo is set as... , The ideal location is:

[0248]

[0249] in: The radius of the circumcircle of the formation (take 5-10m depending on the size of the cargo, for medium-sized cargo). ); To bypass Rotation matrix of the axis (vertical axis), The overall heading angle of the formation.

[0250] For direction unit vectors:

[0251]

[0252]

[0253]

[0254] When the formation is square (4 AUVs), The ideal location is:

[0255]

[0256] Directional unit vector for:

[0257]

[0258]

[0259]

[0260]

[0261] Take 8-12m (large cargo) ).

[0262] Formation switching trigger conditions include: Cargo size trigger: Medium cargo → Triangular formation; Large cargo → Square formation; Mission phase trigger: Initial phase (cargo not stable): Loose formation, allowable deviation ±3m, or no strict formation, only maintain a safe distance. Cruise phase (cargo stable): Standard formation; Obstacle avoidance phase (encountering large obstacles): Temporarily disband formation (maintain a safe distance of 3m), restore the original formation within 10 time steps after obstacle avoidance.

[0263] AUV speed increment during formation switching angular velocity increment (To avoid sudden changes in movement that could cause cargo to shift); Switching completion determination: Deviation of all AUVs from the ideal position. And it lasts for 3 time steps.

[0264] The conversion from a voxel map to an equivalent sphere includes: the local map The probability of occupying the middle Euclidean clustering is performed on the voxels to divide spatially adjacent voxels into independent obstacle clusters. .

[0265] Computational clusters The geometric center (centroid) as the location of the obstacle .

[0266] Traverse all voxel centers in the cluster Calculate its maximum distance to the centroid, and use it as the equivalent radius. :

[0267]

[0268] (in It is half the diagonal of the voxel and is used to compensate for voxel discretization errors.

[0269] The obstacle is represented by a spherical equivalent with an equivalent radius of . , The radius of the AUV, For additional safe distance:

[0270]

[0271] Static obstacles: in the global map For the voxel region, AUV must meet the following requirements:

[0272]

[0273] For a safe distance, ( () is the center of mass of the obstacle.

[0274] Dynamic obstacles: Moving targets (such as other AUVs and aquatic organisms) detected in real time on a local map are designated as dynamic obstacles. The predicted location is (In the dynamic obstacle trajectory prediction stage, considering the limited underwater computing resources and the high-frequency characteristics of system decision-making, this embodiment adopts a short-time prediction model based on first-order kinematic approximation:)

[0275]

[0276] Then AUV must meet the following requirements:

[0277] .

[0279] Step 2: Reinforcement learning path planning control module structure diagram as shown below Figure 8 As shown, the mathematical modeling process of the MADDPG algorithm is as follows:

[0280] Define MAMDP tuple The mathematical definitions of each symbol are as follows:

[0281] Global state , No. Local state of an AUV ( (The "feature vector dimension" of the engineering adaptation is 103 dimensions after direct concatenation of the original states)), specifically its structure:

[0282]

[0283] in, (Location, 3D) (Speed, 3D) (Attitude, roll angle, pitch angle, yaw angle, 3D); It is the feature vector of the local map (occupancy probability of 64 voxels sampled within a 50m range, 64 dimensions). yes The relative positions of the five nearest obstacles within 50m ( dimension); yes Relative position to other AUVs ( dimension, (Time is 9-dimensional) ( Relative position to the cargo center (3D). (Relative position of cargo center and target point, 3D); The state space structured feature design table is shown in Table 1.

[0284] Table 1. State-space structured feature design table

[0285]

[0286] The module as a whole follows a serial structure of "feature grouping → attention weight calculation → weighted fusion → feature output". The three attention branches (local, global, and temporal) are calculated in parallel and then weighted and fused, as detailed below:

[0287] ① Local attention branch (focusing on "critical threats at close range")

[0288] The core objective is to make the model prioritize obstacles within 5m and teammates within 2m, thereby enhancing its sensitivity to local obstacle avoidance and formation maintenance.

[0289] The mathematical modeling method is as follows:

[0290] Step 1: Calculate the "Threat Score" (for obstacle group G3 and teammate group G4): For each obstacle k, the threat level is... ( The equivalent radius of the obstacle. The distance between the AUV and the obstacle is expressed in Euclidean form. (To avoid a denominator of 0); for each companion j, the threat level ( The distance between the AUV and its companions; the closer the distance, the higher the threat level.

[0291] Step 2: Normalize the threat level to the attention weight:

[0292] (Obstacle attention weights are adaptively assigned using the Softmax function, allowing the network to automatically focus on high-threat obstacles.) (Peer attention weights); Local attention weight vector

[0293]

[0294] (There are 5 feature groups, only G3 and G4 have non-zero weights).

[0295] ② Global attention branch (focusing on "task-oriented information")

[0296] Core objective: To enable the model to focus on the cargo-to-target path and its own global position, avoiding deviation from the task's direction.

[0297] The mathematical modeling method is as follows:

[0298] Step 1: Calculate the "Task Relevance Score" (for its own state group G1 and task group G5): Path Relevance :

[0299]

[0300] The shorter the remaining path, the higher the correlation.

[0301] Location correlation :

[0302]

[0303] in, This is the distance from the AUV to the global map boundary, to avoid weight boosting at the boundary.

[0304] Step 2: Normalize the correlation as attention weights:

[0305] (Weight of its own state group)

[0306] (Task group weight)

[0307] Global attention weight vector (Only G1 and G5 have non-zero weights).

[0308] ③ Time-based attention branch (focusing on "recent state changes")

[0309] The core objective is to enable the model to focus on the state differences of the three most recent time steps and capture the dynamic relationship between actions and states (such as rapidly approaching obstacles).

[0310] The methods for mathematical modeling are as follows:

[0311] Step 1: Calculate the "Time Change Rate Score" (for time series of all feature groups): Let the states of the most recent 3 time steps be: Calculate the rate of change for each feature group. :

[0312] ( (The greater the rate of change, the higher the weight).

[0313] Step 2: Normalize the rate of change for attention weights :

[0314] (Time attention weights for each feature group)

[0315] Temporal attention weight vector

[0316] .

[0317] ④ Multi-scale attention fusion

[0318] The weight vectors of the three branches are weighted and fused to obtain the final feature group attention weights. :

[0319] (g=1-5);

[0320] Weighting coefficients (empirical values): α=0.5 (local obstacle avoidance priority), β=0.3 (task-oriented), γ=0.2 (dynamic capture); Constraints: (Normalization ensures that the sum of weights is 1).

[0321] ⑤ Feature weighting and dimensionality preservation

[0322] The original features of each feature group are weighted while maintaining the 103-dimensional dimension:

[0323]

[0324] In this case, each element of G3 (15-dimensional) is multiplied by Each element of G5 (6-dimensional) is multiplied by This ensures that the information from key feature groups is amplified.

[0325] The action of each AUV is a continuous control quantity. ( ), defined as:

[0326]

[0327] in: (Velocity increment, limiting the maximum acceleration of the AUV) ); (Angular velocity increment, limiting the maximum angular acceleration of the AUV) ).

[0328] Joint Action Space Satisfying AUV dynamic constraints:

[0329]

[0330] in, ;

[0331] .

[0332] State transition probability Description in joint actions Below, from the state arrive The transfer law is derived based on the AUV kinematic model:

[0333]

[0334]

[0335]

[0336]

[0337] in, For time step, local map According to the new location Resample and update.

[0338] Employing a weighted multi-objective reward function Balancing obstacle avoidance, formation maintenance, mission progress, energy consumption, motion smoothness, and boundary constraint rewards is defined as follows:

[0339]

[0340] The mathematical definitions of each sub-reward are as follows:

[0341] ① Obstacle avoidance reward ( ):

[0342]

[0343] in yes With the The Euclidean distance between the obstacles yes and The Euclidean distance. When the distance is less than the safety distance, the penalty increases as the distance decreases; when the distance is greater than the safety distance, .

[0344] ② Formation maintenance reward ( ):

[0345] Triangular formation:

[0346] Square formation:

[0347] in yes The ideal formation position has a negative correlation between the reward value and the square of the formation deviation; the smaller the deviation, the greater the reward.

[0348] ③ Task progress rewards ( ):

[0349]

[0350] in This is the current location of the cargo center. It is the center position of the cargo at the next moment (calculated by weighting the AUV positions): As the goods move toward the target point, , A positive value indicates a positive outcome; conversely, a negative value indicates a negative outcome.

[0351] ④ Energy consumption bonus ( ):

[0352]

[0353] in, (Speed ​​energy consumption coefficient) (Linear acceleration energy consumption coefficient) (Angular acceleration energy consumption coefficient) is used to fit the actual energy consumption model of the AUV propulsion system and attitude control system, and to penalize high-energy-consuming actions.

[0354] ⑤ Smooth motion reward ( )

[0355]

[0356] in, The motion vector at time t (velocity increment + angular velocity increment). Indicates the motion smoothing threshold. Smooth reward factor, Mutation penalty factor.

[0357] ⑥ Boundary penalty function ( )

[0358] Safe area (within 5m safety belt):

[0359]

[0360] Warning area (outside the seatbelt, not exceeding the global boundary):

[0361]

[0362] Restricted Area (outside the global map):

[0363] x<0 or y<0 or z<0 or .

[0364]

[0365] Where λ represents the penalty intensity coefficient. This represents the distance from the AUV to the nearest global boundary (in meters).

[0366] Punishment logic:

[0367] Safe Zone ( ≥5m):

[0368] =0 (No penalty)

[0369] Warning area (0< <5m): The penalty increases linearly as the distance decreases.

[0370] restricted area( ≤0):

[0371] =−5.0 (Mission failure level penalty), triggering AUV emergency braking.

[0372] Mission termination conditions include: mission success conditions, cargo center reaching the target area: (Target area radius 2m), and maintain for 2s (corresponding to 20 time steps); and mission failure conditions (any one condition must be met): Collision occurs: Distance between the AUV and the obstacle / other AUV << Out of bounds: The AUV or cargo center is outside the global map range; Energy depletion: The AUV's remaining power is ≤10%; Formation collapse: The AUV deviates from its ideal position. >3m, and lasts for 5 time steps; Timeout: Total time > set time, target point not reached.

[0373] Actor network for each AUV Input is a 103-dimensional local state after multi-scale attention fusion. Output deterministic actions The network structure is a fully connected neural network. :

[0374]

[0375] Where: Input layer dimension: Hidden layer 1 dimension: 256, Hidden layer 2 dimension: 256, Output layer dimension: ; , , This is the weight matrix; , , The bias vector is used; the ReLU activation function introduces nonlinearity. The Tanh activation function limits the range of actions. It matches the physical constraints of the action space.

[0376] The Critic network only operates during the centralized training phase. At this time, the input data... It is directly taken from the ExperienceReplayBuffer. The ExperienceReplayBuffer stores all AUVs at historical moments. The generated joint action tuple and joint state Therefore, Critic When updating parameters, you can directly "see" the actions of other AUVs. There is no need for real-time motion communication between AUVs.

[0377] Only running Actor networks It is based entirely on local state decisions, without the need for Critics, thus achieving decentralized control.

[0378] Critic Network Input global state With joint actions Output action value The network structure is as follows:

[0379]

[0380] in: It is the global state. ( (V) and joint actions ( (Dimensions) concatenation, input dimensions: ( (436 dimensions); Hidden layer 1 dimension: 512, Hidden layer 2 dimension: 256, Output layer dimension: 1; , , This is the weight matrix; , , The bias vector is used; the output layer is a linear activation, directly outputting the action value. .

[0381] To improve training stability, a Target-Actor network is introduced. With Target-Critic Network The parameters are synchronized via soft update:

[0382]

[0383]

[0384] in A soft update rate is used to slowly update the target network parameters, avoiding excessive fluctuations in the target Q-value.

[0385] The loss function of the Critic network is the squared loss of the temporal difference (TD) error. :

[0386]

[0387] in, It is an experience pool (storage) tuple) The target Q value:

[0388]

[0389] It is the joint action output by the Tar1et-Actor network. This is the discount factor.

[0390] The Adam optimizer is used to minimize the loss function, and the learning rate is... Gradient update formula:

[0391]

[0392] The goal of the Actor network is to maximize the action value evaluated by the Critic network. It updates parameters using the policy gradient method. for:

[0393]

[0394] in Depend on It is composed of spliced ​​images of other AUVs. Depend on It is composed of spliced ​​motions from other AUVs.

[0395] The Adam optimizer is used to maximize the objective function, with a learning rate of 100%. Gradient update formula:

[0396]

[0397] Step 3: Mathematical Logic of Distributed Execution

[0398] Each AUV relies solely on its own sensors (sonar, inertial navigation, camera) to acquire local information within a 50m radius and construct a local state. This includes position and attitude perception, acquired through inertial navigation. , , , Errors are calibrated using sonar and seabed beacons (accuracy). , Obstacle detection; sonar scanning within a 50m range to generate a local map. ,calculate (Select the 5 nearest obstacles); Companion perception, obtain the position of other AUVs through communication (communication distance 50m, delay) ),calculate Task information, preset cargo center With the target point ,calculate and .

[0399] The execution phase employs a "decentralized decision-making" model, eliminating the need for global communication. This includes: action generation. Local state Input an Actor network, output actions Formation coordination, through the reward function The AUVs spontaneously adjust their movements to maintain the ideal formation—if Increase Reduce (penalty) and guide the AUV towards Movement; obstacle avoidance decision-making, if or This indicates a risk of collision. This generates a penalty, guiding the AUV to adjust its speed and direction (e.g.) Reduce (Increase) to avoid obstacles.

[0400] Potential conflicts between dynamic obstacles and companions are addressed through the following mechanism: Conflict detection: predicting the future. Internal location: ,like or obstacles satisfy or If a conflict occurs, the AUV is identified as a conflict; conflict resolution: when a conflict occurs, the AUV adjusts its action according to the gradient direction of the reward function—for example, if there is an obstacle on the right, right The partial derivative is negative, which guides the AUV to decrease. (Turn left); Avoid chain blocking: Through a Critic network with global state input, the AUV's action decisions implicitly consider the behavior of its companions—for example, The Critic network input includes If the state is Due to obstacle avoidance, the speed was reduced. of The value will be adjusted accordingly to avoid The rapid pace of action led to new conflicts.

[0401] In an embodiment of the present invention, the intelligent energy management module 150 is used to continuously monitor, predict, and schedule the battery energy state of multiple AUVs. Based on the AUV's battery system and battery management system (BMS), this module combines an energy consumption model of task load, motion state, and communication behavior to dynamically estimate remaining energy. Based on this, it completes charging trigger determination, charging station selection, docking control, and charging process management, providing energy assurance for the system's long-term autonomous operation.

[0402] Figure 9 The schematic diagram illustrates the workflow of an intelligent energy management module according to an embodiment of the present invention.

[0403] like Figure 9 As shown, the workflow of the intelligent energy management module according to an embodiment of the present invention may specifically include:

[0404] Step 1: Each AUV is equipped with a power battery pack and a battery management system (BMS). The BMS collects the battery terminal voltage V, current I, and temperature T in real time, and estimates the battery state of charge (SOC) based on this data. It then performs power monitoring and trend analysis. The power percentage formula can be expressed as: Rate of change in electricity volume Sliding window estimation is used; abnormal consumption is detected based on historical mean and standard deviation.

[0405] Step 2: Charging triggering strategy and task constraints, setting thresholds. , , With target charging level .

[0406] The strategy rules are as follows: When → Emergency return to base / recharging priority; when And the task is not urgent → Schedule charging; When → Schedule charging during task breaks.

[0407] The above determination results are synchronized to the task management module through the communication module to prevent conflicts between task allocation and energy status.

[0408] Step 3: Charging station management and selection. Each charging station has CS attributes: location, charging rate, current load, reliability, availability, etc. The energy management module constructs a comprehensive scoring function for charging stations. :

[0409]

[0410] After normalization, the charging station with the highest score is selected and reservation and queuing scheduling are supported; load balancing and redirection are performed when the station is occupied or in failure.

[0411] Step 4: Charging Execution. Upon entering the near-field range of the charging station, the AUV switches to magnetic induction communication mode and simultaneously initiates the wireless charging docking process. An electromagnetic locking mechanism ensures stable positioning after docking. The charging process theoretically consists of pre-charging, constant current, constant voltage, and float charging stages. The specific stage adaptively selects pre-charging, constant current, constant voltage, or float charging based on the AUV's battery status. Charging completion is determined based on reaching the specified charging level. The charging time limit or the charging current drops to the threshold. Once charging is complete, the BMS sends a "fully charged" signal, the charging station unlocks its mechanical structure, the energy management module notifies the task management module to resume the interrupted transport task, and the communication mode is switched.

[0412] Figure 10 The flowchart illustrating the fault detection and processing module according to an embodiment of the present invention is shown.

[0413] like Figure 10 As shown, the fault detection and processing module according to an embodiment of the present invention may specifically include the following process for fault detection and processing:

[0414] Step 1: Fault Detection (Statistics and Confidence)

[0415] Sliding window estimation is used for each key metric X (e.g., SNR, packet loss rate, positioning error, thruster current / temperature, sensor consistency, battery voltage). , .like And duration If so, it is considered abnormal (typically k=3).

[0416] Fault confidence It can be composed of anomaly scores or the probability output of the learned classifier, and is used to determine whether to take further action.

[0417] Step 2: Fault Classification and Strategy

[0418] Fault severity: 0 = Normal, 1 = Minor, 2 = Moderate, 3 = Severe, 4 = Fatal. The severity level is determined by a weighted average of the number of abnormal indicators, the degree of deviation, and the duration.

[0419] Handling measures: Minor: Try restarting / resetting / switching redundant modules; Moderate: Enable backup system, request relay assistance, or change operating mode; Severe: Mark as unavailable, trigger task reassignment, return to base or perform emergency buoy if necessary; Fatal: Perform emergency buoy or await recovery.

[0420] Step 3: Task Reassignment

[0421] When an AUV malfunctions and needs replacement, task reassignment is triggered. The affected task is marked as unassigned, and the CBAA process described above is re-entered. However, at this time, the task value is dynamically adjusted based on factors such as task urgency, AUV's current location, and battery level. The weight of each task is determined to prioritize fault recovery and system efficiency. If no reasonable alternative exists, the task is marked as delayed or reported to the platform.

[0422] Based on the above-mentioned multi-agent cooperative underwater logistics based on reinforcement learning, this invention proposes a multi-agent cooperative underwater logistics method based on reinforcement learning.

[0423] Figure 11 A flowchart illustrating a multi-agent cooperative underwater logistics method based on reinforcement learning according to an embodiment of the present invention is shown.

[0424] like Figure 11 As shown, the multi-agent cooperative underwater logistics method based on reinforcement learning according to an embodiment of the present invention includes steps S1110 to S1160.

[0425] In step S1110, task information is received, task urgency is assessed, and initial task allocation is completed based on consensus auction algorithm.

[0426] In step S1120, the underwater communication environment is sensed, the current and predicted performance of each communication mode is evaluated, and the communication mode is dynamically switched based on utility evaluation and interruption probability prediction, while maintaining acoustic control link redundancy during the switching.

[0427] In an embodiment of the present invention, dynamically switching communication modes includes: evaluating the total utility of each communication mode, wherein the total utility is a weighted fusion result of the current utility and the predicted utility; and triggering a switch to the optimal candidate mode when the predicted utility of the current mode is lower than a set threshold and there are other modes whose predicted utility is higher than the threshold plus a set margin.

[0428] In step S1130, each vehicle performs local positioning and mapping. When it observes companions or common features, it calculates dynamic weights based on communication quality, battery status, computing power, and historical accuracy, and uses a weighted consensus algorithm to perform cooperative positioning solution.

[0429] In an embodiment of the present invention, the cooperative positioning solution using a weighted consensus algorithm includes: calculating dynamic weights for each vehicle, wherein the dynamic weights are composed of a quality factor reflecting communication reliability, a battery factor reflecting energy state, a calculation factor reflecting processing capability, and a historical factor reflecting past positioning accuracy; within a communication window, each vehicle exchanges its local positioning estimate and corresponding dynamic weights; based on the received estimates and weights from neighboring vehicles, the weighted consensus algorithm is used to perform interpolation updates in the manifold space, and the cooperative positioning result is output after iterative convergence.

[0430] In step S1140, an environment map is constructed and updated, a cooperative formation is set according to the type of goods, and a multi-agent reinforcement learning controller that integrates the environment and task status inputs with a multi-branch attention mechanism is used to output control commands and drive the formation to perform transport.

[0431] In an embodiment of the present invention, the multi-branch attention fusion mechanism includes: calculating local attention weights through local attention branches based on the proximity of obstacles and companions; calculating global attention weights through global attention branches based on path progress and boundary distance; calculating temporal attention weights through temporal attention branches based on the changing trend of state features; and fusing local attention weights, global attention weights, and temporal attention weights according to a preset ratio to perform weighted processing on the input state features.

[0432] In step S1150, the vehicle's battery level is monitored, and charging decisions are triggered based on multi-level battery thresholds and mission urgency levels. Charging stations are selected and scheduled based on a comprehensive charging station score.

[0433] In an embodiment of the present invention, the charging decision triggered based on multi-level power thresholds and task urgency levels includes: when the power level is below the emergency threshold, interrupting the task and performing emergency charging; when the power level is below the low power threshold and the current task is a non-urgent task, reserving a charging station; when the power level is below the medium threshold, planning charging during task intervals; wherein the emergency threshold is lower than the low power threshold, and the low power threshold is lower than the medium threshold.

[0434] In step S1160, key indicators are monitored through a sliding window to detect anomalies and assess the fault level, and corresponding handling measures are implemented. If necessary, task reassignment is triggered.

[0435] Those skilled in the art will understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention can be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

Claims

1. A multi-agent cooperative underwater logistics system based on reinforcement learning, characterized in that, include: The module includes a task management module, a multimodal dynamic communication module, a dynamic weighted collaborative localization module, a reinforcement learning path planning and control module, an intelligent energy management module, and a fault detection and handling module. The multimodal dynamic communication module supports three communication methods: acoustic, optical, and magnetic induction, and can adaptively switch communication modes based on environmental perception information and task status prediction. The dynamic weighted collaborative positioning module, based on the local positioning of each underwater vehicle, dynamically allocates weights by comprehensively considering the communication quality, battery status, computing power and historical positioning accuracy of each vehicle, and uses a distributed consensus algorithm to achieve collaborative positioning. The reinforcement learning path planning and control module adopts a multi-agent reinforcement learning algorithm with a multi-branch attention mechanism, and generates cooperative control instructions by combining environmental map, obstacle information, companion status and task objective. The intelligent energy management module triggers charging decisions in a tiered manner based on the vehicle's battery status and the urgency of the mission, and schedules charging based on the comprehensive score of the charging station. The fault detection and handling module monitors key operating indicators through a sliding window, enabling fault level assessment and adaptive task reassignment.

2. The system according to claim 1, characterized in that, The task management module uses a consensus-based auction algorithm for task allocation. The value of a task is calculated by weighting factors such as task urgency, capability matching, execution efficiency, and estimated cost. The task urgency is calculated by combining the normalized result of the original urgency with the time urgency factor, and a penalty coefficient is applied to overdue tasks.

3. The system according to claim 1, characterized in that, The multimodal dynamic communication module includes a performance evaluation unit and a switching decision unit; The performance evaluation unit is used to evaluate the channel quality of acoustic, optical, and magnetic induction communications in current and predicted environments. The switching decision unit triggers a communication mode switch when the predictive utility of the current communication mode is below a threshold and a better candidate mode exists.

4. The system according to claim 1, characterized in that, The dynamic weighted cooperative positioning module assigns dynamic weights to each autonomous underwater vehicle, which are jointly determined by communication quality assessment factors, battery status assessment factors, computing power assessment factors, and historical positioning accuracy assessment factors. The communication quality assessment factor is used to reflect the stability and latency of the communication link; The battery status assessment factor reflects the current battery level of the aircraft; The computing power evaluation factor reflects the performance and load status of the vehicle's processor. The historical positioning accuracy assessment factor reflects the reliability of the vehicle's past positioning estimates.

5. The system according to claim 1, characterized in that, The multi-branch attention mechanism includes: Local attention branch is used to calculate obstacle threat level and companion threat level and normalize them into weights; The global attention branch is used to calculate path relevance and boundary distance relevance and normalize them into weights. The temporal attention branch is used to calculate the temporal rate of change of each state feature group and normalize it into weights. The weights of the local attention branch, the global attention branch, and the temporal attention branch are fused according to a preset ratio to form the final attention weight, which is used for weighting state features.

6. A multi-agent cooperative underwater logistics method based on reinforcement learning, characterized in that, include: Receive task information, assess task urgency, and complete initial task allocation based on consensus auction algorithm; The system senses the underwater communication environment, assesses the current and predicted performance of each communication mode, and dynamically switches communication modes based on utility assessment and interruption probability prediction, while maintaining acoustic control link redundancy during the switching process. Each vehicle performs local positioning and mapping. When it observes companions or common features, it calculates dynamic weights based on communication quality, battery status, computing power and historical accuracy, and uses a weighted consensus algorithm to perform cooperative positioning solution. Build and update the environment map, set the cooperative formation according to the type of goods, and use a multi-agent reinforcement learning controller that integrates the environment and task status inputs with a multi-branch attention mechanism to output control commands and drive the formation to perform transport. Monitor the vehicle's battery level, trigger charging decisions based on multi-level battery thresholds and mission urgency, and select and dispatch charging stations based on a comprehensive rating of charging stations. By monitoring key indicators through a sliding window, anomalies are detected and the fault level is assessed. Corresponding handling measures are implemented, and task reassignment is triggered when necessary.

7. The method according to claim 6, characterized in that, The dynamically switching communication modes include: The total utility of each communication mode is evaluated, which is a weighted fusion result of the current utility and the predicted utility; When the predictive utility of the current mode is lower than a set threshold, and there are other modes whose predictive utility is higher than the threshold plus a set margin, the switch to the optimal candidate mode is triggered.

8. The method according to claim 6, characterized in that, The use of a weighted consensus algorithm for collaborative localization includes: Dynamic weights are calculated for each vehicle. These dynamic weights are composed of a quality factor reflecting communication reliability, a battery factor reflecting energy state, a calculation factor reflecting processing capability, and a historical factor reflecting past positioning accuracy. Within the communication window, each vehicle exchanges its local positioning estimate and corresponding dynamic weights; Based on the received neighbor vehicle estimates and weights, a weighted consensus algorithm is used to perform interpolation updates in the manifold space, and the cooperative localization results are output after iterative convergence.

9. The method according to claim 6, characterized in that, The fusion multi-branch attention mechanism includes: Based on the proximity of obstacles to companions, local attention weights are calculated through local attention branches; Global attention weights are calculated using a global attention branch based on path progress and boundary distance. Based on the changing trends of state features, the temporal attention weights are calculated through a temporal attention branch. The local attention weight, the global attention weight, and the temporal attention weight are fused according to a preset ratio to perform weighted processing on the input state features.

10. The method according to claim 6, characterized in that, The method of triggering charging decisions based on multi-level power thresholds and task urgency includes: When the battery level falls below the emergency threshold, the task is interrupted and emergency charging is performed. When the battery level is below the low battery threshold and the current task is not urgent, reserve a charging station. When the battery level is below a medium threshold, schedule charging during task breaks; Wherein, the emergency threshold is lower than the low battery threshold, and the low battery threshold is lower than the medium threshold.

Images