Multi-agent offshore uav-assisted communication method based on timing prediction
By constructing a maritime vessel time-series prediction model and using FCM and MATD3 algorithms to optimize UAV resources and trajectories, the problem of low communication coverage efficiency of maritime UAVs was solved, and energy optimization of the efficient UAV communication system was achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANTONG UNIV
- Filing Date
- 2026-03-26
- Publication Date
- 2026-07-21
AI Technical Summary
Traditional communication methods are insufficient to meet the reliable and continuous wireless communication needs of modern maritime vessels. Shore-based base stations have limited coverage, satellite communication is costly, and drone resource scheduling and trajectory optimization are difficult.
A multi-agent maritime UAV-assisted communication method based on time-series prediction is proposed. This method constructs a maritime vessel time-series prediction model that combines spatiotemporal information, and uses the FCM dynamic partitioning algorithm and the MATD3 algorithm to optimize UAV resource allocation and flight trajectory, thereby achieving joint optimization of UAV communication coverage.
It has achieved a reasonable allocation of UAV resources and improved communication coverage efficiency, optimized the energy efficiency of UAV communication systems, and solved the problems of limited UAV resources and trajectory optimization.
Smart Images
Figure CN122437585A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of wireless communication technology, and more specifically, relates to a multi-agent maritime unmanned aerial vehicle (UAV) assisted communication method based on time-series prediction. Background Technology
[0002] With the increasing demand for reliable and continuous wireless communication from ships at sea, traditional communication methods face numerous challenges. While shore-based base stations can provide stable communication, their coverage is limited by geographical factors, making it difficult to achieve comprehensive coverage of vast sea areas. Satellite communication, although possessing wide-area coverage capabilities, is prohibitively expensive for ordinary users. Therefore, neither of these methods can fully meet the communication needs of modern ships at sea.
[0003] In contrast, unmanned aerial vehicles (UAVs) offer advantages such as flexible deployment and rapid coverage, providing a novel solution for maritime communication. Despite the broad application prospects of UAV-based maritime communication, several key challenges remain: 1. Limited UAV resources for maritime communication necessitate efficient resource scheduling strategies; 2. UAV trajectory optimization requires careful consideration of user spatial distribution characteristics, as maritime users are widely and sparsely distributed, unlike land-based communication scenarios; 3. The joint optimization problem for UAV communication coverage is inherently non-convex, making it difficult to solve. Summary of the Invention
[0004] To address the aforementioned issues, this invention proposes a multi-agent maritime UAV-assisted communication method based on temporal prediction. It uses a maritime vessel temporal prediction model incorporating spatiotemporal information to predict the number of vessels in a target sea area, enabling rational allocation of UAV resources. Considering the widespread and sparse distribution of vessels at sea, a dynamic partitioning method using FCM (Flexible Coding Mechanism) is introduced to construct virtual maritime communication cells, thereby improving UAV communication coverage efficiency. Furthermore, the MATD3 algorithm is used to jointly optimize the flight trajectories and propulsion power of multiple UAVs, maximizing the energy efficiency of the UAV-assisted communication system.
[0005] To address at least one of the aforementioned technical problems, according to one aspect of the present invention, a multi-agent maritime unmanned aerial vehicle (UAV) assisted communication method based on time-series prediction is provided, comprising the following steps:
[0006] S1: Construct a spatial-temporal information time series forecasting model (STM) for maritime vessels. The maritime vessel time series dataset is defined as follows: Where T is the number of time slices and N is the number of time series. STM uses a time feature matrix. , and spatial feature matrix Extracting time series data The spatiotemporal information characteristics are used to predict the future number of ships.
[0007] S2: The FCM dynamic partitioning algorithm is used to cluster sparsely distributed maritime vessels, and an outlier detection mechanism is used to exclude isolated vessels. Then, the number of virtual maritime cells is dynamically adjusted using the fuzzy partition coefficient (FPC), and corresponding UAVs are assigned.
[0008] S3: Construct a discrete-time motion model for the UAV, and based on this, construct a propulsion energy consumption model for the UAV. Communication between the UAV and the ship uses a probabilistic line-of-sight channel model.
[0009] S4: Construct a multi-UAV communication energy efficiency optimization problem P1, with the total energy efficiency (EE) as the objective function, and the UAV's propulsion power, trajectory, and minimum communication rate between the UAV and the ship as constraints.
[0010] S5: Use the MATD3 multi-UAV communication optimization algorithm to optimize problem P1, obtain the optimal UAV communication coverage flight trajectory and maximize the EE.
[0011] The specific steps of step S1 are as follows:
[0012] Step S1.1: Construct a time-series dataset of ships at sea Where T is the number of time slices and N is the number of time series. Let be the number of ships at the t-th time step of the i-th time series. The goal of time series prediction is to use P historical time slices of data. Predict the data for the next F time slices Specifically, it involves using historical data. Data for predicting the future .
[0013] Step S1.2: Using the time feature matrix , and spatial feature matrix Extracting time series data The spatiotemporal information, among which This indicates the time information for a 24-hour day. This represents the time information for the 7 days of the week, where D is the hidden dimension of the neural network. The temporal data features that integrate spatiotemporal information are defined as:
[0014] (1)
[0015] in, Historical data after incorporating spatiotemporal information Feature representation, It is a fully connected neural network. For historical data eigenmatrix These are the learnable parameters of the neural network.
[0016] Step S1.3: Use a Temporal Convolutional Network (TCN) to extract long-term dependency features from the time-series data of ships at sea. The formula is defined as follows:
[0017] (2)
[0018] in, This represents a dilated convolution operation, where f is the convolution kernel, k is the kernel size, x is the input sequence, and d is the dilation rate.
[0019] Meanwhile, TCN uses residual connections to improve the stability and convergence speed of deep networks, as defined in the following formula:
[0020] (3)
[0021] in, For activation function, For historical data Its long-term dependency characteristics.
[0022] Finally, after acquiring spatiotemporal information and long-term dependency features, a regression layer is used to predict the future number of ships, and the formula is defined as:
[0023] (4)
[0024] in, This indicates the predicted number of ships.
[0025] The specific steps of step S2 are as follows:
[0026] Step S2.1: Use the FCM algorithm to identify ships at sea. Divided into virtual communities at sea ,in The number of ships predicted for a given time slice t is... , For the number of drones, The partition satisfies the constraints , and The objective function of FCM is defined as follows:
[0027] (5)
[0028] (6)
[0029] (7)
[0030] in, Let i be the position of the i-th ship. Let m be the cluster center location of the j-th virtual cell, and m be the fuzzy coefficient set to 2. For ships and The degree of membership between them. distance The closer, The larger the value, the greater the distance; conversely, the smaller the value, the greater the distance. The smaller the value.
[0031] Step S2.2: Use the FCM outlier detection mechanism to exclude isolated ships from the virtual cell, improve the cluster structure of the virtual cell, and reduce the coverage path of the UAV. The formula is defined as follows:
[0032] (8)
[0033] in, This represents the average distance between a ship and the cluster center of its respective subgroup. Let be the outlier coefficient, a positive real number representing the tolerance for outliers. The higher the value, the higher the tolerance. If the ship If the above conditions are met, the vessel will remain in the community. If it is in the middle, then remove it.
[0034] Step S2.3: Use the Fuzzy Partition Coefficient (FPC) to quantify the clarity of the partitioning results and measure the rationality of the virtual cell partitioning. The FPC formula is defined as follows:
[0035] (9)
[0036] The FPC value ranges from [1 / C, 1], where C is the number of virtual cells. A higher FPC value (closer to 1) indicates a clearer partitioning effect and a better cluster structure. The number of virtual cells corresponding to the highest FPC value is selected as the number of drones to be dispatched.
[0037] The specific steps of step S4 are as follows:
[0038] Step S4.1: Construct a multi-UAV communication energy efficiency optimization problem. The objective of this optimization problem is to optimize the energy efficiency of UAV communication within a finite total time. Optimize drone ensemble The flight trajectory is transmitted to the collection of ships at sea to the maximum extent. The ratio of the total number of information bits to the total propulsion energy consumption of the UAV. Energy efficiency (EE) is defined as the ratio of the total number of information bits to the total propulsion energy consumption of the UAV during time. The ratio of the total number of bits transmitted internally to its total power consumption is defined by the formula:
[0039] (19)
[0040] Constraints C1 and C2 restrict the UAV to remain within the target sea area throughout the mission. Constraint C3 restricts the UAV's speed within each time slice, ensuring that the speed remains at the minimum permissible speed. and maximum speed Constraint C4 limits the acceleration of the drone within each time slice, ensuring that the acceleration is within a specified range. Constraint C5 ensures that the heading angle of the drone is within... Between. Constraint C6 is the threshold requirement for the minimum transmission rate between the drone and the ship.
[0041] The specific steps of step S5 are as follows:
[0042] Step S5.1: Optimize problem P1 using the MATD3 multi-UAV communication optimization algorithm to obtain the optimal UAV communication coverage flight trajectory and maximize the EE. Problem P1 can be viewed as a multi-agent cooperative scenario because it involves multiple UAVs collaboratively covering a target sea area to meet the communication needs of ships. Therefore, the MATD3 multi-agent reinforcement learning method is used to optimize problem P1. This process is modeled as a Markov decision process, consisting of triples... Define a space where S is the state space of the drone, A is the action space of the drone, and R is the space associated with each state-action pair. The associated reward function. Specifically, at each time step λ, the agent, according to policy λ, rewards the agent based on the observed state. Select Action Interact with the environment and then receive a reward. And the new status And use the Q-value function to evaluate the action to be performed in state s. Expected return .
[0043] Specifically, each drone is considered an intelligent agent, for The status of the drone The definition is as follows:
[0044] (20)
[0045] Drone Actions The definition is as follows:
[0046] (twenty one)
[0047] drone reward function The definition is as follows:
[0048] (twenty two)
[0049] in, It is a flight reward factor. , . It is an indicator function that indicates when the distance between the drone and the ship is less than a set threshold. ;otherwise .
[0050] Step S5.2: In MATD3, each agent uses two main evaluation networks (Critic Networks) to estimate the Q-value, and uses the smaller of the two estimates as the target for updating the evaluation network. Let the parameters of these two main evaluation networks be... and Correspondingly, each of them has a target network with parameters as follows: and In addition, each agent also includes two policy networks: a main policy network with parameters of... A target policy network Its parameter MATD3 uses the smaller Q-value of the two target evaluation networks as the Q-value estimate for the next time step, and its formula is defined as:
[0051] (twenty three)
[0052] in, For strategy In state The actions generated below, To add Gaussian noise, `clip` is a clipping operation that limits the action value to a certain range. and between.
[0053] Main evaluation network and The loss function is defined as follows:
[0054] (twenty four)
[0055] Target evaluation network and Through the main evaluation network and The formula for performing a soft update is defined as follows:
[0056] (25)
[0057] in, It is the update factor of the target network.
[0058] Main policy network The objective function is defined as follows:
[0059] (26)
[0060] Main policy network The update is achieved by updating the strategy. gradient propagation to The formula for implementation is defined as follows:
[0061] (27)
[0062] Target Policy Network Through the main policy network The formula for performing a soft update is defined as follows:
[0063] (28)
[0064] By optimizing the action network and policy network of the agent, the optimal drone path planning is generated, thereby maximizing the EE of the multi-drone communication system.
[0065] According to another aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the multi-agent maritime unmanned aerial vehicle-assisted communication method based on time-series prediction of the present invention.
[0066] According to another aspect of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the multi-agent maritime unmanned aerial vehicle-assisted communication method based on time-series prediction of the present invention.
[0067] Compared with existing technologies, the beneficial effects of the above-described method of the present invention are as follows:
[0068] This invention constructs a maritime vessel time-series prediction model to predict the changing trend of the number of vessels in a target sea area, thereby enabling advance planning and dynamic adjustment of UAV deployment strategies and optimizing the allocation of communication resources.
[0069] This invention uses the FCM dynamic partitioning method to divide sparsely distributed maritime vessels into multiple virtual cells and assigns a UAV as a communication relay to each virtual cell, thereby improving the efficiency of UAV-assisted communication.
[0070] This invention addresses the problem of UAV collaboration and resource optimization in dynamic environments through a multi-agent, dual-delay, deep deterministic policy gradient algorithm. Each UAV is treated as an agent, enabling it to autonomously optimize its flight trajectory and communication resource allocation strategy, thereby improving the overall communication energy efficiency of the system. Attached Figure Description
[0071] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of the present invention and are not intended to limit the present invention.
[0072] Figure 1 This is a schematic diagram of the system architecture of a preferred embodiment of the present invention;
[0073] Figure 2 This is a diagram of a preferred embodiment of the maritime vessel timing prediction model of the present invention;
[0074] Figure 3 This is a flowchart of the FCM dynamic programming algorithm according to a preferred embodiment of the present invention;
[0075] Figure 4 The flowchart of the MATD3 multi-UAV communication optimization algorithm is a preferred embodiment of the present invention. Detailed Implementation
[0076] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention.
[0077] Unless otherwise defined, the technical or scientific terms used herein shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.
[0078] Example 1:
[0079] like Figure 1-4 As shown, this invention provides a multi-agent maritime unmanned aerial vehicle (UAV) assisted communication method based on time-series prediction, comprising the following steps:
[0080] Step 1: Construct a time-series dataset of ships at sea Where T is the number of time slices and N is the number of time series. Let be the number of ships at the t-th time step of the i-th time series. The goal of time series prediction is to use P historical time slices of data. Predict the data for the next F time slices Specifically, it involves using historical data. Data for predicting the future Using the time feature matrix , and spatial feature matrix Extracting time series data The spatiotemporal information, among which This indicates the time information for a 24-hour day. This represents the time information for the 7 days of the week, where D is the hidden dimension of the neural network. The temporal data features that integrate spatiotemporal information are defined as:
[0081] (1)
[0082] in, Historical data after incorporating spatiotemporal information Feature representation, It is a fully connected neural network. For historical data eigenmatrix These are the learnable parameters of the neural network.
[0083] Temporal convolutional networks (TCNs) are used to extract long-term dependency features from time-series data of ships at sea. The formula is defined as follows:
[0084] (2)
[0085] in, This represents a dilated convolution operation, where f is the convolution kernel, k is the kernel size, x is the input sequence, and d is the dilation rate.
[0086] Meanwhile, TCN uses residual connections to improve the stability and convergence speed of deep networks, as defined in the following formula:
[0087] (3)
[0088] in, For activation function, For historical data Its long-term dependency characteristics.
[0089] Finally, after acquiring spatiotemporal information and long-term dependency features, a regression layer is used to predict the future number of ships, and the formula is defined as:
[0090] (4)
[0091] in, This indicates the predicted number of ships.
[0092] The predicted sea area for maritime vessels is located between 18.305° and 18.350° north latitude and 64.830° and 64.783° west longitude, within a 5km radius. A 5km area. Time range: 09:00:00 to 20:00:00 on October 24, 2019. Ship changes in the next 12 hours are predicted using the past 72 hours. The predicted number of ships is: [13, 21, 32, 41, 54, 62, 72, 78, 83, 88, 92, 91], while the actual number of ships is: [13, 20, 28, 41, 51, 63, 72, 78, 82, 84, 83, 82].
[0093] Step 2: Use the FCM algorithm to classify ships at sea. Divided into virtual communities at sea ,in The number of ships predicted for a given time slice t is... , For the number of drones, The partition satisfies the constraints , and The objective function of FCM is defined as follows:
[0094] (5)
[0095] (6)
[0096] (7)
[0097] in, Let i be the position of the i-th ship. Let m be the cluster center location of the j-th virtual cell, and m be the fuzzy coefficient set to 2. For ships and The degree of membership between them. distance The closer, The larger the value, the greater the distance; conversely, the smaller the value, the greater the distance. The smaller the value.
[0098] The FCM outlier detection mechanism is used to exclude isolated ships from the virtual cell, improving the cluster structure of the virtual cell and reducing the coverage path of drones. The formula is defined as follows:
[0099] (8)
[0100] in, This represents the average distance between a ship and the cluster center of its respective subgroup. Let be the outlier coefficient, a positive real number representing the tolerance for outliers. The higher the value, the higher the tolerance. If the ship If the above conditions are met, the vessel will remain in the community. If it is in the middle, then remove it.
[0101] The Fuzzy Partition Coefficient (FPC) is used to quantify the clarity of the partitioning results and measure the rationality of the virtual cell partitioning. The FPC formula is defined as follows:
[0102] (9)
[0103] The FPC value ranges from [1 / C, 1], where C is the number of virtual cells. A higher FPC value (closer to 1) indicates a clearer partitioning effect and a better cluster structure. The number of virtual cells corresponding to the highest FPC value is selected as the number of drones to be dispatched.
[0104] Table 1. FPC coefficients corresponding to the number of drones
[0105] FPC 0.740 0.766 0.830 0.778
[0106] When the predicted number of ships in the target sea area is 21, the FPC value is the highest when the number of drones is 4. Therefore, 4 drones are dispatched in the target sea area.
[0107] Step 3: Construct a discrete-time motion model of the UAVs, where the set of UAVs is... The Cartesian coordinates of the drone u at time t are: Where H is the flight altitude of the UAV. The coordinates of the ship ms at time t are... The total mission completion time for the drone is... ,in For the number of time slices, It is the duration of the t-th time slice. The discrete equations of motion for the UAV u from time t to t+1 are defined as follows:
[0108] (10)
[0109] in, , and Let represent the velocity, acceleration, and elevation angle of the UAV u at time t, respectively.
[0110] The propulsion energy consumption model for UAVs is constructed, and its formula is defined as follows:
[0111] (11)
[0112] in, Let u be the flight speed of the drone at time t. The tip velocity of the rotor blades. The average induced velocity of the drone while it is hovering. and These represent the blade profile power and induced power during hovering, respectively. For the fuselage drag ratio, air density, For rotor realism, It is the rotor disk area.
[0113] A maritime channel model for UAVs and ships is constructed, and its formula is defined as follows:
[0114] (12)
[0115] (13)
[0116] Where a and b are environmental coefficients. Let represent the elevation angle between the u-th UAV and the ms-th ship at time t.
[0117] The path loss between drones and ships is defined as follows:
[0118] (14)
[0119] Line-of-sight path loss Non-line-of-sight path loss The definition is as follows:
[0120] (15)
[0121] (16)
[0122] in, For wavelength, and Link factor, and This is for additional path loss.
[0123] The signal-to-noise ratio (SNR) between drones and ships is defined as follows:
[0124] (17)
[0125] in, This represents the transmission power from the u-th UAV to the ms-th ship at time t. It is additive white Gaussian noise. It is the channel gain between drones and ships.
[0126] The transmission rate of the u-th UAV and the ms-th ship at time t is defined as follows:
[0127] (18)
[0128] Environmental parameters a and b were set to 9.61 and 0.16, respectively, and the wavelength was... Set to 0.15, link factor and Set to 1 and 20, additional path loss and Set to 2 and 2, drone flight speed The rotor blade tip speed is set to 0 to 50 m / s. The average induced velocity of the drone while hovering is set at 120 m / s. The blade profile power and induced power are set at 4.03 m / s during hovering. and Set at 99.66W and 120.16W, the fuselage drag ratio Set to 0.6, air density The rotor solidity is set at 1.225 kg / m³. The rotor disk area of the drone is set to 0.05. Set to 0.503m². Drone flight altitude H is set to 100m. Bandwidth B is set to 1MHz. Time slice duration. Set to 0.5.
[0129] Step 4: Construct a multi-UAV communication energy efficiency optimization problem. The goal of this optimization problem is to optimize the energy efficiency of UAV communication within a finite total time. Optimize drone ensemble The flight trajectory is transmitted to the collection of ships at sea to the maximum extent. The ratio of the total number of information bits to the total propulsion energy consumption of the UAV. Energy efficiency (EE) is defined as the ratio of the total number of information bits to the total propulsion energy consumption of the UAV during time. The ratio of the total number of bits transmitted internally to its total power consumption is defined by the formula:
[0130] (19)
[0131] Constraints C1 and C2 restrict the UAV to remain within the target sea area throughout the mission. Constraint C3 restricts the UAV's speed within each time slice, ensuring that the speed remains at the minimum permissible speed. and maximum speed Constraint C4 limits the acceleration of the drone within each time slice, ensuring that the acceleration is within a specified range. Constraint C5 ensures that the heading angle of the drone is within... Between. Constraint C6 is the threshold requirement for the minimum transmission rate between the drone and the ship.
[0132] Constraint C1 is set as m, constraint C2 is set as m, constraint C3 is set as Constraint C4 is set as Constraint C5 is set as Constraint C6 is set as .
[0133] Step S5: Optimize problem P1 using the MATD3 multi-UAV communication optimization algorithm to obtain the optimal UAV communication coverage flight trajectory and maximize the EE. Problem P1 can be viewed as a multi-agent cooperative scenario because it involves multiple UAVs collaboratively covering a target sea area to meet the communication needs of ships. Therefore, the MATD3 multi-agent reinforcement learning method is used to optimize problem P1. This process is modeled as a Markov decision process, consisting of triples... Define a space where S is the state space of the drone, A is the action space of the drone, and R is the space associated with each state-action pair. The associated reward function. Specifically, at each time step λ, the agent, according to policy λ, rewards the agent based on the observed state. Select Action Interact with the environment and then receive a reward. And the new status And use the Q-value function to evaluate the action to be performed in state s. Expected return .
[0134] Specifically, each drone is considered an intelligent agent, for The status of the drone The definition is as follows:
[0135] (20)
[0136] Drone Actions The definition is as follows:
[0137] (twenty one)
[0138] drone reward function The definition is as follows:
[0139] (twenty two)
[0140] in, It is a flight reward factor. , . It is an indicator function that indicates when the distance between the drone and the ship is less than a set threshold. ;otherwise .
[0141] In MATD3, each agent uses two main evaluation networks (Critic Networks) to estimate the Q-value, and uses the smaller of the two estimates as the target for updating the evaluation network. Let the parameters of these two main evaluation networks be... and Correspondingly, each of them has a target network with parameters as follows: and In addition, each agent also includes two policy networks: a main policy network with parameters of... A target policy network Its parameter MATD3 uses the smaller Q-value of the two target evaluation networks as the Q-value estimate for the next time step, and its formula is defined as:
[0142] (twenty three)
[0143] in, is a discount factor used to control how much the agent prioritizes short-term rewards. For strategy In state The actions generated below, To add Gaussian noise, `clip` is a clipping operation that limits the action value to a certain range. and between.
[0144] Main evaluation network and The loss function is defined as follows:
[0145] (twenty four)
[0146] in, This serves as an experience replay buffer, used to store experience tuples generated during agent interactions. .
[0147] Target evaluation network and Through the main evaluation network and The formula for performing a soft update is defined as follows:
[0148] (25)
[0149] in, It is the update factor of the target network.
[0150] Main policy network The objective function is defined as follows:
[0151] (26)
[0152] Main policy network The update is achieved by updating the strategy. gradient propagation to The formula for implementation is defined as follows:
[0153] (27)
[0154] Target Policy Network Through the main policy network The formula for performing a soft update is defined as follows:
[0155] (28)
[0156] By optimizing the action network and policy network of the agent, the optimal drone path planning is generated, thereby maximizing the EE of the multi-drone communication system.
[0157] The MATD3 reinforcement learning algorithm is configured as follows: Experience replay buffer pool The size is set to 1e6, the batch size for training is set to 512, and the discount factor is set to... The target network update factor is set to 0.985. The learning rates were set to 0.01, and the evaluation network and policy network were set to 0.0001 and 0.005, respectively. The network structures were 128-128 and 256-256, respectively.
[0158] After training, it can achieve stable convergence and maximize EE. Meanwhile, MATD3 (No OutlierDetection) is a version of the MATD3 algorithm without an outlier detection mechanism, and its EE is lower than that of the MATD3 algorithm, which proves the effectiveness of our proposed outlier detection mechanism.
[0159] The drone successfully covered the virtual maritime area and provided communication assistance. When the drone was far from the target virtual maritime area, it adopted a straight-line flight strategy to reduce unnecessary propulsion energy consumption. During the communication coverage mission over the virtual maritime area, the drone used a circling flight pattern to communicate with every ship as fairly as possible. At the same time, the drone was able to autonomously avoid isolated vessels and achieve higher energy efficiency. Its flight trajectory was smooth and stable, effectively avoiding sharp turns and unnecessary detours.
[0160] In summary, this invention uses a maritime vessel time-series prediction model incorporating spatiotemporal information to predict the number of vessels in a target sea area, thereby achieving a rational allocation of UAV resources. Addressing the widespread and sparse distribution of vessels at sea, a dynamic partitioning method based on FCM is introduced to construct virtual maritime communication cells, thus improving UAV communication coverage efficiency. Furthermore, the MATD3 algorithm is used to jointly optimize the flight trajectories and propulsion power of multiple UAVs, maximizing the energy efficiency of the UAV-assisted communication system.
[0161] Example 2:
[0162] The computer-readable storage medium of this embodiment stores a computer program that, when executed by a processor, implements the steps in the multi-agent maritime unmanned aerial vehicle-assisted communication method based on time-series prediction in Embodiment 1.
[0163] The computer-readable storage medium in this embodiment can be an internal storage unit of the terminal, such as the terminal's hard disk or memory; the computer-readable storage medium in this embodiment can also be an external storage device of the terminal, such as a plug-in hard disk, smart memory card, secure digital card, flash memory card, etc. equipped on the terminal; furthermore, the computer-readable storage medium can include both the terminal's internal storage unit and external storage devices.
[0164] The computer-readable storage medium of this embodiment is used to store computer programs and other programs and data required by the terminal. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.
[0165] Example 3:
[0166] The computer device of this embodiment includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the multi-agent maritime unmanned aerial vehicle assisted communication method based on time-series prediction of embodiment 1.
[0167] In this embodiment, the processor can be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The memory can include read-only memory and random access memory, and provides instructions and data to the processor. A portion of the memory can also include non-volatile random access memory. For example, the memory can also store device type information.
[0168] Those skilled in the art will understand that the content disclosed in the embodiments can be provided as a method, system, or computer program product. Therefore, this solution can take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this solution can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage) containing computer-usable program code.
[0169] This solution is described with reference to flowchart illustrations and / or block diagrams of methods and computer program products according to embodiments of this solution. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0170] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0171] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0172] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.
[0173] The examples described herein are merely preferred embodiments of the invention and are not intended to limit the concept and scope of the invention. Any modifications and improvements made by those skilled in the art to the technical solutions of the invention without departing from the design concept of the invention should fall within the protection scope of the invention.
[0174] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the specific embodiments described above. The specific embodiments and descriptions in the specification are merely for further illustrating the principles of the invention. Various changes and modifications can be made to the present invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the claims and their equivalents.
Claims
1. A multi-agent maritime unmanned aerial vehicle (UAV) assisted communication method based on time-series prediction, characterized in that, Includes the following steps: Construct a time-series prediction model for maritime vessels that incorporates spatiotemporal information; The FCM dynamic partitioning algorithm is used to cluster sparsely distributed maritime vessels, and an outlier detection mechanism is used to exclude isolated vessels. Then, the number of virtual maritime cells is dynamically adjusted using fuzzy partitioning coefficients and corresponding drones are assigned. A discrete-time motion model for the UAV is constructed, as well as a propulsion energy consumption model for the UAV; a probabilistic line-of-sight channel model is used for communication between the UAV and the ship. Construct a multi-UAV communication energy efficiency optimization problem P1, with the total energy efficiency as the objective function, and the UAV's propulsion power, trajectory, and minimum communication rate between the UAV and the ship as constraints; The MATD3 multi-UAV communication optimization algorithm is used to optimize problem P1, obtaining the optimal UAV communication coverage flight trajectory and the maximized energy efficiency.
2. The method as described in claim 1, characterized in that, Constructing a marine vessel time-series prediction model that combines spatiotemporal information includes: Constructing a time-series dataset of ships at sea Where T is the number of time slices and N is the number of time series. Let be the number of ships at the t-th time step of the i-th time series; the goal of time series prediction is to use P historical time slices of data. Predict the data for the next F time slices Specifically, it involves using historical data. Data for predicting the future ; Using the time feature matrix , and spatial feature matrix Extracting time series data The spatiotemporal information, among which This indicates the time information for a 24-hour day. This represents the time information for the 7 days of the week, where D is the hidden dimension of the neural network; the temporal data features that integrate spatiotemporal information are defined as: (1) in, Historical data after incorporating spatiotemporal information Feature representation, It is a fully connected neural network. For historical data eigenmatrix These are the learnable parameters of the neural network; Temporal convolutional networks are used to extract long-term dependency features from time-series data of maritime vessels. The formula is defined as follows: (2) in, This represents a dilated convolution operation, where f is the convolution kernel, k is the kernel size, x is the input sequence, and d is the dilation rate. Meanwhile, temporal convolutional networks use residual connections to improve the stability and convergence speed of deep networks, as defined in the following formula: (3) in, For activation function, For historical data Its long-term dependency characteristics; After acquiring spatiotemporal information and long-term dependency features, a regression layer is used to predict the future number of ships, and the formula is defined as: (4) in, This indicates the predicted number of ships.
3. The method as described in claim 2, characterized in that, Using the FCM algorithm to classify maritime vessels Divided into virtual communities at sea ,in The number of ships predicted for a given time slice t is... , For the number of drones, The partition satisfies the constraints , and ; The objective function of FCM is defined as follows: (5) (6) (7) in, Let i be the position of the i-th ship. Let m be the cluster center location of the j-th virtual cell, and m be the fuzzy coefficient set to 2. For ships and Membership degree between them; distance The closer, The larger the value, the greater the distance; conversely, the smaller the value, the greater the distance. The smaller the value; The FCM outlier detection mechanism is used to exclude isolated ships from the virtual cell, improve the cluster structure of the virtual cell, and reduce the coverage path of UAVs. The definition is as follows: (8) in, This represents the average distance between a ship and the cluster center of its respective subgroup. Let be the outlier coefficient, a positive real number representing the tolerance for outliers. The higher the value, the higher the tolerance. If the ship If the conditions are met, the vessel will remain in the community. If it is in the middle, then remove it; The fuzzy partitioning coefficient is used to quantify the clarity of the partitioning results and measure the rationality of the virtual cell partitioning; the formula for the fuzzy partitioning coefficient is defined as follows: (9) The value of FPC ranges from [1 / C, 1], where C is the number of virtual cells. The closer the FPC value is to 1, the clearer the division effect and the better the cluster structure. The number of virtual cells corresponding to the largest FPC value is selected as the number of drones to be dispatched.
4. The method as described in claim 3, characterized in that, The goal of constructing a multi-UAV communication energy efficiency optimization problem is to achieve the desired energy efficiency within a finite total time. Optimize drone ensemble The flight trajectory is transmitted to the collection of ships at sea to the maximum extent. The ratio of the total number of information bits to the total propulsion energy consumption of the UAV; Energy efficiency is defined as the efficiency of a drone in terms of time. The ratio of the total number of bits transmitted to the total energy consumption is defined by the formula: (19) Among them, constraints C1 and C2 restrict the UAV to remain within the target sea area throughout the mission; constraint C3 restricts the speed of the UAV in each time slice, ensuring that the speed is maintained at the minimum allowed speed. and maximum speed Between; Constraint C4 limits the acceleration of the UAV within each time slice, ensuring that the acceleration is within a specified range; Constraint C5 ensures that the heading angle of the UAV is within... Between; constraint C6 is the threshold requirement for the minimum transmission rate between the UAV and the ship.
5. The method as described in claim 4, characterized in that, Problem P1 is optimized using the MATD3 multi-UAV communication optimization algorithm to obtain the optimal UAV communication coverage flight trajectory and maximized energy efficiency. Problem P1 is considered as a multi-agent cooperative scenario involving multiple UAVs coordinating to cover a target sea area to meet the communication needs of ships. Problem P1 is optimized using the MATD3 multi-agent reinforcement learning method. A Markov decision process is modeled, consisting of triples. Define a space where S is the state space of the drone, A is the action space of the drone, and R is the space associated with each state-action pair. The associated reward function; at each time step, the agent, according to the policy, based on the observed state... Select Action Interact with the environment and then receive a reward. And the new status ; Use the Q-value function to evaluate the action to be performed in state s Expected return ; Each drone is considered an intelligent agent, for The status of the drone The definition is as follows: (20) Drone Actions The definition is as follows: (21) drone reward function The definition is as follows: (22) in, It is a flight reward factor. , ; It is an indicator function that indicates when the distance between the drone and the ship is less than a set threshold. ;otherwise ; In MATD3, each agent uses two main evaluation networks to estimate the Q-value, and takes the smaller estimate as the target for updating the evaluation network; let the parameters of the two main evaluation networks be... and , and Each has a target network with the following parameters: and Each agent also includes two policy networks: a main policy network with parameters of... A target policy network The parameter MATD3 uses the smaller Q-value from the two target evaluation networks as the Q-value estimate for the next time step, defined by the formula: (23) in, For strategy In state The actions generated below, To add Gaussian noise, `clip` is a clipping operation that limits the action value to a certain range. and Between; main evaluation network and The loss function is defined as follows: (24) Target evaluation network and Through the main evaluation network and For soft updates, the formula is defined as follows: (25) in, It is the update factor of the target network; Main policy network The objective function is defined as follows: (26) Main policy network The update is achieved by updating the strategy. gradient propagation to The formula is defined as follows: (27) Target Policy Network Through the main policy network For soft updates, the formula is defined as follows: (28) By optimizing the action network and policy network of the intelligent agent, the optimal drone path planning is generated, thereby maximizing the energy efficiency of the multi-drone communication system.
6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by the processor, it implements the steps in the multi-agent maritime unmanned aerial vehicle-assisted communication method based on time-series prediction as described in any one of claims 1 to 5.
7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the multi-agent maritime unmanned aerial vehicle-assisted communication method based on time-series prediction as described in any one of claims 1 to 5.