Task offloading and computing resource allocation method for cross-domain ocean network based on td3

Abstract

This invention discloses a method for task offloading and computational resource allocation in cross-domain marine networks based on TD3, applicable to cross-domain marine networks composed of autonomous underwater vehicles, surface buoy nodes, and low-Earth orbit satellite nodes. For local execution mode, surface buoy node collaborative execution mode, and satellite-end execution mode, a comprehensive processing cost is constructed, including underwater optical communication transmission, surface buoy node transcoding processing, radio frequency transmission, and node computation processing. A fixed three-dimensional offloading preference vector and computational resource allocation ratio are output from the TD3 current action network. A service-aware mapping mechanism filters the mode components of the offloading preference vector according to the service availability status, and generates task offloading decisions from the mode components not masked by the service availability status through maximum component selection and one-hot encoding. The underwater optical communication uplink rate and radio frequency uplink rate are used to calculate the transmission latency, transmission energy consumption, and comprehensive processing cost of the corresponding processing mode. This invention enables joint optimization of task offloading decisions and computational resource allocation in cross-domain marine networks, reducing task processing latency and energy consumption.

Description

Technical Field

[0001] This invention belongs to the field of marine edge computing and task collaborative optimization technology. Specifically, it relates to a method for cross-domain marine network task offloading and computing resource allocation based on Twin Delayed Deep Deterministic Policy Gradient (TD3). Background Technology

[0002] With the continuous development of smart ocean and ocean of things (OoT) applications, marine environmental monitoring, underwater target identification, resource exploration, and maritime emergency support are placing higher demands on the real-time performance, reliability, and energy efficiency of task processing. Autonomous underwater vehicles (AUVs) typically require computationally intensive and latency-sensitive processing of perceived data during actual missions. However, limited by local computing power, energy supply, and underwater communication conditions, relying solely on local execution often fails to meet the complex operational needs.

[0003] To enhance maritime mission processing capabilities, cross-domain maritime networks comprised of autonomous underwater vehicles (AUVs), surface buoy nodes, and low-Earth orbit (LEO) satellite nodes are gaining increasing attention. Surface buoy nodes connect AUVs to underwater optical communication links and surface buoy nodes to LEO satellite nodes via radio frequency links, participating in mission processing as edge nodes. LEO satellite nodes possess strong computing resources and wide coverage, enabling them to form a cross-domain collaborative mission processing system with AUVs and surface buoy nodes. By offloading tasks and allocating computing resources within this system, performance bottlenecks caused by limited local resources on AUVs can be alleviated to some extent.

[0004] In the prior art, various methods for task offloading and resource allocation have been proposed for air-space-ground networks or air-sea collaborative networks. For example, the patent document "A Joint Optimization Method for Task Offloading and Resource Allocation in Air-Space-Ground Edge Computing Networks" (application date: July 1, 2025, application number: 202510901684.0, publication number: CN120751445 A, publication date: October 3, 2025, the content of this application can still be cited) gives an implementation idea for constructing task, communication and computing models and making joint decisions in air-space-ground edge computing networks. However, its solution is mainly aimed at air-space-ground scenarios and does not address the characteristics of underwater optical links and airborne radio frequency links coexisting and the link states changing significantly in cross-domain marine networks, nor does it construct a unified processing mechanism adapted to local execution mode, surface buoy node collaborative execution mode and satellite execution mode. For example, the patent document "Method for Task Offloading and Resource Allocation of Marine Edge Computing Based on Air-Sea Collaboration" (application date: July 31, 2025, application number: 202511069018.1, publication number: CN120812662 A, publication date: October 17, 2025, the content of this application can still be cited) focuses on research on air-sea collaborative marine edge computing systems. However, its solution focuses more on performance modeling such as throughput and queuing under the air-sea collaborative architecture. It still lacks targeted solutions for pattern selection under service availability constraints in cross-domain marine networks, collaborative generation of discrete task offloading decisions and continuous computing resource allocation, and online optimization mechanisms oriented towards long-term average integrated processing costs.

[0005] Therefore, existing technologies still struggle to simultaneously address the following issues in cross-domain marine network scenarios: First, under heterogeneous time-varying link conditions, how to construct a unified comprehensive processing cost representation for local execution mode, surface buoy node collaborative execution mode, and satellite execution mode; second, under service availability constraints, how to avoid generating task offloading results inconsistent with the service deployment state required by the task; and third, how to achieve joint optimization of task offloading decisions and continuous computing resource allocation. These issues make it difficult for existing methods to simultaneously ensure consistency between task offloading decisions and service deployment states, as well as reduce task processing latency and energy consumption. Summary of the Invention

[0006] Purpose of the invention: To address the problems of insufficient comprehensive processing cost representation of the three processing modes in existing technologies in cross-domain marine network scenarios, weak consistency between task offloading decisions and service deployment status, and limited joint optimization effect of task offloading decisions and continuous computing resource allocation, this invention proposes a cross-domain marine network task offloading and computing resource allocation method based on TD3.

[0007] This invention constructs a cross-domain marine network architecture comprising autonomous underwater vehicles, surface buoy nodes, and low-Earth orbit satellite nodes. It establishes three task processing modes: local execution mode, surface buoy node collaborative execution mode, and satellite-end execution mode. Furthermore, it constructs a joint optimization mechanism aimed at minimizing the long-term average comprehensive processing cost. Further, through a service-aware mapping mechanism, it filters the fixed three-dimensional offloading preference vector output by the current action network for service availability. From the mode components not masked by service availability status, it generates task offloading decisions through maximum component selection and one-hot encoding, and outputs these decisions in conjunction with the continuous computing resource allocation results, thereby reducing task processing latency and energy consumption.

[0008] To achieve the above-mentioned objectives, the present invention employs the following technical solution: A method for task offloading and computational resource allocation in cross-domain marine networks based on TD3 includes the following steps: S1: Construct a cross-domain ocean network consisting of autonomous underwater vehicles (AUVs), surface buoy nodes, and low-Earth orbit (LEO) satellite nodes, and establish a computational task model and a service availability status model; wherein, the surface buoy nodes are deployed on the water surface to connect heterogeneous links between AUVs and LEO satellite nodes, and the heterogeneous links include underwater optical communication links from AUVs to surface buoy nodes and radio frequency links from surface buoy nodes to LEO satellite nodes; S2: Acquire the mission information, service availability status, available computing resource status of each node, and heterogeneous link transmission status generated by the autonomous underwater vehicle in each time slot, and construct three processing modes: local execution mode, surface buoy node collaborative execution mode, and satellite execution mode. The satellite execution mode includes at least underwater optical communication transmission from the autonomous underwater vehicle to the surface buoy node, transcoding processing of the surface buoy node, and radio frequency transmission from the surface buoy node to the low-Earth orbit satellite node. S3: Construct a service-aware mapping mechanism. Based on the service availability status, perform service availability filtering on the fixed three-dimensional unloading preference vector output by the current action network. Mask the mode components corresponding to the processing modes where services are unavailable, and select the maximum component and perform one-hot encoding from the mode components that have not been masked by the service availability status to generate a task unloading decision vector. The three mode components of the fixed three-dimensional unloading preference vector correspond to the local execution mode, the surface buoy node collaborative execution mode, and the satellite execution mode, respectively. Since the low-Earth orbit satellite node deploys all service types, the mode component corresponding to the satellite execution mode is not masked by the service availability status due to the lack of service types required by the task on the low-Earth orbit satellite node. S4: Calculate the overall processing cost based on the task unloading decision vector and the computing resource allocation ratio, and with the goal of minimizing the long-term average overall processing cost, the current action network of the TD3 algorithm outputs a continuous fixed three-dimensional unloading preference vector and a computing resource allocation ratio; the heterogeneous link transmission status is used to calculate the transmission delay, transmission energy consumption, and overall processing cost in the corresponding processing mode; the overall processing cost is composed of task delay and task energy consumption weighted by a preset fixed conversion factor; the task unloading and computing resource allocation process is iteratively solved, and the task unloading decision and computing resource allocation results are output.

[0009] Furthermore, step S1 is specifically as follows: S1-1: Let the set of autonomous underwater vehicles be... The set of surface buoy nodes is The set of low-orbit satellite nodes is The service type set is The time slot set is ;in, Indicates the index of autonomous underwater vehicles; Indicates the index of the surface buoy node; Indicates the index of low-Earth orbit satellite nodes; Indicates a service type index; Indicates the time slot index; Indicates the total number of time slots; S1-2: In any time slot Inside, autonomous underwater vehicle Generate the computation task, denoted as: , in, Indicates the amount of task data; Indicates the number of CPU cycles required for the task; Indicates the type of service required for the task; S1-3: Definition These represent the service types. autonomous underwater vehicles and surface buoy nodes The service availability status at the location; a value of 1 indicates availability, and a value of 0 indicates unavailability; low-Earth orbit satellite nodes deploy all service types; autonomous underwater vehicle and surface buoy nodes deploy some service types; S1-4: In the cross-domain marine network, autonomous underwater vehicles, surface buoy nodes, and low-Earth orbit satellite nodes each have corresponding computing, caching, and communication capabilities. Among them, autonomous underwater vehicles and surface buoy nodes upload tasks through an underwater optical communication link, and surface buoy nodes and low-Earth orbit satellite nodes relay tasks through a radio frequency link, thus forming a basic environment for cross-domain collaborative processing.

[0010] Furthermore, step S2 is specifically as follows: S2-1: In each time slot Inside, collect data from autonomous underwater vehicles. Task information Service availability status and Autonomous underwater vehicles To the surface buoy node underwater optical communication uplink rate Surface buoy nodes To low Earth orbit satellite nodes RF uplink rate Autonomous underwater vehicles Available computing speed Surface buoy nodes Available computing speed Low Earth Orbit Satellite Nodes Available computing speed The underwater optical communication uplink rate is mentioned above. and RF uplink rate Used to characterize the transmission status of heterogeneous links; S2-2: Three processing modes are constructed for computing tasks: local execution mode, surface buoy node collaborative execution mode, and satellite execution mode. Among them, the local execution mode means that the task is processed locally by the autonomous underwater vehicle; the surface buoy node collaborative execution mode means that the task is offloaded to the surface buoy node via the underwater optical communication link and then processed by the surface buoy node; the satellite execution mode means that the task is first uploaded by the autonomous underwater vehicle to the surface buoy node, transcoded by the surface buoy node, and then uploaded to the low-Earth orbit satellite node via the radio frequency link and processed by the low-Earth orbit satellite node. S2-3: Definition and They are time slots Internal allocation to autonomous underwater vehicles The buoy-side computing resource allocation ratio and the satellite-side computing resource allocation ratio, and satisfying the following:

[0011] S2-4: Define the comprehensive processing costs corresponding to the local execution mode, the surface buoy node collaborative execution mode, and the satellite execution mode. The comprehensive processing cost of the local execution mode consists of the task latency and energy consumption generated by the autonomous underwater vehicle's local computation. The comprehensive processing cost of the surface buoy node collaborative execution mode consists of the underwater optical communication transmission cost from the autonomous underwater vehicle to the surface buoy node and the computation cost of the surface buoy node. The comprehensive processing cost of the satellite execution mode consists of the underwater optical communication transmission cost from the autonomous underwater vehicle to the surface buoy node, the transcoding processing cost of the surface buoy node, the radio frequency transmission cost from the surface buoy node to the low-Earth orbit satellite node, and the computation cost of the low-Earth orbit satellite node. Furthermore, step S3 is specifically as follows: S3-1: Defining Autonomous Underwater Vehicles In the time slot The unloading preference vector is:

[0012] in, , and These represent the mode preference values ​​output by the current action network for the local execution mode, the surface buoy node collaborative execution mode, and the satellite execution mode, respectively. S3-2: Perform service availability filtering on the fixed three-dimensional unloading preference vector to obtain a fixed three-dimensional candidate preference vector:

[0013] in:

[0014]

[0015] ; in, , and These represent the mode preference values ​​after service availability filtering for local execution mode, surface buoy node collaborative execution mode, and satellite execution mode, respectively. when At that time, the mode component corresponding to the local execution mode is masked by the service availability status; when At that time, the mode component corresponding to the collaborative execution mode of the surface buoy node is masked by the service availability status; since the low-orbit satellite node deploys all service types, the mode component corresponding to the satellite execution mode is not masked by the service availability status due to the lack of service types required for the task by the low-orbit satellite node.

[0016] S3-3: The unloading preference vector before and after the service availability filtering is a fixed three-dimensional vector. The service availability status is only used to determine whether the corresponding mode component participates in the maximum component comparison in the current time slot, and does not change the output dimension of the current action network.

[0017] S3-4: Select the maximum component from the pattern components that are not masked by the service availability state, and generate the task offloading decision vector through one-hot encoding:

[0018] in, ; To handle schema indexes, Indicates local execution mode. This indicates the collaborative execution mode of the surface buoy nodes. Indicates the star-end execution mode; This represents a one-hot coding function that maps the selected processing mode index to a three-dimensional binary vector.

[0019] S3-5: It should be noted that the service availability screening in this step is only used to ensure that the task offloading decision is consistent with the deployment status of the services required by the task at the corresponding nodes. Since low-Earth orbit satellite nodes deploy all service types, the mode component corresponding to the satellite-end execution mode is not masked by the service availability status due to the lack of the required service types on the low-Earth orbit satellite nodes. The heterogeneous link transmission status is not a newly added mode masking condition in this step, but is used, as described in steps S2 and S4, for calculating transmission latency, transmission energy consumption, and overall processing cost under the corresponding processing mode.

[0020] Furthermore, step S4 is specifically as follows: S4-1: Task unloading decision vector and computing resource allocation ratio , As a decision quantity to be determined, define autonomous underwater vehicle In the time slot The overall processing cost is as follows; , , These represent the task latency corresponding to local execution mode, surface buoy node collaborative execution mode, and satellite execution mode, respectively. , , These represent the task energy consumption for local execution mode, surface buoy node collaborative execution mode, and satellite execution mode, respectively. Based on the task unloading decision vector, the autonomous underwater vehicle In the time slot The task latency is as follows: ; Autonomous underwater vehicle In the time slot The energy consumption of the following task is: ; The overall processing cost is represented by a weighted sum of task latency and task energy consumption:

[0021] in, This is the task delay reduction factor, and its unit is seconds (s). -1 ; This is the energy consumption conversion factor for the task, and its unit is J. -1 Due to task delays The unit is seconds (s), and the task energy consumption is... The unit is J, therefore and All are dimensionless values ​​and can be added under the same comprehensive processing cost scale; and All of these are non-negative constants pre-set before training begins and remain unchanged during a single training and execution process; S4-2: In time slot System input There are exactly seven pieces of information, namely, task information. Autonomous underwater vehicles To the surface buoy node underwater optical communication uplink rate Surface buoy nodes To low Earth orbit satellite nodes RF uplink rate Autonomous underwater vehicles Available computing speed Surface buoy nodes Available computing speed Low Earth Orbit Satellite Nodes Available computing speed Service availability status set The underwater optical communication uplink rate is mentioned above. and RF uplink rate Used to calculate transmission latency, transmission energy consumption, and overall processing cost in the collaborative execution mode and satellite execution mode of surface buoy nodes; System output There are exactly three pieces of information: the unloading preference vector. 1. Calculate the resource allocation ratio on the buoy side. and satellite-side computing resource allocation ratio ; wherein, the unloading preference vector The filtered candidate preference vector is obtained after processing by the service-aware mapping mechanism. Then, the task unloading decision vector is generated through maximum component selection and one-hot encoding. Instant reward function Defined as the comprehensive processing cost in the current time slot. The negative value, that is ; S4-3: The TD3 algorithm includes a current action network, a target action network, two current evaluation networks and their corresponding target evaluation networks; the current action network continuously acts according to the system input and output of the current time slot, and the continuous actions include a fixed three-dimensional unloading preference vector, a buoy-side computing resource allocation ratio and a satellite-side computing resource allocation ratio. The fixed three-dimensional unloading preference vector is processed by the service-aware mapping mechanism to generate the task unloading decision vector used in the environment execution phase. Service availability filtering, maximum component selection and one-hot encoding do not change the continuous action dimension of the current action network output. The current evaluation network takes the system state and the continuous actions output by the current action network as input, and updates the temporal difference error based on the immediate reward calculated according to the task unloading decision vector and the computing resource allocation ratio in the environment. The current action network updates the policy gradient based on the evaluation results of the continuous actions by the current evaluation network. Maximum component selection and one-hot encoding are not located in the policy gradient calculation path from the current evaluation network to the current action network. In each time slot, the current action network re-outputs a continuous action of fixed dimensions based on the system input of that time slot. Changes in service availability do not change the input and output dimensions of the current action network and the current evaluation network. The current evaluation network is updated based on the smaller evaluation result between the two target evaluation networks, and the target action network and target evaluation network are updated using a soft update method. After the network training converges, the task offloading decision and the corresponding computational resource allocation results for each time slot are output.

[0022] The advantages and technical effects of this invention are as follows: Compared with existing technologies, this invention is aimed at cross-domain marine networks composed of autonomous underwater vehicles, surface buoy nodes, and low-Earth orbit satellite nodes. It establishes a unified and comprehensive processing cost representation method for local execution mode, surface buoy node collaborative execution mode, and satellite execution mode. It can adapt to heterogeneous time-varying environments where underwater optical communication links and airborne radio frequency links coexist, and provides a foundation for joint optimization of task offloading and computing resource allocation.

[0023] Meanwhile, this invention proposes a service-aware mapping mechanism, which filters the pattern components of a fixed three-dimensional offloading preference vector for service availability, and generates task offloading decisions from the pattern components that have not been masked by the service availability status through maximum component selection and one-hot encoding, so that the discrete task offloading decisions are consistent with the deployment status of the services required by the task.

[0024] Furthermore, this invention employs TD3 to jointly solve the task offloading decision and computing resource allocation process, and evaluates the joint decision results through comprehensive processing cost and immediate reward, thereby reducing the long-term average comprehensive processing cost, task processing latency, and energy consumption. Attached Figure Description

[0025] Figure 1 This is an overall flowchart of one embodiment of the present invention; Figure 2 This is a network architecture diagram of one embodiment of the present invention; Figure 3 This is a structural diagram of the TD3 used in one embodiment of the present invention; Figure 4 This is a comparison chart of the average task processing latency between the method provided by this invention and existing methods under different comprehensive channel degradation coefficients according to an embodiment of this invention; Figure 5 This is a graph showing the comparison of average task energy consumption between the method provided by this invention and existing methods under a fixed integrated channel degradation coefficient, according to an embodiment of this invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0027] This embodiment proposes a method for task offloading and computational resource allocation in cross-domain marine networks based on TD3, and its overall flowchart is as follows. Figure 1 As shown, it includes the following steps: S1: Construct a cross-domain ocean network consisting of autonomous underwater vehicles (AUVs), surface buoy nodes, and low-Earth orbit (LEO) satellite nodes, and establish a computational task model and a service availability status model. The surface buoy nodes are deployed on the water surface to connect heterogeneous links between AUVs and LEO satellite nodes. These heterogeneous links include underwater optical communication links from AUVs to surface buoy nodes and radio frequency links from surface buoy nodes to LEO satellite nodes. The specific steps are as follows: S1-1: Constructing a cross-domain ocean edge network in a cross-domain ocean scenario, consisting of autonomous underwater vehicles, surface buoy nodes, and low-Earth orbit satellite nodes; such as... Figure 2As shown, three low-Earth orbit satellite nodes, three surface buoy nodes, and five autonomous underwater vehicles (AUVs) are deployed; the AUVs are grouped as follows: The set of surface buoy nodes is The set of low-orbit satellite nodes is The service type set is In this embodiment, there are 6 different types of services; the time slot set is... ;in, Indicates the index of autonomous underwater vehicles; Indicates the index of the surface buoy node; Indicates the index of low-Earth orbit satellite nodes; Indicates a service type index; Indicates the time slot index; Indicates the total number of time slots; S1-2: In any time slot Inside, autonomous underwater vehicle Generate the computation task, denoted as: , in, Indicates the amount of task data; Indicates the number of CPU cycles required for the task; Indicates the type of service required for the task; S1-3: Definition These represent the service types. autonomous underwater vehicles and surface buoy nodes The service availability status at the location; a value of 1 indicates availability, and a value of 0 indicates unavailability; low-Earth orbit satellite nodes deploy all service types; autonomous underwater vehicle and surface buoy nodes deploy some service types; S1-4: In the cross-domain marine network, autonomous underwater vehicles, surface buoy nodes, and low-Earth orbit satellite nodes each have corresponding computing, caching, and communication capabilities. Among them, autonomous underwater vehicles and surface buoy nodes upload tasks through an underwater optical communication link, and surface buoy nodes and low-Earth orbit satellite nodes relay tasks through a radio frequency link, thus forming a basic environment for cross-domain collaborative processing.

[0028] S2: Acquire mission information, service availability status, available computing resource status of each node, and heterogeneous link transmission status generated by the autonomous underwater vehicle in each time slot. Construct three processing modes: local execution mode, surface buoy node collaborative execution mode, and satellite execution mode. The satellite execution mode includes at least underwater optical communication transmission from the autonomous underwater vehicle to the surface buoy node, transcoding processing at the surface buoy node, and radio frequency transmission from the surface buoy node to the low-Earth orbit satellite node. The specific steps are as follows: S2-1: In each time slot Inside, collect data from autonomous underwater vehicles. Task information Service availability status and Autonomous underwater vehicles To the surface buoy node underwater optical communication uplink rate Surface buoy nodes To low Earth orbit satellite nodes RF uplink rate Autonomous underwater vehicles Available computing speed Surface buoy nodes Available computing speed Low Earth Orbit Satellite Nodes Available computing speed The underwater optical communication uplink rate is mentioned above. and RF uplink rate Used to characterize the transmission status of heterogeneous links; S2-2: Three processing modes are constructed for computing tasks: local execution mode, surface buoy node collaborative execution mode, and satellite execution mode. Among them, the local execution mode means that the task is processed locally by the autonomous underwater vehicle; the surface buoy node collaborative execution mode means that the task is offloaded to the surface buoy node via the underwater optical communication link and then processed by the surface buoy node; the satellite execution mode means that the task is first uploaded by the autonomous underwater vehicle to the surface buoy node, transcoded by the surface buoy node, and then uploaded to the low-Earth orbit satellite node via the radio frequency link and processed by the low-Earth orbit satellite node. S2-3: Definition and They are time slots Internal allocation to autonomous underwater vehicles The buoy-side computing resource allocation ratio and the satellite-side computing resource allocation ratio, and satisfying the following:

[0029] S2-4: Define the comprehensive processing costs corresponding to the local execution mode, the surface buoy node collaborative execution mode, and the satellite execution mode. The comprehensive processing cost of the local execution mode consists of the task latency and energy consumption generated by the autonomous underwater vehicle's local computation. The comprehensive processing cost of the surface buoy node collaborative execution mode consists of the underwater optical communication transmission cost from the autonomous underwater vehicle to the surface buoy node and the computation cost of the surface buoy node. The comprehensive processing cost of the satellite execution mode consists of the underwater optical communication transmission cost from the autonomous underwater vehicle to the surface buoy node, the transcoding processing cost of the surface buoy node, the radio frequency transmission cost from the surface buoy node to the low-Earth orbit satellite node, and the computation cost of the low-Earth orbit satellite node. S3: Construct a service-aware mapping mechanism. Based on the service availability status, perform service availability filtering on the fixed three-dimensional offloading preference vector output by the current action network. Mask the mode components corresponding to the service unavailability processing modes, and select the maximum component and perform one-hot encoding from the mode components not masked by the service availability status to generate a task offloading decision vector. The three mode components of the fixed three-dimensional offloading preference vector correspond to the local execution mode, the surface buoy node collaborative execution mode, and the satellite execution mode, respectively. Since low-Earth orbit satellite nodes deploy all service types, the mode components corresponding to the satellite execution mode are not masked by the service availability status due to the lack of service types required by the task on the low-Earth orbit satellite nodes. The specific steps are as follows: S3-1: Defining Autonomous Underwater Vehicles In the time slot The unloading preference vector is:

[0030] in, , and These represent the mode preference values ​​output by the current action network for the local execution mode, the surface buoy node collaborative execution mode, and the satellite execution mode, respectively. S3-2: Perform service availability filtering on the fixed three-dimensional unloading preference vector to obtain a fixed three-dimensional candidate preference vector:

[0031] in:

[0032]

[0033] ; in, , and These represent the mode preference values ​​after service availability filtering for local execution mode, surface buoy node collaborative execution mode, and satellite execution mode, respectively. when At that time, the mode component corresponding to the local execution mode is masked by the service availability status; when At that time, the mode component corresponding to the collaborative execution mode of the surface buoy node is masked by the service availability status; since the low-orbit satellite node deploys all service types, the mode component corresponding to the satellite execution mode is not masked by the service availability status due to the lack of service types required for the task by the low-orbit satellite node.

[0034] S3-3: The unloading preference vectors before and after the service availability filtering are both fixed three-dimensional vectors. The service availability status is only used to determine whether the corresponding mode component participates in the maximum component comparison in the current time slot, and does not change the output dimension of the current action network.

[0035] S3-4: Select the maximum component from the pattern components that are not masked by the service availability state, and generate the task offloading decision vector through one-hot encoding:

[0036] in, ; To handle schema indexes, Indicates local execution mode. This indicates the collaborative execution mode of the surface buoy nodes. Indicates the star-end execution mode; This represents a one-hot coding function that maps the selected processing mode index to a three-dimensional binary vector.

[0037] S3-5: It should be noted that the service availability screening in this step is only used to ensure that the task offloading decision is consistent with the deployment status of the services required by the task at the corresponding nodes. Since low-Earth orbit satellite nodes deploy all service types, the mode component corresponding to the satellite-end execution mode is not masked by the service availability status due to the lack of the required service types on the low-Earth orbit satellite nodes. The heterogeneous link transmission status is not a newly added mode masking condition in this step, but is used, as described in steps S2 and S4, for calculating transmission latency, transmission energy consumption, and overall processing cost under the corresponding processing mode.

[0038] S4: Calculate the overall processing cost based on the task offloading decision vector and the computing resource allocation ratio, and aim to minimize the long-term average overall processing cost. The current action network of the TD3 algorithm outputs a continuous fixed three-dimensional offloading preference vector and a computing resource allocation ratio. The heterogeneous link transmission status is used to calculate the transmission delay, transmission energy consumption, and overall processing cost in the corresponding processing mode. The overall processing cost is composed of task delay and task energy consumption weighted by a preset fixed conversion factor. The task offloading and computing resource allocation process is iteratively solved to output the task offloading decision and computing resource allocation results. The specific steps are as follows: S4-1: Task unloading decision vector and computing resource allocation ratio , As a decision quantity to be determined, define autonomous underwater vehicle In the time slot The overall processing cost is as follows; , , These represent the task latency corresponding to local execution mode, surface buoy node collaborative execution mode, and satellite execution mode, respectively. , , These represent the task energy consumption for local execution mode, surface buoy node collaborative execution mode, and satellite execution mode, respectively. Based on the task unloading decision vector, the autonomous underwater vehicle In the time slot The task latency is as follows: ; Autonomous underwater vehicle In the time slot The energy consumption of the following task is: ; The overall processing cost is represented by a weighted sum of task latency and task energy consumption:

[0039] in, This is the task delay reduction factor, and its unit is seconds (s). -1 ; This is the energy consumption conversion factor for the task, and its unit is J. -1 Due to task delays The unit is seconds (s), and the task energy consumption is... The unit is J, therefore and All are dimensionless values ​​and can be added under the same comprehensive processing cost scale; and All of these are non-negative constants pre-set before training begins and remain unchanged during a single training and execution process; S4-2: In time slot System input There are exactly seven pieces of information, namely, task information. Autonomous underwater vehicles To the surface buoy node underwater optical communication uplink rate Surface buoy nodes To low Earth orbit satellite nodes RF uplink rate Autonomous underwater vehicles Available computing speed Surface buoy nodes Available computing speed Low Earth Orbit Satellite Nodes Available computing speed Service availability status set The underwater optical communication uplink rate is mentioned above. and RF uplink rate Used to calculate transmission latency, transmission energy consumption, and overall processing cost in the collaborative execution mode and satellite execution mode of surface buoy nodes; System output There are exactly three pieces of information: the unloading preference vector. 1. Calculate the resource allocation ratio on the buoy side. and satellite-side computing resource allocation ratio ; wherein, the unloading preference vector The filtered candidate preference vector is obtained after processing by the service-aware mapping mechanism. Then, the task unloading decision vector is generated through maximum component selection and one-hot encoding. Instant reward function Defined as the comprehensive processing cost in the current time slot. The negative value, that is ; S4-3: As Figure 3 As shown, the TD3 algorithm includes a current action network, a target action network, two current evaluation networks and their corresponding target evaluation networks; the current action network continuously performs actions based on the system input and output of the current time slot, and the continuous actions include a fixed three-dimensional offloading preference vector, a buoy-side computing resource allocation ratio and a satellite-side computing resource allocation ratio; The fixed three-dimensional unloading preference vector is processed by the service-aware mapping mechanism to generate the task unloading decision vector used in the environment execution phase. Service availability filtering, maximum component selection and one-hot encoding do not change the continuous action dimension of the current action network output. The current evaluation network takes the system state and the continuous actions output by the current action network as input, and updates the temporal difference error based on the immediate reward calculated according to the task unloading decision vector and the computing resource allocation ratio in the environment. The current action network updates the policy gradient based on the evaluation results of the continuous actions by the current evaluation network. Maximum component selection and one-hot encoding are not located in the policy gradient calculation path from the current evaluation network to the current action network. In each time slot, the current action network re-outputs a continuous action of fixed dimensions based on the system input of that time slot. Changes in service availability do not change the input and output dimensions of the current action network and the current evaluation network. The current evaluation network is updated based on the smaller evaluation result between the two target evaluation networks, and the target action network and target evaluation network are updated using a soft update method. After the network training converges, the task offloading decision and the corresponding computational resource allocation results for each time slot are output.

[0040] To verify the effectiveness of the TD3-based cross-domain marine network task offloading and computational resource allocation method provided in this invention under cross-domain marine heterogeneous channel conditions, this embodiment sets up a joint optimization scheme based on Deep Deterministic Policy Gradient (DDPG), an ablation scheme removing the service-aware mapping mechanism, and a local execution-only scheme as comparative schemes, and conducts comparative experiments under the same simulation conditions. The comparison results are as follows: Figure 4 and Figure 5 As shown, where, Figure 4 Used to compare average task processing latency Figure 5 Used to compare average task energy consumption.

[0041] This embodiment was simulated in Matlab R2021a. The experimental platform was an Intel Core i7-12700H 2.70 GHz processor based on x64 architecture, with 16 GB of memory. The same random seed and parameter configuration were used for location and channel state, task generation, and available computing resources for each node in all schemes to ensure fairness in the comparison. To reduce the impact of random fluctuations on the experimental conclusions, each scheme was independently repeated multiple times. The results in the figure are the average of the repeated experiments. The main simulation parameters are listed in Table 1.

[0042] This embodiment introduces a comprehensive channel degradation coefficient. As a dimensionless parameter, it is used to characterize the degree of attenuation enhancement of underwater optical links and the deterioration of cross-domain transmission conditions; The larger the value, the worse the heterogeneous channel conditions. Figure 4The average task processing latency is measured by the average completion latency of all AUV tasks within the statistical window. Figure 5 The average task energy consumption is measured by the average energy consumption of all AUV tasks within the system within the statistical window. For both metrics, lower values ​​are preferred. In this embodiment, task latency is expressed in seconds, and task energy consumption in joules. The task latency and task energy consumption conversion factors remain constant throughout the entire training and testing process.

[0043] Table 1 Simulation Parameters

[0044] Under the above simulation environment and parameter configuration, the method of this invention and the comparative scheme were used to conduct joint optimization simulation of task offloading and computational resource allocation under cross-domain marine heterogeneous channel conditions. The comparison results are as follows: Figure 4 and Figure 5 As shown.

[0045] Figure 4 The average task processing delay of each scheme under different comprehensive channel degradation coefficients is compared. It can be seen that, with... The average task processing latency of all schemes increases with the increase in latency. Among them, the local execution scheme, which relies entirely on the local computing resources of the AUV, cannot effectively utilize the collaborative processing capabilities of the surface buoy nodes and low-Earth orbit satellite nodes, resulting in the most significant increase in latency. Although the ablation scheme that removes the service-aware mapping mechanism introduces continuous action learning, it lacks effective screening of candidate processing modes under service availability constraints. This may lead to decision results that are inconsistent with the service deployment state during the discrete task offloading decision generation process, resulting in limited latency improvement. The DDPG joint optimization scheme can reduce task processing latency to some extent, but its joint decision quality and stability still have certain limitations under cross-domain heterogeneous time-varying channel conditions. In contrast, the method of this invention maintains a low average task processing latency across the entire range, indicating that it can more effectively coordinate the local execution mode, the surface buoy node collaborative execution mode, and the satellite execution mode, and perform cost-aware joint optimization based on the transmission cost and computing resource status reflected by the heterogeneous link transmission state.

[0046] Figure 5 The following is given under the condition of fixed synthetic channel degradation coefficient ( =1.8) Comparison of average task energy consumption among various schemes. It can be seen that the method of this invention has the lowest average task energy consumption, followed by the DDPG joint optimization scheme, while the ablation scheme removing the service-aware mapping mechanism and the local execution only scheme have higher energy consumption. This is because this invention treats task offloading decisions and computational resource allocation as a unified optimization problem, and ensures the consistency between discrete task offloading decisions and service states through the service-aware mapping mechanism, thereby reducing additional transmission and computational energy consumption caused by offloading decisions inconsistent with service deployment states or unreasonable resource allocation; simultaneously, using TD3 for joint optimization is beneficial for obtaining better task offloading and resource allocation results in dynamic environments.

[0047] In summary, this invention can effectively achieve joint optimization of task offloading decision-making and computing resource allocation in cross-domain marine networks, reducing network task processing latency and energy consumption under heterogeneous channel conditions.

[0048] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions claimed by the present invention.

Claims

1. A method for TD3-based cross-domain ocean network task offloading and computing resource allocation, characterized in that, Includes the following steps: S1: Construct a cross-domain ocean network consisting of autonomous underwater vehicles (AUVs), surface buoy nodes, and low-Earth orbit (LEO) satellite nodes, and establish a computational task model and a service availability status model; wherein, the surface buoy nodes are deployed on the water surface to connect heterogeneous links between AUVs and LEO satellite nodes, and the heterogeneous links include underwater optical communication links from AUVs to surface buoy nodes and radio frequency links from surface buoy nodes to LEO satellite nodes; S2: Acquire the mission information, service availability status, available computing resource status of each node, and heterogeneous link transmission status generated by the autonomous underwater vehicle in each time slot, and construct three processing modes: local execution mode, surface buoy node collaborative execution mode, and satellite execution mode. The satellite execution mode includes at least underwater optical communication transmission from the autonomous underwater vehicle to the surface buoy node, transcoding processing of the surface buoy node, and radio frequency transmission from the surface buoy node to the low-Earth orbit satellite node. S3: Construct a service-aware mapping mechanism. Based on the service availability status, perform service availability filtering on the fixed three-dimensional unloading preference vector output by the current action network. Mask the mode components corresponding to the processing modes where services are unavailable, and select the maximum component and perform one-hot encoding from the mode components that have not been masked by the service availability status to generate a task unloading decision vector. The three mode components of the fixed three-dimensional unloading preference vector correspond to the local execution mode, the surface buoy node collaborative execution mode, and the satellite execution mode, respectively. Since the low-Earth orbit satellite node deploys all service types, the mode component corresponding to the satellite execution mode is not masked by the service availability status due to the lack of service types required by the task on the low-Earth orbit satellite node. S4: Calculate the overall processing cost based on the task unloading decision vector and the computing resource allocation ratio, and with the goal of minimizing the long-term average overall processing cost, the current action network of the TD3 algorithm outputs a continuous fixed three-dimensional unloading preference vector and a computing resource allocation ratio; the heterogeneous link transmission status is used to calculate the transmission delay, transmission energy consumption, and overall processing cost in the corresponding processing mode; the overall processing cost is composed of task delay and task energy consumption weighted by a preset fixed conversion factor; the task unloading and computing resource allocation process is iteratively solved, and the task unloading decision and computing resource allocation results are output.

2. The method for task offloading and computational resource allocation in a cross-domain marine network based on TD3 according to claim 1, characterized in that: Let the set of autonomous underwater vehicles be... The set of surface buoy nodes is The set of low-orbit satellite nodes is The service type set is The time slot set is ;in, Indicates the index of autonomous underwater vehicles; Indicates the index of the surface buoy node; Indicates the index of low-Earth orbit satellite nodes; Indicates a service type index; Indicates the time slot index; Indicates the total number of time slots; In any time slot Inside, autonomous underwater vehicle Generate computational tasks, define These represent the service types. autonomous underwater vehicles and surface buoy nodes The service availability status at the location; a value of 1 indicates availability, and a value of 0 indicates unavailability; low-Earth orbit satellite nodes deploy all service types; autonomous underwater vehicle and surface buoy nodes deploy some service types; Define autonomous underwater vehicles In the time slot The unloading preference vector is: in, , and These represent the mode preference values ​​output by the current action network for the local execution mode, the surface buoy node collaborative execution mode, and the satellite execution mode, respectively. Define autonomous underwater vehicles In the time slot The task unloading decision vector is: in, , and Let represent the binary decision variables for the local execution mode, the surface buoy node collaborative execution mode, and the satellite execution mode, respectively, and satisfy . ; definition and They are time slots Internal allocation to autonomous underwater vehicles The buoy-side computing resource allocation ratio and the satellite-side computing resource allocation ratio, and satisfying the following: 。 3. The method for task offloading and computational resource allocation in a cross-domain marine network based on TD3 according to claim 2, characterized in that: Service availability filtering is performed on the fixed three-dimensional offloading preference vector to obtain a fixed three-dimensional candidate preference vector: in: ; in, , and These represent the mode preference values ​​after service availability filtering for local execution mode, surface buoy node collaborative execution mode, and satellite execution mode, respectively. when At that time, the mode component corresponding to the local execution mode is masked by the service availability status; when At that time, the mode component corresponding to the collaborative execution mode of the surface buoy node is masked by the service availability status; because the low-orbit satellite node deploys all service types, the mode component corresponding to the satellite execution mode is not masked by the service availability status due to the lack of service types required for the task by the low-orbit satellite node. The unloading preference vectors before and after the service availability filtering are both fixed three-dimensional vectors; the mode components masked by the service availability status do not participate in the maximum component comparison. The maximum component is selected from the pattern components that are not masked by the service availability state, and the task offloading decision vector is generated through one-hot encoding: in, ; To process the schema index, Indicates local execution mode. This indicates the collaborative execution mode of the surface buoy nodes. Indicates the star-end execution mode; This represents a one-hot coding function that maps the selected processing mode index to a three-dimensional binary vector.

4. The method for task offloading and computational resource allocation in a cross-domain marine network based on TD3 according to claim 3, characterized in that: The overall processing cost of the local execution mode consists of the task latency and energy consumption generated by the autonomous underwater vehicle's local computing and processing; the overall processing cost of the surface buoy node collaborative execution mode consists of the underwater optical communication transmission cost from the autonomous underwater vehicle to the surface buoy node and the computing and processing cost of the surface buoy node; the overall processing cost of the satellite-end execution mode consists of the underwater optical communication transmission cost from the autonomous underwater vehicle to the surface buoy node, the transcoding processing cost of the surface buoy node, the radio frequency transmission cost from the surface buoy node to the low-Earth orbit satellite node, and the computing and processing cost of the low-Earth orbit satellite node. Based on the task latency and energy consumption corresponding to the three processing modes, a comprehensive processing cost is constructed, which is represented by the weighted sum of task latency and energy consumption. With minimizing the long-term average comprehensive processing cost as the optimization objective, the TD3 algorithm is used to solve the task unloading and computing resource allocation process. set up , , These represent the task latency corresponding to the local execution mode, the surface buoy node collaborative execution mode, and the satellite execution mode, respectively. , , These represent the task energy consumption for local execution mode, surface buoy node collaborative execution mode, and satellite execution mode, respectively. Based on the task unloading decision vector, the autonomous underwater vehicle In the time slot The task latency is as follows: ; Autonomous underwater vehicle In the time slot The energy consumption of the following task is: ; The overall processing cost is represented by a weighted sum of task latency and task energy consumption: in, This is the task delay reduction factor, and its unit is seconds (s). -1 ; This is the energy consumption conversion factor for the task, and its unit is J. -1 Due to task delay The unit is seconds (s), and the task energy consumption is... The unit is J, therefore and All are dimensionless values ​​and can be added under the same comprehensive processing cost scale; and All of these are non-negative constants pre-set before training begins and remain unchanged during a single training and execution process; In the time slot System input There are exactly seven pieces of information, namely, task information. Autonomous underwater vehicles To the surface buoy node underwater optical communication uplink rate Surface buoy nodes To low Earth orbit satellite nodes RF uplink rate Autonomous underwater vehicles Available computing speed Surface buoy nodes Available computing speed Low Earth Orbit Satellite Nodes Available computing speed Service availability status set The underwater optical communication uplink rate is mentioned above. and RF uplink rate Used to calculate transmission latency, transmission energy consumption, and overall processing cost in the collaborative execution mode and satellite execution mode of surface buoy nodes; System output There are exactly three pieces of information: the unloading preference vector.

1. Calculate the resource allocation ratio on the buoy side and satellite-side computing resource allocation ratio ; wherein, the unloading preference vector The filtered candidate preference vector is obtained after processing by the service-aware mapping mechanism. Then, the task unloading decision vector is generated through maximum component selection and one-hot encoding. ; Instant reward function Defined as the comprehensive processing cost in the current time slot. The negative value, that is The TD3 algorithm includes a current action network, a target action network, two current evaluation networks and their corresponding target evaluation networks; the current action network continuously performs actions based on the system input and output of the current time slot, and the continuous actions include a fixed three-dimensional offloading preference vector, a buoy-side computing resource allocation ratio and a satellite-side computing resource allocation ratio; The fixed three-dimensional unloading preference vector is processed by the service-aware mapping mechanism to generate the task unloading decision vector used in the environment execution phase. Service availability filtering, maximum component selection and one-hot encoding do not change the continuous action dimension of the current action network output. The current evaluation network takes the system state and the continuous actions output by the current action network as input, and updates the temporal difference error based on the immediate reward calculated according to the task unloading decision vector and the computing resource allocation ratio in the environment. The current action network updates the policy gradient based on the evaluation results of the continuous actions by the current evaluation network. Maximum component selection and one-hot encoding are not located in the policy gradient calculation path from the current evaluation network to the current action network. In each time slot, the current action network re-outputs a continuous action of fixed dimensions based on the system input of that time slot. Changes in service availability do not change the input and output dimensions of the current action network and the current evaluation network. The current evaluation network is updated based on the smaller evaluation result between the two target evaluation networks, and the target action network and target evaluation network are updated using a soft update method. After the network training converges, the task offloading decision and the corresponding computational resource allocation results for each time slot are output.

Images