A method for task offloading and collaborative caching in two-layer mobile ad hoc networks

By jointly optimizing task offloading and caching in a two-layer mobile ad hoc network and combining multi-agent reinforcement learning, the performance shortcomings of traditional methods in high mobility and topology change scenarios are solved, and system latency is minimized and resources are optimized.

CN123124018APending Publication Date: 2026-11-17STATE GRID SHANDONG ELECTRIC POWER CO +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611168386.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-03
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Traditional single-layer mobile ad hoc networks struggle to achieve globally optimal performance in scenarios with high mobility and frequent topology changes. Furthermore, existing joint optimization methods for task offloading and caching suffer from insufficient computational complexity and real-time decision-making capabilities in large-scale networks, leading to system resource redundancy and high communication overhead.

Method used

A mobile ad hoc network with a two-layer architecture is adopted. By jointly optimizing task offloading decisions, program caching strategies and routing planning, and combining a multi-agent reinforcement learning framework, distributed collaborative decision-making is achieved, thereby reducing the average system latency.

Benefits of technology

It significantly reduced task processing latency, improved cache hit rate and resource utilization, adapted to dynamic environmental changes, and optimized system performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123124018A_ABST
    Figure CN123124018A_ABST
Patent Text Reader

Abstract

This invention discloses a task offloading and collaborative caching method for two-layer mobile ad hoc networks, belonging to the fields of wireless communication and mobile edge computing. It constructs a two-layer mobile ad hoc network system model and establishes a joint optimization problem with the goal of minimizing the average task processing latency. This problem is decomposed into a collaborative decision-making sub-problem and a routing planning sub-problem. The collaborative decision-making sub-problem is solved using a multi-agent deep reinforcement learning framework. For tasks with cache misses, multi-hop routing planning is performed using the shortest path algorithm based on the real-time network topology and cache distribution status. Based on collaborative operation and dynamic optimization using synchronization time slots, each node can dynamically coordinate the decision-making, planning, and data transmission processes within each time slot. This achieves efficient integration and dynamic optimization of computing, communication, and caching resources in a dynamic mobile ad hoc network environment, significantly improving task cache hit rate and reducing end-to-end task processing latency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of wireless communication and mobile edge computing technology, specifically relating to a task offloading and collaborative caching method for two-layer mobile ad hoc networks. Background Technology

[0002] Mobile Ad Hoc Networks (MANETs) are widely used in emergency communications, unmanned systems, and tactical collaboration scenarios due to their ability to quickly deploy without fixed infrastructure. However, traditional single-layer MANETs are limited in scale. As the number of nodes increases, a large amount of routing information needs to be maintained. Furthermore, the highly dynamic network topology leads to a sharp increase in communication and control overhead, making it difficult to support large-scale, highly mobile networking requirements. In contrast, two-layer MANET architectures can assign differentiated functions to different nodes based on their characteristics, effectively improving network scalability and resource utilization. In a two-layer MANET architecture, the lower layer consists of a large number of ordinary nodes responsible for data collection and task generation; the upper layer consists of mobile nodes with stronger computing capabilities responsible for global information aggregation, collaborative computing, and cross-domain communication. Based on this two-layer architecture, lower-layer nodes typically offload tasks to upper-layer nodes, which can reduce program retrieval latency and further improve task processing efficiency by pre-caching relevant programs.

[0003] Existing research generally optimizes the task unloading and program caching processes independently to reduce solution complexity and decouple the computation and caching modules. While this approach simplifies system design, in practice, separate caching and computation nodes are often required, leading to resource redundancy and significantly increasing deployment costs. Furthermore, independent optimization methods struggle to simultaneously balance cache hit rate, computational load balancing, and network communication overhead, making it difficult for the system to achieve globally optimal performance in scenarios involving high-speed movement and frequent topology changes.

[0004] Driven by the aforementioned problems, joint optimization of task offloading and caching has gradually become a research hotspot. The aim is to achieve a more efficient task processing flow by jointly optimizing task offloading and caching decisions to cope with high-mobility, high-load application scenarios. However, with the continuous expansion of network scale and the increasing dynamics of topology, traditional joint optimization methods based on centralized control or heuristic algorithms are gradually showing shortcomings in terms of global state information acquisition, algorithm computational complexity, and real-time decision-making capabilities. To address this challenge, there is an urgent need for a joint optimization method for task offloading and caching in two-layer mobile ad hoc networks, capable of achieving distributed collaborative decision-making in highly dynamic environments, in order to effectively reduce system task processing latency. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a joint optimization method for task offloading and caching in a two-layer mobile ad hoc network. By jointly optimizing task offloading decisions, program caching strategies, routing planning, and the movement trajectories of computing nodes, the network can adapt to dynamically changing environments and minimize the average system latency.

[0006] To solve the above-mentioned technical problems, the technical solution provided by the present invention is as follows: This invention provides a method for task offloading and collaborative caching in a two-layer mobile ad hoc network, wherein the two-layer mobile ad hoc network includes upper-layer mobile computing nodes and lower-layer task generation nodes, and includes the following steps: S1. Construct a two-layer architecture mobile ad hoc network system model including upper-layer mobile computing nodes and lower-layer task generation nodes. With minimizing the average system latency as the optimization objective, establish a mixed-integer nonlinear programming problem that jointly optimizes task offloading decisions, the movement trajectory of upper-layer mobile computing nodes, program caching strategies, and packet forwarding routing strategies. S2. The mixed integer nonlinear programming problem is decomposed into a sequentially solved collaborative decision-making subproblem and a routing planning subproblem; the collaborative decision-making subproblem is used to jointly optimize task unloading decisions, moving computing node trajectories, and program caching strategies; the routing planning subproblem is responsible for planning multi-hop forwarding paths for tasks when cache misses. S3. A multi-agent reinforcement learning framework is used to solve the cooperative decision-making sub-problem. Each lower-level task generation node and each upper-level mobile computing node are modeled as agents. A global cooperative strategy is obtained through centralized training, so that each node can perform task unloading, node movement and program caching decisions in a distributed manner, relying only on local observation information. S4. For tasks that fail to be cached after collaborative decision-making, based on the real-time network topology and cache distribution status, the shortest path algorithm is used through the routing planning subproblem to determine the multi-hop forwarding path from the receiving node to the target computing node. S5. Based on the unified synchronization time slot of the whole network, the nodes perform collaborative operation. In each time slot, each node executes the task offloading decision, node movement, program cache update, and completes data packet forwarding according to the routing planning results, and dynamically coordinates to minimize the average system latency.

[0007] Preferably, in S1, the system model includes a task offloading model, a computing model of upper-layer mobile computing nodes, a data packet routing model, and a program caching model; The system uses orthogonal frequency division multiple access technology for data transmission and pre-allocates non-overlapping fixed frequency bands for each upper-layer mobile computing node. All task generation nodes connected to the same mobile computing node communicate without interference based on the orthogonal sub-channel resources pre-allocated by that node.

[0008] Preferably, in S1, the objective of the optimization problem is to minimize the average system latency; the system latency includes task unloading latency, task forwarding latency, and task computation latency; The average task processing latency of the system includes task unloading latency, task forwarding latency, and task computation latency; The constraints of the mixed-integer nonlinear programming problem include binary variable constraints and single-objective unloading constraints for task unloading decisions, safe distance constraints between upper-layer mobile computing nodes, binary variable constraints for program caching decisions, cache capacity constraints for each node, uniqueness constraints for computing nodes, and flow conservation constraints for multi-hop routing.

[0009] Preferably, in step S1, the task offloading model comprehensively considers task attributes, wireless data transmission rate, and the dynamic movement status of the upper-layer mobile computing node to determine the optimal strategy for offloading the task to the upper-layer mobile computing node, thereby obtaining the task offloading delay. The computational model of the upper-layer mobile computing node is based on the mobile computing node's moving speed and dynamically adjusted moving direction angle, and performs deterministic modeling and planning of its flight trajectory between consecutive time slots; The packet routing model optimizes the multi-hop forwarding route of tasks that need to be forwarded due to program cache misses based on the real-time constructed network topology, the computational load of candidate nodes, and the cumulative congestion of direct links of each node, thereby obtaining the task forwarding latency. The program caching model determines the storage, updating, and retrieval strategies of the upper-layer mobile computing nodes for task execution programs.

[0010] Preferably, in step S3, the collaborative decision-making sub-problem is modeled as a partially observable Markov decision process, defining a local observation state space, a mixed action space, and a reward function for each agent, and employing a multi-agent deep reinforcement learning framework for distributed solution, specifically including: S3.1 Define each task generation node and each mobile computing node as an independent intelligent agent; S3.2 Define the local observation state space, action space, and reward function for each agent; S3.3 The agents are trained in a centralized manner using a multi-agent deep deterministic policy gradient algorithm to obtain a collaborative optimization policy network; S3.4 Deploy the trained policy network to each network node to achieve distributed collaborative decision-making based on local observations.

[0011] Preferably, the definition of the local observation state space in S3.2 is as follows: The state of the task-generating node agent includes the tasks it generates. Attributes, position of this node The positions of all upper-level nodes and historical channel status information Task generation node intelligent agent In the time slot status for: ; The state of the mobile computing node agent includes the positions of all lower-level nodes and the attributes of the generated tasks. Location information of all nodes in the upper layer and the current network topology connections Mobile computing node intelligent agents In the time slot status for: ; The definition of the action space is as follows: Task generation node intelligent agent In the time slot action Probability distribution for unloading decisions , representing the probability of unloading the task to each mobile computing node: ; in: In time slot lower-level nodes Unload the task to the parent node. The probability of; Mobile computing node intelligent agent In the time slot action Cache the probability distribution of decision variables for the program With normalized direction of movement : ; in: The normalized direction of movement; In time slot upper-level nodes caching program The probability of; The reward function includes: Task generation node intelligent agent The reward is the negative of its task unloading latency, and the task-generating node agent... In the time slot Rewards for: ; Mobile computing node intelligent agent It is negatively correlated with the system's average latency and penalizes behaviors that violate cache capacity constraints and inter-node safe distance constraints, enabling mobile computing node agents to... In the time slot Rewards for: ; in: A binary variable indicating a node Does the total number of cached programs exceed its cache capacity? A binary variable indicating a node Does it satisfy the distance constraint? and As punishment.

[0012] Preferably, in step S3.3, the training process of the multi-agent deep deterministic policy gradient algorithm specifically includes: S3.3.1 Configure an actor network and a commentator network for each agent; S3.3.2 Collect experience data through interaction between the agent and the environment and store it in a shared experience replay buffer; S3.3.3 Sample small batches of empirical data from the buffer and update its parameters by minimizing the temporal difference error loss of the commentator network; S3.3.4 Maximize the expected long-term cumulative discount reward by updating the executor network parameters along the policy gradient direction; S3.3.5 Employs a soft update method to smoothly update the target network parameters to stabilize training.

[0013] Preferably, in S4, the route planning based on the shortest path algorithm specifically includes: After the task unloading and program caching strategies are determined, all tasks to be forwarded are sorted in ascending order according to their deadlines to ensure that urgent tasks receive better link resources first, reducing system latency. For each task, among the candidate computing nodes that meet the program caching conditions, the node with the highest comprehensive utility is selected as the computing node. This comprehensive utility takes into account the current computing load of the candidate node and the cumulative congestion of its direct links. Based on the selected computing node, a multi-hop forwarding path from the current node to the computing node is constructed using Dijkstra's shortest path algorithm.

[0014] Preferably, in S5, the unified synchronization time slot structure of the entire network operates, and each time slot is divided into the following four stages: S5.1 Collaborative Decision-Making and Resource Request Phase The lower-level task generation node generates an offloading decision based on its own task attributes, its own location information, and historical data transmission rate through a locally deployed lightweight policy network, and sends a scheduling request containing task and location information to the target upper-level mobile computing node. The upper-layer mobile computing node generates its own movement direction decision and program cache decision through the local policy network based on its local observations. S5.2 Route Calculation Phase For tasks that need to be forwarded due to a cache miss, the initial receiving node of the task is used as the source node for data forwarding. Based on the current network topology, the load of each node, and the cache distribution status, the shortest path algorithm is executed to determine the multi-hop forwarding path of the data packets. S5.3 Action Execution and Data Transmission Phase The lower-level task generation node unloads task data from the target computing node on the pre-allocated channel; The upper-layer mobile computing node adjusts its movement trajectory, updates the program cache content, and forwards data packets based on the routing decisions made during the routing calculation phase; S5.4 Experience Collection and Environmental Status Update Phase Each node collects local experience data based on the execution results of this time slot, updates the global network status synchronously, and then enters the next time slot.

[0015] The present invention also provides a task offloading and collaborative caching system for two-layer mobile ad hoc networks, employing the method described in any of the above-mentioned embodiments, the system comprising: The centralized training and management module, deployed at the ground control station, performs centralized training of all agents, lightweight model processing, and unified generation and distribution of policy network parameters. The distributed policy execution module includes an upper-layer mobile computing node policy execution unit deployed on each mobile computing node, used to run a lightweight policy network based on local observations and generate node mobility control policies and program cache update policies; and a lower-layer task generation node policy execution unit deployed on each task generation node, used to run a lightweight policy network based on local observations and generate task unloading decisions. The program cache management module is deployed on the upper-layer mobile computing node to store a limited number of computing programs. Based on the decisions of the strategy execution module, it dynamically updates the cache content and responds to task requests, performing program matching and invocation. The distributed routing planning module, deployed on the upper-layer mobile computing node, is used to determine the multi-hop transmission path of a task based on the real-time network topology and cache distribution when the task cache is not hit. The wireless communication subsystem includes: a multiple access unit based on orthogonal frequency division multiple access technology, used to support parallel and interference-free data transmission between lower-layer nodes and upper-layer nodes; and a self-organizing network multi-hop communication unit based on mobile ad hoc network technology, used to support the establishment of multi-hop links, exchange of control information and data forwarding between upper-layer mobile computing nodes. The synchronization module, embedded in each network node, is used for time slot synchronization to ensure that all nodes perform operations collaboratively under a unified timing, maintaining global consistency of the system.

[0016] The beneficial effects of this invention are: This invention optimizes task offloading, program caching, node mobility, and routing planning in a joint manner, and achieves collaborative decision-making based on distributed multi-agent systems. This enables the system to significantly reduce task processing latency and effectively improve cache hit rate and resource utilization in dynamic mobile self-organizing network environments. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating a joint optimization method for task offloading and caching in a two-layer mobile ad hoc network according to the present invention. Figure 2 This is a schematic diagram of a two-layer architecture mobile ad hoc network system model provided in an embodiment of the present invention. Figure 3 This is a schematic diagram of the process for solving collaborative decision-making sub-problems based on a multi-agent reinforcement learning framework, as described in this invention. Detailed Implementation

[0018] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings and examples. It should be understood that the embodiments described are only for illustrating the technical solutions of the present invention and should not be considered as limiting the scope of protection of the present invention. Those skilled in the art can modify or substitute the relevant embodiments without departing from the core ideas and spirit of the present invention, and such modifications or substitutions should all fall within the scope of protection defined by the claims of the present invention.

[0019] like Figure 1 As shown, the present invention provides a joint optimization method for task offloading and caching in a two-layer mobile ad hoc network. The specific process is as follows: S1. System Model Construction and Optimization Problem: Construct a two-layer mobile ad hoc network system model, including a task offloading model for lower-layer task generation nodes, a movement and computation model for upper-layer mobile computing nodes, a packet routing model, and a program caching model; with the goal of minimizing the average system latency, establish a mixed-integer nonlinear programming (MINLP) problem that jointly optimizes task offloading decisions, the movement trajectories of upper-layer mobile computing nodes, program caching strategies, and packet forwarding routing strategies; S2. Decompose the original optimization problem into a collaborative decision-making subproblem and a routing planning subproblem: Decompose the mixed-integer nonlinear programming problem in S1 into a sequentially solved collaborative decision-making subproblem and a routing planning subproblem; the collaborative decision-making subproblem is used to jointly optimize task unloading decisions, moving computing node trajectories, and program caching strategies; the routing planning subproblem is responsible for planning multi-hop forwarding paths for tasks when the cache is missed; S3. Solving the cooperative decision-making sub-problem based on a multi-agent reinforcement learning framework: Each lower-level task generation node and each upper-level mobile computing node are modeled as agents, and a global cooperative strategy is obtained through centralized training, so that each node can perform task unloading, node movement and program caching decisions in a distributed manner, relying only on local observation information. S4. Routing planning based on the shortest path algorithm: For tasks that are not cached after the decision in S3, the routing planning sub-problem is based on the real-time network topology and cache distribution status. The shortest path algorithm is used to calculate the multi-hop forwarding path from the receiving node to the target computing node for each task, based on the program cache location, network link status and node load. S5. Cooperative operation and dynamic optimization based on synchronization time slots: The trained policy network is deployed to each network node, and the nodes are driven to make cooperative decisions and execute in units of time slots during system operation; Each node uses local observation data to call the policy network in real time to generate decisions such as task unloading, node movement, and cache update, and completes packet forwarding in conjunction with the routing planning process. As the network topology, task requirements, and link status continue to change, nodes can dynamically adjust their decisions in continuous time slots to minimize the average system latency.

[0020] In S1: The task offloading model comprehensively considers task attributes, wireless data transmission rate, and the dynamic movement status of upper-layer mobile computing nodes to determine the optimal strategy for offloading tasks to upper-layer mobile computing nodes, thereby obtaining the task offloading latency.

[0021] The node's movement model is based on the moving speed and dynamically adjusted moving direction angle of the moving computing node, and deterministically models and plans its flight trajectory between consecutive time slots; The packet routing model optimizes the multi-hop forwarding route of tasks that need to be forwarded due to program cache misses based on the real-time constructed network topology, the computational load of candidate nodes, and the cumulative congestion of direct links of each node, thereby obtaining the task forwarding latency. The program caching model determines the storage, updating, and retrieval strategies of the upper-layer mobile computing nodes for task execution programs.

[0022] The goal of the optimization problem is to minimize the average system latency; system latency includes task unloading latency, task forwarding latency, and task computation latency.

[0023] The system uses orthogonal frequency division multiple access (OFDMA) technology for data transmission and pre-allocates non-overlapping fixed frequency bands for mobile computing nodes. All task generation nodes connected to the same mobile computing node share the pre-allocated orthogonal sub-channel resources of that node for interference-free communication based on the OFDMA mechanism.

[0024] The specific steps for constructing the system model and solving the optimization problem are as follows: S1.1 Construct a two-tier architecture mobile self-organizing network system model.

[0025] The mobile two-layer self-organizing network system consists of upper-layer mobile computing nodes and lower-layer task generation nodes. The mobile computing nodes have edge computing capabilities and limited program cache space; the task generation nodes are responsible for data acquisition and generating corresponding computing tasks.

[0026] Specifically, such as Figure 2 As shown, the system includes Mobile computing nodes and Each task generation node is defined as a set of nodes. and The system time is evenly divided into... There are 1 time slot, and the length of each time slot is 1. The time slot index set is denoted as .

[0027] S1.2 Construct the task unloading model of the lower-level task generation node to obtain the task unloading latency.

[0028] Assume that each task generation node generates at most one task to be computed in a time slot. Since the computational tasks are independent of each other, a binary offloading model is considered, meaning that a task generated by a lower-level node can be offloaded to at most one upper-level node. To indicate the task offloading decision, a binary offloading decision variable is introduced. :when When, it indicates the lower-level node. In the time slot The generated tasks are unloaded to the mobile compute node. ,otherwise Since each task can only select one unloading target per time slot, the constraints that the unloading decision variables must satisfy are: ; This invention addresses two-layer mobile ad hoc networks. To ensure efficient and interference-free data transmission between lower-layer task-generating nodes and upper-layer mobile computing nodes, the system employs Orthogonal Frequency Division Multiple Access (OFDMA) technology as the access method. The communication bandwidth is pre-divided into several mutually orthogonal sub-channels. Each lower-layer node can autonomously choose to offload its computing tasks to any upper-layer mobile computing node and establish a communication connection with the selected upper-layer node. For multiple lower-layer nodes connected to the same upper-layer node, the system allocates pre-set orthogonal sub-channels based on the OFDMA mechanism. That is, several orthogonal sub-channels corresponding to each upper-layer node are fixedly assigned to each lower-layer node during system initialization. Different lower-layer nodes transmit data on independent sub-channels, effectively avoiding co-channel interference by utilizing the orthogonality of the sub-channels, thereby improving the stability and reliability of the link.

[0029] In any time slot The moving speed of the upper-level moving node is fixed at 1. Direction of movement satisfy The horizontal position of the upper-level node. for And its height is fixed as The horizontal position of the lower-level node. for Lower-level nodes and upper-level nodes Horizontal distance between for Mobile computing nodes No. The position of each time slot for: ; Considering that the link between upper and lower layer nodes is mainly affected by line-of-sight propagation, this invention uses a free-space path loss model to model the channel. Lower layer node and upper-level nodes Channel gain between for: ; in: For upper-level nodes The received power.

[0030] In the time slot lower-level nodes and upper-level nodes Transmission rate between for: ; in: In time slot upper-level nodes With lower-level nodes Bandwidth for data transmission; For lower-level nodes In the time slot The transmission power; For lower-level nodes and upper-level nodes The power of Gaussian white noise between them.

[0031] lower-level nodes In the time slot Unload the task to the upper-level node. latency for: ; in: For lower-level nodes The amount of task data.

[0032] S1.3 Construct the program caching model for the upper-layer mobile computing nodes.

[0033] Each unloading computation task needs to be executed by a specific processor; the set of all available processors in the system is as follows: In real-world scenarios, multiple tasks may request the same program; therefore, mobile computing nodes can pre-cache some programs. If a mobile computing node receives a task but its cache does not contain the corresponding processing program, the task can be transmitted to other mobile computing nodes that have the program cached via multi-hop forwarding. This can be done in any time slot. Each lower-level node Generate a computation task This task is based on three attributes. It means that, among them, For computational tasks Required number of CPU cycles; For the task The amount of data; For computational tasks Required procedures Represents computational task Program needs to be called .

[0034] However, mobile computing nodes have limited storage space and cannot cache all programs, so they need to determine their caching strategy. In the time slot The program caching decision is , Represents a node In the time slot caching program ,otherwise The program caching decision should meet the following constraints: ; in: For the program Storage capacity occupied; For mobile computing nodes Cache capacity.

[0035] S1.4 Construct a routing model for the data packets to be forwarded and obtain the routing forwarding delay.

[0036] time slot The upper-level mobile self-organizing network is modeled as a connected graph. ,in For the set of upper-level mobile nodes, This is the set of upper-layer links. If the upper-layer mobile node... Received from lower-level node Generated computational tasks Later, it was discovered that it did not cache the processing tasks. Required procedures At this time, the task The program can be transmitted to other mobile computing nodes that have cached it via multi-hop forwarding. All such tasks constitute a time slot. Set of tasks to be forwarded .

[0037] For any It is necessary to select a candidate upper-level node for the final computation of this task. Define a binary computational decision variable. ,in Indicates task In the time slot By mobile computing nodes Computation. Because candidate compute nodes must cache the programs required to process tasks. Furthermore, each task must be assigned to a unique candidate upper-level node, so the decision variables are calculated. The following constraints must be met: ; Task Once the compute nodes are determined, it is necessary to construct the unloading nodes. To compute node Multi-hop routing paths between them. Define time slots. Link indicator variable for: ; in: Indicates task In the time slot From the unload node To compute node The forwarding path includes links .

[0038] The system pre-allocates non-overlapping fixed frequency bands for upper-layer links, enabling upper-layer nodes to transmit multiple tasks in parallel on different frequency bands. This avoids interference within the same layer and eliminates queuing delays caused by bandwidth contention, ensuring stable data transmission rates. Data transmission rate for: ; in: Link bandwidth; For sending node The transmission power; For nodes To the node Channel power gain; This represents the noise power spectral density.

[0039] Task In the link Transmission delay for: ; Task From the unload node To compute node Total path delay for: ; S1.5 Construct the computing model of the upper-layer mobile computing node and obtain the computing latency.

[0040] Set time slot Domestic demand is driven by mobile computing nodes The total number of tasks calculated is This includes locally unloaded tasks and tasks forwarded from other nodes. Compute nodes use a processor-sharing model to model the allocation of computing resources, i.e., nodes... Total computing resources On its load The tasks are evenly distributed among the tasks. At the computing node computational latency for: ; S1.6 Calculate the average system latency.

[0041] The total system latency consists of three parts: task unloading latency, task forwarding latency, and edge computing latency. Depending on whether a task needs to be forwarded across nodes, it can be divided into the following two scenarios: S1.6.1 If the task Unload to mobile computing node Furthermore, the node has cached the program required for the task. That is, satisfying When, the total task delay for: ; S1.6.2 If node The required program is not cached, i.e. Then the task The task needs to be forwarded to other upper-level nodes for computation, at which point the total task latency is... for: ; Combining the two scenarios above, the task end-to-end delay for: ; in: For indicator functions, when When true, the indicator function has a value of 1; when true, the value of the function is 1. When the value is false, the indicator function has a value of 0.

[0042] The transmission latency of the returned results is negligible because the amount of data in the calculated results is much smaller than that of the original task. (Time slot) The average system latency in for: ; S1.7 Optimization Problem. With the objective of minimizing the average system latency, this invention jointly optimizes the task offloading strategy. The movement trajectory of the upper-level node caching strategy of the program and multi-hop routing strategies The corresponding optimization problem can be described using the following formula: ; Wherein: the optimization objective is to minimize the system average latency; constraints (P1a) and (P1d) are the definition constraints of the offloading decision variable and the program cache decision, respectively; constraint variable (P1b) indicates that each task can select at most one offloading target in a time slot; constraint (P1c) is the safety distance constraint between upper-layer mobile computing nodes. The minimum safe distance that must be maintained between any two upper-layer nodes; constraint (P1e) indicates that the total number of programs cached by each upper-layer node cannot exceed its cache capacity; constraint (P1f) indicates that each task can only be computed by one upper-layer node; constraint (P1g) is the flow conservation constraint for multi-hop packet routing, ensuring that each task can be correctly forwarded to the selected target compute node.

[0043] S2. Decompose the original optimization problem into a collaborative decision-making subproblem and a route planning subproblem, as follows: Since the optimization problem P1 is a mixed-integer non-convex programming problem with strongly coupled optimization variables, directly solving it and obtaining the optimal solution is significantly challenging. Furthermore, the dynamic changes in task computational requirements and the high dynamism of the mobile computing node network topology further increase the complexity of program caching and packet routing strategy design.

[0044] By analyzing the coupling relationships among the optimization variables, this invention decomposes the original joint optimization problem into two sub-problems to be solved sequentially. Specifically, decision variables directly related to task processing are included in sub-problem one (SP1), including task unloading decisions, upper-layer node movement trajectories, and program caching strategy optimization. Solving this sub-problem aims to determine the network topology and task distribution status of the current time slot. Subsequently, sub-problem two (SP2) focuses on the task forwarding process, responsible for planning the transmission path of tasks between multi-hop upper-layer nodes.

[0045] Although the two subproblems are solved sequentially online in each time slot, the temporal coupling between optimization variables can be uniformly characterized by a Markov Decision Process (MDP). This decomposition strategy effectively avoids strong coupling between variables, thereby reducing the overall solution difficulty. Based on this, this invention first designs a framework based on Deep Reinforcement Learning (DRL) to jointly solve task offloading, node trajectories, and program caching strategies. Furthermore, this invention designs an optimization subproblem for multi-hop route planning. Specifically, this route planning subproblem is embedded in the DRL training process to collaboratively promote policy learning, ultimately obtaining a complete solution to the original optimization problem P1.

[0046] S3. Solving collaborative decision-making sub-problems based on a multi-agent reinforcement learning framework, such as... Figure 3 As shown, the details are as follows: This subproblem involves the coupling of continuous and binary variables, with the decision-making processes of upper and lower level nodes influencing each other and jointly affecting the objective function value. Therefore, given the unacceptable complexity of centralized optimization, this invention proposes a multi-agent deep reinforcement learning-based method to effectively solve this subproblem. This method allows agents to share information during the learning process and choose independent policies during the execution phase. This step transforms the first subproblem into a distributed collaborative decision-making problem and solves it using a multi-agent reinforcement learning framework, specifically including the following sub-steps: S3.1 Generate nodes for each task With each mobile computing node Each agent is modeled as an independent intelligent agent. All agents together constitute a multi-agent system, with each agent making decisions based on local observations.

[0047] S3.2 Defines the local observation state space, action space, and reward function for each agent.

[0048] S3.2.1 Define the state space of each agent.

[0049] In each time slot Each agent forms local observations based on locally available information. The definition of the local observation state space includes: For the task generation node intelligent agent Its states include: self-generated tasks Attributes, position of this node The positions of all upper-level nodes and historical channel status information Task generation node intelligent agent In the time slot status for: ; For mobile computing node intelligent agents Its state includes: the position of all lower-level nodes and the attributes of the generated task. Location information of all nodes in the upper layer and the current network topology connections Mobile computing node intelligent agents In the time slot status for: ; S3.2.2 Define the action space for each agent.

[0050] The actions of lower-level nodes are unloading decisions, which are binary integer variables. The actions of upper-level nodes are program cache decisions and movement trajectories, where the program cache decision is also a binary integer variable, and the movement speed of the upper-level nodes... Fixed, therefore the movement trajectory will be fixed. Optimization is transformed into the direction of movement. The optimization is a continuous variable. To effectively handle multi-agent learning problems involving mixed action spaces, this invention maps discrete decision variables to probability distributions, and then samples real actions based on these probability distributions.

[0051] Task generation node intelligent agent In the time slot action for: ; in: In time slot lower-level nodes Unload the task to the parent node. The probability of.

[0052] Mobile computing node intelligent agent In the time slot action for: ; in: The normalized direction of movement; In time slot upper-level nodes caching program The probability of.

[0053] S3.2.3 Design the system reward function.

[0054] Task generation node intelligent agent The reward is the negative of its task unloading latency, and the task-generating node agent... In the time slot Rewards for: ; Mobile computing node agents need to collaboratively select flight trajectories and caching strategies based on the distribution characteristics of lower-level task demands to reduce system latency. Furthermore, this invention also needs to consider penalties for violating cache capacity constraints (P1c) and inter-node safe distance constraints (P1e). Mobile computing node agents In the time slot Rewards for: ; in: A binary variable indicating a node Does the total number of cached programs exceed its cache capacity? A binary variable indicating a node Does it satisfy the distance constraint? and As punishment.

[0055] S3.3 Employs a multi-agent deep deterministic policy gradient algorithm to centrally train the agents in order to obtain a collaboratively optimized policy network.

[0056] S3.3.1 Configure an actor network and a commentator network for each agent; S3.3.2 Collect experience data through interaction between the agent and the environment and store it in a shared experience replay buffer; S3.3.3 Sample small batches of empirical data from the buffer and update its parameters by minimizing the temporal difference error loss of the commentator network; S3.3.4 Maximize the expected long-term cumulative discount reward by updating the executor network parameters along the policy gradient direction; S3.3.5 Employs a soft update method to smoothly update the target network parameters to stabilize training.

[0057] Specifically: This invention employs a multi-agent deep deterministic policy gradient framework for centralized training of the collaborative decision-making sub-problem. For each agent in the network... Construct the corresponding Actor network for each. With Critic Network These are used for policy generation and action value evaluation, respectively, and the network parameters are denoted as follows: and .

[0058] During the training phase, the Actor network outputs continuous actions based on the agents' local observations, while the Critic network centrally evaluates these actions using joint observations and actions from all agents, thus achieving an architecture of centralized training and distributed execution. The training objective is to maximize the long-term cumulative discount reward. ,in This is the cumulative discount factor, which aligns with the objective of minimizing the system's average latency.

[0059] The Critic network updates by minimizing the mean squared error loss: ; in: This represents the target value for the Critic network.

[0060] Critic network target value for: ; in: ; and The target network has the following parameters: and This is used to eliminate the correlation between the original neural network and the target.

[0061] Each agent updates the weights of the Actor network using policy gradients: ; Target Network and The weights are updated using the following formula: ; ; in: The update step size is typically a very small value. This means that in each training step, the target network only absorbs a small portion of the changes in the main network, thus maintaining the stability of its parameters.

[0062] S3.4 The trained policy network is deployed to each network node to achieve distributed collaborative decision-making based on local observations, as follows: After centralized training, the Actor policy network parameters for each agent are distributed to the corresponding nodes in the network, forming a lightweight policy model that can be used in a real-world operating environment. During the execution phase, the Critic network is no longer used. Each node uses only its available local observations, including node location and task attributes, as input to the Actor network to generate continuous actions such as unloading, routing, cache updating, and movement direction.

[0063] S4. Route planning is performed based on the shortest path algorithm, as follows: After the task offloading and program caching strategies are determined, all tasks are first sorted in ascending order according to their deadlines to ensure that urgent tasks receive priority access to better link resources, thereby reducing system latency. Then, for each task, a comprehensive utility function is calculated for each node in the candidate node set. This function comprehensively considers the current computing load of the candidate node and the cumulative congestion level of its directly connected links. Finally, the node with the highest comprehensive utility value is selected as the task's computing node, and a multi-hop forwarding path from the current node to the computing node is constructed based on Dijkstra's shortest path algorithm.

[0064] S5. Cooperative operation and dynamic optimization based on synchronization time slots: The system operates based on a unified synchronization time slot structure across the entire network. Each time slot is divided into the following four stages: S5.1 Collaborative Decision-Making and Resource Request Phase The lower-level task generation node generates an offloading decision based on its own task attributes, its own location information, and historical data transmission rate through a locally deployed lightweight policy network, and sends a scheduling request containing task and location information to the target upper-level mobile computing node. The upper-layer mobile computing node generates its own movement direction decision and program caching decision through the local policy network based on the location information of the upper-layer node and the task and location information received from the lower-layer node.

[0065] S5.2 Route Calculation Phase For tasks that need to be forwarded due to a cache miss, the initial receiving node of the task is used as the source node for data forwarding. Based on the current network topology, the load of each node, and the cache distribution status, the shortest path algorithm is executed to determine the multi-hop forwarding path of the data packets. S5.3 Action Execution and Data Transmission Phase The lower-level task generation node unloads task data from the target computing node on the pre-allocated channel; The upper-layer mobile computing node adjusts its movement trajectory, updates the program cache content, and forwards data packets based on the routing decisions made during the routing calculation phase; S5.4 Experience Collection and Environmental Status Update Phase Each node collects local experience data based on the execution results of this time slot, updates the global network status synchronously, and then enters the next time slot.

[0066] This invention also provides a task offloading and collaborative caching system for two-layer mobile ad hoc networks, comprising the following modules: The centralized training and management module, deployed at the ground control station, performs centralized training of all agents, lightweight model processing, and unified generation and distribution of policy network parameters. The distributed policy execution module includes: The upper-layer mobile computing node policy execution unit is deployed on each mobile computing node and is used to run a lightweight policy network based on local observations, generating node mobility control policies and program cache update policies. The lower-level task generation node policy execution unit is deployed on each task generation node and is used to run a lightweight policy network based on local observations to generate task unloading decisions. The program cache management module is deployed on the upper-layer mobile computing node to store a limited number of computing programs. Based on the decisions of the policy execution module, it dynamically updates the cache content and responds to task requests to perform program matching and invocation. The distributed routing planning module, deployed on the upper-layer mobile computing node, is used to determine the multi-hop transmission path of a task based on the real-time network topology and cache distribution when the task cache is not hit. The wireless communication subsystem includes: Multiple access unit, based on orthogonal frequency division multiple access technology, is used to support parallel and interference-free data transmission between lower-layer nodes and upper-layer nodes; The self-organizing network multi-hop communication unit, based on mobile ad hoc network technology, is used to support the establishment of multi-hop links, exchange of control information, and data forwarding between upper-layer mobile computing nodes; The synchronization module, embedded in each network node, is used for time slot synchronization to ensure that all nodes perform operations collaboratively under a unified timing, maintaining global consistency of the system.

[0067] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural changes made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

[0068] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

Claims

1. A task offloading and cooperative caching method for a two-tier architecture mobile ad hoc network, characterized in that, The two-layer mobile ad hoc network includes upper-layer mobile computing nodes and lower-layer task generation nodes, and includes the following steps: S1. Construct a two-layer architecture mobile ad hoc network system model including upper-layer mobile computing nodes and lower-layer task generation nodes. With minimizing the average system latency as the optimization objective, establish a mixed-integer nonlinear programming problem that jointly optimizes task offloading decisions, the movement trajectory of upper-layer mobile computing nodes, program caching strategies, and packet forwarding routing strategies. S2. The mixed integer nonlinear programming problem is decomposed into a sequentially solved collaborative decision-making subproblem and a routing planning subproblem; the collaborative decision-making subproblem is used to jointly optimize task unloading decisions, moving computing node trajectories, and program caching strategies; the routing planning subproblem is responsible for planning multi-hop forwarding paths for tasks when cache misses. S3. A multi-agent reinforcement learning framework is used to solve the cooperative decision-making sub-problem. Each lower-level task generation node and each upper-level mobile computing node are modeled as agents. A global cooperative strategy is obtained through centralized training, so that each node can perform task unloading, node movement and program caching decisions in a distributed manner, relying only on local observation information. S4. For tasks that fail to be cached after collaborative decision-making, based on the real-time network topology and cache distribution status, the shortest path algorithm is used through the routing planning subproblem to determine the multi-hop forwarding path from the receiving node to the target computing node. S5. Based on the unified synchronization time slot of the whole network, the nodes perform collaborative operation. In each time slot, each node executes the task offloading decision, node movement, program cache update, and completes data packet forwarding according to the routing planning results, and dynamically coordinates to minimize the average system latency.

2. The method according to claim 1, characterized in that, In S1, the system model includes a task offloading model, a computing model for upper-layer mobile computing nodes, a data packet routing model, and a program caching model. The system uses orthogonal frequency division multiple access technology for data transmission and pre-allocates non-overlapping fixed frequency bands for each upper-layer mobile computing node. All task generation nodes connected to the same mobile computing node communicate without interference based on the orthogonal sub-channel resources pre-allocated by that node.

3. The method according to claim 1, characterized in that, In S1, the objective of the optimization problem is to minimize the average system latency; the system latency includes task unloading latency, task forwarding latency, and task computation latency. The average task processing latency of the system includes task unloading latency, task forwarding latency, and task computation latency; The constraints of the mixed-integer nonlinear programming problem include binary variable constraints and single-objective unloading constraints for task unloading decisions, safe distance constraints between upper-layer mobile computing nodes, binary variable constraints for program caching decisions, cache capacity constraints for each node, uniqueness constraints for computing nodes, and flow conservation constraints for multi-hop routing.

4. The method according to claim 2, characterized in that, In S1, the task offloading model comprehensively considers task attributes, wireless data transmission rate, and the dynamic movement status of the upper-layer mobile computing node to determine the optimal strategy for offloading the task to the upper-layer mobile computing node, thereby obtaining the task offloading delay. The computational model of the upper-layer mobile computing node is based on the mobile computing node's moving speed and dynamically adjusted moving direction angle, and performs deterministic modeling and planning of its flight trajectory between consecutive time slots; The packet routing model optimizes the multi-hop forwarding route of tasks that need to be forwarded due to program cache misses based on the real-time constructed network topology, the computational load of candidate nodes, and the cumulative congestion of direct links of each node, thereby obtaining the task forwarding latency. The program caching model determines the storage, updating, and retrieval strategies of the upper-layer mobile computing nodes for task execution programs.

5. The method according to claim 1, characterized in that, In S3, the collaborative decision-making subproblem is modeled as a partially observable Markov decision process. A local observation state space, a mixed action space, and a reward function are defined for each agent. A multi-agent deep reinforcement learning framework is used for distributed solution, specifically including: S3.1 Define each task generation node and each mobile computing node as an independent intelligent agent; S3.2 Define the local observation state space, action space, and reward function for each agent; S3.3 The agents are trained in a centralized manner using a multi-agent deep deterministic policy gradient algorithm to obtain a collaborative optimization policy network; S3.4 Deploy the trained policy network to each network node to achieve distributed collaborative decision-making based on local observations.

6. The method according to claim 5, characterized in that, The definition of the local observation state space in S3.2 is as follows: The state of the task-generating node agent includes the tasks it generates. Attributes, position of this node The positions of all upper-level nodes and historical channel status information Task generation node intelligent agent In the time slot status for: ; The state of the mobile computing node agent includes the positions of all lower-level nodes and the attributes of the generated tasks. Location information of all nodes in the upper layer and the current network topology connections Mobile computing node intelligent agents In the time slot status for: ; The definition of the action space is as follows: Task generation node intelligent agent In the time slot action Probability distribution for unloading decisions , representing the probability of unloading the task to each mobile computing node: ; in: In time slot lower-level nodes Unload the task to the parent node. The probability of; Mobile computing node intelligent agent In the time slot action Cache the probability distribution of decision variables for the program With normalized direction of movement : ; in: The normalized direction of movement; In time slot upper-level nodes caching program The probability of; The reward function includes: Task generation node intelligent agent The reward is the negative of its task unloading latency, and the task-generating node agent... In the time slot Rewards for: ; Mobile computing node intelligent agent It is negatively correlated with the system's average latency and penalizes behaviors that violate cache capacity constraints and inter-node safe distance constraints, enabling mobile computing node agents to... In the time slot Rewards for: ; in: A binary variable indicating a node Does the total number of cached programs exceed its cache capacity? A binary variable indicating a node Does it satisfy the distance constraint? and As punishment.

7. The method according to claim 5, characterized in that, In S3.3, the training process of the multi-agent deep deterministic policy gradient algorithm specifically includes: S3.3.1 Configure an actor network and a commentator network for each agent; S3.3.2 Collect experience data through interaction between the agent and the environment and store it in a shared experience replay buffer; S3.3.3 Sample small batches of empirical data from the buffer and update its parameters by minimizing the temporal difference error loss of the commentator network; S3.3.4 Maximize the expected long-term cumulative discount reward by updating the executor network parameters along the policy gradient direction; S3.3.5 Employs a soft update method to smoothly update the target network parameters to stabilize training.

8. The method according to claim 1, characterized in that, In S4, route planning based on the shortest path algorithm specifically includes: After the task unloading and program caching strategies are determined, all tasks to be forwarded are sorted in ascending order according to their deadlines to ensure that urgent tasks receive better link resources first, reducing system latency. For each task, among the candidate computing nodes that meet the program caching conditions, the node with the highest comprehensive utility is selected as the computing node. This comprehensive utility takes into account the current computing load of the candidate node and the cumulative congestion of its direct links. Based on the selected computing node, a multi-hop forwarding path from the current node to the computing node is constructed using Dijkstra's shortest path algorithm.

9. The method according to claim 1, characterized in that, In S5, the unified synchronization time slot structure across the entire network operates, and each time slot is divided into the following four stages: S5.1 Collaborative Decision-Making and Resource Request Phase The lower-level task generation node generates an offloading decision based on its own task attributes, its own location information, and historical data transmission rate through a locally deployed lightweight policy network, and sends a scheduling request containing task and location information to the target upper-level mobile computing node. The upper-layer mobile computing node generates its own movement direction decision and program cache decision through the local policy network based on its local observations. S5.2 Route Calculation Phase For tasks that need to be forwarded due to a cache miss, the initial receiving node of the task is used as the source node for data forwarding. Based on the current network topology, the load of each node, and the cache distribution status, the shortest path algorithm is executed to determine the multi-hop forwarding path of the data packets. S5.3 Action Execution and Data Transmission Phase The lower-level task generation node unloads task data from the target computing node on the pre-allocated channel; The upper-layer mobile computing node adjusts its movement trajectory, updates the program cache content, and forwards data packets based on the routing decisions made during the routing calculation phase; S5.4 Experience Collection and Environmental Status Update Phase Each node collects local experience data based on the execution results of this time slot, updates the global network status synchronously, and then enters the next time slot.

10. A task offloading and collaborative caching system for a two-layer mobile ad hoc network, employing the method described in any one of claims 1 to 9, characterized in that, The system includes: The centralized training and management module, deployed at the ground control station, performs centralized training of all agents, lightweight model processing, and unified generation and distribution of policy network parameters. The distributed policy execution module includes an upper-layer mobile computing node policy execution unit deployed on each mobile computing node, used to run a lightweight policy network based on local observations and generate node mobility control policies and program cache update policies; and a lower-layer task generation node policy execution unit deployed on each task generation node, used to run a lightweight policy network based on local observations and generate task unloading decisions. The program cache management module is deployed on the upper-layer mobile computing node to store a limited number of computing programs. Based on the decisions of the strategy execution module, it dynamically updates the cache content and responds to task requests, performing program matching and invocation. The distributed routing planning module, deployed on the upper-layer mobile computing node, is used to determine the multi-hop transmission path of a task based on the real-time network topology and cache distribution when the task cache is not hit. The wireless communication subsystem includes: a multiple access unit based on orthogonal frequency division multiple access technology, used to support parallel and interference-free data transmission between lower-layer nodes and upper-layer nodes; and a self-organizing network multi-hop communication unit based on mobile ad hoc network technology, used to support the establishment of multi-hop links, exchange of control information and data forwarding between upper-layer mobile computing nodes. The synchronization module, embedded in each network node, is used for time slot synchronization to ensure that all nodes perform operations collaboratively under a unified timing, maintaining global consistency of the system.