A marine mobile edge computing dynamic offloading method for incidental tasks

Abstract

The application discloses a kind of offshore mobile edge computing dynamic unloading methods for occasional tasks, comprising: obtaining the original image data of task and pre-processing, obtain task data;Based on task data, the system model of offshore maritime mobile edge computing is constructed, including calculation model, communication model and task queue model;Based on system model, the optimization target of minimizing total blocking cost is converted into Markov decision model problem;Based on Markov decision model and task data, ON-PPO decision model is trained based on the introduction of task category orthogonal coding and state normalization;The trained ON-PPO decision model is deployed to edge device;ON-PPO decision model outputs optimal unloading decision according to real-time state, and edge device distributes task to edge local or cloud device according to optimal unloading decision and executes.The application reduces the average delay and blocking probability of occasional target detection task in offshore mobile edge computing environment.

Description

Technical Field

[0001] This invention relates to the field of maritime mobile edge computing technology, and in particular to a dynamic offloading method for maritime mobile edge computing for occasional tasks. Background Technology

[0002] Target detection technology plays a crucial role in scenarios such as maritime search and rescue, ship inspection, and aquaculture. However, high-precision, robust image algorithm models typically employ hyperparameterized architectures, demanding extremely high computational resources. The maritime environment presents challenges such as limited communication bandwidth and insufficient computing power on edge devices, making it difficult to directly deploy large-scale, high-precision models. Furthermore, in the maritime environment, edge devices often suffer from limited computing power, insufficient storage space, and inadequate energy supply. Maritime communication conditions are highly dynamic and unstable, influenced by environmental factors such as weather and waves. Mobile edge computing (MEC) technology, by offloading computational tasks from local devices to edge servers or cloud platforms, provides computational support and resource expansion for resource-constrained devices, proving effective in providing computational resources to resource-limited local environments and improving resource utilization.

[0003] However, due to the complex structure of image algorithm models and the sporadic nature of maritime tasks, existing offloading strategies cannot effectively implement a layered offloading mechanism for image algorithm models. Furthermore, the alternation of sparse and dense task loads significantly impacts resource supply and demand balance. Layered offloading requires precise division of the "edge processing layer" and the "cloud processing layer," and control of inter-layer data transmission to adapt to limited maritime bandwidth. These shortcomings prevent existing methods from achieving efficient layered offloading of image algorithm models in maritime mobile edge computing scenarios and fail to fully consider the arrival characteristics of sporadic tasks. Specifically, this deficiency manifests as follows: First, it exacerbates resource supply and demand imbalances, reducing resource utilization efficiency. The alternation of sparse and dense sporadic tasks leads to extreme fluctuations in system resources, making supply and demand matching difficult. Second, maritime mobile edge computing systems require a three-stage sequential process: "edge computing → data transmission → cloud computing." The random arrival of sporadic tasks disrupts this temporal continuity, causing cumulative congestion and delays, thus increasing overall latency.

[0004] In summary, existing technologies in the Maritime Mobile Edge Computing (MMEC) environment struggle to simultaneously balance the dynamic adaptability of model hierarchical offloading with the resource scheduling flexibility for occasional task characteristics. A novel offloading method is needed that can adaptively adjust the model hierarchical strategy based on task arrival intensity and optimize blocking latency and resource utilization. Summary of the Invention

[0005] In view of this, the purpose of this invention is to propose a dynamic offloading method for sporadic target detection tasks in maritime mobile edge computing. When resource-constrained devices in maritime mobile edge computing (MMEC) are handling sporadic target detection tasks, a dynamic offloading strategy based on near-end policy optimization (PPO) reinforcement learning is formulated, thereby significantly reducing the average latency and blocking probability of sporadic target detection tasks in the maritime mobile edge computing environment, which helps maritime edge devices to handle tasks smoothly.

[0006] To achieve the above-mentioned technical objectives, the technical solution adopted by this invention is as follows:

[0007] This invention provides a dynamic offloading method for maritime mobile edge computing for occasional tasks, comprising the following steps:

[0008] Step 1: Acquire the raw image data for the maritime target detection mission and preprocess it to obtain mission data;

[0009] Step 2: Construct a system model for nearshore maritime mobile edge computing based on the task data, including a computing model, a communication model, and a task queue model, to quantify the latency and total blocking cost of task data in edge computing, data transmission, and cloud computing.

[0010] Step 3: Based on the system model, the optimization objective of minimizing the total blocking cost is transformed into a Markov decision model problem, and the state, action, and reward functions of the Markov decision model are defined.

[0011] Step 4: Based on the Markov decision model and task data, train the ON-PPO decision model that incorporates orthogonal coding of task categories and state normalization to obtain the trained ON-PPO decision model; the specific execution process of the ON-PPO decision model is as follows:

[0012] Step 41: The ON-PPO decision model includes an actor network and a critic network; initialize the parameters of the actor network. Parameters of the critic network ;

[0013] Step 42: For each training round in the training process, perform the following sub-steps:

[0014] a. Current strategy using actor networks Interact with the environment to collect the status of a batch. ,action and reward function sequence;

[0015] b. The current strategy of the actor network The parameters are copied as the old strategy. This is used for subsequent calculations;

[0016] Step 43: Analyze the collected status. Perform normalization processing and adjust the state. The image algorithm model categories contained therein are subject to orthogonal encoding processing corresponding to the task categories.

[0017] Step 44: For each time step in the batch Each time step Corresponding task set One of the tasks Reward function based on data collection and under the current strategy, the first State at each time step Value function estimate Calculate the task k Corresponding advantage function estimate ;

[0018]

[0019] in, For the first Discount factor for each time step, For the first Discount factor for each time step, Indicates the index of the time step and ; Indicates the first [number]th ... State at each time step The estimated value function;

[0020] Step 45: Within each training round, perform M internal update iterations, performing the following operations in each iteration:

[0021] a. For each time step in the sequence, calculate the sampling probability ratio based on the old and new strategies. ;

[0022]

[0023] in, Indicates task k Corresponding to the current strategy of the actor network, Indicates task k This corresponds to the old strategy of actor networks;

[0024] b. Based on the sampling probability ratio With advantage function estimate The policy loss function of the actor network is calculated using the pruning function. ;

[0025]

[0026]

[0027] in, This represents the clipping function. The preset cropping threshold is set between 0 and 1.

[0028] c. Use the calculated policy loss function Through about The Adam optimizer updates the parameters of the actor network. ;

[0029] d. Reward function based on data collection Calculate the value loss function of the critic network. and use the calculated value loss function The parameters φ of the critic network are updated using the Adam optimizer with respect to φ.

[0030] Value loss function The formula is:

[0031]

[0032] in, For the first The reward function for each time step; Indicates from the first Cumulative discount rewards after a certain time step This indicates that the critics' network is in its current phase under the current strategy. State at each time step The value function estimate is given by the parameter . The network of critics calculated the data used to assess the situation. state The corresponding long-term cumulative reward expectation;

[0033] Step 46: After iteration, the trained ON-PPO decision model, based on the input state and the task... Output the optimal unloading action. That is, determining the set of image algorithm model layers to be executed on the edge device. and the collection of image algorithm model layers executed on cloud devices ;

[0034] Step 5: Deploy the trained ON-PPO decision model to edge devices;

[0035] Step 6: The ON-PPO decision model outputs the optimal offloading decision based on the real-time status. The edge device allocates tasks to the edge local or cloud devices and executes them based on the output optimal offloading decision.

[0036] Furthermore, step 1 specifically includes:

[0037] Step 11: Acquire raw image data for the maritime target detection mission using an acquisition device, which includes a near-shore fixed camera, an unmanned aerial vehicle (UAV)-borne high-definition camera, and / or an image sensor mounted on an unmanned surface vessel.

[0038] Step 12, the task set generated during equipment operation is represented as follows: , Indicates the number of tasks. Represents the index of the task in the task set and Image algorithm model category set , Indicates the number of image algorithm model categories. Represents the index of the image algorithm model category in the set of image algorithm model categories and Based on each task in the task set The task category determines the corresponding image algorithm model category within the set of image algorithm model categories; one task category corresponds to one image algorithm model category. All were classified as Layer, represented as , Indicates the number of layers in the image algorithm model. Indicates the index of the image algorithm model layer. This represents the set of image algorithm model layers;

[0039] Step 13: Perform convolution operation on the original image data using a Gaussian filtering algorithm to remove noise, wherein the convolution kernel size is 3×3, and accelerate the noise reduction process using an FPGA;

[0040] Step 14: Identify the target region in the denoised image data based on the Canny edge detection algorithm, and crop the background region within the target region;

[0041] Step 15: Convert the cropped image data into a preset unified format to obtain task data containing image algorithm model category, image size, and preliminary target contour information;

[0042] Step 16: Output the task data to the edge processing layer as input to the computing model, task queue model, and ON-PPO decision model.

[0043] Furthermore, the computational model constructed in step 2 is used to calculate the computational latency of the task on the edge device and the computational latency on the cloud device, specifically quantifying the computational latency in the following ways:

[0044] (1) Regarding the task k Its computational latency on edge devices Represented as:

[0045]

[0046] in, Indicates the task performed on the edge device. k In the category of image algorithm models The image algorithm model layer set in the middle, Represents the category of image algorithm model The Middle n The number of floating-point operations required for the layer Indicates assignment to task k The computing power of edge devices, measured in floating-point operations per second; Indicates task k The sum of the computational power of all image algorithm model layers on the edge device;

[0047] (2) Regarding the task k Its computing latency on cloud devices Represented as:

[0048]

[0049] in, Indicates the task performed on a cloud device. k In the category of image algorithm models The image algorithm model layer set in the middle, Indicates assignment to task k The computing power of cloud devices Indicates task k The sum of the computing power of all image algorithm model layers on the cloud device.

[0050] Furthermore, the communication model constructed in step 2 is used to calculate the transmission rate of the task from the edge device to the base station and the transmission time of the intermediate data of the task from the edge device to the base station. Specifically, the transmission latency is quantified in the following ways:

[0051] (1) Task k Transmission rate from edge device to base station Represented as:

[0052]

[0053] Where B is the transmission bandwidth. For the transmission power of edge devices, For the task k Corresponding to the large-scale attenuation coefficient of maritime communication, Noise power;

[0054] (2) Edge devices will perform tasks k Transmission time of intermediate data sent to the base station Represented as:

[0055]

[0056] in, Image algorithm model category The Middle n The amount of data output from the layer.

[0057] Furthermore, the task queue model constructed in step 2 is used to calculate the blocking latency caused by resource contention in the three stages of edge computing, data transmission, and cloud computing. Specifically, the blocking latency is quantified in the following ways:

[0058] (1) The processing of each task is divided into three sequential stages: edge computing, data transmission and cloud computing;

[0059] (2) For the edge computing stage, the computing task k Blocking time on edge devices :

[0060]

[0061]

[0062]

[0063] in, The end time for executing the previous task on the edge device. This indicates that the task is performed on an edge device. k End time, To perform tasks on edge devices k Arrival time, This indicates that the task is performed on an edge device. k The start time; Indicates task k Computation latency on edge devices;

[0064] (3) For the data transmission stage, the computing task k Blocking time on edge devices :

[0065]

[0066]

[0067]

[0068] in, It refers to the end time of the previous task executed on the edge device. It executes tasks on edge devices. k End time, Performing tasks on edge devices k The start time; This indicates that the edge device will perform the task. k The transmission time of intermediate data sent to the base station;

[0069] (4) For the cloud computing stage, the blocking time of computing task k on the edge device :

[0070]

[0071]

[0072]

[0073] in, It refers to the end time of the previous task executed on the cloud device. It is to perform tasks on cloud devices. k End time, Performing tasks on cloud devices k The start time, Indicates task k Computing latency on cloud devices.

[0074] Furthermore, in step 2, quantifying the total blocking cost specifically involves integrating the computational model, communication model, and task queue model to form a system model for quantifying the total blocking cost, and taking minimizing the total blocking cost as the optimization objective. The system model quantifies the total blocking cost in the following ways:

[0075] (1) The total blocking time is used as the total blocking cost for the task. k Total blocking cost Represented as:

[0076]

[0077] in, This is a weighting coefficient for the blocking latency in the edge computing phase. This is a weighting coefficient for the blocking delay during the data transmission phase. This is a weighting coefficient for blocking latency during the cloud computing phase. This indicates the task in the edge computing phase. k Blocking time on edge devices, This indicates the task during the data transmission phase. k Blocking time on edge devices, This represents the blocking time of task k on the edge device during the cloud computing phase;

[0078] (2) Minimizing the total blocking cost is taken as the optimization objective, and the optimization objective formula is:

[0079]

[0080] Furthermore, the optimization objective must satisfy the following constraints:

[0081] Constraint 1: Edge devices are assigned to tasks k computing power The maximum computing power of the edge device must not be exceeded. ,Right now:

[0082]

[0083] Constraint 2: Cloud devices are assigned to tasks k computing power It must not exceed the maximum computing power of the cloud device. ,Right now:

[0084]

[0085] Constraint 3: The image algorithm model category to which all tasks to be processed belong. All belong to the predefined set L of image algorithm model categories, that is:

[0086]

[0087] Constraint 4: For any image algorithm model category Task k The collection of image algorithm model layers that execute on edge devices Compared to the collection of image algorithm model layers executed on cloud devices The union of the values ​​is equal to the category of the image algorithm model. Complete image algorithm model layer set ,Right now:

[0088]

[0089]

[0090]

[0091] Constraint 5: For the task k The collection of image algorithm model layers that execute on edge devices Compared to the collection of image algorithm model layers executed on cloud devices The intersection of the two sets is the empty set. ,express and There is no overlap between them, that is:

[0092]

[0093] Constraint 6: The sum of the weighting coefficients for blocking delay in each stage is 1, that is: .

[0094] Furthermore, the state, action, and reward functions of the Markov decision model in step 3 are defined as follows:

[0095] (1) Task k status Defined as:

[0096]

[0097]

[0098] in, This indicates that the edge device is assigned to the task. k computing power This indicates that the cloud device assigns the task. k computing power Indicates task k Transmission rate from edge devices to base stations; Image algorithm model category The set of computational requirements for all image algorithm model layers in the dataset. Represents the category of image algorithm model The computational requirements of the first layer in the middle, Represents the category of image algorithm model The Middle n The computational requirements of the layer Represents the category of image algorithm model The Middle N The computational requirements of the layer; Image algorithm model category The set of data outputs from all layers of the image algorithm model. Represents the category of image algorithm model The data output of the first layer in the middle. Represents the category of image algorithm model The Middle n The data output volume of the layer Represents the category of image algorithm model The Middle N Layer data output volume;

[0099] (2) Handling tasks k action Defined as:

[0100] (3) In state The reward function below Defined as: .

[0101] Furthermore, in step 43, the state... The image algorithm model categories included in the dataset undergo orthogonal coding processing corresponding to the task categories. Specifically, each task category is mapped to a binary vector of length equal to the total number of task categories using one-hot coding. This vector has a 1 in the l-th position and 0 in the remaining positions. In step 43, the collected states... Perform normalization processing, specifically: use min-max normalization to normalize the states. Scaling to :

[0102] in, It is the minimum value of the state characteristic. = - Scaling factor This represents the maximum value of the state characteristic.

[0103] Furthermore, step 6 specifically includes:

[0104] Step 61: Collect the status in real time and input it into the ON-PPO decision model, and output the unloading action of the current task;

[0105] Step 62: The edge device distributes the task data to the local edge device or the cloud device for execution based on the unloading action;

[0106] Step 63: Real-time statistics of the blocking time data of each task in each stage of edge computing, data transmission and cloud computing, and feed this blocking time data into the ON-PPO decision model as a new training sample to form a closed-loop optimization.

[0107] By adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art:

[0108] (1) Dynamic adaptability and long-term optimization capability:

[0109] Compared to static offloading strategies or short-term optimization methods, this invention models the problem as a Markov Decision Process (MDP) and utilizes reinforcement learning (ON-PPO decision model) to achieve long-term policy optimization. This enables the system to respond in real-time to network fluctuations, task load changes, and dynamic resource allocation, significantly reducing the probability of task blocking and average latency. The ON-PPO decision model learns the optimal offloading strategy through a policy gradient method, avoiding dependence on the environment and making it suitable for dynamic maritime network environments.

[0110] (2) Full utilization of the sparsity characteristics of the task:

[0111] Most existing studies neglect the sparsity of task arrivals. This invention, however, constructs a system model integrating computation, communication, and task queue models, dynamically adjusting resource allocation based on task interval characteristics. When tasks are sparse, it increases the use of local computing resources to alleviate cloud pressure; when tasks are intensive, it formulates refined offloading strategies to reduce the risk of system blockage. This method can maintain a stable task queue and low latency under different task arrival intervals.

[0112] (3) Model layered unloading and transmission optimization:

[0113] To address the sequentiality and separability of image algorithm models, this invention employs a vertical layered offloading strategy to reduce the number of data transfers between layers. Compared to complete offloading or static layering methods, this method significantly reduces transmission overhead and bandwidth dependence. By selecting the optimal offloading point (e.g., layer n as the splitting point), the computational load at the edge and in the cloud is balanced, minimizing the total processing time. Attached Figure Description

[0114] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0115] Figure 1 This is an execution flowchart of a dynamic offloading method for maritime mobile edge computing oriented to occasional tasks provided by an embodiment of the present invention.

[0116] Figure 2 This is a schematic diagram of the task queue status provided in an embodiment of the present invention.

[0117] Figure 3This is the workflow of the unloading algorithm based on the ON-PPO decision model provided in this embodiment of the invention. Detailed Implementation

[0118] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be particularly noted that the following embodiments are for illustrative purposes only and do not limit the scope of the invention. Similarly, the following embodiments are only some, not all, embodiments of the present invention, and all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0119] Please see Figure 1 The present invention provides a dynamic offloading method for maritime mobile edge computing for occasional tasks, comprising the following steps:

[0120] Step 1, Data Preparation: Acquire the raw image data of the maritime target detection mission and preprocess it to obtain mission data; this reduces the subsequent computation and transmission load and provides suitable data for hierarchical model calculation.

[0121] In this embodiment, step 1 specifically includes:

[0122] Step 11: Acquire raw image data for the maritime target detection mission using an acquisition device, which includes a near-shore fixed camera, an UAV-borne high-definition camera, and / or an image sensor mounted on an unmanned surface vessel (USV) (adapted to strong light, fog, and turbulent environments at sea, and employing a high dynamic range sensor).

[0123] Step 12, the task set generated during equipment operation is represented as follows: , Indicates the number of tasks. Represents the index of the task in the task set and Image algorithm model category set , Indicates the number of image algorithm model categories. Represents the index of the image algorithm model category in the set of image algorithm model categories and Based on each task in the task set The task category determines the corresponding image algorithm model category within the set of image algorithm model categories. One task category corresponds to one image algorithm model category. Task categories include maritime search and rescue target detection, ship detection, and aquaculture organism detection, among others. All were classified as Layer, represented as , Indicates the number of layers in the image algorithm model. Indicates the index of the image algorithm model layer. This represents the set of image algorithm model layers;

[0124] Step 13: Perform convolution operation on the original image data using a Gaussian filtering algorithm to remove noise. The convolution kernel size is 3×3, and the noise reduction process is accelerated by using an FPGA (Field Programmable Gate Array).

[0125] Step 14: Identify target regions (such as ship outlines or people who have fallen into the water) in the denoised image data based on the Canny (edge ​​detector) edge detection algorithm, and crop the background regions (such as open sea surfaces) in the target regions to reduce the amount of data for subsequent model calculations;

[0126] Step 15: Convert the cropped image data into a preset unified format (e.g., JPEG format) to ensure that the edges are consistent with the cloud model input format, and obtain task data containing image algorithm model category, image size and preliminary target contour information;

[0127] Step 16: Output the task data to the edge processing layer as input to the computing model, task queue model, and ON-PPO decision model.

[0128] Step 2, Model Building: Based on the task data, and according to the task characteristics and marine environmental constraints, a system model for near-shore maritime mobile edge computing is constructed, including a computing model, a communication model, and a task queue model, to quantify the latency and total blocking cost of task data in edge computing, data transmission, and cloud computing; providing parameter basis for subsequent MDP modeling.

[0129] In this embodiment, the computational model constructed in step 2 is used to calculate the computational latency of the task on the edge device and the computational latency on the cloud device. Specifically, the computational latency is quantified in the following way:

[0130] (1) In the task During the computation process, the different layers of the corresponding model are independent of each other and are executed sequentially. For the task... k Its computational latency on edge devices Represented as:

[0131]

[0132] in, Indicates the task performed on the edge device. k In the category of image algorithm models The image algorithm model layer set in the middle, Represents the category of image algorithm model The Middle n The number of floating-point operations (FLOPs) required for the layer. Indicates assignment to taskk The computing power of edge devices, measured in floating-point operations per second; Indicates task k The sum of the computing power of all image algorithm model layers on the edge device; among them, the image algorithm model layer is the core carrier of the layered offloading strategy, the modular component of image algorithm models (such as YOLOv5s, YOLOv5s-MobileNetv3), each layer undertakes specific computing functions (such as feature extraction, target candidate box generation, classification regression), has independent floating-point operation requirements (FLOPs) and data output volume, and is the split object of edge and cloud collaborative computing.

[0133] (2) Regarding the task k Its computing latency on cloud devices Represented as:

[0134]

[0135] in, Indicates the task performed on a cloud device. k In the category of image algorithm models The image algorithm model layer set in the middle, Indicates assignment to task k The computing power of cloud devices Indicates task k The sum of the computing power of all image algorithm model layers on the cloud device.

[0136] In this embodiment, the communication model constructed in step 2 is used to calculate the transmission rate of the task from the edge device to the base station and the transmission time of the intermediate data of the task from the edge device to the base station. Specifically, the transmission latency is quantified in the following way:

[0137] (1) Considering that the amount of data in the detection results is negligible compared to the sample data, this invention ignores wired transmission and downlink delay, and only considers the uplink transmission process from the edge device to the base station. k Transmission rate from edge device to base station Represented as:

[0138]

[0139] Where B is the transmission bandwidth. For the transmission power of edge devices, For the task k Corresponding to the large-scale attenuation coefficient of maritime communication, Noise power;

[0140] (2) Edge devices will perform tasks kTransmission time of intermediate data sent to the base station Represented as:

[0141]

[0142] in, Image algorithm model category l The Middle n The amount of data output from the layer.

[0143] In this embodiment, the task queue model constructed in step 2 is used to calculate the blocking latency caused by resource contention in the three stages of edge computing, data transmission, and cloud computing. Specifically, the blocking latency is quantified in the following ways:

[0144] Due to the sequential nature of image algorithm models, we divide the task into three stages: edge computing, data transmission, and cloud device processing. Each stage of the device can only process one task at a time. If a new task arrives while a stage is still processing the previous task, a blockage will occur, such as... Figure 2 As shown.

[0145] (1) The processing of each task is divided into three sequential stages: edge computing, data transmission and cloud computing;

[0146] (2) For the edge computing stage, the computing task k Blocking time on edge devices :

[0147]

[0148]

[0149]

[0150] in, The end time for executing the previous task on the edge device. This indicates that the task is performed on an edge device. k The end time (the time when the task is completed). To perform tasks on edge devices k The arrival time (the time when the task was initiated). This indicates that the task is performed on an edge device. k Start time (the time when the task begins to be executed);

[0151] (3) For the data transmission stage, the computing task k Blocking time on edge devices :

[0152]

[0153]

[0154]

[0155] in, It refers to the end time of the previous task executed on the edge device. It executes tasks on edge devices. k End time, Performing tasks on edge devices k The start time;

[0156] (4) For the cloud computing stage, the blocking time of computing task k on the edge device :

[0157]

[0158]

[0159]

[0160] in, It refers to the end time of the previous task executed on the cloud device. It is to perform tasks on cloud devices. k End time, Performing tasks on cloud devices k The start time.

[0161] By recording the time-related information and operating status of each device, the impact of different tasks, segmentation methods, and environmental conditions on the total system latency in near-shore image sensing dynamic IoT scenarios can be further analyzed. In the formula, the blocking time represents the maximum value of the end time minus the start time, and the arrival time of edge computing is... The transmission phase uses the end time of edge computing as the start time, and so on.

[0162] In this embodiment, quantifying the total blocking cost in step 2 specifically involves integrating the computation model, communication model, and task queue model to form a system model for quantifying the total blocking cost, and taking minimizing the total blocking cost as the optimization objective. The system model quantifies the total blocking cost in the following ways:

[0163] (1) The objective of the solution in this invention is to identify the optimal partitioning points in the image algorithm model and offload specific image algorithm model layers to suitable cloud devices to optimize system latency. Since system blocking prolongs user waiting time and reduces task service quality, the total blocking time is used as the total blocking cost. kTotal blocking cost Represented as:

[0164]

[0165] in, This is a weighting coefficient for the blocking latency in the edge computing phase. This is a weighting coefficient for the blocking delay during the data transmission phase. These are the weighting coefficients for blocking latency in the cloud computing phase; adjusting the values ​​of these weighting coefficients can further adapt to the different needs of tasks in different environments.

[0166] (2) Considering that different task partitioning will change the computational load between edge computing and cloud computing, and that each layer produces different outputs, the task partitioning point is adjusted to affect system blocking. Therefore, minimizing the total blocking cost is taken as the optimization objective, and the optimization objective formula is:

[0167]

[0168] Furthermore, the optimization objective must satisfy the following constraints:

[0169] Constraint 1: Edge devices are assigned to tasks k computing power The maximum computing power of the edge device must not be exceeded. ,Right now:

[0170]

[0171] Constraint 2: Cloud devices are assigned to tasks k computing power It must not exceed the maximum computing power of the cloud device. ,Right now:

[0172]

[0173] Constraint 3: The image algorithm model category to which all tasks to be processed belong. All belong to the predefined set L of image algorithm model categories, that is:

[0174]

[0175] Constraint 4: For any image algorithm model category Task k The collection of image algorithm model layers that execute on edge devices Compared to the collection of image algorithm model layers executed on cloud devices The union of the values ​​is equal to the category of the image algorithm model. Complete image algorithm model layer set ,Right now:

[0176]

[0177]

[0178]

[0179] Constraint 5: For the task k The collection of image algorithm model layers that execute on edge devices Compared to the collection of image algorithm model layers executed on cloud devices The intersection of the two sets is the empty set. ,express and There is no overlap between them, that is:

[0180]

[0181] Constraint 6: The sum of the weighting coefficients for blocking delay in each stage is 1, that is: .

[0182] Step 3, Model Transformation: Each device is regarded as an intelligent agent. The problem of task partitioning and resource allocation is further represented as a multi-agent Markov decision process (MAMDP), and a Markov decision model (MDP) is constructed. Based on the system model, the optimization objective of minimizing the total blocking cost is transformed into a Markov decision model problem, and the state, action and reward functions of the Markov decision model are defined to establish the mapping relationship between decision and environmental feedback.

[0183] The total cost of the system is determined by the current task state and actions, which trigger the environment and lead to new states. In this case, the adaptive offloading problem is modeled as an MDP. In this MDP, offloading decisions for edge devices are adaptively determined based on the dynamic system environment state to minimize long-term blocking latency of the system. The MDP uses tuples... It means that, among them, For state space, For the action space, The state transition probability matrix is... For the reward function, This is the discount factor.

[0184] In this embodiment, the state, action, and reward function of the Markov decision model in step 3 are defined as follows:

[0185] (1) As edge devices continuously receive new tasks, the system's blocking latency dynamically changes with the task progress through decision-making. To reduce task blocking time, the task... The attribute is defined as: task k Transmission rate from edge device to base station Edge devices are assigned to tasks k computing power Cloud devices are assigned to tasks k computing power Image algorithm model categories The Middle n Computational requirements of the layer and image algorithm model categories The Middle n Layer data output , and Therefore, the task k status Defined as:

[0186]

[0187]

[0188] in, Image algorithm model category The set of computational requirements for all image algorithm model layers in the dataset. Represents the category of image algorithm model The computational requirements of the first layer in the middle, Represents the category of image algorithm model The Middle n The computational requirements of the layer Represents the category of image algorithm model The Middle N The computational requirements of the layer; Image algorithm model category The set of data outputs from all layers of the image algorithm model. Represents the category of image algorithm model The data output of the first layer in the middle. Represents the category of image algorithm model The Middle n The data output volume of the layer Represents the category of image algorithm model The Middle N Layer data output volume;

[0189] (2) The equipment affects the environment through its actions. This invention is based on the task. The state determines the set of image algorithm model layers on the edge device. Image algorithm model layer collection on cloud devices Therefore, processing tasks k action Defined as:

[0190] (3) In the MDP framework, rewards Indicates the state The system gain obtained under each new task Upon arrival, through analysis of Task 1 to Use state information to calculate tasks The blocking time. To minimize this blocking time, the state... The reward is defined as the negative of the blocking time. Therefore, in state... The reward function below Defined as: .

[0191] Step 4, Model Training: Based on the Markov decision model and task data, train the ON-PPO (improved (normalized-orthogonal coding) online proximal policy optimization algorithm) decision model that incorporates task category orthogonal coding and state normalization to obtain the trained ON-PPO decision model; the task category orthogonal coding is only implemented between different categories of tasks;

[0192] In this embodiment, as Figure 3 As shown, the ON-PPO decision model workflow involves the interaction of actor and critic networks with the environment to determine the optimal computational offloading strategy. Online training is used instead of a memory pool to save memory. During algorithm implementation, two policy networks are maintained: the current policy network and the network itself. Continuously improved, old strategies Used for sample collection but not yet updated. Then, based on the collected samples, the estimated task is calculated. Corresponding advantage function Finally, the network parameters are updated by maximizing the objective function, thereby optimizing the strategy. , This indicates the state during a task execution. The corresponding actions taken .

[0193] The specific execution process of the ON-PPO decision model in step 4 is as follows:

[0194] Step 41: The ON-PPO decision model includes an actor network and a critic network; initialize the parameters of the actor network. Parameters of the critic network ;

[0195] Step 42: For each training round in the training process, perform the following sub-steps:

[0196] a. Current strategy using actor networks Interact with the environment to collect the status of a batch. ,action and reward function sequence;

[0197] b. The current strategy of the actor network The parameters are copied as the old strategy. This is used for subsequent calculations;

[0198] Step 43: Analyze the collected status. Perform normalization processing and adjust the state. The image algorithm model categories contained therein are subject to orthogonal encoding processing corresponding to the task categories.

[0199] Step 44: For each time step in the batch Each time step Corresponding task set One of the tasks Reward function based on data collection and under the current strategy, the first State at each time step Value function estimate Calculate the task k Corresponding advantage function estimate ;

[0200]

[0201] in, For the first Discount factor for each time step, For the first Discount factor for each time step, Indicates the index of the time step and Indicates the first [number]th ... State at each time step The value function estimate; the advantage function estimate. By using only the state as input, this method avoids the over-parameterization problem that may arise due to differences in action dimensions.

[0202] Step 45: Within each training round, perform M internal update iterations, performing the following operations in each iteration:

[0203] a. For each time step in the sequence, calculate the sampling probability ratio based on the old and new strategies. ;

[0204]

[0205] in, Indicates task k Corresponding to the current strategy of the actor network, Indicates task k This corresponds to the old strategy of actor networks;

[0206] b. Based on the sampling probability ratio With advantage function estimate The policy loss function of the actor network is calculated using the pruning function. ;

[0207]

[0208]

[0209] in, This represents the clipping function. The preset cropping threshold is set between 0 and 1.

[0210] c. Use the calculated policy loss function Through about The Adam optimizer (Adaptive Moment Estimation, a commonly used first-order gradient optimization algorithm in deep learning) updates the parameters θ of the actor network;

[0211] d. Reward function based on data collection Calculate the value loss function of the critic network. and use the calculated value loss function The parameters φ of the critic network are updated using the Adam optimizer with respect to φ.

[0212] Value loss function The formula is:

[0213]

[0214] in, For the first The reward function for each time step; Indicates from the first Cumulative discount rewards after a certain time step This indicates that the critics' network is in its current phase under the current strategy. State at each time step The value function estimate is given by the parameter . The network of critics calculated the data used to assess the situation. state The corresponding long-term cumulative reward expectation;

[0215] Step 46: After iteration, the trained ON-PPO decision model, based on the input state and the task... Output the optimal unloading action. That is, determining the set of image algorithm model layers to be executed on the edge device. and the collection of image algorithm model layers executed on cloud devices .

[0216] When edge devices handle different tasks, using different numerical values ​​to represent different task categories can cause interference during network weight updates. Orthogonal encoding addresses this issue by introducing orthogonality between category representations, effectively decoupling them and reducing interference during training. Although this orthogonality is achieved through linear transformation, it allows the network to more easily separate internal states associated with different tasks, ultimately enhancing its policy representation capabilities.

[0217] In this embodiment, the state in step 43 is... The image algorithm model categories included in the model are subjected to orthogonal encoding processing corresponding to their corresponding task categories. Specifically, one-hot encoding is used to map each task category to a binary vector of length equal to the total number of task categories. This vector has a 1 only in the l-th position and 0 in the remaining positions. In this paper, given the relatively small number of maritime task categories, the increase in input dimensionality caused by one-hot encoding is negligible. Therefore, this encoding method is used to orthogonalize the task category states. For example, if the total number of task categories is... It is represented as a vector [0,0,0,1]. This method helps to mitigate interference between different task categories.

[0218] In a MIoT environment, the task states input to the ON-PPO algorithm have different dimensions. If the dimensionality difference is too large, features with larger dimensions may dominate the update direction, ignoring state features with smaller values ​​and causing training instability. To accelerate the convergence speed of the algorithm and enhance the generalization ability of the model, this invention adds a feature normalization step before training. In this embodiment, step 43 involves normalizing the collected states... Perform normalization processing, specifically: use min-max normalization to normalize the states. Scaling to :

[0219] in, It is the minimum value of the state characteristic. = - Scaling factor This represents the maximum value of the state characteristic.

[0220] Complexity analysis:

[0221] During the training phase, operations such as state normalization, orthogonal encoding of task category states, and hyperparameter configuration have constant complexity. The training phase is executed using a double loop. The outer loop iterates through the set... Iterate, the inner loop... Each element in the process is executed The complexity of these iterations depends primarily on the network architecture and the size of the data being processed. Assume... This indicates the number of parameters in the original network. Let represent the number of network layers. Then, the computational complexity of a single iteration can be approximated as . Updating network weights is a constant-time operation, denoted as . Therefore, the total computational complexity of training can be expressed as: .

[0222] Step 5, Model Deployment: Deploy the trained ON-PPO decision model to edge devices; specifically including:

[0223] Based on dynamic environmental information, the trained ON-PPO decision model is deployed to edge devices and cloud devices at predetermined time intervals. During the deployment process, it is necessary to adapt to the characteristics of limited edge computing resources and dynamic communication changes at sea.

[0224] Step 6, Decision Implementation: The ON-PPO decision model outputs the optimal offloading decision based on the real-time status. The edge device allocates tasks to the edge local or cloud devices and executes them based on the output optimal offloading decision.

[0225] In this embodiment, step 6 specifically includes:

[0226] Step 61: Collect the status in real time and input it into the ON-PPO decision model, and output the unloading action of the current task;

[0227] Step 62: The edge device distributes the task data to the local edge device or the cloud device for execution based on the unloading action;

[0228] Step 63: Real-time statistics of the blocking time data of each task in each stage of edge computing, data transmission and cloud computing, and feed this blocking time data into the ON-PPO decision model as a new training sample to form a closed-loop optimization.

[0229] The advantages of this invention are as follows:

[0230] This invention develops a maritime monitoring scenario implemented on resource-constrained edge devices, specifically addressing the offloading problem of sporadic target detection tasks. (1) A target detection task-driven scheduling and collaboration strategy for MMEC is proposed. This strategy considers for the first time the hierarchical structure of the image algorithm model and the sporadic nature of maritime observation tasks, thereby achieving fine-grained and dynamic computational offloading of edge-cloud collaboration while ensuring monitoring accuracy. (2) Based on the structure of the image algorithm model and the sporadic nature of the tasks, a system model integrating a computation model, a communication model, and a task queue model is constructed. Minimizing the total blocking delay of the task queue is taken as the optimization objective. Furthermore, an improved near-end policy optimization algorithm (PPO) is designed to achieve rapid convergence and long-term optimality of offloading decisions in dynamic maritime environments.

[0231] Specifically, in the MMEC offloading scenario, this invention proposes a scheduling and offloading strategy for the target monitoring task-driven MMEC scenario, addressing the sporadic nature of maritime observation tasks and the resource constraints of edge devices in the maritime IoT. First, considering the hierarchical nature of the image algorithm model and the sporadic nature of maritime observation tasks, a target monitoring task-driven MMEC scenario scheduling and offloading strategy is adopted. By vertically partitioning the image algorithm model, the number of transmissions between layers is reduced, making it suitable for limited maritime communication resources. This strategy rationally allocates the load among local devices, the cloud platform, and task transmission by combining the intermittent characteristics of the task and the structure of the image algorithm model. Second, based on the structure of the image algorithm model and the sporadic nature of the task, a system model integrating a computational model, a communication model, and a task queue model is constructed, defining the optimization objective as minimizing the total blocking latency of the task queue. Furthermore, the above problem is transformed into a Markov Decision Process (MDP), and a PPO algorithm (named ON-PPO) is proposed, which involves orthogonal encoding of task category states and normalization of system states. The intelligent agent dynamically perceives the system's workload by analyzing real-time resource information (such as the availability of edge and cloud computing resources) in the MDP state. Based on this perception, the system implements dynamic load scheduling: when tasks are sparse and resources are idle, the agent strategically increases the computing load on edge devices, allowing more computation to be completed locally, thereby reducing the occupation and dependence on cloud resources; when tasks are intensive, the agent can clearly distinguish between different types of tasks, and based on the differences in computing requirements of task categories, it differentiates different tasks through orthogonal coding and formulates differentiated offloading schemes to avoid system blockage. This invention, by combining model structure analysis, task characteristic modeling, and reinforcement learning algorithm optimization, systematically solves the core problems caused by high computing requirements, dynamic network environment, and task sparsity in maritime mobile edge computing, significantly improving task processing efficiency and system stability.

[0232] The above description is only a part of the embodiments of the present invention and does not limit the scope of protection of the present invention. Any equivalent device or equivalent process transformation made based on the content of the present invention specification and drawings, or direct or indirect application in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A dynamic offloading method for maritime mobile edge computing for occasional tasks, characterized in that, Includes the following steps: Step 1: Acquire the raw image data for the maritime target detection mission and preprocess it to obtain mission data; Step 2: Construct a system model for nearshore maritime mobile edge computing based on the task data, including a computing model, a communication model, and a task queue model, to quantify the latency and total blocking cost of task data in edge computing, data transmission, and cloud computing. Step 3: Based on the system model, the optimization objective of minimizing the total blocking cost is transformed into a Markov decision model problem, and the state, action, and reward functions of the Markov decision model are defined. Step 4: Based on the Markov decision model and task data, train the ON-PPO decision model that incorporates orthogonal coding of task categories and state normalization to obtain the trained ON-PPO decision model; the specific execution process of the ON-PPO decision model is as follows: Step 41: The ON-PPO decision model includes an actor network and a critic network; initialize the parameters of the actor network. Parameters of the critic network ; Step 42: For each training round in the training process, perform the following sub-steps: a. Current strategy using actor networks Interact with the environment to collect the status of a batch. ,action and reward function sequence; b. The current strategy of the actor network The parameters are copied as the old strategy. This is used for subsequent calculations; Step 43: Analyze the collected status. Perform normalization processing, specifically: use min-max normalization to normalize the states. Scaling to : in, It is the minimum value of the state characteristic. = - Scaling factor The maximum value of the state characteristic; and the state The image algorithm model categories contained herein are subjected to orthogonal coding processing corresponding to the task categories; specifically, one-hot coding is used to map each task category to a binary vector of length equal to the total number of task categories, in which only the l-th position is 1 and the rest are 0; Step 44: For each time step in the batch Each time step Corresponding task set One of the tasks Reward function based on data collection and under the current strategy, the first State at each time step Value function estimate Calculate the task k Corresponding advantage function estimate ; in, For the first Discount factor for each time step, For the first Discount factor for each time step, Indicates the index of the time step and ; Indicates the first [number]th ... State at each time step The estimated value function; Step 45: Within each training round, perform M internal update iterations, performing the following operations in each iteration: a. For each time step in the sequence, calculate the sampling probability ratio based on the old and new strategies. ; in, Indicates task k Corresponding to the current strategy of the actor network, Indicates task k This corresponds to the old strategy of actor networks; b. Based on the sampling probability ratio With advantage function estimate The policy loss function of the actor network is calculated using the pruning function. ; in, This represents the clipping function. The preset cropping threshold is set between 0 and 1. c. Use the calculated policy loss function Through about The Adam optimizer updates the parameters of the actor network. ; d. Reward function based on data collection Calculate the value loss function of the critic network. and use the calculated value loss function The parameters φ of the critic network are updated using the Adam optimizer with respect to φ. Value loss function The formula is: in, For the first The reward function for each time step; Indicates from the first Cumulative discount rewards after a certain time step This indicates that the critics' network is in its current phase under the current strategy. State at each time step The value function estimate is given by the parameter . The network of critics calculated the data used to assess the situation. state The corresponding long-term cumulative reward expectation; Step 46: After iteration, the trained ON-PPO decision model, based on the input state and the task... Output the optimal unloading action. That is, determining the set of image algorithm model layers to be executed on the edge device. and the collection of image algorithm model layers executed on cloud devices ; Step 5: Deploy the trained ON-PPO decision model to edge devices; Step 6: The ON-PPO decision model outputs the optimal offloading decision based on the real-time status. The edge device allocates tasks to the edge local or cloud devices and executes them based on the output optimal offloading decision.

2. The method for dynamic offloading of maritime mobile edge computing for occasional tasks as described in claim 1, characterized in that, Step 1 specifically includes: Step 11: Acquire raw image data for the maritime target detection mission using an acquisition device, which includes a near-shore fixed camera, an unmanned aerial vehicle (UAV)-borne high-definition camera, and / or an image sensor mounted on an unmanned surface vessel. Step 12, the task set generated during equipment operation is represented as follows: , Indicates the number of tasks. Represents the index of the task in the task set and Image algorithm model category set , Indicates the number of image algorithm model categories. Represents the index of the image algorithm model category in the set of image algorithm model categories and According to each task in the task set The task category determines the corresponding image algorithm model category within the set of image algorithm model categories; one task category corresponds to one image algorithm model category. All were classified as Layer, represented as , Indicates the number of layers in the image algorithm model. Indicates the index of the image algorithm model layer. This represents the set of image algorithm model layers; Step 13: Perform convolution operation on the original image data using a Gaussian filtering algorithm to remove noise, wherein the convolution kernel size is 3×3, and accelerate the noise reduction process using an FPGA; Step 14: Identify the target region in the denoised image data based on the Canny edge detection algorithm, and crop the background region within the target region; Step 15: Convert the cropped image data into a preset unified format to obtain task data containing image algorithm model category, image size, and preliminary target contour information; Step 16: Output the task data to the edge processing layer as input to the computing model, task queue model, and ON-PPO decision model.

3. The method for dynamic offloading of maritime mobile edge computing for occasional tasks as described in claim 2, characterized in that, The computational model constructed in step 2 is used to calculate the computational latency of the task on the edge device and the computational latency on the cloud device. Specifically, the computational latency is quantified in the following ways: (1) Regarding the task k Its computational latency on edge devices Represented as: in, Indicates the task performed on the edge device. k In the category of image algorithm models The image algorithm model layer set in the middle, Represents the category of image algorithm model The Middle n The number of floating-point operations required for the layer Indicates assignment to task k The computing power of edge devices, measured in floating-point operations per second; Indicates task k The sum of the computational power of all image algorithm model layers on the edge device; (2) Regarding the task k Its computing latency on cloud devices Represented as: in, Indicates the task performed on a cloud device. k In the category of image algorithm models The image algorithm model layer set in the middle, Indicates assignment to task k The computing power of cloud devices Indicates task k The sum of the computing power of all image algorithm model layers on the cloud device.

4. The method for dynamic offloading of maritime mobile edge computing for occasional tasks as described in claim 3, characterized in that, The communication model constructed in step 2 is used to calculate the transmission rate of the task from the edge device to the base station and the transmission time of the intermediate data of the task from the edge device to the base station. Specifically, the transmission latency is quantified in the following ways: (1) Task k Transmission rate from edge device to base station Represented as: Where B is the transmission bandwidth. For the transmission power of edge devices, For the task k Corresponding to the large-scale attenuation coefficient of maritime communication, Noise power; (2) Edge devices will perform tasks k Transmission time of intermediate data sent to the base station Represented as: in, Image algorithm model category The Middle n The amount of data output from the layer.

5. The method for dynamic offloading of maritime mobile edge computing for occasional tasks as described in claim 4, characterized in that, The task queue model constructed in step 2 is used to calculate the blocking latency caused by resource contention in the three stages of edge computing, data transmission, and cloud computing. Specifically, the blocking latency is quantified in the following ways: (1) The processing of each task is divided into three sequential stages: edge computing, data transmission and cloud computing; (2) For the edge computing stage, the computing task k Blocking time on edge devices : in, The end time for executing the previous task on the edge device. Indicates performing tasks on edge devices. k End time, To perform tasks on edge devices k Arrival time, Indicates performing tasks on edge devices. k Start time; Indicates task k Computation latency on edge devices; (3) For the data transmission stage, the computing task k Blocking time on edge devices : in, It refers to the end time of the previous task executed on the edge device. It performs tasks on edge devices. k End time, Performing tasks on edge devices k Start time; This indicates that the edge device will perform the task. k The transmission time of intermediate data sent to the base station; (4) For the cloud computing stage, the blocking time of computing task k on the edge device : in, It refers to the end time of the previous task executed on the cloud device. It is to perform tasks on cloud devices. k End time, Performing tasks on cloud devices k The start time, Indicates task k Computing latency on cloud devices.

6. The method for dynamic offloading of maritime mobile edge computing for occasional tasks as described in claim 5, characterized in that, Step 2, quantifying the total blocking cost, specifically involves integrating the computational model, communication model, and task queue model to form a system model for quantifying the total blocking cost, with minimizing the total blocking cost as the optimization objective. The system model quantifies the total blocking cost in the following ways: (1) The total blocking time is used as the total blocking cost for the task. k Total blocking cost Represented as: in, This is a weighting coefficient for the blocking latency in the edge computing phase. This is a weighting coefficient for the blocking delay during the data transmission phase. This is a weighting coefficient for blocking latency during the cloud computing phase. This indicates the task in the edge computing phase. k Blocking time on edge devices, This indicates the task during the data transmission phase. k Blocking time on edge devices, This represents the blocking time of task k on the edge device during the cloud computing phase; (2) Minimizing the total blocking cost is taken as the optimization objective, and the optimization objective formula is: Furthermore, the optimization objective must satisfy the following constraints: Constraint 1: Edge devices are assigned to tasks k computing power The maximum computing power of the edge device must not be exceeded. ,Right now: Constraint 2: Cloud devices are assigned to tasks k computing power It must not exceed the maximum computing power of the cloud device. ,Right now: Constraint 3: The image algorithm model category to which all tasks to be processed belong. All belong to a predefined set of image algorithm model categories. L ,Right now: Constraint 4: For any image algorithm model category Task k The collection of image algorithm model layers that execute on edge devices Compared to the collection of image algorithm model layers executed on cloud devices The union of the two sets equals the image algorithm model. Complete image algorithm model layer set ,Right now: Constraint 5: For the task k The collection of image algorithm model layers that execute on edge devices Compared to the collection of image algorithm model layers executed on cloud devices The intersection of the two sets is the empty set. ,express and There is no overlap between them, that is: Constraint 6: The sum of the weighting coefficients for blocking delay in each stage is 1, that is: .

7. The method for dynamic offloading of maritime mobile edge computing for occasional tasks as described in claim 6, characterized in that, In step 3, the state, action, and reward functions of the Markov decision model are defined as follows: (1) Task k status Defined as: in, This indicates that the edge device is assigned to the task. k computing power This indicates that the cloud device assigns the task. k computing power Indicates task k Transmission rate from edge devices to base stations; Image algorithm model category The set of computational requirements for all image algorithm model layers in the dataset. Represents the category of image algorithm model The computational requirements of the first layer in the middle, Represents the category of image algorithm model The Middle n The computational requirements of the layer Represents the category of image algorithm model The Middle N The computational requirements of the layer; Image algorithm model category The set of data outputs from all layers of the image algorithm model. Represents the category of image algorithm model The data output of the first layer in the middle. Represents the category of image algorithm model The Middle n The data output volume of the layer Represents the category of image algorithm model The Middle N Layer data output volume; (2) Handling tasks k action Defined as: (3) In state The reward function below Defined as: .

8. The method for dynamic offloading of maritime mobile edge computing for occasional tasks as described in claim 1, characterized in that, Step 6 specifically includes: Step 61: Collect the status in real time and input it into the ON-PPO decision model, and output the unloading action of the current task; Step 62: The edge device distributes the task data to the local edge device or the cloud device for execution based on the unloading action; Step 63: Real-time statistics of the blocking time data of each task in each stage of edge computing, data transmission and cloud computing, and feed this blocking time data into the ON-PPO decision model as a new training sample to form a closed-loop optimization.

Images