Edge computing task offloading method, system and device for port facility monitoring

Abstract

The present invention provides an edge computing task offloading method, system and device for port facility monitoring, which belongs to the field of mobile edge computing, including: constructing an edge computing system architecture for long-term service monitoring of port infrastructure; establishing local computing and edge computing models, performing feature analysis and local computing and edge computing modeling on long-term service monitoring tasks of port infrastructure; establishing a delay-energy consumption joint optimization model that meets delay requirements while considering energy consumption sensitivity differences; establishing a deep reinforcement learning method-based optimal computing task offloading strategy; and performing edge computing task offloading according to the optimal offloading strategy. The method solves the problem through dynamic edge computing task offloading and computing power allocation algorithms, obtains the optimal task offloading decision and computing power allocation plan, and solves the problem of edge computing task offloading for port facility monitoring.

Description

Technical Field

[0001] The present invention belongs to the field of mobile edge computing technology, and specifically relates to an edge computing task offloading method, system and device for port facility monitoring. Background Art

[0002] Long-term service monitoring of port infrastructure is the basis for building a scientific observation network for the long-term service performance of port infrastructure, which helps to achieve preventive maintenance of transportation infrastructure and improve the level of intelligent maintenance. However, the intelligent business data volume and computing volume of long-term service monitoring of port infrastructure are large, and the latency requirements are high. The computing power of the monitoring equipment alone is difficult to meet the service quality requirements, and computing power supplementation is needed.

[0003] Cloud computing has abundant computing resources, but has large transmission delay. Edge computing has a good compromise performance in computing resources and transmission delay, which can effectively make up for the shortcomings of cloud computing, but it also has certain shortcomings. The invention patent with publication number CN116489712A provides a mobile edge computing task offloading method based on deep reinforcement learning. First, the user equipment terminal in the mobile edge computing environment is calculated to offload the computing task, and the delay consumption and energy consumption consumed in the process of uploading the computing task to the mobile edge computing MEC server are consumed; under the constraints of the maximum tolerable delay and computing power, the objective function is established with the goal of minimizing the energy consumption of the mobile edge computing system, and the objective function is solved based on the improved deep reinforcement learning algorithm, and the computing task offloading decision and resource allocation plan of the user equipment terminal are obtained, so as to achieve device energy saving. However, this solution lacks optimization for delay and fails to achieve joint optimization of delay and energy consumption. The invention patent with the publication number CN116193471A discloses an edge computing offloading method based on deep reinforcement learning. It first constructs the computing offloading problem in the mobile edge network as a multi-objective optimization problem, comprehensively considers latency and energy consumption, decomposes the occasional problem into sub-problems and constructs it into a Markov decision process. According to the deep reinforcement learning algorithm, the model parameters of all sub-problems are coordinated and optimized to further obtain the optimal solution to the problem, thereby achieving joint optimization of latency and energy consumption. However, this solution lacks consideration of the different energy consumption sensitivities of different devices caused by geographical factors, and is of limited help in solving the problem of computing power sharing in port scenarios.

[0004] Through the above analysis, the existing technical solutions have the following deficiencies: (1) Most solutions that consider both latency and energy consumption effects often use one as a constraint to optimize the other indicator, and lack joint optimization of latency and energy consumption. Existing task offloading and computing power allocation solutions lack consideration of latency requirements when jointly optimizing latency and energy consumption, which may cause task timeouts and task failures. (2) A small number of solutions that jointly optimize latency and energy consumption omit the requirements for task information freshness, and lack consideration of the differences in energy constraints caused by different energy sources of equipment in different equipment deployment environments. For example, the existing technical solutions lack consideration of the differences in equipment energy consumption sensitivity caused by the geographical environment and equipment power supply methods, and the algorithm performance for solving edge computing task offloading and computing power allocation problems in complex scenarios such as ports is poor. Summary of the invention

[0005] In order to solve the problem that the existing technology lacks consideration of latency requirements and energy constraint differences, the present invention provides an edge computing task offloading method for port facility monitoring, which can realize task offloading and computing power allocation under latency constraints while considering the differences in equipment energy sensitivity, thereby achieving the joint optimization of task processing latency and energy consumption to the greatest extent.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] An edge computing task offloading method for port facility monitoring, comprising:

[0008] Based on the area of ​​the port scenario, the energy supply heterogeneity brought by the deployment environment of the edge device server, and the characteristics of the long-term service monitoring task of the port infrastructure, an edge computing system architecture for long-term service monitoring of the port infrastructure is constructed;

[0009] According to the edge computing system architecture, the local computing and edge computing in the edge computing system for long-term service monitoring of port infrastructure are modeled, and the delay and energy consumption model of local computing and edge computing is obtained considering the impact of energy supply differences.

[0010] Based on the latency and energy consumption model, the optimization problem is modeled according to the scenario requirements with the goal of achieving joint optimization of latency and energy consumption;

[0011] The modeled optimization problem is transformed into a constrained Markov problem. The offloading algorithm based on deep reinforcement learning is used to solve the joint optimization problem of latency and energy consumption, and the computing task offloading strategy is obtained.

[0012] Offload edge computing tasks based on the offloading strategy.

[0013] Preferably, the edge computing system architecture for long-term service monitoring of port infrastructure is constructed as follows:

[0014] Several sensor terminals and several edge servers are deployed in the port scenario, each server is equipped with a single antenna; the edge servers are divided into offshore edge servers and land edge servers.

[0015] Preferably, the local computing and edge computing in the edge computing system for long-term service monitoring of port infrastructure are modeled according to the edge computing system architecture to obtain the delay and energy consumption model of the local computing and edge computing considering the influence of energy supply differences, specifically:

[0016] The port infrastructure edge computing system model is constructed as a two-layer edge offloading architecture. There are K sensor terminals for generating tasks, and the sensor terminal set is D{d1,d2,…d k …,d K ,}, where the kth sensor terminal d k The state of the vector express; Indicates the maximum computing frequency of the sensor terminal, where Represents the computing time required for the remaining computing amount. There are I edge servers in total, using the set N{n1,n2,…n i …,n I} indicates that Indicates edge server n i The maximum calculation frequency, Represents edge server n i The computation time required for the current remaining computation; Represents edge server n i The edge server type is When , it means that the edge server belongs to the land server. When the edge server is an offshore server;

[0017] Time slot Indicates that the generated tasks are represented by a collection Indicates that the task is m(t) k {s(t) k ,c(t) k ,δ(t) k},s(t) k Indicates the size of the computing task data, c(t) k represents the CPU cycles required to complete the task, δ(t) k Represents the maximum tolerable delay of the computing task;

[0018] The connection relationship between devices is represented by the offloading matrix G. The number of rows of the offloading matrix G is equal to the number of sensor terminals K, and the number of columns is equal to the number of edge servers I, as shown below:

[0019]

[0020] Indicates sensor terminal and edge server n i If the sensor terminal and edge server n i If there is a connection between the sensor terminals The task m generated above k Ability to offload to edge servers i ,at this time Conversely, if the sensor terminal and edge server n i There is no connection between them, and tasks cannot be offloaded between them.

[0021] where X(t) is the set of task offloading decisions, X(t) = {x(t)1, x(t)2, … x(t) k …,x(t) K},x(t) k is the task offloading decision, x(t) k = 0 means that the task is calculated locally, x(t) k = 1, it means that the task is offloaded to the edge server for calculation; the offloaded object of the task is represented by the set N(t), N(t) = {n(t)1,n(t)2,…n(t) k …,n(t) K}, if x(t) k =0, then the corresponding n(t) k =0, otherwise n(t) k = the corresponding offloading edge server number, the task computing resource allocation is represented by the set F(t). When computing resource allocation is performed, F(t) = {f(t)1, f(t)2, ... f(t) k …,f(t) K}, f(t) k is the edge server assigned to task m(t) in time slot t k computing resources.

[0022] Preferably, the task calculation is divided into two cases: local calculation and offloading to the edge server for calculation. The two cases are modeled respectively, specifically:

[0023] Local computational modeling:

[0024] For task m(t) k , when the unloading decision x(t) k = 0 means that the task is calculated locally, and the total delay of the task is Including the task calculation delay and queuing delay, that is:

[0025]

[0026] in is the task m(t) k The required computation time, is the task m(t) k The time required to wait, that is, the task m(t) currently queued on the local device k The sum of the total computing time of the previous tasks to be calculated:

[0027]

[0028] Task m(t) k The computational delay for:

[0029]

[0030] Where c(t) k is the computational task m(t) k The number of CPU cycles required, f(t) k is assigned to task m(t) in time slot t k computing resources;

[0031]

[0032] Total energy consumption of the task Equal to computing energy consumption As shown below:

[0033]

[0034] Where λ1 is the energy dissipation coefficient of the sensor terminal, which can take different values ​​according to the different conditions of the sensor terminal itself;

[0035] Edge Offloading Modeling:

[0036] When x(t) k =1, it means that the task is offloaded to the edge server for calculation. At this time, the total delay is composed of transmission delay, calculation delay, queuing delay and result return delay. When task m(t) k The total latency of the task when offloading to the edge server for:

[0037]

[0038] in is the transmission task n(t) k The required transmission time, Task m(t) k The required computation time, is the task m(t) k The time required to wait, that is, the task m(t) currently scheduled on the unloading edge server k The sum of the total computing time of the previous tasks to be calculated:

[0039]

[0040] Task m(t) k Unload transfer time for:

[0041]

[0042] Where s(t) k is the computational task m(t) k The data size, R(t) k is the transmission task m(t) k The transmission rate can be given by:

[0043]

[0044] Where W represents the task m(t) k The transmission bandwidth, P(t) k Represents the transmission task m(t) k The transmission power, L(t) k represents the path loss during transmission, and N0 represents the noise power of Gaussian white noise during transmission;

[0045] Task f(t) k Calculation time for:

[0046]

[0047] Where c(t) k is the computational task m(t) k The number of CPU cycles required, f(t) k is the edge server assigned to task m(t) in time slot t k computing resources;

[0048] Allocated computing resources f(t) k Not exceeding the computing power of the edge server, that is:

[0049]

[0050] Task m(t) k Total energy consumption Equal to the transmission energy consumption and calculate energy consumption The sum of the transmission energy consumption for:

[0051]

[0052] Calculating energy consumption for:

[0053]

[0054] Where λ2 and λ3 are the energy dissipation coefficients of the land server and the offshore server respectively. Considering the characteristics of the land server and the offshore server, generally λ2>λ3;

[0055] Task m(t) k The total delay T(t) k It is expressed as:

[0056]

[0057] Total energy consumption E(t) k Can be expressed as

[0058]

[0059] Preferably, the optimization problem is modeled based on the delay and energy consumption model, according to the scenario requirements, with the goal of achieving joint optimization of delay and energy consumption, specifically:

[0060]

[0061] st

[0062]

[0063] in, Represents the optimization goal, the objective function is to minimize the sum of delay and energy consumption, α' and β' are weight coefficients that balance delay and energy consumption; constraint C1 indicates that the task intelligence is calculated locally or offloaded to the edge server for calculation; constraint C2 indicates that the server where the task is offloaded should be able to communicate with the local device of the task; constraint C3 indicates that the computing resources allocated to the task should not exceed the maximum computing frequency of the device; constraint C4 indicates that the total delay of the task should not exceed the tolerable delay of the task.

[0064] Preferably, the optimization problem after modeling is converted into a constrained Markov problem, and the delay-energy consumption joint optimization problem is solved using an offloading algorithm based on deep reinforcement learning to obtain a computing task offloading strategy, which is specifically:

[0065] State modeling: The state consists of the system state and the generated task state. in is the state of the sensor terminal at time t, Indicates the maximum computing frequency of the sensor terminal, where Represents the computation time required for the remaining computation; is the state of the edge server at time t, Indicates that Indicates the maximum computing frequency of the edge server. Represents the computation time required for the remaining computation. Represents the edge server type; are the parameters for generating tasks, Task m(t) k {s(t) k ,c(t) k ,δ(t) k},s(t) k Indicates the size of the computing task data, c(t) k represents the CPU cycles required to complete the task, δ(t) k Represents the maximum tolerable delay of the computing task;

[0066] Action modeling: The set of actions that the agent can take is where X(t) is the set of task offloading decisions, X(t) = {x(t)1, x(t)2, … x(t) k …,x(t) K},x(t) k is the task offloading decision, x(t) k = 0 means that the task is calculated locally, x(t) k =1, it means that the task is offloaded to the edge server for calculation; F(t) is the task computing resource allocation set. When computing resources are allocated, F(t) = {f(t)1, f(t)2, …f(t) k …,f(t) K}, f(t) k is the edge server assigned to task m(t) in time slot t kWhen only offloading decisions are made without computing resource allocation, N(t) is the set of offloaded objects of the task N(t) = {n(t)1,n(t)2,…n(t) k …,n(t) K}, if x(t) k =0, then the corresponding n(t) k =0, otherwise n(t) k = the corresponding offloading edge server number;

[0067] Reward modeling: define the function S(t) k To indicate whether the task is successfully completed within the tolerable delay, if the total delay T(t) k Less than the task's tolerance delay δ(t) k , then the work is considered to be successfully completed, S(t) k =1; otherwise, if the total delay T(t) k Greater than the task's tolerance delay δ(t) k , then the task is considered to have timed out and discarded, S(t) k =0, that is:

[0068]

[0069] The single-step return function is set up as follows:

[0070] r(t)=-αT(t) k -βlog2E(t) k +γS(t) k (20)

[0071] where α, β, and γ are parameters that normalize the time cost, energy cost, and profit of completing a task on time to the same scale;

[0072] Current actions are evaluated by long-term payoffs:

[0073]

[0074] where ξ is the discount factor for future rewards.

[0075] Preferably, the edge computing task is unloaded according to the unloading strategy, specifically:

[0076] Initialize the number of IoT devices, the number of edge servers, the offloading matrix, the distance matrix, and the path loss matrix, and define the hyperparameters;

[0077] Initialize the global neural network parameters μ and θ, and start multiple asynchronous training workers, where each worker’s local neural network parameter μ i,θ i Independent of the random number generator, for each thread Workeri, reset the environment, initialize the device queue time to 0, generate a task at the same time, and complete the generation of the initial state s; different computing power sharing scenarios require the generation of action space AS, initialize the reward and time slot count, and start training;

[0078] During training, the Actor network calculates the action probability based on the state s, outputs the action a, calculates the time and energy consumption to generate the reward r, and waits until the new state s; finally, the new state s, action a, reward r and next state s are put into the training set for subsequent training of the intelligent agent; when the training set is full, data is extracted to calculate the loss function and update the network parameters until the iteration ends, and the computing task offloading decision and resource allocation plan of the user device terminal are obtained.

[0079] The present invention also provides an edge computing task offloading system for port facility monitoring, comprising:

[0080] The system framework building module is used to build an edge computing system architecture for long-term service monitoring of port infrastructure based on the area of ​​the port scenario, the energy supply heterogeneity brought by the edge device server deployment environment, and the characteristics of the long-term service monitoring task of port infrastructure;

[0081] A joint model building module is used to model the local computing and edge computing in the edge computing system for long-term service monitoring of port infrastructure according to the edge computing system architecture, and obtain the delay and energy consumption model of local computing and edge computing that takes into account the impact of energy supply differences;

[0082] The problem model building module is used to model the optimization problem based on the latency and energy consumption models and the scenario requirements, with the goal of achieving joint optimization of latency and energy consumption;

[0083] The offloading strategy building module is used to transform the modeled optimization problem into a constrained Markov problem, and use the offloading algorithm based on deep reinforcement learning to solve the joint optimization problem of latency and energy consumption to obtain the computing task offloading strategy;

[0084] The offloading module is used to offload edge computing tasks according to the offloading strategy.

[0085] The present invention also provides a computer device, comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement any step of the edge computing task offloading method for port facility monitoring.

[0086] The edge computing task offloading method for port facility monitoring provided by the present invention has the following beneficial effects:

[0087] The present invention constructs an edge computing system architecture for long-term service monitoring of port infrastructure based on the characteristics of port scenarios, the heterogeneity of energy supply brought by the deployment environment of edge device servers, and the characteristics of long-term service monitoring tasks of port infrastructure. It grasps the task characteristics of long-term service monitoring tasks of port infrastructure, and establishes local computing and edge computing modeling by allocating computing power to different tasks to perform different offloading decisions. The joint optimization problem of latency and energy consumption is established on the premise of meeting the latency requirements of port infrastructure service monitoring tasks. The problem is solved through dynamic edge computing task offloading and computing power allocation algorithms, and the optimal task offloading decision and computing power allocation scheme are obtained, solving the problem of edge computing task offloading for port facility monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] In order to more clearly illustrate the embodiment of the present invention and its design scheme, the following briefly introduces the drawings required for this embodiment. The drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0089] Figure 1 This is a flowchart of the edge computing task offloading method for port facility monitoring according to Example 1 of the present invention. DETAILED DESCRIPTION

[0090] In order to enable those skilled in the art to better understand the technical solution of the present invention and implement it, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the scope of protection of the present invention.

[0091] Example 1

[0092] Edge computing has a good compromise performance in terms of computing resources and transmission delay, which can effectively make up for the shortcomings of cloud computing and replace cloud computing to supplement computing power for long-term service monitoring of port infrastructure. Considering the large area of ​​the port area, the complex deployment environment of facilities and equipment, and the diverse computing business needs, edge computing equipment needs to be deployed in a distributed manner. Designing a reasonable offloading solution to achieve distributed computing power sharing is of great significance to improving the quality of port infrastructure monitoring, reducing the overall energy consumption of the system, and improving user experience.

[0093] Based on the existing edge computing paradigm and the implementation of latency requirements and energy constraints, the present invention takes into account the system characteristics and business requirements of edge computing for long-term service monitoring of port infrastructure, including the large computational complexity and high latency requirements of port detection tasks, as well as the differences in power supply and energy constraints brought about by the edge server deployment environment. A dynamic edge computing task offloading and computing power allocation algorithm is proposed to solve the task offloading and computing power allocation problems when there are differences in the energy constraints of equipment in a port environment, thereby achieving the joint optimization of latency and energy consumption under latency constraints.

[0094] The present invention mainly involves the following contents:

[0095] 1. Constructed an edge computing system architecture for long-term service monitoring of port infrastructure.

[0096] 2. Establish local computing and edge computing models to perform feature analysis, local computing, and edge computing modeling for long-term service monitoring tasks of port infrastructure.

[0097] 3. The differences in energy consumption sensitivity caused by different energy supply methods are taken into account to meet the constraints of geographical restrictions in the port environment.

[0098] 4. A joint optimization problem of delay and energy consumption is established to meet the delay requirements of port infrastructure service monitoring tasks.

[0099] 5. A dynamic edge computing task offloading and computing power allocation algorithm is proposed to solve the problem, which effectively reduces the system latency and energy consumption, and realizes the joint optimization of energy consumption and latency under latency constraints.

[0100] Specifically, Figure 1 As shown, the present invention provides an edge computing task offloading method for port facility monitoring, comprising the following steps:

[0101] Step 1: Extract the characteristics of the port scenario, consider the energy supply heterogeneity brought by the edge device server deployment environment, and conduct feature analysis on the long-term service monitoring task of port infrastructure. On this basis, build an edge computing system architecture for long-term service monitoring of port infrastructure.

[0102] Specifically, several sensor terminals and edge servers are deployed in the port scenario, and each server is equipped with a single antenna. Edge servers can be divided into offshore edge servers and land edge servers. After the task is generated on the sensor terminal, the sensor terminal sends the task information to the control center. The control center collects global information and makes decisions. The decision content includes whether the task is unloaded, the unloading object, and the allocated computing resources. After the decision is made, the corresponding decision is sent to the sensor terminal, and the sensor terminal unloads the edge computing task according to the control information. The transmission time and resource consumption of the status and control information can be ignored.

[0103] Step 2: Model the local computing and edge computing in the edge computing system for long-term service monitoring of port infrastructure, and obtain the latency and energy consumption model of local computing and edge computing that takes into account the impact of energy supply differences.

[0104] The intelligent business of long-term service monitoring of port infrastructure refers to the use of artificial intelligence technology to intelligently process and analyze the long-term service monitoring of port infrastructure, thereby improving the operational efficiency and safety of port infrastructure. Port infrastructure failures often affect port operations in a short period of time and even cause safety accidents. Therefore, the intelligent business of long-term service monitoring of port infrastructure needs to process and analyze monitoring data in real time in order to find problems in time and take corresponding measures, so as to minimize the stagnation of port operations and related safety hazards. Therefore, the intelligent business of long-term service monitoring of port infrastructure has the characteristics of high latency requirements. Secondly, the number of port infrastructure is large, the types of sensors are many, the types of data are diverse, and the amount of calculation is large. Therefore, the long-term service monitoring of port infrastructure needs to introduce edge computing to supplement computing power. In addition, the long-term service monitoring of port infrastructure requires long-term stable operation and high energy consumption. At the same time, due to the geographical limitations of port scenarios, equipment in port scenarios often has different energy supply methods, which will lead to different energy consumption sensitivities of equipment.

[0105] The present invention constructs the port infrastructure edge computing system model as a two-layer edge unloading architecture, generating tasks with a total of K sensor terminals, and the sensor terminal set is D{d1, d2, ... d k …,d K ,}, where the kth sensor terminal d k The state of the vector express. Indicates the maximum computing frequency of the sensor terminal, where Represents the computing time required for the remaining computing amount. There are a total of I edge servers using the set N{n1,n2,…n i …,n I} indicates that Indicates edge server n iThe maximum calculation frequency, Represents edge server n i The computing time required for the remaining computing amount at present. Since the working edge computing scenario of the present invention is for the long-term service monitoring scenario of port infrastructure, the port scenario includes breakwaters and offshore, so in this scenario the edge server is deployed on land and offshore at the same time. Considering the impact of the geographical location of the edge server on the server performance, the heterogeneous type of the edge server should be taken into consideration. The present invention can divide the edge server into offshore servers and land servers, where the computing resources of the two servers are significantly richer than those of the sensor terminals, but the offshore server relies on battery power and there is no stable power supply. Although the battery capacity is larger than that of the Internet of Things monitoring equipment, there is still energy consumption limitation; the land edge server is deployed on the shore and relies on cable connection for power supply. The energy supply is sufficient and stable, and the energy is not limited. Represents edge server n i The edge server type is When , it means that the edge server belongs to the land server. At this time, the edge server belongs to the offshore server.

[0106] The time slot of the present invention is used The generated tasks are represented by a collection Indicates that the present invention stipulates that the time slot size is smaller than the task generation interval, so at most one task is generated on each IoT device in each time slot. Similarly, the time slot interval is smaller than the task tolerance time. In a time slot, there may be multiple different offload tasks that choose the same edge server for calculation. At this time, the calculation will be performed in the order in which the tasks arrive. Task m(t) k {s(t) k ,c(t) k ,δ(t) k},s(t) k Indicates the size of the computing task data, c(t) k represents the CPU cycles required to complete the task, δ(t) k Represents the maximum tolerable delay of the computing task.

[0107] The connection relationship between devices is represented by the offloading matrix G. The number of rows of the offloading matrix G is equal to the number of sensor terminals K, and the number of columns is equal to the number of edge servers I, as shown below:

[0108]

[0109] Indicates sensor terminal and edge server n i If the sensor terminal and edge server n iIf there is a connection between the sensor terminals The task m generated above k Can be offloaded to edge servers i ,at this time Conversely, if the sensor terminal and edge server n i There is no connection between them, and tasks cannot be offloaded between them.

[0110] This scenario is a binary offloading scenario, that is, the task can only be calculated locally or offloaded to the edge server for calculation. The offloading decision set X(t) is the set of task offloading decisions, X(t) = {x(t)1, x(t)2, … x(t) k …,x(t) K},x(t) k is the task offloading decision, x(t) k = 0 means that the task is calculated locally, x(t) k = 1, it means that the task is offloaded to the edge server for calculation. The offloaded object of the task is represented by the set N(t), N(t) = {n(t)1, n(t)2, … n(t) k …,n(t) K}, if x(t) k =0, then the corresponding n(t) k =0, otherwise n(t) k = The corresponding offloading edge server number. The task computing resource allocation is represented by the set F(t). When computing resources are allocated, F(t) = {f(t)1, f(t)2, ... f(t) k …,f(t) K}, f(t) k is the edge server assigned to task m(t) in time slot t k computing resources.

[0111] Task computing can be divided into two situations: local computing and offloading to edge servers for computing. The present invention models the two situations respectively.

[0112] (1) Local computational modeling:

[0113] For task m(t) k , when the unloading decision x(t) k = 0 means that the task is calculated locally, and the total delay of the task is Including the task calculation delay and queuing delay, that is:

[0114]

[0115] in is the task m(t) k The required computation time, is the task m(t) k The time required to wait, that is, the task m(t) currently queued on the local device k The sum of the total computing time of the previous tasks to be calculated:

[0116]

[0117] Task m(t) k The computational delay for:

[0118]

[0119] Where c(t) k is the computational task m(t) k The number of CPU cycles required, f(t) k is assigned to task m(t) in time slot t k computing resources.

[0120] Considering that local device computing resources are very limited, this model does not allocate local computing resources, so:

[0121]

[0122] Total energy consumption of the task Equal to computing energy consumption As shown below:

[0123]

[0124] Where λ1 is the energy dissipation coefficient of the sensor terminal, which can take different values ​​according to different conditions of the sensor terminal itself.

[0125] (2) Edge offloading modeling:

[0126] When x(t) k = 1, it means that the task is offloaded to the edge server for calculation. At this time, the total delay is composed of transmission delay, calculation delay, queuing delay and result return delay. Considering that the size of the calculation result is negligible relative to the calculation task, the return time is ignored. When task m(t) k The total latency of the task when offloading to the edge server for:

[0127]

[0128] in is the transmission task m(t) kThe required transmission time, Task m(t) k The required computation time, is the task m(t) k The time required to wait, that is, the task m(t) currently scheduled on the unloading edge server k The sum of the total computing time of the previous tasks to be calculated:

[0129]

[0130] Task m(t) k Unload transfer time for:

[0131]

[0132] Where s(t) k is the computational task m(t) k The data size, R(t) k is the transmission task m(t) k The transmission rate can be given by the following formula:

[0133]

[0134] Where W represents the task m(t) k The transmission bandwidth, P(t) k Represents the transmission task m(t) k The transmission power, L(t) k represents the path loss during the transmission process, and N0 represents the noise power of Gaussian white noise during the transmission process.

[0135] Task m(t) k Calculation time for:

[0136]

[0137] Where c(t) k is the computational task m(t) k The number of CPU cycles required, f(t) k is the edge server assigned to task m(t) in time slot t k computing resources.

[0138] Allocated computing resources f(t) k The computing power of the edge server should not be exceeded, that is:

[0139]

[0140] Task m(t) k Total energy consumption Equal to the transmission energy consumption and calculate energy consumption The sum of the transmission energy consumption for:

[0141]

[0142] Calculating energy consumption for:

[0143]

[0144] Where λ2 and λ3 are the energy dissipation coefficients of the land server and the offshore server respectively. Considering the characteristics of the land server and the offshore server, generally λ2>λ3.

[0145] So the task m(t) k The total delay T(t) k It can be expressed as:

[0146]

[0147] Total energy consumption E(t) k It can be expressed as:

[0148]

[0149] Step 3: Based on the delay energy consumption model and scenario requirements, the optimization problem is modeled with the goal of achieving joint optimization of delay and energy consumption.

[0150] The edge computing system for long-term service monitoring of port infrastructure should achieve joint optimization of latency and energy consumption to meet the intelligent business needs in this scenario. In order to improve the quality of user services and maximize the overall utility of the system, this paper proposes a modeling of edge computing task offloading problem that pursues joint optimization of latency and energy consumption.

[0151]

[0152] st,

[0153]

[0154]

[0155] Among them, the objective function is to minimize the sum of latency and energy consumption. Constraint C1 means that the task intelligence is calculated locally or offloaded to the edge server for calculation; Constraint C2 means that the server where the task is offloaded should be able to communicate with the local device of the task; Constraint C3 means that the computing resources allocated to the task should not exceed the maximum computing frequency of the device; Constraint C4 means that the total latency of the task should not exceed the tolerable latency of the task.

[0156] Step 4: Convert the modeled optimization problem into a constrained Markov problem, use the offloading algorithm based on deep reinforcement learning to solve the joint optimization problem of delay and energy consumption, and obtain the computing task offloading strategy.

[0157] Since the system model of the present invention is a dynamic system, taking into account the dynamic arrival and waiting of tasks, the Markov decision process is selected to model the problem. Markov decision is suitable for offloading optimization, where dynamic decisions need to be made based on changing environmental parameters (such as wireless channel conditions and computing load). The Markov decision system usually consists of five tuples: Indicates the decision-making period; Represents a set of states; It is a set of actions; represents the state transition probability; is the reward. In this model, it is crucial to define the quintuple according to the unloading process. The following Bellman equation can be formulated:

[0158] V * (s)=max a∈A {R(s,a)+λ∑ s'∈ sp(s'|s,a)V * (s')} (18)

[0159] Where R is the immediate reward, p is the state transition probability, λ is the discount factor, and V is the expected total reward. Then, using classical solutions such as numerical iteration algorithms or policy iteration algorithms, optimal offloading decisions can be obtained, such as the decision on whether to perform offloading locally and the decision on executing offloaded tasks on edge computing servers.

[0160] State modeling: The state of the present invention consists of the system state and the generated task state. in is the state of the sensor terminal at time t, Indicates the maximum computing frequency of the sensor terminal, where Represents the computation time required for the remaining computation. is the state of the edge server at time t, Indicates that Indicates the maximum computing frequency of the edge server. Represents the computation time required for the remaining computation. Indicates the edge server type. are the parameters for generating tasks, Task m(t) k {s(t) k ,c(t) k ,δ(t) k},s(t) k Indicates the size of the computing task data, c(t) k represents the CPU cycles required to complete the task, δ(t) k Represents the maximum tolerable delay of the computing task.

[0161] Action modeling: The set of actions that the agent can take is where X(t) is the set of task offloading decisions, X(t) = {x(t)1, x(t)2, … x(t) k …,x(t) K},x(t) k is the task offloading decision, x(t) k = 0 means that the task is calculated locally, x(t) k =1, it means that the task is offloaded to the edge server for calculation. F(t) is the task computing resource allocation set. When computing resources are allocated, F(t) = {f(t)1, f(t)2, ... f(t) k …,f(t) K}, f(t) k is the edge server assigned to task m(t) in time slot t k When only offloading decisions are made without computing resource allocation, N(t) is the set of offloaded objects of the task N(t) = {n(t)1,n(t)2,…n(t) k …,n(t) K}, if x(t) k =0, then the corresponding n(t) k =0, otherwise n(t) k =The corresponding offloading edge server number.

[0162] Reward modeling: In order to ensure the service quality of delay-sensitive tasks, the present invention defines the function S(t) k To indicate whether the task is successfully completed within the tolerable delay, if the total delay T(t) k Less than the task's tolerance delay δ(t) k , then the work is considered to be successfully completed, S(t) k =1; otherwise, if the total delay T(t) k Greater than the task's tolerance delay δ(t) k , then the task is considered to have timed out and discarded, S(t) k =0, that is:

[0163]

[0164] The goal of the present invention is to balance latency and energy consumption while completing as many tasks as possible, and to achieve joint optimization of latency and energy consumption for distributed computing power sharing for long-term service monitoring of port infrastructure. Therefore, the single-step reward function of the present invention is set as follows:

[0165] r(t)=-αT(t) k -βlog2E(t) k +γS(t) k (20)

[0166] Among them, α, β, and γ are parameters that normalize the time cost, energy cost, and profit of completing tasks on time to the same scale. The three interact to balance the number of tasks completed and the time cost, the weight of energy consumption, and are adjusted according to different edge computing task offloading scenarios or different task emphases. For example, for latency-sensitive tasks, α takes a larger value, and β and γ take relatively smaller values; for energy-sensitive scenarios, such as green energy-saving scenarios, β takes a larger value, and α and γ take relatively smaller values; relative to some scenarios with higher reliability requirements, γ takes a larger value, and β and γ take relatively smaller values. In addition, the energy cost term is proportional to the logarithm, because it is approximately proportional to the square of the MEC operating frequency, and it fluctuates greatly. If it is not processed, it is difficult to control its order of magnitude, which will interfere with the importance of latency and task completion.

[0167] Because the goal is to maximize the expected long-term reward, but the learning agent can only get the immediate reward from the current time step, and the rewards of future time steps are unknown. Therefore, it is necessary to consider the future expected reward to consider the impact of the future reward of the current action, using the immediate reward and the future expected reward estimated by the learned strategy. Specifically, the current action is evaluated by the long-term reward:

[0168]

[0169] where ξ is the discount factor for future rewards.

[0170] The solution algorithm consists of a main network and a training worker. Each worker is an independent process with an independent policy-value (Actor-Critic) network and does not share parameters with other workers, which enhances the algorithm's exploration ability. Each worker is synchronized with the main network (i.e., the global network) regularly.

[0171] In the solution, the Actor-Critic network of each worker is trained based on the policy gradient method. The policy gradient method uses the back-propagation algorithm to calculate the gradient of the loss function and uses the gradient descent algorithm to update the network parameters. During training, the parameters of the neural network corresponding to the worker's policy-value network are first initialized to obtain a neural network with parameters μ and θ. The state at time t is represented by s(t). The Actor network inputs the state s(t) and outputs the action a(t) based on the state s(t), which is the task offloading and computing resource allocation plan. The Critic network is responsible for estimating the value function of the state s(t). The input of the Critic network is the state s, and the output is the value V(s) of the state s. When the action a(t) is taken, the state is transformed from s(t) to s(t+1) through the action of the environment, and the reward r(t) is generated at the same time. s(t), a(t), r(t), s(t+1) are called trajectory data, which are stored in the training set for regular training.

[0172] The goal of the Actor network is to maximize the expectation of long-term rewards, that is, to maximize the state value function V(s). By introducing the advantage function A(s i ,a i )=Q(s i ,a i )-V(s i ) to describe the quality of each action so as to update and train the Actor network. i ,a i ) means in state s i Take action a i The long-term benefits (i.e., cumulative rewards), V(s i ) means in state s i The estimated value of the long-term benefit of taking any action under i ,a i ) means taking action a i The difference in long-term returns relative to the average level is called the advantage.

[0173] When calculating the gradient of the Actor network parameters, the Advantage function is used as the target value, and the gradient of the Actor network parameters is calculated by comparing the difference between the current action value and the target value. The gradient calculation formula is as follows:

[0174]

[0175] Where N is the number of samples in the current round in the training set, μ is the parameter of the Actor network, and π μ (a i ∣s i) represents the probability of taking action a(i) in state s(i). By maximizing the gradient of the Advantage function, the probability of taking excellent actions is increased, thereby improving the performance of the strategy.

[0176] In order to improve the exploration ability of the intelligent agent and avoid falling into the local optimum, the entropy value is introduced into consideration. i )=-∑ a p(a i ∣s i )log p(a i ∣s i ) is used as a penalty term to encourage the agent to explore more strategies. At this time, the loss function of the Actor network is expressed as follows:

[0177]

[0178] Among them, ω is an adjustable parameter used to balance the ratio between entropy penalty and reward. The goal of the entire algorithm is to minimize the loss function and enable the agent to consider both rewards and action diversity, thereby achieving a balance between development and exploration.

[0179] Finally, the gradient ascent method is used to update the Actor network parameters:

[0180]

[0181] The goal of the Critic network is to minimize the mean squared error between the predicted value and the true value, so the loss function of the Critic network is the mean squared error (MSE) of the average return and state value:

[0182]

[0183] Where N represents the batch size, represents the n-step return of the i-th sample, Indicates that the Critic network is in state The predicted state value on .

[0184] Then use the gradient descent algorithm to update the parameters θ of the Critic network to minimize the loss function. The gradient of the loss function with respect to the parameters θ of the Critic network is It is calculated by the back propagation algorithm. First, the gradient of the loss function with respect to the state value is calculated. The chain rule is then used to calculate the gradient of the loss function with respect to the Critic network parameters.

[0185]

[0186] Finally, the gradient descent algorithm is used to update the parameters θ of the Critic network:

[0187]

[0188] in, is the learning rate, which controls the step size of parameter update. The size of can adjust the learning speed of the model to avoid slow convergence or falling into local optimum.

[0189] Step 5: Offload edge computing tasks based on the offloading strategy.

[0190] Initialize the basic settings of edge computing, such as the number of IoT devices, the number of edge servers, the offloading matrix, the distance matrix, the path loss matrix, etc. Then set the algorithm hyperparameters such as the learning rate. Next, initialize the global neural network parameters μ and θ. Start multiple asynchronous training workers, where the local neural network parameters μ of each worker are i and θ i It is independent of the random number generator. For each thread Workeri, reset the environment, initialize the device queue time to 0, generate a task, and complete the generation of the initial state s. Different computing power sharing scenarios require the generation of action space AS, initialize the reward and time slot count, and start training.

[0191] During training, the Actor network calculates the action probability based on the state s, outputs the action a, generates the reward r based on the computation time and computation energy consumption, and waits until the new state s. Finally, the new state s, action a, reward r and next state s are put into the training set for subsequent training of the agent. When the training set is full, data is extracted to calculate the loss function and update the network parameters until the iteration ends, and the computing task offloading decision and resource allocation plan of the user device terminal are obtained.

[0192] The edge computing task offloading method for port facility monitoring provided in this embodiment has the following advantages:

[0193] 1. By deploying sensor devices and edge servers with antennas, an edge computing system architecture for long-term service monitoring of port infrastructure was built.

[0194] 2. The task characteristics of long-term service monitoring tasks of port infrastructure were mastered, and local computing and edge computing modeling was established by allocating computing power to execute different unloading decisions for different tasks.

[0195] 3. A method for characterizing and quantifying the energy consumption sensitivity caused by different energy supply methods was designed. By setting different energy consumption coefficients, the impact of the geographical environment of equipment deployment in the port environment was distinguished.

[0196] 4. Based on the local computing and edge computing models established above, the joint optimization problem of latency and energy consumption was established on the premise of meeting the latency requirements of the port infrastructure service monitoring task.

[0197] 5. The problem was solved through dynamic edge computing task offloading and computing power allocation algorithm, and the optimal task offloading decision and computing power allocation plan were obtained, which solved the problem of computing power sharing for long-term service monitoring of port infrastructure.

[0198] Therefore, the method proposed in the present invention effectively solves the problem of edge computing task offloading for port facility monitoring.

[0199] Based on the same inventive concept, the present invention also discloses an edge computing task unloading system for port facility monitoring, including a system framework building module, a joint model building module, a problem model building module, an unloading strategy building module and an unloading module.

[0200] Specifically, the system framework construction module is used to extract the characteristics of the port scenario, consider the energy supply heterogeneity brought by the edge device server deployment environment, and perform feature analysis on the long-term service monitoring tasks of port infrastructure. On this basis, an edge computing system architecture for long-term service monitoring of port infrastructure is constructed.

[0201] The joint model building module is used to model the local computing and edge computing in the edge computing system for long-term service monitoring of port infrastructure, and to obtain the latency and energy consumption models of local computing and edge computing that take into account the impact of energy supply differences.

[0202] The problem model building module is used to model the optimization problem according to the delay-energy consumption model and scenario requirements, with the goal of achieving joint optimization of delay and energy consumption.

[0203] The offloading strategy building module is used to transform the modeled optimization problem into a constrained Markov problem, and use the offloading algorithm based on deep reinforcement learning to solve the joint optimization problem of delay and energy consumption to obtain the computing task offloading strategy.

[0204] The offloading module is used to offload edge computing tasks according to the offloading strategy.

[0205] Each module in the above-mentioned edge computing task offloading device for port facility monitoring can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0206] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps in the embodiment of the edge computing task offloading method for port facility monitoring. The specific implementation method can be found in the method embodiment, which will not be repeated here.

[0207] Furthermore, the present invention also provides a non-temporary computer-readable storage medium containing instructions, and a computer program is stored on the storage medium. For example, a memory containing instructions, the above instructions can be executed by a processor of a computer device to complete the above method. For example, a non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. When the computer program is executed by the processor, the steps in the embodiment of the edge computing task offloading method for port facility monitoring can be implemented. The specific implementation method can be found in the method embodiment, which will not be repeated here.

[0208] It will be appreciated by those skilled in the art that embodiments of the present invention may provide methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0209] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0210] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0211] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0212] It should be pointed out that the specific implementation methods described above can enable those skilled in the art to understand the invention more comprehensively, but do not limit the invention in any way. Therefore, although the invention has been described in detail in this specification and embodiments, those skilled in the art should understand that the invention can still be modified or replaced by equivalents; and all technical solutions and improvements that do not deviate from the spirit and scope of the invention are included in the protection scope of the patent for the invention. Any figure mark in the claims should not be regarded as limiting the claims involved. Any simple change or equivalent replacement of the technical solution that can be obviously obtained by any technician familiar with the field within the technical scope disclosed in the present invention belongs to the protection scope of the present invention.

Claims

1. An edge computing task offloading method for port facility monitoring, characterized in that: The following steps are involved: Based on the area of ​​the port scenario, the energy supply heterogeneity brought by the deployment environment of the edge device server, and the characteristics of the long-term service monitoring task of the port infrastructure, an edge computing system architecture for long-term service monitoring of the port infrastructure is constructed; According to the edge computing system architecture, the local computing and edge computing in the edge computing system for long-term service monitoring of port infrastructure are modeled, and the delay and energy consumption model of local computing and edge computing is obtained considering the impact of energy supply differences. Based on the latency and energy consumption model, the optimization problem is modeled according to the scenario requirements with the goal of achieving joint optimization of latency and energy consumption; The modeled optimization problem is transformed into a constrained Markov problem. The offloading algorithm based on deep reinforcement learning is used to solve the joint optimization problem of latency and energy consumption, and the computing task offloading strategy is obtained. Offload edge computing tasks based on the offloading strategy.

2. The edge computing task offloading method for port facility monitoring according to claim 1 is characterized in that: The edge computing system architecture for long-term service monitoring of port infrastructure is specifically: Several sensor terminals and several edge servers are deployed in the port scenario, each server is equipped with a single antenna; the edge servers are divided into offshore edge servers and land edge servers.

3. The edge computing task offloading method for port facility monitoring according to claim 2 is characterized in that: The local computing and edge computing in the edge computing system for long-term service monitoring of port infrastructure are modeled according to the edge computing system architecture, and the delay and energy consumption model of local computing and edge computing considering the influence of energy supply differences is obtained, which is specifically: The port infrastructure edge computing system model is constructed as a two-layer edge offloading architecture. There are K sensor terminals for generating tasks, and the sensor terminal set is D{d1,d2,…d k …,d K ,}, where the kth sensor terminal d k The state of the vector express; Indicates the maximum computing frequency of the sensor terminal, where Represents the computing time required for the remaining computing amount. There are I edge servers in total, using the set N{n1,n2,…n i …,n I } indicates that Indicates edge server n i The maximum calculation frequency, Represents edge server n i The computation time required for the current remaining computation; Represents edge server n i The edge server type is When , it means that the edge server belongs to the land server. When the edge server is an offshore server; Time slot Indicates that the generated tasks are represented by a collection Indicates that the task is m(t) k {s(t) k ,c(t) k ,δ(t) k },s(t) k Indicates the computing task data size, c(t) k represents the CPU cycles required to complete the task, δ(t) k Represents the maximum tolerable delay of the computing task; The connection relationship between devices is represented by the offloading matrix G. The number of rows of the offloading matrix G is equal to the number of sensor terminals K, and the number of columns is equal to the number of edge servers I, as shown below: Indicates sensor terminal and edge server n i If the sensor terminal and edge server n i If there is a connection between the sensor terminals The task m generated above k Ability to offload to edge servers i ,at this time Conversely, if the sensor terminal and edge server n i There is no connection between them, and tasks cannot be offloaded between them. where X(t) is the set of task offloading decisions, X(t) = {x(t)1, x(t)2, … x(t) k …,x(t) K },x(t) k is the task offloading decision, x(t) k = 0 means that the task is calculated locally, x(t) k = 1, it means that the task is offloaded to the edge server for calculation; the offloaded object of the task is represented by the set N(t), N(t) = {n(t)1,n(t)2,…n(t) k …,n(t) K }, if x(t) k =0, then the corresponding n(t) k =0, otherwise n(t) k = the corresponding offloading edge server number, the task computing resource allocation is represented by the set F(t). When computing resource allocation is performed, F(t) = {f(t)1, f(t)2, ... f(t) k …,f(t) K }, f(t) k is the edge server assigned to task m(t) in time slot t k computing resources.

4. The edge computing task offloading method for port facility monitoring according to claim 3 is characterized in that: The task calculation is divided into two cases: local calculation and offloading to the edge server for calculation. The two cases are modeled respectively, specifically: Local computational modeling: For task m(t) k , when the unloading decision x(t) k = 0 means that the task is calculated locally, and the total delay of the task is Including the task calculation delay and queuing delay, that is: in is the task m(t) k The computation time required, is the task m(t) k The time required to wait, that is, the task m(t) currently queued on the local device k The sum of the total computing time of the previous tasks to be calculated: Task m(t) k The computational delay for: Where c(t) k is the computational task m(t) k The number of CPU cycles required, f(t) k is assigned to task m(t) in time slot t k computing resources; Total energy consumption of the task Equal to computing energy consumption As shown below: Where λ1 is the energy dissipation coefficient of the sensor terminal; Edge Offloading Modeling: When x(t) k =1, it means that the task is offloaded to the edge server for calculation. At this time, the total delay is composed of transmission delay, calculation delay, queuing delay and result return delay. When task m(t) k The total latency of the task when offloading to the edge server for: in is the transmission task m(t) k The required transmission time, Task m(t) k The computation time required, is the task m(t) k The time required to wait, that is, the task m(t) currently scheduled on the unloading edge server k The sum of the total computing time of the previous tasks to be calculated: Task m(t) k Unload transfer time for: Where s(t) k is the computational task m(t) k The data size, R(t) k is the transmission task m(t) k The transmission rate can be given by: Where W represents the task m(t) k The transmission bandwidth, P(t) k Represents the transmission task m(t) k The transmission power, L(t) k represents the path loss during transmission, and N0 represents the noise power of Gaussian white noise during transmission; Task m(t) k Calculation time for: Where c(t) k is the computational task m(t) k The number of CPU cycles required, f(t) k is the edge server assigned to task m(t) in time slot t k computing resources; Allocated computing resources f(t) k Not exceeding the computing power of the edge server, that is: Task m(t) k Total energy consumption Equal to the transmission energy consumption and calculate energy consumption The sum of the transmission energy consumption for: Calculating energy consumption for: Where λ2 and λ3 are the energy dissipation coefficients of the land server and the offshore server respectively. Considering the characteristics of the land server and the offshore server, λ2>λ3; Task m(t) k The total delay T(t) k It is expressed as: Total energy consumption E(t) k Can be expressed as 5. The edge computing task offloading method for port facility monitoring according to claim 4 is characterized in that: According to the delay and energy consumption model, the optimization problem is modeled according to the scenario requirements with the goal of achieving joint optimization of delay and energy consumption, specifically: st, in, represents the optimization goal, the objective function is to minimize the sum of delay and energy consumption, α′ and β′ are weight coefficients that balance delay and energy consumption; constraint C1 indicates that the task intelligence is calculated locally or offloaded to the edge server for calculation; constraint C2 indicates that the server where the task is offloaded should be able to communicate with the local device of the task; constraint C3 indicates that the computing resources allocated to the task should not exceed the maximum computing frequency of the device; constraint C4 indicates that the total delay of the task should not exceed the tolerable delay of the task.

6. The edge computing task offloading method for port facility monitoring according to claim 5 is characterized in that: The optimization problem after modeling is transformed into a constrained Markov problem, and the delay and energy consumption joint optimization problem is solved using an offloading algorithm based on deep reinforcement learning to obtain a computing task offloading strategy, which is specifically: State modeling: The state consists of the system state and the generated task state. in is the state of the sensor terminal at time t, Indicates the maximum computing frequency of the sensor terminal, where Represents the computation time required for the remaining computation; is the state of the edge server at time t, Indicates that Indicates the maximum computing frequency of the edge server. Represents the computation time required for the remaining computation. Represents the edge server type; are the parameters for generating tasks, Task m(t) k {s(t) k ,c(t) k ,δ(t) k },s(t) k Indicates the computing task data size, c(t) k represents the CPU cycles required to complete the task, δ(t) k Represents the maximum tolerable delay of the computing task; Action modeling: The set of actions that the agent can take is where X(t) is the set of task offloading decisions, X(t) = {x(t)1, x(t)2, … x(t) k …,x(t) K },x(t) k is the task offloading decision, x(t) k = 0 means that the task is calculated locally, x(t) k =1, it means that the task is offloaded to the edge server for calculation; F(t) is the task computing resource allocation set. When computing resources are allocated, F(t) = {f(t)1, f(t)2, …f(t) k …,f(t) K }, f(t) k is the edge server assigned to task m(t) in time slot t k When only offloading decisions are made without computing resource allocation, N(t) is the set of offloaded objects of the task N(t) = {n(t)1,n(t)2,…n(t) k …,n(t) K }, if x(t) k =0, then the corresponding n(t) k =0, otherwise n(t) k = the corresponding offloading edge server number; Reward modeling: define the function S(t) k To indicate whether the task is successfully completed within the tolerable delay, if the total delay T(t) k Less than the task's tolerance delay δ(t) k , then the task is considered to be successfully completed, S(t) k =1; otherwise, if the total delay T(t) k Greater than the task's tolerance delay δ(t) k , then the task is considered to have timed out and discarded, S(t) k =0, that is: The single-step return function is set up as follows: r(t)=-αT(t) k -βlog2E(t) k +γS(t) k (20) where α, β, and γ are parameters that normalize the time cost, energy cost, and profit of completing a task on time to the same scale; Current actions are evaluated by long-term payoffs: where ξ is the discount factor for future rewards.

7. The edge computing task offloading method for port facility monitoring according to claim 6 is characterized in that: The edge computing task is unloaded according to the unloading strategy, specifically: Initialize the number of IoT devices, the number of edge servers, the offloading matrix, the distance matrix, and the path loss matrix, and define the hyperparameters; Initialize the global neural network parameters μ and θ, and start multiple asynchronous training workers, where each worker’s local neural network parameter μ i ,θ i Independent of the random number generator, for each thread Workeri, reset the environment, initialize the device queue time to 0, generate a task at the same time, and complete the generation of the initial state s; different computing power sharing scenarios require the generation of action space AS, initialize the reward and time slot count, and start training; During training, the Actor network calculates the action probability based on the state s, outputs the action a, calculates the time and energy consumption to generate the reward r, and waits until the new state s; finally, the new state s, action a, reward r and next state s are put into the training set for subsequent training of the intelligent agent; when the training set is full, data is extracted to calculate the loss function and update the network parameters until the iteration ends, and the computing task offloading decision and resource allocation plan of the user device terminal are obtained.

8. An edge computing task offloading system for port facility monitoring, characterized in that: include: The system framework building module is used to build an edge computing system architecture for long-term service monitoring of port infrastructure based on the area of ​​the port scenario, the energy supply heterogeneity brought by the edge device server deployment environment, and the characteristics of the long-term service monitoring task of port infrastructure; A joint model building module is used to model the local computing and edge computing in the edge computing system for long-term service monitoring of port infrastructure according to the edge computing system architecture, and obtain the delay and energy consumption model of local computing and edge computing considering the impact of energy supply differences; The problem model building module is used to model the optimization problem based on the latency and energy consumption models and the scenario requirements, with the goal of achieving joint optimization of latency and energy consumption; The offloading strategy building module is used to transform the modeled optimization problem into a constrained Markov problem, and use the offloading algorithm based on deep reinforcement learning to solve the joint optimization problem of latency and energy consumption to obtain the computing task offloading strategy; The offloading module is used to offload edge computing tasks according to the offloading strategy.

9. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

Images