Industrial internet of things complex task computing offloading method, device, equipment and medium

Abstract

The application provides an industrial Internet of Things complex task computing offloading method, device, equipment and medium, and relates to the technical field of industrial Internet of Things task offloading. The method comprises the following steps: S1, acquiring an industrial task set of an industrial Internet of Things system, and constructing a multi-constraint task model of a directed acyclic graph structure according to the dependency relationship between tasks; S2, constructing a total time delay model and a total energy consumption model of the industrial Internet of Things system, and acquiring an optimization target; S3, constructing a Markov decision process according to the optimization target; and S4, regarding an edge server as an intelligent agent by using the Markov decision process, adopting a deep reinforcement learning method based on multi-agent near-end policy optimization based on an observation state, training a model and outputting a final offloading strategy. In the training process, a continuous learning mechanism is introduced.

Description

Technical Field

[0001] This invention relates to the field of task offloading technology in the Industrial Internet of Things (IIoT), and more specifically, to a method, apparatus, device, and medium for offloading complex tasks in the Industrial Internet of Things. Background Technology

[0002] With the rapid development of Industrial Internet of Things (IIoT) technology, modern industrial production systems are entering a period of digital transformation. IIoT connects various industrial devices, sensors, and control systems to achieve interoperability, aiming to enable real-time monitoring, intelligent analysis, and optimized decision-making in the production process. The large-scale interconnection of industrial environments has spurred demand for sophisticated computing. Tasks generated by terminal devices often require millisecond-level response times, and complex dependencies exist between various computing tasks. However, traditional cloud platforms face challenges such as network latency and insufficient bandwidth, making it difficult to meet the real-time processing needs of industry. This highlights the urgent need for efficient, dynamic computation offloading methods.

[0003] To address these needs, mobile edge computing technology has been introduced, significantly improving the computing power of the Industrial Internet of Things (IIoT) by distributing computing tasks to the edge layer near the data source. Simultaneously, deep reinforcement learning is applied as a core scheduling engine. Leveraging the feature extraction capabilities of deep neural networks, it achieves adaptive decision optimization in high-dimensional and complex environments, thereby supporting real-time scheduling responses. Existing research, such as multi-objective deep reinforcement learning algorithms, improved policy gradient methods, and intelligent resource scheduling schemes based on SARSA, has achieved positive results in optimizing latency, energy consumption, and cost of task offloading, demonstrating the potential of reinforcement learning in edge computing scenarios.

[0004] However, existing deep reinforcement learning methods still have significant drawbacks in the dynamic environment of the Industrial Internet of Things (IIoT). Task arrival patterns and resource distribution exhibit significant time-varying properties, easily leading to non-stationary task distribution and making it difficult for models to maintain long-term effectiveness. Furthermore, models are prone to catastrophic forgetting during continuous learning of new tasks, with early knowledge being rapidly overwritten, weakening the long-term robustness and stability of the scheduling system.

[0005] Existing methods mostly focus on multi-objective optimization or dependency handling, but fail to comprehensively consider the balance between complex task dependencies and dynamic environment adaptability, resulting in a decline in the effectiveness of strategies in long-term applications and limiting the overall performance of industrial IoT systems. Summary of the Invention

[0006] The present invention provides a method, apparatus, device and medium for offloading complex task computations in the Industrial Internet of Things (IIoT) to improve at least one of the above-mentioned technical problems.

[0007] Firstly, the present invention provides a method for offloading complex task computations in the Industrial Internet of Things, comprising steps S1 to S4.

[0008] S1. Obtain the set of industrial tasks from the Industrial Internet of Things (IIoT) system, and construct a multi-constraint task model with a directed acyclic graph structure based on the dependencies between tasks.

[0009] S2. Construct the total latency model and total energy consumption model of the industrial IoT system to obtain the optimization target.

[0010] S3. Construct a Markov decision process based on the optimization objective.

[0011] S4. Using a Markov decision process, the edge server is treated as an agent. Based on the observed state, a deep reinforcement learning method based on multi-agent proximal policy optimization is used to train the model and output the final offloading policy. A continuous learning mechanism is introduced during the training process.

[0012] Secondly, the present invention provides an industrial Internet of Things (IoT) complex task computation offloading device, which includes a graph construction module, a target construction module, a framework construction module and a solution module.

[0013] The graph construction module is used to obtain the set of industrial tasks of the Industrial Internet of Things system and construct a multi-constraint task model with a directed acyclic graph structure based on the dependencies between tasks.

[0014] The target building module is used to construct the total latency model and total energy consumption model of the industrial IoT system and obtain the optimization target.

[0015] The framework building module is used to construct a Markov decision process based on the optimization objective.

[0016] The solution module utilizes a Markov decision process, treating the edge server as an agent, and trains the model using a deep reinforcement learning method based on multi-agent proximal policy optimization, based on the observed state, to output the final offloading policy. The training process incorporates a continuous learning mechanism.

[0017] Thirdly, the present invention provides an industrial Internet of Things (IIoT) complex task computation offloading device, which includes a processor, a memory, and a computer program stored in the memory. The computer program can be executed by the processor to implement an IIoT complex task computation offloading method as described in any paragraph of the first aspect.

[0018] Fourthly, the present invention provides a computer-readable storage medium. The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device containing the computer-readable storage medium to perform a method for offloading complex tasks in the Industrial Internet of Things as described in any paragraph or claim of the first aspect.

[0019] By adopting the above technical solution, the present invention can achieve the following technical effects:

[0020] This invention effectively addresses the challenges of poor dynamic adaptability and insufficient long-term stability in task offloading within the Industrial Internet of Things (IIoT) environment by organically combining complex task dependency modeling, multi-agent reinforcement learning, and continuous learning mechanisms. The method utilizes task modeling and topology sorting based on directed acyclic graphs (DAGs) to accurately characterize and dynamically schedule industrial tasks with strong temporal constraints and complex dependencies. By introducing continuous learning memory replay and hierarchical sampling strategies, it significantly alleviates the catastrophic forgetting problem inherent in traditional deep reinforcement learning when the environment changes, thus ensuring the robustness and continuous optimization capability of the scheduling strategy in long-term operation. Simultaneously, by employing MAPPO-based multi-agent collaborative decision-making and a soft-time-window reward design, it achieves comprehensive optimization of latency and energy consumption while meeting industrial real-time constraints, improving the overall efficiency of task offloading and the system's ability to adapt to dynamic resource changes. Ultimately, it provides an efficient, stable, and continuously learning-capable intelligent offloading solution for complex computing tasks in IIoT scenarios. Attached Figure Description

[0021] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the specific embodiments of the present invention will be briefly introduced below. It should be understood that the following drawings only show some specific embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating a method for offloading complex tasks in the Industrial Internet of Things (IIoT).

[0023] Figure 2 This is a schematic diagram of the system architecture for a method of computationally unloading complex tasks in the Industrial Internet of Things (IIoT).

[0024] Figure 3 This is a detailed flowchart of the computational offloading method for complex tasks in the Industrial Internet of Things (IIoT). Detailed Implementation

[0025] The technical solutions of the present invention will now be clearly and completely described with reference to the accompanying drawings in the embodiments of the present invention.

[0026] Example 1, please refer to Figures 1 to 3The first embodiment of this invention provides a method for offloading complex tasks in the Industrial Internet of Things (IIoT). Based on the MAPPO algorithm and incorporating a continuous learning mechanism, it solves the problems of catastrophic forgetting and insufficient adaptability in traditional DRL in dynamic environments, achieving efficient and stable task offloading. The MAPPO algorithm is a multi-agent proximal policy optimization algorithm.

[0027] See Figure 2 As shown in this embodiment, the system architecture model is a three-layer model, consisting of local terminal devices, edge servers, and cloud servers. During operation, the industrial production line continuously generates various computing tasks. These tasks generally have strict dependencies. To meet real-time requirements while considering system energy consumption, the generated tasks are transmitted via data transmission and offloaded to edge servers or remote cloud servers for execution according to a scheduling strategy. Edge servers can quickly handle latency-sensitive tasks, while cloud servers are suitable for handling computationally complex tasks. After task allocation and computation are completed, the execution results and corresponding optimization feedback information are transmitted back to the local terminal devices. This enables real-time monitoring, dynamic allocation, and comprehensive optimization of industrial production steps, ensuring stable system operation under complex dependencies and dynamic environments.

[0028] See Figure 1 As shown, the industrial IoT complex task computation offloading method of this embodiment can be executed by an industrial IoT complex task computation offloading device (hereinafter referred to as: task offloading device). In particular, it is executed by one or more processors in the task offloading device to implement the following steps.

[0029] S1. Obtain the set of industrial tasks from the industrial IoT system, and construct a multi-constraint task model with a directed acyclic graph (DAG) structure based on the dependencies between tasks. Preferably, the industrial IoT system includes local terminal devices that generate tasks, edge servers deployed at the edge, and cloud servers.

[0030] Based on the above embodiments, in an optional embodiment of the present invention, modeling the tasks within the factory production line to obtain a multi-constraint task model specifically includes steps S11 to S12.

[0031] S11. Obtain the industrial task set Treat each task as a node in a graph, identify the dependencies between tasks, and if tasks... Execution depends on the task The output result then establishes directed edges in the task graph. This allows us to construct a task dependency graph. .in A collection of industrial tasks. For the task. The number of industrial tasks. This is the first index. This is the second index. A set of directed edges representing task dependencies.

[0032] S12. At each scheduling time, execute the topology sorting algorithm based on the graph structure to identify the set of all tasks with an in-degree of zero.

[0033] .

[0034] In the formula For the ready task set, This is the set of completed tasks. The set of ready tasks is used for subsequent scheduling action selection.

[0035] Step S1 achieves efficient dynamic scheduling of complex hybrid tasks by constructing a task dependency graph and performing topological sorting on the tasks.

[0036] The following example illustrates the "automatic self-inspection" process of a robotic arm in a smart factory.

[0037] This self-inspection process comprises five tasks, forming an industrial task set. ,in, This indicates system initialization and sensor calibration. This indicates a self-check of the mechanical structure. This indicates a self-test of the electrical system. Indicates to and The results were analyzed comprehensively. This indicates that the final self-inspection report has been generated and uploaded.

[0038] Construct a directed task dependency graph based on the logical dependencies between tasks. The set of edges The graphical model clearly illustrates the task. and There is no direct dependency between them, and they can be executed in parallel. (Task) Execution depends on and The task is complete. The execution depends on The completion of.

[0039] At each scheduling moment, the system executes a topological sorting algorithm based on the graph structure to dynamically identify the set of ready tasks with an in-degree of zero. The scheduling process is as follows.

[0040] Initial state phase, set of completed tasks The ready task set was identified through topological sorting. The scheduler selects to execute .Task After completion, The system recalculates the task in-degree and identifies a new set of ready tasks. At this point, the scheduler can separately... and By allocating resources to different computing units for parallel execution, efficiency is significantly improved. and All have reached the completion status. The system updates the status graph and identifies ready tasks. The scheduler begins execution. .Task After completion, Finally, the ready tasks were identified. The scheduler executes the task. To complete the entire process.

[0041] S2. Construct the total latency model and total energy consumption model of the industrial IoT system to obtain the optimization objective. The optimization objective is the system optimization model. Preferably, step S2 includes steps S21 to S25.

[0042] S21. Obtain the local execution latency of the task during local computation. and local execution energy consumption .

[0043] .

[0044] .

[0045] In the formula It is to perform tasks Total number of cycles required. It is the number of CPU cycles per second for the local device. This refers to the execution power of the local device.

[0046] S22. Edge execution latency when the task is offloaded to the edge server and edge execution energy consumption .

[0047] .

[0048] .

[0049] In the formula Indicates task and Dependency relationship between them, if ,but Otherwise, it is 0. For the task Pre-transfer data volume. For the task Edge transmission channel bandwidth. This represents the number of CPU cycles per second for the edge server. This refers to the idle power of the equipment.

[0050] S23. Cloud execution latency when the task is unloaded to the cloud server. Energy consumption of cloud execution .

[0051] .

[0052] .

[0053] In the formula For the task Cloud transmission channel bandwidth. This refers to the number of CPU cycles per second for the cloud server.

[0054] Edge execution latency consists of transmission latency and execution latency. Cloud execution latency consists of transmission latency and execution latency.

[0055] S24. Obtain system latency and system energy consumption .

[0056] .

[0057] .

[0058] S25. Obtain the optimization target.

[0059] .

[0060] .

[0061] .

[0062] In the formula For time delay constraints. Energy consumption constraints.

[0063] S3. Based on the optimization objective, construct a Markov decision process. Specifically, model the computational offloading problem of complex tasks as a Markov decision process. This defines the state space, action space, and reward function required for reinforcement learning. This step provides the mathematical framework for subsequent solutions.

[0064] Preferably, step S3 includes steps S31 to S33.

[0065] S31, will The information observed by the agent at any given moment is used in the state space. Represented as:

[0066] .

[0067] In the formula, This indicates task dependencies. This indicates that the task is ready. This indicates the node's CPU utilization. Indicates the status of network resources. For time delay constraints. Energy consumption constraints;

[0068] S32, Move space Represented as:

[0069] .

[0070] In the formula, For task unloading strategy, 0 indicates the task is executed locally, 1 to 10 indicate the task is executed locally. Indicates the corresponding edge node. This indicates execution in the cloud.

[0071] S33, The reward function is based on the task completion time. Based on this, normalization penalties are introduced for latency and energy consumption, and a soft time window is used to assign adjustable weights to smoothing penalties for early and late completion, thereby achieving comprehensive optimization of latency and energy consumption while satisfying the soft time window constraints.

[0072] .

[0073] .

[0074] In the formula This is the reward function. For time delay dynamic weight parameters. This is the dynamic weighting parameter for energy consumption. The penalty is a dynamic weight parameter. This refers to system latency. This refers to the system's energy consumption. This is the penalty function for a soft time window. This is the initial penalty coefficient. This is a buffer window. This represents the penalty coefficient for severe timeouts.

[0075] S4. Using a Markov decision process, the edge server is treated as an agent. Based on the observed state, a deep reinforcement learning method based on multi-agent proximal policy optimization (MAPPO algorithm) is used to train the model and output the final offloading policy. A continuous learning mechanism is introduced during the training process.

[0076] Specifically, all information is stored in an experience replay pool and used to train a multi-agent reinforcement learning model, introducing a continuous learning mechanism. This not only overcomes the problems of catastrophic forgetting and insufficient adaptability in dynamic environments, but also ensures the continuous stability of task unloading.

[0077] Preferably, step S4 includes the following steps.

[0078] S401, Define Policy Network and value network .

[0079] .

[0080] .

[0081] In the formula These are the parameters of the neural network. For a moment The system status. For a moment The action chosen.

[0082] S402, Define the memory replay buffer for continuous learning. .

[0083] .

[0084] In the formula, This is empirical data. For a moment The reward. This is the source of the task.

[0085] S403. Initialize the policy network and value network. The policy network obtains the current system state from the environment. The original scores for each action are calculated using a multi-layer fully connected network. The scores are then converted into a probability distribution using the Softmax function, ensuring that the sum of the probabilities of all actions is 1.

[0086] S404. Randomly select an action using a sampling method. .

[0087] S405, Select the action The code is passed to the environment for execution and returns the corresponding reward. and update the status to The experience data generated during the interaction process is stored in the continuous learning memory replay buffer. . This is new experience data.

[0088] S406. Introduce timing difference error as a quantitative indicator for empirical priority. :

[0089] .

[0090] .

[0091] In the formula This represents the timing difference error at the current moment. To prevent values ​​from being 0. This is the discount factor. This is the parameterization function for the value network.

[0092] S407. Add a hierarchical sampling mechanism based on task identifiers:

[0093] .

[0094] .

[0095] In the formula This is the training batch for this training exercise. A subset sampled from the experience of the current task. A subset sampled from experience of previous tasks.

[0096] S408, Introducing the Standardized Advantage Function Regarding the original dominance value Standardization is performed to obtain the normalized dominance value. This is the standardized advantage value. This represents the average of the current batch's advantage values. This represents the standard deviation of the current batch's advantage value.

[0097] S409, Define the probability ratio between the old and new strategies. It is used to quantify the magnitude and direction of strategy updates.

[0098] .

[0099] In the formula These are the parameters of the neural network. This is a new strategy. This is the old strategy.

[0100] Indicating a new strategy Next system status When selecting an action The probability of. Similarly.

[0101] S410. Define the policy loss function based on the standardized advantage function and probability ratio. .

[0102] .

[0103] In the formula For the desired operation. It is a minimum value function. This is the shearing function.

[0104] The operation will limit the strategy ratio to Within the range. Specifically: when the performance is better than expected ( Furthermore, the new strategy favors this action. Increase the probability of the action. When the action performance is worse than expected ( And the new strategy reduces the probability of selection. When ), reduce the probability of the action.

[0105] S411. Introducing the mean square error function To calculate the loss of value:

[0106] .

[0107] In the formula This is the value network's prediction of the current state. For actual returns.

[0108] S412. Introducing the entropy regularization term This is used to prevent the action probability from converging to a local optimum too early.

[0109] .

[0110] S413. Introducing a regularization term: Knowledge distillation loss and gradient penalty This is to mitigate the catastrophic forgetting problem in continuous learning. To prevent the formula from becoming too long, a definition is provided. for The abbreviation of . for The abbreviation of .

[0111] .

[0112] .

[0113] In the formula This represents the expected value based on past experience. Let KL divergence be denoted as KL divergence. The gradient of the policy loss function of the new policy network parameters. It is the square of the L2 norm.

[0114] S414. Obtain the final loss function. .

[0115] .

[0116] In the formula for The weight. for The weight. for The weight. for The weight.

[0117] S415. Repeat the above steps and obtain the final action set after the iteration is completed.

[0118] See Figure 3 As shown in the figure, in this embodiment, a computational offloading method for complex tasks in the industrial Internet of Things (IIoT) environment based on MAPPO is proposed to address the computational offloading problem. The specific process is as follows.

[0119] First, a policy network and a value network are defined for the multi-agent system. The policy network generates the probability distribution of actions, and the value network estimates the state value. Simultaneously, a memory replay buffer is built to support continuous learning, and a task identifier is added to each piece of experience data to facilitate subsequent hierarchical sampling and mitigate catastrophic forgetting.

[0120] During the initialization phase, the policy network obtains the current system state from the environment, calculates the raw scores for each action using a multi-layer neural network, and transforms them into a probability distribution using a normalization function to ensure that the sum of the probabilities of each action equals 1. An action is randomly selected using a sampling method and passed to the environment for execution. The environment provides a corresponding reward and updates its state to reflect the system state for the next time step. The experience data generated through this interaction is stored in a memory replay buffer.

[0121] To optimize the efficiency of experience utilization and the ability to continuously learn, this embodiment introduces temporal difference error as an experience priority indicator. Based on this indicator, the importance of samples is measured. Combined with task identification information, a hierarchical sampling mechanism is designed: current and historical task experiences are sampled simultaneously at a certain ratio to achieve a balance between learning new tasks and preserving knowledge from old tasks.

[0122] In the process of policy optimization, the advantage function is first standardized to obtain a normalized advantage value, preventing training instability caused by scale differences. Simultaneously, a probability ratio between the old and new policies is defined to measure the magnitude of policy updates. Based on this, a pruning-form policy loss function is constructed to prevent instability caused by excessively large step sizes while ensuring effective policy updates.

[0123] In the process of value optimization, the mean squared error function is used to calculate the value loss, and an entropy regularization term is added to the loss function to enhance the ability of policy exploration.

[0124] In addition, to alleviate the catastrophic forgetting problem in continuous learning, this embodiment introduces regularization constraints, combining the knowledge distillation loss of old task strategy snapshots with the penalty term of gradient magnitude adjustment, thereby maintaining the memory of existing knowledge during version iteration.

[0125] Finally, the overall loss function is composed of a weighted combination of policy loss, value loss, entropy regularization term, and continuous learning regularization term. This hybrid loss function is optimized to achieve synergistic optimization of the policy network and the value network. After multiple progressive iterative training iterations, this embodiment can generate the optimal offloading strategy for complex tasks in industrial IoT scenarios, ensuring rapid task assignment and effective implementation, significantly suppressing knowledge forgetting, and improving the overall system performance and robustness.

[0126] Example 2: The present invention provides an industrial Internet of Things (IoT) complex task computation offloading device, which includes a graph construction module, a target construction module, a framework construction module and a solution module.

[0127] The graph construction module is used to obtain the set of industrial tasks of the Industrial Internet of Things system and construct a multi-constraint task model with a directed acyclic graph structure based on the dependencies between tasks.

[0128] The target building module is used to construct the total latency model and total energy consumption model of the industrial IoT system and obtain the optimization target.

[0129] The framework building module is used to construct a Markov decision process based on the optimization objective.

[0130] The solution module utilizes a Markov decision process, treating the edge server as an agent, and trains the model using a deep reinforcement learning method based on multi-agent proximal policy optimization, based on the observed state, to output the final offloading policy. The training process incorporates a continuous learning mechanism.

[0131] Example 3: This invention provides an industrial IoT complex task computation offloading device, which includes a processor, a memory, and a computer program stored in the memory. The computer program can be executed by the processor to implement an industrial IoT complex task computation offloading method as described in any paragraph of Example 1.

[0132] It is understood that the industrial IoT complex task computing offloading device (i.e., task offloading device) can be an electronic device with computing performance, such as a portable laptop computer, desktop computer, server, smartphone or tablet computer.

[0133] Example 4: This invention provides a computer-readable storage medium. The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform a complex task computation offloading method for the Industrial Internet of Things as described in any paragraph or item of Example 1.

[0134] Obviously, the embodiments described above are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0135] In the several embodiments provided in this invention, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus and method embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0136] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0137] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, electronic device, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks. It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0138] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention are also intended to include the plural forms unless the context clearly indicates otherwise.

[0139] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0140] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0141] The terms "first" and "second" used in the embodiments are merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" can be interchanged in a specific order or sequence where permitted. It should be understood that the objects distinguished by "first" and "second" can be interchanged where appropriate so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0142] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for offloading complex tasks in the Industrial Internet of Things (IIoT), characterized in that, Include: S1. Obtain the set of industrial tasks of the industrial Internet of Things system, and construct a multi-constraint task model with a directed acyclic graph structure based on the dependencies between tasks. S2. Construct the total latency model and total energy consumption model of the industrial IoT system to obtain the optimization objective; S3. Construct a Markov decision process based on the optimization objective. S4. Using Markov decision process, the edge server is regarded as an agent. Based on the observation state, a deep reinforcement learning method based on multi-agent proximal policy optimization is adopted to train the model and output the final offloading policy. Among them, the training process introduces a continuous learning mechanism. S4 includes the following steps; Define policy network and value network ; ； ； In the formula For neural network parameters; For a moment The system status; For a moment The chosen action; Define a memory replay buffer for continuous learning ; ； In the formula, This is empirical data; For a moment The reward; For the source of the task; Initialize the policy network and value network; the policy network obtains the current system state from the environment. The original scores of each action are calculated through a multi-layer fully connected network, and the scores are converted into a probability distribution using the Softmax function to ensure that the sum of the probabilities of all actions is 1. A random action is selected using a sampling method. ; Select the action The code is passed to the environment for execution and returns the corresponding reward. and update the status to The experience data generated during the interaction process is stored in the continuous learning memory replay buffer. ; For new experience data; Introducing time-series difference error as a quantitative indicator of empirical priority. : ； ； In the formula This represents the timing difference error at the current moment; To prevent values ​​from being 0; Discount factor; Add a hierarchical sampling mechanism based on task identifiers: ； ； In the formula This refers to the training batch for this training session; A subset sampled from the experience of the current task; A subset sampled from experience of previous tasks; Introducing a standardized advantage function Regarding the original dominance value Standardization is performed to obtain the normalized dominance value; where This is the standardized advantage value; This represents the average of the current batch's advantage values; This represents the standard deviation of the current batch's advantage value. Define the probability ratio between the old and new strategies. This is used to quantify the magnitude and direction of strategy updates; ； In the formula For neural network parameters; For the new strategy; This is the old strategy.

2. The method for offloading complex tasks in the Industrial Internet of Things according to claim 1, characterized in that, definition: The number of industrial tasks; As the first index; For the second index; S2 includes the following steps; Get the local execution latency of the task when it is computed locally. and local execution energy consumption ; ； ； In the formula It is to perform tasks Total number of cycles required; It is the number of CPU cycles per second for the local device; The execution power of the local device; Edge execution latency when the task is offloaded to the edge server and edge execution energy consumption ; ； ； In the formula Indicates task and Dependency relationship between them, if ,but Otherwise, it is 0; For the task Pre-transmission data volume; For the task Edge transmission channel bandwidth; This represents the number of CPU cycles per second for the edge server. This refers to the idle power of the equipment. Get the cloud execution latency when the task is unloaded to the cloud server Energy consumption of cloud execution ; ； ； In the formula For the task Cloud transmission channel bandwidth; This refers to the number of CPU cycles per second for the cloud server. Get system latency and system energy consumption ; ； ； The optimization objective is to obtain the following: ； ； ； In the formula For time delay constraints; Energy consumption constraints.

3. The method for offloading complex tasks in the Industrial Internet of Things according to claim 1, characterized in that, S3 includes the following steps; Will The information observed by the agent at any given moment is used in the state space. Represented as: ； In the formula, Indicates task dependencies; Indicates that the task is ready; Indicates node CPU utilization; Indicates the status of network resources; For time delay constraints; Energy consumption constraints; Action space Represented as: ； In the formula, For task unloading strategy, 0 indicates the task is executed locally, 1 to 10 indicate the task is executed locally. Indicates the corresponding edge node. Indicates execution in the cloud; The number of industrial tasks; As the first index; The reward function is based on the task completion time. Based on this, normalization penalties are introduced for latency and energy consumption, and a soft time window is used to assign adjustable weights to smooth penalties for early and late completion, thereby achieving comprehensive optimization of latency and energy consumption while satisfying the soft time window constraints. ； ； In the formula For the reward function; For time delay dynamic weight parameters; This is a dynamic weighting parameter for energy consumption; For the penalty dynamic weight parameter; For system latency; This refers to system energy consumption; This is a penalty function for a soft time window; This is the initial penalty coefficient; For buffer windows; This represents the penalty coefficient for severe timeouts.

4. The method for offloading complex tasks in the Industrial Internet of Things according to claim 1, characterized in that, S4 also includes the following steps; Define the policy loss function based on the standardized advantage function and probability ratio. ; ； In the formula For the desired operation; It is a minimum value function; It is the shearing function; Introducing the mean square error function To calculate the loss of value: ； In the formula This represents the value network's prediction of the current state. For actual returns; Introducing an entropy regularization term ; ； Introducing a regularization term: knowledge distillation loss and gradient penalty ; ； ； In the formula For the expected value regarding past experience; Let KL divergence be a metric. The gradient of the policy loss function of the new policy network parameters; The square of the L2 norm; Obtain the final loss function ; ； In the formula for The weights; for The weights; for The weights; for The weights; Repeat the above steps, and obtain the final set of actions after the iteration is complete.

5. The method for offloading complex tasks in the Industrial Internet of Things according to claim 1, characterized in that, S1 includes the following steps; Obtain industrial task set Treat each task as a node in a graph, identify the dependencies between tasks, and if tasks... Execution depends on the task The output result then establishes directed edges in the task graph. This allows us to construct a task dependency graph. ;in A collection of industrial tasks; For the task; The number of industrial tasks; As the first index; For the second index; A set of directed edges representing task dependencies; At each scheduling time, a topological sorting algorithm is executed based on the graph structure to identify the set of all tasks with an in-degree of zero. In the formula For the ready task set, This is a collection of completed tasks.

6. The method for offloading complex tasks in the Industrial Internet of Things according to claim 1, characterized in that, An industrial IoT system includes local terminal devices that generate tasks, edge servers deployed at the edge, and cloud servers.

7. An industrial Internet of Things (IIoT) complex task computation offloading device, characterized in that, Suitable for performing an industrial IoT complex task computation offloading method as described in any one of claims 1 to 6; the industrial IoT complex task computation offloading device comprises: The graph construction module is used to obtain the set of industrial tasks of the industrial Internet of Things system and construct a multi-constraint task model with a directed acyclic graph structure based on the dependencies between tasks. The target building module is used to construct the total latency model and total energy consumption model of the industrial IoT system to obtain the optimization target. A framework building module is used to construct a Markov decision process based on the optimization objective. The solution module utilizes Markov decision processes to treat edge servers as agents. Based on the observed states, it employs a deep reinforcement learning method based on multi-agent proximal policy optimization to train the model and output the final offloading policy. The training process incorporates a continuous learning mechanism.

8. An industrial Internet of Things (IIoT) complex task computation and unloading device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory; the computer program can be executed by the processor to implement a method for offloading complex tasks in the Industrial Internet of Things as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform a complex task computation offloading method for industrial Internet of Things as described in any one of claims 1 to 6.

Images