Distributed task unloading method and system based on fault perception and reinforcement learning

Abstract

The invention relates to the technical field of edge computing, and provides a distributed task unloading method and system based on fault perception and reinforcement learning, and the method comprises the steps: obtaining an unloading task for unloading a task to an edge node by an Internet of Things terminal user with an autonomous decision right; constructing a link number control method based on reinforcement learning, and adaptively deciding the optimal number of unloading nodes of the unloading task according to the link number control method based on reinforcement learning; selecting a target node with the optimal transmission performance by circularly calculating and comparing the expected transmission time of each candidate link according to the optimal number of unloaded nodes, and establishing a target transmission link according to the target node; and dynamically adjusting the target transmission link according to the congestion degree of the transmission link. According to the method, the overall efficiency and robustness of task unloading of the Internet of Things are effectively improved, the method is adaptive to a large-scale dynamically changing Internet of Things network environment, and a more efficient and reliable technical support is provided for terminal task unloading under edge computing.

Description

Technical Field

[0001] This invention relates to the field of edge computing technology, and in particular to a distributed task offloading method and system based on fault perception and reinforcement learning. Background Technology

[0002] With the proliferation of IoT devices, such as wearables and smartphones, a large number of compute-intensive and latency-sensitive applications have emerged, including facial recognition and augmented reality. Mobile edge computing, with its powerful computing capabilities and ultra-low latency, has become one of the ideal solutions for handling these applications with high network bandwidth requirements. However, compute-intensive and latency-sensitive applications typically operate in wireless network environments. Due to the dynamic nature of wireless environments, their transmission links are unstable, exacerbating competition between edge terminals and potentially leading to node overload, ultimately reducing resource utilization.

[0003] To reduce terminal contention and improve resource utilization, leveraging collaboration among edge nodes to enhance task offloading efficiency is a viable approach. This primarily includes centralized and distributed methods. Centralized methods require global knowledge of the system state to make task offloading decisions for each task in the edge network. However, for rapidly changing information, such as environmental information, task information, and the workload of edge nodes, up-to-date global knowledge would require prohibitively high transmission overhead, which is impractical in large-scale networks. Traditional distributed heuristics are based on greedy optimization, which may lead to suboptimal results in the long run. Reinforcement learning methods, optimized based on traditional methods, can adaptively make task offloading decisions based on the dynamic environment, aiming to maximize the discounted cumulative reward, thus avoiding suboptimal problems. Compared to centralized and traditional distributed methods, this represents a potential technology applicable to large-scale networks. However, when existing reinforcement learning methods are directly applied to select offloading strategies, they suffer from large policy spaces and slow algorithm convergence speeds.

[0004] Nodes in mobile edge clouds often exhibit high heterogeneous failure and recovery rates due to harsh environments. Frequent failures can interfere with task data transmission and processing, significantly increasing the risk of task failure. Existing research primarily optimizes task offloading efficiency in ideal, fault-free environments, improving performance but proving difficult to deploy in practice. To address this, passive fault tolerance mechanisms are introduced to improve task success rates in faulty environments; however, this strategy is reactive, and the losses caused by failures are often irreversible. Active fault tolerance aims to mitigate the impact of failures before they occur. However, these methods are often based on oversimplified models, ignoring key characteristics of real-world networks, resulting in unresolved task offloading issues in truly fault-prone scenarios.

[0005] Therefore, the following challenges are faced when designing distributed task offloading algorithms: How can users perceive and dynamically adapt to fluctuations in the wireless environment to maximize task transmission rate? How can users make effective decisions when wireless links interfere with each other and lack information about other people's policies (such as due to privacy restrictions)? How can users efficiently find the optimal strategy when the policy space is large (such as the combination of node selection and power allocation) and policy switching is limited? How can users design offloading strategies to reduce the impact of potential long-term failures of edge nodes?

[0006] Given the above considerations, current research has introduced reinforcement learning into the task offloading domain, enabling agents to autonomously learn optimal policies in environments where rewards are unknown. However, these methods often face privacy concerns due to policy sharing, or problems such as excessively large policy spaces and slow convergence caused by balancing factors like power allocation. Attempts have been made to protect privacy through non-shared policies, but the core challenge of low decision-making efficiency remains unresolved. Summary of the Invention

[0007] This invention aims to at least solve one of the technical problems existing in related technologies. To this end, this invention provides a distributed task offloading method and system based on fault perception and reinforcement learning. The method involves using a reinforcement learning agent to determine the optimal number of offloading edge nodes, employing an ε-greedy strategy for action selection and Q-value updates, selecting edge nodes to establish transmission links based on expected transmission time, and considering the impact of edge node failure rates. The method dynamically adjusts the number of links based on network congestion levels, removing inefficient and redundant links. This invention decouples link quantity decisions from edge node selection, reduces the dimensionality of the policy space, improves convergence speed, and comprehensively considers edge node failures and network congestion, significantly improving task offloading success rate and system transmission efficiency.

[0008] This invention provides a distributed task offloading method based on fault perception and reinforcement learning, comprising: S1: Obtain the unloading task from IoT terminal users with autonomous decision-making power to unload tasks to edge nodes; S2: Construct a link number control method based on reinforcement learning, and adaptively determine the optimal number of unloading edge nodes for the unloading task based on the link number control method based on reinforcement learning. S3: Construct a fault-aware expected transmission time selection model, and use the fault-aware expected transmission time selection model to calculate the expected transmission time of each candidate link. S4: Compare the expected transmission times of each candidate link and select the edge node with the best transmission performance to obtain the target offloading edge node; S5: Apply steps S3 and S4 iteratively based on the optimal number of offloading edge nodes to obtain all target offloading edge nodes, and establish the target transmission link based on all target offloading edge nodes. S6: Dynamically adjust the target transmission link based on the congestion level of the transmission link.

[0009] Furthermore, the unloading task is an indivisible minimum unloading unit, and the goal of the unloading task is to maximize the actual transmission rate within the runtime.

[0010] Furthermore, the fault-aware expected transmission time selection model selects the most suitable edge node by calculating the expected transmission time of the task. The expected transmission time is the conditional expectation under two scenarios during edge node transmission: failure and successful transmission. It is determined by the task data volume, the current channel transmission rate, and the historical average transmission rate. The calculation expression is as follows: in, For tasks to edge nodes Expected transmission time For edge nodes The expectation of a failure during transmission. For edge nodes Expectations for successful transmission For the amount of task data, For edge nodes Based on the expected estimated channel transmission rate, For average transmission rate, For edge node failures related to time slots The probability density distribution function.

[0011] Furthermore, reinforcement learning-based link number control methods include: S21: Model the user equipment as a reinforcement learning agent; S22: The agent's action is defined as selecting the maximum number of links, the agent's state is defined as the influence of the joint policies of other user equipment, and is indirectly represented by channel state information. The reward function is the total number of bits transmitted between two decision intervals.

[0012] Furthermore, an ε-greedy strategy is adopted for action selection. The agent selects the action with the highest current Q value with a probability of 1-ε, or explores randomly with a probability of ε. It obtains immediate rewards and the maximum Q value of the next state, updates the Q table using the temporal difference method, and gradually learns the optimal configuration of the number of edge nodes under different network states. The maximum number of links is established based on the optimal configuration of the number of edge nodes.

[0013] Furthermore, the agent's state is divided into a less congested state and a more congested state based on the channel load. A less congested state is when the current channel quality is better than the historical average. A relatively congested state is defined as the current channel quality being lower than or equal to the historical average.

[0014] Furthermore, channel quality is reflected by aggregated information, and smoothed channel information is obtained by exponential weighted averaging.

[0015] Furthermore, aggregated information is defined as: in, For time slots user edge nodes Aggregated information, For time slots user edge nodes Channel gain, Background noise power, For time slots user edge nodes Channel gain, For time slots user edge nodes Transmission power, In addition to users The collection of all other users besides [the user group].

[0016] Furthermore, the expression for calculating channel information is as follows: in, For time slots user edge nodes Channel information, For channel weights, For time slots user edge nodes Channel information, For time slots user edge nodes Aggregated information between them.

[0017] This invention also provides a distributed task offloading system based on fault perception and reinforcement learning, for executing the aforementioned distributed task offloading method based on fault perception and reinforcement learning, comprising: The task acquisition module acquires the unloading tasks of IoT terminal users with autonomous decision-making power to unload tasks to edge nodes; The optimal node number decision module constructs a link number control method based on reinforcement learning, and adaptively determines the optimal number of unloading edge nodes for the unloading task based on the link number control method based on reinforcement learning. The expected transmission time calculation module constructs an expected transmission time selection model based on fault perception and uses the expected transmission time selection model based on fault perception to calculate the expected transmission time of each candidate link. The target node selection module compares the expected transmission time of each candidate link and selects the edge node with the best transmission performance to obtain the target offloading edge node. The target transmission link construction module applies steps S3 and S4 iteratively according to the optimal number of offload edge nodes to obtain all target offload edge nodes, and establishes the target transmission link based on all target offload edge nodes. A dynamic adjustment module that dynamically adjusts the target transmission link based on the congestion level of the transmission link.

[0018] The above-described one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects: This invention utilizes a reinforcement learning agent to determine the optimal number of offloaded edge nodes, employing an ε-greedy strategy for action selection and Q-value updates. Edge nodes are selected to establish transmission links based on expected transmission time, taking into account the impact of edge node failure rates. The number of links is dynamically adjusted according to network congestion levels, removing inefficient and redundant links. This invention decouples link quantity decision-making from edge node selection, reducing the dimensionality of the policy space and improving convergence speed. Simultaneously, by comprehensively considering edge node failures and network congestion, it significantly improves task offloading success rate and system transmission efficiency.

[0019] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating a distributed task offloading method based on fault perception and reinforcement learning provided by the present invention.

[0022] Figure 2This refers to the actual transmission rate under the Rice and Rayleigh distributions provided in the embodiments of the present invention.

[0023] Figure 3 This refers to the number of tasks completed under the Rice and Rayleigh distributions provided in this embodiment of the invention.

[0024] Figure 4 This is a schematic diagram of the structure of a distributed task offloading system based on fault perception and reinforcement learning provided by the present invention.

[0025] Figure label: 101. Task Acquisition Module; 102. Optimal Node Quantity Decision Module; 103. Expected Transmission Time Calculation Module; 104. Target Node Selection Module; 105. Target Transmission Link Construction Module; 106. Dynamic Adjustment Module. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention. The following embodiments are used to illustrate this invention but cannot be used to limit the scope of this invention.

[0027] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0028] The following is combined with Figures 1 to 4 This invention describes a distributed task offloading method and system based on fault perception and reinforcement learning.

[0029] like Figure 1 As shown, a distributed task offloading method based on fault perception and reinforcement learning includes: S1: Obtain the unloading task from IoT terminal users with autonomous decision-making power to unload tasks to edge nodes; For IoT terminal users with autonomous decision-making power, offloading computing tasks to edge nodes is a viable solution. Given that centralized decision-making is inefficient as the scale of terminals expands, and users are unwilling to relinquish scheduling authority due to privacy and security concerns, coupled with the improvement of the terminal's own capabilities, the realization of distributed autonomous decision-making is jointly promoted.

[0030] Task unloading efficiency is measured by the number of tasks that are successfully unloaded within a specified time and the actual data transmission rate of the tasks. Multiple user equipment (UE) and A task offloading system consisting of edge nodes (ENs). , ,in, For the first end user, For the second end user, For the first One end user, As the first edge node, For the second edge node, For the first There are several edge nodes, and the task offloading system operates in discrete time slots, with each time slot having a length of [length missing]. time slot The power allocation strategy for each user is as follows: , , For time slots user The transmission power of the first edge node, For time slots user The transmission power of the second edge node, For time slots user No. The transmission power of each edge node; The total power constraint is satisfied: in, For time slots user edge nodes Transmission power, For users Maximum transmission power.

[0031] The transmission rate model takes into account interference between users at the same edge node. edge nodes The transmission rate is: in, In state space Time slot user edge nodes transmission rate For edge nodes The corresponding channel bandwidth, Background noise power, For users edge nodes Channel gain, User edge nodes Channel gain, For time slots user edge nodes Transmission power, In addition to users The collection of all other users besides [the user group].

[0032] In the computational model, the offloading task is the smallest indivisible offloading unit, and the edge nodes have unlimited parallel processing capabilities. The goal of the offloading task is to maximize the actual transmission rate within the runtime, and the calculation expression is: in, This is the actual transmission rate. For the first The first user device The size of the data in a successfully transmitted task For runtime, The time slot length, For a successful transmission of the task, This represents the total number of users.

[0033] This objective is equivalent to maximizing the total number of successfully completed tasks. The system considers the resource waste caused by edge node failures and transmission interruptions, aiming to improve task success rate and system efficiency by optimizing the offloading strategy.

[0034] The premise of the task offloading system and method design in this invention is that the number of links is key to improving system efficiency. In typical edge computing scenarios, when When multiple user equipments compete for the same channel, the transmission rate of a single device can be expressed as: in, for The transmission rate of a single device when multiple user equipments compete for the same channel. For channel width, For the transmission power of a single device, For channel gain, Background noise power.

[0035] The ratio of the transmission rate of a dedicated channel to that of a shared channel is: in, The transmission rate for a dedicated channel; Therefore, even if only two user equipments share the channel, the total transmission rate will be at least an order of magnitude lower than that of a dedicated channel. Given that there may be more user equipments competing for the channel in the actual environment, in order to ensure transmission efficiency, the channel of the edge node should be kept as dedicated as possible to avoid competition between user equipments.

[0036] S2: Construct a link number control method based on reinforcement learning, and adaptively determine the optimal number of unloading edge nodes for the unloading task based on the link number control method based on reinforcement learning. Traditional methods tend to converge slowly when coordinating computational and communication resources due to the large policy space. Theoretical analysis shows that the number of links is the key to improving efficiency, so this invention takes it as the core decision objective of reinforcement learning.

[0037] The optimal number of unloading edge nodes for the unloading task is adaptively determined using a link number control method based on reinforcement learning.

[0038] Specifically, an ε-greedy strategy is employed for action selection: the agent chooses the action with the highest current Q-value (i.e., the number of links) with probability 1-ε, or explores randomly with probability ε. After each decision, the algorithm updates the Q-table using a temporal difference method based on the immediate reward and the maximum Q-value of the next state, thereby gradually learning the optimal configuration of the number of edge nodes under different network states. Through continuous interaction with the environment, the agent can adaptively adjust the number of offloaded edge nodes to optimize overall offloading efficiency.

[0039] A link number control method based on reinforcement learning adaptively determines the optimal number of offloading edge nodes for offloading tasks, which is applicable to user equipment in distributed wireless communication systems. S21: Model the user equipment as a reinforcement learning agent; S22: The agent's action is defined as selecting the maximum number of links, the agent's state is defined as the influence of the joint policies of other user equipment, and is indirectly characterized by measurable channel state information. The reward function is the total number of bits transmitted between two decision intervals.

[0040] User equipment is modeled as a reinforcement learning agent, enabling adaptive adjustment of the number of links through interaction with the environment.

[0041] An ε-greedy strategy is adopted for action selection. The agent selects the action with the highest current Q value with a probability of 1-ε, or explores randomly with a probability of ε. It obtains immediate reward and the maximum Q value of the next state, updates the Q table using the temporal difference method, and gradually learns the optimal configuration of the number of edge nodes under different network states. The maximum number of links is established based on the optimal configuration of the number of edge nodes.

[0042] The agent's state is divided into a less congested state and a more congested state based on the channel load. A less congested state is when the current channel quality is better than the historical average. A relatively congested state is defined as the current channel quality being lower than or equal to the historical average.

[0043] Channel quality is reflected by aggregated information, and smoothed channel information is obtained by exponential weighted averaging.

[0044] To address the issue of unobservable policies of other user equipment in distributed decision-making, measurable channel state information is used to indirectly characterize the system state.

[0045] Specifically, aggregated information is defined to reflect channel quality, and smoothed channel information is obtained through exponentially weighted averaging, serving as a reliable approximation of the state input. In the decision time slot... Aggregated information can be defined as: in, For time slots user edge nodes Aggregated information, For time slots user edge nodes Channel gain, Background noise power, For time slots user and edge nodes Channel gain, For time slots user edge nodes Transmission power, In addition to users The collection of all other users besides [the user group].

[0046] For users In the same edge node The power of interference plus noise experienced on the channel is: Time slot The transmission rate is: in, For time slots user edge nodes transmission rate For edge nodes The corresponding channel bandwidth; Will Substituting into the above equation, we get: but: We can obtain: in, For time slots user edge nodes The transmission rate; All parameters on the right side of the equation are known quantities, and the aggregated information can be directly calculated. The user transmission rate can be expressed as: in, In state space Time slot user edge nodes transmission rate For edge nodes The corresponding channel bandwidth.

[0047] This indicates that the aggregated information fully includes the time slots. The information required for transmission rate decisions can accurately characterize the impact of joint strategies. Considering the random fluctuation characteristics of the channel, an exponentially weighted average is used to eliminate random bias, defining... With edge nodes Average aggregate information: in, For time slots user edge nodes Channel information, For channel weights, For time slots user edge nodes Channel information, For time slots user edge nodes Aggregated information between them.

[0048] The parameter is a real number in the interval (0,1]. As an effective approximation of the state in reinforcement learning.

[0049] A significant advantage of this invention lies in the compact design of the action space. Unlike traditional methods that treat the entire discrete policy space as the action space, this invention limits the action space to the maximum number of selectable links (i.e., the number of optimal offloading edge nodes), making the size of the action space independent of the number of network nodes. This effectively avoids the curse of dimensionality in the action space and ensures the convergence speed of the algorithm.

[0050] In this invention, the agent directly determines the upper limit of the number of edge nodes that each user can simultaneously unload tasks by deciding the maximum number of links. That is, the maximum number of links is the optimal number of unloaded edge nodes in the final implementation stage.

[0051] In terms of reward function design, the reward is defined as the cumulative number of transmitted bits, which directly reflects the transmission efficiency of the system and provides a clear learning objective for the agent.

[0052] At the algorithm implementation level, a tabular Q-learning algorithm is adopted, and continuous states are approximated by tile coding. In order to achieve intelligent perception of channel congestion, the state space is divided into two categories according to the channel load: when the current channel quality is better than the historical average, it is defined as a less congested state, and vice versa.

[0053] Calculate the current state, select an action based on the ε-greedy policy, and update the Q-value function. When the decision requires reducing the number of links, the link with the worst channel quality is removed to ensure that the actual number of links matches the decision.

[0054] This invention achieves scalability and practicality in link number control in large-scale network environments through reasonable state representation, compact action space design, and clear reward mechanism, providing an effective technical solution for distributed wireless resource management.

[0055] S3: Construct a fault-aware expected transmission time selection model, and use the fault-aware expected transmission time selection model to calculate the expected transmission time of each candidate link. After determining the number of offloaded edge nodes, a heuristic strategy is adopted to determine the target offloaded edge nodes, taking into account the failure rate of edge nodes, and a transmission link is established. The selected edge nodes must be in a healthy state to provide services. Internally, the receiving, processing and result feedback processes of tasks are managed by maintaining receiving, execution and feedback queues.

[0056] After determining the target number of links, the designed heuristic strategy iteratively calculates and compares the expected transmission time of each candidate link, continuously selecting the edge node with the best transmission performance to establish a connection until the actual number of links reaches the target value. This process ensures that, considering factors such as edge node failure rate, the system can adaptively construct an efficient set of transmission links.

[0057] The fault-aware expected transmission time selection model selects the most suitable edge node by calculating the expected transmission time of the task. The expected transmission time is the conditional expectation under two scenarios: failure and successful transmission during the edge node's transmission process. It is determined by the task data volume, the current channel transmission rate, and the historical average transmission rate. The calculation expression is as follows: in, For tasks to edge nodes Expected transmission time For edge nodes The expectation of a failure during transmission. For edge nodes Expectations for successful transmission For the amount of task data, For edge nodes Based on the expected estimated channel transmission rate, For average transmission rate, For edge node failures related to time slots The probability density distribution function.

[0058] S4: Compare the expected transmission times of each candidate link and select the edge node with the best transmission performance to obtain the target offloading edge node; After determining the maximum number of links, the user equipment needs to allocate suitable target edge nodes for each task to be offloaded. To maximize task offloading efficiency, this invention proposes selecting edge nodes with the goal of minimizing the transmission time of a single task. The calculation of the expected transmission time comprehensively considers three key factors: the size of the task data, the fault distribution characteristics of the edge nodes, and the channel transmission rate.

[0059] According to the full expectation formula in probability theory, the expected transmission time is decomposed into conditional expectations under two scenarios: failure during edge node transmission and successful transmission. (Task to edge node) Expected transmission time It is determined by three factors: the amount of task data. Fault distribution of edge nodes (probability density function is) and the channel transmission rate based on expected estimation. The calculation expression is: in, For tasks to edge nodes Expected transmission time For edge nodes The expectation of a failure during transmission. For edge nodes Expectations for successful transmission For the amount of task data, For the channel transmission rate estimated based on expectations, For average transmission rate, For edge node failures related to time slots The probability density distribution function.

[0060] This invention further subdivides the expected transmission time into three components, based on a probabilistic analysis of the timing of failures during transmission: additional costs incurred due to potential failures at edge nodes. It is essentially the time of failure occurrence. The expected value within the failure probability range represents the time invested but lost due to failure; the cost required for data retransmission when an edge node fails. ,in For fixed retransmission time, this integral term reflects the result of weighting this cost by the probability of failure; and the cost required for successful transmission. Basic transmission time Multiply by the probability of successful transmission.

[0061] In a preferred embodiment of the invention, it is assumed that edge node failures are related to time slots. The probability density distribution follows a Poisson process, and the fault time interval follows an exponential distribution, i.e.: in, is the scale parameter of the exponential distribution; Based on this assumption, a closed-form expression for the expected transmission time is derived through integration: The final calculation results show that the expected transmission time is indeed determined by only three variables: the amount of task data, the current channel transmission rate, and the historical average transmission rate. These parameters can all be obtained through actual measurement or estimation from historical data.

[0062] The specific execution flow of this method includes: calculating the expected transmission time for each available channel, selecting the edge node with the minimum expected transmission time to establish a link, and updating the actual link set. Through this edge node selection mechanism based on expected transmission time, this method can effectively improve the success rate of task offloading, reduce transmission latency, and thus improve the overall system performance.

[0063] S5: Apply steps S3 and S4 iteratively based on the optimal number of offloading edge nodes to obtain all target offloading edge nodes, and establish the target transmission link based on all target offloading edge nodes. By calculating the expected transmission time of the task, the most suitable edge node is selected to establish a transmission link.

[0064] This invention provides an efficient and reliable edge node selection scheme for task offloading in distributed computing systems by establishing an accurate mathematical model and considering uncertainties in the actual network environment.

[0065] For end users, the number of links can be dynamically adjusted based on congestion levels during task execution. This mechanism is not only an immediate response to network changes but also a proactive fault-tolerance strategy designed to avoid transmission failures caused by congestion, thereby ensuring end-to-end task reliability.

[0066] S6: Dynamically adjust the target transmission link based on the congestion level of the transmission link.

[0067] When the actual number of links exceeds the optimal number for reinforcement learning decisions, the redundant links with the lowest performance are removed based on the performance metrics of each link.

[0068] To verify the effectiveness of this invention, a simulation of a mobile edge computing environment was conducted to evaluate the performance of link number control and edge node selection. The simulation system was implemented in C++ and ran on a computing platform configured with an AMD Ryzen 7 5800H processor.

[0069] In the simulation environment settings, edge nodes and user equipment are on sides of length [missing information]. The channels are uniformly distributed within a square area. The fast fading characteristics of the channel are modeled using a Ricean distribution, and the channel gain is mainly determined by the distance between devices and the path loss exponent.

[0070] Regarding edge node reliability modeling, the average failure time interval for each edge node is randomly generated to ensure that the failure rate remains within the range of [3.5%, 5%] over a one-minute runtime. The recovery time for a failed edge node is set to 100 milliseconds. The task size is simulated using a Pareto distribution with the following shape parameter: .

[0071] To reduce the impact of random factors and obtain accurate statistical results, ensuring the reliability and statistical significance of the evaluation results, all simulation data were within a 6×10⁻⁶ range. 4 The experiment was run on multiple time slots, and the average of 50 independent experiments was taken as the final result.

[0072] To provide sufficient evidence for verifying the effectiveness of the present invention in practical application scenarios, the present invention (Scalable Reinforcement Learning, SRL) was compared with Random Offloading (RO), Greedy Offloading (GO), and Multi-armed Bandits (MAB). Random Offloading (RO): Performance lower limit benchmark, where user equipment randomly selects target edge nodes and allocates power; Greedy Offloading (GO): In each time slot, the user device selects the best edge node based on a preference metric, pursuing a local optimum. Multi-armed Bandits (MAB): User equipment selects the number of links using a gambling machine algorithm, ignoring state changes, to compare the effectiveness of state-aware designs.

[0073] This invention considers different target selection strategies and edge node states, as follows: Average channel information (ACI) strategy: When selecting target edge nodes for user tasks, this strategy mainly considers the long-term channel quality of the link and calculates the channel state information by exponential weighted averaging. It aims to select nodes with more stable channel conditions and lighter loads and is an evaluation method based on the physical state of the link.

[0074] Expected Transmission Time (ETT) strategy: This strategy performs an end-to-end comprehensive performance evaluation when selecting target edge nodes for user tasks. It not only considers the channel transmission rate, but also introduces the edge node failure probability and task retransmission cost. It calculates the expected time required to complete the transmission through a mathematical model, aiming to select the edge node with the highest overall transmission efficiency. It is an application-oriented evaluation method.

[0075] Idealized situation (IS): Under this setting, the simulation environment ignores the failure mechanism of edge nodes and assumes that all edge nodes remain in a normal state throughout the entire operation. This assumption is used to evaluate the upper limit of the algorithm's performance in a perfect hardware environment. At the same time, in this case, since there is no need to consider edge node failures, the ACI strategy based on channel quality and the ETT strategy that comprehensively considers failures will tend to be consistent in decision-making performance.

[0076] The adaptability of this invention under different channel conditions was verified through simulation. The tests employed two typical wireless channel models: Ricean distribution and Rayleigh distribution, using the user's actual transmission rate and the number of tasks completed per unit time as performance evaluation indicators.

[0077] Simulation results show that the present invention (SRL) exhibits superior performance under both channel models. Regarding transmission rate, as... Figure 2 As shown, the present invention significantly outperforms the comparative algorithm when considering different target selection strategies (ACI and ETT) under ideal state IS and fault state. Figure 2 As shown in Figure (a), under the Rice distribution, in the three cases of ideal state (IS) and fault state using ACI or ETT strategies, the present invention shows significant improvement compared to the RO algorithm, GO algorithm, and MAB algorithm. Figure 2 As shown in Figure (b), SRL maintains a stable performance advantage even under Rayleigh distribution conditions with worse channel conditions. Regarding the number of tasks completed, as... Figure 3 As shown, SRL also demonstrates significant advantages when considering different target selection strategies, ACI and ETT, under ideal state IS and fault state conditions. Figure 3 As shown in Figure (a), under the Rice distribution, in the ideal state (IS) and the fault state using the ACI or ETT strategy, this invention also shows a significant improvement over the RO algorithm, GO algorithm, and MAB algorithm. Figure 3 As shown in Figure (b), the effectiveness of SRL is further verified under Rayleigh distribution with worse channel conditions, proving that the present invention has a wide channel adaptability, can maintain stable performance in different wireless channel environments, and is suitable for various channel conditions in practical application scenarios.

[0078] like Figure 4 As shown, a distributed task offloading system based on fault perception and reinforcement learning is used to execute the aforementioned distributed task offloading method based on fault perception and reinforcement learning, including: The task acquisition module 101 acquires the unloading tasks of IoT terminal users with autonomous decision-making power to unload tasks to edge nodes; The optimal node number decision module 102 constructs a link number control method based on reinforcement learning, and adaptively determines the optimal number of unloading edge nodes for the unloading task based on the link number control method based on reinforcement learning. The expected transmission time calculation module 103 constructs an expected transmission time selection model based on fault perception, and uses the expected transmission time selection model based on fault perception to calculate the expected transmission time of each candidate link. The target node selection module 104 compares the expected transmission time of each candidate link and selects the edge node with the best transmission performance to obtain the target offloading edge node. The target transmission link construction module 105 applies steps S3 and S4 repeatedly according to the optimal number of offloading edge nodes to obtain all target offloading edge nodes, and establishes the target transmission link based on all target offloading edge nodes. The dynamic adjustment module 106 dynamically adjusts the target transmission link according to the congestion level of the transmission link.

[0079] Through the collaborative work of the above modules, the optimal number of offloading edge nodes is determined by the reinforcement learning agent, and an ε-greedy strategy is used for action selection and Q-value updates. Edge nodes are selected to establish transmission links based on the expected transmission time, taking into account the impact of edge node failure rates. The number of links is dynamically adjusted according to network congestion levels, and inefficient redundant links are removed. This invention decouples link quantity decision-making from edge node selection, reduces the dimensionality of the policy space, improves convergence speed, and comprehensively considers edge node failures and network congestion, significantly improving task offloading success rate and system transmission efficiency.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A distributed task offloading method based on fault perception and reinforcement learning, characterized in that, include: S1: Obtain the unloading task from IoT terminal users with autonomous decision-making power to unload tasks to edge nodes; S2: Construct a link number control method based on reinforcement learning, and adaptively determine the optimal number of unloading edge nodes for the unloading task based on the link number control method based on reinforcement learning. S3: Construct a fault-aware expected transmission time selection model, and use the fault-aware expected transmission time selection model to calculate the expected transmission time of each candidate link. S4: Compare the expected transmission times of each candidate link and select the edge node with the best transmission performance to obtain the target offloading edge node; S5: Apply steps S3 and S4 iteratively based on the optimal number of offloading edge nodes to obtain all target offloading edge nodes, and establish the target transmission link based on all target offloading edge nodes. S6: Dynamically adjust the target transmission link based on the congestion level of the transmission link.

2. The distributed task offloading method based on fault perception and reinforcement learning according to claim 1, characterized in that, The unloading task is the smallest indivisible unloading unit, and the goal of the unloading task is to maximize the actual transmission rate within the runtime.

3. The distributed task offloading method based on fault perception and reinforcement learning according to claim 1, characterized in that, The fault-aware expected transmission time selection model selects the most suitable edge node by calculating the expected transmission time of the task. The expected transmission time is the conditional expectation under two scenarios: failure and successful transmission during the edge node's transmission process. It is determined by the task data volume, the current channel transmission rate, and the historical average transmission rate. The calculation expression is as follows: in, For tasks to edge nodes Expected transmission time For edge nodes The expectation of a failure during transmission. For edge nodes Expectations for successful transmission For the amount of task data, For edge nodes Based on the expected estimated channel transmission rate, For average transmission rate, For edge node failures related to time slots The probability density distribution function.

4. The distributed task offloading method based on fault perception and reinforcement learning according to claim 1, characterized in that, Link number control methods based on reinforcement learning include: S21: Model the user equipment as a reinforcement learning agent; S22: The agent's action is defined as selecting the maximum number of links, the agent's state is defined as the influence of the joint policies of other user equipment, and is indirectly represented by channel state information. The reward function is the total number of bits transmitted between two decision intervals.

5. A distributed task offloading method based on fault perception and reinforcement learning according to claim 4, characterized in that, An ε-greedy strategy is adopted for action selection. The agent selects the action with the highest current Q value with a probability of 1-ε, or explores randomly with a probability of ε. It obtains immediate reward and the maximum Q value of the next state, updates the Q table using the temporal difference method, and gradually learns the optimal configuration of the number of edge nodes under different network states. The maximum number of links is established based on the optimal configuration of the number of edge nodes.

6. The distributed task offloading method based on fault perception and reinforcement learning according to claim 4, characterized in that, The agent's state is divided into a less congested state and a more congested state based on the channel load. A less congested state is when the current channel quality is better than the historical average. A relatively congested state is defined as the current channel quality being lower than or equal to the historical average.

7. A distributed task offloading method based on fault perception and reinforcement learning according to claim 6, characterized in that, Channel quality is reflected by aggregated information, and smoothed channel information is obtained by exponential weighted averaging.

8. A distributed task offloading method based on fault perception and reinforcement learning according to claim 7, characterized in that, Aggregated information is defined as: in, For time slots user edge nodes Aggregated information, For time slots user edge nodes Channel gain, Background noise power, For time slots user edge nodes Channel gain, For time slots user edge nodes Transmission power, In addition to users The collection of all other users besides [the user group].

9. A distributed task offloading method based on fault perception and reinforcement learning according to claim 7, characterized in that, The expression for calculating channel information is: in, For time slots user edge nodes Channel information, For channel weights, For time slots user edge nodes Channel information, For time slots user edge nodes Aggregated information between them.

10. A distributed task offloading system based on fault perception and reinforcement learning, characterized in that, To execute a distributed task offloading method based on fault perception and reinforcement learning as described in any one of claims 1 to 9, comprising: The task acquisition module acquires the unloading tasks of IoT terminal users with autonomous decision-making power to unload tasks to edge nodes; The optimal node number decision module constructs a link number control method based on reinforcement learning, and adaptively determines the optimal number of unloading edge nodes for the unloading task based on the link number control method based on reinforcement learning. The expected transmission time calculation module constructs an expected transmission time selection model based on fault perception and uses the expected transmission time selection model based on fault perception to calculate the expected transmission time of each candidate link. The target node selection module compares the expected transmission time of each candidate link and selects the edge node with the best transmission performance to obtain the target offloading edge node. The target transmission link construction module applies steps S3 and S4 iteratively according to the optimal number of offload edge nodes to obtain all target offload edge nodes, and establishes the target transmission link based on all target offload edge nodes. A dynamic adjustment module that dynamically adjusts the target transmission link based on the congestion level of the transmission link.

Images