Self-sensing multi-edge node calculation unloading method for smart pension

Abstract

The invention discloses a self-sensing multi-edge node calculation unloading method for smart pension, which comprises a plurality of edge nodes and a center node, and user request collection modules and tokens are carried on the edge nodes. The user request collection module collects a service request of an elderly user in a smart pension mode and sends the service request to the center node, and the token is used for self-sensing design of an edge node; the central node carries a data storage module and a calculation task management module, the data storage module stores data of the elderly users, converts the data into a partial observable Markov decision process, and sends the partial observable Markov decision process to the calculation task management module; and the calculation task management module obtains an optimal unloading strategy according to an algorithm, and issues calculation resources to edge nodes according to the unloading strategy for collection of a next round of requests. According to the method, the computing resources are reasonably deployed to the edge nodes, the time delay of unloading is reduced, and the efficiency of resource scheduling is improved.

Description

Technical Field

[0001] The present invention belongs to the field of edge computing technology, and in particular relates to a self-aware multi-edge node computing offloading method for smart elderly care. Background Art

[0002] As the global aging problem becomes increasingly serious, smart elderly care has become a key technical means for modern society to solve problems such as elderly care, health monitoring and daily life management. Through the widespread application of Internet of Things (IoT) technology, a large number of smart devices and sensors in smart elderly care environments can monitor the health status, activity behavior and emergency events of the elderly in real time. However, the amount of data generated by these devices is huge and has high timeliness requirements. Therefore, a single data center or cloud processing solution may face challenges such as network delay, bandwidth limitation and computing resource bottleneck, which makes it difficult to meet the requirements of real-time and reliability in smart elderly care scenarios. Edge computing, as an emerging computing paradigm, can effectively reduce the load of central servers, reduce network delays, and improve data processing efficiency and system response speed by distributing computing tasks to edge nodes close to data sources. Therefore, the application of edge computing in smart elderly care environments has gradually attracted attention. However, the smart elderly care environment is dynamic, complex and changeable, and user needs and network status change frequently. Traditional static computing task offloading solutions are difficult to adapt to this complex environment. Therefore, a self-aware computing offloading method based on multiple edge nodes is proposed, which can perform intelligent resource scheduling and task allocation according to the real-time status in the smart elderly care environment, becoming an effective way to solve current problems. Summary of the invention

[0003] The purpose of the present invention is to provide a self-aware multi-edge node computing offloading method for smart elderly care to solve the computing offloading problem in the above-mentioned existing smart elderly care environment. In order to achieve the above-mentioned purpose, the present invention provides the following scheme: a self-aware multi-edge node computing offloading method for smart elderly care, comprising the following steps:

[0004] Step 1: A user request collection module is installed on each edge node to obtain the data volume and service delay upper limit parameters of the newly arrived user service requests in the smart elderly care environment, the wireless channel state parameters, and the user service request buffer queue length parameters on the base station and edge server sides, and the obtained parameter data is normalized and uploaded to the central node;

[0005] Step 2: The central node is equipped with a data storage module to receive data information uploaded by the edge node, and convert the data information into a partially observable Markov decision process and send it to the computing task management module; Step 3: The central node is equipped with a computing task management module. After receiving the data information of the Markov decision process, the optimal strategy for unloading is obtained using the double-delay deep deterministic policy gradient algorithm; Step 4: The computing task management module sends computing resources to the edge node according to the optimal strategy; Step 5: The edge node group is equipped with a token Token to realize the dynamic expansion and monitoring of the edge node. All edge nodes are arranged in a virtual closed chain structure, and each node is recorded by a token transfer. According to the optimal strategy, when adding or reducing edge nodes, the token will automatically update the device information to ensure the dynamic adaptability of the edge node group. If a node fails, the node group can automatically detect and update the node status through token transfer to ensure the normal operation of the system. Step 6: After the edge node obtains the computing resources, it continues to run the user request collection module to obtain parameter data. Further, step 1 specifically includes the following steps: Step 1.1: The edge node interacts with the smart elderly care environment, collects user requests, and obtains a triple (s t ,r t ,s t+1 ), where s t represents the state of the node environment at the current time t, including the characteristic information of the user service request i arriving at time t, the service delay upper limit parameter, the wireless channel state parameter, the user service request buffer queue length parameter on the base station and edge server side, and the computing resources of the edge node; r t Represents a reward value fed back by the edge node after executing the current collection task; s t+1 Indicates the next state of the node; Step 1.2, store the triples obtained by the interaction between the edge node and the smart elderly care environment in the storage library; if the number of triples in the storage library does not exceed the maximum capacity of the storage library, directly store the triples in the memory library; if it exceeds the maximum capacity of the storage library, suspend this collection first and normalize the triples in the existing storage library; Step 1.3, normalize the obtained parameter data according to the following formula:

[0006]

[0007] In the formula, x k Indicates the kth parameter obtained by the system, x k,max represents the maximum value of the kth parameter, represents the minimum value of the kth parameter, x ′ kRepresents the value of the kth parameter after normalization, and k represents the number of different parameters. Step 1.4, upload the normalized data to the central node. Further, step 2 specifically includes the following steps: Step 2.1, the central node inputs the normalized user request data into the data storage module; Step 2.2, converts the data into a Markov decision process, the process is specifically as follows: State space: In the smart elderly care system, the system state is composed of a user, i edge nodes and the environment, so the system state of time slot k can be defined as Among them, E re (k) represents the residual energy of the edge node in time slot k, p(k) represents the location information of the user in time slot k, and q i (k) represents the location information of the edge node in the kth time slot, D re (k) represents the remaining request collection task size of the entire system, d i (k) represents the size of the requested resources collected by edge node i in time slot k, f i (k) indicates whether edge node i is blocked by an obstacle in time slot k; Action space: The edge node collects requests based on the current user status and environment, including the unloaded device, the angle and speed of the user's action. The action space can be expressed as:

[0008] Action k =(i(k),β(k),v(k),r i (k))

[0009] Among them, i(k) represents the action variable of the edge node in time slot k, β(k) represents the movement angle of the edge node in time slot k, v(k) represents the movement speed of the edge node in time slot k, and r i (k) represents the task offloading rate of the edge node in time slot k;

[0010] Since the user input is a continuous action, the action variable Selected by the edge node, it needs to be discretized. The user's movement speed and angle and the request offloading ratio can be accurately optimized in a continuous action space. The system jointly optimizes the above four variables to minimize the cost; Reward function: The reward function is a quantitative measure of device behavior, which will affect the decisions and actions made by the edge node in a specific environment. Therefore, in order to obtain the best performance, it is necessary to design a suitable reward function that can guide the edge node to make better decisions and behaviors while avoiding unexpected behaviors and adverse consequences. Therefore, when selecting a reward function, it is necessary to fully consider factors such as the system's goals, task requirements, and constraints, and design a reasonable reward function to improve the performance and effect of the algorithm; the goal is to minimize the maximum processing delay of the edge node requesting task offloading, and define it as a reward function:

[0011] Reword k =R(State k , Action k )=-Delay(k),

[0012] Where Delay(k) is expressed as: αik represents the activation variable of offload task i at time slot k, which indicates whether task i is executed at time slot k. If task i is executed at time slot k, then α i (k) = 1; otherwise, α i (k) = 0; t loc，i (k) represents the local processing time of task i in time slot k; t edge，i (k) represents the time that task i is processed by the edge node in time slot k; t t，i (k) represents the transmission time of task i in time slot k. It represents the time required for task i to be transmitted from the local system to the edge node for processing;

[0013] Step 2.3, sending the result to the computing task management module; further, step 3 specifically includes the following steps: step 3.1, the computing task management module receives the Markov decision process;

[0014] Step 3.2: Use the double-delayed deep deterministic policy gradient algorithm to obtain the optimal strategy for unloading, as follows:

[0015] Establish two value function networks Q1 and Q2. In order to stabilize the learning process, create two target value function networks Q1' and Q2'; create two strategy function networks μ and μ'; introduce Gaussian noise as noise to increase the exploratory nature of edge nodes, and use the behavior strategy to add Gaussian noise to the strategy function to explore the environment;

[0016] Initialization value function network parameters and policy function network parameters θ μ , copy the two network parameters to the corresponding target value function network and target strategy function network, that is Initialize memory playback library D;

[0017] Traverse all episodes, initialize Gaussian noise, i.e., noise ε, receive the starting state s1 from the environment, and traverse all steps for each episode;

[0018] From the memory playback library D, randomly sample N (s i ,r i ,s i+1 ) as a mini-batch training data for the value function network and the policy function network;

[0019] Use the target value function network Q1' and Q2' to calculate the Q value of the next state, Q'(s t+1 ,μ's t+1 |θμ')), select the minimum value minQ(s of the two value function networks t+1 ,a) As the Q value of the next state, update the value function network, y t =r t+1 +γminQ'(s t+1 ,μ'(s t+1 |θμ')), using the gradient descent algorithm to update the parameters of the value function network and

[0020] Every several time steps, use the policy function network μ to generate a new behavior strategy and update the policy function network, use the target value function network to calculate the estimated value of the policy gradient, and update the parameters θμ of the policy function network;

[0021] Use soft update method to update the target value function network Q1' and

[0022] By continuously updating the value function network and the policy function network, they will eventually converge to the optimal policy. Using the converged policy function network, the optimal unloading decision can be obtained.

[0023] Furthermore, step 4 specifically includes the following steps:

[0024] Step 4.1: Allocate the required computing resources to the relevant edge nodes according to the obtained optimal offloading strategy.

[0025] Furthermore, step 5 specifically includes the following steps:

[0026] Step 5.1: To ensure the flexible expansion and robust operation of the edge computing network, each edge node is equipped with a token mechanism. The edge nodes are arranged in a virtual closed chain structure, and each node maintains state information by passing tokens. The token mechanism enables edge nodes to dynamically increase or decrease nodes, and automatically update device information and network topology when failures occur, ensuring the adaptability and robustness of the system. When the optimal strategy requires increasing or decreasing edge nodes, the token will be automatically updated to ensure that new or removed nodes do not affect the overall computing efficiency.

[0027] Step 5.2: The token is represented as follows:

[0028]

[0029] Among them, workgroup represents the work group of the edge node; is a list of status information for each node in the work group; each All represent the information of edge working node i;

[0030] servergroup indicates the server group; is a list of status information for each server in the server group; each Represents the information of server node i;

[0031] All node devices are divided into two parts: workgroups (k devices) and server groups (t devices). The initial token is generated by a preset test program, recording the computing performance, status and IP address of the device. The system automatically allocates the number of workgroups and server groups suitable for the distributed training task based on the number of devices provided by the user and the predefined workgroup / server group ratio. As the devices start one by one, they add relevant information to the token. After all devices are tested, the last device sorts the information in the token and groups the devices. The system dynamically adjusts the configuration by modulating the token information, deletes or adds devices, and reallocates the workgroup / server roles if the server group requirements are not met to ensure that the distributed acquisition process is consistent with the latest device information. Step 5.3. In order to enhance the reliability and resilience of the system, a self-healing method is proposed to reduce the risk of failure. The failure of any PS node may cause the entire cluster to be out of order, resulting in system failure. For example, starting from device 1, the token is sent to the next device according to the order in the token, which is recorded as device 2. Before sending, device 1 tries to communicate with device 2 to determine whether it is still online. If yes, the token is sent; if no, device 1 tries to communicate with device 3, and continues in this way until a device that is online is found.

[0032] Step 5.4: When any device is found offline during training, the detection process determines the group it belongs to based on its IP address. If it is in the work group, the device is removed from the token and the system continues to run. If it is in the server group, a global notification is triggered and broadcast to restart all devices on the most recent checkpoint in the cluster.

[0033] Step 5.5, before delivering the token, device 1 removes all offline device information from the token. Device 2 waits to receive the token and then performs the same process to check if the next device in the token is online. Therefore, no matter which group it belongs to, as long as the token holder works, the system should always remain operational.

[0034] Furthermore, step 6 specifically includes the following steps:

[0035] After obtaining computing resources, the edge node continues to run the user request collection module to dynamically obtain new service request data and network status parameters. By periodically uploading the latest parameter data, the central node can continuously optimize the unloading strategy according to the latest network status, realizing adaptive updates and dynamic adjustments of the system.

[0036] Compared with the prior art, the present invention has achieved the following beneficial technical effects:

[0037] 1. For the smart elderly care scenario with multiple edge nodes, the problems of computing offloading and resource allocation are studied. According to the dynamic changes of user requests and computing resources at each edge node in the system, the optimal target offloading edge node and computing resources are selected for dynamically arriving user requests. The system throughput is maximized while meeting the service delay upper limit of user requests in the multi-edge node system and the system computing resource constraints.

[0038] 2. The self-aware multi-edge node computing offloading method for smart elderly care provided by the present invention converts the user request problem in the smart elderly care scenario into a partially observable Markov decision process, uses a double-delay deep deterministic policy gradient algorithm to obtain the optimal offloading strategy, and adds two independent Gaussian noise networks to the optimal strategy for state enhancement, further improving the delay performance of the multi-edge node system. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A flowchart of a self-aware multi-edge node computing offloading method for smart elderly care provided by an embodiment of the present invention;

[0040] Figure 2 A dynamic self-organizing method based on token passing of edge node devices in the present invention;

[0041] Figure 3The flowchart is a self-healing method based on token passing of edge node equipment in the present invention. DETAILED DESCRIPTION

[0042] In order to better understand the purpose, structure and function of the present invention, the following is a further detailed description of a self-aware multi-edge node computing offloading method for smart elderly care in conjunction with the accompanying drawings.

[0043] The core of the present invention is to provide a self-aware multi-edge node computing offloading method for smart elderly care, so as to solve the delay optimization and resource scheduling problems in existing edge computing.

[0044] A self-aware multi-edge node computing offloading method for smart elderly care in an embodiment of the present invention mainly includes 6 steps, such as: Figure 1 As shown, the following steps are included:

[0045] Step 1: The edge node collects user requests, processes them and uploads them to the central node. The steps include:

[0046] The edge node interacts with the smart elderly care environment, collects user requests, and obtains a triplet (s t ,r t ,s t+1 ), where s t represents the state of the node environment at the current time t, including the characteristic information of the user service request i arriving at time t, the service delay upper limit parameter, the wireless channel state parameter, the user service request buffer queue length parameter on the base station and edge server side, and the computing resources of the edge node; r t Represents a reward value fed back by the edge node after executing the current collection task; s t+1 Indicates the next state of the node;

[0047] The triples obtained by the interaction between the edge node and the smart elderly care environment are stored in the storage library; if the number of triples in the storage library does not exceed the maximum capacity of the storage library, the triples are directly stored in the memory library; if the maximum capacity of the storage library is exceeded, the current collection is suspended first, and the triples in the existing storage library are normalized;

[0048] The obtained parameter data is normalized according to the following formula:

[0049]

[0050] In the formula, x k Indicates the kth parameter obtained by the system, x k,max Indicates the maximum value of the kth parameter represents the minimum value of the kth parameter, x ′ kIt represents the value of the kth parameter after normalization, and k represents the number of different parameters.

[0051] Upload the normalized data to the central node.

[0052] Step 2: The central node is equipped with a data storage module, which receives data information uploaded by the edge node, converts the data information into a partially observable Markov decision process, and sends it to the computing task management module. The steps include: the central node inputs the normalized user request data into the data storage module;

[0053] The data is converted into a Markov decision process, which is as follows:

[0054] State space: In the smart elderly care system, the system state is composed of a user, i edge nodes and the environment, so the system state at time slot k can be defined as Among them, E re (k) represents the residual energy of the edge node in time slot k, p(k) represents the location information of the user in time slot k, and q i (k) represents the location information of the edge node in the kth time slot, D re (k) represents the remaining request collection task size of the entire system, d i (k) represents the size of the requested resources collected by edge node i in time slot k, f i (k) indicates whether edge node i is blocked by an obstacle in time slot k; Action space: The edge node collects requests based on the current user status and environment, including the unloaded device, the angle and speed of the user's action. The action space can be expressed as:

[0055] Action k=(i(k), β(k), v(k), r i (k))

[0056] Among them, i(k) represents the action variable of the edge node in time slot k, β(k) represents the movement angle of the edge node in time slot k, v(k) represents the movement speed of the edge node in time slot k, and r i (k) represents the task offloading rate of the edge node in time slot k;

[0057] Since the user input is a continuous action, the action variable Selected by the edge node, it needs to be discretized. The user's movement speed and angle and the request offloading ratio can be accurately optimized in a continuous action space. The system jointly optimizes the above four variables to minimize the cost; Reward function: The reward function is a quantitative measure of device behavior, which will affect the decisions and actions made by the edge node in a specific environment. Therefore, in order to obtain the best performance, it is necessary to design a suitable reward function that can guide the edge node to make better decisions and behaviors while avoiding unexpected behaviors and adverse consequences. Therefore, when selecting a reward function, it is necessary to fully consider factors such as the system's goals, task requirements, and constraints, and design a reasonable reward function to improve the performance and effect of the algorithm; the goal is to minimize the maximum processing delay of the edge node requesting task offloading, and define it as a reward function:

[0058] Reword k =R(State k , Action k )=-Delay(k)

[0059] Where Delay(k) is expressed as: αik represents the activation variable of offload task i at time slot k, which indicates whether task i is executed at time slot k. If task i is executed at time slot k, then α i (k) = 1; otherwise, α i (k) = 0; t loc，i (k) represents the local processing time of task i in time slot k; t edge，i (k) represents the time that task i is processed by the edge node in time slot k; t t，i (k) represents the transmission time of task i in time slot k. It represents the time required for task i to be transmitted from the local system to the edge node for processing;

[0060] The results are sent to the computing task management module.

[0061] Step 3: The central node is equipped with a computing task management module. After receiving the data information of the Markov decision process, the optimal strategy for unloading is obtained using the double-delayed deep deterministic policy gradient algorithm. The specific steps include:

[0062] The computational task management module receives the Markov decision process;

[0063] The optimal strategy for offloading is obtained using the double-delayed deep deterministic policy gradient algorithm, as follows:

[0064] Establish two value function networks Q1 and Q2. In order to stabilize the learning process, create two target value function networks Q1' and Q2'; create two strategy function networks μ and μ'; introduce Gaussian noise as noise to increase the exploratory nature of edge nodes, and use the behavior strategy to add Gaussian noise to the strategy function to explore the environment;

[0065] Initialization value function network parameters and policy function network parameters θμ, copy the two network parameters to the corresponding target value function network and target policy function network, that is, Initialize memory playback library D;

[0066] Traverse all episodes, initialize Gaussian noise, i.e., noise ε, receive the starting state s1 from the environment, and traverse all steps for each episode;

[0067] From the memory playback library D, randomly sample N (s i ,r i ,s i+1 ) as a mini-batch training data for the value function network and the policy function network;

[0068] Use the target value function network Q1' and Q2' to calculate the Q value of the next state, Q'(s t+1 ,μ's t+1 |θμ')), select the minimum value minQ(s of the two value function networks t+1 ,a) As the Q value of the next state, update the value function network, y t =r t+1 +γminQ'(s t+1 ,μ'(s t+1 |θμ')), using the gradient descent algorithm to update the parameters of the value function network and

[0069] Every several time steps, use the policy function network μ to generate a new behavior strategy and update the policy function network, use the target value function network to calculate the estimated value of the policy gradient, and update the parameters θμ of the policy function network;

[0070] Use soft update method to update the target value function network Q1' and

[0071] By continuously updating the value function network and the policy function network, they will eventually converge to the optimal policy. Using the converged policy function network, the optimal unloading decision can be obtained.

[0072] Step 4: The computing task management module sends computing resources to edge nodes according to the optimal strategy;

[0073] Step 5: The edge node group is equipped with a token to achieve dynamic expansion and monitoring of edge nodes. The specific steps include:

[0074] To ensure the flexible expansion and robust operation of the edge computing network, each edge node is equipped with a token mechanism. The edge nodes are arranged in a virtual closed chain structure, and each node maintains state information by passing tokens. The token mechanism enables edge nodes to dynamically increase or decrease nodes, and automatically update device information and network topology when failures occur, ensuring the adaptability and robustness of the system. When the optimal strategy requires increasing or decreasing edge nodes, the token will be automatically updated to ensure that new or removed nodes do not affect the overall computing efficiency.

[0075] The token is represented as follows:

[0076]

[0077] Among them, workgroup represents the work group of the edge node; is a list of status information for each node in the work group; each All represent the information of edge working node i;

[0078] servergroup indicates the server group; is a list of status information for each server in the server group; each Represents the information of server node i;

[0079] All node devices are divided into two parts: workgroups (k devices) and server groups (t devices). The initial token is generated by a preset test program, recording the computing performance, status and IP address of the device. The system automatically allocates the number of workgroups and server groups suitable for the distributed training task based on the number of devices provided by the user and the predefined workgroup / server group ratio. As the devices start one by one, they add relevant information to the token. After all devices are tested, the last device sorts the information in the token and groups the devices. The system dynamically adjusts the configuration by modulating the token information, removes or adds devices, and reallocates the workgroup / server roles if the server group requirements are not met to ensure that the distributed acquisition process is consistent with the latest device information. In order to enhance the reliability and resilience of the system, a self-healing method is proposed to reduce the risk of failure. The failure of any PS node may cause the entire cluster to be out of order, resulting in system failure. For example, starting from device 1, the token is sent to the next device according to the order in the token, which is recorded as device 2. Before sending, device 1 tries to communicate with device 2 to determine if it is still online. If so, the token is sent; if not, device 1 tries to communicate with device 3 and continues in this way until an online device is found.

[0080] When any device is found offline during training, the detection process determines the group it belongs to based on its IP address. If it is in the work group, the device is removed from the token and the system continues to run. If it is in the server group, a global notification is triggered and broadcast to restart all devices in the cluster on the most recent checkpoint.

[0081] Before delivering the token, device 1 removes all offline device information from the token. Device 2 waits to receive the token and then performs the same process to check if the next device in the token is online. Therefore, no matter which group it belongs to, as long as the token holder works, the system should always remain operational.

[0082] After the edge node obtains computing resources, it continues to run the user request collection module to obtain parameter data. The specific steps include:

[0083] After obtaining computing resources, the edge node continues to run the user request collection module to dynamically obtain new service request data and network status parameters. By periodically uploading the latest parameter data, the central node can continuously optimize the unloading strategy according to the latest network status, realizing adaptive updates and dynamic adjustments of the system.

[0084] The above is a detailed introduction to the self-aware multi-edge node computing offloading method for smart elderly care provided by the present invention.

[0085] The contents not described in detail in the specification of the present invention belong to the prior art known to the professional and technical personnel in this field.

Claims

1. A self-aware multi-edge node computing offloading method for smart elderly care, characterized in that: include: Step 1: Each edge node is equipped with a user request collection module to obtain the data volume and service delay upper limit parameters of the newly arrived user service requests in the smart elderly care environment, the wireless channel state parameters, and the user service request buffer queue length parameters on the base station and edge server sides, and normalize the obtained parameter data and upload it to the central node; Step 2: The central node is equipped with a data storage module, which receives data information uploaded by the edge nodes, converts the data information into a partially observable Markov decision process, and sends it to the computing task management module; Step 3: The central node is equipped with a computing task management module. After receiving the data information of the Markov decision process, the optimal strategy for unloading is obtained using the double-delayed deep deterministic policy gradient algorithm. Step 4: The computing task management module sends computing resources to edge nodes according to the optimal strategy; Step 5: The edge node group is equipped with a token to achieve dynamic expansion and monitoring of edge nodes. All edge nodes are arranged in a virtual closed chain structure, and each node is recorded by passing a token. According to the optimal strategy, when adding or reducing edge nodes, the token will automatically update the device information to ensure the dynamic adaptability of the edge node group. If a node fails, the node group automatically detects and updates the node status through token transfer to ensure the normal operation of the system; Step 6: After the edge node obtains computing resources, it continues to run the user request collection module to obtain parameter data.

2. The self-aware multi-edge node computing offloading method for smart elderly care according to claim 1 is characterized in that: Step 1 specifically includes the following steps: Step 1.1: The edge node interacts with the smart elderly care environment, collects user requests, and obtains a triplet (s t ,r t ,s t+1 ), where s t represents the state of the node environment at the current time t, including the characteristic information of the user service request i arriving at time t, the service delay upper limit parameter, the wireless channel state parameter, the user service request buffer queue length parameter on the base station and edge server side, and the computing resources of the edge node; r t Represents a reward value fed back by the edge node after executing the current collection task; s t+1 Indicates the next state of the node; Step 1.2: Store the triples obtained by the interaction between the edge node and the smart elderly care environment in the storage repository; if the number of triples in the storage repository does not exceed the maximum capacity of the storage repository, directly store the triples in the memory repository; if it exceeds the maximum capacity of the storage repository, suspend the current collection and normalize the triples in the existing storage repository; Step 1.3: Normalize the acquired parameter data according to the following formula: In the formula, x k Indicates the kth parameter obtained by the system, x k,max represents the maximum value of the kth parameter, represents the minimum value of the kth parameter, x′ k It represents the value of the kth parameter after normalization, and k represents the number of different parameters; Step 1.4: Upload the normalized data to the central node.

3. The self-aware multi-edge node computing offloading method for smart elderly care according to claim 1 is characterized in that: Step 2 specifically includes the following steps: Step 2.1, the central node inputs the normalized user request data into the data storage module; Step 2.2: Convert the data into a Markov decision process. The specific process is as follows: State space: In the smart elderly care system, the system state is composed of a user, i edge nodes and the environment, so the system state at time slot k is defined as Among them, E re (k) represents the residual energy of the edge node in time slot k, p(k) represents the location information of the user in time slot k, and q i (k) represents the location information of the edge node in the kth time slot, D re (k) represents the remaining request collection task size of the entire system, D i (k) represents the size of the requested resources collected by edge node i in time slot k, f i (k) indicates whether edge node i is blocked by an obstacle in time slot k; Action space: The edge node collects requests based on the current user state and environment, including the uninstalled device, the angle and speed of the user's action. The action space is represented as Action k =(i(k),β(k),v(k),r i (k)) Among them, i(k) represents the action variable of the edge node in time slot k, β(k) represents the movement angle of the edge node in time slot k, v(k) represents the movement speed of the edge node in time slot k, and r i (k) represents the task offloading rate of the edge node in time slot k; Since the user input is a continuous action, the action variable The edge nodes are selected and need to be discretized. The user's movement speed and angle as well as the request offloading ratio are accurately optimized in a continuous action space. The system jointly optimizes the above four variables to minimize the cost. Reward function: The reward function is a quantitative measure of device behavior, which affects the decisions and actions made by edge nodes in a specific environment. Therefore, in order to obtain the best performance, a suitable reward function is designed to guide edge nodes to make better decisions and behaviors, while avoiding unexpected behaviors and adverse consequences. Therefore, when selecting a reward function, it is necessary to fully consider the system's goals, task requirements, and constraints, and design a reasonable reward function to improve the performance and effect of the algorithm. The goal is to minimize the maximum processing delay of edge nodes requesting task offloading, which is defined as a reward function: Reword k =R(State k ,Action k )=-Delay(k) Where Delay(k) is expressed as: αik represents the activation variable of offload task i in time slot k, which indicates whether task i is executed in time slot k; if task i is executed in time slot k, then α i (k) = 1; otherwise, α i (k) = 0; t loc,i (k) represents the local processing time of task i in time slot k; t edge,i (k) represents the time that task i is processed by the edge node in time slot k; t t,i (k) represents the transmission time of task i in time slot k. It represents the time required for task i to be transmitted from the local system to the edge node for processing; Step 2.3: Send the result to the computing task management module.

4. The self-aware multi-edge node computing offloading method for smart elderly care according to claim 1 is characterized in that: Step 3 specifically includes the following steps: Step 3.1, the computing task management module receives the Markov decision process; Step 3.2: Use the double-delayed deep deterministic policy gradient algorithm to obtain the optimal strategy for unloading, as follows: Establish two value function networks Q1 and Q2. In order to stabilize the learning process, create two target value function networks Q1' and Q2'; create two strategy function networks μ and μ'; introduce Gaussian noise as noise to increase the exploratory nature of edge nodes, and use the behavior strategy to add Gaussian noise to the strategy function to explore the environment; Initialization value function network parameters and policy function network parameters θμ, copy the two network parameters to the corresponding target value function network and target policy function network, that is, θμ'←θμ, initialize the memory playback library D; Traverse all episodes, initialize Gaussian noise, i.e., noise ε, receive the starting state s1 from the environment, and traverse all steps for each episode; From the memory playback library D, randomly sample N (s i ,r i ,s i+1 ) as a mini-batch training data for the value function network and the policy function network; Use the target value function network Q1' and Q2' to calculate the Q value of the next state, Q'(s t+1 ,μ's t+1 |θμ')), select the minimum value minQ(s of the two value function networks t+1 ,a) As the Q value of the next state, update the value function network, y t =r t+1 +γminQ'(s t+1 ,μ'(s t+1 |θμ')), using the gradient descent algorithm to update the parameters of the value function network and Every several time steps, use the policy function network μ to generate a new behavior strategy and update the policy function network, use the target value function network to calculate the estimated value of the policy gradient, and update the parameters θμ of the policy function network; Use soft update method to update the target value function network Q1' and Q2', By continuously updating the value function network and the policy function network, they will eventually converge to the optimal policy. Using the converged policy function network, the optimal unloading decision can be obtained.

5. The self-aware multi-edge node computing offloading method for smart elderly care according to claim 1 is characterized in that: Step 4 specifically includes the following steps: Step 4.1: Allocate the required computing resources to the relevant edge nodes according to the obtained optimal offloading strategy.

6. The self-aware multi-edge node computing offloading method for smart elderly care according to claim 1 is characterized in that: Step 5 specifically includes the following steps: Step 5.

1. To ensure the flexible expansion and robust operation of the edge computing network, each edge node is equipped with a token mechanism. The edge nodes are arranged in a virtual closed chain structure, and each node maintains status information by passing tokens. The token mechanism enables edge nodes to dynamically increase or decrease nodes, and automatically update device information and network topology when a failure occurs, ensuring the adaptability and robustness of the system. When the optimal strategy requires increasing or decreasing edge nodes, the token will be automatically updated to ensure that new or removed nodes do not affect the overall computing efficiency. Step 5.2: The token is represented as follows: Among them, workgroup represents the work group of the edge node; is a list of status information for each node in the work group; each All represent the information of edge working node i; servergroup indicates the server group; is a list of status information for each server in the server group; each Represents the information of server node i; All node devices are divided into two parts: workgroups and server groups, where a workgroup has k devices and a server group has t devices; the initial token is generated through a preset test program to record the computing performance, status, and IP address of the device; the system automatically allocates the number of workgroups and server groups suitable for the distributed training task based on the number of devices provided by the user and the predefined workgroup / server group ratio; as the devices are started one by one, they add relevant information to the token; after all devices have been tested, the last device sorts the information in the token and groups the devices; the system dynamically adjusts the configuration by modulating the token information, deleting or adding devices, and reallocates the workgroup / server role if the server group requirements are not met to ensure that the distributed acquisition process is consistent with the latest device information; Step 5.3: In order to enhance the reliability and resilience of the system, a self-healing method is proposed to reduce the risk of failure. The failure of any PS node may cause the entire cluster to be out of order, resulting in system failure. If the token is sent to the next device according to the order in the token, the device is recorded as device 2. Before sending, device 1 tries to communicate with device 2 to determine whether it is still online. If so, send the token. If not, device 1 tries to communicate with device 3 and continues in this way until an online device is found. Step 5.4: When any device is found offline during training, the detection process determines the group it belongs to based on its IP address; if it is in the work group, the device is removed from the token and the system continues to run; if it is in the server group, a global notification is triggered and broadcast to restart all devices on the most recent checkpoint in the cluster; Step 5.

5. Before delivering the token, device 1 removes all offline device information from the token; device 2 waits to receive the token and then performs the same process to check if the next device in the token is online; therefore, no matter which group it belongs to, as long as the token holder works, the system should always remain operational.

7. The self-aware multi-edge node computing offloading method for smart elderly care according to claim 1 is characterized in that: Step 6 specifically includes the following steps: After obtaining computing resources, the edge node continues to run the user request collection module to dynamically obtain new business request data and network status parameters; by periodically uploading the latest parameter data, the central node can continuously optimize the offloading strategy according to the latest network conditions to achieve adaptive updates and dynamic adjustments of the system.

Images