A lightweight edge terminal computing resource sharing collaborative inference method

Abstract

The application discloses a kind of lightweight edge terminal computing resource sharing's collaborative inference method, when the computing resource of certain edge terminal is insufficient to complete task, apply to edge server collaborative inference;Edge server is built-in first strategy network, according to the computing resource of edge terminal and the priority of task to be cooperated, task to be cooperated is distributed to suitable target cooperative equipment, and collaborative inference is realized;Edge server calculates reward, and the state, action before and after collaborative inference and reward are packaged into memory and pressed into memory pool;When reward does not satisfy convergence condition, according to whether the memory quantity of memory pool exceeds threshold value, first decision network, second strategy network, target strategy network, first evaluation network, second evaluation network, first target evaluation network, second target evaluation network built-in in edge server are updated.The application realizes collaborative inference task in the power distribution network edge environment of scarce computing resource, improves task inference real-time performance.

Description

Technical Field

[0001] This invention relates to the technical field of edge intelligence, and in particular to a lightweight collaborative reasoning method for sharing computing resources on edge terminals. Background Technology

[0002] With the rapid improvement of the intelligence level of distribution networks, edge terminals are widely used in scenarios such as status monitoring, fault diagnosis, and protection control, generating a large number of computationally intensive inference tasks. Traditional deployment methods usually rely on the local computing resources of the edge terminal to complete the tasks independently. However, distribution network edge terminals generally have limited hardware capabilities, strong resource heterogeneity, and dynamic load changes, which makes them prone to resource contention and response delays in high-concurrency task scenarios, making it difficult to meet the real-time requirements of intelligent applications.

[0003] Existing technologies have attempted to improve edge computing efficiency through task offloading or resource collaboration, such as migrating tasks to edge servers or the cloud for processing. However, these methods often fail to fully consider the dynamic and heterogeneous nature of distribution network terminal resources, lacking a comprehensive perception and optimization allocation mechanism for multiple resource dimensions (such as CPU, memory, and bandwidth), which can easily lead to problems such as load imbalance, low efficiency, and insufficient resource utilization. Furthermore, most methods rely on complex decision-making models, making it difficult to achieve lightweight, fast, and real-time decision-making in resource-constrained distribution network edge environments.

[0004] Therefore, there is an urgent need for an efficient collaborative reasoning method for distribution network edge terminals that can dynamically schedule edge terminal computing resources based on the real-time resource status and task characteristics of the terminals, especially sharing the computing resources of idle edge terminals. While ensuring the balance of computing resources and real-time response, it can achieve lightweight intelligent decision-making and resource optimization, thereby supporting highly reliable and low-latency edge intelligent applications in distribution networks. Summary of the Invention

[0005] The purpose of this invention is to solve the problems of limited resources, unbalanced load, and difficulty in ensuring the real-time performance of tasks at the edge of the power distribution network. It proposes a lightweight collaborative reasoning method for sharing computing resources at the edge of the network. This method achieves dynamic perception of the terminal computing status and adaptive allocation of collaborative tasks by constructing an intelligent decision-making system based on a dual-policy network. It also introduces a reward function with efficiency as the core to optimize the resource scheduling and sharing process, thereby significantly improving the response speed, resource utilization efficiency, and overall balance of intelligent applications at the edge of the power distribution network.

[0006] To achieve the above objectives, the technical solution provided by this invention is: a lightweight collaborative reasoning method for sharing computing resources on edge terminals, comprising the following steps: S1: Edge terminals at fixed time intervals Upload its computing resources to the edge server; when the computing resources of an edge terminal are insufficient to complete the task, upload the data, priority, and completion progress of the task to be coordinated, as well as the edge terminal's number, to the edge server to request collaborative inference; S2: Edge server with built-in first policy network First Strategy Network It comprises three fully connected layers. Its input states include the computing resources of each edge terminal and the priorities of the tasks to be coordinated. Its output actions are the probability distribution of each action taken given a particular state, selecting the action with the highest probability. This is then processed through a first policy network. Obtain the target collaborative edge terminal number to guide the edge server decision-making process, then allocate the collaborative tasks according to the target collaborative edge terminal number, and continue to complete the collaborative tasks according to the data and completion progress of the collaborative tasks, thereby realizing collaborative reasoning of edge terminal computing resource sharing; S3: After the collaborative task is assigned, the computing resources of the edge terminal are updated; the edge server calculates the reward, and at the same time packages the state, actions and rewards before and after collaborative reasoning into memory and pushes them into the memory pool; S4: If the reward meets the convergence condition, return to step S1; otherwise, proceed to step S5. S5: When the number of memories in the memory pool exceeds the first threshold At that time, randomly select One memory is used to update the second policy network built into the edge server. Target-Policy Network First Evaluation Network Second evaluation network First objective evaluation network Second objective evaluation network Otherwise, return to step S1; second strategy network and target policy network It consists of five identical fully connected layers. Its input is the state, and its output is the action, yielding the probability distribution of each action to be taken in a given state, thus guiding the agent's decision-making process; the first evaluation network... Second evaluation network First objective evaluation network Second objective evaluation network It contains five fully connected layers with the same structure. Its input is state and action, and its output is evaluation value. It is used to evaluate whether the selected target collaborative edge terminal is reasonable under the given edge terminal computing resources and priority of the task to be collaborated. S6: When the number of memories in the memory pool exceeds the second threshold At that time, randomly select One memory is used to update the first policy network. Otherwise, return to step S1; when the first policy network After the update is complete, return to step S1; where the second threshold... Greater than the first threshold .

[0007] Furthermore, in step S1, the computing resources of the edge terminal include CPU utilization, RAM utilization, and network bandwidth; determining that the computing resources of a certain edge terminal are insufficient to complete the task specifically involves: for the i-th edge terminal, the r-th type of computing resource... This includes CPU utilization and RAM utilization. Any computing resource must meet the following conditions: ; In the formula, The amount of computational resource r required for the j-th task is... A threshold for computing resources for edge terminals. Let be the total amount of the r-th type of computing resources for the i-th edge terminal.

[0008] Furthermore, in step S3, a larger edge server computing reward indicates a more balanced and efficient sharing of computing resources. Based on this, the reward... Expressed as efficiency ,Right now: ; In the formula, Indicates the state Get Action in the case The reward, where the subscript t represents time; efficiency Average task completion time Performance balance rate and task completion rate Decide; Average task completion time This is the ratio of the sum of task completion times to the number of tasks completed, i.e.: ; In the formula, Let be the task completion time for the j-th task. The number of tasks completed; Performance Balance Rate Represented as a weighted sum of different computing resource balance rates, for the i-th edge terminal with the r-th type of computing resource... This includes CPU utilization and RAM utilization, and its resource balancing rate. Represented as: ; In the formula, Let r be the average value of the computing resource type r across all edge terminals, i.e. ,in For the number of edge terminals, To find the sign of the absolute value; Ultimately, performance balance rate This is expressed as a weighted sum of the resource balancing rate, which includes CPU utilization and RAM utilization. ; In the formula, Resource balancing rate is the CPU utilization rate. Resource balancing rate for RAM utilization These are weighting coefficients; Task completion rate This is expressed as the ratio of the number of tasks completed to the number of tasks generated, i.e.: ; In the formula, Generate the number of tasks; Performance Balance Rate The higher the completion rate, the better. The higher the average task completion time The shorter, the more efficient. The higher the efficiency, the better. Defined as: .

[0009] Furthermore, in step S4, the convergence condition for the reward is: ; In the formula, To be in state at time t+1 Get Action in the case The reward To be in state at time t Get Action in the case The reward Let lim be the convergence threshold, and let lim be the limit sign. It is the symbol for infinity.

[0010] Furthermore, in step S5, when the number of memory pools exceeds the first threshold... Update the first evaluation network at that time. Second evaluation network The method involves calculating the loss function, namely: ; ; In the formula, To assess the loss of the first evaluation network, The loss of the second evaluation network, The parameters of the first evaluation network are... The parameters of the second evaluation network, To calculate the expected value, The first evaluation network in state and actions The output evaluation value under the given circumstances. For the second evaluation network in state and actions The output evaluation value under the given circumstances. Let be the depreciation factor, and min be the minimum function. The primary objective is to evaluate the network in state. and actions The output evaluation value under the given circumstances. To evaluate the network in the state for the second objective and actions The output evaluation value under the given circumstances. Let ln(x) be the entropy coefficient, and ln() be the logarithmic function. For the second policy network In state Execute action at time The probability value, The next time step is the state-action policy entropy, which is then used to account for the randomness of the policy. Gradient backpropagation updates the parameters of the first and second evaluation networks, i.e.: ; ; In the formula, The learning rate is used to evaluate the network first. The learning rate of the second evaluation network. The gradient of the first evaluation network is used. The gradient of the second evaluation network, To determine the gradient sign; Update the second strategy network The method involves calculating the loss function, namely: ; In the formula, For the second policy network The loss, For the second policy network In state Execute action at time The probability value, For the second policy network The parameters, The entropy is the state-action policy at the current moment. Gradient backpropagation updates the second policy network The parameters, namely: ; In the formula, For the second policy network The learning rate For the second policy network The gradient; Update the first objective evaluation network Second objective evaluation network and target policy network The method is as follows: ; ; ; In the formula, The parameters of the network are evaluated for the primary objective. The parameters of the network are evaluated for the second objective. For target policy network The parameters, This is for updating coefficients.

[0011] Furthermore, in step S6, when the number of memory pools exceeds the second threshold... Update the first policy network at that time. The method involves calculating the Kullback-Leibler loss function, i.e.: ; In the formula, For the first strategy network The loss is calculated as follows: k is the index of the output node, and the number of output nodes is proportional to the number of edge terminals. same, For the Softmax function, i.e. ,in Let C be the output value of the nth node in the network, and C be the number of output nodes in the network. Let c be the output value of the c-th node in the network, where c is the index of the output node. It is an exponential function; For the first strategy network The output probability of the k-th node For target policy network The output probability of the k-th node This is the temperature coefficient.

[0012] Compared with the prior art, the present invention has the following advantages and beneficial effects: 1. This invention innovatively adopts a dual-policy network architecture (i.e., including a first policy network). Second policy network ), where the first strategy network is used for online decision-making Lightweight (with second policy network) Compared to the first-policy network It contains fewer fully connected layers, enabling lightweight decision-making, has low computational overhead, and fast response speed, meeting the stringent low-latency requirements of scenarios such as distribution network fault handling. The more complex second-policy network... This is used for offline learning and optimization. The two work together to ensure decision-making efficiency without sacrificing final performance.

[0013] 2. Unlike traditional methods that only consider a single resource indicator, this invention comprehensively considers the status of multi-dimensional computing resources such as CPU utilization and RAM utilization, and designs an efficiency reward that integrates average task completion time, performance balancing rate, and task completion rate. This design can effectively avoid overload of a single terminal, promote balanced sharing and efficient utilization of computing resources across terminals, and thus significantly improve the overall stability and task throughput of edge computing in the power distribution network.

[0014] 3. This invention employs a policy network to separate the learning process from real-time decision-making. By calculating the state-action policy entropy, it enhances exploration capabilities and prevents the policy from prematurely converging to local optima. This approach enables the method to adaptively learn the dynamically changing distribution network edge environment, continuously optimize the collaborative reasoning strategy, and ultimately obtain higher and more stable long-term rewards. Its intelligence level is significantly superior to traditional methods based on fixed rules. Attached Figure Description

[0015] Figure 1 This is a flowchart of the method of the present invention.

[0016] Figure 2 This is a diagram showing the network update relationships and environmental interaction relationships between the various networks in this invention. Detailed Implementation

[0017] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0018] like Figure 1 As shown, this embodiment discloses a collaborative reasoning method for lightweight edge terminal computing resource sharing. The method involves edge terminals periodically reporting resource status and requesting collaborative reasoning when resources are insufficient. The edge server utilizes a lightweight policy network to make real-time decisions on task allocation and continuously optimizes the scheduling strategy through a dual-network training mechanism, ultimately achieving efficient and balanced sharing of edge computing resources in the power distribution network. The specific details are as follows: S1: Edge terminals at fixed time intervals Upload its computing resources to the edge server; when the computing resources of an edge terminal are insufficient to complete the task, upload the data, priority, and completion progress of the task to be coordinated, as well as the edge terminal's number, to the edge server to request collaborative inference.

[0019] S2: Edge server with built-in first policy network First Strategy Network It comprises three fully connected layers, whose input states (including the computing resources of each edge terminal and the priorities of the tasks to be coordinated) and output actions (i.e., the probability distribution of taking each action given a state, and selecting the action with the highest probability); through the first policy network. Obtain the target collaborative edge terminal number to guide the edge server decision-making process, then allocate the collaborative tasks according to the target collaborative edge terminal number, and continue to complete the collaborative tasks according to the data and completion progress of the collaborative tasks, thereby realizing collaborative reasoning of edge terminal computing resource sharing; S3: After the collaborative task is assigned, the computing resources of the edge terminal are updated; the edge server calculates the reward, and at the same time packages the state, actions and rewards before and after collaborative reasoning into memory and pushes them into the memory pool; S4: When the reward meets the convergence condition, return to step S1; otherwise, proceed to step S5.

[0020] S5: When the number of memories in the memory pool exceeds the first threshold At that time, randomly select One memory is used to update the second policy network built into the edge server. Target-Policy Network First Evaluation Network Second evaluation network First objective evaluation network Second objective evaluation network Otherwise, return to step S1; second strategy network and target policy network It consists of five identical fully connected layers. Its input is the state, and its output is the action, yielding the probability distribution of each action to be taken in a given state, thus guiding the agent's decision-making process; the first evaluation network... Second evaluation network First objective evaluation network Second objective evaluation network It contains five fully connected layers with the same structure. It takes input states and actions as inputs and outputs evaluation values. It is used to evaluate whether the selected target collaborative edge terminal is reasonable given the computing resources of the specified edge terminal and the priority of the task to be collaborated on. S6: When the number of memories in the memory pool exceeds the second threshold At that time, randomly select One memory is used to update the first policy network. Otherwise, return to step S1; when the first policy network After the update is complete, return to step S1; where the second threshold... Greater than the first threshold .

[0021] Specifically, the computing resources of an edge terminal include CPU utilization, RAM utilization, and network bandwidth, with time intervals... To determine if the computing resources of a certain edge terminal are insufficient to complete the task, specifically, for the i-th edge terminal with the r-th type of computing resources... This includes CPU utilization and RAM utilization. Any computing resource must meet the following conditions: ; In the formula, The amount of computational resource r required for the j-th task is... A threshold for computing resources for edge terminals. Let be the total amount of the r-th type of computing resources for the i-th edge terminal.

[0022] like Figure 2 As shown, this embodiment provides a network update relationship and environmental interaction relationship diagram between various networks, and the edge terminal and the edge server interact through state-action.

[0023] Specifically, the first strategy network It contains three fully connected layers, with 128, 256, and 128 neurons in each layer, respectively. ,in The number of edge terminals is used to map the state to the probability distribution of each action. Each action is the number of the edge terminal, and the action with the highest probability is the number of the target edge terminal.

[0024] A larger reward for edge server computing indicates a more balanced and efficient sharing of computing resources. Based on this, the reward... Expressed as efficiency ,Right now: ; In the formula, Indicates the state Get Action in the case The reward, where the subscript t represents time; efficiency Average task completion time Performance balance rate and task completion rate Decide; Average task completion time This is the ratio of the sum of task completion times to the number of tasks completed, i.e.: ; In the formula, Let be the task completion time for the j-th task. The number of tasks completed; Performance Balance Rate Represented as a weighted sum of different computing resource balance rates, for the i-th edge terminal with the r-th type of computing resource... This includes CPU utilization and RAM utilization, and its resource balancing rate. Represented as: ; In the formula, Let r be the average value of the computing resource type r across all edge terminals, i.e. ,in For the number of edge terminals, To find the sign of the absolute value; Ultimately, performance balance rate This is expressed as a weighted sum of the resource balancing rate, which includes CPU utilization and RAM utilization. ; In the formula, Resource balancing rate is the CPU utilization rate. Resource balancing rate for RAM utilization These are weighting coefficients; Task completion rate This is expressed as the ratio of the number of tasks completed to the number of tasks generated, i.e.: ; In the formula, Generate the number of tasks; Performance Balance Rate The higher the completion rate, the better. The higher the average task completion time The shorter, the more efficient. The higher the efficiency, the better. Defined as: ; Specifically, the convergence condition for the reward is: ; In the formula, To be in state at time t+1 Get Action in the case The reward To be in state at time t Get Action in the case The reward The convergence threshold is set to 1e-4 in this embodiment, and lim represents the limit sign. It is the symbol for infinity.

[0025] like Figure 2 As shown, this embodiment provides a network update relationship and environment interaction relationship diagram between various networks, and the networks iteratively update each other; Specifically, the second strategy network Target-Policy Network It contains five fully connected layers with 128, 256, 512, and 256 neurons respectively. ,in Given the number of edge terminals, the state is mapped to a probability distribution of each action, where each action is represented by an edge terminal number, and the action with the highest probability is the target edge terminal number; First evaluation network Second evaluation network First objective evaluation network Second objective evaluation network It contains five fully connected layers with 128, 256, 512, 256, and 1 neurons respectively. The input is state and action, and the output is evaluation value.

[0026] When the number of memory pools exceeds the first threshold Update the first evaluation network at that time. Second evaluation network The method involves calculating the loss function, namely: ; ; In the formula, To assess the loss of the first evaluation network, The loss of the second evaluation network, The parameters of the first evaluation network are... The parameters of the second evaluation network, To calculate the expected value, The first evaluation network in state and actions The output evaluation value under the given circumstances. For the second evaluation network in state and actions The output evaluation value under the given circumstances. Let be the depreciation factor, and min be the minimum function. The primary objective is to evaluate the network in state. and actions The output evaluation value under the given circumstances. To evaluate the network in the state for the second objective and actions The output evaluation value under the given circumstances. Let ln(x) be the entropy coefficient, and ln() be the logarithmic function. For the second policy network In state Execute action at time The probability value, The next time step is the state-action policy entropy, which is then used to account for the randomness of the policy. Gradient backpropagation updates the parameters of the first and second evaluation networks, i.e.: ; ; In the formula, The learning rate is used to evaluate the network first. The learning rate of the second evaluation network. The gradient of the first evaluation network is used. The gradient of the second evaluation network, To determine the gradient sign; Update the second strategy network The method involves calculating the loss function, namely: ; In the formula, For the second policy network The loss, For the second policy network In state Execute action at time The probability value, For the second policy network The parameters, The entropy is the state-action policy at the current moment. Gradient backpropagation updates the second policy network The parameters, namely: ; In the formula, For the second policy network The learning rate For the second policy network The gradient; Update the first objective evaluation network Second objective evaluation network and target policy network The method is as follows: ; ; ; In the formula, The parameters of the network are evaluated for the primary objective. The parameters of the network are evaluated for the second objective. For target policy network The parameters, This is for updating coefficients.

[0027] When the number of memory pools exceeds the second threshold Update the first policy network at that time. The method involves calculating the Kullback-Leibler loss function, i.e.: ; In the formula, For the first strategy network The loss is calculated as follows: k is the index of the output node, and the number of output nodes is proportional to the number of edge terminals. same, For the Softmax function, i.e. ,in Let C be the output value of the nth node in the network, and C be the number of output nodes in the network. Let c be the output value of the c-th node in the network, where c is the index of the output node. It is an exponential function; For the first strategy network The output probability of the k-th node For target policy network The output probability of the k-th node This is the temperature coefficient.

[0028] Below, we will conduct simulation verification by simulating a real-world distribution network edge terminal deployment scenario. In a distribution network, different types of edge terminals exist, generating inference tasks at different time intervals. Using smart circuit breakers, power quality monitors, smart meters, data terminal units (DTUs), feeder terminal units (FTUs), distribution transformer monitoring terminals (TTUs), and remote terminal units (RTUs) as different types of edge terminals, inference tasks are generated at time intervals of 2s, 5s, 10s, 15s, 20s, and 5 minutes, respectively. For example, smart circuit breakers perform topology identification inference tasks at 2-second intervals. The shorter the time interval for generating inference tasks, the greater the likelihood of insufficient computing resources for the edge terminals, and the more necessary it is to request collaborative inference.

[0029] To verify the effectiveness of the method of this invention, experiments were conducted in the above-mentioned scenario, and the average task completion time, performance balancing rate, task completion rate, and efficiency of this invention were used as evaluation criteria. The experiment lasted 60 minutes and was repeated 10 times to eliminate the influence of chance. To provide a benchmark, this embodiment was compared with the method proposed in this invention without its application. The experimental results are shown in Table 1.

[0030] Table 1

[0031] As shown in the table above, applying the method of this invention reduces the average task completion time, improves the performance balancing rate, increases the task completion rate, and significantly improves efficiency. The experimental results fully demonstrate the effectiveness of the method of this invention, which helps to utilize the computing resources of idle edge terminals to achieve collaborative reasoning between the edge and terminal, and improves the real-time performance of edge terminal task reasoning.

[0032] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A lightweight collaborative reasoning method for sharing computing resources on edge terminals, characterized in that, Includes the following steps: S1: Edge terminals at fixed time intervals Upload its computing resources to the edge server; when the computing resources of an edge terminal are insufficient to complete the task, upload the data, priority, and completion progress of the task to be coordinated, as well as the edge terminal's number, to the edge server to request collaborative inference; S2: Edge server with built-in first policy network First Strategy Network It comprises three fully connected layers. Its input states include the computing resources of each edge terminal and the priorities of the tasks to be coordinated. Its output actions are the probability distribution of each action taken given a particular state, selecting the action with the highest probability. This is then processed through a first policy network. Obtain the target collaborative edge terminal number to guide the edge server decision-making process, then allocate the collaborative tasks according to the target collaborative edge terminal number, and continue to complete the collaborative tasks according to the data and completion progress of the collaborative tasks, thereby realizing collaborative reasoning of edge terminal computing resource sharing; S3: After the collaborative task is assigned, the computing resources of the edge terminal are updated; the edge server calculates the reward, and at the same time packages the state, actions and rewards before and after collaborative reasoning into memory and pushes them into the memory pool; S4: If the reward meets the convergence condition, return to step S1; otherwise, proceed to step S5. S5: When the number of memories in the memory pool exceeds the first threshold At that time, randomly select One memory is used to update the second policy network built into the edge server. Target-Policy Network First Evaluation Network Second evaluation network First objective evaluation network Second objective evaluation network Otherwise, return to step S1; second strategy network and target policy network It consists of five identical fully connected layers. Its input is the state, and its output is the action, yielding the probability distribution of each action to be taken in a given state, thus guiding the agent's decision-making process; the first evaluation network... Second evaluation network First objective evaluation network Second objective evaluation network It contains five fully connected layers with the same structure. Its input is state and action, and its output is evaluation value. It is used to evaluate whether the selected target collaborative edge terminal is reasonable under the given edge terminal computing resources and priority of the task to be collaborated. S6: When the number of memories in the memory pool exceeds the second threshold At that time, randomly select One memory is used to update the first policy network. Otherwise, return to step S1; when the first policy network After the update is complete, return to step S1; where the second threshold... Greater than the first threshold .

2. The collaborative reasoning method for lightweight edge terminal computing resource sharing according to claim 1, characterized in that, In step S1, the computing resources of the edge terminal include CPU utilization, RAM utilization, and network bandwidth; determining that the computing resources of a certain edge terminal are insufficient to complete the task specifically involves: for the i-th edge terminal, the r-th type of computing resource... This includes CPU utilization and RAM utilization. Any computing resource must meet the following conditions: ； In the formula, The amount of computational resource r required for the j-th task is... A threshold for computing resources for edge terminals. Let be the total amount of the r-th type of computing resources for the i-th edge terminal.

3. The collaborative reasoning method for lightweight edge terminal computing resource sharing according to claim 2, characterized in that, In step S3, a larger reward for edge server computation indicates a more balanced and efficient sharing of computing resources. Based on this, the reward... Expressed as efficiency ,Right now: ； In the formula, Indicates the state Get Action in the case The reward, where the subscript t represents time; efficiency Average task completion time Performance balance rate and task completion rate Decide; Average task completion time It is the ratio of the sum of task completion times to the number of tasks completed, i.e.: ； In the formula, Let be the task completion time for the j-th task. The number of tasks completed; Performance Balance Rate Represented as a weighted sum of different computing resource balance rates, for the i-th edge terminal with the r-th type of computing resource... This includes CPU utilization and RAM utilization, and its resource balancing rate. Represented as: ； In the formula, Let r be the average value of the computing resource type r across all edge terminals, i.e. ,in For the number of edge terminals, To find the sign of the absolute value; Ultimately, performance balance rate This is expressed as a weighted sum of the resource balancing rate, which includes CPU utilization and RAM utilization. ； In the formula, Resource balancing rate is the CPU utilization rate. Resource balancing rate for RAM utilization These are weighting coefficients; Task completion rate This is expressed as the ratio of the number of tasks completed to the number of tasks generated, i.e.: ； In the formula, Generate the number of tasks; Performance Balance Rate The higher the completion rate, the better. The higher the average task completion time The shorter, the more efficient. The higher the efficiency, the better. Defined as: 。 4. The collaborative reasoning method for lightweight edge terminal computing resource sharing according to claim 3, characterized in that, In step S4, the convergence condition for the reward is: ； In the formula, To be in state at time t+1 Get Action in the case The reward To be in state at time t Get Action in the case The reward Let lim be the convergence threshold, and let lim be the limit sign. It is the symbol for infinity.

5. The collaborative reasoning method for lightweight edge terminal computing resource sharing according to claim 4, characterized in that, In step S5, when the number of memory pools exceeds the first threshold Update the first evaluation network at that time. Second evaluation network The method involves calculating the loss function, namely: ； ； In the formula, To assess the loss of the first evaluation network, The loss of the second evaluation network, The parameters of the first evaluation network are... The parameters of the second evaluation network, To calculate the expected value, The first evaluation network in state and actions The output evaluation value under the given circumstances. For the second evaluation network in state and actions The output evaluation value under the given circumstances. Let be the depreciation factor, and min be the minimum function. The primary objective is to evaluate the network in state. and actions The output evaluation value under the given circumstances. To evaluate the network in the state for the second objective and actions The output evaluation value under the given circumstances. Let ln(x) be the entropy coefficient, and ln() be the logarithmic function. For the second policy network In state Execute action at time The probability value, The next time step is the state-action policy entropy, which is then used to account for the randomness of the policy. Gradient backpropagation updates the parameters of the first and second evaluation networks, i.e.: ； ； In the formula, The learning rate is used to evaluate the network first. The learning rate of the second evaluation network. The gradient of the first evaluation network is used. The gradient of the second evaluation network, To determine the gradient sign; Update the second strategy network The method involves calculating the loss function, namely: ； In the formula, For the second policy network The loss, For the second policy network In state Execute action at time The probability value, For the second policy network The parameters, The entropy is the state-action policy at the current moment. Gradient backpropagation updates the second policy network The parameters, namely: ； In the formula, For the second policy network The learning rate For the second policy network The gradient; Update the first objective evaluation network Second objective evaluation network and target policy network The method is as follows: ； ； ； In the formula, The parameters of the network are evaluated for the primary objective. The parameters of the network are evaluated for the second objective. For target policy network The parameters, This is for updating coefficients.

6. The collaborative reasoning method for lightweight edge terminal computing resource sharing according to claim 5, characterized in that, In step S6, when the number of memory pools exceeds the second threshold Update the first policy network at that time. The method involves calculating the Kullback-Leibler loss function, i.e.: ； In the formula, For the first strategy network The loss is calculated as follows: k is the index of the output node, and the number of output nodes is proportional to the number of edge terminals. same, For the Softmax function, i.e. ,in Let C be the output value of the nth node in the network, and C be the number of output nodes in the network. Let c be the output value of the c-th node in the network, where c is the index of the output node. It is an exponential function; For the first strategy network The output probability of the k-th node For target policy network The output probability of the k-th node This is the temperature coefficient.

Images