Reliable reasoning scheduling method based on edge hybrid expert large model

Abstract

A reliable reasoning scheduling method based on an edge hybrid expert large model relates to the field of computers, and comprises the following steps: deploying a MoE large model to an edge network in a distributed edge manner; malicious node behavior modeling is carried out, distributed cooperative reasoning is simulated, and an auto-encoder input vector space set V and a reasoning task state set S are generated respectively; performing abnormal node detection neural network training to obtain calculation reliability of each edge node; performing deep reinforcement learning neural network training to obtain an expert selection strategy; and performing reliable reasoning scheduling of the edge MoE large model based on the trained deep reinforcement learning neural network. According to the method, the service quality and privacy protection advantages of distributed edge deployment are utilized, according to an original routing result of a large model gating function to an expert network, the node reliability and real-time load in an actual dynamic deployment environment are combined, the reasoning process is effectively optimized, and the reasoning efficiency is improved. Therefore, the cooperative reasoning efficiency and reliability of the MoE large model under distributed edge deployment are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more specifically, to a reliable reasoning scheduling method based on an edge hybrid expert large model. Background Art

[0002] In recent years, large language models (LLMs) have made significant progress in natural language processing, multimodal interaction and other fields. As an innovative model architecture, mixture of experts (MoE) has achieved an important breakthrough in balancing model capability expansion and computing efficiency improvement. Its sparse activation and parallel computing working characteristics also make edge-enabled distributed LLM deployment possible. Compared with centralized cloud deployment, edge computing can significantly reduce service latency and improve user data privacy. However, LLMs have extremely high requirements for computing resources and usually rely on cloud-edge collaboration or joint edge computing to achieve collaborative reasoning among multiple nodes. However, this distributed deployment is susceptible to uncertain factors such as device heterogeneity and node failure, resulting in uneven node load and reduced reasoning efficiency. At the same time, edge devices are limited by their own security protection capabilities and are susceptible to external environmental interference and malicious attacks, resulting in reduced reasoning reliability.

[0003] Model quantization technology can improve reasoning efficiency while reducing model resource usage, facilitating lightweight deployment of LLMs on edge devices, but it will affect the reasoning accuracy to a certain extent; model sharding strategy optimization technology can achieve resource-friendly distributed deployment of static large models, but it cannot cope with the problems of uneven node load and malicious attacks faced in actual reasoning environments. Summary of the invention

[0004] In order to solve the problems of low inference accuracy and inability to cope with uneven node load and malicious attacks in actual inference environments in existing model quantization technology and model sharding strategy optimization technology, the present invention provides a reliable inference scheduling method based on edge hybrid expert large model. The present invention uses an anomaly detection algorithm to calculate node reliability, and uses deep reinforcement learning to achieve expert activation with optimal inference efficiency and reliability, thereby improving the inference accuracy to solve the problems of uneven node load and malicious attacks in actual inference environments.

[0005] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0006] The present invention provides a reliable reasoning scheduling method based on an edge hybrid expert large model, comprising the following steps:

[0007] Step S1: Distributed edge deployment of the MoE large model to the edge network;

[0008] Step S2: Model malicious node behavior, simulate distributed collaborative reasoning, and generate the autoencoder input vector space set V and the reasoning task state set S respectively;

[0009] Step S3: Perform abnormal node detection neural network training to obtain the calculation reliability of each edge node;

[0010] Step S4: Perform deep reinforcement learning neural network training to obtain an expert selection strategy;

[0011] Step S5: Perform reliable inference scheduling of the edge MoE large model based on the trained deep reinforcement learning neural network.

[0012] Furthermore, the specific implementation process of step S1 is as follows:

[0013] S1.1: Divide the MoE large model into N layers according to the MoE blocks, and deploy each layer on a group of devices;

[0014] The MoE block includes: a MoE layer, a residual connection layer, a normalization layer, a self-attention layer and a gating network; the MoE layer includes multiple expert networks, each MoE layer contains E experts, and E is an integer greater than 1; the residual connection layer, the normalization layer, and the self-attention layer constitute a self-attention mechanism, and the gating network constitutes a gating mechanism, which is used to realize expert routing selection;

[0015] S1.2: The expert networks deployed on the edge nodes belonging to the same device group constitute a complete set of MoE layers, and the remaining layer structures are shared among the edge nodes in the same group.

[0016] Furthermore, the specific implementation process of step S2 is as follows:

[0017] S2.1: Upload batch inference tasks to the distributed edge deployed MoE large model; the inference task represents an end-to-end inference from the first layer MoE block to the Nth layer MoE block;

[0018] S2.2: Perform a Top-2 expert routing for each inference task and route it to the expert network deployed on the first layer of devices;

[0019] S2.3: The reasoning task is queued at the expert network of the corresponding device edge node, waiting for calculation with the expert network;

[0020] S2.4: Based on the malicious node behavior modeling, each malicious node participating in the reasoning task at this layer performs malicious behavior;

[0021] S2.5: Calculate the reasoning time of the reasoning task, merge the reasoning results of each routed expert at the edge node expert network with the slowest reasoning progress, and perform subsequent calculations;

[0022] S2.6: Calculate the reasoning time of the task at this level, return to step S2.2, and perform reasoning at the next level until the reasoning task is completed.

[0023] Furthermore, the specific implementation process of step S2.4 is as follows:

[0024] S2.4.1: The node is offline, which means that the node has no calculation results and the reasoning task waits for a response indefinitely;

[0025] S2.4.2: Energy-delay attack, manifested as a node's reasoning time being too long;

[0026] S2.4.3: Dishonest calculation, which takes the form of reducing the credibility of the reasoning result C;

[0027] S2.4.4: Each malicious node only participates in one type of malicious behavior: if the node is offline, it will not participate in subsequent reasoning; if the node participates in energy-delay attacks or dishonest calculations, it will perform malicious behavior according to probability in each round of reasoning.

[0028] Furthermore, the specific implementation process of step S2.5 is as follows:

[0029] S2.5.1: Calculate the floating point computation required for an expert network to perform a reasoning task:

[0030] D comp =4m·m h +2m h ·m+η·m h +m h ;

[0031] Where m represents the embedding dimension of the token, m h represents the dimension of the hidden layer of the neural network, and η represents the FLOPS (floating point operations per second) required for the activation function calculation;

[0032] S2.5.2: Calculate the amount of data D transmitted between nodes tran =ζ·m; where ζ represents the quantization accuracy of the neural network weights;

[0033] S2.5.3: When no malicious behavior occurs during the reasoning process, the reasoning time between adjacent layers of a reasoning task is:

[0034]

[0035] In the formula, i and j represent the experts being routed, B i represents the communication bandwidth of the node device where expert i is located, F i represents the FLOPS of the node device where expert i is located, B j represents the communication bandwidth of the node device where expert j is located, Fj represents the FLOPS of the node device where expert j is located, Q i represents the number of queued tasks of the GPU of the node device where expert i is located, Q j represents the queued task volume of the GPU of the node where expert j is located, T i represents the time required for expert i to complete the current reasoning task, T j represents the time required for expert j to complete the current reasoning task, represents the subsequent calculation time of each device except the expert network calculation. If T i Greater than or equal to T j ,but If T i Less than T j ,but represents the subsequent computing time of the node device where expert i is located, excluding the expert network computing. It represents the subsequent calculation time of the node device where expert j is located, except for the expert network calculation, which is expressed as

[0036] if T i ≥T j ,else

[0037] Furthermore, when the reasoning task at this level involves malicious behavior, the node is offline, which is represented by T = +∞;

[0038] For energy-delay attack, it manifests as:

[0039]

[0040] In the formula, q i represents the number of queued tasks in the expert network where expert i is located, q j represents the amount of queued tasks in the expert network where expert j is located;

[0041] For dishonest calculations, there is no significant impact on the reasoning time, but the credibility of the reasoning results will be reduced from C=1 to C=0.

[0042] Furthermore, the specific implementation process of step S3 is as follows:

[0043] S3.1: Initialize sample pool D1; Initialize compression network W(θ e ,θ d ), θ e and θ d Both represent compressed networks; initialize the estimated network θ m Represents the estimated network parameters; the reliability of each node is initialized to P = [p1, p2, ..., pg ,…,p G ]=[1,1,…,1,…,1];

[0044] S3.2: Store the input vector space set V constructed in step S2 into the sample pool D1; randomly take I input vector samples x from the sample pool D1 i ∈V;

[0045] S3.3: Batch input of randomly selected I input vector samples into the compression network to obtain the low-dimensional spatial feature information z of the i-th input vector after compression ci ∈Z c and the compressed and reconstructed high-dimensional vector x i ′, and then the reconstruction error is obtained

[0046] S3.4: Input I length-2 constructed vectors z = [z1, z2, ..., z i ,...,z I ]stz i =(z ci ,z ri ), i∈[1,I], the estimation network specifically adopts Gaussian mixture model GMM, the number of Gaussian clusters it contains is k=3, and the estimation network output through the softmax layer is obtained: Respectively represent the probability that a node belongs to each Gaussian cluster;

[0047] S3.5: Estimate the weight values, mean vectors and covariance matrices of various Gaussian clusters of the Gaussian mixture model GMM based on the above probabilities:

[0048]

[0049] In the formula, represents the weight value of the Gaussian cluster, represents the mean vector of the Gaussian cluster, = represents the covariance matrix of the Gaussian cluster, Indicates i number Base The probability of a Gaussian cluster;

[0050] S3.6: Calculate the sample energy of each input vector:

[0051]

[0052] Mode middle, Indicates high The inverse matrix of the cluster covariance matrix;

[0053] S3.7: Update network parameters based on the objective function;

[0054] S3.8: Compute node reliability based on the trained DAGMM network.

[0055] Furthermore, the specific implementation process of step S3.7 is as follows:

[0056] S3.7.1: Calculate the objective function of DAGMM:

[0057]

[0058] In the formula, E(z i )+ represents the sample energy of the input vector, represents the regularization term related to the covariance matrix, λ1 and λ2 represent the weight factors of the second sample energy and the third regularization term respectively;

[0059] S3.7.2: Update the neural network parameters θ separately by stochastic gradient descent e ,θ d With θ m ;

[0060] S3.7.3: Return to step S3.2 for the next round of parameter update until the training is completed.

[0061] Furthermore, the specific implementation process of step S3.8 is as follows:

[0062] S3.8.1: Input vector samples x∈V to the trained DAGMM network and obtain the energy E of all vector samples;

[0063] S3.8.2: Select the energy value that accounts for 70% of the energy E as the abnormality judgment threshold;

[0064] S3.8.3: When the energy of the vector sample constructed in step S2 is greater than the abnormality judgment threshold, that is, E i >threshold,stx i ∈V, the sample is judged as abnormal, and the reliability of the corresponding node to which the sample belongs is updated to: p g ←p g +0.3,g∈[1,G]; otherwise, it is judged as a normal update sample, and the reliability of the corresponding node to which the update belongs is: p g ←p g +1,g∈[1,G];

[0065] S3.8.4: The final calculated reliability of each node is: In the formula, k g Indicates the total number of vector samples belonging to the node.

[0066] Furthermore, the specific implementation process of step S4 is as follows:

[0067] S4.1: Initialize sample pool D2; Initialize Q neural network Q(θ) of action-value function; Initialize target action-value function Neural Networks The initial weight parameters of the two types of neural networks are equal, that is,

[0068] S4.2: Store the inference task state set S constructed in step S2 into the sample pool D2; randomly take J samples (s j ,a j ,r j ,s j+1 ), s j With s j+1 Respectively represent the current state and next state of the sample, a j Indicates action, r j Indicates reward;

[0069] S4.3: Calculate the target value of each current state in the sample: by performing action a j The reward after j To update the Q value as the target value y j : If the next state is an absorbing state, then y j =r j ,otherwise:

[0070]

[0071] In the formula, a′ represents the action to be performed in the next state;

[0072] S4.4: Calculate the objective function based on the target value and update the parameter θ through SGD;

[0073] S4.5: Update neural network parameters after every C iterations make C is an integer greater than 1.

[0074] Furthermore, the specific implementation process of step S4.4 is as follows:

[0075] S4.4.1: Based on target value y j Calculate the objective function L(θ)=E[(y j -Q(s j ,a j ;θ)) 2 ];

[0076] S4.4.2: Calculate the gradient of the parameter θ with respect to the objective function:

[0077]

[0078] S4.4.3: Update the parameters θ using SGD.

[0079] The beneficial effects of the present invention are:

[0080] The present invention is studied under a distributed edge network, and the feasibility and usability of the scheme in the scenario are verified by simulating the collaborative reasoning process of the MoE large model. The anomaly detection algorithm DAGMM is used to dynamically generate and adjust the reliability values ​​of the edge nodes, and then the deep reinforcement learning algorithm DDQN is used to obtain the expert activation decision with the best reasoning efficiency and reliability. The experiment verified the optimization effectiveness of the present invention in reasoning efficiency and result reliability under different node reliability scenarios compared with the traditional Top-2 activation and greedy strategies. The present invention utilizes the service quality and privacy protection advantages of distributed edge deployment, and according to the original routing results of the expert network of the large model gating function, combined with the node reliability and real-time load in the actual dynamic deployment environment, effectively optimizes the reasoning process, thereby improving the collaborative reasoning efficiency and reliability of the MoE large model under distributed edge deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 A flow chart of a reliable reasoning scheduling method based on edge hybrid expert large model provided by the present invention;

[0082] Figure 2 This is a diagram of the LLM edge distributed deployment structure of the present invention;

[0083] Figure 3 It is a schematic diagram of reliable reasoning scheduling simulation of the present invention;

[0084] Figure 4 It is the DAGMM training flow chart of the present invention;

[0085] Figure 5 It is the DDQN training flow chart of the present invention;

[0086] Figure 6 It is a relationship diagram between the number of training cycles and training loss of the present invention;

[0087] Figure 7 This is a comparison chart between the present invention and the traditional Top-2 activation strategy and greedy strategy;

[0088] Figure 8 A comparison chart of rewards at different levels of the present invention;

[0089] Fig. 9 This is a relationship diagram between different deployment environments of the present invention and the average reasoning time and result reliability of tasks. DETAILED DESCRIPTION

[0090] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described in detail with reference to the accompanying drawings.

[0091] The reliable reasoning scheduling method based on the edge hybrid expert big model of the present invention utilizes the service quality and privacy protection advantages of distributed edge deployment, and effectively optimizes the reasoning process according to the original routing results of the expert network based on the big model gating function, combined with the node reliability and real-time load in the actual dynamic deployment environment. In the present invention, by simulating multiple types of node internal attacks, real-time monitoring of service time and reasoning results, a discretization algorithm based on Gaussian mixture model (DAGMM) anomaly detection is used to dynamically adjust the reliability evaluation of nodes. Subsequently, the present invention combines the node reliability and the current load of the expert network to solve a multi-layer expert selection optimization problem for reasoning scheduling. The problem is based on the state-action-reward MDP modeling, and a deep reinforcement learning method based on DDQN (Double DQN algorithm) is used for training and updating. In this way, through continuous iteration, the relationship between node reliability and node behavior, expert selection and reasoning environment is autonomously learned, and reliable and efficient plug-in large model reasoning for actual distributed reasoning environment is realized. The simulation results in various edge network scenarios show that the expert activation strategy based on node reliability calculation of the present invention has better reasoning efficiency and result reliability.

[0092] See also Figure 1 To illustrate, the present invention provides a reliable reasoning scheduling method based on edge hybrid expert large model, and its specific implementation process is as follows:

[0093] Step S1: Distributed edge deployment of the MoE large model to the edge network;

[0094] The distributed simulation deploys a MoE large model containing an N-layer hybrid expert block structure to an edge network containing G node devices; where N and G are both integers greater than 1.

[0095] See also Figure 2 To illustrate, the specific implementation process of the MoE large model distributed edge deployment method is as follows:

[0096] S1.1: Divide the MoE large model into N layers according to the MoE blocks, and deploy each layer on a group of devices;

[0097] Among them, the MoE block mainly includes: MoE layer, residual connection layer, normalization layer, self-attention layer and gated network.

[0098] The MoE layer mainly includes multiple expert networks, and the types of expert networks include but are not limited to feed-forward neural networks (FFN). Each MoE layer contains E experts, where E is an integer greater than 1.

[0099] Among them, the residual connection layer, normalization layer, and self-attention layer constitute the self-attention mechanism, and the gated network constitutes the gated mechanism to realize expert routing selection.

[0100] S1.2: The expert networks deployed on the edge nodes of the same device group constitute a complete set of MoE layers, and the remaining layer structures such as residual connection layers, normalization layers, self-attention layers, and gating networks are shared among the edge nodes of the same group.

[0101] Step S2: Model malicious node behavior, simulate distributed collaborative reasoning, and generate the autoencoder input vector space set V and the reasoning task state set S respectively;

[0102] By modeling multiple types of malicious node behaviors, distributed collaborative reasoning is simulated based on the initial node reliability and expert selection strategy, the reasoning information of each node in the process is statistically analyzed, and the input vector space set V of the deep autoencoder is constructed. At the same time, the reasoning task state set S and the corresponding action set A are constructed.

[0103] In the present invention, the input vector sample x of the deep autoencoder is i = is defined as a triple, including the node number g∈[1,G] where the current reasoning task is located, the time T required for the node to perform the reasoning task between adjacent layers, and the credibility of the reasoning result C∈{0,1}, that is, x i =[g,T i ,C i ]∈V.

[0104] In the present invention, the reasoning task state s is defined as a triple, including the model layer number l where the current reasoning task is located, the reliability p of the node where the expert to be activated for the current reasoning task is located = {p i ,p j ,p k ,...}sti,j,k∈[1,G] and the real-time queued task volume q={q i ,q j ,q k ,...}sti,j,k∈[1,G], that is, s j=[l,p,q]∈S. The reliability value of each node is obtained by the trained DAGMM. At the same time, the expert set to be activated is determined by the output of the gated network. The present invention performs secondary expert selection based on this output. Therefore, as a plug-in method, the present invention does not require retraining of the large model.

[0105] In the present invention, the action a of the reasoning task is defined as a binary group, which indicates the two experts finally selected and activated, i.e., a = {a1, a2} ∈ {i, j, k...} ∈ A. After the reasoning task performs the expert selection at the lth layer, the reasoning task is routed to the node where the corresponding expert is located for subsequent reasoning calculation. If the reasoning time of this calculation is low and the reasoning result is highly credible, it will be rewarded; otherwise, it will be punished.

[0106] See also Figure 3 To illustrate, the specific implementation process of the distributed collaborative reasoning simulation process is as follows:

[0107] S2.1: Upload batch inference tasks to the MoE large model deployed at the distributed edge; where the inference task represents an end-to-end inference from the first-layer MoE block to the Nth-layer MoE block;

[0108] S2.2: Top-2 activation;

[0109] Perform a Top-2 expert routing for each inference task and route it to the expert network deployed on the first-layer device;

[0110] S2.3: reasoning task queuing;

[0111] The reasoning tasks are queued at the expert network of the corresponding device edge node, waiting for calculation with the expert network;

[0112] S2.4: Based on the malicious node behavior modeling, each malicious node participating in the reasoning task at this layer performs malicious behavior;

[0113] Among them, the specific implementation process of modeling malicious node behavior is as follows:

[0114] S2.4.1: The node is offline, which means that the node has no calculation results and the reasoning task waits for a response indefinitely;

[0115] S2.4.2: Energy-delay attack, manifested as a node's reasoning time being too long;

[0116] S2.4.3: Dishonest calculation, which takes the form of reducing the credibility of the reasoning result C;

[0117] S2.4.4: Each malicious node only participates in one type of malicious behavior: if the node is offline, it will not participate in subsequent reasoning; if the node participates in energy-delay attacks or dishonest calculations, it will perform malicious behavior according to probability in each round of reasoning.

[0118] S2.5: Calculate the inference time of the inference task, merge the inference results of each routed expert at the edge node expert network with the slowest inference progress, and perform subsequent calculations, including residual calculation, attention calculation, etc.;

[0119] The specific calculation method of the inference time of the inference task is as follows:

[0120] S2.5.1: Calculate the floating point computation required for an expert network to perform a reasoning task:

[0121] D comp =4m·m h +2m h ·m+η·m h +m h ;

[0122] Where m represents the embedding dimension of the token, m h represents the dimension of the hidden layer of the neural network, and η represents the FLOPS (floating point operations per second) required for the activation function calculation;

[0123] S2.5.2: Calculate the amount of data D transmitted between nodes tran =ζ·m; where ζ represents the quantization accuracy of the neural network weights;

[0124] S2.5.3: When no malicious behavior occurs during the reasoning process, the reasoning time between adjacent layers of a reasoning task is:

[0125]

[0126] In the formula, i and j represent the experts being routed, B i represents the communication bandwidth of the node device where expert i is located, F i represents the FLOPS of the node device where expert i is located, B j represents the communication bandwidth of the node device where expert j is located, F j represents the FLOPS of the node device where expert j is located, Q i represents the number of queued tasks of the GPU of the node device where expert i is located, Q j represents the queued task volume of the GPU of the node where expert j is located, T i represents the time required for expert i to complete the current reasoning task, T j represents the time required for expert j to complete the current reasoning task, represents the subsequent calculation time of each device except the expert network calculation. If T i Greater than or equal to T j ,but If T i Less than T j ,but represents the subsequent computing time of the node device where expert i is located, excluding the expert network computing. It represents the subsequent calculation time of the node device where expert j is located, except for the expert network calculation, which is expressed as

[0127] if T i ≥T j ,else

[0128] In particular, when the reasoning task at this level involves malicious behavior, the node is offline, which is represented by T = +∞;

[0129] For energy-delay attack, it manifests as:

[0130]

[0131] In the formula, q i represents the number of queued tasks in the expert network where expert i is located, q j Represents the amount of queued tasks in the expert network where expert j is located.

[0132] For dishonest calculations, there is no significant impact on the reasoning time, but the credibility of the reasoning results will be reduced from C=1 to C=0.

[0133] S2.6: Calculate the reasoning time of the task at this level, return to step S2.2, and perform reasoning at the next level until the reasoning task is completed.

[0134] Step S3: Perform abnormal node detection neural network training to obtain the calculation reliability of each edge node;

[0135] According to the DAGMM training method, the input vector space set V is used as the input of the compression network, and the low-dimensional space feature information Z c and the reconstruction error Z r As the input of the estimation network (Gaussian mixture model GMM), the abnormal node detection neural network training is carried out, and the node reliability calculation method based on DAGMM is used to calculate the computational reliability of each edge node in the current reasoning task round.

[0136] See also Figure 4 To illustrate, the specific implementation process of the node reliability calculation method based on DAGMM is as follows:

[0137] S3.1: Initialize sample pool D1; Initialize compression network W(θ e ,θ d ), θ e and θ d Both represent compressed networks; initialize the estimated network θ m Represents the estimated network parameters; the reliability of each node is initialized to P = [p1, p2, ..., p g ,...,p G ]=[1,1,...,1,...,1];

[0138] S3.2: Store the input vector space set V constructed in step S2 into the sample pool D1; randomly take I input vector samples x from the sample pool D1 i ∈V;

[0139] S3.3: Batch input of randomly selected I input vector samples into the compression network to obtain the low-dimensional spatial feature information z of the i-th input vector after compression ci ∈Z c and the compressed and reconstructed high-dimensional vector x i ′, and then the reconstruction error is obtained Where i∈[1,I];

[0140] S3.4: Input I length-2 constructed vectors z = [z1, z2, ..., z i ,...,z I ]stz i =(z ci ,z ri ), i∈[1,I], the estimation network specifically adopts Gaussian mixture model GMM, the number of Gaussian clusters it contains is k=3, and the estimation network output through the softmax layer is obtained: Respectively represent the probability that a node belongs to each Gaussian cluster;

[0141] S3.5: Estimate the weight values, mean vectors and covariance matrices of various Gaussian clusters of the Gaussian mixture model GMM based on the above probabilities:

[0142]

[0143] In the formula, represents the weight value of the Gaussian cluster, represents the mean vector of the Gaussian cluster, = represents the covariance matrix of the Gaussian cluster, Indicates i number Base The probability of a Gaussian cluster;

[0144] S3.6: Calculate the sample energy of each input vector:

[0145]

[0146] In the formula, Indicates high The inverse of the cluster covariance matrix.

[0147] S3.7: Update network parameters based on the objective function; the specific implementation process is as follows:

[0148] S3.7.1: Calculate the objective function of DAGMM:

[0149]

[0150] In the formula, E(z i )+ represents the sample energy of the input vector, represents the regularization term related to the covariance matrix, λ1 and λ2 represent the weight factors of the second sample energy and the third regularization term respectively;

[0151] S3.7.2: Update the neural network parameters θ separately by stochastic gradient descent (SGD) e ,θ d With θ m ;

[0152] S3.7.3: Return to step S3.2 for the next round of parameter update until the training is completed.

[0153] S3.8: Calculate node reliability based on the trained DAGMM network; the specific implementation process is as follows:

[0154] S3.8.1: Input vector samples x∈V to the trained DAGMM network and obtain the energy E of all vector samples;

[0155] S3.8.2: Select the energy value that accounts for 70% of the energy E as the abnormality judgment threshold;

[0156] S3.8.3: When the energy of the vector sample constructed in step S2 is greater than the abnormality judgment threshold, that is, E i >threshold,stx i ∈V, the sample is judged as abnormal, and the reliability of the corresponding node to which the sample belongs is updated to: p g ←p g +0.3,g∈[1,G]; otherwise, it is judged as a normal update sample, and the reliability of the corresponding node to which the update belongs is: p g ←p g +1,g∈[1,G];

[0157] S3.8.4: The final calculated reliability of each node is: In the formula, k g Indicates the total number of vector samples belonging to the node.

[0158] Step S4: Perform deep reinforcement learning neural network training to obtain an expert selection strategy;

[0159] Synchronous with DAGMM training, according to the DDQN training method, the reasoning task state set S and the corresponding action set A are used as training data sets to perform deep reinforcement learning neural network training to obtain the expert selection strategy under the current task round; determine whether the training is completed, if not, return to step S2 for the next round of training, if yes, execute step S5;

[0160] See also Figure 5 To illustrate, the specific implementation process of DDQN's training method is as follows:

[0161] S4.1: Initialize sample pool D2; Initialize Q neural network Q(θ) of action-value function; Initialize target action-value function Neural Networks The initial weight parameters of the two types of neural networks are equal, that is,

[0162] S4.2: Store the inference task state set S constructed in step S2 into the sample pool D2; randomly take J samples (s j ,a j ,r j ,s j+1 ), where s j With s j+1 Respectively represent the current state and next state of the sample, a j Indicates action, r j Indicates reward;

[0163] S4.3: Calculate the target value of each current state in the sample, specifically: by executing action a j The reward after j To update the Q value as the target value y j : If the next state is an absorbing state, then y j =r j ,otherwise:

[0164]

[0165] In the formula, a′ represents the action to be performed in the next state.

[0166] S4.4: Calculate the objective function based on the target value and then update the parameter θ through SGD;

[0167] The specific implementation process of step S4.4 is as follows:

[0168] S4.4.1: Based on target value y j Calculate the objective function L(θ)=E[(y j -Q(s j ,a j ;θ)) 2 ];

[0169] S4.4.2: Calculate the gradient of the parameter θ with respect to the objective function:

[0170]

[0171] S4.4.3: Update the parameters θ using SGD.

[0172] S4.5: Update neural network parameters after every C iterations make C is an integer greater than 1.

[0173] Among them, in a reliable reasoning scheduling method based on a large edge hybrid expert model of the present invention, the reasoning task starts from the existing state and continuously optimizes the expert selection strategy. In the entire system, there is a reasoning task state set S, an action set A, and an expert selection strategy π. The next moment behavior a=π(s) is selected according to the current state. For each state s in the state set, there is a corresponding reward value R(s) corresponding to it; for each next state in the state sequence, an attenuation coefficient γ is set, and for each expert selection strategy π, a corresponding weight function V is set. π (s0) = E[R(s0) + γR(s1) + γ 2 R(s2)+...|s0=S,π], this expression satisfies the Bellman equation and can be rewritten as V π (s0)=E[R(s0)+γV π (s1)], where s0 represents the initial state and s1 represents the next state of the initial state.

[0174] Step S5: Perform reliable inference scheduling of the edge MoE large model based on the trained deep reinforcement learning neural network;

[0175] Specifically, the same method as step S2 is used to perform vector space and action state modeling on the edge MoE large model to implement reliable reasoning scheduling, and generate the initial input vector v0 and the initial reasoning task state s0 of the deep autoencoder; after the neural network training of steps S3 and S4, a converged abnormal node detection and expert selection neural network model can be obtained; the initial input vector v0 and the initial reasoning task state s0 are respectively input into the expert selection neural network model to obtain the node reliability and the expert selection strategy for the current MoE layer reasoning task; then, according to the expert selection strategy, the task is routed to the corresponding expert network of the next layer for subsequent reasoning. This process is the reliable reasoning scheduling of the edge MoE large model.

[0176] In order to verify the effect of the reliable reasoning scheduling method based on edge hybrid expert large model of the present invention, an LLM containing 10 layers of MoE blocks is deployed on an edge distributed network containing 10 groups of device nodes. Each MoE layer contains 8 expert networks, and 2 experts are selected from the expert set to be activated each time to complete the decision process. Using the reasoning scheduling method of the present invention, the relationship between the number of DDQN training cycles and the training loss is as follows: Figure 6 shown. Figure 6 The horizontal axis in the upper middle figure represents the number of training cycles, and the vertical axis represents the training loss. It can be seen that as the number of training cycles increases, the training loss under different discount factor γ experimental settings decreases significantly and converges in the end. The larger the discount factor γ, the more training volatility is shown. In addition, Figure 6 The lower middle figure also shows the normalized average computation time and reliability of the task under different experimental settings of discount factors γ. It can be seen that for an LLM with a shallower depth structure like the one used in this experiment, a discount factor γ of 0.9 can achieve the best training effect.

[0177] The comparison between the present invention and the traditional Top-2 activation strategy and the greedy strategy is as follows: Figure 7 shown. Figure 7 The horizontal axis represents the average task reliability, and the vertical axis represents the task reliability distribution. Figure 7 The middle dotted line represents the normalized average task reasoning time. It can be seen that the present invention has made a good trade-off between reasoning reliability and efficiency, and is ahead of the traditional TOP-2 activation strategy in both indicators. At the same time, although the greedy strategy achieves better optimization results on the optimization targets of their respective concerns, it sacrifices another indicator. For example, the delay greedy strategy that aims to reduce the reasoning delay performs very poorly in reasoning reliability.

[0178] Under the guidance of the present invention, the average time reward and reliability reward of the reasoning task at each layer are statistically analyzed to obtain the reward comparison of the present invention at different layers. Figure 8As shown in the figure, it can be seen that in the early stage of reasoning, that is, the task is located in the shallow position of LLM, the average reliability reward value is large and the average time reward is small, which means that the expert selection strategy at this time is more focused on ensuring the reliability of the task; while in the late stage of reasoning (that is, the task is located in the deep position of LLM), the average reliability reward value is small and the average time reward is large, which means that the expert selection strategy at this time is more focused on improving the efficiency of task execution.

[0179] Under the guidance of this invention, the relationship between different malicious node ratios (MNR), malicious node participation in dishonest computing ratios (DCR) and average task reasoning time and reasoning result reliability in the deployment environment is shown as follows: Fig. 9 Obviously, with the increase of MNR, the average task reasoning latency will increase significantly and the reasoning reliability will decrease significantly; and when MNR is large, the increase of DCR will significantly reduce the average reliability of task reasoning.

[0180] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A reliable reasoning scheduling method based on edge hybrid expert large model, characterized in that: The following steps are involved: Step S1: Distributed edge deployment of the MoE large model to the edge network; Step S2: Model malicious node behavior, simulate distributed collaborative reasoning, and generate the autoencoder input vector space set V and the reasoning task state set S respectively; Step S3: Perform abnormal node detection neural network training to obtain the calculation reliability of each edge node; Step S4: Perform deep reinforcement learning neural network training to obtain an expert selection strategy; Step S5: Perform reliable inference scheduling of the edge MoE large model based on the trained deep reinforcement learning neural network.

2. According to claim 1, a reliable reasoning scheduling method based on edge hybrid expert large model is characterized in that: The specific implementation process of step S1 is as follows: S1.1: Divide the MoE large model into N layers according to the MoE blocks, and deploy each layer on a group of devices; The MoE block includes: a MoE layer, a residual connection layer, a normalization layer, a self-attention layer and a gating network; the MoE layer includes multiple expert networks, each MoE layer contains E experts, and E is an integer greater than 1; the residual connection layer, the normalization layer, and the self-attention layer constitute a self-attention mechanism, and the gating network constitutes a gating mechanism, which is used to realize expert routing selection; S1.2: The expert networks deployed on the edge nodes belonging to the same device group constitute a complete set of MoE layers, and the remaining layer structures are shared among the edge nodes in the same group.

3. According to claim 1, a reliable reasoning scheduling method based on edge hybrid expert large model is characterized in that: The specific implementation process of step S2 is as follows: S2.1: Upload batch inference tasks to the distributed edge deployed MoE large model; the inference task represents an end-to-end inference from the first layer MoE block to the Nth layer MoE block; S2.2: Perform a Top-2 expert routing for each inference task and route it to the expert network deployed on the first layer of devices; S2.3: The reasoning task is queued at the expert network of the corresponding device edge node, waiting for calculation with the expert network; S2.4: Based on the malicious node behavior modeling, each malicious node participating in the reasoning task at this layer performs malicious behavior; S2.5: Calculate the reasoning time of the reasoning task, merge the reasoning results of each routed expert at the edge node expert network with the slowest reasoning progress, and perform subsequent calculations; S2.6: Calculate the reasoning time of the task at this level, return to step S2.2, and perform reasoning at the next level until the reasoning task is completed.

4. The reliable reasoning scheduling method based on edge hybrid expert large model according to claim 3 is characterized in that: The specific implementation process of step S2.4 is as follows: S2.4.1: The node is offline, which means that the node has no calculation results and the reasoning task waits for a response indefinitely; S2.4.2: Energy-delay attack, manifested as a node's reasoning time being too long; S2.4.3: Dishonest calculation, which takes the form of reducing the credibility of the reasoning result C; S2.4.4: Each malicious node only participates in one type of malicious behavior: if the node is offline, it will not participate in subsequent reasoning; if the node participates in energy-delay attacks or dishonest calculations, it will perform malicious behavior according to probability in each round of reasoning.

5. The reliable reasoning scheduling method based on edge hybrid expert large model according to claim 3 is characterized in that: The specific implementation process of step S2.5 is as follows: S2.5.1: Calculate the floating point computation required for an expert network to perform a reasoning task: D comp =4m·m h +2m h ·m+η·m h +m h ; Where m represents the embedding dimension of the token, h represents the dimension of the hidden layer of the neural network, and η represents the FLOPS (floating point operations per second) required for the activation function calculation; S2.5.2: Calculate the amount of data D transmitted between nodes tran =ζ·m; where ζ represents the quantization accuracy of the neural network weights; S2.5.3: When no malicious behavior occurs during the reasoning process, the reasoning time between adjacent layers of a reasoning task is: In the formula, i and j represent the experts being routed, B i represents the communication bandwidth of the node device where expert i is located, F i represents the FLOPS of the node device where expert i is located, B j represents the communication bandwidth of the node device where expert j is located, F j represents the FLOPS of the node device where expert j is located, Q i represents the number of queued tasks of the GPU of the node device where expert i is located, Q j represents the queued task volume of the GPU of the node where expert j is located, T i represents the time required for expert i to complete the current reasoning task, T j represents the time required for expert j to complete the current reasoning task, represents the subsequent calculation time of each device except the expert network calculation. If T i Greater than or equal to T j ,but If T i Less than T j ,but represents the subsequent computing time of the node device where expert i is located, excluding the expert network computing. It represents the subsequent calculation time of the node device where expert j is located, except for the expert network calculation, which is expressed as 6. The reliable reasoning scheduling method based on edge hybrid expert large model according to claim 5 is characterized in that: When the reasoning task at this level involves malicious behavior, the node is offline, which is represented by T = +∞; For energy-delay attack, it manifests as: In the formula, q i represents the number of queued tasks in the expert network where expert i is located, q j represents the amount of queued tasks in the expert network where expert j is located; For dishonest calculations, there is no significant impact on the reasoning time, but the credibility of the reasoning results will be reduced from C=1 to C=0.

7. The reliable reasoning scheduling method based on edge hybrid expert large model according to claim 1 is characterized in that: The specific implementation process of step S3 is as follows: S3.1: Initialize sample pool D1; Initialize compression network W(θ e ,θ d ), θ e and θ d Both represent compressed networks; initialize the estimated network θ m Represents the estimated network parameters; the reliability of each node is initialized to P = [p1, p2, ..., p g ,...,p G ]=[1,1,…,1,…,1]; S3.2: Store the input vector space set V constructed in step S2 into the sample pool D1; randomly take I input vector samples x from the sample pool D1 i ∈V; S3.3: Batch input of randomly selected I input vector samples into the compression network to obtain the low-dimensional spatial feature information z of the i-th input vector after compression ci ∈Z c and the compressed and reconstructed high-dimensional vector x i ′, and then the reconstruction error is obtained S3.4: Input I length-2 constructed vectors z = [z1, z2, ..., z i ,...,z I ]stz i =(z ci ,z ri ), i∈[1,I], the estimation network specifically adopts Gaussian mixture model GMM, the number of Gaussian clusters it contains is k=3, and the estimation network output through the softmax layer is obtained: Respectively represent the probability that a node belongs to each Gaussian cluster; S3.5: Estimate the weight values, mean vectors and covariance matrices of various Gaussian clusters of the Gaussian mixture model GMM based on the above probabilities: In the formula, represents the weight value of the Gaussian cluster, represents the mean vector of the Gaussian cluster, = represents the covariance matrix of the Gaussian cluster, Indicates i number Base The probability of a Gaussian cluster; S3.6: Calculate the sample energy of each input vector: Mode middle, Indicates high The inverse matrix of the cluster covariance matrix; S3.7: Update network parameters based on the objective function; S3.8: Compute node reliability based on the trained DAGMM network.

8. The reliable reasoning scheduling method based on edge hybrid expert large model according to claim 7 is characterized in that: The specific implementation process of step S3.7 is as follows: S3.7.1: Calculate the objective function of DAGMM: In the formula, E(z i )+ represents the sample energy of the input vector, represents the regularization term related to the covariance matrix, λ1 and λ2 represent the weight factors of the second sample energy and the third regularization term respectively; S3.7.2: Update the neural network parameters θ separately by stochastic gradient descent e ,θ d With θ m ; S3.7.3: Return to step S3.2 for the next round of parameter update until the training is completed.

9. The reliable reasoning scheduling method based on edge hybrid expert large model according to claim 7 is characterized in that: The specific implementation process of step S3.8 is as follows: S3.8.1: Input vector samples x∈V to the trained DAGMM network and obtain the energy E of all vector samples; S3.8.2: Select the energy value that accounts for 70% of the energy E as the abnormality judgment threshold; S3.8.3: When the energy of the vector sample constructed in step S2 is greater than the abnormality judgment threshold, that is, E i >threshold,stx i ∈V, the sample is judged as abnormal, and the reliability of the corresponding node to which the sample belongs is updated to: p g ←p g +0.3,g∈[1,G]; otherwise, it is judged as a normal update sample, and the reliability of the corresponding node to which the update belongs is: p g ←p g +1,g∈[1,G]; S3.8.4: The final calculated reliability of each node is: g∈[1,G]; where k g Indicates the total number of vector samples belonging to the node.

10. The reliable reasoning scheduling method based on edge hybrid expert large model according to claim 1 is characterized in that: The specific implementation process of step S4 is as follows: S4.1: Initialize sample pool D2; Initialize Q neural network Q(θ) of action-value function; Initialize target action-value function Neural Networks The initial weight parameters of the two types of neural networks are equal, that is, S4.2: Store the inference task state set S constructed in step S2 into the sample pool D2; randomly take J samples (s j ,a j ,r j ,s j+1 ), s j With s j+1 Respectively represent the current state and next state of the sample, a j Indicates action, r j Indicates reward; S4.3: Calculate the target value of each current state in the sample: by performing action a j The reward after j To update the Q value as the target value y j : If the next state is an absorbing state, then y j =r j ,otherwise: In the formula, a′ represents the action to be performed in the next state; S4.4: Calculate the objective function based on the target value and update the parameter θ through SGD; S4.4.1: Based on target value y j Calculate the objective function L(θ)=E[(y j -Q(s j ,a j ;θ)) 2 ]; S4.4.2: Calculate the gradient of the parameter θ with respect to the objective function: S4.4.3: Update the parameters θ using SGD. S4.5: Update neural network parameters after every C iterations make C is an integer greater than 1.

Images