Dynamic load balancing method and system of AI model intelligent machine

Abstract

The invention discloses a dynamic load balancing method and system for an AI model intelligent machine, and relates to the technical field of computer science, and the method comprises the steps: constructing an agent behavior strategy model through combining deep reinforcement learning and a multi-agent system MAS; constructing an intelligent scheduling model based on user request data and resource data, inputting the data into the model, performing resource demand prediction and task characteristic analysis, and outputting an optimal resource allocation scheme; constructing a dynamic load balancing function based on the optimal resource allocation scheme, and automatically adjusting the dynamic load balancing function when the network topology and the load mode change; collecting user feedback data, and performing deep analysis on the user feedback data to obtain a user satisfaction score and a service quality score as a basis for constructing a user feedback evaluation function; constructing a resource scheduling optimization model based on the intelligent scheduling model, the dynamic load balancing function and the user feedback evaluation function, and predicting a user experience strategy; and according to a prediction result, carrying out strategy adjustment and resource redistribution decision making.

Description

Technical Field

[0001] The present invention relates to the field of computer science and technology, and in particular to a dynamic load balancing method and system for an AI model intelligent machine. Background Art

[0002] With the rapid development of artificial intelligence technology, especially the increasing application of deep learning models, data centers and cloud computing platforms are facing tremendous computing load pressure. In order to improve the utilization of computing resources, reduce operating costs, and meet users' demand for service quality, dynamic load balancing technology has become a research focus. Traditional load balancing technology mainly relies on static configuration and simple algorithms to allocate computing tasks. This method often seems incapable when faced with complex and changeable workloads. In recent years, with the development of advanced technologies such as deep reinforcement learning DRL and multi-agent systems MAS, these technologies have been applied to the field of load balancing and have shown good potential.

[0003] However, the existing load balancing methods based on DRL and MAS still have some shortcomings. In most cases, these methods only consider single-dimensional optimization objectives, such as maximizing resource utilization and minimizing task response time, but fail to take into account multiple performance indicators at the same time, such as resource utilization, task response time, user satisfaction and system stability. Secondly, due to the complexity of DRL and MAS systems, they cannot respond quickly to rapidly changing network topologies and load patterns, resulting in delayed adjustment of load balancing strategies. In addition, existing methods lack an effective mechanism to integrate user feedback, making it difficult to accurately predict and improve user experience. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a dynamic load balancing method and system solution for an AI model intelligent machine. In most cases, these methods only consider single-dimensional optimization objectives, such as maximizing resource utilization and minimizing task response time, but fail to take into account multiple performance indicators at the same time, such as resource utilization, task response time, user satisfaction and system stability. Secondly, due to the complexity of DRL and MAS systems, they cannot respond quickly to rapidly changing network topologies and load patterns, resulting in delayed adjustment of load balancing strategies. In addition, existing methods lack an effective mechanism to integrate user feedback, making it difficult to accurately predict and improve user experience.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, an embodiment of the present invention provides a dynamic load balancing method for an AI model intelligent machine, which includes:

[0008] Combine deep reinforcement learning with multi-agent system MAS to build an agent behavior strategy model;

[0009] Build an intelligent scheduling model based on user request data and resource data, input the data into the model, perform resource demand forecasting and task characteristic analysis, and output the optimal resource allocation plan;

[0010] Build a dynamic load balancing function based on the optimal resource allocation solution, and automatically adjust the dynamic load balancing function when the network topology and load pattern change;

[0011] Collect user feedback data, conduct in-depth analysis on it, and obtain user satisfaction scores and service quality scores as the basis for building user feedback evaluation functions;

[0012] Build a resource scheduling optimization model based on intelligent scheduling model, dynamic load balancing function and user feedback evaluation function to identify resource bottlenecks and potential optimization points and predict user experience strategies;

[0013] Make policy adjustments and resource reallocation decisions based on the predicted user experience and system performance status.

[0014] As a preferred solution of the dynamic load balancing method of the AI ​​model intelligent machine of the present invention, wherein: a state space S is defined based on a multi-agent system MAS, representing a set of all system states s;

[0015] Based on the multi-agent system MAS, the action space A is defined, which represents the set of all actions a;

[0016] Based on the deep reinforcement learning model, the network parameters are set to θ, the experience replay buffer is B, and the four-tuple used for storage is (s, a, r, s'), where s represents the current state, a represents the action performed, r represents the reward, and s' represents the next state;

[0017] Combining the deep reinforcement learning model with the multi-agent system MAS, the update rules of the agent behavior strategy model are set, and the expression is:

[0018]

[0019] Among them, △θ i is the parameter update amount of agent i, α is the learning rate, γ is the discount factor, r is the immediate reward, Q is the current Q value, which represents the expected cumulative reward for executing action a in the current state s calculated using the current policy network parameters θi of agent i, and Q' is the target Q value, using the target network parameters θ i -The expected cumulative reward of performing action a' in the next state s' is calculated, and max represents the maximum value. Indicates the sensitivity of Q value to network parameters;

[0020] The resource utilization, task response time, user satisfaction and system stability of the agent behavior strategy model are tested in a simulation environment. Thresholds are set according to the four performance indicators. When the four performance indicators are greater than or equal to the preset thresholds, it means that the performance of the agent behavior strategy model is good.

[0021] When the four performance indicators are less than the preset thresholds, it indicates that the performance of the agent behavior strategy model is poor, and the system parameters are input into the update rule expression for parameter optimization;

[0022] By implementing iterative strategies through multiple training cycles, the agent continuously updates its behavior strategy until the maximum number of iterations is reached and the optimal strategy is obtained;

[0023] Integrate the optimized agent behavior strategy model into the dynamic load balancing system of the AI ​​model intelligent machine.

[0024] As a preferred solution of the dynamic load balancing method of the AI ​​model intelligent machine of the present invention, wherein: the intelligent scheduling model is constructed based on user request data and resource data, the data is input into the model, resource demand prediction and task characteristic analysis are performed, and the optimal resource allocation plan is output. The specific steps are:

[0025] Collect request type, request size, priority, and estimated execution time to form user request data;

[0026] Collect CPU usage, memory usage, GPU occupancy and network bandwidth to form resource data;

[0027] Clean and standardize all collected data;

[0028] The recursive neural network RNN ​​is used to extract features from user request data to obtain the request feature vector set X req ;

[0029] The long short-term memory network LSTM is used to extract the resource data to obtain the resource feature vector set X res ;

[0030] Build an intelligent scheduling model based on recurrent neural network RNN ​​and long short-term memory network LSTM;

[0031] The feature vector set X req Input intelligent scheduling model to capture the time series dependency in user request data. The output of RNN is H req ;

[0032] The resource feature vector set X res Input intelligent scheduling model to capture the long-term dependency of resource data usage. The output of LSTM is H res ;

[0033] Add a fusion layer inside the intelligent scheduling model to transform H req and H res Fusion is performed to form a prediction vector, which is expressed as:

[0034]

[0035] Among them, Y* is the output vector of the fusion layer, which represents the predicted resource demand, σ represents the activation function ReLU, and W f represents the weight matrix of the fusion layer, represents vector concatenation, b f Represents the bias vector of the fusion layer;

[0036] Set the loss function L and calculate the mean square error (MSE) between the predicted resource demand Y* and the actual resource demand Y;

[0037] Set the threshold MT. When the difference mean square error value is greater than or equal to MT, the hybrid neural network model parameters need to be updated.

[0038] Update the model parameters W using the gradient descent algorithm f and b f , minimize the loss function L, and the update rule expression is:

[0039]

[0040] Among them, △W f and △b f represents the update amount of the weight matrix and bias vector, η represents the learning rate, and They are the partial derivatives of the loss function with respect to the weight matrix and the bias vector, respectively, which are used to calculate the gradient;

[0041] The model parameters after the updated rules are input into the intelligent scheduling model again to obtain the optimal resource allocation solution Y*.

[0042] As a preferred solution of the dynamic load balancing method of the AI ​​model intelligent machine of the present invention, wherein: the dynamic load balancing function is constructed based on the optimal resource allocation solution, and the dynamic load balancing function is automatically adjusted when the network topology and load mode change. The specific steps are:

[0043] Use online monitoring algorithms to continuously detect changes in network topology and changes in load patterns;

[0044] Based on the changes in network topology and load pattern, a load change evaluation function is constructed to evaluate the impact of changes in network topology and load pattern on system performance. The expression is:

[0045]

[0046] Among them, G△ represents the load change evaluation function, T represents the current network topology vector, L represents the current load mode vector, N represents the number of nodes in the network, and l i represents the current load of node i, l i old represents the load before node i, l i max represents the maximum load of node i, t i represents the number of other nodes connected to node i, t i old represents the number of topological connections before node i, t i max Indicates the maximum number of nodes that node i can establish connections with;

[0047] A dynamic load balancing function is constructed based on the load change evaluation function G△, and the expression is:

[0048]

[0049] Among them, F LB represents the dynamic load balancing function, Y* represents the optimal resource allocation scheme, and α represents the adjustment parameter;

[0050] When nodes join, leave, load increases, and load decreases, record the current state T and L, and calculate the current state T and L based on the previous state T old and L old , input the data into the load change evaluation function formula to calculate G△(T, L);

[0051] According to the output of G△, use the dynamic load balancing function F LB Adjust the optimal resource allocation plan Y* to obtain a new resource allocation plan Y new .

[0052] As a preferred solution of the dynamic load balancing method of the AI ​​model intelligent machine of the present invention, wherein: the user feedback data is collected and deeply analyzed to obtain the user satisfaction score and service quality score as the basis for constructing the user feedback evaluation function, and the specific steps are as follows:

[0053] Use market research methods to collect user satisfaction scores and service quality scores;

[0054] Clean the collected user feedback data;

[0055] Standardize the cleaned data;

[0056] Based on the satisfaction score and service quality score after processing, a user feedback evaluation function is constructed to quantify the impact of user feedback on resource allocation. The expression is:

[0057]

[0058] Among them, G U represents the user feedback evaluation function, U represents the user feedback data, N U Indicates the number of feedback items in the user feedback data, u i represents the current satisfaction score of user i, u i old represents the previous satisfaction score of user i, u i max represents the maximum value of user i’s satisfaction score, s i represents the current service quality score of user i, s i old represents the previous service quality score of user i, s i max represents the maximum value of the service quality score of user i, w u and w s Represents the weight coefficient.

[0059] As a preferred solution of the dynamic load balancing method of the AI ​​model intelligent machine described in the present invention, wherein: the resource scheduling optimization model is constructed based on the intelligent scheduling model, the dynamic load balancing function and the user feedback evaluation function, resource bottlenecks and potential optimization points are identified, and user experience strategies are predicted. The specific steps are:

[0060] Combine intelligent scheduling model, dynamic load balancing function and user feedback evaluation function;

[0061] A deep reinforcement learning algorithm is used to build a resource scheduling optimization model. The combined results and user feedback evaluation function are input into the resource scheduling optimization model. The expression is:

[0062]

[0063] Among them, M o It represents the resource scheduling optimization model, α and β are two adjustment parameters;

[0064] The response time, service availability and system stability in the user feedback data are used as the experience index e, and a prediction user experience function is constructed based on the experience index. The expression is:

[0065]

[0066] Among them, P E represents the predicted user experience function, N E Indicates the number of experience indicators, e i represents the current value of the i-th experience indicator, e i old represents the old value of the i-th experience indicator, e i max represents the maximum value of the i-th experience index, w pe is the weight coefficient.

[0067] As a preferred solution of the dynamic load balancing method of the AI ​​model intelligent machine of the present invention, wherein: the strategy adjustment and resource reallocation decision are made according to the predicted user experience and system performance status, and the specific steps are:

[0068] According to the resource scheduling optimization model M o Output to obtain the current resource allocation plan and system status;

[0069] Use the load change evaluation function G△ to calculate the impact of changes in the current network topology and load pattern on system performance;

[0070] Use user feedback to evaluate function G U Quantify the impact of user feedback on resource allocation;

[0071] Use the predicted user experience function P E Predict changes in user experience after resource allocation strategy adjustments;

[0072] Based on the optimal resource allocation solution Y*, predict the user experience function value P E (M o ), user feedback evaluation function value G U (U) and the load change evaluation function value G△(T, L) construct the strategy adjustment function, which is expressed as:

[0073]

[0074] Among them, D A represents the policy adjustment function, α, β, and γ are adjustment parameters, which are used to control the influence of G△ on resource allocation adjustment and control G U The degree of influence and control on resource allocation adjustment E the extent of the impact on resource allocation adjustments;

[0075] Adjust the function D according to the strategy A output, adjust resource allocation strategy, and implement new resource allocation plan;

[0076] Observe changes in user experience based on the adjusted resource allocation plan, collect new user feedback data, and continuously optimize the user experience.

[0077] In a second aspect, the present invention provides a dynamic load balancing system for an AI model intelligent machine, comprising:

[0078] Behavior strategy model, responsible for combining deep reinforcement learning with multi-agent system MAS to build the agent's behavior strategy model;

[0079] Intelligent scheduling model, which is responsible for extracting features based on user request data and resource data using recurrent neural network (RNN) and long short-term memory (LSTM) networks, building an intelligent scheduling model, and outputting the optimal resource allocation plan;

[0080] Dynamic load balancing module, which is responsible for automatically adjusting the dynamic load balancing function when the network topology and load pattern change;

[0081] User feedback evaluation model, responsible for collecting and analyzing user feedback data, building a user feedback evaluation function, and quantifying the impact of user feedback on resource allocation;

[0082] The resource scheduling model is responsible for combining the intelligent scheduling model, dynamic load balancing function and user feedback evaluation function to build a resource scheduling optimization model, identify resource bottlenecks and potential optimization points, and predict user experience strategies.

[0083] In a third aspect, an embodiment of the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the dynamic load balancing method of the AI ​​model intelligent machine as described in the first aspect of the present invention is implemented.

[0084] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the dynamic load balancing method of the AI ​​model intelligent machine as described in the first aspect of the present invention.

[0085] The beneficial effects of the present invention are as follows: the present invention constructs an agent behavior strategy model by combining deep reinforcement learning with a multi-agent system, which can not only effectively predict resource requirements and task characteristics, but also automatically adjust load balancing strategies according to changes in network topology and load patterns, and further optimize resource scheduling through user feedback. The present invention belongs to the field of computer science, and in particular, relates to resource management technologies for cloud computing platforms and data centers. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0087] Figure 1 This is a flow chart of the dynamic load balancing method of the AI ​​model intelligent machine in Example 1.

[0088] Figure 2 This is a diagram of the structure of the resource scheduling optimization model in Example 1. DETAILED DESCRIPTION

[0089] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0090] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0091] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0092] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, and provides a dynamic load balancing method for an AI model intelligent machine, comprising the following steps:

[0093] S1 combines deep reinforcement learning with multi-agent system MAS to build an agent behavior strategy model;

[0094] Based on the multi-agent system MAS, the state space S is defined, which represents the set of all system states s, including but not limited to the current network topology, resource usage and task queue status, and the expression is:

[0095]

[0096] Among them, S represents the set of all system states s, and n represents the number of actions;

[0097] Based on the multi-agent system MAS, the action space A is defined, which represents the set of all actions a, including but not limited to task allocation and resource adjustment. The expression is:

[0098]

[0099] Among them, A represents the set of all actions a, and n represents the number of actions;

[0100] Based on the deep reinforcement learning model, the network parameters are set to θ, the experience replay buffer is B, and the four-tuple used for storage is (s, a, r, s'), where s represents the current state, a represents the action performed, r represents the reward, and s' represents the next state;

[0101] Combining the deep reinforcement learning model with the multi-agent system MAS, the update rules of the agent behavior strategy model are set, and the expression is:

[0102]

[0103] Among them, △θ i is the parameter update of agent i, α is the learning rate, γ is the discount factor, which indicates the discount degree of future rewards relative to immediate rewards, r is the immediate reward, which is calculated based on the agent behavior and the environment state, Q is the current Q value, which indicates the expected cumulative reward for performing action a in the current state s calculated using the current policy network parameters θi of agent i, and Q' is the target Q value, which is calculated using the target network parameters θ i - The expected cumulative reward of performing action a' in the next state s' is calculated, and max represents the maximum value. Indicates the sensitivity of Q value to network parameters;

[0104] The resource utilization, task response time, user satisfaction and system stability of the agent behavior strategy model are tested in a simulation environment. Thresholds are set according to the four performance indicators. When the four performance indicators are greater than or equal to the preset thresholds, it means that the performance of the agent behavior strategy model is good.

[0105] When the four performance indicators are less than the preset thresholds, it indicates that the performance of the agent behavior strategy model is poor, and the system parameters are input into the update rule expression for parameter optimization;

[0106] By implementing iterative strategies through multiple training cycles, the agent continuously updates its behavior strategy until the maximum number of iterations is reached and the optimal strategy is obtained;

[0107] Integrate the optimized agent behavior strategy model into the dynamic load balancing system of the AI ​​model intelligent machine;

[0108] The learning rate α ranges from 0 to 1. A small value means that the learning process is more stable but converges slowly, while a large value means the opposite.

[0109] The discount factor γ is a constant between 0 and 1. When γ is close to 1, it means that the agent attaches more importance to potential future rewards; when γ is close to 0, it means that the agent only cares about immediate rewards;

[0110] The instant reward r includes setting the reward function to R. The instant reward calculation formula is:

[0111] r=w1·R ut (s,a,s')+w2·R re (s,a,s')+w3·R sa (s,a,s')+w4·R st (s,a,s');

[0112] Among them, w1, w2, w3, and w4 are weight coefficients used to adjust the relative importance of different reward functions in the total reward. ut represents the resource utilization reward function, high resource utilization is usually regarded as a positive outcome, R re Represents the response time reward function. Reducing the response time will get positive rewards. R sa represents the user satisfaction reward function. Improving user satisfaction can increase rewards. R st represents the system stability reward function. Keeping the system stable usually results in positive rewards.

[0113] S2 builds an intelligent scheduling model based on user request data and resource data, inputs the data into the model, performs resource demand forecasting and task characteristic analysis, and outputs the optimal resource allocation plan;

[0114] Collect request type, request size, priority, and estimated execution time to form user request data;

[0115] Collect CPU usage, memory usage, GPU occupancy and network bandwidth to form resource data;

[0116] Clean and standardize all collected data, including removing outliers and standardizing them;

[0117] The recursive neural network RNN ​​is used to extract features from user request data to obtain the request feature vector set X req ;

[0118] The long short-term memory network LSTM is used to extract the resource data to obtain the resource feature vector set X res ;

[0119] Build an intelligent scheduling model based on recurrent neural network RNN ​​and long short-term memory network LSTM;

[0120] The feature vector set X req Input intelligent scheduling model to capture the time series dependency in user request data. The output of RNN is H req ;

[0121] The resource feature vector set X res Input intelligent scheduling model to capture the long-term dependency of resource data usage. The output of LSTM is H res ;

[0122] Add a fusion layer inside the intelligent scheduling model to transform H req and H res Fusion is performed to form a prediction vector, which is expressed as:

[0123]

[0124] Among them, Y* is the output vector of the fusion layer, which represents the predicted resource demand, σ represents the activation function ReLU, and W f Represents the weight matrix of the fusion layer, which is used to adjust the relative contribution of the request feature vector and the resource feature vector in the fusion process. Represents vector concatenation, used to concatenate H req and H res merged into a larger vector, b f Represents the bias vector of the fusion layer;

[0125] Set the loss function L and calculate the mean square error (MSE) between the predicted resource demand Y* and the actual resource demand Y;

[0126] Set the threshold MT. When the difference mean square error value is greater than or equal to MT, the hybrid neural network model parameters need to be updated.

[0127] Update the model parameters W using the gradient descent algorithm f and b f , minimize the loss function L, and the update rule expression is:

[0128]

[0129] Among them, △W f and △b f represents the update amount of the weight matrix and bias vector, η represents the learning rate, which is used to control the speed of parameter update, and They are the partial derivatives of the loss function with respect to the weight matrix and the bias vector, respectively, which are used to calculate the gradient;

[0130] The model parameters after the updated rules are input into the intelligent scheduling model again to obtain the optimal resource allocation solution Y*.

[0131] S3 builds a dynamic load balancing function based on the optimal resource allocation solution, and automatically adjusts the dynamic load balancing function when the network topology and load pattern change;

[0132] Use online monitoring algorithms to continuously detect changes in network topology and changes in load patterns, including node joining, node leaving, load increase, and load decrease;

[0133] Based on the changes in network topology and load pattern, a load change evaluation function is constructed to evaluate the impact of changes in network topology and load pattern on system performance. The expression is:

[0134]

[0135] Among them, G△ represents the load change evaluation function, T represents the current network topology vector, which includes the connection relationship between each node in the network, L represents the current load mode vector, which includes the load status of each node in the network, N represents the number of nodes in the network, and l i represents the current load of node i, namely, CPU usage and memory usage, l i old represents the load before node i, l i max represents the maximum load of node i, t i represents the number of other nodes connected to node i, t i old represents the number of topological connections before node i, t i max Indicates the maximum number of nodes that node i can establish connections with;

[0136] A dynamic load balancing function is constructed based on the load change evaluation function G△, and the expression is:

[0137]

[0138] Among them, F LB represents the dynamic load balancing function, Y* represents the optimal resource allocation scheme, and α represents the adjustment parameter, which is used to control the influence of G△ on the resource allocation adjustment. A larger α value means a more conservative response to changes, and a smaller α value means a more sensitive response to changes.

[0139] When nodes join, leave, load increases, and load decreases, record the current state T and L, and calculate the current state T and L based on the previous state T old and L old, input the data into the load change evaluation function formula to calculate G△(T, L);

[0140] According to the output of G△, use the dynamic load balancing function F LB Adjust the optimal resource allocation plan Y* to obtain a new resource allocation plan Y new .

[0141] S4 collects user feedback data, conducts in-depth analysis on it, and obtains user satisfaction scores and service quality scores as the basis for constructing user feedback evaluation functions;

[0142] Use market research methods to collect user satisfaction ratings and service quality ratings, including but not limited to user ratings on service speed, availability, and stability;

[0143] Clean the collected user feedback data, including removing outliers and missing values;

[0144] Standardize the cleaned data to ensure data consistency and comparability;

[0145] Based on the satisfaction score and service quality score after processing, a user feedback evaluation function is constructed to quantify the impact of user feedback on resource allocation. The expression is:

[0146]

[0147] Among them, G U represents the user feedback evaluation function, U represents the user feedback data, N U Indicates the number of feedback items in the user feedback data, u i represents the current satisfaction score of user i, u i old represents the previous satisfaction score of user i, u i max represents the maximum value of user i’s satisfaction score, s i represents the current service quality score of user i, s i old represents the previous service quality score of user i, s i max represents the maximum value of the service quality score of user i, w u and w s represents the weight coefficient, which is used to adjust the relative importance of satisfaction score and service quality score in the overall evaluation;

[0148] G U The value range of G is 0 to 2, indicating that user feedback has not changed to the extent that user feedback has the greatest impact on resource allocation. UThe closer the value is to 2, the greater the impact of user feedback on resource allocation, and vice versa.

[0149] S5 builds a resource scheduling optimization model based on intelligent scheduling model, dynamic load balancing function and user feedback evaluation function to identify resource bottlenecks and potential optimization points and predict user experience strategies;

[0150] Combine intelligent scheduling model, dynamic load balancing function and user feedback evaluation function;

[0151] A deep reinforcement learning algorithm is used to build a resource scheduling optimization model. The combined results and user feedback evaluation function are input into the resource scheduling optimization model. The expression is:

[0152]

[0153] Among them, M o represents the resource scheduling optimization model, α and β are two adjustment parameters, which are used to control the influence of G△ on resource allocation adjustment and control G U the extent of the impact on resource allocation adjustments;

[0154] The value range of G△ is 0 to 2, which means that the system performance has not changed to the extent that the system performance is most affected. The closer the value of G△ is to 2, the greater the impact of changes in network topology and load patterns on system performance, and vice versa;

[0155] The response time, service availability and system stability in the user feedback data are used as the experience index e, and a prediction user experience function is constructed based on the experience index. The expression is:

[0156]

[0157] Among them, P E represents the predicted user experience function, N E It represents the number of experience indicators, with values ​​ranging from 1 to 3, corresponding to response time, service availability, and system stability, respectively. i represents the current value of the i-th experience indicator, e i old represents the old value of the i-th experience indicator, e i max represents the maximum value of the i-th experience index, w pe is the weight coefficient, which is used to adjust the relative importance of different experience indicators in the overall evaluation.

[0158] S6 makes policy adjustments and resource reallocation decisions based on the predicted user experience and system performance status;

[0159] According to the resource scheduling optimization model M oOutput to obtain the current resource allocation plan and system status;

[0160] Use the load change evaluation function G△ to calculate the impact of changes in the current network topology and load pattern on system performance;

[0161] Use user feedback to evaluate function G U Quantify the impact of user feedback on resource allocation;

[0162] Use the predicted user experience function P E Predict changes in user experience after resource allocation strategy adjustments;

[0163] Based on the optimal resource allocation solution Y*, predict the user experience function value P E (M o ), user feedback evaluation function value G U (U) and the load change evaluation function value G△(T, L) construct the strategy adjustment function, which is expressed as:

[0164]

[0165] Among them, D A represents the policy adjustment function, α, β, and γ are adjustment parameters, which are used to control the influence of G△ on resource allocation adjustment and control G U The degree of influence and control on resource allocation adjustment E the extent of the impact on resource allocation adjustments;

[0166] Adjust the function D according to the strategy A output, adjust resource allocation strategy, and implement new resource allocation plan;

[0167] Observe changes in user experience based on the adjusted resource allocation plan, collect new user feedback data, and continuously optimize the user experience.

[0168] This embodiment also provides a dynamic load balancing system for an AI model intelligent machine, including:

[0169] Behavior strategy model, responsible for combining deep reinforcement learning with multi-agent system MAS to build the agent's behavior strategy model;

[0170] Intelligent scheduling model, which is responsible for extracting features based on user request data and resource data using recurrent neural network (RNN) and long short-term memory (LSTM) networks, building an intelligent scheduling model, and outputting the optimal resource allocation plan;

[0171] Dynamic load balancing module, which is responsible for automatically adjusting the dynamic load balancing function when the network topology and load pattern change;

[0172] User feedback evaluation model, responsible for collecting and analyzing user feedback data, building a user feedback evaluation function, and quantifying the impact of user feedback on resource allocation;

[0173] The resource scheduling model is responsible for combining the intelligent scheduling model, dynamic load balancing function and user feedback evaluation function to build a resource scheduling optimization model, identify resource bottlenecks and potential optimization points, and predict user experience strategies.

[0174] This embodiment also provides a computer device, which is suitable for the dynamic load balancing method of an AI model intelligent machine, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the dynamic load balancing method of the AI ​​model intelligent machine proposed in the above embodiment.

[0175] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0176] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by the processor, the dynamic load balancing method for implementing the AI ​​model intelligent machine proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic storage, flash memory, disk or optical disk.

[0177] In summary, the present invention combines deep reinforcement learning with a multi-agent system to build an agent behavior strategy model, which can not only effectively predict resource requirements and task characteristics, but also automatically adjust the load balancing strategy according to changes in network topology and load patterns, and further optimize resource scheduling through user feedback. The present invention belongs to the field of computer science, and in particular, it relates to resource management technology for cloud computing platforms and data centers.

[0178] Example 2, referring to Table 1, is the second example of the present invention. To further verify the advancement of the present invention, experimental simulation data of the dynamic load balancing method of the AI ​​model intelligent machine is provided.

[0179] This embodiment aims to verify the effectiveness of a dynamic load balancing method for an AI model intelligent machine. This method combines deep reinforcement learning with a multi-agent system MAS to realize the construction of agent behavior strategies, and optimizes resource allocation strategies through intelligent scheduling models, dynamic load balancing functions, and user feedback evaluation functions.

[0180] The experiment preparation phase includes setting up the experimental environment, defining the state and action space, setting up the agent behavior strategy model, building the intelligent scheduling model, dynamic load balancing function and user feedback evaluation function. The specific implementation process is as follows:

[0181] A virtual data center environment is created, including 20 server nodes. Each server is configured with a 4-core CPU, 8GB RAM, and 1 GPU to simulate the real network topology and load changes.

[0182] The state space S is defined to include network topology, resource usage, and task queue status; the action space A includes task allocation and resource adjustment.

[0183] Set the deep reinforcement learning model parameters θ and define the update rules of the experience replay buffer B agent behavior strategy model as:

[0184]

[0185] Among them, α is the learning rate and its value is 0.01, and γ is the discount factor and its value is 0.99;

[0186] Collect user request data and resource data, including request type, request size, priority, estimated execution time, CPU usage, memory usage, GPU occupancy, and network bandwidth, and use recursive neural network RNN ​​and long short-term memory network LSTM for feature extraction. The extracted features are fused through the fusion layer to form a prediction vector Y*.

[0187] Use online monitoring algorithms to continuously detect changes in network topology and load patterns, and build load change evaluation functions to assess the impact of these changes on system performance. Based on this, build a dynamic load balancing function.

[0188] Collect user feedback data through market research, including satisfaction scores and service quality scores, and build a user feedback evaluation function.

[0189] The intelligent scheduling model, dynamic load balancing function and user feedback evaluation function are combined to build a resource scheduling optimization model, and the user experience strategy is predicted based on it.

[0190] Based on the predicted user experience and system performance status, the policy adjustment function is used to make policy adjustments and resource reallocation decisions.

[0191] The details are shown in Table 1:

[0192] Table 1 Experimental record table

[0193]

[0194]

[0195] By analyzing the data in the above table, we can see that the dynamic load balancing method of the AI ​​model smart machine in the embodiment has been significantly improved compared to the baseline.

[0196] The resource utilization rate of Examples 1-5 reached a maximum of 92%, which is nearly 42% higher than the baseline of 65%. This shows that the intelligent agent behavior strategy model can utilize computing resources more efficiently.

[0197] The task response time is reduced from 1500ms in the baseline to 800ms in Example 1-5, a reduction of about 46.7%, which means that the intelligent scheduling model can complete task allocation faster and improve user experience.

[0198] The user satisfaction score increased from 6.5 to 9.5, an increase of 46.2%. The introduction of the user feedback evaluation function made the resource allocation strategy more in line with user needs.

[0199] The system stability score also increased from 7.0 to 9.8, an increase of 40%, which shows that the intelligent agent behavior strategy model not only improves resource utilization but also enhances the overall stability of the system.

[0200] The performance score of the agent behavior policy model increased from 5.0 to 9.5, an increase of 90%, which shows that the combination of deep reinforcement learning and multi-agent systems is very effective in resource allocation decisions.

[0201] The performance score of the intelligent scheduling model increased from 5.0 to 9.0, an increase of 80%, indicating that the use of recurrent neural networks (RNN) and long short-term memory networks (LSTM) significantly improved the ability to predict resource requirements.

[0202] In summary, the dynamic load balancing method of the AI ​​model intelligent machine in Example 1 shows significant advantages in terms of resource utilization, task response time, user satisfaction, system stability, and the performance of the intelligent agent behavior strategy model and the intelligent scheduling model. This fully proves the innovation and practicality of the invention.

[0203] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A dynamic load balancing method for an AI model intelligent machine, characterized in that: include, Combine deep reinforcement learning with multi-agent system MAS to build an agent behavior strategy model; Build an intelligent scheduling model based on user request data and resource data, input the data into the model, perform resource demand forecasting and task characteristic analysis, and output the optimal resource allocation plan; Build a dynamic load balancing function based on the optimal resource allocation solution, and automatically adjust the dynamic load balancing function when the network topology and load pattern change; Collect user feedback data, conduct in-depth analysis on it, and obtain user satisfaction scores and service quality scores as the basis for building user feedback evaluation functions; Build a resource scheduling optimization model based on intelligent scheduling model, dynamic load balancing function and user feedback evaluation function to identify resource bottlenecks and potential optimization points and predict user experience strategies; Make policy adjustments and resource reallocation decisions based on the predicted user experience and system performance status.

2. The dynamic load balancing method of the AI ​​model intelligent machine according to claim 1, characterized in that: Combining the deep reinforcement learning model with the multi-agent system MAS to build an agent behavior strategy model, the specific steps are: Based on the multi-agent system MAS, the state space S is defined, which represents the set of all system states s; Based on the multi-agent system MAS, the action space A is defined, which represents the set of all actions a; Based on the deep reinforcement learning model, the network parameters are set to θ, the experience replay buffer is B, and the four-tuple used for storage is (s, a, r, s'), where s represents the current state, a represents the action performed, r represents the reward, and s' represents the next state; Combining the deep reinforcement learning model with the multi-agent system MAS, the update rules of the agent behavior strategy model are set, and the expression is: Among them, △θ i is the parameter update amount of agent i, α is the learning rate, γ is the discount factor, r is the immediate reward, Q is the current Q value, which represents the expected cumulative reward for executing action a in the current state s calculated using the current policy network parameters θi of agent i, and Q' is the target Q value, using the target network parameters θ i - The expected cumulative reward of executing action a' in the next state s' is calculated, max represents the maximum value, and ▽Q represents the sensitivity of Q value to network parameters; The resource utilization, task response time, user satisfaction and system stability of the agent behavior strategy model are tested in a simulation environment. Thresholds are set according to the four performance indicators. When the four performance indicators are greater than or equal to the preset thresholds, it means that the performance of the agent behavior strategy model is good. When the four performance indicators are less than the preset thresholds, it indicates that the performance of the agent behavior strategy model is poor, and the system parameters are input into the update rule expression for parameter optimization; By implementing iterative strategies through multiple training cycles, the agent continuously updates its behavior strategy until the maximum number of iterations is reached and the optimal strategy is obtained; Integrate the optimized agent behavior strategy model into the dynamic load balancing system of the AI ​​model intelligent machine.

3. The dynamic load balancing method of the AI ​​model intelligent machine according to claim 2, characterized in that: The intelligent scheduling model is constructed based on user request data and resource data, the data is input into the model, resource demand prediction and task characteristic analysis are performed, and the optimal resource allocation plan is output. The specific steps are as follows: Collect request type, request size, priority, and estimated execution time to form user request data; Collect CPU usage, memory usage, GPU occupancy and network bandwidth to form resource data; Clean and standardize all collected data; The recursive neural network RNN ​​is used to extract features from user request data to obtain the request feature vector set X req ; The long short-term memory network LSTM is used to extract the resource data to obtain the resource feature vector set X res ; Build an intelligent scheduling model based on recurrent neural network RNN ​​and long short-term memory network LSTM; The feature vector set X req Input intelligent scheduling model to capture the time series dependency in user request data. The output of RNN is H req ; The resource feature vector set X res Input intelligent scheduling model to capture the long-term dependency of resource data usage. The output of LSTM is H res ; Add a fusion layer inside the intelligent scheduling model to transform H req and H res Fusion is performed to form a prediction vector, which is expressed as: Among them, Y* is the output vector of the fusion layer, which represents the predicted resource demand, σ represents the activation function ReLU, and W f represents the weight matrix of the fusion layer, represents vector concatenation, b f Represents the bias vector of the fusion layer; Set the loss function L and calculate the mean square error (MSE) between the predicted resource demand Y* and the actual resource demand Y; Set the threshold MT. When the difference mean square error value is greater than or equal to MT, the hybrid neural network model parameters need to be updated. Update the model parameters W using the gradient descent algorithm f and b f , minimize the loss function L, and the update rule expression is: Among them, △W f and △b f represents the update amount of the weight matrix and bias vector, η represents the learning rate, and They are the partial derivatives of the loss function with respect to the weight matrix and the bias vector, respectively, which are used to calculate the gradient; The model parameters after the updated rules are input into the intelligent scheduling model again to obtain the optimal resource allocation solution Y*.

4. The dynamic load balancing method of the AI ​​model intelligent machine according to claim 3, characterized in that: The dynamic load balancing function is constructed based on the optimal resource allocation solution, and the dynamic load balancing function is automatically adjusted when the network topology and load mode change. The specific steps are as follows: Use online monitoring algorithms to continuously detect changes in network topology and changes in load patterns; Based on the changes in network topology and load pattern, a load change evaluation function is constructed to evaluate the impact of changes in network topology and load pattern on system performance. The expression is: Among them, G△ represents the load change evaluation function, T represents the current network topology vector, L represents the current load mode vector, N represents the number of nodes in the network, and l i represents the current load of node i, l i old represents the load before node i, l i max represents the maximum load of node i, t i represents the number of other nodes connected to node i, t i old represents the number of topological connections before node i, t i max Indicates the maximum number of nodes that node i can establish connections with; A dynamic load balancing function is constructed based on the load change evaluation function G△, and the expression is: Among them, F LB represents the dynamic load balancing function, Y* represents the optimal resource allocation scheme, and α represents the adjustment parameter; When nodes join, leave, load increases, and load decreases, record the T and L of the current state, and the T and L of the previous state. old and L old , input the data into the load change evaluation function formula to calculate G△(T, L); According to the output of G△, use the dynamic load balancing function F LB Adjust the optimal resource allocation plan Y* to obtain a new resource allocation plan Y new .

5. The dynamic load balancing method of the AI ​​model intelligent machine according to claim 4, characterized in that: The user feedback data is collected and analyzed in depth to obtain the user satisfaction score and service quality score as the basis for constructing the user feedback evaluation function. The specific steps are as follows: Use market research methods to collect user satisfaction scores and service quality scores; Clean the collected user feedback data; Standardize the cleaned data; Based on the satisfaction score and service quality score after processing, a user feedback evaluation function is constructed to quantify the impact of user feedback on resource allocation. The expression is: Among them, G U represents the user feedback evaluation function, U represents the user feedback data, N U Indicates the number of feedback items in the user feedback data, u i represents the current satisfaction score of user i, u i old represents the previous satisfaction score of user i, u i max represents the maximum value of user i’s satisfaction score, s i represents the current service quality score of user i, s i old represents the previous service quality score of user i, s i max represents the maximum value of the service quality score of user i, w u and w s Represents the weight coefficient.

6. The dynamic load balancing method of the AI ​​model intelligent machine according to claim 5, characterized in that: The resource scheduling optimization model is constructed based on the intelligent scheduling model, dynamic load balancing function and user feedback evaluation function to identify resource bottlenecks and potential optimization points and predict user experience strategies. The specific steps are as follows: Combine intelligent scheduling model, dynamic load balancing function and user feedback evaluation function; A deep reinforcement learning algorithm is used to build a resource scheduling optimization model. The combined results and user feedback evaluation function are input into the resource scheduling optimization model. The expression is: Among them, M o It represents the resource scheduling optimization model, α and β are two adjustment parameters; The response time, service availability and system stability in the user feedback data are used as the experience index e, and a prediction user experience function is constructed based on the experience index. The expression is: Among them, P E represents the predicted user experience function, N E Indicates the number of experience indicators, e i represents the current value of the i-th experience indicator, e i old represents the old value of the i-th experience indicator, e i max represents the maximum value of the i-th experience index, w pe is the weight coefficient.

7. The dynamic load balancing method of the AI ​​model intelligent machine according to claim 6, characterized in that: The specific steps of making policy adjustments and resource reallocation decisions based on the predicted user experience and system performance status are as follows: According to the resource scheduling optimization model M o Output to obtain the current resource allocation plan and system status; Use the load change evaluation function G△ to calculate the impact of changes in the current network topology and load pattern on system performance; Use user feedback to evaluate function G U Quantify the impact of user feedback on resource allocation; Use the predicted user experience function P E Predict changes in user experience after resource allocation strategy adjustments; Based on the optimal resource allocation solution Y*, predict the user experience function value P E (M o ), user feedback evaluation function value G U (U) and the load change evaluation function value G△(T, L) construct the strategy adjustment function, the expression is: Among them, D A represents the policy adjustment function, α, β, and γ are adjustment parameters, which are used to control the influence of G△ on resource allocation adjustment and control G U The degree of influence and control on resource allocation adjustment E the extent of the impact on resource allocation adjustments; Adjust the function D according to the strategy A output, adjust resource allocation strategy, and implement new resource allocation plan; Observe changes in user experience based on the adjusted resource allocation plan, collect new user feedback data, and continuously optimize the user experience.

8. A dynamic load balancing system for an AI model intelligent machine, based on the dynamic load balancing method for an AI model intelligent machine according to any one of claims 1 to 7, characterized in that: include: Behavior strategy model, responsible for combining deep reinforcement learning with multi-agent system MAS to build the agent's behavior strategy model; Intelligent scheduling model, which is responsible for extracting features based on user request data and resource data using recurrent neural network (RNN) and long short-term memory (LSTM) networks, building an intelligent scheduling model, and outputting the optimal resource allocation plan; Dynamic load balancing module, which is responsible for automatically adjusting the dynamic load balancing function when the network topology and load pattern change; User feedback evaluation model, responsible for collecting and analyzing user feedback data, building a user feedback evaluation function, and quantifying the impact of user feedback on resource allocation; The resource scheduling model is responsible for combining the intelligent scheduling model, dynamic load balancing function and user feedback evaluation function to build a resource scheduling optimization model, identify resource bottlenecks and potential optimization points, and predict user experience strategies.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the dynamic load balancing method of the AI ​​model intelligent machine described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the dynamic load balancing method of the AI ​​model intelligent machine described in any one of claims 1 to 7 are implemented.

Images