Large model cloud computing dynamic updating method based on elastic adaptive incremental learning

Abstract

The invention discloses a large model cloud computing dynamic updating method based on elastic adaptive incremental learning, which comprises the following steps: S1, in a cloud computing platform, acquiring a current version of a large model and a new data stream to be processed; s2, local parameter updating is conducted on the large model according to the new data, and parameters or model layers needing to be updated are dynamically selected according to the characteristics of the data in the updating process; s3, introducing an adaptive learning strategy, and monitoring the change condition of data distribution in real time; s4, in a large model updating process, splitting a large model updating task to a plurality of computing nodes for parallel execution through a multi-node collaborative scheduling mechanism of the cloud; s5, after the large model is updated, the system verifies the performance of the large model; and S6, in the process of updating and deploying the large model, the system dynamically recycles computing resources which are not used any more. According to the method, idle computing resources can be fully utilized in a large-scale cloud computing environment, so that computing tasks are distributed in a balanced manner, and dynamic adjustment of task loads is ensured.

Description

Technical Field

[0001] The present invention relates to the field of large model technology, and in particular to a large model cloud computing dynamic update method based on elastic adaptive incremental learning. Background Art

[0002] In the existing technology, with the widespread application of large models, the dynamic update and maintenance of large models has become an important issue in the cloud computing environment. Large models usually need to process massive amounts of data and improve performance through frequent training and updates. However, existing large model update methods often rely on retraining the entire large model, which brings serious resource waste problems in the cloud computing environment. First, retraining the entire large model means that a large amount of computing resources are required. When the amount of data continues to grow, the traditional global training method cannot effectively cope with this growth, resulting in a continuous increase in the occupancy rate and energy consumption of cloud computing resources. In addition, since the training of the entire large model takes a long time, the delay in the update process is also relatively obvious, which cannot meet certain application scenarios with high real-time requirements.

[0003] Although incremental learning technology for large models can alleviate the resource consumption problem of retraining to a certain extent, existing incremental learning methods are usually only effective for fixed data structures or specific scenarios. When the data distribution changes significantly, the update strategy of the large model is difficult to adjust adaptively, resulting in a significant reduction in the prediction accuracy and robustness of the large model. At the same time, cloud resource scheduling in existing technologies often relies on static resource allocation strategies and fails to make dynamic adjustments based on real-time task load and system resource usage, resulting in the inability to optimize computing resources. Some computing node resources are overloaded, while other node resources are idle, resulting in unbalanced resource allocation.

[0004] In addition, existing large model update methods lack flexibility and usually require keeping the computing nodes synchronized during the large model update process, which not only increases the complexity of the system but also reduces the update efficiency. Especially in large-scale distributed cloud computing environments, parallel scheduling and asynchronous execution of tasks are the key to improving system performance. However, the synchronization mechanism of existing technologies cannot fully utilize the collaborative computing advantages of multiple nodes in the cloud, which in turn affects the operating efficiency of the overall system.

[0005] In summary, the existing technology has the following problems in updating large models: first, the global retraining of large models causes resource waste and delay problems; second, the incremental learning mechanism lacks adaptability in dealing with dynamic data changes; third, the cloud computing resource scheduling strategy lacks real-time dynamic adjustment capabilities and cannot fully utilize cloud resources; fourth, the existing synchronous update method reduces the efficiency of multi-node parallel execution and limits the improvement of system performance. Summary of the invention

[0006] One purpose of the present invention is to propose a large-model cloud computing dynamic update method based on elastic adaptive incremental learning. The present invention can make full use of idle computing resources in a large-scale cloud computing environment, so that computing tasks are evenly distributed and dynamic adjustment of task loads is ensured.

[0007] A large model cloud computing dynamic update method based on elastic adaptive incremental learning according to an embodiment of the present invention includes the following steps:

[0008] S1. In the cloud computing platform, obtain the current version of the big model and the new data stream to be processed, and determine the elastic resource allocation strategy according to the current task load and system resource situation of the big model;

[0009] S2. Adopting the incremental learning mechanism, local parameters of the large model are updated according to the new data without retraining the entire large model. During the update process, the parameters or model layers that need to be updated are dynamically selected according to the characteristics of the data.

[0010] S3, introduce adaptive learning strategies to monitor changes in data distribution in real time, and automatically adjust the model's learning rate, parameter structure, and update frequency according to data changes;

[0011] S4. During the large model update process, the large model update task is split into multiple computing nodes for parallel execution through the multi-node collaborative scheduling mechanism in the cloud, and each node performs asynchronous updates according to its own task progress;

[0012] S5. After completing the large model update, the system verifies the performance of the large model. If the performance reaches the preset threshold, the updated large model version is saved and deployed. If the performance does not reach the threshold, the parameters are adjusted and the update iteration is performed again.

[0013] S6. During the large model update and deployment process, the system dynamically reclaims computing resources that are no longer in use.

[0014] Optionally, the S1 step includes:

[0015] S11. In the cloud computing platform, obtain the current version M of the large model t and the new data stream D to be processed new , the resource scheduling mechanism based on the cloud computing platform is used to dispatch new data streams D new Perform pre-processing;

[0016] S12, according to the current task L of the large model t and system available resources R t Determine the elastic resource allocation strategy, the current task load L of the large model t By processing the current large model v t 、Memory usage mt and the task queue length q t The parameters are comprehensively evaluated and the current task load of the large model is calculated as follows:

[0017]

[0018] Among them, v max 、m max ,q max They represent the maximum values ​​of system processing speed, memory usage and task queue length respectively, α 1 , α 2 , α 3 is the weight coefficient of the task load, corresponding to the weights of different resources;

[0019] Real-time monitoring of available resources in cloud computing platforms t , including the number of available CPU cores CPU t , GPU availability performance GPU t and available memory capacity RAM t :

[0020]

[0021] Among them, CPU t , GPU t and RAM t They are the current number of CPU cores, GPU performance and memory capacity of the system, CPU max , GPU max and RAM max is the maximum available amount of these resources, β 1 , β 2 , β 3 is the corresponding resource weight, reflecting the relative contribution of each resource in the system’s available resources;

[0022] S13, based on the current task load L of the large model t R t The relationship between the two is that the adaptive allocation strategy f(L t ,R t )Dynamically adjust the allocation ratio of resources:

[0023]

[0024] Among them, R alloc L is the computing resource allocated to the large model update process. thr represents the threshold of task load, β is the adaptive coefficient, which controls the sensitivity of resource allocation to task load changes, γ is the smoothing factor, and R t-1 is the amount of resources allocated in the previous time step, R maxis the maximum amount of system resources;

[0025] S14: allocate the allocated computing resources R according to the elastic resource allocation strategy. alloc Used for data preprocessing, parameter adjustment and large model structure optimization during large model updating process.

[0026] Optionally, the S2 step includes:

[0027] S21. Obtain new data stream D based on incremental learning mechanism new The eigenvector X new , the feature vector and the current parameter matrix W of the large model t Compared with the similarity function S(X new ,W t ) Calculate the similarity between the features and the large model parameters:

[0028]

[0029] Among them, X new is the feature vector of the new data, W t is the current parameter matrix of the large model, ∥·∥ represents the norm operation of the vector;

[0030] S22, determine the large model part parameters W that need to be updated according to the similarity calculation results update , when the similarity S(X new ,W t ) is less than the set threshold δ, the parameter layer or large model structure associated with the new data features is selected for local update;

[0031] S23, the selected large model parameter W update Perform incremental learning updates and use an adaptive learning rate η t Update the large model parameters by gradient descent:

[0032]

[0033] in, is the loss function of the current large model relative to the large model parameter W t The gradient of t Dynamically adjust according to data flow changes and large model convergence speed;

[0034] S24. After completing the incremental learning update, the system updates the updated large model parameters W t+1 Verification is performed so that the updated part can improve the performance of the large model without retraining the entire large model.

[0035] Optionally, the S3 step includes:

[0036] S31, real-time monitoring of new data stream D new The distribution of the new data is obtained by new and standard deviation σ new , and compare it with the mean μ of the current data distribution of the large model t and standard deviation σ t Make comparisons and monitor changes in data distribution;

[0037] S32, rate of change based on data distribution Δ μ and Δ σ Dynamically adjust the learning rate η of large models t and update frequency f t ;

[0038] S33, according to the data distribution change rate Δ μ and Δ σ , using adaptive learning strategy to adjust the learning rate η t , the learning rate adjustment rule is:

[0039]

[0040] Among them, α is the adaptive coefficient, which controls the sensitivity of the learning rate to changes in data distribution;

[0041] S34, adjust the update frequency f of the large model t , set the update cycle according to the amplitude of data change, if Δ μ and Δ σ If the preset threshold ∈ is exceeded, the update frequency is increased:

[0042]

[0043] Among them, β is the adaptive adjustment coefficient of the update frequency;

[0044] S35, based on the adjusted learning rate η t+1 and update frequency f t+1 , dynamically update the large model.

[0045] Optionally, the S4 step includes:

[0046] S41. During the large model update process, obtain the number of computing nodes N available in the cloud and their corresponding computing capacity C i , computing power C i The number of CPU cores of the node i ,GPU performanceGPU i and available memory RAM i Comprehensive calculation:

[0047]

[0048] Among them, α, β, and γ are the weight coefficients of CPU, GPU, and memory, respectively, reflecting the contribution of each resource to the computing power. The nonlinear contribution of memory resources to computation is expressed in square root form, where i represents the i-th computing node;

[0049] S42. Split the update task of the large model into T j subtasks, and the computational complexity of each subtask is L j By the dimension W of the parameter matrix j The node task allocation strategy is based on the node's computing power C i and task complexity L j , whose load balancing dynamically allocates tasks while considering time constraints and remaining resources:

[0050]

[0051] Among them, t i is the time currently consumed by the node, T max is the maximum processing time allowed, R i represents the current remaining resources of node i, R total is the total remaining resources of all nodes;

[0052] S43, each node is assigned a task T j and the large model parameter W j Perform asynchronous updates, with each node independently calculating the subtask gradient update:

[0053]

[0054] Among them, η j is the learning rate, For the corresponding gradient calculation, the update process is Controls the adaptive adjustment of learning rate;

[0055] S44, real-time monitoring of the task progress of each node P i , when a node completes its task, re-evaluate the computational complexity of the remaining tasks L remain and the remaining computing capacity of the node C remain ;

[0056] S45. After all subtasks are completed, the system updates the parameters W of each node. j Merge to generate the updated large model parameter matrix W t+1 :

[0057]

[0058] Optionally, the step S5 includes:

[0059] S51. After the large model is updated, the updated large model version M is obtained. t+1 , through the preset performance index P target Verify the performance of the large model. The performance indicators include the accuracy of the model A t+1 , calculation efficiency E t+1 and resource utilization rate R t+1 :

[0060]

[0061] Among them, α, β, and γ are the weights of different performance indicators, A t+1 is the accuracy of the model, E t+1 is the computational efficiency, R t+1 is the resource utilization rate, R max is the maximum allowed resource usage;

[0062] S52, the verification result P t+1 With the preset threshold P target For comparison, if P t+1 ≥P target , then save and deploy the updated large model version M t+1 , replacing it with the currently running large model version;

[0063] S53, if P t+1 <P target , then adjust the relevant parameters of the large model and reset the model's learning rate η, update frequency f and parameter matrix W;

[0064] S54, after the parameter adjustment is completed, the system re-executes the update process until the model performance index P t+1 Reach or exceed the preset threshold P target , and finally deploy the updated large model version.

[0065] Optionally, the step S6 includes:

[0066] S61. After the large model update deployment is completed, the system monitors the resource utilization U of each computing node on the cloud computing platform. i ;

[0067] S62: Perform resource recovery evaluation on all computing nodes. If the resource utilization rate of a node is U i Below the preset threshold U min , the system starts a dynamic resource recovery mechanism to recover underutilized computing resources Release it back to the cloud platform for use in other tasks:

[0068]

[0069] in, is the computing resource used by the i-th node during the update process, is the total computing resources of the node;

[0070] S63, dynamically adjust the resource allocation strategy of each computing node according to the recovered idle resources Allocate resources for other high-load nodes;

[0071] S64. During the resource recovery process, the system monitors the overall resource utilization rate of the cloud platform in real time. total , when U total Reach the preset high efficiency utilization threshold U opt When the resource recovery is stopped, the system re-evaluates the computing resource requirements of the large model based on the utilization of the resources after recovery, and adjusts the priority and scheduling strategy of resource allocation in the future model update process.

[0072] The beneficial effects of the present invention are:

[0073] (1) The present invention adopts an elastic adaptive incremental learning mechanism to locally update some parameters or layers of the large model according to new data without retraining the entire large model. Compared with the traditional global update method, the present invention monitors the changes in data distribution in real time and adaptively adjusts the learning rate and update frequency of the large model, so that the large model can respond quickly to data changes, thereby improving the adaptability and update efficiency of the large model. Through the incremental learning strategy, the system can selectively update the parameters of the large model, thereby greatly reducing the usage of computing resources. Experimental results show that compared with the traditional global update method, the resource usage rate in the cloud computing environment is reduced by 31.2%, and the large model training time is shortened by more than 50%.

[0074] (2) The present invention combines the multi-node collaborative scheduling mechanism of the cloud computing platform to split the large model update task into multiple subtasks, which are distributed to multiple computing nodes in parallel for execution, and each node asynchronously updates the large model parameters according to the task progress. Compared with the traditional synchronous update method, the present invention avoids the communication delay and synchronization bottleneck problems between nodes through the asynchronous scheduling mechanism, significantly improves the efficiency of parallel computing, and can make full use of idle computing resources in a large-scale cloud computing environment, so that computing tasks are evenly distributed and dynamic adjustment of task loads is ensured.

[0075] (3) The present invention monitors the utilization rate of cloud computing resources in real time by introducing a dynamic resource monitoring and recycling mechanism, and dynamically recycles computing resources that are no longer in use after the large model is updated, thereby ensuring the efficient operation of the cloud computing platform. Compared with the static resource allocation strategy in the prior art, the dynamic resource management mechanism of the present invention can flexibly adjust the allocation and recycling of resources according to the actual needs of the task, thereby preventing excessive occupation and waste of resources. Through this optimization strategy, the system improves resource utilization by 20% during the peak period of the task, and can quickly recycle idle resources after the task is completed. The overall resource utilization efficiency is improved by 15%, providing more reliable and efficient computing support for large-scale data processing and large model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0077] Figure 1 A flowchart of a large model cloud computing dynamic update method based on elastic adaptive incremental learning proposed by the present invention;

[0078] Figure 2 This is a schematic diagram of asynchronously updating a large model based on a cloud-based multi-node collaborative scheduling mechanism in a large model cloud computing dynamic update method based on elastic adaptive incremental learning proposed by the present invention. DETAILED DESCRIPTION

[0079] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0080] refer to Figure 1-2 , a large model cloud computing dynamic update method based on elastic adaptive incremental learning, comprising the following steps:

[0081] S1. In the cloud computing platform, obtain the current version of the big model and the new data stream to be processed, and determine the elastic resource allocation strategy according to the current task load and system resource situation of the big model;

[0082] S2. Adopting the incremental learning mechanism, local parameters of the large model are updated according to the new data without retraining the entire large model. During the update process, the parameters or model layers that need to be updated are dynamically selected according to the characteristics of the data.

[0083] S3, introduce adaptive learning strategies to monitor changes in data distribution in real time, and automatically adjust the model's learning rate, parameter structure, and update frequency according to data changes;

[0084] S4. During the large model update process, the large model update task is split into multiple computing nodes for parallel execution through the multi-node collaborative scheduling mechanism in the cloud, and each node performs asynchronous updates according to its own task progress;

[0085] S5. After completing the large model update, the system verifies the performance of the large model. If the performance reaches the preset threshold, the updated large model version is saved and deployed. If the performance does not reach the threshold, the parameters are adjusted and the update iteration is performed again.

[0086] S6. During the large model update and deployment process, the system dynamically reclaims computing resources that are no longer in use.

[0087] In this implementation, step S1 includes:

[0088] S11. In the cloud computing platform, obtain the current version M of the large model t and the new data stream D to be processed new , the resource scheduling mechanism based on the cloud computing platform is used to dispatch new data streams D new Perform pre-processing;

[0089] S12, according to the current task L of the large model t and system available resources R t Determine the elastic resource allocation strategy, the current task load L of the large model t By processing the current large model v t 、Memory usage m t and the task queue length q t The parameters are comprehensively evaluated and the current task load of the large model is calculated as follows:

[0090]

[0091] Among them, v max 、m max ,q max They represent the maximum values ​​of system processing speed, memory usage and task queue length respectively, α 1 , α 2 , α 3 is the weight coefficient of the task load, corresponding to the weights of different resources;

[0092] Real-time monitoring of available resources in cloud computing platforms t , including the number of available CPU cores CPU t , GPU availability performance GPU t and available memory capacity RAM t :

[0093]

[0094] Among them, CPUt , GPU t and RAM t They are the current number of CPU cores, GPU performance and memory capacity of the system, CPU max , GPU max and RAM max is the maximum available amount of these resources, β 1 , β 2 , β 3 is the corresponding resource weight, reflecting the relative contribution of each resource in the system’s available resources;

[0095] S13, based on the current task load L of the large model t R t The relationship between the two is that the adaptive allocation strategy f(L t ,R t )Dynamically adjust the allocation ratio of resources:

[0096]

[0097] Among them, R alloc L is the computing resource allocated to the large model update process. thr represents the threshold of task load, β is the adaptive coefficient, which controls the sensitivity of resource allocation to task load changes, γ is the smoothing factor, and R t-1 is the amount of resources allocated in the previous time step, R max is the maximum amount of system resources;

[0098] S14: allocate the allocated computing resources R according to the elastic resource allocation strategy. alloc Used for data preprocessing, parameter adjustment and large model structure optimization during large model updating process.

[0099] In this embodiment, step S2 includes:

[0100] S21. Obtain new data stream D based on incremental learning mechanism new The eigenvector X new , the feature vector and the current parameter matrix W of the large model t Compared with the similarity function S(X new ,W t ) Calculate the similarity between the features and the large model parameters:

[0101]

[0102] Among them, X new is the feature vector of the new data, W t is the current parameter matrix of the large model, ∥·∥ represents the norm operation of the vector;

[0103] S22, determine the large model part parameters W that need to be updated according to the similarity calculation results update , when the similarity S(X new ,W t ) is less than the set threshold δ, the parameter layer or large model structure associated with the new data features is selected for local update;

[0104] S23, the selected large model parameter W update Perform incremental learning updates and use an adaptive learning rate η t Update the large model parameters by gradient descent:

[0105]

[0106] in, is the loss function of the current large model relative to the large model parameter W t The gradient of t Dynamically adjust according to data flow changes and large model convergence speed;

[0107] S24. After completing the incremental learning update, the system updates the updated large model parameters W t+1 Verification is performed so that the updated part can improve the performance of the large model without retraining the entire large model.

[0108] In this implementation, step S3 includes:

[0109] S31, real-time monitoring of new data stream D new The distribution of the new data is obtained by new and standard deviation σ new , and compare it with the mean μ of the current data distribution of the large model t and standard deviation σ t Make comparisons and monitor changes in data distribution;

[0110] S32, rate of change based on data distribution Δ μ and Δ σ Dynamically adjust the learning rate η of large models t and update frequency f t ;

[0111] S33, according to the data distribution change rate Δ μ and Δ σ , using adaptive learning strategy to adjust the learning rate η t , the learning rate adjustment rule is:

[0112]

[0113] Among them, α is the adaptive coefficient, which controls the sensitivity of the learning rate to changes in data distribution;

[0114] S34, adjust the update frequency f of the large model t , set the update cycle according to the amplitude of data change, if Δ μ and Δ σ If the preset threshold ∈ is exceeded, the update frequency is increased:

[0115]

[0116] Among them, β is the adaptive adjustment coefficient of the update frequency;

[0117] S35, based on the adjusted learning rate η t+1 and update frequency f t+1 , dynamically update the large model.

[0118] In this embodiment, step S4 includes:

[0119] S41. During the large model update process, obtain the number of computing nodes N available in the cloud and their corresponding computing capacity C i , computing power C i The number of CPU cores of the node i ,GPU performanceGPU i and available memory RAM i Comprehensive calculation:

[0120]

[0121] Among them, α, β, and γ are the weight coefficients of CPU, GPU, and memory, respectively, reflecting the contribution of each resource to the computing power. The nonlinear contribution of memory resources to computation is expressed in square root form, where i represents the i-th computing node;

[0122] S42. Split the update task of the large model into T j subtasks, and the computational complexity of each subtask is L j By the dimension W of the parameter matrix j The node task allocation strategy is based on the node's computing power C i and task complexity L j , whose load balancing dynamically allocates tasks while considering time constraints and remaining resources:

[0123]

[0124] Among them, t i is the time currently consumed by the node, T max is the maximum processing time allowed, R i represents the current remaining resources of node i, R totalis the total remaining resources of all nodes;

[0125] S43, each node is assigned a task T j and the large model parameter W j Perform asynchronous updates, with each node independently calculating the subtask gradient update:

[0126]

[0127] Among them, η j is the learning rate, For the corresponding gradient calculation, the update process is Controls the adaptive adjustment of learning rate;

[0128] S44, real-time monitoring of the task progress of each node P i , when a node completes its task, re-evaluate the computational complexity of the remaining tasks L remain and the remaining computing capacity of the node C remain ;

[0129] S45. After all subtasks are completed, the system updates the parameters W of each node. j Merge to generate the updated large model parameter matrix W t+1 :

[0130]

[0131] In this implementation, step S5 includes:

[0132] S51. After the large model is updated, the updated large model version M is obtained. t+1 , through the preset performance index P target Verify the performance of the large model. The performance indicators include the accuracy of the model A t+1 , Computational efficiency E t+1 and resource utilization rate R t+1 :

[0133]

[0134] Among them, α, β, and γ are the weights of different performance indicators, A t+1 is the accuracy of the model, E t+1 is the computational efficiency, R t+1 is the resource utilization rate, R max is the maximum allowed resource usage;

[0135] S52, the verification result P t+1 With the preset threshold P target For comparison, if P t+1 ≥P target, then save and deploy the updated large model version M t+1 , replacing it with the currently running large model version;

[0136] S53, if P t+1 <P target , then adjust the relevant parameters of the large model and reset the model's learning rate η, update frequency f and parameter matrix W;

[0137] S54, after the parameter adjustment is completed, the system re-executes the update process until the model performance index P t+1 Reach or exceed the preset threshold P target , and finally deploy the updated large model version.

[0138] In this implementation, step S6 includes:

[0139] S61. After the large model update deployment is completed, the system monitors the resource utilization U of each computing node on the cloud computing platform. i ;

[0140] S62: Perform resource recovery evaluation on all computing nodes. If the resource utilization rate of a node is U i Below the preset threshold U min , the system starts a dynamic resource recovery mechanism to recover underutilized computing resources Release it back to the cloud platform for use in other tasks:

[0141]

[0142] in, is the computing resource used by the i-th node during the update process, is the total computing resources of the node;

[0143] S63, dynamically adjust the resource allocation strategy of each computing node according to the recovered idle resources Allocate resources for other high-load nodes;

[0144] S64. During the resource recovery process, the system monitors the overall resource utilization rate of the cloud platform in real time. total , when U total Reach the preset high efficiency utilization threshold U opt When the resource recovery is stopped, the system re-evaluates the computing resource requirements of the large model based on the utilization of the resources after recovery, and adjusts the priority and scheduling strategy of resource allocation in the future model update process.

[0145] Embodiment 1:

[0146] In order to verify the feasibility and practical effect of the present invention in the dynamic update of large-scale cloud computing models, the experimenters selected the publicly available data sets Criteo and Movielens for experimental verification. The Criteo data set contains the historical records of users clicking on advertisements, which is about 45 million data items. The Movielens data set contains the user rating records for movies, which is about 10 million data items. In order to test the performance of the present invention in different data scenarios, the experimenters divided the two data sets into training set and test set in a ratio of 8:2.

[0147] In the experiment, the Criteo dataset was used for the ad click prediction task, and the Movielens dataset was used for the movie recommendation task. Both tasks are based on large models for prediction, so the experimenter selected the BERT model as the benchmark large model. In order to verify the cloud computing dynamic update method based on elastic adaptive incremental learning of the present invention, the experimenter simulated the process of real-time updating of the model in a dynamic data stream scenario. In the traditional full model retraining method, after each new data is added, the system needs to re-import all new data and train the entire model. In the implementation of the present invention, the experimenter selectively updates only some parameters of the model through the mechanism of incremental learning, and introduces adaptive learning rate and update frequency adjustment strategies, so that the model can quickly adapt to changes in new data.

[0148] The experimenters first preprocessed the Criteo and Movielens datasets, dividing the training set and the test set in a ratio of 8:2 to ensure that the new data and historical data can be reasonably distributed. In the Criteo dataset, the features include user ID, ad ID, and click status, and the experimenters standardized these features. In the Movielens dataset, the features include user ID, movie ID, and ratings. The experimenters encoded the features. To adapt to the large model input, the data was processed into vector form. The embedding dimension of the Criteo dataset is 1024, and the embedding dimension of the Movielens dataset is 768.

[0149] The experimenters used the pre-trained BERT model as the benchmark large model and performed initial training on the Criteo and Movielens datasets respectively. In order to improve efficiency, the experimenters trimmed the number of layers and parameters of the model to make it suitable for ad click prediction and movie recommendation tasks. After the initial training was completed, the accuracy of the benchmark large model on the Criteo dataset was 84.5% and the accuracy on the Movielens dataset was 79.3%.

[0150] During the incremental learning process, the experimenter introduces a new data stream, and the system selectively updates some parameters in the model through similarity calculation. Specifically, for each batch of new data, the experimenter calculates the similarity between its feature vector and the current parameters of the model. If the similarity is lower than the set threshold, the system will update the corresponding parameters. The experimenter uses an adaptive learning rate (Adam optimizer) and an adaptive update frequency mechanism to dynamically adjust the learning rate and update frequency of the model during training to ensure that the model can converge quickly.

[0151] In order to verify the multi-node collaborative update mechanism of the present invention, the experimenters simulated a multi-node environment on a cloud computing platform, split the update task of the large model into multiple subtasks, assigned them to multiple computing nodes for parallel execution, and updated parameters between nodes through an asynchronous mechanism to ensure efficient use of computing resources.

[0152] In order to verify the effectiveness of the present invention, the experimenters compared the experimental results with the traditional full model retraining method. The traditional method requires re-importing all data and retraining the entire model after each new data is added, which is time-consuming and resource-intensive.

[0153] Comparing the update efficiency, on the Criteo dataset, the traditional full model retraining method takes about 6 hours for each update, while the method of the present invention only takes 2.5 hours through incremental learning and adaptive updating, and the update efficiency is improved by 58.3%. On the Movielens dataset, the traditional method takes about 4 hours to update, while the method of the present invention only takes 1.7 hours, and the update efficiency is improved by 57.5%.

[0154] Computing resource usage comparison: The experimenters compared the resource usage of the two methods in the cloud computing environment. On the Criteo dataset, the computing resource usage of the traditional method was as high as 90%, while that of the method of the present invention was only 65%. On the Movielens dataset, the traditional method occupied 85% of the computing resources, while that of the method of the present invention was reduced to 60%.

[0155] Comparison of model performance. In terms of model performance, the experimenters used accuracy and mean square error as the main evaluation indicators. Experimental data show that the method of the present invention not only has high updating efficiency, but also outperforms the traditional method in model performance. On the Criteo dataset, the accuracy of the traditional method is 84.5%, while the accuracy of the method of the present invention is improved to 86.3% after updating. On the Movielens dataset, the accuracy of the traditional method is 79.3%, while the accuracy of the method of the present invention is improved to 81.2%. In addition, the mean square error of the model is also reduced. On the Criteo dataset, the MSE is reduced from 0.154 to 0.138, and on the Movielens dataset, the MSE is reduced from 0.217 to 0.198.

[0156] Comparison of resource utilization and recycling. The present invention can effectively recycle unused computing resources after the model update is completed through dynamic resource monitoring and recycling mechanism. Experiments show that on the Criteo dataset, the resource utilization rate of the traditional method is 78%, while the resource utilization rate of the present invention reaches 93%. On the Movielens dataset, the resource utilization rate of the traditional method is 80%, while the resource utilization rate of the present invention is 90%. The computing resources after resource recycling can be quickly allocated to other tasks, greatly improving the overall efficiency of the cloud computing platform.

[0157] Table 1 Comparison of experimental results

[0158]

[0159] Through the above experiments and data comparisons, the large-model cloud computing dynamic update method based on elastic adaptive incremental learning of the present invention significantly improves the model update efficiency, resource utilization and prediction accuracy, and verifies the feasibility and effectiveness of the method in practical applications.

[0160] The present invention adopts an elastic adaptive incremental learning mechanism to locally update some parameters or layers of the large model according to new data without retraining the entire large model. Compared with the traditional global update method, the present invention monitors the changes in data distribution in real time and adaptively adjusts the learning rate and update frequency of the large model, so that the large model can respond quickly to data changes, thereby improving the adaptability and update efficiency of the large model. Through the incremental learning strategy, the system can selectively update the large model parameters, thereby greatly reducing the occupancy of computing resources. Experimental results show that compared with the traditional global update method, the resource occupancy rate in the cloud computing environment is reduced by 31.2%, and the large model training time is shortened by more than 50%.

[0161] The present invention combines the multi-node collaborative scheduling mechanism of the cloud computing platform to split the large model update task into multiple subtasks, which are distributed to multiple computing nodes in parallel for execution, and each node asynchronously updates the large model parameters according to the task progress. Compared with the traditional synchronous update method, the present invention avoids communication delays and synchronization bottlenecks between nodes through the asynchronous scheduling mechanism, significantly improves the efficiency of parallel computing, and can make full use of idle computing resources in a large-scale cloud computing environment, so that computing tasks are evenly distributed and dynamic adjustment of task loads is ensured.

[0162] The present invention introduces a dynamic resource monitoring and recovery mechanism to monitor the utilization rate of cloud computing resources in real time, and dynamically recovers computing resources that are no longer in use after the large model is updated, thereby ensuring the efficient operation of the cloud computing platform. Compared with the static resource allocation strategy in the prior art, the dynamic resource management mechanism of the present invention can flexibly adjust the allocation and recovery of resources according to the actual needs of the task, and prevent excessive occupation and waste of resources. Through this optimization strategy, the system improves resource utilization by 20% during task peak periods, and can quickly recover idle resources after the task is completed, and the overall resource utilization efficiency is improved by 15%, providing more reliable and efficient computing support for large-scale data processing and large model training.

[0163] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A large model cloud computing dynamic update method based on elastic adaptive incremental learning, characterized in that: The steps include: S1. In the cloud computing platform, obtain the current version of the big model and the new data stream to be processed, and determine the elastic resource allocation strategy according to the current task load and system resource situation of the big model; S2. Adopting the incremental learning mechanism, local parameters of the large model are updated according to the new data without retraining the entire large model. During the update process, the parameters or model layers that need to be updated are dynamically selected according to the characteristics of the data. S3, introduce adaptive learning strategies to monitor changes in data distribution in real time, and automatically adjust the model's learning rate, parameter structure, and update frequency according to data changes; S4. During the large model update process, the large model update task is split into multiple computing nodes for parallel execution through the multi-node collaborative scheduling mechanism in the cloud, and each node performs asynchronous updates according to its own task progress; S5. After completing the large model update, the system verifies the performance of the large model. If the performance reaches the preset threshold, the updated large model version is saved and deployed. If the performance does not reach the threshold, the parameters are adjusted and the update iteration is performed again. S6. During the large model update and deployment process, the system dynamically reclaims computing resources that are no longer in use.

2. According to claim 1, a large model cloud computing dynamic update method based on elastic adaptive incremental learning is characterized in that: The S1 step includes: S11. In the cloud computing platform, obtain the current version M of the large model t and the new data stream D to be processed new , the resource scheduling mechanism based on the cloud computing platform is used to dispatch new data streams D new Perform pre-processing; S12, according to the current task L of the large model t and system available resources R t Determine the elastic resource allocation strategy, the current task load L of the large model t By processing the current large model v t 、Memory usage m t and the task queue length q t The parameters are comprehensively evaluated and the current task load of the large model is calculated as follows: Among them, v max 、m max ,q max They represent the maximum values ​​of system processing speed, memory usage and task queue length respectively. α1, α2 and α3 are the weight coefficients of task load, corresponding to the weights of different resources respectively. Real-time monitoring of available resources in cloud computing platforms t , including the number of available CPU cores CPU t , GPU availability performance GPU t and available memory capacity RAM t : Among them, CPU t , GPU t and RAM t They are the current number of CPU cores, GPU performance and memory capacity of the system, CPU max , GPU max and RAM max is the maximum available amount of these resources, β1, β2, and β3 are the corresponding resource weights, reflecting the relative contribution of each resource in the available resources of the system; S13, based on the current task load L of the large model t R t The relationship between the two is that the adaptive allocation strategy f(L t , R t )Dynamically adjust the allocation ratio of resources: Among them, R alloc L is the computing resource allocated to the large model update process. thr represents the threshold of task load, β is the adaptive coefficient, which controls the sensitivity of resource allocation to task load changes, γ is the smoothing factor, and R t-1 is the amount of resources allocated in the previous time step, R max is the maximum amount of system resources; S14: allocate the allocated computing resources R according to the elastic resource allocation strategy. alloc Used for data preprocessing, parameter adjustment and large model structure optimization during large model updating process.

3. According to the large model cloud computing dynamic update method based on elastic adaptive incremental learning according to claim 1, it is characterized in that: The S2 step includes: S21. Obtain new data stream D based on incremental learning mechanism new The eigenvector X new , the feature vector and the current parameter matrix W of the large model t Compared with the similarity function S(X new , W t ) Calculate the similarity between the features and the large model parameters: Among them, X new is the feature vector of the new data, W t is the current parameter matrix of the large model, ‖·‖ represents the norm operation of the vector; S22, determine the large model part parameters W that need to be updated according to the similarity calculation results update , when the similarity S(X new , W t ) is less than the set threshold δ, the parameter layer or large model structure associated with the new data features is selected for local update; S23, the selected large model parameter W update Perform incremental learning updates and use an adaptive learning rate η t Update the large model parameters by gradient descent: in, is the loss function of the current large model relative to the large model parameter W t The gradient of t Dynamically adjust according to data flow changes and large model convergence speed; S24. After completing the incremental learning update, the system updates the updated large model parameters W t+1 Verification is performed so that the updated part can improve the performance of the large model without retraining the entire large model.

4. According to the large model cloud computing dynamic update method based on elastic adaptive incremental learning according to claim 1, it is characterized in that: The S3 step includes: S31, real-time monitoring of new data stream D new The distribution of new data is obtained by new and standard deviation σ new , and compare it with the mean μ of the current data distribution of the large model t and standard deviation σ t Make comparisons and monitor changes in data distribution; S32, rate of change based on data distribution Δ μ and Δ σ Dynamically adjust the learning rate η of large models t and update frequency f t ; S33, according to the data distribution change rate Δ μ and Δ σ , using adaptive learning strategy to adjust the learning rate η t , the learning rate adjustment rule is: Among them, α is the adaptive coefficient, which controls the sensitivity of the learning rate to changes in data distribution; S34, adjust the update frequency f of the large model t , set the update cycle according to the amplitude of data change, if Δ μ and Δ σ If the preset threshold ∈ is exceeded, the update frequency is increased: Among them, β is the adaptive adjustment coefficient of the update frequency; S35, based on the adjusted learning rate η t+1 and update frequency f t+1 , dynamically update the large model.

5. According to the large model cloud computing dynamic update method based on elastic adaptive incremental learning according to claim 1, it is characterized in that: The S4 step includes: S41. During the large model update process, obtain the number of computing nodes N available in the cloud and their corresponding computing capacity C i , computing power C i The number of CPU cores of the node i 、GPU performance GPUJ i and available memory RAM i Comprehensive calculation: Among them, α, β, and γ are the weight coefficients of CPU, GPU, and memory, respectively, reflecting the contribution of each resource to the computing power. The nonlinear contribution of memory resources to computation is expressed in square root form, where i represents the i-th computing node; S42. Split the update task of the large model into T i subtasks, and the computational complexity of each subtask is L j The node task allocation strategy is based on the node's computing power C, which is calculated by the dimension Wj of the parameter matrix and the gradient operation to be performed. i and task complexity L j , whose load balancing dynamically allocates tasks while considering time constraints and remaining resources: Among them, t i is the time currently consumed by the node, T max is the maximum processing time allowed, R i represents the current remaining resources of node i, R total is the total remaining resources of all nodes; S43, each node is assigned a task T j Asynchronously update the large model parameters Wj, and each node independently calculates the subtask gradient update: Among them, η j is the learning rate, For the corresponding gradient calculation, the update process is Controls the adaptive adjustment of learning rate; S44, real-time monitoring of the task progress of each node P i , when a node completes its task, re-evaluate the computational complexity L of the remaining tasks remain and the remaining computing capacity of the node C remain ; S45. After all subtasks are completed, the system updates the parameters W of each node. j Merge to generate the updated large model parameter matrix W t+1 :

6. According to the large model cloud computing dynamic update method based on elastic adaptive incremental learning according to claim 1, it is characterized in that: The S5 step includes: S51. After the large model is updated, the updated large model version M is obtained. t+1 , through the preset performance index P target Verify the performance of the large model. The performance indicators include the accuracy of the model A t+1 , calculation efficiency E t+1 and resource utilization rate R t+1 : Among them, α, β, and γ are the weights of different performance indicators, A t+1 is the accuracy of the model, E t+1 is the computational efficiency, R t+1 is the resource utilization rate, R max is the maximum allowed resource usage; S52, the verification result P t+1 With the preset threshold P target For comparison, if P t+1 ≥P target , then save and deploy the updated large model version M t+1 , replacing it with the currently running large model version; S53, if P t+1 <P target , then adjust the relevant parameters of the large model and reset the model's learning rate η, update frequency f and parameter matrix W; S54, after the parameter adjustment is completed, the system re-executes the update process until the model performance index P t+1 Reach or exceed the preset threshold P target , and finally deploy the updated large model version.

7. According to the large model cloud computing dynamic update method based on elastic adaptive incremental learning according to claim 1, it is characterized in that: The S6 step includes: S61. After the large model update deployment is completed, the system monitors the resource utilization U of each computing node on the cloud computing platform. i ; S62: Perform resource recovery evaluation on all computing nodes. If the resource utilization rate of a node is U i Below the preset threshold U min , the system starts a dynamic resource recovery mechanism to recover underutilized computing resources Release it back to the cloud platform for use in other tasks: in, is the computing resource used by the i-th node during the update process, is the total computing resources of the node; S63, dynamically adjust the resource allocation strategy of each computing node according to the recovered idle resources Allocate resources for other high-load nodes; S64. During the resource recovery process, the system monitors the overall resource utilization rate of the cloud platform in real time. total , when U total Reach the preset high efficiency utilization threshold U opt When the resource recovery is stopped, the system re-evaluates the computing resource requirements of the large model based on the utilization of the resources after recovery, and adjusts the priority and scheduling strategy of resource allocation in the future model update process.

Images