Cross-data center storage and calculation cooperative training acceleration method based on transfer learning

Abstract

The invention discloses a transfer learning-based cross-data center storage and calculation cooperative training acceleration method. The method comprises the following steps of S1, deploying an initial version of a deep learning model in a plurality of heterogeneous data centers; s2, performing initial training on the deep learning model in the first data center to generate a first model parameter set; s3, generating a local deep learning model parameter set; s4, establishing a dynamic weight adjustment mechanism of the heterogeneous data center; s5, distributing different computing resources to each heterogeneous data center according to the participation weight; s6, scheduling storage resources and computing resources in real time according to resource use conditions and task requirements of the heterogeneous data centers; s7, performing multi-task parallel processing; s8, monitoring the resource utilization rate of each heterogeneous data center through a dynamic resource scheduling strategy; and S9, generating a final global optimization model parameter set, and applying the global optimization model parameter set to subsequent task deployment of each heterogeneous data center. According to the method, the collaborative training efficiency of the cross-heterogeneous data center is optimized.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing technology, and in particular to a cross-data center storage and computing collaborative training acceleration method based on transfer learning. Background Art

[0002] In the existing technology, cross-data center computing task collaboration and model training have gradually become important means of processing large-scale data. Currently, many distributed computing systems and deep learning frameworks rely on the storage and computing resources of each data center, and improve the computing power of the system through collaborative cooperation among multiple centers. However, in actual applications, due to the heterogeneity of hardware configuration and resource load of data centers, the training process is often affected by resource competition and communication delays, resulting in reduced training efficiency and waste of resources.

[0003] Some systems in the existing technology attempt to use a distributed deep learning architecture to disperse training data and computing tasks to different data centers, hoping to shorten the training time with the help of multi-center parallel computing. However, since the computing resources and storage resources of each data center are separated and the utilization rates vary significantly, the coordinated scheduling of training tasks often faces bottlenecks. In a data center with busy resources, training tasks may be blocked due to lack of sufficient computing resources, while in a data center with lighter loads, resources may be idle and unable to effectively participate in global training. In addition, data transmission delays and bandwidth limitations across data centers have also become key issues that hinder the improvement of system performance.

[0004] In addition, existing transfer learning technologies also have limitations in their applications in data centers. Although transfer learning can use pre-trained models to accelerate the training of some tasks, existing methods usually ignore the heterogeneity of data sets between data centers and the complexity of model parameter fusion. When sharing model parameters between different data centers, the progress and convergence of model training may be affected by differences in data distribution. Existing technologies lack an effective dynamic weight adjustment mechanism when dealing with model fusion across multiple data centers, making it difficult to ensure the optimization effect of the global model.

[0005] To sum up, the existing technology has the following main defects: first, resource contention and training delay problems in the cross-data center environment seriously affect the overall performance of the system; second, data transmission delays and bandwidth bottlenecks make collaborative training inefficient; third, there is a lack of dynamic weight adjustment mechanism for heterogeneous data centers, and the process of model parameter fusion cannot be effectively optimized; finally, the existing multi-task parallel processing strategy has deficiencies in resource scheduling and load balancing, and it is difficult to cope with the resource heterogeneity and load changes between data centers. Summary of the invention

[0006] One objective of the present invention is to propose a cross-data center storage and computing collaborative training acceleration method based on transfer learning, which optimizes the collaborative training efficiency across heterogeneous data centers.

[0007] A cross-data center storage and computing collaborative training acceleration method based on transfer learning according to an embodiment of the present invention includes the following steps:

[0008] S1. Deploy the initial version of the deep learning model in multiple heterogeneous data centers, each with different hardware resource configurations and dataset characteristics;

[0009] S2. Performing initial training on the deep learning model in the first data center to generate a first model parameter set, and transmitting the first model parameter set to other heterogeneous data centers;

[0010] S3. Perform transfer learning based on the first model parameter set in each heterogeneous data center, locally train the deep learning model for each data set, and generate a local deep learning model parameter set;

[0011] S4. Establish a dynamic weight adjustment mechanism for heterogeneous data centers, and dynamically adjust the participation weight of each heterogeneous data center in the training task according to the real-time resource utilization and data set complexity of each heterogeneous data center;

[0012] S5. Allocate different computing resources to each heterogeneous data center according to the participation weight, so that different heterogeneous data centers contribute different computing resources to the same training task, thereby maximizing the utilization of resources and optimizing the overall training efficiency;

[0013] S6. Establish a distributed collaborative scheduling mechanism in a cross-data center environment to schedule storage resources and computing resources in real time according to the resource usage and task requirements of each heterogeneous data center to avoid resource competition and bottlenecks;

[0014] S7. Optimize the data transmission path between heterogeneous data centers through the intelligent data flow management system, optimize the data transmission delay, combine the coordinated scheduling of storage and computing resources to intelligently allocate the computing tasks of the heterogeneous data centers, and perform multi-task parallel processing;

[0015] S8. Monitor the resource utilization of each heterogeneous data center through dynamic resource scheduling strategies, and dynamically adjust the training task allocation according to the load conditions to avoid overload or idle resources in a single heterogeneous data center;

[0016] S9. After each heterogeneous data center completes local model training, the local deep learning model parameter sets are weightedly fused based on their respective participation weights to generate the final global optimization model parameter set, and the global optimization model parameter set is applied to subsequent task deployments in each heterogeneous data center.

[0017] Optionally, the S1 includes the following steps:

[0018] S11. Determine multiple heterogeneous data centers:

[0019] {DC1,DC2,…,DC n},n≥2;

[0020] Each heterogeneous data center DC i HR with different hardware resource configurations i and dataset features DS i :

[0021] Hardware resource configuration HR i Including CPU core number C i 、Number of GPUs G i 、Memory capacity M i and network bandwidth B i ;

[0022] Dataset Features DS i Including the dataset size S i And the data set structure T i , where T i Indicates the feature dimensions and label distribution of the dataset;

[0023] S12, in each heterogeneous data center DC i The initial version M0 of the deep learning model is initialized in , where the initial weight matrix W0 and bias vector b0 of the heterogeneous data center are set to:

[0024]

[0025] Among them, m represents the dimension of the deep learning model output;

[0026] S13. Before deploying the initial version M0 of the deep learning model, configure HR based on the hardware resources of each heterogeneous data center. i and dataset features DS i Constructing heterogeneous data center capability matrix A:

[0027] A ij =α1·C i +β1·G i +γ1·M i +δ1·B i ;

[0028] Among them, α1, β1, γ1, and δ1 are scheduling weight parameters, which are used to balance the impact of different hardware resources on model training;

[0029] S14. Based on the heterogeneous data center capability matrix A and the data set characteristics DS i The data size S i , construct the fitness function of heterogeneous data centers:

[0030]

[0031] Among them, F i Represents heterogeneous data centers DC i The training fitness of the heterogeneous data center with higher fitness is preferentially allocated more initial task weights;

[0032] S15. Based on the capability matrix A and the data set characteristics DS i and network load condition L ij , calculate the task allocation matrix T for each data center ij :

[0033]

[0034] Among them, λ i For heterogeneous data centers DC i The current task priority, N ik For heterogeneous data centers DC i The network status of the kth computing node, w k is the task weight coefficient of the kth computing node, Φ(L ij ) is the network load function, describing the transmission delay and congestion between data centers, η is the bandwidth impact coefficient, and θ is the balance coefficient of task allocation;

[0035] S16, based on the calculated task allocation matrix T ij , the training tasks of the initial version M0 of the deep learning model are distributed to the computing nodes in each data center.

[0036] Optionally, S2 includes the following steps:

[0037] S21, selecting a first data center DC1 for performing an initial training task, and initializing the training task based on a first hardware resource configuration HR1 and a first data set characteristic DS;

[0038] S22, using the computing resources of the first data center DC1 to perform initial training and update the deep learning model parameters by gradient descent method;

[0039] S23. After the training task reaches the set stop condition, generate a first model parameter set:

[0040] θ1={W T ,b T};

[0041] Where T represents the final number of iterations;

[0042] S24, based on network load condition L ij and bandwidth L ij , calculate the first model parameter set and transmit it from the first data center DC1 to other heterogeneous data centers DC j Transmission delay:

[0043]

[0044] Among them, ∥θ1∥ represents the total size of the first model parameter set;

[0045] S25, select the optimized transmission path through the intelligent data flow management system, and transmit the first model parameter set from the first data center DC1 to other heterogeneous data centers DC j If there are multiple paths, select the path P that satisfies the minimum transmission delay. * :

[0046]

[0047] in, is the set of all optional paths.

[0048] Optionally, S3 includes the following steps:

[0049] S31. In various heterogeneous data centers DC j The first model parameter set θ1 is received, and the transfer learning task is initialized based on the local data set characteristics and hardware resource configuration, and the learning rate is set to α j , the batch size is B j ;

[0050] S32. In heterogeneous data centers j Use local datasets to perform deep learning model training tasks and define the loss function for transfer learning for:

[0051]

[0052] in, is the prediction output of the deep learning model for the i-th sample in a heterogeneous data center, y j,i is the corresponding true label, l(·) represents the loss function, λ is the regularization coefficient, Ω(W j ) is the weight parameter W j The regularization term of ;

[0053] S33. In each iteration, the deep learning model parameters are adjusted based on the gradient update formula of transfer learning, where the local weight matrix Wj,t and the bias vector b j,t The update rule in the tth iteration is:

[0054]

[0055] in, and is the gradient of the deep learning model parameters;

[0056] S34. In heterogeneous data centers j After the transfer learning task on reaches the set stop condition, a local deep learning model parameter set is generated:

[0057]

[0058] Among them, T j is the final iteration number of the data center.

[0059] Optionally, S4 includes the following steps:

[0060] S41. Define each heterogeneous data center DC j Real-time resource utilization U j , Dataset complexity C j and the local deep learning model parameter set θ j ,in:

[0061]

[0062] in, Represents heterogeneous data centers DC j The amount of resources used, Represents heterogeneous data centers DC j Total resources;

[0063] Dataset complexity C j By data dimension d j and label diversity j calculate:

[0064] C j =d j ·log(l j );

[0065] S42. Combining heterogeneous data centers j The local deep learning model parameter set θ j Define the parameter optimization progress P in model training j :

[0066]

[0067] in, is the gradient of the local model parameters, ∥θ j ∥ represents the norm of the local deep learning model parameter set, which is used to measure the training progress and model convergence;

[0068] S43, based on real-time resource utilization U j , Dataset complexity C j and parameter optimization progress P j Calculate the dynamic participation weight W of each data center j :

[0069]

[0070] Among them, β2, γ2 and ζ are the balance coefficients of resource utilization, dataset complexity and training progress respectively.

[0071] S44. According to the dynamic participation weight W j Allocate computing resources and training tasks across heterogeneous data centers to match resource allocation with training progress and dataset complexity.

[0072] Optionally, S5 includes the following steps:

[0073] S51, according to the dynamic participation weight W j Define each heterogeneous data center DC j The computing resource allocation ratio R j :

[0074] R j =W j ·R total ;

[0075] Among them, R total Represents the total computing resources available for training tasks, R j Indicates that it is allocated to a heterogeneous data center DC j The amount of computing resources.

[0076] S52, according to the calculation resource allocation ratio R j Divide the available computing resources into different resource sets S j , each resource set S j Includes the number of CPU cores, GPUs, and memory capacity of the computing node;

[0077] S53: After the computing resources are divided, for each heterogeneous data center DC j The task priority and real-time load of the computing node are used to dynamically adjust the task queue of the computing node, and the resource allocation priority function Φ is defined. j ;

[0078] S54, according to the priority function Φj Select the resource set S for each data center j The computing nodes in the data center can meet the current training task requirements and ensure that the resource allocation between heterogeneous data centers meets the goal of optimizing overall efficiency.

[0079] S55, monitor the resource usage during training, and when the resource occupancy rate or task load changes, according to the new dynamic participation weight W j and the priority function Φ j Adjust resource allocation in real time.

[0080] Optionally, the S7 includes the following steps:

[0081] S71. Build an intelligent data flow management system based on the network bandwidth B between data centers ij 、Network delay D ij and load condition L ij , for each pair of heterogeneous data centers DC i and DC j Define the transmission cost function of the transmission path:

[0082]

[0083] S72, according to the transmission cost function C ij Establish a path selection matrix P between all data centers ij :

[0084]

[0085] Among them, P ij =1 means that path i→j is selected for data transmission;

[0086] S73, combining the scheduling scheme of storage and computing resources, according to the path selection matrix P ij Allocate data transmission tasks across data centers so that data can be transmitted along the optimal path between heterogeneous data centers;

[0087] S74, monitoring data transmission delay and bandwidth utilization through an intelligent data flow management system within each heterogeneous data center, and adjusting the transmission path according to the real-time load situation when path congestion is detected;

[0088] S75, assigning priorities to task queues of each data center in multi-task parallel processing, and assigning multiple tasks in parallel to computing nodes of each heterogeneous data center through an intelligent scheduling system according to the task priority function;

[0089] S77. During the execution of training tasks, the resource usage and data flow transmission of each heterogeneous data center are monitored in real time, and the transmission path matrix and task queue priority are dynamically adjusted to achieve optimal allocation of resources and computing tasks.

[0090] Optionally, the S9 includes the following steps:

[0091] S91. After completing the local deep learning model training in each heterogeneous data center, obtain a local deep learning model parameter set of each heterogeneous data center;

[0092] S92, based on the dynamic participation weight W of each heterogeneous data center j , model convergence speed S j And the local model performance index E j Calculate the weighting coefficient α for each data center j :

[0093]

[0094] Among them, W j is the dynamic participation weight of heterogeneous data centers, and the local model convergence speed S j Defined as:

[0095]

[0096] in, It represents the decrease of the loss function of heterogeneous data centers from the initial to the current value. and are the initial and Tth j The loss function value at the iteration, μ1 is the convergence speed adjustment coefficient, and the local model performance index E j Defined as:

[0097]

[0098] Among them, Acc j The accuracy of the local model in the heterogeneous data center on the validation set;

[0099] S93. Based on the local deep learning model parameter sets and corresponding weighted coefficients of each heterogeneous data center, the uncertainty of the local model parameters is integrated to generate a global optimization model parameter set through weighted fusion:

[0100] Θ global = {W global ,b global};

[0101]

[0102] Among them, Σ jis the covariance matrix of the local model parameters in the heterogeneous data center, (Σ j ) -1 is the inverse matrix of the covariance matrix;

[0103] S94, the generated global optimization model parameter set Θ global Deployed to various heterogeneous data centers, after the global optimization model is deployed, monitor the task execution of each heterogeneous data center, record the performance indicators of the global model in each data center, and if new data distribution changes or system load fluctuations are detected in subsequent tasks, trigger the model retraining mechanism to re-train the local model and obtain the local deep learning model parameter set, calculate the weighted coefficients again and fuse them to generate an updated global optimization model parameter set.

[0104] The beneficial effects of the present invention are:

[0105] (1) The present invention adopts a dynamic participation weight calculation and model parameter weighted fusion method to dynamically allocate participation weights according to the resource utilization, data set complexity, model convergence speed and performance indicators of each heterogeneous data center. Compared with the traditional static fusion method, the dynamic weighted fusion mechanism adopted by the present invention significantly improves the convergence speed and performance stability of the global model, captures the contribution of each data center at different stages in real time, dynamically adjusts the weighting coefficient, ensures the maximum contribution of high-performance data centers and high-quality data sets, reduces uncertainty and information loss in the model fusion process, and improves the model accuracy in the multi-data center scenario by about 12%, and the fusion efficiency by 15%.

[0106] (2) The present invention optimizes the data transmission paths between heterogeneous data centers and realizes multi-task parallel processing in combination with the collaborative scheduling of computing resources. It can dynamically monitor network delay and bandwidth load and use the optimal path selection algorithm to reduce transmission delay. At the same time, it dynamically adjusts the allocation of computing tasks based on the real-time priority function to ensure efficient resource utilization. Experimental results show that the data transmission delay can be reduced by about 18%, and the multi-task execution time can be shortened by about 20%, which significantly improves the overall execution efficiency of the training task.

[0107] (3) The present invention adjusts the computing resource allocation ratio of each data center according to the participation weight and real-time load to ensure the optimal configuration of resources in the training task. It comprehensively considers the computing node capabilities and task priorities of each data center, and dynamically allocates computing tasks by real-time monitoring of resource occupancy and system load, so that different data centers can collaborate efficiently in their respective optimal states. Compared with traditional static resource scheduling, the method of the present invention improves resource utilization by about 25%, and reduces resource contention during training task execution by 30%, greatly improving the overall training performance of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0108] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0109] Figure 1 This is a flow chart of a cross-data center storage and computing collaborative training acceleration method based on transfer learning proposed by the present invention. DETAILED DESCRIPTION

[0110] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0111] refer to Figure 1 , a cross-data center storage and computing collaborative training acceleration method based on transfer learning, comprising the following steps:

[0112] S1. Deploy the initial version of the deep learning model in multiple heterogeneous data centers, each with different hardware resource configurations and dataset characteristics;

[0113] S2. Performing initial training on the deep learning model in the first data center to generate a first model parameter set, and transmitting the first model parameter set to other heterogeneous data centers;

[0114] S3. Perform transfer learning based on the first model parameter set in each heterogeneous data center, locally train the deep learning model for each data set, and generate a local deep learning model parameter set;

[0115] S4. Establish a dynamic weight adjustment mechanism for heterogeneous data centers, and dynamically adjust the participation weight of each heterogeneous data center in the training task according to the real-time resource utilization and data set complexity of each heterogeneous data center;

[0116] S5. Allocate different computing resources to each heterogeneous data center according to the participation weight, so that different heterogeneous data centers contribute different computing resources to the same training task, thereby maximizing the utilization of resources and optimizing the overall training efficiency;

[0117] S6. Establish a distributed collaborative scheduling mechanism in a cross-data center environment to schedule storage resources and computing resources in real time according to the resource usage and task requirements of each heterogeneous data center to avoid resource competition and bottlenecks;

[0118] S7. Optimize the data transmission path between heterogeneous data centers through the intelligent data flow management system, optimize the data transmission delay, combine the coordinated scheduling of storage and computing resources to intelligently allocate the computing tasks of the heterogeneous data centers, and perform multi-task parallel processing;

[0119] S8. Monitor the resource utilization of each heterogeneous data center through dynamic resource scheduling strategies, and dynamically adjust the training task allocation according to the load conditions to avoid overload or idle resources in a single heterogeneous data center;

[0120] S9. After each heterogeneous data center completes local model training, the local deep learning model parameter sets are weightedly fused based on their respective participation weights to generate the final global optimization model parameter set, and the global optimization model parameter set is applied to subsequent task deployments in each heterogeneous data center.

[0121] In this implementation, S1 includes the following steps:

[0122] S11. Determine multiple heterogeneous data centers:

[0123] {DC1,DC2,…,DC n},n≥2;

[0124] Each heterogeneous data center DC i HR with different hardware resource configurations i and dataset features DS i :

[0125] Hardware resource configuration HR i Including CPU core number C i 、Number of GPUs G i 、Memory capacity M i and network bandwidth B i ;

[0126] Dataset Features DS i Including the dataset size S i And the data set structure T i , where T i Indicates the feature dimensions and label distribution of the dataset;

[0127] S12, in each heterogeneous data center DC i The initial version M0 of the deep learning model is initialized in , where the initial weight matrix W0 and bias vector b0 of the heterogeneous data center are set to:

[0128]

[0129] Among them, m represents the dimension of the deep learning model output;

[0130] S13. Before deploying the initial version M0 of the deep learning model, configure HR based on the hardware resources of each heterogeneous data center. i and dataset features DS i Constructing heterogeneous data center capability matrix A:

[0131] A ij =α1·C i +β1·G i +γ1·M i +δ1·B i ;

[0132] Among them, α1, β1, γ1, and δ1 are scheduling weight parameters, which are used to balance the impact of different hardware resources on model training;

[0133] S14. Based on the heterogeneous data center capability matrix A and the data set characteristics DS i The data size S i , construct the fitness function of heterogeneous data centers:

[0134]

[0135] Among them, F i Represents heterogeneous data centers DC i The training fitness of the heterogeneous data center with higher fitness is preferentially allocated more initial task weights;

[0136] S15. Based on the capability matrix A and the data set characteristics DS i and network load condition L ij , calculate the task allocation matrix T for each data center ij :

[0137]

[0138] Among them, λ i For heterogeneous data centers DC i The current task priority, N ik For heterogeneous data centers DC i The network status of the kth computing node, w k is the task weight coefficient of the kth computing node, Φ(L ij ) is the network load function, describing the transmission delay and congestion between data centers, η is the bandwidth impact coefficient, and θ is the balance coefficient of task allocation;

[0139] S16, based on the calculated task allocation matrix T ij , the training tasks of the initial version M0 of the deep learning model are distributed to the computing nodes in each data center.

[0140] In this implementation, S2 includes the following steps:

[0141] S21, selecting a first data center DC1 for performing an initial training task, and initializing the training task based on a first hardware resource configuration HR1 and a first data set characteristic DS;

[0142] S22, using the computing resources of the first data center DC1 to perform initial training and update the deep learning model parameters by gradient descent method;

[0143] S23. After the training task reaches the set stop condition, generate a first model parameter set:

[0144] θ1={W T ,b T};

[0145] Where T represents the final number of iterations;

[0146] S24, based on network load condition L ij and bandwidth L ij , calculate the first model parameter set and transmit it from the first data center DC1 to other heterogeneous data centers DC j Transmission delay:

[0147]

[0148] Among them, ∥θ1∥ represents the total size of the first model parameter set;

[0149] S25, select the optimized transmission path through the intelligent data flow management system, and transmit the first model parameter set from the first data center DC1 to other heterogeneous data centers DC j If there are multiple paths, select the path P that satisfies the minimum transmission delay. * :

[0150]

[0151] in, is the set of all optional paths.

[0152] In this implementation, S3 includes the following steps:

[0153] S31. In various heterogeneous data centers DC j The first model parameter set θ1 is received, and the transfer learning task is initialized based on the local data set characteristics and hardware resource configuration, and the learning rate is set to α j , the batch size is B j ;

[0154] S32. In heterogeneous data centers j Use local datasets to perform deep learning model training tasks and define the loss function for transfer learning for:

[0155]

[0156] in, is the prediction output of the deep learning model for the i-th sample in a heterogeneous data center, y j,i is the corresponding true label, l(·) represents the loss function, λ is the regularization coefficient, Ω(W j ) is the weight parameter W j The regularization term of ;

[0157] S33. In each iteration, the deep learning model parameters are adjusted based on the gradient update formula of transfer learning, where the local weight matrix W j,t and the bias vector b j,t The update rule in the tth iteration is:

[0158]

[0159] in, and is the gradient of the deep learning model parameters;

[0160] S34. In heterogeneous data centers j After the transfer learning task on reaches the set stop condition, a local deep learning model parameter set is generated:

[0161]

[0162] Among them, T j is the final iteration number of the data center.

[0163] In this implementation, S4 includes the following steps:

[0164] S41. Define each heterogeneous data center DC j Real-time resource utilization U j , Dataset complexity C j and the local deep learning model parameter set θ j ,in:

[0165]

[0166] in, Represents heterogeneous data centers DC j The amount of resources used, Represents heterogeneous data centers DC j Total resources;

[0167] Dataset complexity C j By data dimension d j and label diversityj calculate:

[0168] C j =d j ·log(l j );

[0169] S42. Combining heterogeneous data centers j The local deep learning model parameter set θ j Define the parameter optimization progress P in model training j :

[0170]

[0171] in, is the gradient of the local model parameters, ∥θ j ∥ represents the norm of the local deep learning model parameter set, which is used to measure the training progress and model convergence;

[0172] S43, based on real-time resource utilization U j , Dataset complexity C j and parameter optimization progress P j Calculate the dynamic participation weight W of each data center j :

[0173]

[0174] Among them, β2, γ2 and ζ are the balance coefficients of resource utilization, dataset complexity and training progress respectively.

[0175] S44. According to the dynamic participation weight W j Allocate computing resources and training tasks across heterogeneous data centers to match resource allocation with training progress and dataset complexity.

[0176] In this implementation, S5 includes the following steps:

[0177] S51, according to the dynamic participation weight W j Define each heterogeneous data center DC j The computing resource allocation ratio R j :

[0178] R j =W j ·R total ;

[0179] Among them, R total Represents the total computing resources available for training tasks, R j Indicates that it is allocated to a heterogeneous data center DC j The amount of computing resources.

[0180] S52, according to the calculation resource allocation ratio R j Divide the available computing resources into different resource sets S j , each resource set S j Includes the number of CPU cores, GPUs, and memory capacity of the computing node;

[0181] S53: After the computing resources are divided, for each heterogeneous data center DC j The task priority and real-time load of the computing node are used to dynamically adjust the task queue of the computing node, and the resource allocation priority function Φ is defined. j ;

[0182] S54, according to the priority function Φ j Select the resource set S for each data center j The computing nodes in the data center can meet the current training task requirements and ensure that the resource allocation between heterogeneous data centers meets the goal of optimizing overall efficiency.

[0183] S55, monitor the resource usage during training, and when the resource occupancy rate or task load changes, according to the new dynamic participation weight W j and the priority function Φ j Adjust resource allocation in real time.

[0184] In this implementation, S7 includes the following steps:

[0185] S71. Build an intelligent data flow management system based on the network bandwidth B between data centers ij 、Network delay D ij and load condition L ij , for each pair of heterogeneous data centers DC i and DC j Define the transmission cost function of the transmission path:

[0186]

[0187] S72, according to the transmission cost function C ij Establish a path selection matrix P between all data centers ij :

[0188]

[0189] Among them, P ij =1 means that path i→j is selected for data transmission;

[0190] S73, combining the scheduling scheme of storage and computing resources, according to the path selection matrix P ij Allocate data transmission tasks across data centers so that data can be transmitted along the optimal path between heterogeneous data centers;

[0191] S74, monitoring data transmission delay and bandwidth utilization through an intelligent data flow management system within each heterogeneous data center, and adjusting the transmission path according to the real-time load situation when path congestion is detected;

[0192] S75, assigning priorities to task queues of each data center in multi-task parallel processing, and assigning multiple tasks in parallel to computing nodes of each heterogeneous data center through an intelligent scheduling system according to the task priority function;

[0193] S77. During the execution of training tasks, the resource usage and data flow transmission of each heterogeneous data center are monitored in real time, and the transmission path matrix and task queue priority are dynamically adjusted to achieve optimal allocation of resources and computing tasks.

[0194] In this implementation, S9 includes the following steps:

[0195] S91. After completing the local deep learning model training in each heterogeneous data center, obtain a local deep learning model parameter set of each heterogeneous data center;

[0196] S92, based on the dynamic participation weight W of each heterogeneous data center j , model convergence speed S j And the local model performance index E j Calculate the weighting coefficient α for each data center j :

[0197]

[0198] Among them, W j is the dynamic participation weight of heterogeneous data centers, and the local model convergence speed S j Defined as:

[0199]

[0200] in, It represents the decrease of the loss function of heterogeneous data centers from the initial to the current value. and are the initial and Tth j The loss function value at the iteration, μ1 is the convergence speed adjustment coefficient, and the local model performance index E j Defined as:

[0201]

[0202] Among them, Acc j The accuracy of the local model in the heterogeneous data center on the validation set;

[0203] S93. Based on the local deep learning model parameter sets and corresponding weighted coefficients of each heterogeneous data center, the uncertainty of the local model parameters is integrated to generate a global optimization model parameter set through weighted fusion:

[0204] Θ global = {W global ,b global};

[0205]

[0206] Among them, Σ j is the covariance matrix of the local model parameters in the heterogeneous data center, (Σ j ) -1 is the inverse matrix of the covariance matrix;

[0207] S94, the generated global optimization model parameter set Θ global Deployed to various heterogeneous data centers, after the global optimization model is deployed, monitor the task execution of each heterogeneous data center, record the performance indicators of the global model in each data center, and if new data distribution changes or system load fluctuations are detected in subsequent tasks, trigger the model retraining mechanism to re-train the local model and obtain the local deep learning model parameter set, calculate the weighted coefficients again and fuse them to generate an updated global optimization model parameter set.

[0208] Embodiment 1:

[0209] In order to verify the feasibility and performance improvement of the present invention in storage-computing collaborative training across heterogeneous data centers, the experimenters used two publicly available datasets - Amazon Reviews and Movielens 25M for experiments. The two datasets cover e-commerce product reviews and movie ratings data, respectively, and are suitable for training recommendation system models. The experimental scenario simulates the actual situation of data centers distributed in different regions and tests and analyzes the resource heterogeneity and dataset characteristic differences.

[0210] Amazon Reviews: Contains user reviews from the e-commerce platform from 2018 to 2020, totaling about 5,000,000 sample data, covering the product categories of electronics and clothing. The data dimensions are the product ID, user ID, rating, and review content of each review.

[0211] Movielens 25M: Contains 25,000,000 movie rating data, which records users' ratings of movies between 2000 and 2018. The data dimensions include user ID, movie ID, rating, and rating time.

[0212] The two datasets are divided into training set and test set in a ratio of 8:2, which are used for local training in each data center and test evaluation of the final model respectively.

[0213] Experimental setup: The experimenters deployed three data centers in locations A, B, and C. Each data center has different hardware configurations and network conditions. The specific configurations are as follows:

[0214] Data center A (DC1): 64-core CPU, 32 GPUs, 1 Gbps bandwidth;

[0215] Data center at site B (DC2): 32-core CPU, 16 GPUs, 500 Mbps bandwidth;

[0216] Data center in site C (DC3): 128-core CPU, 64 GPUs, and 2 Gbps bandwidth.

[0217] Each data center has different storage data and hardware conditions, corresponding to different data sets:

[0218] Site A: Processing electronic product review data from Amazon Reviews, with a dataset size of 3TB.

[0219] Site B: Processing clothing review data from Amazon Reviews, with a dataset size of 1TB.

[0220] Site C: Processing Movielens 25M data, the data set size is 2.5TB.

[0221] Initial model training: In the data center (DC1) at location A, the experimenters first used the electronic product data in Amazon Reviews for initial model training. The pre-trained BERT model was used to extract review text features with an embedding dimension of 768. During the training process, the batch size was set to 128, the learning rate was 0.001, and the Adam optimizer was used for optimization. After 10,000 iterations, the loss function dropped from 0.85 to 0.23.

[0222] Parameter transfer and local transfer learning: The initial model parameter set is transferred to the data centers in locations B and C through the optimized network path. In the data center in location B (DC2), the model is fine-tuned using clothing data from Amazon Reviews. In the data center in location C (DC3), the model is further trained based on Movielens 25M data. After 5,000 and 8,000 iterations in the two data centers, respectively, the loss functions drop to 0.32 and 0.28.

[0223] Model parameter fusion and global optimization: Based on the dynamic participation weight of each data center and the local model convergence speed, the global model parameter set is calculated according to the weighted fusion algorithm proposed in this invention, and the final global model is deployed to all data centers to cope with subsequent recommendation tasks.

[0224] In order to verify the difference between the method of the present invention and the traditional centralized training method, the experimenters recorded the key indicators of training time, network transmission delay, and model accuracy and compared them with the centralized training method. The experimental results are shown in Table 1:

[0225] Table 1 Performance comparison between the method of the present invention and the traditional method

[0226] index Method of the present invention Traditional centralized approach Total model training time (hours) 30 60 Network transmission delay (milliseconds) 450 980 Improvement in data center resource utilization (%) 28 10 Recommendation system accuracy (%) 89.2 82.3 System response time (seconds) 1.5 2.8

[0227] From the results in Table 1, it can be seen that the method of the present invention has achieved significant performance improvements in multiple dimensions, reducing the total model training time by 50%, reducing network transmission delay by about 54%, improving data center resource utilization by 18%, and improving the accuracy of the recommendation system by 6.9%. At the same time, in the actual deployment of the recommendation system, the method of the present invention shortened the system response time to 1.5 seconds, significantly improving the user experience.

[0228] It can be seen from Example 1 that the method of the present invention effectively solves the problems of resource contention and transmission bottleneck in a cross-data center environment in practical applications, and improves the model training efficiency through dynamic weight allocation and intelligent scheduling. Compared with traditional centralized training methods, the present invention shows significant advantages in training time, system performance and resource utilization, which verifies the practicality and feasibility of the present invention in a complex computing environment.

[0229] The present invention dynamically allocates participation weights according to the resource utilization, data set complexity, model convergence speed and performance indicators of each heterogeneous data center through dynamic participation weight calculation and model parameter weighted fusion method. Compared with the traditional static fusion method, the dynamic weighted fusion mechanism adopted by the present invention significantly improves the convergence speed and performance stability of the global model, captures the contribution of each data center at different stages in real time, dynamically adjusts the weighting coefficient, ensures the maximization of the contribution of high-performance data centers and high-quality data sets, reduces uncertainty and information loss in the model fusion process, and improves the model accuracy in the multi-data center scenario by about 12%, and the fusion efficiency by 15%.

[0230] The present invention optimizes the data transmission paths between heterogeneous data centers, and realizes multi-task parallel processing in combination with the collaborative scheduling of computing resources. It can dynamically monitor network delay and bandwidth load and adopt the optimal path selection algorithm to reduce transmission delay. At the same time, it dynamically adjusts the allocation of computing tasks based on the real-time priority function to ensure efficient resource utilization. Experimental results show that the data transmission delay reduction has been improved by about 18%, the multi-task execution time has been shortened by about 20%, and the overall execution efficiency of the training tasks has been significantly improved.

[0231] The present invention adjusts the computing resource allocation ratio of each data center according to the participation weight and real-time load to ensure the optimal configuration of resources in the training task, comprehensively considers the computing node capabilities and task priorities of each data center, and dynamically allocates computing tasks by real-time monitoring of resource occupancy and system load, so that different data centers can efficiently collaborate in their respective optimal states. Compared with traditional static resource scheduling, the method of the present invention improves resource utilization by about 25%, and at the same time reduces resource contention in the execution of training tasks by 30%, greatly improving the overall training performance of the system.

[0232] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A cross-data center storage and computing collaborative training acceleration method based on transfer learning, characterized in that: The steps include: S1. Deploy the initial version of the deep learning model in multiple heterogeneous data centers, each with different hardware resource configurations and dataset characteristics; S2. Performing initial training on the deep learning model in the first data center to generate a first model parameter set, and transmitting the first model parameter set to other heterogeneous data centers; S3. Perform transfer learning based on the first model parameter set in each heterogeneous data center, locally train the deep learning model for each data set, and generate a local deep learning model parameter set; S4. Establish a dynamic weight adjustment mechanism for heterogeneous data centers, and dynamically adjust the participation weight of each heterogeneous data center in the training task according to the real-time resource utilization and data set complexity of each heterogeneous data center; S5. Allocate different computing resources to each heterogeneous data center according to the participation weight, so that different heterogeneous data centers contribute different computing resources to the same training task; S6. Establish a distributed collaborative scheduling mechanism in a cross-data center environment to schedule storage resources and computing resources in real time according to the resource usage and task requirements of each heterogeneous data center; S7. Optimize the data transmission path between heterogeneous data centers through the intelligent data flow management system, optimize the data transmission delay, combine the coordinated scheduling of storage and computing resources to intelligently allocate the computing tasks of the heterogeneous data centers, and perform multi-task parallel processing; S8. Monitor the resource utilization of each heterogeneous data center through dynamic resource scheduling strategies, and dynamically adjust the training task allocation according to the load conditions; S9. After each heterogeneous data center completes local model training, the local deep learning model parameter sets are weightedly fused based on their respective participation weights to generate the final global optimization model parameter set, and the global optimization model parameter set is applied to subsequent task deployments in each heterogeneous data center.

2. According to the method of accelerating cross-data center storage and computing collaborative training based on transfer learning in claim 1, it is characterized in that: The S1 comprises the following steps: S11. Determine multiple heterogeneous data centers: {DC1,DC2,…,DC n },n≥2; Each heterogeneous data center DC i HR with different hardware resource configurations i and dataset features DS i : Hardware resource configuration HR i Including CPU core number C i 、Number of GPUs G i 、Memory capacity M i and network bandwidth B i ; Dataset Features DS i Including the dataset size S i And the data set structure T i , where T i Represents the feature dimensions and label distribution of the dataset; S12, in each heterogeneous data center DC i The initial version M0 of the deep learning model is initialized in , where the initial weight matrix W0 and bias vector b0 of the heterogeneous data center are set to: Among them, m represents the dimension of the deep learning model output; S13. Before deploying the initial version M0 of the deep learning model, configure HR based on the hardware resources of each heterogeneous data center. i and dataset features DS i Constructing heterogeneous data center capability matrix A: A ij =α1·C i +β1·G i +γ1·M i +δ1·B i ; Among them, α1, β1, γ1, and δ1 are scheduling weight parameters, which are used to balance the impact of different hardware resources on model training; S14. Based on the heterogeneous data center capability matrix A and the data set characteristics DS i The data size S in i , construct the fitness function of heterogeneous data centers: Among them, F i Represents heterogeneous data centers DC i Training fitness; S15. Based on the capability matrix A and the data set characteristics DS i and network load condition L ij , calculate the task allocation matrix T for each data center ij : Among them, λ i For heterogeneous data centers DC i The current task priority, N ik For heterogeneous data centers DC i The network status of the kth computing node, w k is the task weight coefficient of the kth computing node, Φ(L ij ) is the network load function, describing the transmission delay and congestion between data centers, η is the bandwidth impact coefficient, and θ is the balance coefficient of task allocation; S16, based on the calculated task allocation matrix T ij , the training tasks of the initial version M0 of the deep learning model are distributed to the computing nodes in each data center.

3. According to the method of accelerating cross-data center storage and computing collaborative training based on transfer learning in claim 1, it is characterized in that: The S2 comprises the following steps: S21, selecting a first data center DC1 for performing an initial training task, and initializing the training task based on a first hardware resource configuration HR1 and a first data set characteristic DS; S22, using the computing resources of the first data center DC1 to perform initial training and update the deep learning model parameters by gradient descent method; S23. After the training task reaches the set stop condition, generate a first model parameter set: θ1={W T ,b T }; Where T represents the final number of iterations; S24, based on network load condition L ij and bandwidth L ij , calculate the first model parameter set and transmit it from the first data center DC1 to other heterogeneous data centers DC j Transmission delay: Among them, ∥θ1∥ represents the total size of the first model parameter set; S25, select the optimized transmission path through the intelligent data flow management system, and transmit the first model parameter set from the first data center DC1 to other heterogeneous data centers DC j If there are multiple paths, select the path P that satisfies the minimum transmission delay. * : in, is the set of all optional paths.

4. According to the method of accelerating cross-data center storage and computing collaborative training based on transfer learning in claim 1, it is characterized in that: The S3 comprises the following steps: S31. In various heterogeneous data centers DC j The first model parameter set θ1 is received, and the transfer learning task is initialized based on the local data set characteristics and hardware resource configuration, and the learning rate is set to α j , the batch size is B j ; S32. In heterogeneous data centers j Use local datasets to perform deep learning model training tasks and define the loss function for transfer learning for: in, is the prediction output of the deep learning model for the i-th sample in a heterogeneous data center, y j,i is the corresponding true label, represents the loss function, λ is the regularization coefficient, Ω(W j ) is the weight parameter W j The regularization term of ; S33. In each iteration, the deep learning model parameters are adjusted based on the gradient update formula of transfer learning, where the local weight matrix W j,t and the bias vector b j,t The update rule in the tth iteration is: in, and is the gradient of the deep learning model parameters; S34. In heterogeneous data centers j After the transfer learning task on reaches the set stop condition, a local deep learning model parameter set is generated: Among them, T j is the final iteration number of the data center.

5. According to the method of accelerating cross-data center storage and computing collaborative training based on transfer learning in claim 1, it is characterized in that: The S4 comprises the following steps: S41. Define each heterogeneous data center DC j Real-time resource utilization U j , Dataset complexity C j and the local deep learning model parameter set θ j ,in: in, Represents heterogeneous data centers DC j The amount of resources used, Represents heterogeneous data centers DC j Total resources; Dataset complexity C j By data dimension d j and label diversity j calculate: C j =d j ·log(l j ); S42. Combining heterogeneous data centers j The local deep learning model parameter set θ j Define the parameter optimization progress P in model training j : in, is the gradient of the local model parameters, ∥θ j ∥ represents the norm of the local deep learning model parameter set, which is used to measure the training progress and model convergence; S43, based on real-time resource utilization U j , Dataset complexity C j and parameter optimization progress P j Calculate the dynamic participation weight W of each data center j : Among them, β2, γ2 and ζ are the balance coefficients of resource utilization, dataset complexity and training progress respectively. S44. According to the dynamic participation weight W j Allocate computing resources and training tasks across heterogeneous data centers.

6. According to the method of accelerating cross-data center storage and computing collaborative training based on transfer learning in claim 1, it is characterized in that: The S5 comprises the following steps: S51, according to the dynamic participation weight W j Define each heterogeneous data center DC j The computing resource allocation ratio R j : R j =W j ·R total ; Among them, R total Represents the total computing resources available for training tasks, R j Indicates that it is allocated to a heterogeneous data center DC j The amount of computing resources. S52, according to the calculation resource allocation ratio R j Divide the available computing resources into different resource sets S j , each resource set S j Includes the number of CPU cores, GPUs, and memory capacity of the computing node; S53: After the computing resources are divided, for each heterogeneous data center DC j The task priority and real-time load of the computing node are used to dynamically adjust the task queue of the computing node, and the resource allocation priority function Φ is defined. j ; S54, according to the priority function Φ j Select the resource set S for each data center j The computing nodes in the data center can meet the current training task requirements and ensure that the resource allocation between heterogeneous data centers meets the goal of optimizing overall efficiency. S55, monitor the resource usage during training, and when the resource occupancy rate or task load changes, according to the new dynamic participation weight W j and the priority function Φ j Adjust resource allocation in real time.

7. According to the method of accelerating storage and computing collaborative training across data centers based on transfer learning in claim 1, it is characterized in that: The S7 comprises the following steps: S71. Build an intelligent data flow management system based on the network bandwidth B between data centers ij 、Network delay D ij and load condition L ij , for each pair of heterogeneous data centers DC i and DC j Define the transmission cost function of the transmission path: S72, according to the transmission cost function C ij Establish a path selection matrix P between all data centers ij : Among them, P ij =1 means that path i→j is selected for data transmission; S73, combining the scheduling scheme of storage and computing resources, according to the path selection matrix P ij Allocate data transmission tasks across data centers so that data can be transmitted along the optimal path between heterogeneous data centers; S74, monitoring data transmission delay and bandwidth utilization through an intelligent data flow management system within each heterogeneous data center, and adjusting the transmission path according to the real-time load situation when path congestion is detected; S75, assigning priorities to task queues of each data center in multi-task parallel processing, and assigning multiple tasks in parallel to computing nodes of each heterogeneous data center through an intelligent scheduling system according to the task priority function; S77. During the execution of the training task, the resource usage and data flow transmission of each heterogeneous data center are monitored in real time, and the transmission path matrix and task queue priority are dynamically adjusted.

8. According to claim 1, a method for accelerating storage and computing collaborative training across data centers based on transfer learning is characterized in that: The S9 comprises the following steps: S91. After completing the local deep learning model training in each heterogeneous data center, obtain a local deep learning model parameter set of each heterogeneous data center; S92, based on the dynamic participation weight W of each heterogeneous data center j , model convergence speed S j And the local model performance index E j Calculate the weighting coefficient α for each data center j : Among them, W j is the dynamic participation weight of heterogeneous data centers, and the local model convergence speed S j Defined as: in, It represents the decrease of the loss function of heterogeneous data centers from the initial to the current value. and are the initial and Tth j The loss function value at the iteration, μ1 is the convergence speed adjustment coefficient, and the local model performance index E j Defined as: Among them, Acc j The accuracy of the local model in the heterogeneous data center on the validation set; S93. Based on the local deep learning model parameter sets and corresponding weighted coefficients of each heterogeneous data center, the uncertainty of the local model parameters is integrated to generate a global optimization model parameter set through weighted fusion: I global ={W global ,b global }; Among them, Σ j is the covariance matrix of the local model parameters in the heterogeneous data center, (Σ j ) -1 is the inverse matrix of the covariance matrix; S94, the generated global optimization model parameter set Θ global Deployed to various heterogeneous data centers, after the global optimization model is deployed, monitor the task execution of each heterogeneous data center, record the performance indicators of the global model in each data center, and if new data distribution changes or system load fluctuations are detected in subsequent tasks, trigger the model retraining mechanism to re-train the local model and obtain the local deep learning model parameter set, calculate the weighted coefficients again and fuse them to generate an updated global optimization model parameter set.

Images