Resource dynamic coordination method and system for data training

Abstract

The application discloses a resource dynamic coordination method and system for data training. The method is used for processing industrial data and comprises the following steps: acquiring industrial data samples, performing data shunting according to the characteristics of the industrial data samples to obtain user data and environment data, and selecting a federal learning mode and initializing a data training model according to the characteristics of the industrial data samples; and training the user data and the environment data until a global model reaches convergence. The application effectively deals with the high dynamic and personalized requirements of user data and the massive and miscellaneous characteristics of environment data, and jointly meets the requirements of industrial learning terminal data transmission and training efficiency and the like through dynamic coordination scheduling of network resources.

Description

Technical Field

[0001] This invention belongs to the field of industrial internet data processing technology, and in particular relates to a method and system for dynamic resource collaboration in data training. Background Technology

[0002] As smart cities become increasingly diversified in their application areas, the market size of the Industrial Internet is gradually expanding. The Industrial Internet requires the Industrial Internet of Things to be further extended to information systems, business processes, and personnel under different application entities. The Cyber-Physical-Social Convergence System Protocol can realize the collaborative perception, collaborative analysis, and collaborative services of the information, physical, and social elements in smart cities.

[0003] Compared to the traditional Internet, the Industrial Internet for smart cities needs to address the efficiency and security issues of multi-source data analysis while achieving ubiquitous broadband interconnection and comprehensive and thorough perception, in order to achieve the goal of data fusion for various smart applications in the city.

[0004] Currently, the construction of the industrial internet in smart cities faces the following problems:

[0005] (1) Data management and resource allocation issues. The centralized federated learning mechanism can enhance the efficiency of smart city network construction and industrial internet operation, but it also needs to consider the management, screening and analysis of training data of each node in the network, the allocation and maintenance of various network communication, computing and cache, and other aspects. While considering various network performance indicators, it is necessary to achieve joint optimization of data management and resource allocation under this mechanism.

[0006] (2) Decision-making dynamics and data backlog issues. On the one hand, the highly dynamic nature of users in the Industrial Internet requires ensuring the timeliness of decision-making results during the cross-system network design process. On the other hand, the wide-area application characteristics of smart cities make it easy for user data, environmental data, training data, etc., to accumulate at various distributed learning terminals, requiring the stability of the backlog amount during the cross-system network design process. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of existing technologies by providing a dynamic resource coordination method and system for data training. This system aims to solve the technical problems of data management and resource allocation, dynamic decision-making, and data backlog in the industrial internet. It can effectively address the highly dynamic and personalized needs of user data as well as the massive and redundant characteristics of environmental data. Through dynamic collaborative scheduling of network resources, it can jointly meet the needs of industrial learning terminals for data transmission and training efficiency.

[0008] The objective of this invention is achieved through the following technical solution:

[0009] A method for dynamic resource coordination based on data training, characterized in that the method is used to process industrial data, and the method includes:

[0010] Acquire industrial data samples, and perform data splitting based on the characteristics of the industrial data samples to obtain user data and environmental data. At the same time, select the federated learning mode based on the characteristics of the industrial data samples and initialize the data training model.

[0011] The user data and the environment data are used to train the global model until it converges.

[0012] Specifically, for the user data, data quality is quantified and training costs are evaluated. Network resources are coordinated and scheduled according to the performance indicators required for each training iteration. Model parameters are sent to each terminal based on resource scheduling. Feedback results from each terminal after completing local model training and updating are received. Global model aggregation and updating are performed based on the feedback results until the global model converges.

[0013] For the aforementioned environmental data, an environmental data queue and a training model queue are loaded for each terminal. The training results output from the training model queues of each terminal are received, and the global model is updated based on the training results until the global model converges.

[0014] Furthermore, the step of selecting the federated learning mode based on the characteristics of the industrial data samples and initializing the data training model specifically includes:

[0015] For data with less than a preset threshold of sample overlap and more than a preset threshold of feature overlap, a horizontal federated learning mode is used; for data with more than a preset threshold of sample overlap and less than a preset threshold of feature overlap, a vertical federated learning mode is used; and for data with less than a preset threshold of sample overlap and less than a preset threshold of feature overlap, a transfer federated learning mechanism is used.

[0016] The central server generates data to train the model based on the selected federated learning mode and issues federated learning tasks. The central server needs to select training collaborators from T intelligent learning terminals. The selected learning terminal maintains a local dataset D t And initialize the locally trained model w t The central server initializes the global training model. Among them, T * Let T be the set of intelligent learning terminals, representing the selected collaborators. This is the update formula for the global training model parameters of the central server during the i-th training iteration.

[0017] Furthermore, the quantified data quality specifically includes:

[0018] Define the data quality function F t (s t )=1-ηexp(-μs t ), where s t This represents the amount of data provided by the learning terminal t, where η and μ are curve fitting parameters, and the data quality function is updated to a data quality function χ based on terminal competitiveness. t (s t ) = F t (s t ) / ∑ t∈T F t (s t ).

[0019] Furthermore, the training cost includes data cost, communication cost, computation cost, and caching cost, and the evaluation of training cost specifically includes:

[0020] Data cost is defined as:

[0021]

[0022] in, This indicates the unit price of data, expressed as unit-price / bits;

[0023] Communication cost is defined as:

[0024]

[0025] Wherein, the communication rate is R t Each learning terminal is allocated a bandwidth of b. t The required communication power is Where h t =|g t | 2 / δ0 is the normalized channel power gain, g t The signal gain from the learning terminal t to the central server is represented by δ0, where δ0 is the one-sided noise power spectral density. This indicates the unit price per unit of power consumption, expressed as unit-price / J. t Let be the size of the training result data at terminal t;

[0026] The computational cost is defined as:

[0027]

[0028] s t This indicates the amount of data used for model training. This indicates the calculation of unit price, with units expressed as unit-price / (bits×epochs), κ g and κ lThese represent the global and local iteration counts, respectively, and θ is the size of the trained model.

[0029] Cache cost is defined as:

[0030]

[0031] in, This represents the unit price of data, expressed in unit-price / bits, z t ρ represents the size of the cached data. t This indicates the probability that the cached data will be retrieved in the future.

[0032] Furthermore, the step of collaboratively scheduling network resources according to the performance metrics required for each training iteration specifically includes:

[0033] Scheduling scheme OP1 is defined as follows:

[0034] OP1:

[0035] stC1:

[0036] C2:

[0037] C3:

[0038] C4:

[0039] C5:

[0040] C6:

[0041] C7:

[0042] C8:

[0043] Among them, T * Let s represent the set of learning terminals. t b represents the amount of user data. t Indicates the allocated bandwidth, z t The vectors represent the local cache pool usage, vector s represents the solution set of the selected learning terminal user data volume, vector b represents the solution set of the selected learning terminal allocated bandwidth, and vector z represents the solution set of the selected learning terminal's local cache status; T * Each of s, b, and z is responsible for a specific area of ​​optimization, and they are coupled together through the objective function and constraints.

[0044] The utility function is defined as:

[0045]

[0046] in, With χ t (s t ) replace F t (s t The details are as follows:

[0047]

[0048] The constraints are as follows: C1 specifies that the learning terminal for this task is selected from the terminal set T; C2 specifies the level of user data provided by each learning terminal, with the upper bound of the provided data volume not exceeding s. max The lower bound is no less than s1; C3 limits the lower bound s of the amount of user data provided by all learning terminals. th C4 limits the data quality provided by each learning terminal to exceed a threshold, with the data quality threshold value being F. th C5 limits the bandwidth allocation level for each learning terminal, with the upper limit of the provided bandwidth not exceeding b. max The lower bound is not lower than b1; C6 stipulates that the bandwidth that the network can provide to the learning terminal for uploading training results at the current moment is limited, with a total bandwidth of b. th C7 stipulates that for each learning terminal making the call, the transmission power used, under the optimized frequency band allocation value, cannot exceed the upper bound p. th C8 limits the level of caching. After each training session, the cached content includes the training dataset, the training result set, and the training model parameter set. The caching level depends on the probability that the data will be retrieved in the future.

[0049] Furthermore, the step of sending model parameters to each terminal based on resource scheduling, receiving feedback results from each terminal after completing local model training and updating, and performing global model aggregation and updating based on the feedback results until the global model converges specifically includes:

[0050] Training model parameters Make the loss function L(w) t Minimize, i.e. Where [i] represents the i-th local training of learning terminal t;

[0051] Minimize the global loss function

[0052] The local loss function at each learning terminal is Among them, the local dataset D of the learning terminal t t It can be derived from the n-dimensional eigenvector x t={x1,...,x n} represents an n-dimensional eigenvector x t ={x1,...,x n The standard basis for each element is y. t ={y1,...,y n}, This indicates that through model w t Based on feature vector x t The obtained prediction results, for the i-th global model aggregation, are obtained by minimizing the global loss function.

[0053] Furthermore, the loading of environment data queues and training model queues for each terminal specifically includes:

[0054] The environmental data at each learning terminal is divided into equal data blocks, each block being s0 bits in size. At time i, assume that at learning terminal t, there is... If a data block arrives, then the length of the newly added environmental data queue at learning terminal t is . Environmental data queue is

[0055]

[0056] Where, d t (i) represents the amount of environmental data that leaves the queue at time i;

[0057] The model parameters trained by each learning terminal are also divided into equal data blocks, each block being s1 bits in size. At time i, assuming the learning terminal at position t has... If a new data block is generated, then the length of the newly added training model queue at the learning terminal t is . The training model queue is as follows:

[0058]

[0059] Among them, b t (i) represents the amount of training data that leaves the queue at time i;

[0060] The amount of environmental data processed by terminal i at time i learning terminal t is Where τ represents the time slot length, f t (i) represents the computation frequency called by the learning terminal t at time i, l t This represents the computational sensitivity of the learning terminal t;

[0061] The training latency of learning terminal t at time i depends on its queue length.

[0062] The corresponding energy consumption is Among them, κ t is a constant, representing the power coefficient.

[0063] Furthermore, the step of receiving the training results output from the training model queues of each terminal, and updating the global model based on the training results until the global model converges, specifically includes:

[0064] The transmission rate of learning terminal t at time i on resource block c is Where B represents the bandwidth of resource block c, The signal-to-interference-plus-noise ratio (SIR) when the central server receives the training model parameters from the learning terminal t.

[0065] The total amount of data uploaded by learning terminal t at time i is Where C represents the set of frequency band resource blocks that can be provided for the learning terminal. Indicates the bandwidth allocation factor. This means that for time i, the frequency band resource block c is allocated to the learning terminal t, and vice versa.

[0066] The corresponding energy consumption is Where, p t Let t be the transmission power of the learning terminal.

[0067] Furthermore, the method allocates network resources in real time according to the OP2 resource scheduling scheme:

[0068] OP2:

[0069] stC1:

[0070] C2:

[0071] C3:

[0072] C4:

[0073] C5:

[0074] C6:

[0075] C7:

[0076] C8:

[0077] Indicates the arrival rate of environmental data. f represents the generation rate of the trained model. t(i) represents the amount of resource allocation to be calculated. This indicates the amount of frequency band resources allocated.

[0078] Minimize long-run energy consumption as the objective utility function is Where I = {1,2,...,t,...I} represents the training time length;

[0079] The constraints are as follows: C1 limits the number of environment data blocks and training data blocks generated by each learning terminal at each time step, with a maximum of q; C2 limits the upper and lower bounds of the computation rate invoked by each learning terminal at each time step (i.e., f). max and f min C3 limits the frequency band resource block allocation factor to a 0-1 variable; C4 and C5 require that each frequency band resource block be allocated to at most one learning terminal at any given time, and the total number of frequency band resource blocks that the network can provide is |C|; C6 sets a lower limit on the upload rate of the learning terminal at each time (i.e., The constraints were set; C7 and C8 are the maximum tolerable latency constraints for the environment data queue and the training model queue. and These are the maximum tolerable latency constraints for the environmental data queue and the training model queue, respectively.

[0080] On the other hand, the present invention also provides a resource dynamic coordination system for data training, the system including a control platform and a data platform, the communication system being used to execute any of the aforementioned data training resource dynamic coordination methods;

[0081] The control platform integrates a software-defined network controller for monitoring virtualization processes, sample and feature extraction, and centralized federated learning.

[0082] The data platform includes a virtual network operator, a central controller, and learning terminals. The central controller schedules network resources in real time according to the performance index requirements during the data training process. The learning terminals interact with the training parameters of the central controller to train user data and environmental data in real time, and store the training data and training results.

[0083] The beneficial effects of this invention are as follows:

[0084] (1) This invention uses a centralized federated learning mechanism to separate industrial user data and industrial environment data and conduct distributed training. While ensuring training quality, it effectively reduces training complexity and alleviates the network load and resource consumption pressure of the Industrial Internet.

[0085] (2) This invention coordinates industrial data and network resources, and through the evaluation of user data quality and training costs, designs a joint deployment scheme for communication, computing, caching and other resources, and real-time schedules network resources to meet the network resources required by intelligent learning devices during training, thereby maximizing the utility of centralized federated learning.

[0086] (3) This invention addresses both the dynamics of decision-making and the problem of data backlog. For training on massive and redundant industrial environment data, it loads an environment data queue and a training model queue for the intelligent learning terminal. Through the design of long-term optimization schemes for resources such as communication, computing, and caching, it achieves stable data outflow from the terminal and minimizes the energy consumption of centralized federated learning. Attached Figure Description

[0087] Figure 1 This is a schematic diagram of the resource dynamic collaboration method for data training provided in an embodiment of the present invention;

[0088] Figure 2 These are training data quality curves under different strategies according to embodiments of the present invention;

[0089] Figure 3 This is a data queue stability curve of the intelligent learning terminal under different strategies according to an embodiment of the present invention;

[0090] Figure 4 This is a diagram of a resource dynamic collaborative system model trained using data, provided in an embodiment of the present invention. Detailed Implementation

[0091] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0092] Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0093] Currently, the construction of the industrial internet in smart cities faces the following problems:

[0094] (1) Data management and resource allocation issues. The centralized federated learning mechanism can enhance the efficiency of smart city network construction and industrial internet operation, but it also needs to consider the management, screening and analysis of training data of each node in the network, the allocation and maintenance of various network communication, computing and cache, and other aspects. While considering various network performance indicators, it is necessary to achieve joint optimization of data management and resource allocation under this mechanism.

[0095] (2) Decision-making dynamics and data backlog issues. On the one hand, the highly dynamic nature of users in the Industrial Internet requires ensuring the timeliness of decision-making results during the cross-system network design process. On the other hand, the wide-area application characteristics of smart cities make it easy for user data, environmental data, training data, etc., to accumulate at various distributed learning terminals, requiring the stability of the backlog amount during the cross-system network design process.

[0096] To address the aforementioned technical problems, the following embodiments of the resource dynamic coordination method and system for data training of the present invention are proposed.

[0097] Example 1

[0098] Reference Figure 1 ,like Figure 1 The diagram shown is a schematic representation of the resource dynamic coordination method for data training provided in this embodiment. The method specifically includes:

[0099] S1. The intelligent learning terminal captures industrial sensor data and performs data splitting based on data samples and features, namely user data and environmental data. It selects an appropriate federated learning mode according to the data samples and features, and initializes the data training model at the same time.

[0100] Specifically, S1 includes the following sub-steps:

[0101] S11. The intelligent learning terminal captures industrial sensor data and performs data splitting based on data samples and features, namely user data and environmental data.

[0102] S12. The central server selects an appropriate federated learning mode based on the samples and features of the split data: horizontal federated learning mode is used for data with little sample overlap and much feature overlap, vertical federated learning mode is used for data with much sample overlap and little feature overlap, and transfer federated learning mechanism is used for data with little sample overlap and little feature overlap.

[0103] S13. The central server generates data to train the model based on the selected federated learning mode and issues the federated learning task. The central server needs to select training collaborators from T intelligent learning terminals. The selected learning terminal maintains a local dataset D t And initialize the locally trained model w tThe central server initializes the global training model. T * Let T be the set of intelligent learning terminals, representing the selected collaborators. This is the update formula for the global training model parameters of the central server during the i-th training iteration.

[0104] S2. For industrial user data, the central controller evaluates the data quality and training cost captured by each terminal. Based on the performance requirements of each training iteration, the central controller coordinates and schedules network resources. The central controller encrypts and distributes model parameters to each terminal based on the resource scheduling. Each terminal independently completes local model training and updates, and feeds back the update results to the central controller to perform global model aggregation and updates until the global model converges.

[0105] Specifically, S2 includes the following sub-steps:

[0106] S21. For industrial user data, the central controller evaluates the data quality and training cost captured by each terminal.

[0107] Data quality assessment follows two principles: first, the larger the amount of data used to train the local model, the better the quality of the resulting federated service; second, the marginal utility of the federated service decreases as the amount of data increases. Based on these two principles, the defined data quality function is a monotonically increasing concave function with respect to the amount of data, specifically:

[0108] F t (s t )=1-ηexp(-μs t ),

[0109] Among them, s t Let represent the amount of data provided by the learning terminal t, and η and μ be curve fitting parameters. To further incentivize the learning terminals to provide higher-quality data for federated learning, this data quality function can be updated to a data quality function based on terminal competitiveness, specifically as follows:

[0110] χ t (s t ) = F t (s t ) / ∑ t∈T F t (s t ).

[0111] This function can characterize the proportion of the data quality provided by learning terminal t in the total data quality provided by all candidate learning terminals.

[0112] In this embodiment, the data training cost includes data cost, communication cost, computing cost, and caching cost.

[0113] Data costs primarily stem from data acquisition costs; the data acquisition cost at learning terminal t is defined as...

[0114]

[0115] in, This indicates the unit price of the data, expressed as unit-price / bits.

[0116] The communication cost mainly stems from the power consumption during the process of uploading the locally trained model parameters from the learning terminal. Since the training results after each local learning iteration can be considered known quantities, a constant-rate transmission mode can be adopted, assuming the communication rate is R. t Each learning terminal is allocated a bandwidth of b. t According to Shannon's formula, the required communication power is... Where h t =|g t | 2 / δ0 is the normalized channel power gain, g t Let δt represent the signal gain from the learning terminal t to the central server, and δ0 be the one-sided noise power spectral density. Based on this, the communication cost at the learning terminal t can be obtained as follows:

[0117]

[0118] in, This indicates the unit price per unit of power consumption, expressed as unit-price / J. t Let be the size of the training result data at terminal t.

[0119] The computational cost primarily stems from the computational resource consumption of the learning terminal during local training. The local computational load of the terminal depends on the amount of data (s) used for model training. t Therefore, the computational cost at the learning terminal t is defined as

[0120]

[0121] in, This indicates the calculation of unit price, with units expressed as unit-price / (bits×epochs), κ g and κ l These represent the global and local iteration counts, respectively, and θ is the size of the training model.

[0122] Caching costs arise from the update of the local data pool by the learning terminal after each training session. The learning terminal can determine the size and even the type of data to cache based on its own cache space, specifically defined as...

[0123]

[0124] in, This represents the unit price of data, expressed in unit-price / bits, z t ρ represents the size of the cached data. t This represents the popularity value of the cached data, i.e., the probability that the data will be retrieved in the future, which can be described by the Zipf distribution function.

[0125] S22. The central controller coordinates the scheduling of network resources according to the performance requirements of each training iteration, as shown in OP1:

[0126] OP1:

[0127] stC1:

[0128] C2:

[0129] C3:

[0130] C4:

[0131] C5:

[0132] C6:

[0133] C7:

[0134] C8:

[0135] The constructed multi-resource joint optimization problem uses the selected set of learning terminals T * User data volume s t , allocated bandwidth b t And the usage of the local cache pool. t As an optimization variable.

[0136] The constructed multi-resource joint optimization problem aims to maximize system utility, and the utility function is defined as follows:

[0137]

[0138] in, To further stimulate competition in the learning terminal market, χ t (s t ) can be used to replace F t (s t The details are as follows:

[0139]

[0140] The constraints of the constructed multi-resource joint optimization problem are as follows: C1 restricts the learning terminals for this task to be selected from the terminal set T; C2 restricts the level of user data provided by each learning terminal, and the upper bound of the provided data volume does not exceed s. max The lower bound is no less than s1, which ensures that each learning terminal can provide sufficient user data for local training. The upper bound ensures that the amount of data is not too large, which would lead to an excessively long federated training cycle. C3 limits the lower bound s of the amount of user data provided by all learning terminals. th This constraint ensures that the user data provided by all learning terminals is sufficient and effective for the global training results at the central server; C4 limits the data quality provided by each calling learning terminal to be acceptable, with a data quality threshold of F. th C5 limits the bandwidth allocation level for each learning terminal, with the upper limit of the provided bandwidth not exceeding b. max The lower bound is not lower than b1; C6 stipulates that the bandwidth that the network can provide to the learning terminal for uploading training results at the current moment is limited, with a total bandwidth of b. th C7 states that for each learning terminal making the call, the transmission power used, under the optimized bandwidth allocation value, cannot exceed the upper bound p. th C8 limits the level of caching. After each training session, the cached content covers different types of data, such as the training dataset, training result set, and training model parameter set. The caching level depends on the likelihood that the data will be frequently accessed in the future.

[0141] To reduce the complexity and signaling overhead of problem solving, deep reinforcement learning is used to solve the optimization problem. To ensure that the constructed multi-dimensional resource joint optimization scheme can meet the dynamic requirements of resource scheduling during federated learning, an offline training and online optimization approach is adopted to quickly generate solutions to the optimization problem.

[0142] S23. The central controller encrypts and distributes model parameters to each terminal based on resource scheduling. The selected learning terminals maintain their local datasets D. t And train the local model w t The local model is then aggregated, integrated, and updated by the central server. And generate a global model w g .

[0143] For local model training on learning terminal t, the goal is to obtain the model parameters. Make the loss function L(w) t Minimize, i.e.

[0144]

[0145] Where [i] represents the i-th local training iteration of learning terminal t. For the central server, the goal is to minimize the global loss function.

[0146]

[0147] Suppose that the local dataset D of the learning terminal t t It can be derived from the n-dimensional eigenvector x t ={x1,...,x n} indicates that the standard basis for each element of this vector is y. t ={y1,...,y n}, This means that through model w t Based on feature vector x t The obtained prediction results. Then the local loss function at each learning terminal is:

[0148]

[0149] For the i-th global model aggregation, to minimize the global loss function, let

[0150]

[0151] S3. For industrial environment data, load environment data queue and training model queue for each learning terminal. Each terminal stably schedules the environment data queue for local training based on the global model parameters issued by the central controller, and stores the training results in the training model queue in an orderly manner. The training model queue stably outputs the training results to the central controller, and the central controller executes the update of the global model until the global model reaches convergence.

[0152] Specifically, S3 includes the following sub-steps:

[0153] S31. For industrial environment data, load the environment data queue and training model queue for each learning terminal.

[0154] The environmental data at each learning terminal is divided into equal data blocks, each block being s0 bits in size. At time i, assume that at learning terminal t... If a data block arrives, then the length of the newly added environmental data queue at learning terminal t is . Based on this, its environmental data queue can be further described as

[0155]

[0156] Where, d t(i) represents the amount of environmental data that leaves the queue at time i.

[0157] The model parameters trained by each learning terminal are also divided into equal data blocks, each block being s1 bits in size. At time i, assume that at learning terminal t, there are... If a new data block is generated, then the length of the newly added training model queue at the learning terminal t is . Based on this, its training model queue can be further described as follows:

[0158]

[0159] Among them, b t (i) represents the amount of training data that leaves the queue at time i.

[0160] S32. Each terminal performs local training on the stable scheduling environment data queue based on the global model parameters issued by the central controller, and stores the training results in the training model queue in an orderly manner.

[0161] For time i, the amount of environmental data processed by the learning terminal t can be calculated using the following formula.

[0162]

[0163] Where τ represents the time slot length (unit: s), f t (i) represents the computation frequency (unit: CPU cycles / s) of the learning terminal t at time i. t This represents the computational sensitivity of the learning terminal t (unit: CPU cycles / bit).

[0164] For time i, the training latency of learning terminal t depends on its queue length, i.e.

[0165]

[0166] The corresponding energy consumption can be expressed as

[0167]

[0168] Among them, κ t This is a constant representing the power factor, which is usually determined by the type of chip in the device.

[0169] S33. Each learning terminal controls the stable output of training results from the training model queue to the central controller, which then performs global model updates until the global model converges.

[0170] For time i, the transmission rate of learning terminal t on resource block c is:

[0171]

[0172] Where B represents the bandwidth of resource block c, Let $\frac{1}{2}$ be the signal-to-interference-plus-noise ratio (SIR) when the central server receives the training model parameters from the learning terminal $t$. Based on this, for time $i$, the total amount of data uploaded by the learning terminal $t$ is $\frac{1}{2}$.

[0173]

[0174] Where C represents the set of frequency band resource blocks that can be provided for the learning terminal. Indicates the bandwidth allocation factor. This means that for time i, the frequency band resource block c is allocated to the learning terminal t, and vice versa.

[0175] For time i, the transmission delay of the learning terminal t is

[0176]

[0177] The corresponding energy consumption can be expressed as

[0178]

[0179] Where, p t Let t be the transmission power of the learning terminal.

[0180] S34. To ensure the training efficiency of the models involved in S31 to S33, the central controller allocates network resources in real time according to the following resource scheduling scheme, as shown in OP2:

[0181] OP2:

[0182] stC1:

[0183] C2:

[0184] C3:

[0185] C4:

[0186] C5:

[0187] C6:

[0188] C7:

[0189] C8:

[0190] The constructed multi-resource joint optimization problem is based on the environmental data arrival rate. Training model generation rate Calculate the resource allocation amount f t (i) and frequency band resource allocation As an optimization variable.

[0191] The constructed multi-resource joint optimization problem takes minimizing long-term energy consumption as the objective utility function, i.e.

[0192]

[0193] Where I = {1,2,...,t,...I} represents the training time length.

[0194] The constraints of the constructed multi-resource joint optimization problem are as follows: C1 limits the number of environmental data blocks and training data blocks generated by each learning terminal at each time step, with a maximum of q; C2 limits the upper and lower bounds of the computation rate invoked by each learning terminal at each time step (i.e., f). max and f min C3 limits the frequency band resource block allocation factor to a 0-1 variable; C4 and C5 require that each frequency band resource block be allocated to at most one learning terminal at any given time, and the total number of frequency band resource blocks that the network can provide is |C|; C6 sets a lower limit on the upload rate of the learning terminal at each time (i.e., The constraints were set; C7 and C8 are the maximum tolerable latency constraints for the environment data queue and the training model queue. and These are the maximum tolerable latency constraints for the environmental data queue and the training model queue, respectively.

[0195] As one implementation method, in order to reduce the complexity of problem solving and signaling overhead, this embodiment uses the Lyapunov algorithm to transform the optimization problem, thereby splitting the original optimization problem into multiple discrete sub-optimization problems for solving.

[0196] Reference Figure 2 ,like Figure 2 The figure shown is a training data quality curve under different strategies in this embodiment. Figure 2 This demonstrates the improvement in data quality under data quality functions that prioritize terminal competitiveness versus those that do not. From Figure 2 As can be seen, both the data quality function with and without terminal competition awareness can effectively improve the training data quality of smart terminals compared to the baseline scheme (without considering data quality evaluation), and this improvement is more significant when the number of training data terminals increases.

[0197] Reference Figure 3,like Figure 3 The figure shown is a stability curve of the data queue of the intelligent learning terminal under different strategies in this embodiment. Figure 3 This demonstrates the queue length per timeslot under both long-term and instantaneous optimization schemes. From Figure 3 As can be seen, the long-term optimization scheme adopted in this embodiment can significantly improve the stability of the data queue compared with the traditional instantaneous optimization scheme.

[0198] The data training resource dynamic collaboration method provided in this embodiment, through the decentralization and distributed training of industrial user data and industrial environment data using a centralized federated learning mechanism, effectively reduces training complexity while ensuring training quality and alleviating the network load and resource consumption pressure of the Industrial Internet. This method jointly schedules industrial data and network resources, designs a joint deployment scheme for communication, computing, and caching resources based on the evaluation of user data quality and training costs, and schedules network resources in real time to meet the network resource requirements of intelligent learning devices during training, thereby maximizing the utility of centralized federated learning. This method also addresses the issues of dynamic decision-making and data backlog. For training with massive and redundant industrial environment data, it loads environmental data queues and training model queues onto the intelligent learning terminal. Through long-term optimization schemes for communication, computing, and caching resources, it achieves stable data outflow from the terminal and minimizes the energy consumption of centralized federated learning.

[0199] Example 2

[0200] Reference Figure 2 ,like Figure 2 The diagram shows a resource dynamic coordination system model for data training provided in this embodiment. This system is used to implement the resource dynamic coordination method for data training provided in the foregoing embodiments. Specifically, the system includes a control platform and a data platform.

[0201] The control platform integrates a software-defined network controller to monitor the virtualization process, sample and feature extraction, and centralized federated learning.

[0202] The data platform includes a virtual network operator, a central controller, and learning terminals. The central controller schedules network resources in real time according to the performance requirements during the data training process. The learning terminals interact with the training parameters of the central controller to train user data and environmental data in real time, and store the training data and training results.

[0203] As one implementation method, virtual network operators include virtual power grids, virtual medical networks, virtual transportation networks, and virtual home networks.

[0204] The above system can be applied to industrial internet systems or wireless communication systems, and can be combined with the methods of the aforementioned embodiments to achieve dynamic resource collaboration for industrial data training.

[0205] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for dynamic resource coordination in data training, characterized in that, The method is used to process industrial data, and the method includes: Acquire industrial data samples, and perform data splitting based on the characteristics of the industrial data samples to obtain user data and environmental data. At the same time, select the federated learning mode based on the characteristics of the industrial data samples and initialize the data training model. The user data and the environment data are used to train the global model until it converges. Specifically, for the user data, data quality is quantified and training costs are evaluated. Network resources are coordinated and scheduled according to the performance indicators required for each training iteration. Model parameters are sent to each terminal based on resource scheduling. Feedback results from each terminal after completing local model training and updating are received. Global model aggregation and updating are performed based on the feedback results until the global model converges. For the aforementioned environmental data, an environmental data queue and a training model queue are loaded for each terminal. The training results output from the training model queues of each terminal are received, and the global model is updated based on the training results until the global model converges. The quantification of data quality specifically includes: defining a data quality function. ,in, Indicates learning terminal The amount of data provided and The parameters are used for curve fitting, and the data quality function is updated to a data quality function based on terminal competitiveness. ; The coordinated scheduling of network resources based on the performance metrics required for each training iteration specifically includes: Scheduling scheme OP1 is defined as follows: in, Represents a set of learning terminals. Indicates the amount of user data. Indicates the allocated bandwidth. The vector s represents the local cache pool usage, the vector b represents the solution set of the selected learning terminal user data volume, the vector z represents the solution set of the selected learning terminal allocated bandwidth, and the vector z represents the solution set of the selected learning terminal local cache status. The utility function is defined as: in, ,by replace The details are as follows: The constraints are as follows: C1 specifies that the learning terminals for this task are from the terminal set. The selection is made within the range of C2; C2 limits the level of user data provided by each learning terminal, with the upper limit of the provided data volume not exceeding [a certain threshold]. The lower limit is not lower than C3 limits the lower bound of the amount of user data provided by all learning terminals. C4 limits the data quality provided by each learning terminal to exceed a threshold, which is set to [value missing]. C5 limits the bandwidth allocation level for each learning terminal, with the upper limit of the provided bandwidth not exceeding [a certain threshold]. The lower limit is not lower than C6 specifies that the bandwidth available to learning terminals for uploading training results at any given time is limited, with a total bandwidth of [missing information]. C7 stipulates that for each learning terminal making the call, the transmission power used, under the optimized frequency band allocation value, cannot exceed the upper bound. C8 limits the level of caching. After each training session, the cached content includes the training dataset, the training result set, and the training model parameter set. The caching level depends on the probability that the data will be retrieved in the future. The aforementioned loading environment data queue and training model queue for each terminal specifically include: The environmental data at each learning terminal is divided into equal data blocks, each block being [size missing]. bits, in the At this moment, assuming the learning terminal There are When a data block arrives, the learning terminal... The length of the newly added environmental data queue is The environmental data queue is ， in, Indicates time The amount of environmental data leaving this queue; The model parameters trained on each learning terminal are also divided into equal data blocks, each block being [size missing]. bits, in the At this moment, assuming the learning terminal There are If a new data block is generated, then the learning terminal will then... The length of the newly added training model queue is The training model queue is as follows: ， in, Indicates time The amount of training data that leaves the queue; Time of each terminal Learning terminal Environmental data processing volume is ,in, Indicates the time slot length. Indicates time Learning terminal The frequency of the call to the calculation, Indicates learning terminal The computational sensitivity; time Learning terminal The training latency depends on its queue length. ; The corresponding energy consumption is ,in, The constant represents the power coefficient; The step of receiving the training results output from the training model queues of each terminal and updating the global model based on the training results until the global model converges specifically includes: time Learning terminal In resource block The transmission rate on is ,in, Represents resource block bandwidth, For the central server to receive learning terminals The signal-to-interference-plus-noise ratio when training model parameters; time Learning terminal The total amount of data uploaded is ,in, This represents the set of frequency band resource blocks that can be provided to learning terminals. Indicates the bandwidth allocation factor. Represents the moment Frequency band resource blocks It was assigned to the learning terminal Conversely ; The corresponding energy consumption is ,in, For learning terminals The transmission power; The method allocates network resources in real time according to the OP2 resource scheduling scheme: ， Indicates the arrival rate of environmental data. Indicates the generation rate of the trained model. This indicates the amount of computational resources allocated. This indicates the amount of frequency band resources allocated. Minimize long-run energy consumption as the objective utility function is ,in, Indicates the length of training time; The constraints are as follows: C1 limits the number of environmental data blocks and training data blocks generated by each learning terminal at each time step, with a maximum not exceeding [a certain value]. C2 defines the upper and lower bounds of the computation rate invoked by each learning terminal at each time step (i.e., ...). and C3 specifies that the frequency band resource block allocation factor is a 0-1 variable; C4 and C5 require that each frequency band resource block can be allocated to at most one learning terminal at any given time, and that the total number of frequency band resource blocks that the network can provide is [missing information]. C6 sets the lower limit of the upload rate of the learning terminal at each time step (i.e., ... The constraints were set; C7 and C8 are the maximum tolerable latency constraints for the environment data queue and the training model queue. and These are the maximum tolerable latency constraints for the environmental data queue and the training model queue, respectively.

2. The resource dynamic coordination method for data training as described in claim 1, characterized in that, The step of selecting a federated learning mode based on the features of the industrial data samples and initializing the data training model specifically includes: For data with less than a preset threshold of sample overlap and more than a preset threshold of feature overlap, a horizontal federated learning mode is used; for data with more than a preset threshold of sample overlap and less than a preset threshold of feature overlap, a vertical federated learning mode is used; and for data with less than a preset threshold of sample overlap and less than a preset threshold of feature overlap, a transfer federated learning mechanism is used. The central server generates data to train the model based on the selected federated learning mode and issues federated learning tasks. The central server needs to select training collaborators from T intelligent learning terminals. The selected learning terminal maintains the local dataset. And initialize the locally trained model The central server initializes the global training model. ,in, For the selected collaborators, A collection of intelligent learning terminals. This is the update formula for the global training model parameters of the central server during the i-th training iteration.

3. The resource dynamic coordination method for data training as described in claim 1, characterized in that, The training cost includes data cost, communication cost, computation cost, and caching cost. The evaluation of the training cost specifically includes: Data cost is defined as: ， in, This indicates the unit price of data, expressed as unit-price / bits; Communication cost is defined as: ， Among them, the communication rate is The bandwidth allocated to each learning terminal is The required communication power is ,in For normalized channel power gain, Indicates learning terminal Signal gain to the central server For the one-sided noise power spectral density, This indicates the unit price per unit of power consumption, expressed as unit-price / J. For the terminal Size of training result data; The computational cost is defined as: ， This indicates the amount of data used for model training. This indicates the calculation of the unit price, expressed as unit-price / (bits) epochs), and These represent the global and local iteration counts, respectively. For training model size; Cache cost is defined as: ， in, This indicates the unit price of data, expressed as unit-price / bit. Indicates the size of the cached data. This indicates the probability that the cached data will be retrieved in the future.

4. The resource dynamic coordination method for data training as described in claim 1, characterized in that, The process of sending model parameters to each terminal based on resource scheduling, receiving feedback results from each terminal after completing local model training and updating, and performing global model aggregation and updating based on the feedback results until the global model converges specifically includes: Training model parameters Make the loss function Minimize, i.e. ,in Indicates learning terminal The Secondary local training; Minimize the global loss function ; The local loss function at each learning terminal is Among them, learning terminals Local dataset can be 3D feature vector express, 3D feature vector The standard basis corresponding to each element is: , Indicates through the model Based on feature vectors The obtained prediction results, for the first Sub-global model aggregation, to minimize the global loss function .

5. A dynamic resource collaboration system for data training, the system comprising a control platform and a data platform, characterized in that, The resource dynamic coordination system is used to execute the resource dynamic coordination method for data training as described in any one of claims 1-4; The control platform integrates a software-defined network controller for monitoring virtualization processes, sample and feature extraction, and centralized federated learning. The data platform includes a virtual network operator, a central controller, and learning terminals. The central controller schedules network resources in real time according to the performance index requirements during the data training process. The learning terminals interact with the training parameters of the central controller to train user data and environmental data in real time, and store the training data and training results.

Images