Federal learning method and system for cloud network integration video conference system

Abstract

The invention discloses a federated learning method and system for a cloud network fusion video conference system, and the method comprises the steps: 1, initializing a global model on a cloud node composed of a video conference server of the video conference system, and distributing the global model to all terminal nodes of the video conference system; step 2, the terminal node receives the issued global model and initializes the local model; step 3, training the local model by the terminal node based on the local data, and taking the sum of RJMMD loss and classification loss as total loss during training; step 4, the terminal node sends the trained local model parameters to the cloud node; 5, aggregating all the received local model parameters by the cloud node, and updating the global model parameters; and 6, the cloud node judges whether the global model converges or not, if yes, the process is ended, and if not, global model parameters are distributed to all the terminal nodes, and the step 2 is executed again. The method has the advantages of few communication rounds and high efficiency.

Description

Technical Field

[0001] The present invention relates to video conferencing technology, and in particular to a federated learning method and system for a cloud-network integrated video conferencing system. Background Art

[0002] Video conferencing system is an indispensable technology in modern office, which can help people communicate and collaborate efficiently between different locations. Video conferencing system includes multi-point controller MCU (video conferencing server), conference room terminal, PC desktop terminal, telephone access gateway (PSTNGateway), Gatekeeper and other parts. Various terminals are connected to MCU for centralized exchange to form a video conferencing network. MCU is responsible for receiving video, audio and data streams from multiple terminals, mixing them at once, and then sending the mixed streams to other terminals. MCU is also responsible for managing the connection of the meeting and coordinating the participants of the meeting.

[0003] With the gradual development of cloud computing technology, new enterprise cloud communication networking and applications based on cloud computing architecture will become the development direction of enterprise cloud communication systems in the future. The application of cloud computing technology in enterprise cloud communication industry networking is to cloudify and port CloudMCU, an important media exchange processing platform in enterprise cloud communication systems, so that multiple CloudMCU devices form an MCU "resource pool", so that there is no need to consider which MCU port resources to use in conference formation. The system meets the normal use of each meeting through intelligent resource scheduling and port allocation principles.

[0004] Federated learning is a distributed machine learning technology or machine learning framework. Its goal is to achieve joint modeling and improve the effectiveness of AI models on the basis of ensuring data privacy, security, and legal compliance. The biggest advantage of federated learning is the protection of user privacy data. What is transmitted to the cloud is the parameters of the terminal model, not the terminal data information. The existing federated learning methods require more rounds of communication, and the efficiency needs to be further optimized. Summary of the invention

[0005] Purpose of the invention: In view of the problems existing in the prior art, the present invention provides a federated learning method and system for a cloud-network integrated video conferencing system with fewer communication rounds and higher efficiency.

[0006] Technical solution: The present invention describes the following technical solution:

[0007] A federated learning method for a cloud-network integrated video conferencing system includes the following steps:

[0008] Step 1: Initialize a global model on the cloud nodes consisting of the video conferencing servers of the video conferencing system and distribute it to all terminal nodes of the video conferencing system;

[0009] Step 2: The terminal node receives the global model and initializes the local model according to the parameters of the global model.

[0010] Step 3: The terminal node trains the local model based on local data. During training, the loss function is calculated using the following formula to update the parameters:

[0011] L(Θ L ∣Θ G )=L cls +L RJMMD

[0012] L RJMMD =λ 2 E[K(θ G (X t ),θ G (X t ))]+E[K(θ L (X t ),θ L (X t ))]-2λE[K(θ G (X t ),θ L (X t ))]

[0013] In the formula, L(Θ L ∣Θ G ) represents the total loss, Θ G represents the global model parameters, Θ L represents the local model parameters, L cls represents the standard classification loss, L RJMMD represents the weighted maximum mean difference (Reweighted Joint Max meandiscrepancy, RJMMD) loss, θ G (X t ) and θ L (X t ) represent the global model and local model input as X t The output when X t represents the training sample data set, specifically local data; K() represents the kernel function, λ represents the weighting parameter, and E[] represents the average value;

[0014] Step 4: The terminal node sends the trained local model parameters to the cloud node;

[0015] Step 5: The cloud node aggregates all received local model parameters and uses the aggregated parameters as updated global model parameters;

[0016] Step 6: The cloud node determines whether the global model has converged. If so, the process ends. If not, the updated global model parameters are distributed to all terminal nodes, and the process returns to step 2.

[0017] Furthermore, in step 1, the general ALEXNET is used as the global model initial value.

[0018] Furthermore, in step 3, the standard classification loss is specifically:

[0019]

[0020] In the formula, represents the training sample dataset, Indicates that X t The corresponding label, represents the i-th training sample, represents the label of the i-th training sample, n t represents the number of training samples in the training sample data set, J() represents the cross entropy loss, Represents local model input Output when .

[0021] Furthermore, the step 3 adopts the gradient descent method to train the local model.

[0022] Furthermore, in step 5, the cloud node performs weighted averaging on all received local model parameters and aggregates them into updated global model parameters:

[0023]

[0024] In the formula, represents the updated global model parameters at the r+1th round, represents the local model parameters of the tth terminal node received by the cloud node in the r+1th round, w L,t represents the weight of the local model parameter of the t-th terminal node, S r A collection of endpoints.

[0025] A federated learning system for a cloud-network integrated video conferencing system, comprising a cloud node formed by a video conferencing server of the video conferencing system and a plurality of terminal nodes wirelessly connected to the cloud node;

[0026] The cloud node is used to initialize a global model at the initial moment and distribute it to all terminal nodes of the video conferencing system; and aggregate all received local model parameters and use the aggregated parameters as updated global model parameters; and determine whether the global model converges. If so, the process ends; if not, the updated global model parameters are distributed to all terminal nodes;

[0027] The terminal node is used to receive the global model sent by the cloud node and initialize the local model according to the parameters of the global model; train the local model based on local data, use the following formula to calculate the loss function to update the parameters during training, and send the trained local model parameters to the cloud node:

[0028] L(Θ L ∣Θ G )=L cls +L RJMMD

[0029] L RJMMD =λ 2 E[K(θ G (X t ),θ G (X t ))]+E[K(θ L (X t ),θ L (X t ))]-2λE[K(θ G (X t ),θ L (X t ))]

[0030] In the formula, L(Θ L ∣Θ G ) represents the total loss, Θ G represents the global model parameters, Θ L represents the local model parameters, L cls represents the standard classification loss, L RJMMD represents the RJMMD loss, θ G (X t ) and θ L (X t ) represent the global model and local model input as X t The output when X t represents the training sample data set, specifically local data; K() represents the kernel function, λ represents the weighting parameter, and E[] represents the average value.

[0031] Furthermore, the general ALEXNET is used as the global model initialization value.

[0032] Furthermore, the standard classification loss is specifically:

[0033]

[0034] In the formula, represents the training sample dataset, Indicates that X t The corresponding label, represents the i-th training sample, represents the label of the i-th training sample, n t represents the number of training samples in the training sample data set, J() represents the cross entropy loss, Represents local model input Output when .

[0035] Furthermore, the gradient descent method is used to train the local model.

[0036] Furthermore, the cloud node performs weighted averaging on all received local model parameters and aggregates them into updated global model parameters:

[0037]

[0038] In the formula, represents the updated global model parameters at the r+1th round, represents the local model parameters of the tth terminal node received by the cloud node in the r+1th round, w L,t represents the weight of the local model parameter of the t-th terminal node, S r A collection of endpoints.

[0039] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: the present invention uses the weighted maximum mean difference RJMMD function to quantify the difference in the joint distribution of the two models, and can dynamically adjust the degree of differentiation according to the performance requirements of model training, thereby achieving a balance between model performance and training overhead. Therefore, on the one hand, by minimizing the RJMMD loss term between the global model and the local model output, the local model can be forced to obtain the representation of the current client data; on the other hand, by adjusting the distribution weights and dynamically approximating the output of the global model according to the RJMMD loss, the number of communication rounds can be reduced, and the convergence of the training process can be accelerated, thereby limiting the training overhead and improving efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flowchart of a federated learning method for a cloud-network integrated video conferencing system provided by an embodiment of the present invention;

[0041] Figure 2 This is the architecture diagram of the cloud-network integrated video conferencing system;

[0042] Figure 3 It is the training flowchart of the local model. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0044] Embodiment 1

[0045] The embodiment of the present invention provides a federated learning method for a cloud-network integrated video conferencing system. Figure 1 The architecture of the cloud-network integrated video conferencing system to which the present invention is applicable is as follows: Figure 2 As shown, the video conferencing server constitutes a cloud node, and all conference room terminals (such as cameras, projectors), computer monitors, mobile phones and other terminals serve as terminal nodes.

[0046] like Figure 1 As shown, the method of the embodiment of the present invention includes the following steps:

[0047] Step 1: Initialize a global model on the cloud node and distribute it to all terminal nodes of the video conferencing system.

[0048] The global model is a model that needs to be optimized. For example, when resource scheduling needs to be optimized, the global model can be a resource scheduling optimization model. When bandwidth allocation optimization needs to be optimized, the global model can be a bandwidth allocation optimization model.

[0049] In specific implementation, the general ALEXNET can be used as the global model initialization value. The model consists of 5 convolutional layers and 3 fully connected layers, such as Figure 3 shown.

[0050] Step 2: The terminal node receives the global model sent down and initializes the local model according to the parameters of the global model.

[0051] After receiving the global model, the terminal node uses the parameters of the global model as the initial parameters of the local model.

[0052] Step 3: The terminal node trains the local model based on local data. During training, the loss function is calculated using the following formula to update the parameters:

[0053] L(Θ L ∣Θ G )=L cls +L RJMMD

[0054] L(Θ L ∣Θ G ) represents the total loss, ΘG represents the global model parameters, Θ L represents the local model parameters, L cls represents the standard classification loss, L RJMMD Represents RJMMD loss, and the RJMMD calculation formula is as follows:

[0055] Given two data distributions x and y, the MMD reweighted square between them can be expressed as:

[0056] RJM 2 (x,y)=‖λE[φ(x)]-E[φ(y)]‖ 2

[0057] λ is a dynamically adjustable weighting parameter, and its actual value can be selected between 0 and 1 as needed. φ(x) represents the mapping of data distribution to the reproducing kernel Hilbert space (RKHS). In fact, this mapping relationship is unknown. However, the kernel technique can be used to replace the inner product to obtain:

[0058] RJMMD2(x,y)=λ 2 E[K(x,x)]+E[K(y,y)]-2λE[K(x,y)]

[0059] Where K(x,y)=<φ(x),φ(y)> represents the kernel function that can maximize the task performance. Corresponding to the RJMMD loss, it is:

[0060] L RJMMD =λ 2 E[K(θ G (X t ),θ G (X t ))]+E[K(θ L (X t ),θ L (X t ))]-2λE[K(θ G (X t ),θ L (X t ))]

[0061] In the formula, θ G (X t ) and θ L (X t ) represent the global model and local model input as X t The output when X t represents the training sample data set, specifically local data; λ represents the weighting parameter, and E[] represents the average value.

[0062] The standard classification loss is specifically:

[0063]

[0064] In the formula, represents the training sample dataset, Indicates that X t The corresponding label, represents the i-th training sample, represents the label of the i-th training sample, n t represents the number of training samples in the training sample data set, J() represents the cross entropy loss, Represents local model input Output when .

[0065] The terminal node uses the RJMMD constraint to calculate the weighted difference between the local model and the global model, and uses gradient descent to update the local model so that it is closer to the global model while limiting the model training overhead.

[0066] Step 4: The terminal node sends the trained local model parameters to the cloud node.

[0067] Step 5: The cloud node aggregates all received local model parameters and uses the aggregated parameters as updated global model parameters.

[0068] In specific implementation, the cloud node performs weighted averaging on all received local model parameters and aggregates them into updated global model parameters:

[0069]

[0070] In the formula, represents the updated global model parameters at the r+1th round, represents the local model parameters of the tth terminal node received by the cloud node in the r+1th round, w L,t represents the weight of the local model parameter of the t-th terminal node, S r A collection of endpoints.

[0071] Step 6: The cloud node determines whether the global model has converged. If so, the process ends. If not, the updated global model parameters are distributed to all terminal nodes, and the process returns to step 2.

[0072] Embodiment 2

[0073] An embodiment of the present invention provides a federated learning system for a cloud-network integrated video conferencing system, including a cloud node composed of a video conferencing server of the video conferencing system and a plurality of terminal nodes wirelessly connected to the cloud node.

[0074] The cloud node is used to initialize a global model at the initial moment and distribute it to all terminal nodes of the video conferencing system; aggregate all received local model parameters and use the aggregated parameters as updated global model parameters; and determine whether the global model converges. If so, the process ends; if not, the updated global model parameters are distributed to all terminal nodes.

[0075] During specific implementation, the general ALEXNET can be used as the global model initial value.

[0076] The terminal node is used to receive the global model sent by the cloud node and initialize the local model according to the parameters of the global model; train the local model based on local data, use the following formula to calculate the loss function to update the parameters during training, and send the trained local model parameters to the cloud node:

[0077] L(Θ L ∣Θ G )=L cls +L RJMMD

[0078] L RJMMD =λ 2 E[K(θ G (X t ),θ G (X t ))]+E[K(θ L (X t ),θ L (X t ))]-2λE[K(θ G (X t ),θ L (X t ))]

[0079] In the formula, L(Θ L ∣Θ G ,X t ,Y t ) represents the total loss, Θ G represents the global model parameters, Θ L represents the local model parameters, L cls represents the standard classification loss, L RJMMD represents the RJMMD loss, θ G (X t ) and θ L (X t ) represent the global model and local model input as X t The output when X t represents the training sample data set, specifically local data; K() represents the kernel function, λ represents the weighting parameter, and E[] represents the average value;

[0080] The standard classification loss is specifically:

[0081]

[0082] In the formula, represents the training sample dataset, Indicates that X t The corresponding label, represents the i-th training sample, represents the label of the i-th training sample, n t represents the number of training samples in the training sample data set, J() represents the cross entropy loss, Represents local model input Output when .

[0083] In specific implementation, the gradient descent method can be used to train the local model

[0084] In specific implementation, the cloud node performs weighted averaging on all received local model parameters and aggregates them into updated global model parameters:

[0085]

[0086] In the formula, represents the updated global model parameters at the r+1th round, represents the local model parameters of the tth terminal node received by the cloud node in the r+1th round, w L,t represents the weight of the local model parameter of the t-th terminal node, S r A collection of endpoints.

[0087] It should be understood that the above embodiments and descriptions only describe the principles, main features and advantages of the present invention. Without departing from the spirit and scope of the present invention, the present invention may be subject to various changes and improvements, and these changes and improvements all fall within the scope of protection of the present invention.

Claims

1. A federated learning method for a cloud-network integrated video conferencing system, characterized in that: The steps include: Step 1: Initialize a global model on the cloud nodes consisting of the video conferencing servers of the video conferencing system and distribute it to all terminal nodes of the video conferencing system; Step 2: The terminal node receives the global model and initializes the local model according to the parameters of the global model. Step 3: The terminal node trains the local model based on local data. During training, the loss function is calculated using the following formula to update the parameters: L(Θ L ∣Θ G )=L cls +L RJMMD L RJMMD =λ 2 E[K(θ G (X t ),θ G (X t ))]+E[K(θ L (X t ),θ L (X t ))]-2λE[K(θ G (X t ),θ L (X t ))] In the formula, L(Θ L ∣Θ G ) represents the total loss, Θ G represents the global model parameters, Θ L represents the local model parameters, L cls represents the standard classification loss, L RJMMD represents the RJMMD loss, θ G (X t ) and θ L (X t ) represent the global model and local model input as X t The output when X t represents the training sample data set, specifically local data; K() represents the kernel function, λ represents the weighting parameter, and E[] represents the average value; Step 4: The terminal node sends the trained local model parameters to the cloud node; Step 5: The cloud node aggregates all received local model parameters and uses the aggregated parameters as updated global model parameters; Step 6: The cloud node determines whether the global model has converged. If so, the process ends. If not, the updated global model parameters are distributed to all terminal nodes, and the process returns to step 2.

2. The federated learning method for a cloud-network integrated video conferencing system according to claim 1, characterized in that: In step 1, the general ALEXNET is used as the global model initial value.

3. The federated learning method for a cloud-network integrated video conferencing system according to claim 1, characterized in that: In step 3, the standard classification loss is specifically: In the formula, represents the training sample dataset, Indicates that X t The corresponding label, represents the i-th training sample, represents the label of the i-th training sample, n t represents the number of training samples in the training sample data set, J() represents the cross entropy loss, Represents local model input Output when .

4. The federated learning method for a cloud-network integrated video conferencing system according to claim 1, characterized in that: The step 3 uses the gradient descent method to train the local model.

5. The federated learning method for a cloud-network integrated video conferencing system according to claim 1, characterized in that: In step 5, the cloud node performs weighted averaging on all received local model parameters and aggregates them into updated global model parameters: In the formula, represents the updated global model parameters at the r+1th round, represents the local model parameters of the tth terminal node received by the cloud node in the r+1th round, w L,t represents the weight of the local model parameter of the t-th terminal node, S r A collection of endpoints.

6. A federated learning system for cloud-network integrated video conferencing system, characterized in that: It includes a cloud node composed of a video conference server of a video conference system and a plurality of terminal nodes wirelessly connected to the cloud node; The cloud node is used to initialize a global model at the initial moment and distribute it to all terminal nodes of the video conferencing system; and aggregate all received local model parameters and use the aggregated parameters as updated global model parameters; and determine whether the global model converges. If so, the process ends; if not, the updated global model parameters are distributed to all terminal nodes; The terminal node is used to receive the global model sent by the cloud node and initialize the local model according to the parameters of the global model; train the local model based on local data, use the following formula to calculate the loss function to update the parameters during training, and send the trained local model parameters to the cloud node: L(Θ L ∣Θ G )=L cls +L RJMMD L RJMMD =λ 2 E[K(θ G (X t ),θ G (X t ))]+E[K(θ L (X t ),θ L (X t ))]-2λE[K(θ G (X t ),θ L (X t ))] In the formula, L(Θ L ∣Θ G ) represents the total loss, Θ G represents the global model parameters, Θ L represents the local model parameters, L cls represents the standard classification loss, L RJMMD represents the RJMMD loss, θ G (X t ) and θ L (X t ) represent the global model and local model input as X t The output when X t represents the training sample data set, specifically local data; K() represents the kernel function, λ represents the weighting parameter, and E[] represents the average value.

7. The federated learning system for cloud-network integrated video conferencing system according to claim 6, characterized in that: Use the generic ALEXNET as the global model initializer.

8. The federated learning method for a cloud-network integrated video conferencing system according to claim 6, characterized in that: The standard classification loss is specifically: In the formula, represents the training sample dataset, Indicates that X t The corresponding label, represents the i-th training sample, represents the label of the i-th training sample, n t represents the number of training samples in the training sample data set, J() represents the cross entropy loss, Represents local model input Output when .

9. The federated learning system for cloud-network integrated video conferencing system according to claim 6, characterized in that: The gradient descent method is used to train the local model.

10. The federated learning system for cloud-network integrated video conferencing system according to claim 6, characterized in that: The cloud node performs a weighted average of all received local model parameters and aggregates them into updated global model parameters: In the formula, represents the updated global model parameters at the r+1th round, represents the local model parameters of the tth terminal node received by the cloud node in the r+1th round, w L,t represents the weight of the local model parameter of the t-th terminal node, S r A collection of endpoints.

Images