Communication methods and communication devices
By combining the posterior distribution of the interaction model between nodes and the mathematical combination of dual variables, the problem of not being able to analyze parameter importance and confidence in existing distributed training methods is solved, achieving more efficient model training results and reducing communication overhead.
Patent Information
- Application Number
- CN202510603915.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-09
- Publication Date
- 2026-11-10
AI Technical Summary
Existing distributed training methods cannot effectively analyze the importance of model parameters and the confidence of model output in communication systems, which affects the training effect of the model.
By mathematically combining the posterior distribution and dual variables of the interaction model between nodes, the number of transmitted parameters is reduced, communication overhead is lowered, and training effectiveness is improved.
This improved the training performance of the model, analyzed the importance of model parameters and output confidence, and reduced communication overhead.
Smart Images

Figure CN123070249A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and more specifically, to a communication method and a communication device. Background Technology
[0002] With the rapid development of artificial intelligence (AI) technology, AI models are being applied in increasingly wider scenarios. For example, AI models can be introduced into certain communication systems to achieve network intelligence.
[0003] To protect user privacy and data security, distributed training methods can be used to train AI models in communication systems. However, the training results of existing distributed training methods are not ideal. Summary of the Invention
[0004] This application provides a communication method and a communication device that can improve the training effect of the model.
[0005] In a first aspect, a communication method is provided, the method being applied to a first node, or to a component in the first node (e.g., a processor, chip, chip system, circuit, or a functional module, etc.), the method comprising:
[0006] Send first information to the second node, the first information being used to indicate a first posterior distribution and / or a first dual variable corresponding to the first posterior distribution, the first posterior distribution representing the posterior distribution of a first model obtained using the local dataset of the first node as evidence; receive second information from the second node, the second information being used to indicate a second posterior distribution, the second posterior distribution being determined based on the first posterior distribution and / or the first dual variable.
[0007] In this embodiment, first information indicating the first posterior distribution and / or the first dual variable is sent to the second node, and second information is received from the second node. The second information indicates the second posterior distribution, which is determined based on the first posterior distribution and / or the first dual variable. In this way, the posterior distribution of the interactive model (such as the first posterior distribution and / or the second posterior distribution) can be analyzed during the training process, thereby analyzing the importance of the parameters in the model and the confidence of the model output, and thus improving the training effect of the model.
[0008] In some possible implementations, the first information includes the parameters of the first posterior distribution and a mathematical combination of the first dual variable.
[0009] In this embodiment of the application, the first information includes a mathematical combination of the parameters of the first posterior distribution and the first dual variable. In this way, compared with directly sending the parameters of the first posterior distribution and the first dual variable, the number of parameters transmitted can be reduced, thereby reducing communication overhead.
[0010] In some possible implementations, the parameters of the first posterior distribution include a first mean and a first variance, the first dual variable includes the dual variable corresponding to the first mean and the dual variable corresponding to the first variance, and the mathematical combination includes a first weighted combination of the first mean and its corresponding dual variable, and a second weighted combination of the first variance and its corresponding dual variable.
[0011] In some possible implementations, the first weighted combination satisfies the following relation:
[0012]
[0013] in, This represents the first mean. Let C1 and C2 represent the dual variable corresponding to the first mean, C1 and C2 represent the weights, t represent the first node being the t-th node among multiple first nodes, and s represent the s-th round of local training.
[0014] In some possible implementations, the second weighted combination satisfies the following relation:
[0015]
[0016] in, This represents the first variance. Let represent the dual variable corresponding to the first variance, ρ represent the penalty coefficient, t represent the t-th node among multiple first nodes, and s represent the s-th round of local training.
[0017] In some possible implementations, before sending the first information to the second node, the method further includes: determining the first posterior distribution and the first dual variable based on the local dataset of the first node.
[0018] Secondly, a communication method is provided, the method being applied to a second node, or to a component in the second node (e.g., a processor, chip, chip system, circuit, or a functional module, etc.), the method comprising:
[0019] Receive first information from a first node, the first information being used to indicate a first posterior distribution and / or a first dual variable corresponding to the first posterior distribution, the first posterior distribution representing the posterior distribution of a first model obtained using the local dataset of the first node as evidence; send second information to the first node, the second information being used to indicate a second posterior distribution, the second posterior distribution being determined based on the first posterior distribution and / or the first dual variable.
[0020] In this embodiment, first information indicating a first posterior distribution and / or a first dual variable is received from a first node, and second information is sent to the first node. The second information indicates a second posterior distribution, which is determined based on the first posterior distribution and / or the first dual variable. In this way, the posterior distribution of the interactive model (such as the first posterior distribution and / or the second posterior distribution) can be analyzed during the training process, thereby analyzing the importance of the parameters in the model and the confidence of the model output, and thus improving the training effect of the model.
[0021] In some possible implementations, the first information includes the parameters of the first posterior distribution and a mathematical combination of the first dual variable.
[0022] In this embodiment of the application, the first information includes a mathematical combination of the parameters of the first posterior distribution and the first dual variable. In this way, compared with directly sending the parameters of the first posterior distribution and the first dual variable, the number of parameters transmitted can be reduced, thereby reducing communication overhead.
[0023] In some possible implementations, the parameters of the first posterior distribution include a first mean and a first variance, the first dual variable includes the dual variable corresponding to the first mean and the dual variable corresponding to the first variance, and the mathematical combination includes a first weighted combination of the first mean and its corresponding dual variable, and a second weighted combination of the first variance and its corresponding dual variable.
[0024] In some possible implementations, the first weighted combination satisfies the following relation:
[0025]
[0026] in, This represents the first mean. Let C1 and C2 represent the dual variable corresponding to the first mean, C1 and C2 represent the weights, t represent the first node being the t-th node among multiple first nodes, and s represent the s-th round of local training.
[0027] In some possible implementations, the second weighted combination satisfies the following relation:
[0028]
[0029] in, This represents the first variance. Let represent the dual variable corresponding to the first variance, ρ represent the penalty coefficient, t represent the t-th node among multiple first nodes, and s represent the s-th round of local training.
[0030] In some possible implementations, the first node is one of a plurality of first nodes, and before sending the second information to the first node, the method further includes: determining the second posterior distribution based on the first posterior distribution and / or the first dual variable reported by the plurality of first nodes.
[0031] In some possible implementations, the parameters of the first posterior distribution include a first mean and a first variance, the first dual variable includes the dual variable corresponding to the first mean and the dual variable corresponding to the first variance, and the parameters of the second posterior distribution include a second mean and a second variance.
[0032] In some possible implementations, the second mean satisfies the following relationship:
[0033]
[0034] in, Let T represent the second mean, T represent the number of first nodes, and t represent the t-th node among the T first nodes. This represents the first mean of the t-th node. Let C1 and C2 represent the dual variable corresponding to the first mean of the t-th node, and let s represent the weights.
[0035] In some possible implementations, the second variance satisfies the following relationship:
[0036]
[0037] in, Let T represent the second variance, T represent the number of first nodes, and t represent the t-th node among the T first nodes. Let represent the first variance of the t-th node. Let represent the dual variable corresponding to the first variance of the t-th node, ρ represent the penalty coefficient, C1 represent the weight, and s represent the s-th round of local training.
[0038] Thirdly, a communication device is provided, comprising: the device can be used for a first node, the device can be the first node, or a device in the first node (e.g., a chip, or a chip system, or a circuit, or a processor), or a device that can be matched and used with the first node, or a logic module or software that can implement all or part of the first node.
[0039] The device includes modules that perform the methods / operations / steps / actions described in the first aspect or any possible implementation of the first aspect. These modules can be hardware circuits, software, or a combination of hardware circuits and software.
[0040] Fourthly, a communication device is provided, comprising: the device can be used for a second node, the device can be the second node, or a device in the second node (e.g., a chip, or a chip system, or a circuit, or a processor), or a device that can be matched with the second node, or a logic module or software that can implement all or part of the second node.
[0041] The device includes modules that perform the methods / operations / steps / actions described in the second aspect or any possible implementation of the second aspect. These modules can be hardware circuits, software, or a combination of hardware circuits and software.
[0042] Fifthly, a communication device is provided, comprising: a processor and a memory, the processor being coupled to the memory, the memory being used to store a computer program (also referred to as code or instructions), the computer program being executed by the processor causing the device to perform the method of the first aspect or any possible implementation thereof.
[0043] In some possible implementations, the device also includes a memory coupled to the processor.
[0044] In some possible implementations, there are one or more processors, and / or one or more memories.
[0045] In some possible implementations, the memory can be integrated with the processor, or the memory can be set up separately from the processor.
[0046] In a sixth aspect, a communication device is provided, comprising: a processor and a memory, the processor being coupled to the memory, the memory being used to store a computer program (also referred to as code or instructions), the computer program being executed by the processor causing the device to perform the method of the second aspect or any possible implementation thereof.
[0047] In some possible implementations, the device also includes a memory coupled to the processor.
[0048] In some possible implementations, there are one or more processors, and / or one or more memories.
[0049] In some possible implementations, the memory can be integrated with the processor, or the memory can be set up separately from the processor.
[0050] In a seventh aspect, a computer-readable storage medium is provided, on which a computer program (also referred to as code or instructions) is stored, which, when executed on a computer, causes the computer to perform the methods of any of the above aspects or any possible implementations thereof.
[0051] Eighthly, a computer program product is provided, comprising: a computer program (also referred to as code or instructions) that, when run on a computer, causes the computer to perform the method in any of the above aspects or any possible implementations of any of the above aspects.
[0052] A ninth aspect provides a chip comprising: a processor and a memory, the memory for storing a computer program (also referred to as code or instructions), the processor for calling and running the computer program stored in the memory, such that an apparatus or device on which the chip is mounted performs the method of any of the above aspects or any possible implementation thereof. Attached Figure Description
[0053] Figure 1 This is a schematic diagram of a neural network structure according to this application.
[0054] Figure 2 This is a schematic structural diagram of a neuron in this application.
[0055] Figure 3 This is a schematic block diagram of a wireless communication system applicable to this application.
[0056] Figure 4 This is a schematic flowchart of a communication method provided in one embodiment of this application.
[0057] Figure 5 This is a schematic flowchart of a communication method provided in another embodiment of this application.
[0058] Figure 6 This is a schematic flowchart of a communication method provided in another embodiment of this application.
[0059] Figure 7This is a schematic structural diagram of a communication device provided in one embodiment of this application.
[0060] Figure 8 This is a schematic structural diagram of a communication device provided in another embodiment of this application.
[0061] Figure 9 This is a schematic structural diagram of an apparatus provided in one embodiment of this application. Detailed Implementation
[0062] To better illustrate the technical solutions of the embodiments of this application, relevant knowledge will be introduced below.
[0063] Before describing the embodiments of this application, the relevant terms involved in this application will be introduced.
[0064] 1. Artificial Intelligence (AI): Artificial intelligence is about enabling machines to possess human-like intelligence. It uses computer hardware and software to simulate certain intelligent behaviors of humans, including machine learning and many other methods.
[0065] 2. Machine Learning (ML): Machine learning is one way to implement artificial intelligence. It's a method that empowers machines to perform functions that are impossible through direct programming. Machine learning involves learning models or rules from raw data and then using those models to make predictions. There are many different machine learning methods, such as neural networks, decision trees, and support vector machines.
[0066] 3. AI Model (or simply Model): An AI model can be understood as a function model that maps an input of a certain dimension to an output of a certain dimension. The parameters of an AI model can be obtained through machine learning training. For example, f(x) = ax 2 +b is a quadratic function that can be viewed as an AI model, where a and b correspond to the parameters of the AI model and can be obtained through machine learning training.
[0067] 4. Neural network (NN): Here it refers to artificial neural network, which is a mathematical model that imitates the behavioral characteristics of animal neural networks to perform distributed parallel information processing. It is a special form of AI model.
[0068] Neural networks are a specific manifestation of machine learning methods. A neural network is a mathematical model that mimics the behavioral characteristics of animal neural networks to process information. For example... Figure 1As shown, a neural network can be composed of three types of computational layers: input layer, hidden layer, and output layer. Each layer can have one or more logical decision units, which are called neurons. Each neuron can perform a weighted summation operation on its input values and output the result through a nonlinear function. Therefore, the weights of the weighted summation operation of neurons in a neural network and the nonlinear function are called the parameters of the neural network. The connections between neurons in a neural network are called the weights of the neural network, and the weights of all neurons in a neural network constitute the parameters of the neural network (also called the weights of the neural network).
[0069] 5. Deep neural network (DNN): A neural network with multiple hidden layers.
[0070] 6. Deep learning (DL): Machine learning that utilizes deep neural networks.
[0071] 7. Dataset: The collection of data used for model training, validation, and testing in machine learning. The quantity and quality of data in the dataset will affect the effectiveness of machine learning.
[0072] 8. Model Training: By selecting an appropriate loss function, the model parameters are trained using optimization algorithms to minimize the loss function value.
[0073] 9. Loss function: Used to measure the difference between the model's predicted value and the true value.
[0074] 10. Model Testing: After model training, use test data to evaluate model performance.
[0075] 11. Model Application: Using the trained model to solve practical problems.
[0076] With the rapid development of AI technology, the application scenarios of AI models (such as neural network models) are becoming more and more widespread, including smart healthcare, finance, smart cities, and intelligent networks. For example, AI models can be introduced into some communication systems to achieve network intelligence.
[0077] Machine learning is an important technological approach to achieving artificial intelligence; the parameters of AI models can be obtained through machine learning training. Machine learning can include supervised learning, unsupervised learning, and reinforcement learning.
[0078] Supervised learning: Supervised learning uses machine learning algorithms to learn the mapping relationship between sample values and sample labels based on collected sample values and labels, and expresses the learned mapping relationship using a machine learning model (such as an AI model). The process of training the machine learning model is the process of learning this mapping relationship. For example, in signal detection, the noisy received signal is the sample, and the corresponding real constellation point is the label. Machine learning aims to learn the mapping relationship between samples and labels through training, that is, to enable the machine learning model to learn a signal detector. During training, the model parameters are optimized by calculating the error between the model's predicted values and the real labels. Once the mapping relationship is learned, the learned mapping can be used to predict the sample label of each new sample. The mapping relationship learned in supervised learning can include linear mappings and nonlinear mappings. Based on the type of label, the learning task can be divided into classification tasks and regression tasks.
[0079] Unsupervised learning (also known as self-supervised learning): Unsupervised learning relies solely on collected sample values, allowing algorithms to discover inherent patterns within the samples themselves. One type of unsupervised learning algorithm uses the samples themselves as supervisory signals, meaning the model learns the mapping relationships from samples to samples; hence, it can also be called self-supervised learning. During model training, the model parameters are optimized by calculating the error between the model's predicted values and the samples themselves. Unsupervised learning can be used for signal compression and decompression recovery applications. Common unsupervised learning algorithms include autoencoders and generative adversarial networks (GANs).
[0080] Reinforcement learning, unlike supervised learning, is a type of algorithm that learns problem-solving strategies through interaction with the environment. Unlike supervised and unsupervised learning, reinforcement learning algorithms do not have explicit "correct" action labels. They need to interact with the environment to obtain reward signals and adjust decision actions to obtain larger reward values. For example, in downlink power control, a reinforcement learning model can adjust the downlink transmission power of each user based on the total system throughput feedback from the wireless network, aiming to achieve higher system throughput. The goal of reinforcement learning is also to learn the mapping relationship between the environment state and the optimal decision action. However, because the label of the "correct action" cannot be obtained in advance, the network cannot be optimized by calculating the error between the action and the "correct action." Therefore, training a reinforcement learning-based model is achieved through iterative interaction with the environment.
[0081] Deep Neural Networks (DNNs) are a specific implementation of machine learning. According to the general approximation theorem, neural networks can theoretically approximate any continuous function, thus enabling them to learn arbitrary mappings. Traditional communication systems rely on extensive expert knowledge to design communication modules, while DNN-based deep learning communication systems can automatically discover hidden pattern structures from large datasets, establish mapping relationships between data, and achieve performance superior to traditional modeling methods.
[0082] The idea behind DNNs (Dual Neural Networks) originates from the neuronal structure of the brain. Each neuron performs a weighted summation of its input values, and the result is passed through a non-linear function to produce the output. For example, ... Figure 2 As shown, assume the input of the neuron is x = [x0, x1, ..., x2]. n The weights corresponding to the inputs are w = [w0, w1, ..., w] n The input is a weighted sum of its corresponding weights, with a bias of b. The nonlinear function can take many forms; for example, a possible example is the max{0,x} maximum value function, in which case the output of a neuron could be... Typically, DNNs can have multi-layered structures, for example, such as Figure 1 As shown, each layer of a DNN can contain multiple neurons. The input layer processes the received values through neurons and then passes them to the hidden layers. Similarly, the hidden layers can then pass the computation results to the final output layer to obtain the output of the DNN.
[0083] DNNs typically have more than one hidden layer, and these hidden layers often directly affect the ability to extract information and fit functions. Increasing the number of hidden layers or widening the width of each layer can improve the function fitting ability of a DNN. The weights in each neuron are the parameters of the DNN network model. The model parameters are optimized through the training process, enabling the DNN network to extract data features and express mapping relationships. DNNs generally use supervised or unsupervised learning strategies to optimize model parameters.
[0084] Common DNNs can include feedforward neural networks (FNNs), convolutional neural networks (CNNs), and recurrent neural networks (RNNs), all of which are based on neurons. Figure 1 The image shows an FNN network, characterized by complete pairwise connections between neurons in adjacent layers. This makes FNNs typically require a large amount of storage space and result in high computational complexity.
[0085] CNNs are a class of neural networks specifically designed to process data with a grid-like structure. For example, time-series data (discrete sampling along the time axis) and image data (two-dimensional discrete sampling) can both be considered grid-like data. CNNs do not use all the input information at once for computation; instead, they use a fixed-size window to extract a portion of the information for convolution operations, which significantly reduces the computational cost of model parameters. Furthermore, depending on the type of information extracted by the window (such as people and objects in an image representing different types of information), each window can use different convolution kernels, allowing CNNs to better extract features from the input data.
[0086] Recurrent Neural Networks (RNNs) are a type of distributed neural network (DNN) that utilizes feedback time-series information. The input to an RNN can include the current input value and its own output value from the previous time step. RNNs are well-suited for acquiring temporally correlated sequence features, and are particularly applicable to applications such as speech recognition and channel coding / decoding.
[0087] The FNN, CNN, and RNN mentioned above are common neural network structures, all built upon neurons. As explained above, each neuron performs a weighted summation of its input values and outputs the result through a non-linear function. Therefore, the weights of the weighted summation operation and the non-linear function in the neural network are called the parameters of the neural network. For example, continuing with the example of a neuron with max{0,x} as the non-linear function, we will... The parameters of the operated neuron can include weights, biases, and nonlinear functions, where the weights are w = [w0, w1, ..., w...]. n The parameters of a neural network are: ], the bias of the weighted summation (for each neuron) is b, and the nonlinear function max{0,x}. The parameters of all neurons in a neural network constitute the parameters of that neural network.
[0088] As described above, most machine learning techniques, such as deep neural network-based learning algorithms, require centralized training with all available data. However, centralized training necessitates collecting a vast amount of data, often from user devices, requiring users to upload this data, which incurs significant upload overhead. Furthermore, concentrating massive amounts of training data on a single node limits training efficiency due to storage space constraints. Additionally, collecting user-side data may infringe on user privacy. Storing massive amounts of data for centralized training could raise serious concerns about privacy breaches among users. On the other hand, if users train machine learning algorithms using only their own existing data, the training effectiveness is often limited by the limited amount of local data available.
[0089] Distributed model training methods are an effective solution to the aforementioned problems, enabling multiple terminal devices to collaboratively train AI models while protecting user privacy and data security. Federated learning (FL) is one such distributed model training method.
[0090] One possible training method is the Federated Learning algorithm (FedAvg) based on stochastic gradient descent (SGD). Federated learning allows child nodes (also called child compute nodes, etc.) to perform multi-step gradient updates using their local datasets, and then feed the updated model back to the central node (also called the central compute node, central server node, etc.), thus reducing the communication overhead caused by each child node needing to interact with the central node for every gradient update. Specifically, in the federated learning framework, K child nodes and the central node can collaboratively train a global model (or, in other words, a globally shared model). The central node aggregates the models sent by the child nodes, and each of the K child node models performs multiple rounds of local training using its local private data.
[0091] In this context, a "round" refers to iterating through all available local data. Assuming the local dataset contains N training samples, and during training, b samples are randomly selected each time to execute the stochastic gradient descent algorithm, then one round of computation... The subgradient, where This indicates that N / b is rounded down. Then, the central node performs model aggregation. After aggregation, the central node and child nodes share the aggregated model, and the above steps are repeated until the model converges. All local models in the FedAvg method are trained on locally stored data, thus protecting the data privacy of child nodes.
[0092] The entire process of the FedAvg framework can include local model training, local dual variable updates, communication between child nodes and the central node, and model aggregation. During the local training phase, the K child nodes train their own models independently and in parallel. The models used for local training by the child nodes can be any type of machine learning model.
[0093] During the communication phase between child nodes and the central node, each communication transmits only the model trained by the child node or the model aggregated by the central node. To improve training efficiency, federated learning can select a subset of child nodes to upload their models. The central node aggregates the received models within a set time window. In the model aggregation phase, federated learning typically uses model averaging, meaning the central node performs a weighted average of the received child node models.
[0094] The above local model training, communication, and model aggregation steps will be performed multiple times until the model at the central node converges.
[0095] While the FedAvg method can protect user privacy and reduce communication overhead between child nodes and the central node, it can only obtain point estimates of the model (such as the global model and / or the local model of the child node), and cannot obtain the posterior distribution of the model. Therefore, it cannot analyze the importance of the parameters in the model or the confidence of the model output, which will affect the training effect of the final model.
[0096] Another possible training method is a federated learning algorithm based on the alternating direction method of multipliers (ADMM) (which can be called FedADMM). In this algorithm, child nodes (also called child compute nodes, etc.) can update their local models by solving a local optimization problem based on local data to obtain the optimal solution, and then update their local dual variables. The updated model is then fed back to the central node (also called the central compute node, central server node, etc.) to reduce the communication overhead caused by the interaction between the child nodes and the central node for each gradient update.
[0097] For example, in the FedADMM method, K (K is a positive integer) child nodes and the central node can collaboratively train a global model (or, it can also be called a globally shared model). The central node aggregates the models sent by the child nodes, while the K child nodes update their local models by solving a local optimization problem and report the updated local models. At the same time, they also update their local dual variables. Then, the central node performs model aggregation. After the aggregation is completed, the central node and the child nodes share the aggregated model, and then the above steps are repeated until the model converges.
[0098] Similar to FedAvg, all local models in the FedADMM method are trained on locally stored data, thus protecting the data privacy of child nodes. The entire FedADMM framework process can include local model training, local dual variable updates, communication between child nodes and the central node, and model aggregation. During the local training phase, K child nodes train their own models independently and in parallel. The models used for local training by child nodes can be any type of machine learning model. Compared to the FedAvg method, the FedADMM method has a faster convergence speed because it is based on the ADMM algorithm.
[0099] Similar to the FedAvg method mentioned above, the FedADMM method also cannot obtain point estimates of the model (such as the global model and / or the local model of the child nodes), nor can it obtain the posterior distribution of the model. Therefore, it cannot analyze the importance of the parameters in the model or the confidence of the model output, which will affect the training effect of the final model.
[0100] To address at least one of the aforementioned technical problems, this application provides a new technical solution. The technical solution in the embodiments of this application will be described below with reference to the accompanying drawings.
[0101] In the description of this application, unless otherwise stated, " / " indicates that the objects before and after are in an "or" relationship. For example, A / B can represent A or B. "And / or" in this application merely describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple. Additionally, to facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or order of execution, and that "first," "second," etc., do not necessarily imply that they are different. It should be understood that in this application, descriptions such as "in the case of," "if," "when," "if," etc., can be used interchangeably.
[0102] The wireless communication system in this application can be various wireless communication systems, such as 5th generation (5G) systems, new radio (NR) systems, long term evolution (LTE) systems, LTE frequency division duplex (FDD) systems, LTE time division duplex (TDD) systems, satellite and other non-terrestrial communication systems, and communication systems that integrate terrestrial and non-terrestrial communication. The technical solution provided in this application can also be applied to future communication systems.
[0103] To facilitate understanding of the embodiments of this application, firstly, in conjunction with Figure 3 This describes a wireless communication system applicable to embodiments of this application. For example... Figure 3 As shown, the wireless communication system includes a wireless access network 100. The wireless access network 100 may include at least one network device (such as...). Figure 3 110 in the text may also include at least one terminal device (such as...) Figure 3 (120a to 120e).
[0104] The scheme in this application embodiment can be applied to a distributed training system. This distributed training system may include a central node and child nodes, and the child nodes and the central node can transmit data to each other. This data can be carried on physical channels, such as a physical downlink control channel (PDCCH), a physical downlink shared channel (PDSCH), a physical uplink shared channel (PUSCH), or a physical uplink control channel (PUCCH), or even, for example, a physical sidelink control channel (PSCCH) or a physical sidelink shared channel (PSSCH).
[0105] In some embodiments, the network device can act as a central node, and the terminal device can act as a child node. Both the network device and the terminal device may further include a processing module based on a machine learning model, such as a neural network module. For example, for the network device, the processing module may be located within the network device, or it may be separate from the network device; this is not limited in the embodiments of this application. For the terminal device, the processing module may be located within the terminal device, and the processing module can access the user's local data. The processing module can implement feature extraction, feature analysis, classification, and regression based on machine learning algorithms. In the embodiments of this application, the machine learning model trained by the distributed training system can be oriented towards non-wireless communication services, such as image recognition and natural language processing, or it can be oriented towards wireless communication services, such as beam selection based on environmental information.
[0106] For example, such as Figure 3 As shown, when the central node is a network device (such as...) Figure 3 (110 in the middle), the child nodes are terminal devices (such as Figure 3In the case of UEs 120a to 120e, network device 110 and UEs 120a to 120e can form a distributed training system. In this communication system, UEs 120a to 120e can send data to network device 110. Network device 110 needs to receive uplink data sent by UEs 120a to 120e. The uplink data can be parameters representing model differences and / or model parameters calculated by the sub-nodes, or feedback quantities containing their state information. Simultaneously, network device 110 can send configuration information to UEs 120a to 120e. The configuration information can be global model parameters used by the central node to synchronize the various sub-nodes, or control information instructing the training methods of the sub-nodes.
[0107] The terminal device in this application embodiment may refer to user equipment (UE), station, access terminal, user unit, user station, mobile station, mobile station (MS), remote station, remote terminal, mobile terminal (MT), user terminal, terminal (or terminal device), wireless communication equipment, user agent or user device, etc., or a device used to provide voice or data connectivity to users, or an Internet of Things device. For example, terminal devices include handheld devices with wireless connection functions, vehicle-mounted devices, etc., but this application embodiment does not limit this. The terminal device in this application embodiment may be a mobile phone, cellular phone, cordless phone, session initiation protocol (SIP) phone, wireless local loop (WLL) station, personal digital assistant (PDA), handheld device with wireless communication function, computing device or other processing device connected to a wireless modem, large screen, vehicle-mounted device (e.g., car, bicycle, electric vehicle, airplane, ship, train, high-speed rail, etc.), wearable device (e.g., smartwatch, smart bracelet, pedometer, smart glasses, etc.), machine type communication (MTC) terminal device, terminal device in 5G network, or terminal device in future evolved public land mobile network (PLMN), etc., and is not limited to this in this application embodiment.The terminal device in the embodiments of this application may also be a tablet computer, a laptop computer, a handheld computer, a mobile internet device (MID), a virtual reality (VR) device, an augmented reality (AR) device, a point of sale (POS) machine, customer-premises equipment (CPE), a light UE, a reduced capability UE (RedCap UE), a wireless terminal in industrial control, a smart home device (e.g., a refrigerator, a television, an air conditioner, an electricity meter, etc.), a smart robot, a robotic arm, workshop equipment, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, or a flying device (e.g., a smart robot, a hot air balloon, a drone, an airplane), etc. Terminal devices can also be vehicle devices, such as vehicle units, vehicle modules, vehicle chips, on-board units (OBU), or telematics boxes (T-BOX). Terminal devices can also be other devices with terminal functions. For example, a terminal device can also be a device that plays a terminal function in device-to-device (D2D) communication.
[0108] In some implementations, the terminal device can be used to act as a base station. Optionally, the terminal device can act as a scheduling entity to provide sidelink signals between terminal devices in vehicle-to-everything (V2X) or device-to-device (D2D) scenarios. For example, cellular phones and cars can communicate using sidelink signals, or cellular phones and smart home devices can communicate using sidelink signals without relaying communication signals through a base station.
[0109] The network device (or communication device) in this application embodiment can refer to a radio access network (RAN) node (or device) that connects a terminal device to a wireless network, and can also be called a base station (BS). For example, the network device can be a NodeB, an evolved NodeB (eNodeB), a next-generation NodeB (gNB) in a 5G mobile communication system, a transmission reception point (TRP), an access point (AP), a network device (such as a satellite) in a nonterrestrial network (NTN) system, a base station in a future mobile communication system or an access point (AP) in a WiFi system, a wireless controller, relay station, access point, vehicle-mounted equipment, wearable device, or network device in other future evolved communication systems, etc.
[0110] In some implementations, multiple RAN nodes can collaborate to assist terminal devices in achieving wireless access, with different RAN nodes each implementing some of the functions of a base station. For example, a RAN node (i.e., the network device in this application) can be a central unit (CU), a distributed unit (DU), a CU-control plane (CP), a CU-user plane (UP), or a radio unit (RU), etc. CUs and DUs can be set up separately or included in the same network element, such as a baseband unit (BBU). RUs can be included in radio frequency equipment or radio frequency units, such as remote radio units (RRUs), active antenna units (AAUs), or remote radio heads (RRHs). In different systems, CUs (or CU-CPs and CU-UPs), DUs, or RUs may have different names, but those skilled in the art will understand their meaning. For example, in an open radio access network (O-RAN) system, a CU can also be called an open CU (O-CU), a DU can also be called an open DU (O-DU), a CU-CP can also be called an O-CU-CP, a CU-UP can also be called an O-CU-UP, and a RU can also be called an O-RU. Any of the units among the CU (or CU-CP, CU-UP), DU, and RU in this application can be implemented through software modules, hardware modules, or a combination of software and hardware modules. It should be understood that this application does not limit the specific technology or specific device form used in the network equipment.
[0111] In some implementations, the network device can be fixed or mobile, and this application does not limit this. For example, a helicopter or drone can be configured as a mobile network device, and one or more cells can move according to the location of the mobile network device. In other examples, a helicopter or drone can be configured as a device to communicate with another network device.
[0112] In some implementations, network devices can be deployed on land or in the air, and this application does not limit this. For example, network devices can be deployed on land, including indoors or outdoors, handheld or vehicle-mounted; they can also be deployed on water; and they can also be deployed in the air on airplanes, balloons, and satellites.
[0113] In this embodiment, the terminal device or network device may include a hardware layer, an operating system layer running on top of the hardware layer, and an application layer running on top of the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and memory (also called main memory). The operating system can be any one or more computer operating systems that implement business processing through processes. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software. Furthermore, this embodiment does not specifically limit the specific structure of the execution entity of the method provided in this embodiment, as long as it can communicate according to the method provided in this embodiment by running a program that records the code of the method provided in this embodiment.
[0114] Furthermore, various aspects or features of this application can be implemented as methods, apparatus, or articles of manufacture using standard programming and / or engineering techniques. The term "article of manufacture" as used herein encompasses a computer program accessible from any computer-readable device, carrier, or medium. For example, computer-readable media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks, or magnetic tapes), optical discs (e.g., compact discs (CDs), digital versatile discs (DVDs), etc.), smart cards, and flash memory devices (e.g., erasable programmable read-only memory (EPROMs), cards, sticks, or key drives, etc.). Additionally, the various storage media described herein may represent one or more devices and / or other machine-readable media for storing information. The term "machine-readable medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.
[0115] To address one or more of the aforementioned technical problems, this application proposes a communication method and communication device that can improve the training effect of the model. The following is a detailed explanation... Figure 4 The communication methods in the embodiments of this application are illustrated in detail.
[0116] Figure 4 This is a schematic flowchart of a communication method provided in one embodiment of this application. Figure 4 The method 400 shown may include steps S410 and S420, as detailed below:
[0117] S410, the first node sends the first message to the second node.
[0118] In some embodiments, the first node can be a child node and the second node can be a central node; or, both the first node and the second node can be child nodes, for example, child nodes in a distributed training architecture without a fixed central node.
[0119] The distributed training architecture without a fixed central node can be understood as follows: During the local model training phase, one or more child nodes can each solve their local optimization problem, update their local model, and update their dual variables. Then, during the inter-node aggregation update phase, a child node can share its model parameters with another randomly selected child node. One or more child nodes can iteratively execute these steps multiple times until all child nodes converge to a consensus model.
[0120] In some embodiments, the first information may be used to indicate the first posterior distribution and / or the first dual variable corresponding to the first posterior distribution.
[0121] Optionally, the first posterior distribution can be determined based on a local dataset. For example, the first node can be obtained by performing Bayesian optimization using a local dataset. Bayesian optimization (also known as Bayesian estimation) can include variational inference, Monte Carlo sampling, and other methods, which are not limited in this embodiment.
[0122] Optionally, the first posterior distribution can be understood as the posterior distribution of the first model obtained using the local dataset of the first node as evidence. Here, the first model can be understood as the local model of the first node (such as the local model trained by the first node).
[0123] Optionally, the first dual variable may be determined based on the first posterior distribution.
[0124] In some embodiments, the first information may include parameters of a first posterior distribution and / or a first dual variable. The parameters of the first posterior distribution may include a first mean and a first variance. For example, the first posterior distribution may be a Gaussian posterior distribution, the first mean may be the mean of the Gaussian posterior distribution, and the first variance may be the variance of the Gaussian posterior distribution. The first dual variable may include the dual variable corresponding to the first mean and the dual variable corresponding to the first variance.
[0125] In some embodiments, before sending the first information to the second node, the first node may determine the first posterior distribution and the first dual variable. For example, before step S410 above, method 400 may further include step S402, as follows:
[0126] S402, the first node determines the first posterior distribution and the first dual variable based on the local dataset of the first node.
[0127] For example, the first node can use the local dataset to perform Bayesian optimization to obtain the first posterior distribution; and determine the first dual variable based on the first posterior distribution.
[0128] S420, the second node sends the second information to the first node.
[0129] In some embodiments, the second information may be used to indicate a second posterior distribution. Optionally, the second posterior distribution may be determined based on a first posterior distribution and / or a first dual variable.
[0130] In some embodiments, the second posterior distribution can be understood as the posterior distribution of the second model obtained using the global dataset as evidence. The second model can be understood as the global model of the second node (such as the global model obtained by aggregating the second node), and the global dataset can include the local datasets of all child nodes participating in training (including the first node).
[0131] In some embodiments, the second information may include parameters of the second posterior distribution. These parameters may include a second mean and a second variance. For example, the second posterior distribution may be a Gaussian posterior distribution, the second mean may be the mean of the Gaussian posterior distribution, and the second variance may be the variance of the Gaussian posterior distribution.
[0132] The second mean and second variance will be further illustrated below using the ADMM method.
[0133] For example, the second mean can satisfy the following relationship:
[0134]
[0135] in, Let T represent the second mean, T represent the number of first nodes, and t represent the t-th node among the T first nodes. This represents the first mean of the t-th node. Let C1 and C2 represent the dual variable corresponding to the first mean of the t-th node, and s represent the weights.
[0136] Local training rotation.
[0137] For example, the second variance can satisfy the following relationship:
[0138]
[0139] in, Let T represent the second variance, T represent the number of first nodes, and t represent the t-th node among the T first nodes. Let represent the first variance of the t-th node. Let represent the dual variable corresponding to the first variance of the t-th node, ρ represent the penalty coefficient, C1 represent the weight, and s represent the s-th round of local training.
[0140] In some embodiments, the second node may determine the second posterior distribution before sending the second information to the first node. For example, before step S420 above, method 400 may further include step S412, as follows:
[0141] S412, the second node determines the second posterior distribution based on the first posterior distribution and / or the first dual variable reported by multiple first nodes.
[0142] The first node can be one of multiple first nodes.
[0143] For example, if the first information only indicates the first posterior distribution, the second node can determine the second posterior distribution based on the first posterior distribution and the dual variable previously uploaded by the first node (such as the most recent upload), or it can determine the second posterior distribution based on the first posterior distribution and preset parameter values (such as preset dual variable values).
[0144] For example, if the first information only indicates the first dual variable, the second node can determine the second posterior distribution based on the first dual variable and the posterior distribution previously uploaded by the first node (such as the most recent upload), or it can determine the second posterior distribution based on the first dual variable and preset parameter values (such as preset posterior distribution values).
[0145] Alternatively, if the first information indicates the first posterior distribution and the first dual variable, the second node can determine the second posterior distribution based on the first posterior distribution and the first dual variable.
[0146] In some embodiments, in step S412 above, the second node can determine the parameters of the second posterior distribution according to the ADMM method. For example, the second node can determine the parameters of the second posterior distribution based on relation one and / or relation two above.
[0147] In some embodiments, the first information may include a mathematical combination of the parameters of the first posterior distribution and the first dual variable. The mathematical combination may include a first weighted combination of the first mean and the corresponding dual variable, and a second weighted combination of the first variance and the corresponding dual variable.
[0148] In this embodiment of the application, the first information includes a mathematical combination of the parameters of the first posterior distribution and the first dual variable. In this way, compared with directly sending the parameters of the first posterior distribution and the first dual variable, the number of parameters transmitted can be reduced, thereby reducing communication overhead.
[0149] The following example, using the ADMM method, further illustrates the mathematical combination of the parameters of the first posterior distribution and the first dual variable.
[0150] After analyzing relational expressions one and two above, the applicant determined that the second node only needs to know... and The second posterior distribution can then be calculated, where, and This can be understood as two combinations of the parameters of the first posterior distribution and the first dual variable. Therefore, the first node only needs to upload these two combinations in step S410 above.
[0151] Considering that the mean and variance of the posterior distribution of the local model, as well as the dimension of the dual variable corresponding to the mean and variance, are the same as the dimension of the model (such as the global model and / or the local model), the communication overhead required to directly send the parameters of the first posterior distribution and the first dual variable is four times that of the model's parameters. However, if the mathematical combination of the parameters of the first posterior distribution and the first dual variable is sent, the communication overhead can be halved.
[0152] For example, the first weighted combination can satisfy the following relation three:
[0153]
[0154] in, This represents the first mean. Let C1 and C2 represent the dual variable corresponding to the first mean, C1 and C2 represent the weights (such as constants derived from the ADMM method), t represents the t-th node among multiple first nodes, and s represents the s-th round of local training.
[0155] For example, the second weighted combination can satisfy the following relation four:
[0156]
[0157] in, Indicates the first variance. Let represent the dual variable corresponding to the first variance, ρ represent the penalty coefficient (such as the penalty coefficient specified by the ADMM method), t represent the t-th node among multiple first nodes, and s represent the s-th round of local training.
[0158] In this embodiment, the first node sends first information to the second node to indicate the first posterior distribution and / or the first dual variable. The second node sends second information to the first node based on the first information. The second information indicates the second posterior distribution, which is determined based on the first posterior distribution and / or the first dual variable. In this way, the posterior distribution of the interactive model (such as the first posterior distribution and / or the second posterior distribution) can be analyzed during the training process. This allows for the analysis of the importance of the parameters in the model and the confidence of the model output, thereby improving the training effect of the model.
[0159] The following is combined with Figure 5 Taking the first node as a child node and the second node as the center node as an example, the embodiments of this application are further illustrated by way of example.
[0160] Figure 5 This is a schematic flowchart of a communication method provided in one embodiment of this application. Figure 5 The method 500 shown may include steps S501 to S510, as follows:
[0161] S501, the central node sends the initial global posterior distribution to the child nodes.
[0162] The central node can broadcast the initial global posterior distribution to its child nodes. The initial global posterior distribution can refer to the posterior distribution of the initial global model, and the initial global model can refer to the global model distributed by the central node in the first round of training.
[0163] The initial global posterior distribution can be a Gaussian posterior distribution, and the central node sends the mean and variance of the Gaussian posterior distribution to the child nodes.
[0164] The central node can also send initialization information of the child node's machine learning model to the child node. For example, the initialization information may include: the model type (such as DNN, CNN, random forest model, etc.), the model's structural information (such as the number of hidden layers in the DNN, the number of neurons and activation functions in each or some layers), and other model parameters and training configuration information.
[0165] The central node can broadcast the same information to all child nodes participating in the training, such as the initial global posterior distribution and initialization information mentioned above.
[0166] S502, child nodes update local posterior distribution.
[0167] Child nodes can undergo Bayesian optimization based on the local dataset and the initial global posterior distribution to update the local posterior distribution. The local posterior distribution can refer to the posterior distribution of the local model.
[0168] The specific methods for updating the local posterior distribution of child nodes can be found in existing technologies, and will not be elaborated here.
[0169] S503, child nodes update local dual variables.
[0170] The local dual variable can be understood as the dual variable corresponding to the local posterior distribution. For example, child nodes can update the local dual variable based on the local posterior distribution.
[0171] The specific methods for updating local dual variables in child nodes can be found in existing technologies, and will not be elaborated here.
[0172] S504, child nodes upload their local posterior distribution and local dual variables to the central node.
[0173] S505, the central node updates the global posterior distribution.
[0174] The central node can aggregate the local posterior distributions and local dual variables uploaded by each child node, and update the global posterior distribution according to the ADMM method.
[0175] For example, the central node can calculate the global posterior distribution based on the above relations one and two. The specific calculation method is described in method 400 above and will not be repeated here.
[0176] S506, the central node sends the global posterior distribution to the child nodes.
[0177] The central node can broadcast the global posterior distribution to all child nodes participating in the training.
[0178] S507, child nodes update local posterior distribution.
[0179] Child nodes can update their local posterior distribution based on the local dataset and the global posterior (issued by the central node in step S506).
[0180] S508, child nodes update local dual variables.
[0181] The local dual variable can be understood as the dual variable corresponding to the local posterior distribution (obtained in step S507). For example, child nodes can update the local dual variable based on the local posterior distribution (obtained in step S507).
[0182] S509, child nodes upload their local posterior distribution and local dual variables to the central node.
[0183] Child nodes can upload the local posterior distribution (calculated in step S507) and the local dual variable (calculated in step S508) to the central node.
[0184] S510, the central node updates the global posterior distribution.
[0185] The central node can aggregate the local posterior distributions and local dual variables uploaded by each child node, and update the global posterior distribution according to the ADMM method.
[0186] The above process can be repeated until training is complete, that is, the global model of the central node converges.
[0187] The following is combined with Figure 6 Taking the first node as a child node and the second node as the center node as an example, the embodiments of this application are further illustrated by way of example.
[0188] Figure 6 This is a schematic flowchart of a communication method provided in one embodiment of this application. Figure 6 The method 600 shown may include steps S601 to S612, as follows:
[0189] S601, the central node sends the initial global posterior distribution to the child nodes.
[0190] The central node can broadcast the initial global posterior distribution to its child nodes. The initial global posterior distribution can refer to the posterior distribution of the initial global model, and the initial global model can refer to the global model distributed by the central node in the first round of training.
[0191] The initial global posterior distribution can be a Gaussian posterior distribution, and the central node sends the mean and variance of the Gaussian posterior distribution to the child nodes.
[0192] The central node can also send initialization information of the child node's machine learning model to the child node. For example, the initialization information may include: the model type (such as DNN, CNN, random forest model, etc.), the model's structural information (such as the number of hidden layers in the DNN, the number of neurons and activation functions in each or some layers), and other model parameters and training configuration information.
[0193] The central node can broadcast the same information to all child nodes participating in the training, such as the initial global posterior distribution and initialization information mentioned above.
[0194] S602, child nodes update local posterior distribution.
[0195] Child nodes can undergo Bayesian optimization based on the local dataset and the initial global posterior distribution to update the local posterior distribution. The local posterior distribution can refer to the posterior distribution of the local model.
[0196] The specific methods for updating the local posterior distribution of child nodes can be found in existing technologies, and will not be elaborated here.
[0197] S603, child nodes update local dual variables.
[0198] The local dual variable can be understood as the dual variable corresponding to the local posterior distribution. For example, child nodes can update the local dual variable based on the local posterior distribution.
[0199] The specific methods for updating local dual variables in child nodes can be found in existing technologies, and will not be elaborated here.
[0200] S604, the child node calculates the mathematical combination of the local posterior distribution and the local dual variable.
[0201] For example, child nodes can calculate the mathematical combination of the local posterior distribution and the local dual variable based on relations three and four above. The specific calculation method is described in method 400 above and will not be repeated here.
[0202] S605, the child node uploads the mathematical combination to the central node.
[0203] S606, the central node updates the global posterior distribution.
[0204] The central node can aggregate the mathematical combinations uploaded by each child node and update the global posterior distribution according to the ADMM method.
[0205] For example, the central node can calculate the global posterior distribution based on the above relations one and two. The specific calculation method is described in method 400 above and will not be repeated here.
[0206] S607, the central node sends the global posterior distribution to the child nodes.
[0207] The central node can broadcast the global posterior distribution to all child nodes participating in the training.
[0208] S608, child nodes update local posterior distribution.
[0209] Child nodes can update their local posterior distribution based on the local dataset and the global posterior (issued by the central node in step S607).
[0210] S609, child nodes update local dual variables.
[0211] The local dual variable can be understood as the dual variable corresponding to the local posterior distribution (obtained in step S608). For example, child nodes can update the local dual variable based on the local posterior distribution (obtained in step S608).
[0212] S610, the child node calculates the mathematical combination of the local posterior distribution and the local dual variable.
[0213] The child node can compute (the local posterior distribution obtained in step S608) and (the local dual variable obtained in step S609) a mathematical combination.
[0214] S611, the child node uploads the mathematical combination to the central node.
[0215] Child nodes can upload mathematical combinations (obtained in step S610) to the central node.
[0216] S612, the central node updates the global posterior distribution.
[0217] The central node can aggregate the mathematical combinations uploaded by each child node and update the global posterior distribution according to the ADMM method.
[0218] The above process can be repeated until training is complete, that is, the global model of the central node converges.
[0219] The above text combined Figures 1 to 6 The method embodiments of this application are described in detail below, in conjunction with... Figures 7 to 9 The present application provides a detailed description of the apparatus embodiments. It should be understood that the descriptions of the method embodiments correspond to the descriptions of the apparatus embodiments; therefore, any parts not described in detail can be found in the foregoing method embodiments.
[0220] Figure 7 This is a schematic structural diagram of a communication device provided in an embodiment of this application. Figure 7 The communication device 700 shown can be used in the first node in the foregoing embodiments. The communication device 700 can be the first node, or a device in the first node (e.g., a chip, a chip system, or a circuit), or a device that can be matched with the first node, or a logic module or software that can implement all or part of the first node.
[0221] like Figure 7 As shown, the communication device 700 includes a transmitting unit 710 and a receiving unit 720, as detailed below:
[0222] The sending unit 710 is used to send first information to the second node. The first information is used to indicate the first posterior distribution and / or the first dual variable corresponding to the first posterior distribution. The first posterior distribution represents the posterior distribution of the first model obtained by using the local dataset of the first node as evidence.
[0223] The receiving unit 720 is configured to receive second information from the second node, the second information being used to indicate a second posterior distribution, the second posterior distribution being determined based on the first posterior distribution and / or the first dual variable.
[0224] In some possible implementations, the first information includes the parameters of the first posterior distribution and a mathematical combination of the first dual variable.
[0225] In some possible implementations, the parameters of the first posterior distribution include a first mean and a first variance, the first dual variable includes the dual variable corresponding to the first mean and the dual variable corresponding to the first variance, and the mathematical combination includes a first weighted combination of the first mean and its corresponding dual variable, and a second weighted combination of the first variance and its corresponding dual variable.
[0226] In some possible implementations, the first weighted combination satisfies the following relation:
[0227]
[0228] in, This represents the first mean. Let C1 and C2 represent the dual variable corresponding to the first mean, C1 and C2 represent the weights, t represent the first node being the t-th node among multiple first nodes, and s represent the s-th round of local training.
[0229] In some possible implementations, the second weighted combination satisfies the following relation:
[0230]
[0231] in, This represents the first variance. Let represent the dual variable corresponding to the first variance, ρ represent the penalty coefficient, t represent the t-th node among multiple first nodes, and s represent the s-th round of local training.
[0232] In some possible implementations, the communication device 700 further includes a determining unit 730, which, before sending the first information to the second node, is configured to: determine the first posterior distribution and the first dual variable based on the local dataset of the first node.
[0233] Figure 8 This is a schematic structural diagram of a communication device provided in an embodiment of this application. Figure 8 The communication device 800 shown can be used in the second node in the foregoing embodiments. The communication device 800 can be the second node, or a device in the second node (e.g., a chip, a chip system, or a circuit), or a device that can be used in conjunction with the second node, or a logic module or software that can implement all or part of the second node.
[0234] like Figure 8As shown, the communication device 800 includes a receiving unit 810 and a transmitting unit 820, as detailed below:
[0235] The receiving unit 810 is used to receive first information from the first node. The first information is used to indicate the first posterior distribution and / or the first dual variable corresponding to the first posterior distribution. The first posterior distribution represents the posterior distribution of the first model obtained by using the local dataset of the first node as evidence.
[0236] The sending unit 820 is used to send second information to the first node, the second information being used to indicate a second posterior distribution, the second posterior distribution being determined based on the first posterior distribution and / or the first dual variable.
[0237] In some possible implementations, the first information includes the parameters of the first posterior distribution and a mathematical combination of the first dual variable.
[0238] In some possible implementations, the parameters of the first posterior distribution include a first mean and a first variance, the first dual variable includes the dual variable corresponding to the first mean and the dual variable corresponding to the first variance, and the mathematical combination includes a first weighted combination of the first mean and its corresponding dual variable, and a second weighted combination of the first variance and its corresponding dual variable.
[0239] In some possible implementations, the first weighted combination satisfies the following relation:
[0240]
[0241] in, This represents the first mean. Let C1 and C2 represent the dual variable corresponding to the first mean, C1 and C2 represent the weights, t represent the first node being the t-th node among multiple first nodes, and s represent the s-th round of local training.
[0242] In some possible implementations, the second weighted combination satisfies the following relation:
[0243]
[0244] in, This represents the first variance. Let represent the dual variable corresponding to the first variance, ρ represent the penalty coefficient, t represent the t-th node among multiple first nodes, and s represent the s-th round of local training.
[0245] In some possible implementations, the first node is one of a plurality of first nodes, and the communication device 800 further includes a determining unit 830, which, before sending the second information to the first node, is configured to: determine the second posterior distribution based on the first posterior distribution and / or the first dual variable reported by the plurality of first nodes.
[0246] In some possible implementations, the parameters of the first posterior distribution include a first mean and a first variance, the first dual variable includes the dual variable corresponding to the first mean and the dual variable corresponding to the first variance, and the parameters of the second posterior distribution include a second mean and a second variance.
[0247] In some possible implementations, the second mean satisfies the following relationship:
[0248]
[0249] in, Let T represent the second mean, T represent the number of first nodes, and t represent the t-th node among the T first nodes. This represents the first mean of the t-th node. Let C1 and C2 represent the dual variable corresponding to the first mean of the t-th node, and let s represent the weights.
[0250] In some possible implementations, the second variance satisfies the following relationship:
[0251]
[0252] in, Let T represent the second variance, T represent the number of first nodes, and t represent the t-th node among the T first nodes. Let represent the first variance of the t-th node. Let represent the dual variable corresponding to the first variance of the t-th node, ρ represent the penalty coefficient, C1 represent the weight, and s represent the s-th round of local training.
[0253] Figure 9 This is a schematic structural diagram of a device provided in an embodiment of this application. Figure 9 The dashed lines indicate that the unit or module is optional. The device 900 can be used to implement the methods described in the above method embodiments. The device 900 can be a chip or a communication device.
[0254] Apparatus 900 may include one or more processors 910. The processor 910 may support apparatus 900 in implementing the methods described in the preceding method embodiments. The processor 910 may be a general-purpose processor or a special-purpose processor. For example, the processor may be a central processing unit (CPU). Alternatively, the processor may be other general-purpose processors, microprocessor units (MPUs), microcontroller units (MCUs), graphics processing units (GPUs), artificial intelligence processors (AI processors) or neural processing units (NPUs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.
[0255] The device 900 may further include one or more memories 920. The memories 920 store a program that can be executed by the processor 910, causing the processor 910 to perform the methods described in the preceding method embodiments. The memories 920 may be independent of the processor 910 or integrated within the processor 910. In this embodiment, the memories 920 may include, but are not limited to, cache, read-only memory (ROM), random access memory (RAM), synchronous dynamic random access memory (SDRAM), hard disk drive (HDD) or solid-state drive (SSD), erasable programmable read-only memory (EPROM), or compact discread-only memory (CD-ROM), etc.
[0256] The device 900 may also include a transceiver 930. The processor 910 can communicate with other devices or chips via the transceiver 930. For example, the processor 910 can send and receive data with other devices or chips via the transceiver 930.
[0257] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.
[0258] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0259] This application also provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the steps described in the various method embodiments above.
[0260] This application also provides a computer program product, which includes a computer program that, when run on a computer, causes the computer to perform the steps described in the various method embodiments above.
[0261] This application also provides a chip, which includes a processor and a memory. The memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory, so that a device or equipment (such as a communication device) with the chip installed performs the steps in the above-described method embodiments.
[0262] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate form. The computer-readable storage medium can include at least: any entity or device capable of carrying computer program code to a device / app, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some possible implementations, the computer-readable storage medium may not be an electrical carrier signal or a telecommunication signal.
[0263] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0264] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0265] In the embodiments provided in this application, it should be understood that the disclosed apparatus / devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.
[0266] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0267] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. A communication method, characterized in that, The method is applied to the first node, and the method includes: Send first information to the second node. The first information is used to indicate the first posterior distribution and / or the first dual variable corresponding to the first posterior distribution. The first posterior distribution represents the posterior distribution of the first model obtained by using the local dataset of the first node as evidence. Receive second information from the second node, the second information being used to indicate a second posterior distribution, the second posterior distribution being determined based on the first posterior distribution and / or the first dual variable.
2. The method according to claim 1, characterized in that, The first information includes the parameters of the first posterior distribution and the mathematical combination of the first dual variable.
3. The method according to claim 2, characterized in that, The parameters of the first posterior distribution include a first mean and a first variance. The first dual variable includes the dual variable corresponding to the first mean and the dual variable corresponding to the first variance. The mathematical combination includes a first weighted combination of the first mean and its corresponding dual variable, and a second weighted combination of the first variance and its corresponding dual variable.
4. The method according to claim 3, characterized in that, The first weighted combination satisfies the following relationship: in, This represents the first mean. Let C1 and C2 represent the dual variable corresponding to the first mean, C1 and C2 represent the weights, t represent the first node being the t-th node among multiple first nodes, and s represent the s-th round of local training.
5. The method according to claim 3 or 4, characterized in that, The second weighted combination satisfies the following relationship: in, This represents the first variance. Let represent the dual variable corresponding to the first variance, ρ represent the penalty coefficient, t represent the t-th node among multiple first nodes, and s represent the s-th round of local training.
6. The method according to any one of claims 1 to 5, characterized in that, Before sending the first information to the second node, the method further includes: The first posterior distribution and the first dual variable are determined based on the local dataset of the first node.
7. A communication method, characterized in that, The method is applied to the second node, and the method includes: Receive first information from the first node, the first information being used to indicate the first posterior distribution and / or the first dual variable corresponding to the first posterior distribution, the first posterior distribution representing the posterior distribution of the first model obtained using the local dataset of the first node as evidence; Send a second message to the first node, the second message indicating a second posterior distribution, the second posterior distribution being determined based on the first posterior distribution and / or the first dual variable.
8. The method according to claim 7, characterized in that, The first information includes the parameters of the first posterior distribution and the mathematical combination of the first dual variable.
9. The method according to claim 8, characterized in that, The parameters of the first posterior distribution include a first mean and a first variance. The first dual variable includes the dual variable corresponding to the first mean and the dual variable corresponding to the first variance. The mathematical combination includes a first weighted combination of the first mean and its corresponding dual variable, and a second weighted combination of the first variance and its corresponding dual variable.
10. The method according to claim 9, characterized in that, The first weighted combination satisfies the following relationship: in, This represents the first mean. Let C1 and C2 represent the dual variable corresponding to the first mean, C1 and C2 represent the weights, t represent the first node being the t-th node among multiple first nodes, and s represent the s-th round of local training.
11. The method according to claim 9 or 10, characterized in that, The second weighted combination satisfies the following relationship: in, This represents the first variance. Let represent the dual variable corresponding to the first variance, ρ represent the penalty coefficient, t represent the t-th node among multiple first nodes, and s represent the s-th round of local training.
12. The method according to any one of claims 7 to 11, characterized in that, The first node is one of a plurality of first nodes. Before sending the second information to the first node, the method further includes: The second posterior distribution is determined based on the first posterior distribution and / or the first dual variable reported by the plurality of first nodes.
13. The method according to claim 12, characterized in that, The parameters of the first posterior distribution include a first mean and a first variance, the first dual variable includes the dual variable corresponding to the first mean and the dual variable corresponding to the first variance, and the parameters of the second posterior distribution include a second mean and a second variance.
14. The method according to claim 13, characterized in that, The second mean satisfies the following relationship: in, Let T represent the second mean, T represent the number of first nodes, and t represent the t-th node among the T first nodes. This represents the first mean of the t-th node. Let C1 and C2 represent the dual variable corresponding to the first mean of the t-th node, and let s represent the weights. Let s represent the local training round s.
15. The method according to claim 13 or 14, characterized in that, The second variance satisfies the following relationship: in, Let T represent the second variance, T represent the number of first nodes, and t represent the t-th node among the T first nodes. Let represent the first variance of the t-th node. Let represent the dual variable corresponding to the first variance of the t-th node, ρ represent the penalty coefficient, C1 represent the weight, and s represent the s-th round of local training.
16. A communication device, characterized in that, include: A module or unit for performing the method as described in any one of claims 1 to 15.
17. A communication device, characterized in that, include: A processor and a memory, the processor being coupled to the memory, the memory being used to store a computer program that, when executed by the processor, causes the apparatus to perform the method as described in any one of claims 1 to 15.
18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 15.
19. A computer program product, characterized in that, include: A computer program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 15.
20. A chip, characterized in that, include: A processor and a memory, the memory for storing a computer program, the processor for calling and running the computer program stored in the memory, causing a device or apparatus on which the chip is mounted to perform the method as described in any one of claims 1 to 15.
