Model decorrelation and subspace division for federated learning

Abstract

Certain aspects of the present disclosure provide techniques and apparatus for training machine learning models. One example method generally includes partitioning a machine learning model into a plurality of partitions. A request to update a respective partition of the plurality of partitions in the machine learning model is sent to each respective participating device of the plurality of participating devices in the federated learning scheme, and the request may specify that the respective partition is to be updated based on unique data at the respective participating device. Updates to one or more partitions in the machine learning model are received from the plurality of participating devices, and the machine learning model is updated based on the received updates.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. patent application serial number 17 / 934,098, filed on September 21, 2022, which is hereby incorporated by reference into this application.

[0003] introduction

[0004] Aspects of the present disclosure relate to federated learning.

[0005] Federated learning generally refers to various techniques that allow training of machine learning models to be distributed across multiple client devices, which beneficially allows a wide variety of data to be used to train the machine learning models. For example, using federated learning to train machine learning models for facial recognition can allow these machine learning models to be trained from a wide range of datasets that include different sets of facial features, different amounts of contrast between foreground data of interest (e.g., a person's face) and background data, etc.

[0006] In some examples, federated learning may be used to learn embeddings across multiple client devices based on the assumption that data heterogeneity across devices may allow a wide variety of data to be used to train machine learning models. However, using federated learning techniques to train machine learning models may not take into account various issues with the data used when training machine learning models. For example, while data on different participating devices used to train machine learning models using federated learning may allow a wide variety of data to be used to train machine learning models, there may be various shifts or biases that may adversely affect the training of the machine learning model (e.g., biasing the model, causing the model to be under-fit or over-fit, etc.). In another example, sharing weights associated with an output layer of a model in a federated learning paradigm may result in exposure of sensitive data or allow sensitive data to be reconstructed from data exposed by the output layer of the model.

[0007] Therefore, there is a need for improved techniques for training machine learning models using federated learning techniques. Summary of the invention

[0008] Certain aspects provide a method for training a machine learning model. The method generally includes: partitioning the machine learning model into a plurality of partitions. Sending a request to update a corresponding partition of the plurality of partitions in the machine learning model to each respective participating device in a plurality of participating devices in a federated learning scheme, and the request may specify that the corresponding partition is to be updated based on unique data at the respective participating device. Receive updates to one or more partitions in the machine learning model from the plurality of participating devices, and update the machine learning model based on the received updates.

[0009] Other aspects provide a method for training a machine learning model. The method generally includes: receiving from a server information defining orthogonal partitions to be updated in the machine learning model and constraints for updating the orthogonal partitions. The orthogonal partitions generally include partitions for a first participating device in a federated learning scheme, which are decorrelated with partitions for a second participating device in the federated learning scheme. The orthogonal partitions in the machine learning model are updated based on local data. Information defining the updated orthogonal partitions in the machine learning model is sent to the server.

[0010] Other aspects provide: a processing system configured to perform the aforementioned methods and those described herein; a non-transitory computer-readable medium comprising instructions that, when executed by one or more processors of the processing system, cause the processing system to perform the aforementioned methods and those described herein; a computer program product embodied on a computer-readable storage medium comprising code for performing the aforementioned methods and those further described herein; and a processing system comprising components for performing the aforementioned methods and those further described herein.

[0011] The following description and the associated drawings set forth in detail certain illustrative features of one or more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings depict certain aspects of the one or more embodiments and therefore are not to be considered limiting of the scope of the present disclosure.

[0013] Figure 1 Depicted is an example environment in which a machine learning model is trained using federated learning techniques by multiple client devices.

[0014] Figure 2 Example operations that may be performed by a server to distribute updates of a machine learning model across multiple client devices in a federated learning scenario are illustrated in accordance with aspects of the present disclosure.

[0015] Figure 3 Example operations that may be performed by a client device for updating a machine learning model in a federated learning scheme according to aspects of the present disclosure are illustrated.

[0016] Figure 4 Examples of messages exchanged between participating devices in a federated learning scheme for training and / or updating a machine learning model in accordance with aspects of the present disclosure are illustrated.

[0017] Figure 5 Example implementations of processing systems in which training of machine learning models may be performed in accordance with aspects of the present disclosure are illustrated.

[0018] Figure 6 Example implementations of processing systems in which machine learning models can be trained in accordance with aspects of the present disclosure are illustrated.

[0019] To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements that are common to the figures. It is contemplated that elements and features of one embodiment may be beneficially incorporated in other embodiments without further recitation. DETAILED DESCRIPTION

[0020] Aspects of the present disclosure provide apparatus, methods, processing systems, and computer-readable media for training machine learning models using federated learning techniques.

[0021] In systems that use federated learning to train machine learning models, the machine learning model is typically defined based on model updates (e.g., changes to weights or other model parameters) generated by each of a plurality of participating client devices. Typically, each of these client devices can train the model using data stored locally on the client device. By doing so, a wide variety of data and the utilization of heterogeneity of data across users can be used to train the machine learning model, which can reduce the likelihood that the resulting global machine learning model will underfit the data (e.g., resulting in a model that neither fits the training data nor generalizes to new data) or overfit the data (e.g., resulting in a model that fits the training data too closely so that new data is inaccurately generalized).

[0022] However, since using federated learning techniques to train machine learning models typically involves training machine learning models on heterogeneous data, various challenges may arise. For example, if a global machine learning model is initially trained using data from a first subset of users and the global machine learning model is retrained using data from a second subset of users, the differences in the data from the first subset of users and the second subset of users may not be fully captured in the model update, or may result in a model trained using an unbalanced dataset that biases the model toward data from scenarios that are more common across data from the first subset of users than data from the second subset of users. As a result, the global performance of the machine learning model may be reduced because the machine learning model may not retain the characteristics of the data provided by any particular participant when using federated learning techniques to train the machine learning model.

[0023] In another example, a client with data in a first class of data may retrain a global machine learning model, which may require updating various weights. The updated weights may include weights associated with the first class of data and weights associated with an unrelated second class of data. When updating the global model based on these weights, the updated weights associated with the second class of data may "tamper" or otherwise modify existing weights that are known to produce acceptable performance relative to the second data set. Therefore, although the performance of the machine learning model may be improved relative to the first data set, the global retraining of client devices in a federated learning scheme may unintentionally degrade the performance of the machine learning model relative to the second data set and may not allow successful continued learning of the machine learning model.

[0024] Various aspects of the present disclosure provide techniques for federated learning of machine learning models that use model decorrelation and subspace partitioning to localize the training and updating (or retraining) of machine learning models to different spaces that may correspond to unique information associated with each device participating in a federated learning scheme to train or update a machine learning model. In general, model decorrelation and subspace partitioning may refer to using various techniques to decompose a machine learning model into multiple parts that may have minimal similarity to each other, so that each of these parts can be trained and updated independently. A client device may be configured to train or update a portion of a machine learning model based on information about the client device, and contribute the retrained portion of the machine learning model to a server that coordinates the training of the machine learning model across multiple participating devices. Since each client device may be configured to train or update only a portion of a machine learning model, various aspects of the present disclosure may reduce the complexity of training operations on client devices and may reduce the amount of resources required to communicate updates to the machine learning model to a server that coordinates a federated learning scheme, because the client device may communicate updated weights for only a portion of the machine learning model instead of the updated weights for the entire machine learning model. Furthermore, the use of model decorrelation and subspace partitioning in a federated learning scheme can lead to improved accuracy in machine learning models trained using federated learning techniques, and can allow continuous learning in machine learning models that preserves the results of previous learning operations on each specific part of the machine learning model.

[0025] Example federated learning architecture for updating partitions in a machine learning model

[0026] Figure 1 An example environment 100 is depicted in which a machine learning model is trained using federated learning techniques by multiple client devices. As shown, the environment 100 includes a server 110 (eg, a federated learning server) and multiple client devices 120A-120F (eg, federated learning clients).

[0027] The server 110 typically maintains a machine learning model that can be updated by one or more client devices 120. The global machine learning model can be based on equation g θ (·): The defined embedding network, where the input data Embedding networks typically take x as input and predict an embedding vector g θ (x). In some examples, the embedding network can be learned based on a classification loss or a metric learning loss. The classification loss can include, for example, a softmax-based cross entropy loss or a margin-based loss, as well as other types of classification losses. The metric learning loss can include, for example, a contrastive loss, a triplet loss, a prototype loss, or other metric-based losses.

[0028] As discussed, using federated learning techniques to update a global model may result in various inaccuracies in the resulting updated model, and because updating the entire model may be computationally expensive and impose significant communication overhead when information about the entire machine learning model is sent between the server 110 and one or more client devices 120. In order to improve the accuracy of a machine learning model trained using federated learning techniques and reduce the messaging overhead involved in updating the machine learning model, the machine learning model may be partitioned into a plurality of decorrelated partitions, and each of these decorrelated partitions may be independently updated by one or more of the client devices 120A-120F based on local data stored at these client devices.

[0029] In order to partition the machine learning model into multiple decorrelated partitions, the model can be initially trained using data from multiple data sources. The trained model can then be partitioned into multiple orthogonalized partitions, where variables that are particularly relevant to one partition are different from variables that are particularly relevant to other partitions from the multiple orthogonalized partitions. Since each partition is substantially orthogonal to the other partitions (as discussed in further detail below), each partition can be updated independently without adversely affecting the performance of other partitions in the machine learning model.

[0030] In some aspects, the machine learning model may be partitioned during initial training of the machine learning model. To partition the machine learning model during initial training of the machine learning model, the kernel of the machine learning model (which represents the portion of the machine learning model that can use a linear classifier to solve a nonlinear problem) may be substantially orthogonalized. For example, a layer in a neural network corresponding to the kernel K of the machine learning model may be represented as a matrix. The kernel K may be partitioned based on the decorrelation loss L. decor Orthogonalizing the kernel K, the decorrelation loss can be expressed by the following equation:

[0031] L decor =‖K T K-αI‖p ,p∈{1,2,…}

[0032] Where K T is represented by the shift of the matrix representing the kernel K, α represents the scaling factor, I represents the identity matrix of the matrix representing the kernel K, and p represents the T K-αI‖ represents the power of the number of matrix multiplication operations performed on the residual matrix. T The multiplication of I and K may result in the computation of an autocorrelation matrix representing the autocorrelation properties of kernel K. Subtracting I or a scaled version of I from the autocorrelation properties of kernel K may allow the autocorrelation matrix K to be T The terms in K other than those along the diagonal of the autocorrelation matrix are kept in quadratic form to be used as loss terms when updating the model.

[0033] Then we can calculate the total loss term L total To train the model, the total loss term can be expressed as the old loss term L legacy and decorrelation loss term L decor In some aspects, the weight β can be applied to the decorrelation loss term L decor To generate the total loss term L total , so that L total =L legacy +βL decor In some aspects, the resulting model can be at least partially orthogonalized (or decorrelated) in that the model can be optimized over loss terms including the legacy loss term.

[0034] In some aspects, the machine learning model may be partitioned into a plurality of orthogonal partitions after initial training. Various techniques may be used to orthogonalize the machine learning model after initial training, including Gram-Schmidt orthogonalization, singular value decomposition (SVD), Cholesky decomposition, and other decorrelation techniques that may allow the model to be partitioned into a plurality of orthogonal partitions (e.g., by removing or reducing weight autocorrelation within the machine learning model and / or kernel cross-correlation within the machine learning model).

[0035] Gram-Schmidt orthogonalization generally allows a set of vectors representing a machine learning model to be converted into a set of orthogonal vectors. Each vector in the set of orthogonal vectors may represent an orthogonal portion of the machine learning model that may be independently updated using federated learning techniques, as discussed herein. Typically, for a set S of vectors of size k, the orthogonal vectors generated using Gram-Schmidt orthogonalization may be a set of vectors that span the same k-dimensional subspace as the set S of vectors. Each vector may, for example, represent a portion of a machine learning model and may be projected into space such that any pair of vectors in the orthogonalized set of vectors are orthogonal to each other.

[0036] In some aspects, singular value decomposition can be used to generate multiple orthogonal partitions of a machine learning model. Singular value decomposition can allow for the generation of multiple orthogonal partitions of a machine learning model by using the equation K = UΣV * The decorrelation kernel of the model is derived by factorizing the matrix representing the kernel K with dimensions m×n. U typically denotes a unitary matrix with dimensions m×m, Σ typically denotes a rectangular diagonal matrix, and V typically denotes a unitary matrix with dimensions n×n.

[0037] In yet another aspect, Cholesky decomposition (or whitening) can be used to generate multiple orthogonal partitions of a machine learning model. Cholesky decomposition generally allows a matrix, such as a matrix representing a kernel K, to be a product of multiple matrices. To generate multiple orthogonalized partitions for a machine learning model, the whitening matrix W can be based on the matrix C and its transformation C T to generate.

[0038] In some aspects, the substantially orthogonal partitions into which the machine learning model may be partitioned may include a public subnetwork and a plurality of non-public subnetworks. A public subnetwork may be a network that may be updated by any participating client device 120 and may include one or more initial parts of the machine learning model. For example, a public subnetwork may include one or more layers for extracting features from an input and then taking any action on the extracted features. Multiple non-public subnetworks may be "scenario-dependent" subnetworks suitable for performing machine learning operations on specific data sets. For example, in a scenario where a machine learning model is used in a wireless communication network (e.g., predicting channel state information (CSI), generating embedded messaging based on various channel state information measurements, etc.), different scenarios may correspond to different sets of channel conditions, different types of devices, etc. One partition in the machine learning model may correspond to, for example, a scenario in which the device has a strong signal strength, another partition in the machine learning model may correspond to a scenario in which the device has a weak signal strength, and so on. It should be recognized that this is only one example of a scenario that may correspond to different partitions within a machine learning model, and other scenarios may exist based on the purpose for which the machine learning model is trained and deployed.

[0039] After partitioning the machine learning model, the model can use various techniques to perform subspace partitioning so that the weights of the machine learning model are separated into unique spaces that can be updated independently. As an illustrative example, the weight matrix W of a machine learning model with two scenarios can be defined as:

[0040]

[0041] Where W0 represents a set of weights common to both the first scenario and the second scenario, W1 represents a set of weights unique to the first scenario, W2 represents a set of weights unique to the second scenario, and W1 and W2 are decorrelated (e.g., as discussed above). In order to divide the model into multiple orthogonal partitions, a feature projection technique can be used, so that features in the forward training path at certain locations in the machine learning model are used to project a set of features, and the projected feature set is passed to an activation function to select a subspace of the kernel or channel in the machine learning model for processing.

[0042] In another example, gradient projection can be used to subspace partition a machine learning model. Typically, gradient projection allows gradients back-propagated at certain locations in a machine learning model to be used to project other gradients. A subset of projected gradients can be updated and used for subsequent processing. For example, partitions of a machine learning model can be grouped into null space and non-null space. Gradients can be updated for partitions of the machine learning model in non-null space; however, gradients may not be updated for partitions of the machine learning model in null space.

[0043] For example, suppose the loss function of a machine learning model is expressed according to the following equation:

[0044]

[0045] Where W represents the set of weights of the neural network, x represents the set of input features, and y represents the activation generated from the set of input features x. The gradient with respect to the weight W can be expressed by the following equation:

[0046]

[0047] in and x T represents the transpose of the input feature x. Therefore, the gradient update can be located within the linear span of x. Based on the orthogonal part of the machine learning model, using gradient projection, it may be necessary to update a subset of the weights W instead of the entire W. In other words, using gradient projection, the weights can be updated

[0048] For example, assume that the device includes data suitable for updating the partitions associated with the first scene in the dual scene example discussed above of the machine learning model. In some aspects, these partitions can be kernels of the machine learning model that allow the use of low-dimensional data in a high-dimensional space, and each kernel can be part of the weights in the same layer or kernel of the neural network. Since W0 represents a set of weights shared across all scenes (e.g., weights associated with a common partition in the model) and W1 represents a set of weights applicable to the first scene (e.g., weights associated with a partition in the model for the first scene), the device can update the partitions of the first scene in the common partition and the model, but may not update the partitions of the second scene in the model. Therefore, by dividing the machine learning model into multiple orthogonalized partitions, only the part of the machine learning model related to a specific scene can be updated by the device participating in the federated learning scheme, and the negative impact of updating the global model relative to other unrelated scenes (e.g., tampering with weights unrelated to a specific scene, weight forgetting or other data contamination) can be avoided. The resulting machine learning model can therefore be updated to improve the performance of the model relative to a specific scene while leaving the performance of the model relative to the specific scene essentially unchanged.

[0049] In some aspects, constraints for updating partitions may be determined based on information reported by client devices 120 that are participants in a federated learning scheme. For example, to define constraints for updating partitions of a machine learning model used in a wireless communication system, each client device 120 (which may be, for example, a user equipment (UE) in a wireless communication system, such as a mobile phone, a tablet computer, an autonomous vehicle UE, etc.) may report various radio measurements to a server 110 (which may be, for example, a network entity, such as a base station serving one or more UEs). The constraints may be based on the reported radio measurements from each client device 120, and may, for example, indicate whether constraints are defined for any particular user, and if so, which weights are to be and are not to be updated by each participating client device 120 in the federated learning scheme.

[0050] In order to update the machine learning model, the server 110 may select a plurality of client devices 120 to participate in a federated learning scheme to update the machine learning model. The plurality of client devices 120 may include any number m of client devices and may be selected based on various characteristics, such as data available at each client device, partitions to be updated in the machine learning model (e.g., due to poor inference performance), etc. For example, as shown, the server 110 may select client devices 120A, 120B, 120D, 120E, and 120F as participating devices in the federated learning scheme for updating the machine learning model. For example, these devices may have reported the presence of data for the selected partitions that the server 110 believes are required to update the machine learning model. However, client device 120C is not selected as a participating device in the federated learning scheme because the device may not have the data that would be needed to update the selected partition of the machine learning model (rather, it may at least implicitly indicate to the server 110 that the client device 120C has data that may be relevant to updating the partition of the machine learning model corresponding to the scenario for which the inference performance is considered acceptable). In some aspects, other considerations may also be used to select client devices 120 that will participate in the federated learning scheme to update the machine learning model. These other considerations may include, for example, how recently the client device has participated in training (or updating) the global machine learning model, mobility and power characteristics, available computing resources that can be dedicated to training or updating the model, etc.

[0051] After the server 110 elects to participate in the client devices 120A, 120B, 120D, 120E, and 120F, the server 110 may invoke a model update process at each of the client devices 120A, 120B, 120D, 120E, and 120F by providing the client devices 120A, 120B, 120D, 120E, and 120F with the current version of the partition in the machine learning model. The client devices 120A, 120B, 120D, 120E, and 120F may generate updated model partitions based on the data stored at each client device and upload the updated model partitions to the server 110 for integration into the global machine learning model.

[0052] Server 110 may use various model aggregation techniques to update the machine learning model. Figure 1As illustrated, the server 110 may update partitions of the machine learning model based on a running average of weights associated with updated model partitions generated by the client device 120 over time or based on other statistical measures of weights associated with updated model partitions generated by the client device 120 over time. In some aspects, newer model information (e.g., weights, parameter values, etc.) may replace older model information in the data set of the model on which the machine learning model was generated. As discussed, when updating the machine learning model, the machine learning model may be updated on a per-partition basis, and updating one partition in the machine learning model may not affect parameters (e.g., weights) associated with different orthogonal partitions within the machine learning model or may have a limited effect on these parameters. After updating the machine learning model, the server 110 may deploy the updated machine learning model to the client devices 120A-120F in the environment 100 for performing inference at each of the client devices 120A-120F.

[0053] Example method for updating partitions in a machine learning model

[0054] Figure 2 The present disclosure illustrates that various aspects of the present disclosure may be performed (eg, by a server, a network entity, etc., in a wireless communication network, such as Figure 1 The illustrated server 110) performs example operations 200 to distribute updates of a machine learning model across multiple client devices in a federated learning scenario.

[0055] As shown, operation 200 begins at box 210: partitioning the machine learning model into a plurality of partitions. Typically, when partitioning the machine learning model into a plurality of partitions, each partition may be made substantially orthogonal to other partitions in the machine learning model. As discussed above, when partitioning the machine learning model into a plurality of partitions, the partitions may be made at least partially orthogonal (e.g., when such orthogonalization is based on an old-style loss and a decorrelation loss) or fully orthogonal (e.g., when such orthogonalization is performed after initial training of the machine learning model); although the partitions in the machine learning model need not be fully orthogonal, the partitions may generally have a high degree of orthogonality. Each partition may correspond to one of a plurality of scenarios in which the machine learning model is trained to perform various inference operations. In some aspects, the partitions may be generated as orthogonal portions of the same kernel, such that the kernel includes a subset of weights associated with a common portion of the machine learning model and multiple subsets of weights associated with different scenarios for which the machine learning model is trained. In some aspects, the partitions may be different kernels, wherein a first kernel corresponds to a common portion of the machine learning model and a plurality of other kernels correspond to scenario-specific portions of the machine learning model.

[0056] In some aspects, the machine learning model may be partitioned into a plurality of partitions during initial training of the machine learning model. To partition the machine learning model into a plurality of partitions during initial training of the machine learning model, the machine learning model may be orthogonalized on a per-kernel basis. When orthogonalizing the machine learning model, the total loss may be defined in terms of a legacy loss term and a decorrelation loss term. The decorrelation loss term may be a matrix K representing the kernel and a transposed K T The decorrelation loss term can thus include the autocorrelation matrix K T The entries in K except those along the diagonal of the quadratic form of the autocorrelation matrix.

[0057] In some aspects, the machine learning model can be partitioned into multiple partitions after initially training the machine learning model. After the initial training of the machine learning model, various techniques can be used to partition the machine learning model. For example, the machine learning model can be partitioned into multiple partitions based on Gram-Schmidt orthogonalization, singular value decomposition, Cholesky decomposition, or other decorrelation techniques that remove or reduce autocorrelation between weights in the machine learning model and / or cross-correlation in the kernel of the machine learning model.

[0058] In some aspects, partitions in a machine learning model may be generated based on a feature projection technique. When using the feature projection technique, features at certain locations in the machine learning model during a forward pass (e.g., during a computational process in which an input traverses from an initial portion of the machine learning model to a later portion) may be used to project features of the input into the machine learning model. These projected features may be passed to an activation function to select a partition of the machine learning model to be used for processing. In the feature projection technique, each of a plurality of partitions may be associated with a set of weights. A first partition of a plurality of partitions may be a common partition having a common set of weights applicable to any scenario in which the machine learning model is trained, and a plurality of second partitions of a plurality of partitions may be partitions having weights associated with one of a plurality of defined scenarios for which the machine learning model is trained. As discussed, by dividing the machine learning model into a common partition and a plurality of scene-specific partitions, the common partition and a scene-specific partition of a plurality of scene-specific partitions may be updated by a client device participating in a federated learning scheme to update the machine learning model, which may allow specific portions of the machine learning model to be updated while other portions of the machine learning model remain unchanged.

[0059] In some aspects, partitions in the machine learning model may be selected based on a gradient projection technique. When using the gradient projection technique, gradients identified in some locations in the machine learning model may be used to project gradients for other locations in the machine learning model. A subspace of projected gradients may be updated, while other projected gradients may remain unchanged. In this example, the multiple partitions into which the machine learning model is divided may be organized into null spaces and non-null spaces. The null space may correspond to the portion of the machine learning model that is to remain unchanged, while the non-null space may correspond to the portion of the machine learning model that is to be updated. In doing so, a subset of weights corresponding to the portion of the machine learning model associated with the specific scenario may be updated, and other portions of the machine learning model may remain unchanged.

[0060] At box 220, operation 200 continues by sending a request to each respective participating device in the plurality of participating devices in the federated learning scheme to update a respective partition in the plurality of partitions in the machine learning model. Typically, the request may indicate that the plurality of partitions may be updated based on unique data at each respective participating device in the federated learning scheme. As discussed herein, the plurality of participating devices in the federated learning scheme may include a subset of client devices connected to the server having relevant data for updating different orthogonalized portions of the machine learning model, but need not necessarily include devices that do not have relevant data for updating the orthogonalized portions of the machine learning model but may have previously participated in updating the machine learning model.

[0061] In some aspects, participating devices in a federated learning scheme may transmit various reports to a server that uses the federated learning scheme to coordinate updates of machine learning models. These reports may include, for example, information identifying (at least generally) the type of data that can be used to update the machine learning model at each client device, and other information that can be used to select client devices to participate in the federated learning scheme. The type of data can be signaled in a manner that minimizes the risk of compromising the privacy or security of local data at each client device. For example, in general, client devices may share information about the type of data stored on the client device, which may indicate scenarios for which the client device can update the machine learning model. In one example, where a machine learning model is used to predict parameters for wireless communications between a network entity (or one or more components of a decomposed network entity) and a user equipment (UE), the data at each client device may include radio measurements at each respective device. These measurements may, for example, correspond to a general classification of radio conditions at each UE, and the classification of these radio conditions may be used to select portions of the machine learning model to be updated by each respective device.

[0062] At block 230, operation 200 continues with receiving updates to one or more partitions in the machine learning model from the plurality of participating devices. In some aspects, the updates received from each of the plurality of participating devices may include weight data and / or other information associated with at least a specific partition of the machine learning model for which the specific device in the plurality of participating devices is responsible for updating. In some aspects, the updates received from each device may also include weights for common partitions of the machine learning model that are shared across different partitions of the machine learning model (and therefore, are common to each of the plurality of scenarios for which the machine learning model is trained to generate inferences).

[0063] At box 240, operation 200 continues to update the machine learning model based on the received update. In some aspects, to update the machine learning model, the update can be combined with existing data in the machine learning model. For example, where the update includes weight data, the weight data can be combined by averaging the existing weight data and the updated weight data for each partition in the machine learning model. In some aspects, the weight updates from each participating device in a plurality of participating devices can be aggregated into an overall update to be applied to the machine learning model. Typically, updating the machine learning model can involve updating partitions of the machine learning model for which updates are requested for participating devices, while leaving partitions for which client devices are not requested to be updated in their pre-update state.

[0064] In some aspects, various constraints for updating a partition in a machine learning model may be identified. These constraints may be identified based on reports received from each of a plurality of participating devices and may be signaled to each of a plurality of participating devices. For example, these constraints, when established, may indicate which parameters within a partition of a machine learning model are to be updated and which parameters within a partition of a machine learning model are to remain unchanged during the update process.

[0065] Typically, the server may maintain a mapping between different portions of the machine learning model and the client devices responsible for updating those portions of the machine learning model. The assignment of which portions of the machine learning model are to be managed and updated by the client devices may be communicated to the relevant client devices via one or more messages to the relevant client devices (e.g., in a message instructing participating devices to update portions of the machine learning model, as discussed above with respect to block 220). The server may assign portions of the machine learning model to a specific client device or multiple client devices, and update the machine learning model based on responses received from participating client devices, as discussed above.

[0066] It should be noted that Figure 2 This is merely one example of a method, and other methods including fewer, additional, or alternative steps may also be consistent with the present disclosure.

[0067] Figure 3 The present disclosure illustrates a method that can be used by a client device (e.g., Figure 1 The illustrated client device 120) performs example operations 300 for updating a machine learning model in a federated learning scheme. The client device may include, for example, a user equipment (UE) in a wireless communication network or other device that can interact with a server to participate in a federated learning scheme to update a machine learning model.

[0068] As shown, operation 300 begins at block 310: receiving information defining an orthogonal partition to be updated in a machine learning model. In some aspects, constraints for updating the orthogonal partition may also be received. In some aspects, the orthogonal partition may be a partition for a first participating device in a federated learning scheme that is decorrelated with a partition for a second participating device in the federated learning scheme.

[0069] At box 320, the operation 300 continues to update the orthogonalized partitions in the machine learning model based on the local data.

[0070] In some aspects, updating orthogonal partitions in a machine learning model may include updating orthogonal partitions based on a feature projection technique and local data. In this case, features at certain locations in the machine learning model during a forward pass may be used to project features of an input into the machine learning model. These projected features may be passed to an activation function to select a partition of the machine learning model to be used for processing. In the feature projection technique, each partition in a plurality of partitions may be associated with a set of weights. A first partition in a plurality of partitions may be a common partition having a common set of weights applicable to any scenario for training the machine learning model, and a plurality of second partitions in a plurality of partitions may be partitions having weights associated with one of a plurality of defined scenarios for which the machine learning model is trained.

[0071] In some aspects, updating the orthogonal partitions in the machine learning model may include updating the orthogonal partitions based on a gradient projection technique and local data. In this case, gradients identified in some locations in the machine learning model may be used to project gradients for other locations in the machine learning model. A subspace of projected gradients may be updated, while other projected gradients may remain unchanged. In this example, the plurality of partitions into which the machine learning model is partitioned may be organized into null spaces and non-null spaces. The null space may correspond to a portion of the machine learning model that is to remain unchanged, while the non-null space may correspond to a portion of the machine learning model that is to be updated.

[0072] At block 330, operation 300 continues with sending information defining the updated orthogonal partitions in the machine learning model to the server. Because the information defining the updated orthogonal partitions in the machine learning model may include sets of weights for the orthogonal partitions (and in some cases, common partitions of the machine learning model), but may not include sets of weights for other partitions in the machine learning model, the amount of communication overhead involved in communicating the updated orthogonal partitions to the server may be a fraction of the communication overhead that would be involved in sending update information for the entire machine learning model to the server.

[0073] In some aspects, operation 300 also includes sending one or more reports to the server. Constraints for updating orthogonal partitions in the machine learning model are based on the one or more reports. The one or more reports may include generalized information about local data, which may include information identifying (at least generally) the type of data that can be used to update the machine learning model at each client device. In some aspects, the reports may include information that can be used to select client devices to participate in the federated learning scheme. In one example, where the machine learning model is used to predict parameters for wireless communications between a network entity (or one or more components of a decomposed network entity) and a user equipment (UE), the report may include radio measurements corresponding to a general classification of radio conditions at each UE.

[0074] It should be noted that Figure 3 This is merely one example of a method, and other methods including fewer, additional, or alternative steps may also be consistent with the present disclosure.

[0075] Example message flow diagram for updating partitions in a machine learning model

[0076] Figure 4 Illustrated is an example 400 of messages exchanged between participating devices in a federated learning scheme for training and / or updating a machine learning model in accordance with aspects of the present disclosure.

[0077] As shown, example 400 includes messages exchanged between a server 402, a first client 404, and a second client 406. In one aspect where the machine learning model being updated is a model for predicting parameters for communications in a wireless communication system, server 402 may correspond to a network entity (e.g., a base station), while first client 404 and second client 406 may correspond to different user equipment in the wireless communication system served by the base station.

[0078] Initially, at box 410, the server 402 partitions the machine learning model into a plurality of updateable orthogonalized partitions. Partitioning the machine learning model into a plurality of updateable orthogonalized partitions can be performed as part of an initial training process for the machine learning model, or can be performed after the machine learning model has been trained. If the partitioning is performed as part of an initial training process for the machine learning model, each partition can be at least partially orthogonalized or partially decorrelated with other partitions in the machine learning model because the total loss minimized during training of the machine learning model can include old-style loss terms and decorrelated loss terms. If the partitioning is performed after initially training the machine learning model, the partitions in the machine learning model can be substantially or completely orthogonal to other partitions in the machine learning model.

[0079] To update the machine learning model, update requests 412 and 414 are sent to the first client 404 and the second client 406, respectively. In general, the update requests 412 and 414 may specify to the respective client devices the partitions of the machine learning model to be updated using local data at each of the client devices. In some aspects, the update requests 412 and 414 may also specify various constraints to be followed by the clients 404 and 406 when updating the specified partitions of the machine learning model. These constraints may indicate, for example, which parameters in a specific partition are to be updated and which parameters in the specific partition are to remain unchanged during the update process.

[0080] At block 416, the first client 404 updates the partition with local data associated with the first client in response to receiving the update request 412. Similarly, at block 418, the second client 406 updates the partition with local data associated with the second client in response to receiving the update request 414. The update of the model by the first client 404 at block 416 and the update of the model by the second client 406 at block 418 may be independent and may be performed substantially in parallel; however, it should be noted that these operations may be completed at different times (e.g., due to different processing capabilities of each client device, the amount of data used to update the partition, etc.) and do not need to be synchronized with each other. After updating the partition with local data, the first client 404 and the second client 406 may send respective update responses 420 and 422 to the server 402.

[0081] At box 424, the server updates the machine learning model based on the update responses 420 and 422 received from the first client 404 and the second client 406. The update may include, for example, updated weight data for different partitions in the machine learning model. The different partitions may include a common partition with weights that can be applied across different scenarios in which the machine learning model is trained to perform inference operations, as well as multiple partitions that are scenario-specific. As discussed, when updating the machine learning model, the update may be combined with existing data in the machine learning model to produce an updated model. These updates may only affect the partitions in the machine learning model that request clients 404 and 406 to update, and the partitions in the machine learning model that do not request clients 404 and 406 to update may remain unchanged.

[0082] Example processing system for updating partitions in a machine learning model

[0083] Figure 5 Describes such as this article for example relative to Figure 2 An example processing system 500 for updating a machine learning model across client devices in a federated learning scenario is described.

[0084] The processing system 500 includes a central processing unit (CPU) 502, which may be a multi-core CPU in some examples. Instructions executed at the CPU 502 may be loaded, for example, from a program memory associated with the CPU 502, or may be loaded from a memory partition (e.g., in the memory 524).

[0085] Processing system 500 also includes additional processing components tailored for specific functions, such as a graphics processing unit (GPU) 504 , a digital signal processor (DSP) 506 , a neural processing unit (NPU) 508 , and wireless connectivity components 512 .

[0086] An NPU, such as NPU 508, is typically a dedicated circuit configured to implement control and arithmetic logic for executing machine learning algorithms, such as algorithms for processing artificial neural networks (ANNs), deep neural networks (DNNs), random forests (RFs), etc. An NPU is sometimes alternatively referred to as a neural signal processor (NSP), a tensor processing unit (TPU), a neural network processor (NNP), an intelligence processing unit (IPU), a vision processing unit (VPU), or a graphics processing unit.

[0087] NPUs such as NPU 508 are configured to accelerate the execution of common machine learning tasks such as image classification, machine translation, object detection, and various other predictive models. In some examples, multiple NPUs may be instantiated on a single chip such as a system on a chip (SoC), while in other examples, multiple NPUs may be part of a dedicated neural network accelerator.

[0088] The NPU can be optimized for training or inference, or in some cases configured to balance performance between the two. For NPUs that can perform both training and inference, the two tasks can usually still be performed independently.

[0089] NPUs designed to accelerate training are typically configured to accelerate the optimization of new models, which is a highly computationally intensive operation that involves inputting an existing dataset (usually labeled or tagged), iterating on the dataset, and then adjusting model parameters (such as weights and biases) to improve model performance. Typically, optimization based on error predictions involves passing back through the layers of the model and determining the gradient to reduce the prediction error.

[0090] NPUs designed to accelerate inference are typically configured to operate on complete models. Thus, such NPUs can be configured to input new data slices and quickly process them through the trained model to generate model outputs (e.g., inferences).

[0091] In one implementation, the NPU 508 is part of one or more of the CPU 502 , the GPU 504 , and / or the DSP 506 .

[0092] In some examples, wireless connectivity component 512 may include, for example, subcomponents for third generation (3G) connectivity, fourth generation (4G) connectivity (e.g., 4G LTE), fifth generation connectivity (e.g., 5G or NR), Wi-Fi connectivity, Bluetooth connectivity, and other wireless data transmission standards. Wireless connectivity component 512 is further connected to one or more antennas 514.

[0093] The processing system 500 may also include one or more input and / or output devices 522, such as a screen, a touch-sensitive surface (including a touch-sensitive display), physical buttons, speakers, microphones, and the like.

[0094] The processing system 500 also includes a memory 524, which represents one or more static and / or dynamic memories, such as dynamic random access memory, flash-based static memory, etc. In this example, the memory 524 includes computer-executable components that are executable by one or more of the aforementioned processors of the processing system 500.

[0095] In particular, in this example, memory 524 includes a model partitioning component 524A, an update request sending component 524B, an update receiving component 524C, a model updating component 524D, and a partitioned machine learning model 524E. The depicted components and other components not depicted may be configured to perform various aspects of the methods described herein.

[0096] Generally, processing system 500 and / or its components may be configured to perform the methods described herein.

[0097] Figure 6 Depicted is used for example in this article relative to Figure 3 An example processing system 600 for updating a machine learning model is described.

[0098] The processing system 600 includes a central processing unit (CPU) 602, which may be a multi-core CPU in some examples. Instructions executed at the CPU 602 may be loaded, for example, from a program memory associated with the CPU 602, or may be loaded from a memory partition (e.g., in the memory 624).

[0099] The processing system 600 also includes additional processing components customized for specific functions, such as a graphics processing unit (GPU) 604 , a digital signal processor (DSP) 606 , a neural processing unit (NPU) 608 , a multimedia processing unit 610 , and a wireless connectivity component 612 .

[0100] An NPU such as NPU 608 may be as described above with respect to Figure 6 In one implementation, the NPU 608 is part of one or more of the CPU 602 , the GPU 604 , and / or the DSP 606 .

[0101] In some examples, the wireless connection component 612 can be as described above with respect to Figure 6 as described.

[0102] The processing system 600 may also include one or more sensor processing units 616 associated with any manner of sensors, one or more image signal processors (ISPs) 618 associated with any manner of image sensors, and / or a navigation processor 620, which may include satellite-based positioning system components (e.g., GPS or GLONASS) and inertial positioning system components.

[0103] The processing system 600 may also include one or more input and / or output devices 622, such as a screen, a touch-sensitive surface (including a touch-sensitive display), physical buttons, speakers, microphones, and the like.

[0104] In some examples, one or more of the processors of processing system 600 may be based on the ARM or RISC-V instruction set.

[0105] The processing system 600 also includes a memory 624, which represents one or more static and / or dynamic memories, such as dynamic random access memory, flash-based static memory, etc. In this example, the memory 624 includes computer-executable components that are executable by one or more of the aforementioned processors of the processing system 600.

[0106] In particular, in this example, memory 624 includes a model partition receiving component 624A, a model partition updating component 624B, an update sending component 624C, a machine learning model partition 624D, and local data 624E. The depicted components, as well as other components not depicted, may be configured to perform various aspects of the methods described herein.

[0107] Generally, the processing system 600 and / or its components may be configured to perform the methods described herein.

[0108] It is worth noting that in other embodiments, various aspects of the processing system 600 may be omitted, such as where the processing system 600 is a server computer, etc. For example, in other embodiments, the multimedia processing unit 610, the wireless connectivity component 612, the sensor processing unit 616, the ISP 618, and / or the navigation processor 620 may be omitted.

[0109] Sample Clauses

[0110] Specific implementation details are described in the following numbered clauses:

[0111] Clause 1: A computer-implemented method comprising: partitioning a machine learning model into a plurality of partitions; sending a request to each respective participating device among a plurality of participating devices in a federated learning scheme to update a respective partition among the plurality of partitions in the machine learning model based on unique data at the respective participating device; receiving updates to one or more partitions in the machine learning model from the plurality of participating devices; and updating the machine learning model based on the received updates.

[0112] Clause 2: The method of clause 1, wherein partitioning the machine learning model into the plurality of partitions comprises partitioning the machine learning model into a common subnetwork and one or more non-common subnetworks.

[0113] Clause 3: A method according to clause 1 or 2, wherein partitioning the machine learning model into the plurality of partitions comprises orthogonalizing the partitions.

[0114] Clause 4: The method of clause 3, wherein orthogonalizing the partitions comprises generating the plurality of partitions based on a Gram-Schmidt orthogonalization.

[0115] Clause 5: The method of clause 3 or 4, wherein orthogonalizing the partitions comprises generating the plurality of partitions based on singular value decomposition.

[0116] Clause 6: The method of any of clauses 3 to 5, wherein orthogonalizing the partitions comprises generating the plurality of partitions based on a Cholesky decomposition.

[0117] Clause 7: A method according to any one of clauses 1 to 6, wherein partitioning the machine learning model includes generating the multiple partitions in the machine learning model based on a feature projection technique.

[0118] Clause 8: A method according to clause 7, wherein: each of the plurality of partitions is associated with a set of weights; a first partition of the plurality of partitions comprises a common partition having a common set of weights; and a plurality of second partitions of the plurality of partitions comprises a partition having a weight associated with one of a plurality of defined scenarios for which the machine learning model is trained.

[0119] Clause 9: A method according to any one of clauses 1 to 8, wherein partitioning the machine learning model comprises generating the multiple partitions in the machine learning model based on a gradient projection technique.

[0120] Clause 10: The method of clause 9, wherein: the plurality of partitions comprises a null space and a non-null space, and the request to update the one or more of the plurality of partitions comprises a request to update a subspace in the non-null space.

[0121] Clause 11: The method according to any one of clauses 1 to 10 further includes: receiving a report from each of the multiple participating devices; and configuring constraints for updating partitions in the machine learning model for each corresponding participating device among the multiple participating devices based on the received report.

[0122] Clause 12: A method according to clause 11, wherein: the one or more participating devices include one or more user equipment (UE) in a wireless communication system, the machine learning model includes a model for predicting parameters for wireless communication between a network entity and the user equipment (UE), and the data at each corresponding device includes one or more radio measurements at each corresponding device.

[0123] Clause 13: A method as described in any of clauses 1 to 12, wherein updating the machine learning model based on the received updates includes aggregating weight updates from each participating device in the plurality of participating devices.

[0124] Clause 14: A method as set forth in any of clauses 1 to 13, wherein the method is performed by a network entity in a wireless communication system, and wherein the plurality of participating devices comprises user equipment (UE) served by the network entity.

[0125] Clause 15: A computer-implemented method comprising: receiving from a server information defining orthogonal partitions to be updated in a machine learning model and constraints for updating the orthogonal partitions, wherein the orthogonal partitions include partitions for a first participating device in a federated learning scheme that are decorrelated with partitions for a second participating device in the federated learning scheme; updating the orthogonal partitions in the machine learning model based on local data; and sending to the server information defining the updated orthogonal partitions in the machine learning model.

[0126] Clause 16: The method of clause 15, further comprising sending one or more reports to the server, wherein the constraint for updating the orthogonal partitions is based on the one or more reports.

[0127] Clause 17: The method of clause 16, wherein the one or more reports include generalized information about the local data.

[0128] Clause 18: A method as set forth in clause 16 or 17, wherein the one or more reports comprise radio measurements corresponding to a general classification of radio conditions at the device.

[0129] Clause 19: A method according to any one of clauses 15 to 18, wherein updating the orthogonal partitions in the machine learning model includes updating the orthogonal partitions based on a feature projection technique and the local data.

[0130] Clause 20: A method according to any one of clauses 15 to 19, wherein updating the orthogonal partitions in the machine learning model comprises updating the orthogonal partitions based on a gradient projection technique and the local data.

[0131] Clause 21: A method as described in any of clauses 15 to 20, wherein the method is performed by a user equipment (UE) in a wireless communication system, and wherein the server is associated with a network entity in the wireless communication system serving the UE.

[0132] Clause 22: An apparatus comprising: a memory having executable instructions stored thereon; and a processor configured to execute the executable instructions so that the apparatus performs the method according to any one of clauses 1 to 21.

[0133] Clause 23: An apparatus comprising: means for performing the method according to any one of clauses 1 to 21.

[0134] Clause 24: A non-transitory computer readable medium having stored thereon instructions which, when executed by a processor, perform the method of any one of clauses 1 to 21.

[0135] Clause 25: A computer program product embodied on a computer readable storage medium, the computer program product comprising code for performing the method according to any one of clauses 1 to 21.

[0136] Additional considerations

[0137] The previous description is provided to enable any person skilled in the art to practice the various embodiments described herein. The examples discussed herein are not limitations on the scope, applicability, or embodiments set forth in the claims. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the universal principles defined herein may be applied to other embodiments. For example, without departing from the scope of the present disclosure, the functions and arrangements of the elements discussed may be changed. Various examples may omit, replace, or add various procedures or components as appropriate. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. In addition, the features described for some examples may be combined in some other examples. For example, any number of aspects set forth herein may be used to implement a device or practice method. In addition, the scope of the present disclosure is intended to cover such devices or methods practiced using other structures, functionality, or structures and functionality that are supplementary or alternative to the various aspects of the present disclosure set forth herein. It should be understood that any aspect of the disclosure disclosed herein may be embodied by one or more elements of the present invention.

[0138] As used herein, the word “exemplary” means “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects.

[0139] As used herein, a phrase referring to "at least one of" a list of items refers to any combination of those items (including single members). For example, "at least one of a, b, or c" is intended to cover a, b, c, ab, ac, bc, and abc, as well as any combination with multiple identical elements (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbc, cc, and ccc, or any other ordering of a, b, and c).

[0140] As used herein, the term "determining" encompasses a wide variety of actions. For example, "determining" may include calculating, computing, processing, deriving, investigating, searching (e.g., searching in a table, database, or other data structure), ascertaining, and the like. Furthermore, "determining" may include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), etc. Furthermore, "determining" may include resolving, selecting, choosing, establishing, etc.

[0141] The method disclosed herein includes one or more steps or actions for implementing the method. The steps and / or actions of the method can be interchangeable with each other without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of specific steps and / or actions can be modified without departing from the scope of the claims. In addition, the various operations of the method described above can be performed by any suitable component capable of performing the corresponding function. The component may include various hardware and / or software components and / or modules, including but not limited to circuits, application specific integrated circuits (ASICs) or processors. Typically, in the case of operations illustrated in the drawings, those operations may have corresponding corresponding components with similar numbers plus functional components.

[0142] The following claims are not intended to be limited to the embodiments shown herein, but should be granted the full scope consistent with the language of the claims. Within the claims, unless otherwise specified, the reference to the singular element is not intended to mean "one and only one", but "one or more". Unless otherwise specified, the term "some" refers to one or more. Any claim element should not be interpreted according to the provisions of 35 U.S.C. § 112 (f), unless the phrase "parts for..." is used to explicitly record the element, or in the case of a method claim, the phrase "step for..." is used to record the element. All structural and functional equivalents of the elements of various aspects described throughout the present disclosure that are known or will be known later to a person of ordinary skill in the art are expressly incorporated herein by reference and are intended to be covered by the claims. In addition, nothing disclosed herein is intended to be dedicated to the public, regardless of whether such disclosure is explicitly recorded in the claims.

Claims

1. A computer-implemented method comprising: Divide the machine learning model into multiple partitions; sending, to each respective one of a plurality of participating devices in a federated learning scheme, a request to update a respective one of the plurality of partitions in the machine learning model based on unique data at the respective one of the participating devices; receiving updates to one or more partitions in the machine learning model from the plurality of participating devices; as well as The machine learning model is updated based on the received updates.

2. The method of claim 1, wherein partitioning the machine learning model into the plurality of partitions comprises partitioning the machine learning model into a common subnetwork and one or more non-common subnetworks.

3. The method of claim 1 , wherein partitioning the machine learning model into the plurality of partitions comprises orthogonalizing the partitions. The method of claim 3 , wherein orthogonalizing the partitions comprises generating the plurality of partitions based on Gram-Schmidt orthogonalization. The method of claim 3 , wherein orthogonalizing the partitions comprises generating the plurality of partitions based on singular value decomposition. The method of claim 3 , wherein orthogonalizing the partitions comprises generating the plurality of partitions based on a Cholesky decomposition.

7. The method according to claim 1, wherein partitioning the machine learning model comprises generating the multiple partitions in the machine learning model based on a feature projection technique.

8. The method according to claim 7, wherein: Each partition of the plurality of partitions is associated with a set of weights; A first partition of the plurality of partitions comprises a common partition having a common set of weights; and A plurality of second ones of the plurality of partitions include partitions having weights associated with one of the plurality of defined scenarios for which the machine learning model was trained.

9. The method according to claim 1, wherein partitioning the machine learning model comprises generating the multiple partitions in the machine learning model based on a gradient projection technique.

10. The method according to claim 9, wherein: The plurality of partitions include null spaces and non-null spaces, and The request to update the one or more of the plurality of partitions comprises a request to update a subspace in the non-zero space.

11. The method according to claim 1, further comprising: receiving a report from each participating device of the plurality of participating devices; as well as Constraints for updating partitions in the machine learning model are configured for each respective one of the plurality of participating devices based on the received report of the respective participating device.

12. The method according to claim 11, wherein: The one or more participating devices include one or more user equipment (UE) in a wireless communication system, The machine learning model includes a model for predicting parameters for wireless communications between a network entity and a user equipment (UE), and The data at each respective device includes one or more radio measurements at each respective device.

13. The method of claim 1, wherein updating the machine learning model based on the received updates comprises aggregating weight updates from each of the plurality of participating devices.

14. The method of claim 1, wherein the method is performed by a network entity in a wireless communication system, and wherein the plurality of participating devices comprises user equipment (UE) served by the network entity.

15. A computer-implemented method comprising: receiving, from a server, information defining orthogonal partitions to be updated in a machine learning model and constraints for updating the orthogonal partitions, wherein the orthogonal partitions include partitions for a first participating device in a federated learning scheme, the partitions being decorrelated with partitions for a second participating device in the federated learning scheme; updating the orthogonal partitions in the machine learning model based on local data; as well as Information defining updated orthogonal partitions in the machine learning model is sent to the server.

16. The method of claim 15, further comprising sending one or more reports to the server, wherein the constraints for updating the orthogonal partitions are based on the one or more reports. The method of claim 16 , wherein the one or more reports include generalized information about the local data.

18. The method of claim 16, wherein the one or more reports include radio measurements corresponding to a general classification of radio conditions at a device.

19. The method of claim 15, wherein updating the orthogonal partitions in the machine learning model comprises updating the orthogonal partitions based on a feature projection technique and the local data.

20. The method of claim 15, wherein updating the orthogonal partitions in the machine learning model comprises updating the orthogonal partitions based on a gradient projection technique and the local data.

21. The method of claim 15, wherein the method is performed by a user equipment (UE) in a wireless communication system, and the server is associated with a network entity serving the UE in the wireless communication system.

22. A processing system comprising: a memory including computer-executable instructions stored thereon; and one or more processors configured to execute the computer-executable instructions to cause the processing system to: Divide the machine learning model into multiple partitions; sending, to each respective one of a plurality of participating devices in a federated learning scheme, a request to update a respective one of the plurality of partitions in the machine learning model based on unique data at the respective one of the participating devices; receiving updates to one or more partitions in the machine learning model from the plurality of participating devices; as well as The machine learning model is updated based on the received updates.

23. The processing system of claim 22, wherein to partition the machine learning model into the plurality of partitions, the one or more processors are configured to: dividing the machine learning model into a common subnetwork and one or more non-common subnetworks, or The partitions are orthogonalized.

24. The processing system of claim 23, wherein to orthogonalize the partitions, the one or more processors are configured to generate the plurality of partitions based on one or more of Gram-Schmidt orthogonalization, singular value decomposition, or Cholesky decomposition.

25. The processing system of claim 22, wherein to partition the machine learning model, the one or more processors are further configured to generate the plurality of partitions in the machine learning model based on a feature projection technique, Each partition of the plurality of partitions is associated with a set of weights, A first partition of the plurality of partitions comprises a common partition having a common set of weights, and A plurality of second ones of the plurality of partitions include partitions having weights associated with one of the plurality of defined scenarios for which the machine learning model was trained.

26. The processing system of claim 22, wherein: To partition the machine learning model, the one or more processors are further configured to generate the plurality of partitions in the machine learning model based on a gradient projection technique, The plurality of partitions include null spaces and non-null spaces, and The request to update the one or more of the plurality of partitions comprises a request to update a subspace in the non-zero space.

27. The processing system of claim 22, wherein the one or more processors are configured to: receiving a report from each of the plurality of participating devices; and Constraints for updating partitions in the machine learning model are configured for each respective one of the plurality of participating devices based on the received report of the respective participating device.

28. The processing system of claim 27, wherein: The plurality of participating devices include one or more user equipment (UE) in a wireless communication system, The machine learning model includes a model for predicting parameters for wireless communications between a network entity and a user equipment (UE), and The data at each respective device includes one or more radio measurements at each respective device.

29. The processing system of claim 22, wherein to update the machine learning model, the one or more processors are configured to aggregate weight updates from each of the plurality of participating devices.

30. A processing system comprising: a memory having computer-executable instructions stored thereon; and one or more processors configured to execute the computer-executable instructions to: receiving, from a server, information defining orthogonal partitions to be updated in a machine learning model and constraints for updating the orthogonal partitions, wherein the orthogonal partitions include partitions for a first participating device in a federated learning scheme, the partitions being decorrelated with partitions for a second participating device in the federated learning scheme; updating the orthogonal partitions in the machine learning model based on local data; as well as Information defining updated orthogonal partitions in the machine learning model is sent to the server.

Images