Systems and methods for performing vertically associative learning

Abstract

Methods and servers for performing vertical federated learning (VFL) over a communications network are disclosed. Some methods include encoder and data selection, encoder customization, and VFL configuration. Encoder and data selection includes receiving a VFL request and parsing it to determine data requirements and encoder requirements, selecting a first encoder and a second encoder based on the encoder requirements, and selecting data sources for the first encoder and the second encoder based on the data requirements. Encoder customization includes selecting a first encoder and a second encoder and customizing the first encoder by muting portions of the first encoder. VFL configuration includes receiving labeled data information and an indication of a loss function, configuring an evaluator using the labeled data information and the indication of the loss function, configuring the encoder output to a combined classifier, and configuring a privacy router.

Description

[Technical Field]

[0001] FIELD OF THE DISCLOSURE The present disclosure relates to the field of federated learning, and more particularly to systems and methods for performing vertical federated learning. [Background technology]

[0002] In telecommunications, "6G" refers to the sixth-generation standard currently under development. The 6G standard is for wireless communications technology that supports cellular data networks. 6G networks will exhibit more heterogeneity than their predecessors, such as 5G networks, and will support applications that go beyond current mobile usage scenarios, such as virtual and augmented reality, ubiquitous instant communications, universal intelligence, and the Internet of Things (IoT). Mobile network operators can adopt flexible, decentralized business models for 6G networks using local spectrum licensing, spectrum sharing, infrastructure sharing, and intelligent automated management supported by mobile edge computing, artificial intelligence (AI), short packet communications, and blockchain technologies.

[0003] Federated learning (FL) is one of several machine learning (ML) techniques that can be leveraged to enable 6G network capabilities. Broadly speaking, FL is an ML technique that trains one or more AI models across multiple distributed participants (e.g., edge devices or servers) using their local datasets without any raw data exchange between them. This approach contrasts with traditional "centralized" ML techniques, which either upload all local datasets to a single practitioner or assume the sharing of raw local data samples among distributed participants.

[0004] In other words, FL allows multiple participants to build a coupled AI model without sharing data, thus addressing important issues such as data privacy, data security, data access rights, and access to heterogeneous data.There are two main approaches to implementing FL: horizontal federated learning (HFL) and vertical federated learning (VFL).

[0005] Roughly speaking, HFL is used for datasets with multiple participants that share the same feature space but different ID spaces. On the one hand, the feature space describes the set of attributes / dimensions of each data entity / sample in a dataset. On the other hand, the ID space describes the identity of each data entity. For example, in a banking dataset, the feature space could be (account balance, mortgage amount, credit score, etc.) and the ID space would be the user list of all accounts.

[0006] In contrast to HFL, VFL is used for datasets with multiple participants that share the same ID space but different feature spaces. For example, Bell TM and Amazon TM has many overlapping entities in ID space (e.g., the same user in the real world) but with different data features (e.g., Bell TM Mobile usage features hosted by Amazon TM If one or both of these participants want to train a joint ML model to assess a given user's credit score by considering the influence of both mobile usage features and online shopping features, VFL can be used to train the joint ML model without leaking raw data and local model information between the two participants.

[0007] Because VFL participants may not belong to the same business or application category, their local ML models usually differ in terms of structure and hyperparameters. Therefore, there is a need for a better VFL architecture that can be deployed for and used by a wide variety of participants. Summary of the Invention

[0008] The purpose of the present technology is to ameliorate at least some of the disadvantages present in conventional VFL architectures.

[0009] Typical VFL Architecture 1, a typical VFL architecture 100 is shown having a first participant with a first ML model 102 and a second participant with a second ML model 104. The first ML model 102 and the second ML model 104 together form a combined model 110.

[0010] In contrast to the HFL, two specific functions may be required to train and use the VFL architecture 100: intermediate result exchange 106 between the ML model and the collaborator 108.

[0011] According to the VFL paradigm, in each training step and each in-use step of the combined model 110, the intermediate outputs or computation results of the first local ML model 102 and the second local ML model 104 may first need to be encrypted by homomorphic encryption and then transmitted to the other of the first local ML model 102 and the second local ML model 104 so that they perform local computations. Each of the first local ML model 102 and the second local ML model 104 combined in the intermediate result exchange process 106 does not decrypt the intermediate results from the other of the first local ML model 102 and the second local ML model 104, i.e., they perform local computations on the homomorphically encrypted data.

[0012] To ensure data confidentiality during the training process, a collaborator 108 is required to perform VFL. For example, the collaborator 108 may be a third-party entity trusted by the first and second participants. The collaborator 108 may be embodied as a secure computing node, as a non-limiting example. The collaborator 108 creates a cryptographic pair and sends a public key to the first and second participants to enable the encrypted intermediate result exchange process 106. Once the intermediate result exchange process 106 is completed for each training and usage step of the combined model 110, the first and second participants send their encrypted outputs and locally generated masks to the collaborator 108. The collaborator 208 decrypts the masked outputs with its private key, calculates the decrypted gradients and losses, and sends this information back to each participant. Each participant unmasks the gradients and losses from the collaborator 208 and updates its respective local ML model parameters accordingly.

[0013] Developers recognize that VFL may need to be supported by the NET4AI architecture to provide native support for the training and in-use phases of AI-based computing services. Generally, NET4AI is a service-oriented architecture for 6G wireless systems that provides end-to-end support for AI applications from the deployment to operational phases. The NET4AI architecture is generally described in the paper entitled "Nine Challenges in Artificial Intelligence and Wireless Communications for 6G" by Wen Tong and Geoffry Ye Li, published in 2021, DOI 10.1109 / MWC.006.2100543, the entire contents of which are incorporated herein by reference. Supporting VFL in the NET4AI architecture is a challenging task.

[0014] The developers recognize that in the existing VFL paradigm, a VFL customer has access to information indicating the number of other VFL participants, the data characteristics of other VFL participants, the datasets of other VFL participants, the labeled data location / ownership, and the encoder structure of other VFL participants. However, when implementing an automated VFL system, a VFL customer may not have access to information about other VFL participants. Therefore, the developers of the present technology recognize that a system and method are needed that allows the NET4AI architecture to select datasets and encoders that have the functionality required for a VFL customer.

[0015] The developers also recognize that in the existing VFL paradigm, labeled data is provided by one single VFL participant, and the loss function can be split without violating homomorphic encryption. As a result, a privacy-preserving intermediate result exchange process can be performed to compute the combined loss. However, in some cases, the labeled data may be at least partially owned by multiple participants. Also, some loss functions may differ from the linear polynomial functions that ensure homomorphic encryption after computation. Therefore, the developers recognize that it may be necessary to configure a combined classifier and evaluator for different encoders. This may simplify the interaction between local classifiers of different VFL participants, and / or support loss functions that violate homomorphic encryption, and / or allow labeled data to be provided by multiple data sources.

[0016] The developers also recognize that in existing VFL paradigms, the encoder structure maintained by the corresponding AI enabler remains the same. These solutions lack procedures for customizing the encoder structure in specific cases, such as when multiple encoders are trained collaboratively to form a combined model. However, when applying VFL in the NET4AI architecture or other architectures, the training data of selected participants may have overlapping data features.

[0017] For example, in a VFL model with a "bank" participant and a "mobile operator" participant, the data of both participants may have a common feature named "mobile bill payment record." In this example, when a VFL combined model is formed with the original encoders of the two participants, the weight of the feature "mobile bill payment record" in the combined output may increase significantly due to overlap in both model inputs. In the same example, the weights of other features may be reduced in the combined output without being duplicated.

[0018] The developers recognize that while it is possible to mitigate the problem of relatively increased weights of overlapping features by setting strict convergence requirements, this may significantly increase overall training costs. Thus, the developers recognize that customization of the encoder structure may be desirable to reduce drawbacks such as relatively increased weights and slow convergence speed caused by overlapping data features.

[0019] A typical encoder-classifier model 2, there is shown a simplified representation of an exemplary local ML model 200. For example, the local ML model 200 may be a deep neural network (DNN) executed by each computing service (e.g., participant).

[0020] The local ML model 200 includes an encoder 202 and a classifier 204, each of which includes multiple layers. As seen in Figure 2, layers 2-4 are sometimes referred to as "hidden" layers. The encoder 202 includes an input layer 206 and multiple hidden layers, and the classifier 204 includes the remaining hidden layers and an output layer.

[0021] The encoder 202 captures low-level features of the local ML model 200, which can be shared with other local ML models implemented in a similar manner to the local ML model 200 (e.g., local ML models aimed at solving a similar problem). The classifier 204 captures high-level features of the local ML model 200, which are typically goal-specific. For the input layer 206 of the encoder 202, each neuron corresponds to one feature of the input data. The dimension of the input layer 206, i.e., the number of neurons in the input layer 206, is equal to the number of features associated with the input data of the local ML model 200.

[0022] The developers of the present technology recognize that when performing Auto-VFL, the encoder can be a network-supplied encoder, i.e., provided by the network and implemented by a network entity called an "AI enabler." During Auto-VFL, it is contemplated that the classifier may be at least one of (i) a customer-supplied classifier, i.e., a local classifier provided by a participant of the corresponding ML enabler, and (ii) a network-supplied classifier, i.e., a combined classifier provided by an orchestrator within the NET4AI architecture or other architecture. It is contemplated that both customer-supplied classifiers and network-supplied classifiers can be implemented via service functions of a computing service without departing from the scope of the present technology.

[0023] The developers of this technology recognize that neural network (NN) pruning techniques can be used to reduce the size of a NN by "muting" certain neurons or links of the NN while maintaining the same or similar performance as a corresponding full-size NN. Muting a neuron can be achieved by setting its output equal to zero with respect to either input. Muting a link can be achieved by setting the link's weight value to zero.

[0024] Typically, during NN pruning, neurons in a given input layer are not muted. However, in some embodiments of the present technology, at least some neurons in at least some input layers of each VFL participant's encoder may be muted to reduce the detrimental effects caused by overlapping data features. The process of modifying the structure of an encoder by muting specific neurons or links in a given layer (potentially the input layer) is sometimes referred to as an encoder customization procedure or encoder pruning.

[0025] In some embodiments of the present technology, a VFL participant includes an encoder, a local classifier (if available), and a corresponding data source. A VFL participant's data source may include multiple data sets implemented at different network locations. All data entities provided by a VFL participant's data source may have the same format and characteristics. Each encoder is associated with an application category, so a VFL participant's data source may be associated with the same application category as the VFL participant's encoder.

[0026] This technology In a first broad aspect of the present technology, there is provided a method including: receiving, by an encoder selector, a Vertical Federated Learning (VFL) request from a VFL customer; determining, by the encoder selector, data requirements and encoder requirements for solving a VFL task in accordance with the VFL request, the VFL task being divisible into multiple subtasks, the multiple subtasks including at least a first subtask and a second subtask; selecting, by the encoder selector, a first encoder and a second encoder from a pool of encoders in a communication network based on the encoder requirements, the first encoder configured to process a first set of features for solving the first subtask, and the second encoder configured to process a second set of features for solving the second subtask; and selecting, by the encoder selector, a first data source for the first encoder and a second data source for the second encoder based on the data requirements, the selected data sources being from the pool of data sources, the first data source including the first set of features, and the second data source including the second set of features.

[0027] In some embodiments of the method, the data requirements indicate requirements for selecting a first data source for solving the first subtask and a second data source for solving the second subtask.

[0028] In some embodiments of the method, the encoder requirements indicate a first encoder structure to be used to solve the first subtask and a second encoder structure to be used to solve the second subtask.

[0029] In some embodiments of the method, the method further comprises dividing, by the encoder selector, the VFL task into a plurality of subtasks.

[0030] In some embodiments of the method, selecting the first encoder includes determining, by the encoder selector, an application category for the first subtask based on the encoder requirements, and the application category is stored in memory in association with the first encoder.

[0031] In some embodiments of the method, selecting the first encoder includes identifying, by an encoder selector, a pool of encoders having the first encoder structure, and selecting, by the encoder selector, the first encoder from the pool of encoders having the first encoder structure.

[0032] In some embodiments of the method, selecting the first data source and the second data source includes sending, by the encoder selector, a request for data source information to the data source manager, the request including data requirements; receiving, by the encoder selector, data source information from the data source manager indicating the quality and characteristics of the data contained in each from the pool of data sources; and selecting, by the encoder selector, a first data source for the first encoder and a second data source for the second encoder using the data source information.

[0033] In some embodiments of the method, the first data source and the second data source are associated with respective IDs, and the method further includes sending, by the encoder selector, the respective IDs of the first data source and the second data source to the data source manager, and receiving, by the encoder selector, from the data source manager, network locations of the first data source and the second data source.

[0034] In a second broad aspect of the present technology, a method is provided that includes customizing, by an encoder customizer, a first encoder by muting a portion of the first encoder, thereby generating a customized first encoder. The first encoder and a second encoder are selected from a pool of encoders in a communication network based on encoder requirements, the encoder requirements being determined for solving a Vertical Federated Learning (VFL) task according to a VFL request. The VFL task is divisible into multiple subtasks, the multiple subtasks including at least a first subtask and a second subtask. The first encoder is configured to process a first set of features for solving the first subtask, and the second encoder is configured to process a second set of features for solving the second subtask.

[0035] In some embodiments of the method, the customizing includes determining overlapping features between the first set of features and the second set of features, and muting a portion of the first encoder that processes the overlapping features.

[0036] In some embodiments of the method, customizing includes determining, by an encoder customizer, that the first encoder is to be customized.

[0037] In some embodiments of the method, determining, by the encoder customizer, that the first encoder should be customized includes determining, by the encoder customizer, that the first encoder should be customized based on data feature weights, the data feature weights being received from a reference AI enabler.

[0038] In some embodiments of the method, customizing the first encoder includes muting, by the encoder customizer, a portion of the first encoder that processes the overlapping features, wherein the muting includes muting at least one of a neuron of the first encoder and a link of the first encoder.

[0039] In a third broad aspect of the present technology, there is provided a method including: configuring, with a Vertical Federated Learning (VFL) configurator, at least one of an evaluator and a combined classifier; and configuring, with the VFL configurator, a privacy router to determine at least one of a data exchange path between the combined classifier and the evaluator, a labeled data merging path between the evaluator and one or more data sources, and a backpropagation path between the combined classifier and customized first and second encoders for backpropagating gradient values, wherein the combined classifier, the evaluator, and the privacy router are configured to perform VFL.

[0040] In some embodiments of the method, configuring the evaluator includes receiving, by a VFL configurator, labeled data information from an encoder selector; determining, by the VFL configurator, a loss function to be used to perform the VFL; and configuring, by the VFL configurator, the evaluator using the labeled data information and the loss function.

[0041] In some embodiments of the method, configuring the evaluator includes configuring, by a VFL configurator, rules for combining the labeled data resulting from the data sources selected for the first encoder and the second encoder with the loss function.

[0042] In some embodiments of the method, configuring the combined classifier includes receiving, by a VFL configurator, information indicating output layer structures of the first encoder and the second encoder from an encoder customizer, and configuring, by the VFL configurator, encoder outputs to the combined classifier based on the output layer structures of the first encoder and the second encoder.

[0043] In some embodiments of the method, the second encoder is a customized second encoder.

[0044] In a fourth broad aspect of the present technology, there is provided an encoder selector configured to: receive a Vertical Federated Learning (VFL) request from a VFL customer; determine data requirements and encoder requirements for solving a VFL task according to the VFL request, the VFL task being divisible into a plurality of subtasks, the plurality of subtasks including at least a first subtask and a second subtask; select a first encoder and a second encoder from a pool of encoders in a communication network based on the encoder requirements, the first encoder configured to process a first set of features for solving the first subtask and the second encoder configured to process a second set of features for solving the second subtask; and select a first data source for the first encoder and a second data source for the second encoder based on the data requirements, the selected data sources being from the pool of data sources, the first data source including the first set of features and the second data source including the second set of features.

[0045] In some embodiments of the encoder selector, the data requirements indicate requirements for selecting a first data source for solving the first subtask and a second data source for solving the second subtask.

[0046] In some embodiments of the encoder selector, the encoder requirements indicate a first encoder structure to be used to solve the first subtask and a second encoder structure to be used to solve the second subtask.

[0047] In some embodiments of the encoder selector, the encoder selector is further configured to divide the VFL task into multiple subtasks.

[0048] In some embodiments of the encoder selector, selecting the first encoder includes the encoder selector being configured to determine an application category for the first subtask based on the encoder requirements, and the application category being stored in memory in association with the first encoder.

[0049] In some embodiments of the encoder selector, selecting the first encoder includes the encoder selector being configured to identify a pool of encoders having the first encoder structure and select the first encoder from the pool of encoders having the first encoder structure.

[0050] In some embodiments of the encoder selector, selecting the first data source and the second data source includes the encoder selector being configured to send a request for data source information to a data source manager, the request including data requirements, receive data source information from the data source manager indicating the quality and characteristics of the data contained in each from a pool of data sources, and use the data source information to select a first data source for the first encoder and a second data source for the second encoder.

[0051] In some embodiments of the encoder selector, the first data source and the second data source are associated with respective IDs, and the encoder selector is further configured to send the respective IDs of the first data source and the second data source to a data source manager and receive, from the data source manager, network locations of the first data source and the second data source.

[0052] In a fifth broad aspect of the present technology, an encoder customizer is provided that is configured to customize a first encoder by muting a portion of the first encoder, thereby generating a customized first encoder. The first encoder and a second encoder are selected from a pool of encoders in a communication network based on encoder requirements, the encoder requirements being determined for solving a Vertical Federated Learning (VFL) task according to a VFL request. The VFL task is divisible into multiple subtasks, and the multiple subtasks include at least a first subtask and a second subtask. The first encoder is configured to process a first set of features for solving the first subtask, and the second encoder is configured to process a second set of features for solving the second subtask.

[0053] In some embodiments of the encoder customizer, the customizing includes the encoder customizer being configured to determine overlapping features between the first set of features and the second set of features, and muting a portion of the first encoder that processes the overlapping features.

[0054] In some embodiments of the encoder customizer, customizing includes the encoder customizer being configured to determine that the first encoder is to be customized.

[0055] In some embodiments of the encoder customizer, determining that the first encoder should be customized includes the encoder customizer being configured to determine that the first encoder should be customized based on data feature weights, the data feature weights being received from a reference AI enabler.

[0056] In some embodiments of the encoder customizer, customizing the first encoder includes the encoder customizer being configured to mute a portion of the first encoder that processes the overlapping features, and the muting includes muting at least one of a neuron of the first encoder and a link of the first encoder.

[0057] In a sixth broad aspect of the present technology, there is provided a Vertical Federated Learning (VFL) configurator configured to configure at least one of an evaluator and a combined classifier, and to configure a privacy router to determine at least one of a data exchange path between the combined classifier and the evaluator, a labeled data merging path between the evaluator and one or more data sources, and a backpropagation path between the combined classifier and customized first and second encoders for backpropagating gradient values, wherein the combined classifier, the evaluator, and the privacy router are configured to perform Vertical Federated Learning (VFL).

[0058] In some embodiments of the VFL configurator, configuring the evaluator includes configuring the VFL configurator to receive labeled data information from an encoder selector, determine a loss function to be used to perform the VFL, and configure the evaluator using the labeled data information and the loss function.

[0059] In some embodiments of the VFL configurator, configuring the evaluator includes configuring the VFL configurator to configure combination rules for the loss function and the labeled data resulting from the selected data sources for the first encoder and the second encoder.

[0060] In some embodiments of the VFL configurator, configuring the combined classifier includes configuring the VFL configurator to receive information indicating output layer structures of the first encoder and the second encoder from an encoder customizer, and to configure the encoder output to the combined classifier based on the output layer structures of the first encoder and the second encoder.

[0061] In some embodiments of the VFL configurator, the second encoder is a customized second encoder.

[0062] In a seventh broad aspect of the present technology, a method includes receiving, by an encoder selector, a Vertical Federated Learning (VFL) request from a VFL customer; determining, by the encoder selector, data requirements and encoder requirements for solving a VFL task in accordance with the VFL request, the VFL task being divisible into a plurality of subtasks, the plurality of subtasks including at least a first subtask and a second subtask; selecting, by the encoder selector, a first encoder and a second encoder from a pool of encoders in a communication network based on the encoder requirements, the first encoder configured to process a first set of features for solving the first subtask, and the second encoder configured to process a second set of features for solving the second subtask; and selecting, by the encoder selector, a first data source for the first encoder and a second data source for the second encoder based on the data requirements, the selected data sources being selected from the pool of data sources. a first data source including a first set of features and a second data source including a second set of features; customizing, by an encoder customizer, the first encoder by muting a portion of the first encoder, thereby generating a customized encoder; configuring, by a VFL configurator, at least one of an evaluator and a combined classifier; and configuring, by the VFL configurator, a privacy router to determine at least one of a data exchange path between the combined classifier and the evaluator, a labeled data merging path between the evaluator and one or more of the selected data sources, and a backpropagation path between the combined classifier and each of the customized first and second encoders for backpropagating gradient values, wherein the combined classifier, the evaluator, and the privacy router are configured to execute the VFL.

[0063] In some embodiments of the method, the data requirements indicate requirements for selecting a first data source for solving the first subtask and a second data source for solving the second subtask.

[0064] In some embodiments of the method, the encoder requirements indicate a first encoder structure to be used to solve the first subtask and a second encoder structure to be used to solve the second subtask.

[0065] In some embodiments of the method, the method further comprises dividing, by the encoder selector, the VFL task into a plurality of subtasks.

[0066] In some embodiments of the method, selecting the first encoder includes determining, by the encoder selector, an application category for the first subtask based on the encoder requirements, and the application category is stored in memory in association with the first encoder.

[0067] In some embodiments of the method, selecting the first encoder includes identifying, by an encoder selector, a pool of encoders having the first encoder structure, and selecting, by the encoder selector, the first encoder from the pool of encoders having the first encoder structure.

[0068] In some embodiments of the method, selecting the first data source and the second data source includes sending, by the encoder selector, a request for data source information to the data source manager, the request including data requirements; receiving, by the encoder selector, data source information from the data source manager indicating the quality and characteristics of the data contained in each from the pool of data sources; and selecting, by the encoder selector, a first data source for the first encoder and a second data source for the second encoder using the data source information.

[0069] In some embodiments of the method, the first data source and the second data source are associated with respective IDs, and the method further includes transmitting, by the encoder selector, the respective IDs of the first data source and the second data source to the data source manager, and receiving, by the encoder selector, from the data source manager, network locations of the first data source and the second data source.

[0070] In some embodiments of the method, the customizing includes determining overlapping features between the first set of features and the second set of features, and muting a portion of the first encoder that processes the overlapping features.

[0071] In some embodiments of the method, customizing includes determining, by an encoder customizer, that the first encoder is to be customized.

[0072] In some embodiments of the method, determining, by the encoder customizer, that the first encoder should be customized includes determining, by the encoder customizer, that the first encoder should be customized based on data feature weights, the data feature weights being received from a reference AI enabler.

[0073] In some embodiments of the method, customizing the first encoder includes muting, by the encoder customizer, a portion of the first encoder that processes the overlapping features, and the muting includes muting at least one of a neuron of the first encoder and a link of the first encoder.

[0074] In some embodiments of the method, configuring the evaluator includes receiving, by a VFL configurator, labeled data information from an encoder selector; determining, by the VFL configurator, a loss function to be used to perform the VFL; and configuring, by the VFL configurator, the evaluator using the labeled data information and the loss function.

[0075] In some embodiments of the method, configuring the evaluator includes configuring, by a VFL configurator, rules for combining the labeled data resulting from the data sources selected for the first encoder and the second encoder with the loss function.

[0076] In some embodiments of the method, configuring the combined classifier includes receiving, by a VFL configurator, information indicating output layer structures of the first encoder and the second encoder from an encoder customizer, and configuring, by the VFL configurator, encoder outputs to the combined classifier based on the output layer structures of the first encoder and the second encoder.

[0077] In some embodiments of the method, the second encoder is a customized second encoder.

[0078] In an eighth broad aspect of the present technology, a method includes receiving, by an encoder selector, a Vertical Federated Learning (VFL) request from a VFL customer; determining, by the encoder selector, data requirements and encoder requirements for solving a VFL task according to the VFL request, the VFL task being divisible into a plurality of subtasks, the plurality of subtasks including at least a first subtask and a second subtask; selecting, by the encoder selector, a first encoder and a second encoder from a pool of encoders in a communication network based on the encoder requirements, the first encoder configured to process a first set of features for solving the first subtask and the second encoder configured to process a second set of features for solving the second subtask; and selecting, by the encoder selector, a first data source for the first encoder and a second data source for the second encoder based on the data requirements, the selected data sources being from the pool of data sources; A system is provided in which the first data source includes a first set of features and the second data source includes a second set of features; an encoder customizer customizes the first encoder by muting portions of the first encoder, thereby generating a customized encoder; a VFL configurator configures at least one of an evaluator and a combined classifier; and the VFL configurator configures a privacy router to determine at least one of a data exchange path between the combined classifier and the evaluator, a labeled data merging path between the evaluator and one or more of the selected data sources, and a backpropagation path between the combined classifier and each of the customized first and second encoders for backpropagating gradient values, wherein the combined classifier, the evaluator, and the privacy router are configured to execute the VFL.

[0079] In some embodiments of the system, the data requirements indicate requirements for selecting a first data source for solving the first subtask and a second data source for solving the second subtask.

[0080] In some embodiments of the system, the encoder requirements indicate a first encoder structure to be used to solve the first subtask and a second encoder structure to be used to solve the second subtask.

[0081] In some embodiments of the system, the encoder selector is further configured to divide the VFL task into multiple subtasks.

[0082] In some embodiments of the system, selecting the first encoder includes the encoder selector being configured to determine an application category for the first subtask based on the encoder requirements, and the application category being stored in memory in association with the first encoder.

[0083] In some embodiments of the system, selecting the first encoder includes the encoder selector being configured to identify a pool of encoders having the first encoder structure and select the first encoder from the pool of encoders having the first encoder structure.

[0084] In some embodiments of the system, selecting the first data source and the second data source includes the encoder selector being configured to send a request for data source information to the data source manager, the request including data requirements, receive data source information from the data source manager indicating the quality and characteristics of the data contained in each from a pool of data sources, and use the data source information to select a first data source for the first encoder and a second data source for the second encoder.

[0085] In some embodiments of the system, the first data source and the second data source are associated with respective IDs, and the encoder selector is further configured to send the respective IDs of the first data source and the second data source to the data source manager and receive, from the data source manager, network locations of the first data source and the second data source.

[0086] In some embodiments of the system, the customizing includes the encoder customizer being configured to determine overlapping features between the first set of features and the second set of features, and muting a portion of the first encoder that processes the overlapping features.

[0087] In some embodiments of the system, the customizing includes the encoder customizer being configured to: obtain (i) encoder structure parameters of the first encoder and the second encoder; (ii) a position of a target AI enabler for the customized encoder; and (iii) a position of a reference AI enabler of the trained encoder; send a request for data feature weights of the trained encoder to the reference AI enabler; receive the data feature weights from the reference AI enabler; determine overlapping features between a first feature set of the first encoder and a second feature set of the second encoder; determine, based on the data feature weights, that the first encoder should be customized; and mute a portion of the first encoder that processes the overlapping features, wherein the muting includes muting at least one of a neuron of the first encoder and a link of the first encoder.

[0088] In some embodiments of the system, configuring the evaluator includes configuring a VFL configurator to receive labeled data information from an encoder selector, determine a loss function to be used to perform the VFL, and configure the evaluator using the labeled data information and the loss function.

[0089] In some embodiments of the system, configuring the evaluator includes configuring a VFL configurator to configure combination rules of the loss function and the labeled data resulting from the selected data sources for the first encoder and the second encoder.

[0090] In some embodiments of the system, configuring the combined classifier includes configuring the VFL configurator to receive information indicating output layer structures of the first encoder and the second encoder from an encoder customizer, and to configure the encoder output to the combined classifier based on the output layer structures of the first encoder and the second encoder.

[0091] In some embodiments of the system, the second encoder is a customized second encoder.

[0092] Implementations of the present technology each have at least one of the above-described objects and / or aspects, but not necessarily all of them. It should be understood that some aspects of the present technology that arise from attempts to achieve the above-described object may not meet that object and / or may meet other objects not specifically set forth herein.

[0093] Additional and / or alternative features, aspects, and advantages of implementations of the present technology will become apparent from the following description, the accompanying drawings, and the appended claims. [Brief explanation of the drawings]

[0094] Embodiments of the present disclosure will now be described, by way of example only, with reference to the accompanying drawings, in which: [Figure 1] 1 shows a schematic diagram of a typical VFL architecture with two participants. [Figure 2] A schematic diagram of a typical local ML model is shown. [Figure 3] 1 shows a schematic diagram of a possible automatic VFL architecture in some embodiments of the present technology; [Figure 4] 4 shows a schematic diagram of a privacy router of the automatic VFL architecture of FIG. 3. [Figure 5] 4 shows a schematic diagram of the encoder and data selection procedure performed by the encoder selector of the VFL architecture of FIG. 3. [Figure 6] 4 shows a schematic diagram of an encoder customization procedure performed by an encoder customizer of the VFL architecture of FIG. 3. [Figure 7] Two customization modes are shown for the encoder customization procedure in FIG. [Figure 8] 4 shows a schematic diagram of a VFL configuration procedure performed by a VFL configurator of the VFL architecture of FIG. 3. [Figure 9] Three VFL frameworks are presented, including traditional VFL, automatic VFL and stitched single-NN. [Figure 10] We show the performance comparison between conventional VFL and automatic VFL in Fig. 9 applied to two test datasets. [Figure 11] We show the performance comparison between automatic VFL and stitched single NN in Fig. 9 applied to two test datasets. DETAILED DESCRIPTION OF THE INVENTION

[0095] Broadly speaking, sixth-generation (6G) communication systems are end-to-end (E2E) systems that support AI-based services and applications in a variety of contexts and scenarios. For example, 6G network components can natively integrate communications, computing, and sensing capabilities to facilitate the transition from centralized intelligence in the cloud to universal intelligence on the "deep" edge. This integration can be enabled by what is referred to herein as a "Network for AI" (NET4AI) architecture. The NET4AI architecture may enable the delivery of AI as a Service (AIaaS). However, as will become apparent from the following description, at least some aspects of the present technology may be embodied in a wide variety of AIaaS architectures.

[0096] In the context of the present technology, the developers have developed a method and system for automating Vertical Federated Learning (VFL). Generally, VFL is a machine learning (ML) technique that trains a combined model using heterogeneous AI models and datasets belonging to different participants. The developers of the present technology have recognized that it may be beneficial to support automated VFL in the NET4AI architecture or other architectures.

[0097] According to some non-limiting embodiments of the present technology, an automated VFL method and system is provided for customers to use the data and computing resources of other customers, even if they belong to different organizations or even industries. As described in further detail herein below, VFL automation may be enabled using at least some of the following features: encoder and data selection, encoder customization, and VFL configuration.

[0098] In a further non-limiting embodiment of the present technology, an automatic VFL solution is provided that supports AI model customization. Broadly speaking, AI model customization is a feature of the automatic VFL solution for improving the scalability of AI models across heterogeneous devices and scenarios. As described in further detail herein below, AI model customization may be enabled via an encoder customization feature.

[0099] Automated VFL Architecture 3, there is shown a non-limiting embodiment of an automated VFL architecture 300. In this embodiment, automated VFL architecture 300 includes an application layer 302, a network layer 304, and a data layer 306.

[0100] Broadly speaking, the various layers of the auto-VFL architecture 300 are part of a framework used to describe the functionality of a networking system that implements a VFL. Different layers can be used to characterize the computing functions performed by one or more networking nodes (e.g., servers) to at least partially support interoperability between the hardware and software components of the auto-VFL structure 300.

[0101] In this embodiment, application layer 302 includes VFL customer 310. However, application layer 302 may include multiple VFL customers without departing from the scope of the present technology. In this embodiment, VFL customer 310 is configured to maintain combined classifier 314 and provide VFL requests, such as VFL request 312, to components of network layer 304.

[0102] In this embodiment, data layer 306 includes multiple data sources 342 potentially used during automated VFL. ID alignment functionality 344 may be enabled in data layer 306. Broadly speaking, a data source may be configured as a logical network entity storage for storing and / or maintaining the necessary raw data (as VFL model input) and labeled data (as benchmarks for calculating training loss) used in the present technology to train a VFL model. It is contemplated that an entire batch of raw data or labeled data required to train a VFL model may be provided by one or more data sources. In some embodiments, the data sources may be either centrally deployed at a network entity or deployed across multiple distributed entities without departing from the scope of the present technology.

[0103] In the illustrated non-limiting embodiment, the network layer 304 maintains an AI hyperparameter optimizer 320, multiple encoders 328, an evaluator 330, and a privacy router 332 (forwarding plane). In the context of the present technology, the AI ​​hyperparameter optimizer 320 is configured to perform one or more computer-implemented procedures to enable the following functions: encoder and data selection, encoder customization, and VFL configuration. In the illustrated non-limiting embodiment, the AI ​​hyperparameter optimizer 320 includes an encoder selector 322, an encoder customizer 324, and a VFL configurator 326 to enable the above functions, respectively. The components of the AI ​​hyperparameter optimizer 320 will now be described in turn.

[0104] Encoder Selector In the illustrated non-limiting embodiment, the encoder selector 322 is configured to receive a VFL request 312 from a VFL customer 310 via an interface between the application controller and the service manager (e.g., as defined in the NET4AI architecture). A VFL customer 310 is referred to as a user of the automated VFL, providing a VFL request 312 containing a VFL problem to be addressed by the network layer 304 and maintaining its own combined classifier 314 to compute a combined ML model output from multiple encoders 328. In some embodiments, the VFL customer 310 may be one of the VFL participants with its associated local encoder and dataset. In other embodiments, the VFL customer 310 may not have an associated local encoder or dataset and may request a combined model to solve one or more VFL problems, which may be considered VFL tasks.

[0105] In the illustrated non-limiting embodiment, the encoder selector 322 maps the VFL request 312 to data requirements and encoder structure requirements that may be needed to solve the VFL problem described in the VFL request 312. The data requirements include an indication of "mandatory" data features and labeled data requirements (e.g., minimum amount of labeled data, labeled data format, etc.) that may need to be provided by the VFL participant's data sources. The data requirements indicate the requirements for selecting a data source for solving the VFL problem.

[0106] The encoder requirements indicate the encoder structure that may be required to provide specific functionality for solving the VFL problem. For example, given a problem described as "traffic violation recognition from surveillance video," an indication of the encoder structure "CNN" may be provided to indicate that a CNN encoder must be included by the combined model to process the video data. The encoder selector 322 can interact with the data source manager over multiple rounds of communication to complete the encoder and data source selection procedure. How the encoder and data selection procedure are implemented in some embodiments of the present technology is described in further detail herein below.

[0107] Encoder Customizer In the illustrated non-limiting embodiment, once a set of encoders (with associated AI enablers) for a VFL problem has been selected, the encoder customizer 324 can be configured to “filter out” redundant data features for the selected encoders and determine customization parameters for each encoder. To calculate the customization parameters for each encoder, the weights of each data feature in the input layer of the corresponding encoder may be determined by the encoder customizer 324. The calculation process requires interaction between the encoder customizer 324 and other AI enablers that maintain trained encoders with the same structure as the corresponding encoder. In some cases, after achieving encoder customization for each selected encoder, the corresponding AI enabler may feed back the customized encoder features to the encoder customizer 324, thereby completing the encoder customization procedure.

[0108] VFL configuration In the illustrated non-limiting embodiment, when customized encoder features are provided by the encoder customizer 324, the VFL configurator 326 configures parameters for the combined classifier 314 and evaluator 330, as well as routing rules for data exchange between the combined classifier 314 and evaluator 330 and the multiple encoders 328. This configuration of parameters and routing rules is referred to herein as the VFL configuration procedure.

[0109] 4, the VFL configuration procedure includes a combined classifier 314, an evaluator 330, and a privacy router 332. In the illustrated non-limiting embodiment, the combined classifier 314 is implemented as a service function executed by a third-party VFL customer 310 on an application server and / or by a network. The combined classifier 314 may perform intermediate result exchange and collaborator functions defined in the conventional VFL framework, such as computing the output of a combined ML model from the outputs of multiple selected VFL encoders, interacting with the evaluator 330 to obtain a combined loss, and backpropagating gradients for each encoder.

[0110] In some non-limiting embodiments of the present technology, the combined classifier 314 may be pre-configured with a fixed structure, in which the VFL configurator 326 can configure the encoder output to the combined classifier 314 according to customized encoder output layer characteristics.

[0111] In the illustrated non-limiting embodiment, the evaluator 330 is implemented as a service function running on the network layer 304. The evaluator 330 receives labeled data forwarded by the privacy router 332 from one or more data sources 308 and calculates a combined loss value by applying a specific loss function to compare the combined ML model output from the combined classifier 314 with the corresponding labeled data.

[0112] In some non-limiting embodiments, the VFL configurator 326 may configure the selected loss function and labeled data format (including combination rules if the labeled data is composed of data from multiple data sources) according to the selected encoder and labeled data information of the data source 308.

[0113] In the illustrated non-limiting embodiment, the privacy router 332 is a predefined NET4AI function. However, the privacy router 332 may be implemented as a predefined function of a different architecture without departing from the scope of the present technology. In an automatic VFL, the privacy router 332 enables secure data exchange between the combined classifier 314 and evaluator 330 and the selected encoder and labeled data source 308.

[0114] In the illustrated non-limiting embodiment, the data combining function 412 can be performed by the privacy router 332 to combine unbalanced data entities (e.g., in terms of data entity quantity) associated with the same data ID during the training phase. The data combining function 412 can provide an interface between the encoder outputs 410 and the combined classifier 314. Broadly speaking, the data combining function 412 combines multiple outputs (each output may be in the form of a batch of data) from different encoders according to predefined combination rules into a single batch of combined output (e.g., concatenates all output batches together), which is then sent to the combined classifier 314 for further training / inference. The privacy router 332 can also perform combined loss transfer 416 between the combined classifier 314 and the evaluator 330, and labeled data transfer 414 from the labeled data source 408 to the evaluator 330.

[0115] Various embodiments of the present technology are described below that may help enable at least some of the encoder and data selection, encoder customization, and VFL configuration features of an automated VFL architecture.

[0116] Encoder and Data Selection Embodiments 5, there is shown a schematic diagram of an encoder and data selection procedure 500. In the illustrated non-limiting embodiment, the procedure begins at step 506, where an encoder selector 504 receives a VFL request 506 from a VFL customer 502. It is contemplated that the VFL request 506 may be received by the encoder selector 504 similar to how a VFL request 312 is received by the encoder selector 322 without departing from the scope of the present technology. As noted above, the VFL request 506 includes a description of a VFL problem to be solved / a VFL problem that the VFL is to be trained to solve.

[0117] Generally, a VFL problem is a general problem that can be "understood" by encoder selector 504 to determine specific data and / or encoder requirements. In embodiments where VFL customer 502 has partial and / or complete knowledge of the detailed data and encoder requirements, VFL customer 502 can indicate them directly in VFL request 502 for encoder selector 504 to process.

[0118] In other embodiments, where VFL customer 502 provides only a general VFL problem in VFL request 506, the procedure may proceed to step 508 for determining data requirements and encoder requirements. Therefore, encoder selector 504 separates or divides the VFL task into subtasks. As an example, the subtasks include a first subtask and a second subtask. It is understood that the number of subtasks can be greater than two.

[0119] Performing a given subtask requires a list of necessary data features and requirements (e.g., amount, content, or data, etc.) for the labeled data, which will be referred to hereinafter as the necessary data requirements for the subtask. Each subtask also corresponds to an application category associated with a particular encoder structure, which will be referred to as the necessary encoder requirements for the subtask.

[0120] For example, the VFL task "traffic violation recognition" can be decomposed into a combination of the following subtasks: 1) recognizing traffic violations through camera recordings, which requires video stream data features and labeled traffic violation video clips (necessary data requirements) and a CNN encoder for video processing (necessary encoder requirements); and 2) recognizing traffic violations through RSU data, which requires RSU sensed data features and labeled data and an encoder structure for RSU data processing. It should be noted that an application category can be a predefined NET4AI concept that describes, for example, a set of applications / problems that can be solved by a particular encoder structure. Therefore, it is said that an encoder structure may be associated with and stored in a respective application category.

[0121] In some embodiments, a VFL task can be implemented as a list of subtask IDs and attributes representing their importance (weight). The configuration and attribute values ​​for each VFL task are maintained by the encoder selector 504 and can be learned from historical data and / or existing knowledge. In the above example with the "traffic violation recognition" task, the VFL task may be defined as [(1, 0.4); (2, 0.6)], where (1, 0.4) corresponds to the "camera recording" subtask with ID equal to "1" and importance equal to "0.4", and (2, 0.6) corresponds to the "RSU" subtask with ID equal to "2" and importance equal to "0.6".

[0122] In this embodiment, the procedure proceeds to step 508, where an encoder selector 504 selects one or more encoders for solving the VFL task, where the one or more encoders may be selected from a pool of encoders in the communication network. Since a given encoder structure corresponds to one application category, the encoder selector 504 may select encoders to participate in the VFL according to the necessary encoder requirements of multiple subtasks. The number of encoders may match the number of subtasks. When a subtask includes a first subtask and a second subtask, the encoder requirements indicate a first encoder structure to be used to solve the first subtask and a second encoder structure to be used to solve the second subtask.

[0123] In some embodiments, the encoder selector 504 may select an encoder with a relatively simpler structure to speed up the convergence rate of the combined model training to ensure a predetermined effectiveness threshold (e.g., classification / prediction accuracy).

[0124] In other embodiments, the encoder selector 504 may need to ensure that the set of selected encoders includes the encoder structures necessary to solve all subtasks of the VFL task. For example, in the "traffic violation recognition" task presented above, the set of selected encoders may need to include at least a CNN for solving the "camera recording" subtask and a DNN for solving the "RSU data" subtask. The CNN can be considered a first encoder, and the DNN can be considered a second encoder. The first encoder is configured to process a first set of features for solving the first subtask, and the second encoder is configured to process a second set of features for solving the second subtask.

[0125] In the illustrated non-limiting embodiment, the procedure continues at step 510 with selecting data to be used to solve the VFL task. Given the selected encoders of step 508, an encoder selector 504 is configured to determine a data source for each selected encoder. The encoder selector 504 is configured to select a first data source for solving a first subtask from the pool of data sources, the first data source including a first set of features, and to select a second data source for solving a second subtask from the pool of data sources, the second data source including a second set of features.

[0126] In embodiments where all data sources are owned by the network, the encoder selector 504 can directly perform the data selection process according to complete information of all available data sources. However, in other embodiments where the available data sources are not owned by the network, the encoder selector 504 may interact with a data source manager to access information of the available data sources for data selection (including the location of the data sources, data quality, data features that can be provided for each data source, etc.).

[0127] For example, a data source manager may be a data analytics and management (DAM) function or entity whose purpose is to monitor data sources and provide data analysis functionality to different network applications (including VFL applications).

[0128] In this example, the data selection process can be completed by the following set of interactions between the data source manager and the encoder selector 504. The first interaction may involve the encoder selector 504 sending all necessary data requirements to the data source manager to request available data source information, where the data requirements are included in the request sent by the encoder selector 504 to the data source manager. The second interaction may involve the data source manager feeding back data quality and data feature information of all available data sources in the pool of data sources.

[0129] Depending on the received available data source information, the encoder selector 504 may select a detailed data source for each selected encoder and a data source for the labeled data. During data source selection, the encoder selector 504 may need to ensure that (i) the set of selected data sources provides all necessary data features required by the subtask, (ii) the selected data sources have a minimum number of overlapping features, and (iii) the labeled data can be original labeled data from one or more selected sources or combined labeled data composed by labeled data with partial features from multiple data sources.

[0130] In this example, a third interaction may include the encoder selector 504 sending all selected data source IDs to the DAM tool. A fourth interaction may include the DAM tool feeding back the locations of all datasets that include the selected data sources. Note that each data source may be partially hosted by multiple datasets at different network locations (e.g., servers, edge devices, etc.). In response to receiving the detailed dataset locations, the encoder selector 504 may determine a location (AI enabler location) for realizing the corresponding encoder that minimizes communication costs between the dataset and the AI ​​enabler.

[0131] Encoder Customization Embodiments Referring to FIG. 6, a schematic diagram of an encoder customization procedure 600 is shown. The customization procedure 600 may include communication and data transfer between an encoder selector 602, an encoder customizer 604, a reference AI enabler 606, and a target AI enabler 608. The encoder selector 602 may be implemented in a manner similar to the encoder selector 322 of FIG. 3 and / or the encoder selector 504 of FIG. 5. The encoder customizer 604 may be implemented in a manner similar to the encoder customizer 324 of FIG. 3. The reference AI enabler 606 is a fully trained "full" encoder available on the network. The target AI enabler 608 is the encoder to be "customized" in accordance with at least some embodiments of the present technology.

[0132] In step 610, the encoder selector 602 sends the encoder selection results to the encoder customizer 604. For example, step 610 may occur after the encoder and data selection procedures are completed, as described above. In some embodiments, the customization procedure 600 may be triggered by the encoder customizer 602 in response to receiving the encoder selection results.

[0133] The content of the encoder selection result is not limited and may vary depending on, among other things, various implementations of the present technology, but in some embodiments, the content of the encoder selection result includes encoder structure parameters (e.g., number of layers, size, etc.) of each selected encoder, the location of a target AI enabler (TE) that implements each customized selected encoder, and the location of a reference AI enabler (RE) that implements each fully trained selected encoder.

[0134] In the illustrated non-limiting embodiment shown in FIG. 6, the encoder selection result includes, among other things, the encoder structure parameters of the first encoder for the first subtask, the position of the corresponding TE608, and the position of the corresponding RE606.

[0135] Without wishing to be bound by any particular theory, the developer recognizes that the purpose of including the RE 606 in the encoder customization procedure 600 is to calculate different weights for data features in a given selected encoder, which may be parameters necessary for performing customization decisions. The data features for which weights are to be calculated by the RE 606 are likely to be data features originating from a first data source. In at least one case, all AI enablers for different applications can be maintained by a network layer (e.g., the NET4AI network), allowing the encoder selector to directly select the most trained RE 606. However, to ensure security and privacy requirements, the encoder selector only "knows" the encoder structure in the RE 606 (the same as the selector encoder structure) and how well the encoder has been trained (e.g., the number of trainings / uses). The detailed values ​​(biases / weights) of the neurons and links in the RE 606 are not known by the encoder selector itself.

[0136] Continuing with FIG. 6, at step 620, the encoder customizer 604 sends a request for data feature weights to the corresponding RE 606. At step 630, the encoder customizer 604 receives the calculated data feature weights. It should be noted that for a given encoder, the weight of each data feature in its input layer indicates the influence / importance of the corresponding data feature on the output of the given encoder. For example, the higher the weight of a given data feature, the greater the variation in the encoder output can be observed when changing the given data feature value.

[0137] 7, an example of the structure of a first encoder 700 and a second encoder 710 selected by the encoder and data selection procedure is shown. For example, the first encoder 700 has input data features (A, B, C, D) with weights (0.3, 0.4, 0.1, 0.2), and the second encoder 710 has input data features (A, B, E, F) with weights (0.1, 0.2, 0.4, 0.3). In this example, the input data features (A, B, C, D) come from a first data source, and the input data features (A, B, E, F) come from a second data source.

[0138] The developers of the present technology recognize that there are multiple ways to calculate the weights of each data feature for a customized encoder according to a fully trained encoder. However, the encoder customizer 604 may not have access to the fully trained values ​​of the neurons and links in the corresponding RE 606 due to security and privacy requirements. Therefore, the calculation of each data feature weight for a given selected encoder can be completed through the interaction between the encoder customizer 604 and the corresponding RE 606 performed during steps 620 and 630. Thus, the encoder customizer 604 sends a request for the data feature weights to the corresponding RE 606 (step 620), and the corresponding RE 606, for example, calculates the data feature weights locally according to known techniques and then feeds back the calculated data feature weights to the encoder customizer 604 (step 630). As an example, the calculated data feature weights include a data feature weight corresponding to each feature in a first feature set and a data feature weight corresponding to each feature in a second feature set, where the first feature set is included in a first data source for a first encoder and the second feature set is included in a second data source for a second encoder. At least one such technique is disclosed in a paper entitled "Problems with Shapley-value-based explanations as feature importance measures" by Kumar, I. Elizabeth et al., published June 30, 2020, the entire contents of which are incorporated herein by reference. Other techniques are contemplated without departing from the scope of the present technology.

[0139] The customization procedure 600 proceeds to step 640, where, given the calculated data feature weights of all selected encoders by the corresponding REs 606, encoder customization parameters are calculated by the encoder customizer 604, which is configured to select "key features" for the combined model being constructed.

[0140] In some embodiments, non-overlapping data features may be automatically selected as primary features. Broadly speaking, non-overlapping data features include data features that are present in each selected encoder without being present in the other selected encoders. In the example shown in Figure 7, the non-overlapping features include features C, D, E, and F.

[0141] In another embodiment, for a given overlapping data feature, the encoder customizer 604 compares its data feature weight in each selected encoder and selects the one with the highest relative weight as the primary feature. Broadly speaking, overlapping data features include data features that are duplicated in at least two selected encoders. In the example shown in Figure 7, the overlapping features include features A and B.

[0142] For example, in the first encoder 700, features A and B are primary features. It should be noted that the relative weights are different from the data feature weights calculated by the RE. The relative weight calculation considers both the data feature weights of each RE and other factors, such as, but not limited to, the encoder structure size (large size means high complexity, which affects output convergence performance), dependency on other features, and the overall importance of the feature. It should be noted that the relative weights are used to determine which data features should be retained and which should be removed in the first encoder 700 and the second encoder 710. In this example, it is said that the relative weights are used to determine whether features A and B should be removed from the first encoder 700 or the second encoder 710. Given the selected primary features, all non-primary data features are removed from the corresponding selected encoder by the encoder customizer. By removing the non-primary features, the side effects on the combined model caused by overlapping data features can be effectively reduced.

[0143] In some embodiments of the present technology, three modes of encoder customization can be selectively implemented by the encoder selector 604, depending on, among other things, different application scenarios. For example, the first mode 720 can be referred to as a minimal customization mode, in which only input layer neurons corresponding to non-primary features in the selected encoder are removed. This is the encoder customization mode that has the least impact on the connection model structure.

[0144] In another example, the second mode can be referred to as weight-based customization mode 730. In weight-based customization mode 220, for a given selected encoder that has non-primary features removed from its input layer, the encoder customizer 604 sends the IDs of the removed neurons to a corresponding RE with a corresponding well-trained encoder. The corresponding RE removes all neurons and links in the encoder that are significantly affected by the removed features. The influence of each data feature on a particular neuron or link in the encoder can be calculated according to various techniques. At least one such technique is disclosed in the aforementioned paper entitled "Problems with Shapley-value-based explanations as feature importance measures." The corresponding RE then feeds back the removed neuron and link IDs / locations to the encoder customizer 604 to complete the encoder customization.

[0145] In a further example, the third mode can be referred to as a learning-based customization mode, in which, for a given selected encoder that has non-key features removed from its input layer, the encoder customizer 604 applies a learning-based NN pruning solution to train a customized encoder. At least one such solution is disclosed in the paper entitled "Learning both Weights and Connections for Efficient Neural Networks" by Han, Song et al., published in 2015, the entire contents of which are incorporated herein by reference.

[0146] The developers recognize that such a solution requires significant computational resources during training. It should be noted that the training phase should be performed by a corresponding TE that implements the customized encoder in order to reduce the computational cost of running the encoder customizer 604.

[0147] In the illustrated non-limiting embodiment, in step 650, the encoder customizer 604 sends the encoder customization parameters to the corresponding TE 608 to perform the encoder customization decision process. The contents of the encoder customization parameters may depend, among other things, on different encoder customization modes. For the minimal customization mode 720, information indicating the removed input layer neurons (non-primary features) may be included in the encoder customization parameters. For the weight-based customization mode 730, information indicating the removed neurons and links for the full encoder may be included in the encoder customization parameters. For the learning-based customization mode, information indicating the removed neurons in the input layer and hyperparameters for customization training may be included in the encoder customization parameters. It is contemplated that in some embodiments of the present technology, the hyperparameters may include a loss threshold for determining convergence, a weight threshold for the neurons and links to be removed, etc.

[0148] In the illustrated non-limiting embodiment, in step 660, the corresponding TE 608 performs an encoder customization decision according to the received encoder customization parameters of step 650. It is contemplated that the decision may be performed based on relative weights. The relative weights may be calculated based on the encoder customization parameters received in step 650. For the minimum and weight-based customization modes 720 and 730, the corresponding TE 608 may directly delete corresponding neurons and / or links. For the learning-based customization mode, the corresponding TE 608 may perform customization training until convergence without departing from the scope of the present technology. The customization training may be triggered in response to receiving the encoder customization parameters.

[0149] For illustrative purposes only, it is assumed that the corresponding TE 608 performs the encoder customization decision according to a learning-based customization mode. In the illustrated non-limiting embodiment, since the final customization result of the learning-based customization is obtained in the corresponding TE 608, after convergence is reached, the corresponding TE 608 may feed back customized encoder features (e.g., customized structure of the encoder) to the encoder customizer 604 in step 660. The customized encoder features may be used, for example, for further VFL configuration.

[0150] VFL Configuration Embodiment 8, there is shown a schematic diagram of a VFL configuration procedure 800. The VFL configuration procedure 800 includes an encoder selector 802, an encoder customizer 804, a VFL configurator 806, a combined classifier 808, an evaluator 810, and a privacy router 332. It is contemplated that at least some of the components involved in the VFL configuration procedure 800 may be implemented similarly to the components of the VFL architecture 300 shown in FIG. 3 without departing from the scope of the present technology.

[0151] In the illustrated non-limiting embodiment, in step 820, the encoder selector 802 sends the labeled data information to the VFL configurator 806, and in step 830, the encoder customizer 804 sends the customized encoder information to the VFL configurator 806. In some embodiments, receipt of the labeled data information and the customized encoder information by the VFL configurator 806 may trigger the following steps of the VFL configuration procedure 800.

[0152] Generally, the labeled data information includes the location of a dataset including labeled data sources and provided data features. In other words, the labeled data information indicates the location (which portion of the labeled data comes from which of the first and second data sources) and how the labeled data should be combined. It is contemplated that a loss function indicated by the VFL customer may also be transmitted as part of the labeled data information. The customized encoder information includes information indicating the output layer structure of each customized encoder. As an example, when the encoder customizer 804 determines that the first encoder needs to be customized according to data feature weights, the customized encoder information includes information indicating the output layer structure of the customized first encoder. When the encoder customizer 804 determines that the first encoder and the second encoder need to be customized according to data feature weights, the customized encoder information includes information indicating the output layer structures of the customized first encoder and the customized second encoder.

[0153] In the illustrated non-limiting embodiment, in step 840, the VFL configurator 806 configures the encoder output to the combined classifier based on the customized encoder information. Configuring the encoder output to the combined classifier may be performed in various ways. In some embodiments, the output layers of multiple encoders may be connected to the input layer of the combined classifier via a concatenation of the output layers of the multiple encoders. At least one other connection mode is disclosed in the paper entitled "SplitNN-driven vertical partitioning" by Iker Ceballos et al., published on August 7, 2020, the contents of which are incorporated herein by reference in their entirety.

[0154] In the illustrated non-limiting embodiment, in step 850, the VFL configurator 806 sends the combined classifier configuration parameters to the combined classifier 808. The contents of the combined classifier configuration parameters are used by the combined classifier to customize or modify its local configuration to match the encoder's output (which is the input of the combined classifier). For example, in some embodiments, encoders customized using weight-based or learning-based methods as described above may have their output layer neurons partially muted and / or deleted. The VFL configurator may include information about muted output layer neurons (e.g., of all encoders) in the combined classifier configuration parameters. By receiving the combined classifier configuration parameters, the combined classifier can customize its input layer architecture (e.g., by muting and / or deleting neurons) to match the encoder output.

[0155] In the illustrated non-limiting embodiment, in step 860, VFL configurator 806 configures evaluator 810. As part of step 860, VFL configurator 806 uses the labeled data information to configure combination rules for partially labeled data from different datasets as they are merged in evaluator 810. Also as part of step 860, VFL configurator 806 configures a loss function to be used during VFL.

[0156] Partially labeled data can be considered labeled data in which the associated label information does not represent the complete label information. All partially labeled data associated with the same ID / sample should be combined together to form the complete labeled data. For example, the complete labeled data may be represented as "[A, blue cat]", where "A" is the associated ID / sample (i.e., the represented entity) and "blue cat" is the label. In some embodiments, this labeled data may come from two different data sources: one providing "[A, blue]" and the other providing "[A, cat]". In this example, both "[A, blue]" and "[A, cat]" are termed partial labeled data.

[0157] In the illustrated non-limiting embodiment, in step 880, the VFL configurator 806 configures an inter-encoder or “privacy” router 812. It is contemplated that the privacy router 812 may be implemented similarly to the privacy router 332 of FIG. 3. As part of step 880, the VFL configurator 806 configures data transmission routing paths for (i) a data exchange path between the combined classifier 808 and the evaluator 810, (ii) a labeled data merge path between the evaluator 810 and each data source, and (iii) a backpropagation path between the combined classifier 808 and each encoder for backpropagation gradient values. The “respective encoders” in (iii) can be understood as all selected encoders selected by the encoder selector. When some of the selected encoders are customized by the encoder customizer, the “respective encoders” in (iii) include the customized encoders and the remaining uncustomized encoders among all selected encoders. When the selected encoders selected by the encoder selector include a first encoder and a second encoder, and the first encoder is customized, the "respective encoders" in (iii) include the customized first encoder and second encoder.

[0158] In the illustrated non-limiting embodiment, in step 890, the VFL configurator 806 sends privacy router configuration parameters to the privacy router 812. In at least some embodiments of the present technology, the routes supported by the privacy router 812 may be those described above with reference to FIG.

[0159] Performance evaluation 9, three simplified representations of the VFL framework are shown. To verify the effectiveness and efficiency of the Auto-VFL architecture contemplated in at least some embodiments of the present technology, simulations were performed to compare the performance of three cases on a binary classification task: traditional VFL 910, Auto-VFL 92, and stitched single NN 930.

[0160] Generally, the conventional VFL 910 includes two participants (P1 and P2) with different encoders (E1 and E2). The detailed parameters of E1 and E2 are shown in Table 1 below. Both participants have independent classifiers with the same structure (C0). The inputs of both participants (I1 and I2) completely overlap in ID space and partially overlap in feature space. The automated VFL 920 includes two participants (PA1 and PA2) with encoders and inputs of P1 and P2. The outputs of the two participants are connected to a combined classifier with the same structure as C0. It should be noted that all classifiers used in these three cases are the same in terms of structure. The stitched single NN 930 connects I1 and I2 as a unified input layer and stitches E1 and E2 layer by layer with a fully connected structure to form a stitched encoder. It should be noted that the remaining hidden layers of the encoder (e.g., the remaining E2 layers) are fully connected to the stitched structure. The independent classifier C0 is fully connected to the output layer of the stitched encoder. [Table 1]

[0161] The simulations were performed on two datasets: an anonymous banking dataset containing user information from Germanbank, and a network traffic dataset used for deep packet inspection of encrypted internet traffic (TLS22).

[0162] Referring to FIG. 10 , a performance comparison between the conventional VFL 910 and the automatic VFL 920 applied to the two test datasets described above is shown. To verify the effectiveness of the encoder customization considered in at least some embodiments of the present technology, both the automatic VFL without encoder customization ("autoVFL_noCut") and the automatic VFL with minimal encoder customization ("autoVFL") were simulated. As shown in graphs 1010 and 1030, it can be seen that, in some implementations of the present technology, the automatic VFL 920 (with or without encoder customization) can converge to a better optimal result with a higher accuracy rate and lower variance compared to the conventional VFL 910. As shown in graphs 1020 and 1040, the classification performance of the conventional VFL 910 and the automatic VFL 920 is compared. It can be said that, in at least some implementations of the present technology, the automatic VFL 920 can improve general performance metrics, particularly accuracy rate (5% to 8% improvement) and precision rate (10% improvement), compared to the conventional VFL 910.

[0163] The developers have recognized that, in at least some embodiments of the present technology, the better performance achieved by Auto-VFL may be made possible by two aspects of Auto-VFL. First, compared to the intermediate result exchange of conventional VFL, the combined classifier can enable stronger mutual influence between participating encoders, which increases the efficiency of training the combined model. Second, encoder customization can further reduce the adverse effects caused by overlapping data features from the inputs of multiple participants. While both the combined classifier and encoder customization can improve overall performance, according to the results shown in FIG. 10, the combined classifier has a higher impact than encoder customization. Furthermore, it should be noted that in the simulation of FIG. 10, only the minimum encoder customization mode is applied. When a more complex encoder customization algorithm is fully realized (other modes described herein above), encoder customization may increase the performance of Auto-VFL without departing from the scope of the present technology.

[0164] Referring to FIG. 11 , a performance comparison between the Auto VFL 920 and the Stitched Single NN 930 applied to the two test datasets described above is shown. For the Stitched Single NN 930, both a single NN with no encoder customization applied to its concatenated input layer, “singleNN_noCut,” and a single NN with minimal encoder customization applied to its concatenated input layer, “singleNN_cut,” were simulated. In at least some embodiments of the present technology, it is believed that the Auto VFL can achieve comparable convergence and classification performance without significant losses compared to that of the Stitched Single NN. Furthermore, as seen in graphs 1110 and 1120, it may be noted that the Stitched Single NN 930 incurs significant performance drawbacks because a simple stitched NN structure may not be suitable for a particular dataset or problem. While a suitable new NN can mitigate these drawbacks, developers recognize that the redesign procedure for a new NN can be costly because clear design principles are not available and existing models cannot be reused. Therefore, it can be said that in at least some embodiments of the present technology, Auto VFL 920 can achieve general performance metrics that are comparable to or better than those of Stitch Single NN 930.

[0165] In some embodiments of the present technology, an automated VFL system is provided that is configured to perform at least some of the following functions: an encoder selector, an encoder customizer, a VFL configurator, and a combined classifier. In some embodiments of the present technology, the automated VFL system can be implemented for the NET4AI architecture.

[0166] In some embodiments, the automated VFL system may achieve better AI model performance compared to traditional VFL and stitched single NNs because (i) the combined classifier can enable deep mutual influence among participating encoders (compared to traditional VFL) and (ii) encoder customization can reduce the negative effects caused by overlapping data features. In other embodiments, the automated VFL system may enable automatic encoder selection and reuse. As a result, this allows existing encoders in the network to be reused to solve the VFL task, and a specific NN for that purpose may not need to be designed. A specific NN need not be designed. Both encoder (and dataset) selection and customization are automated by NET4AI. This may also avoid VFL configuration through the application layer, so that VFL management overhead at the application layer may not be required. In further embodiments, the automated VFL system may solve complex tasks by supporting loss functions that violate homomorphic encryption, allowing labeled data to be provided separately by multiple data sources.

[0167] Those skilled in the art will understand that the description of various embodiments is illustrative only and is not intended to be limiting in any way. Other embodiments will readily suggest themselves to those skilled in the art having the benefit of this disclosure. Moreover, at least some of the disclosed embodiments may be customized to provide useful solutions to existing needs and problems related to VFLs. In the interest of clarity, not all of the routine features of at least some implementations of the disclosed embodiments are shown and described.

[0168] In particular, combinations of elements recited in the appended claims form an integral part of this disclosure, and therefore combinations of features are not limited to those presented in the above description. Of course, in developing any such actual implementation of at least some of the disclosed embodiments, it will be recognized that numerous implementation-specific decisions may need to be made to achieve the developer's particular goals, such as compliance with application, system, and business-related constraints, and that these particular goals will vary from implementation to implementation and from developer to developer. Furthermore, it will be recognized that the development effort may be complex and time-consuming, but will nevertheless be a routine engineering undertaking for those skilled in the art of feedback equalization at high data rates having the benefit of this disclosure.

[0169] In accordance with this disclosure, the components, process operations, and / or data structures described herein may be implemented using various types of operating systems, computing platforms, network devices, computer programs, and / or general-purpose machines. Furthermore, those skilled in the art will recognize that devices of a less general-purpose nature, such as hardwired devices, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and the like, may also be used. When a method including a series of operations is implemented by a computer, a processor operatively connected to a memory, or a machine, the operations may be stored as a series of instructions readable by the machine, processor, or computer, or may be stored on a non-transitory, tangible medium.

[0170] The systems and modules described herein may include software, firmware, hardware, or any combination of software, firmware, or hardware suitable for the purposes described herein. The software and other modules may be executed by a processor and reside in the memory of a server, workstation, personal computer, computerized tablet, personal digital assistant (PDA), and other device suitable for the purposes described herein. The software and other modules may be accessible via local memory, over a network, via a browser or other application, or via other means suitable for the purposes described herein. The data structures described herein may include computer files, variables, programming arrays, programming structures, or any electronic information storage scheme or method, or any combination thereof, suitable for the purposes described herein.

[0171] The present disclosure has been described in the above specification by way of non-limiting exemplary embodiments provided as examples. These exemplary embodiments may be optionally modified. The scope of the claims should not be limited by the embodiments described in these examples, but should be given the broadest interpretation consistent with the description as a whole.

Claims

1. receiving, by an encoder selector, a vertical federation learning (VFL) request from a VFL customer; determining, by the encoder selector, data requirements and encoder requirements for solving a VFL task according to the VFL request, the VFL task being divisible into a plurality of subtasks, the plurality of subtasks including at least a first subtask and a second subtask; selecting, by the encoder selector, a first encoder and a second encoder from a pool of encoders in a communication network based on the encoder requirements, the first encoder configured to process a first set of features for solving the first subtask and the second encoder configured to process a second set of features for solving the second subtask; selecting, by the encoder selector, a first data source for the first encoder and a second data source for the second encoder based on the data requirements, the selected data sources being from a pool of data sources, the first data source including the first set of features and the second data source including the second set of features; A method comprising:

2. 2. The method of claim 1, wherein the data requirements indicate requirements for selecting the first data source for solving the first subtask and the second data source for solving the second subtask.

3. 3. The method of claim 1, wherein the encoder requirements indicate a first encoder structure to be used to solve the first subtask and a second encoder structure to be used to solve the second subtask.

4. The method comprises: The method of claim 1 , further comprising the step of dividing the VFL task into the plurality of subtasks by the encoder selector.

5. Selecting the first encoder comprises: determining, by the encoder selector, an application category for the first subtask based on the encoder requirements; Including, 5. The method of claim 1, wherein the application category is stored in a memory in association with the first encoder.

6. Selecting the first encoder comprises: identifying, by the encoder selector, the pool of encoders having a first encoder structure; selecting, by the encoder selector, the first encoder from the pool of encoders having the first encoder structure; 5. The method of claim 1, further comprising:

7. Selecting the first data source and the second data source includes: sending, by said encoder selector, a request for data source information to a data source manager, said request including said data requirements; receiving, by the encoder selector, from the data source manager, data source information indicating the quality and characteristics of data contained in each from the pool of data sources; selecting, by the encoder selector, the first data source for the first encoder and the second data source for the second encoder using the data source information; The method of claim 1 , comprising:

8. The first data source and the second data source are associated with respective identities, and the method comprises: sending, by the encoder selector, respective IDs of the first data source and the second data source to the data source manager; receiving, by the encoder selector, network locations of the first data source and the second data source from the data source manager; The method of claim 7 further comprising:

9. customizing, with an encoder customizer, the first encoder by muting a portion of the first encoder, thereby generating a customized first encoder. Including, the first encoder and the second encoder are selected from a pool of encoders in a communication network based on encoder requirements, the encoder requirements being determined for solving a vertical federated learning (VFL) task according to VFL requirements; The VFL task is divisible into a plurality of subtasks, the plurality of subtasks including at least a first subtask and a second subtask; 1. A method according to claim 1, wherein the first encoder is configured to process a first set of features for solving the first subtask, and the second encoder is configured to process a second set of features for solving the second subtask.

10. The customizing step includes: determining overlapping features between the first set of features and the second set of features; muting the portion of the first encoder that processes the overlapping features; and 10. The method of claim 9, comprising:

11. The customizing step includes: determining, by the encoder customizer, that the first encoder should be customized; 11. The method of claim 9 or 10, comprising:

12. determining, by the encoder customizer, that the first encoder should be customized includes: determining, by the encoder customizer, that the first encoder should be customized based on data feature weights, the data feature weights being received from a reference AI enabler; The method of claim 11 , comprising:

13. Customizing the first encoder includes: muting, by the encoder customizer, the portion of the first encoder that processes overlapping features. Including, 13. The method of claim 9, wherein the muting comprises muting at least one of a neuron of the first encoder and a link of the first encoder.

14. configuring at least one of the evaluator and the combined classifier with a vertical federated learning (VFL) configurator; configuring, by the VFL configurator, a privacy router to determine at least one of a data exchange path between the combined classifier and the evaluator, a labeled data merge path between the evaluator and one or more data sources, and a backpropagation path between the combined classifier and customized first and second encoders for backpropagating gradient values; Including, The method, wherein the combined classifier, the evaluator, and the privacy router are configured to perform a VFL.

15. Configuring the evaluator comprises: receiving, by said VFL configurator, labeled data information from an encoder selector; determining, by the VFL configurator, a loss function to be used to implement the VFL; configuring the evaluator using the labeled data information and the loss function by the VFL configurator; The method of claim 11 , comprising:

16. Configuring the evaluator comprises: configuring, by the VFL configurator, rules for combining the loss function with the labeled data resulting from the selected data sources for the first encoder and the second encoder; 13. The method of claim 12, comprising:

17. Configuring the combined classifier comprises: receiving, by the VFL configurator, information indicating an output layer structure of the first encoder and the second encoder from an encoder customizer; configuring, by the VFL configurator, encoder outputs to the combined classifier based on the output layer structures of the first encoder and the second encoder; 14. The method of any one of claims 11 to 13, comprising:

18. 18. The method of any one of claims 14 to 17, wherein the second encoder is a customized second encoder.

19. An encoder selector, receiving a vertical federation learning (VFL) request from a VFL customer; determining data requirements and encoder requirements for solving a VFL task according to the VFL request, the VFL task being divisible into a plurality of subtasks, the plurality of subtasks including at least a first subtask and a second subtask; selecting a first encoder and a second encoder from a pool of encoders in a communication network based on the encoder requirements, the first encoder configured to process a first set of features for solving the first sub-task and the second encoder configured to process a second set of features for solving the second sub-task; selecting a first data source for the first encoder and a second data source for the second encoder based on the data requirements, the selected data sources being from a pool of data sources, the first data source including the first set of features and the second data source including the second set of features; an encoder selector configured to:

20. 20. The encoder selector of claim 19, wherein the data requirements indicate requirements for selecting the first data source for solving the first subtask and the second data source for solving the second subtask.

21. 21. The encoder selector of claim 19 or 20, wherein the encoder requirements indicate a first encoder structure to be used for solving the first subtask and a second encoder structure to be used for solving the second subtask.

22. 22. The encoder selector of claim 19, wherein the encoder selector is further configured to divide a VFL task into the plurality of subtasks.

23. Selecting the first encoder comprises: the encoder selector is configured to determine an application category for the first subtask based on the encoder requirements; the application category is stored in a memory in association with the first encoder; 23. An encoder selector according to any one of claims 19 to 22, comprising:

24. Selecting the first encoder comprises: The encoder selector identifying a pool of encoders having a first encoder structure; selecting the first encoder from the pool of encoders having the first encoder structure; be configured to 23. An encoder selector according to any one of claims 19 to 22, comprising:

25. Selecting the first data source and the second data source includes: The encoder selector sending a request for data source information to a data source manager, said request including said data requirements; receiving data source information from the data source manager indicating the quality and characteristics of the data contained in each from the pool of data sources; using the data source information to select the first data source for the first encoder and the second data source for the second encoder; be configured to 20. The encoder selector of claim 19, comprising:

26. the first data source and the second data source are associated with respective IDs; The encoder selector is sending respective identities of the first data source and the second data source to the data source manager; receiving from the data source manager the network locations of the first data source and the second data source; 20. The encoder selector of claim 19, further configured to:

27. An encoder customizer, configured to customize the first encoder by muting a portion of the first encoder, thereby generating a customized first encoder; the first encoder and the second encoder are selected from a pool of encoders in a communication network based on encoder requirements, the encoder requirements being determined for solving a vertical federated learning (VFL) task according to VFL requirements; The VFL task is divisible into a plurality of subtasks, the plurality of subtasks including at least a first subtask and a second subtask; an encoder customizer, wherein the first encoder is configured to process a first set of features for solving the first subtask and the second encoder is configured to process a second set of features for solving the second subtask.

28. To customize, The encoder customizer: determining overlapping features between the first set of features and the second set of features; muting the portion of the first encoder that processes the overlapping features; and be configured to 28. The encoder customizer of claim 27, comprising:

29. To customize, The encoder customizer: being configured to determine that the first encoder should be customized; 29. The encoder customizer of claim 27 or 28, comprising:

30. Determining that the first encoder should be customized includes: The encoder customizer: configured to determine, based on data feature weights, that the first encoder should be customized; the data feature weights are received from a reference AI enabler; 30. The encoder customizer of claim 29, comprising:

31. Customizing the first encoder includes: The encoder customizer: configured to mute the portion of the first encoder that processes overlapping features. Including, 31. The encoder customizer of claim 27, wherein the muting comprises muting at least one of a neuron of the first encoder and a link of the first encoder.

32. 1. A vertically federated learning (VFL) configurator, comprising: configuring at least one of an evaluator and a combined classifier; configuring a privacy router to determine at least one of a data exchange path between the combined classifier and the evaluator, a labeled data merge path between the evaluator and one or more data sources, and a backpropagation path between the combined classifier and customized first and second encoders for backpropagating gradient values; configured to: The VFL configurator, wherein the combined classifier, the evaluator, and the privacy router are configured to execute a VFL.

33. Configuring the evaluator comprises: The VFL configurator: receiving labeled data information from an encoder selector; Determining a loss function to be used to perform VFL; and configuring the evaluator using the labeled data information and the loss function; and be configured to 33. The VFL configurator of claim 32, comprising:

34. Configuring the evaluator comprises: The VFL configurator: configured to configure a combination rule of the loss function with labeled data originating from selected data sources for the first encoder and the second encoder; 34. The VFL configurator of claim 33, comprising:

35. Configuring the combined classifier comprises: The VFL configurator: receiving information indicating an output layer structure of the first encoder and the second encoder from an encoder customizer; constructing an encoder output to the combined classifier based on the output layer structures of the first encoder and the second encoder; be configured to 35. A VFL configurator according to any one of claims 32 to 34, comprising:

36. 36. The VFL configurator of any one of claims 32 to 35, wherein the second encoder is a customized second encoder.

37. receiving, by an encoder selector, a vertical federation learning (VFL) request from a VFL customer; determining, by the encoder selector, data requirements and encoder requirements for solving a VFL task according to the VFL request, the VFL task being divisible into a plurality of subtasks, the plurality of subtasks including at least a first subtask and a second subtask; selecting, by the encoder selector, a first encoder and a second encoder from a pool of encoders in a communication network based on the encoder requirements, the first encoder configured to process a first set of features for solving the first subtask and the second encoder configured to process a second set of features for solving the second subtask; selecting, by the encoder selector, a first data source for the first encoder and a second data source for the second encoder based on the data requirements, the selected data sources being from a pool of data sources, the first data source comprising the first set of features and the second data source comprising the second set of features; customizing, with an encoder customizer, the first encoder by muting a portion of the first encoder, thereby generating a customized first encoder; configuring, with a VFL configurator, at least one of an evaluator and a combined classifier; configuring, by the VFL configurator, a privacy router to determine at least one of a data exchange path between the combined classifier and the evaluator, a labeled data merge path between the evaluator and one or more of the selected data sources, and a backpropagation path between the combined classifier and each of a customized first encoder and a customized second encoder for backpropagating gradient values; Including, The method, wherein the combined classifier, the evaluator, and the privacy router are configured to perform a VFL.

38. 38. The method of claim 37, wherein the data requirements indicate requirements for selecting the first data source for solving the first subtask and the second data source for solving the second subtask.

39. 39. The method of claim 37 or 38, wherein the encoder requirements indicate a first encoder structure to be used to solve the first subtask and a second encoder structure to be used to solve the second subtask.

40. The method comprises:

40. The method of any one of claims 37 to 39, further comprising dividing the VFL task into the plurality of subtasks by the encoder selector.

41. Selecting the first encoder comprises: determining, by the encoder selector, an application category for the first subtask based on the encoder requirements; Including, 41. A method according to any one of claims 37 to 40, wherein the application category is stored in memory in association with the first encoder.

42. Selecting the first encoder comprises: identifying, by the encoder selector, the pool of encoders having a first encoder structure; selecting, by the encoder selector, the first encoder from the pool of encoders having the first encoder structure; 42. The method of any one of claims 37 to 41, comprising:

43. Selecting the first data source and the second data source includes: sending, by said encoder selector, a request for data source information to a data source manager, said request including said data requirements; receiving, by the encoder selector, from the data source manager, data source information indicating the quality and characteristics of data contained in each from the pool of data sources; selecting, by the encoder selector, the first data source for the first encoder and the second data source for the second encoder using the data source information; 38. The method of claim 37, comprising:

44. The first data source and the second data source are associated with respective identities, and the method comprises: sending, by the encoder selector, respective IDs of the first data source and the second data source to the data source manager; receiving, by the encoder selector, network locations of the first data source and the second data source from the data source manager; 44. The method of claim 43, further comprising:

45. The customizing step includes: determining overlapping features between the first set of features and the second set of features; muting the portion of the first encoder that processes the overlapping features; and 45. The method of any one of claims 37 to 44, comprising:

46. The customizing step includes: determining, by the encoder customizer, that the first encoder should be customized; 46. ​​The method of any one of claims 37 to 45, comprising:

47. determining, by the encoder customizer, that the first encoder should be customized includes: determining, by the encoder customizer, that the first encoder should be customized based on data feature weights, the data feature weights being received from a reference AI enabler; 47. The method of claim 46, comprising:

48. Customizing the first encoder includes: muting, by the encoder customizer, the portion of the first encoder that processes overlapping features. Including, 48. The method of any one of claims 37 to 47, wherein the muting comprises muting at least one of a neuron of the first encoder and a link of the first encoder.

49. Configuring the evaluator comprises: receiving, by said VFL configurator, labeled data information from an encoder selector; determining, by the VFL configurator, a loss function to be used to implement the VFL; configuring the evaluator using the labeled data information and the loss function by the VFL configurator; 38. The method of claim 37, comprising:

50. Configuring the evaluator comprises: configuring, by the VFL configurator, rules for combining the loss function with labeled data originating from selected data sources for the first encoder and the second encoder; 50. The method of claim 49, comprising:

51. Configuring the combined classifier comprises: receiving, by the VFL configurator, information indicating an output layer structure of the first encoder and the second encoder from an encoder customizer; configuring, by the VFL configurator, encoder outputs to the combined classifier based on the output layer structures of the first encoder and the second encoder; 51. The method of any one of claims 47 to 50, comprising:

52. 52. The method of any one of claims 37 to 51, wherein the second encoder is a customized second encoder.

53. 1. A system comprising: receiving, by an encoder selector, a vertically federated learning (VFL) request from a VFL customer; determining, by the encoder selector, data requirements and encoder requirements for solving a VFL task according to the VFL request, wherein the VFL task is divisible into a plurality of subtasks, the plurality of subtasks including at least a first subtask and a second subtask; selecting, by the encoder selector, a first encoder and a second encoder from a pool of encoders in a communication network based on the encoder requirements, the first encoder configured to process a first set of features for solving the first subtask and the second encoder configured to process a second set of features for solving the second subtask; selecting, by the encoder selector, a first data source for the first encoder and a second data source for the second encoder based on the data requirements, the selected data sources being from a pool of data sources, the first data source including the first set of features and the second data source including the second set of features; customizing, with an encoder customizer, the first encoder by muting a portion of the first encoder, thereby generating a customized first encoder; configuring at least one of an evaluator and a combined classifier with a VFL configurator; configuring, by the VFL configurator, a privacy router to determine at least one of a data exchange path between the combined classifier and the evaluator, a labeled data merge path between the evaluator and one or more of the selected data sources, and a backpropagation path between the combined classifier and each of a customized first encoder and a customized second encoder for backpropagating gradient values; configured to: The system, wherein the combined classifier, the evaluator, and the privacy router are configured to execute a VFL.

54. 54. The system of claim 53, wherein the data requirements indicate requirements for selecting the first data source for solving the first subtask and the second data source for solving the second subtask.

55. 55. The system of claim 53 or 54, wherein the encoder requirements indicate a first encoder structure to be used to solve the first subtask and a second encoder structure to be used to solve the second subtask.

56. 56. The system of any one of claims 53 to 55, wherein the encoder selector is further configured to divide the VFL task into the plurality of subtasks.

57. Selecting the first encoder comprises: the encoder selector is configured to determine an application category for the first subtask based on the encoder requirements; the application category is stored in a memory in association with the first encoder; 57. A system according to any one of claims 53 to 56, comprising:

58. Selecting the first encoder comprises: The encoder selector identifying a pool of encoders having a first encoder structure; selecting the first encoder from the pool of encoders having the first encoder structure; be configured to 58. A system according to any one of claims 53 to 57, comprising:

59. Selecting the first data source and the second data source includes: The encoder selector sending a request for data source information to a data source manager, said request including said data requirements; receiving data source information from the data source manager indicating the quality and characteristics of the data contained in each from the pool of data sources; using the data source information to select the first data source for the first encoder and the second data source for the second encoder; be configured to 54. The system of claim 53, comprising:

60. the first data source and the second data source are associated with respective IDs; The encoder selector sending respective identities of the first data source and the second data source to the data source manager; receiving from the data source manager the network locations of the first data source and the second data source; 60. The system of claim 59, further configured to:

61. To customize, The encoder customizer determining overlapping features between the first set of features and the second set of features; muting the portion of the first encoder that processes the overlapping features; and be configured to 61. The system of any one of claims 53 to 60, comprising:

62. To customize, The encoder customizer obtains (i) encoder structure parameters of the first encoder and the second encoder, (ii) a position of a target AI enabler for the customized encoder, and (iii) a position of a reference AI enabler of a trained encoder; Sending a request for data feature weights of the trained encoder to the reference AI enabler; receiving the data feature weights from the reference AI enabler; determining overlapping features between the first set of features of the first encoder and the second set of features of the second encoder; determining, based on the data feature weights, that the first encoder should be customized; muting the portion of the first encoder that processes the overlapping features; and be configured to Including, 62. The system of claim 61 , wherein muting comprises muting at least one of a neuron of the first encoder and a link of the first encoder.

63. Configuring the evaluator comprises: The VFL configurator: receiving labeled data information from an encoder selector; Determining a loss function to be used to perform VFL; and configuring the evaluator using the labeled data information and the loss function; and be configured to 54. The system of claim 53, comprising:

64. Configuring the evaluator comprises: The VFL configurator: configured to configure a combination rule of the loss function with labeled data originating from selected data sources for the first encoder and the second encoder; 64. The system of claim 63, comprising:

65. Configuring the combined classifier comprises: The VFL configurator: receiving information indicating an output layer structure of the first encoder and the second encoder from an encoder customizer; constructing an encoder output to the combined classifier based on the output layer structures of the first encoder and the second encoder; be configured to 65. A system according to any one of claims 63 to 64, comprising:

66. 66. The system of any one of claims 53 to 65, wherein the second encoder is a customized second encoder.

Images