System and method of training a learning model in a distributed client / server data processing system

Abstract

A method for training multiple learning models in a distributed client / server data processing system (202), which includes several clients (204A-K, 304A-K) interacting with a server (206, 306) and a model warehouse (208, 308) over a network is provided. The model warehouse, which has access to pre-trained multi-modal encoders and decoders, performs the method. It involves receiving a request from the server that includes identifiers for the server, task, clients, dataset sizes, computational resources, and data modes. The model warehouse selects an appropriate learning model and derives estimates of the marginal and joint structures of the encoder output based on client datasets and processing capabilities. It sends these structures and decoders to the server and marginal structures to the clients. Each client then selects an encoder based on its processing capability. The clients and server jointly train the encoders and decoders using their respective data and the provided structures.

Description

[0001] SYSTEM AND METHOD OF TRAINING A LEARN ING MODEL IN A DISTRIBUTED CLIENT / SERVER DATA

[0002] PROCESSING SYSTEM

[0003] TECHNICAL FIELD

[0004] The disclosure relates generally to training learning models in a distributed system, and more particularly, the disclosure relates to a method of training one or more learning models in a distributed client / server data processing system having one or more clients interacting with a server and with a model warehouse, over a network. Moreover, the disclosure relates to a system of training the learning models in the distributed client / server data processing system having the clients interacting with the server and with the model warehouse, over the network.

[0005] BACKGROUND

[0006] Many real-life Artificial intelligence, (Al)-based applications deal with distributed multi-modal data. Typically, a set of distributed nodes with access to limited distributed multi-modal data and possessing limited computational power are required to collaboratively learn suitable models for an inference task. This scenario highlights the need for solutions that can leverage the capabilities of the 6G network to assist these nodes in learning such models and determining the optimal structure for these models.

[0007] More precisely, as depicted in FIG. 1, a typical model warehouse 108 has access to a set of large pre-trained multi-modal encoders and decoders, derived from extensive datasets. There are K clients 104A-K, each client 104A-K having access to one data mode (Si-Sk) (e.g., text, image or video) of correlated data in a multi-modal setup. In case, where a client has multiple data modes, it can be virtually considered as multiple clients and each having access to only one data mode. Both clients 104A- K and a server 106 needs to receive the pre-trained encoders and decoders and jointly train or fine-tune them.

[0008] Additionally, the existing multi-modal setup includes the following constraints or limitations such as (i) the clients 104A-K have access to small datasets and possess limited computational resources / power, (ii) the existing trained encoders are potentially too large to be loaded on the client side, and (iii) the bandwidth for transmitting data between the model warehouse 108 and the clients 104A-K is limited.

[0009] The technical drawback / problem of the existing multi-modal setup includes inefficient transferring of a learning model from the model warehouse 108 to the clients 104A-K and the server 106 within the described multi-modal setup of FIG. 1, and inability to perform distributed training or fine-tuning. The primary challenges of the existing multi-modal setup include designing methods that minimize bandwidth usage while ensuring high inference accuracy, considering the clients' limited datasets and computational power.

[0010] Therefore, there arises a need to address the aforementioned technical problems / drawbacks for training learning models with limited resources and data availability.

[0011] SUMMARY

[0012] It is an object of the disclosure to provide a method of training one or more learning models in a distributed client / server data processing system having one or more clients interacting with a server and with a model warehouse, over a network. Moreover, the disclosure relates to a system of training the one or more learning models in the distributed client / server data processing system having the one or more clients interacting with the server and with the model warehouse, over the network. This object is achieved by the features of the independent claims. Further, implementation forms are apparent from the dependent claims, the description, and the figures.

[0013] The disclosure provides a method of training one or more learning models in a distributed client / server data processing system having one or more clients interacting with a server and with a model warehouse, over a network. Moreover, the disclosure relates to a system of training the one or more learning models in the distributed client / server data processing system having the one or more clients interacting with the server and with the model warehouse, over the network.

[0014] According to a first aspect, there is provided a method of training a plurality of learning models in a distributed client / server data processing system having a plurality of clients interacting with a server and with a model warehouse, over a network. The method is performed by the model warehouse which has access to the plurality of learning models associated with a respective plurality of pre-trained multi-modal encoders and decoders. The method includes receiving, from the server, a request including an identifier for the server, an identifier for a task to be performed by the system, an identifier for each client, a dataset size for each client, a computational resource identifier for each client, and a mode identifier for each client. The mode identifier identifies a type of data mode, of a plurality of data modes, that the respective client has access to, and the computational resource identifier identifies a processing capability of the respective client. The method includes selecting a learning model corresponding to the task included in the request from the server. The method includes deriving an estimate of the marginal structure of the output of an encoder corresponding to the selected learning model, for each of the respective clients, and the joint structure of the outputs of the encoders for each of the respective clients, based on the dataset size for each client and the processing capability for each client. The method includes sending, to the server, the derived estimate of the joint structure and a decoder corresponding to the selected learning model. The method includes sending, to each client, the corresponding marginal structure specific to the client. Each client selects an encoder based on the processing capability of the respective client. The clients and the server jointly train the encoders by using the corresponding marginal structure for the respective client, data available to each respective client, and the joint structure. The clients and the server jointly train the encoders and decoder using the respective data at each of the clients and the server.

[0015] The method considers the processing capabilities and dataset sizes of each client, ensuring that the learning model is suitable for the computational resources available, leading to efficient learning model training. By transmitting only the necessary components of the learning model (i.e., the joint structure to the server and the marginal structures to the clients), the method significantly reduces the number of bits required for data transmission compared to existing solutions, leading to substantial bandwidth savings. The training and fine-tuning of the learning models become feasible even in resource-constrained environments, such as cellular networks, due to the efficient use of available computational resources and bandwidth. The learning models that is trained exhibits enhanced robustness to change in data distribution, as it leverages a structure learned from a large dataset to ensure more reliable performance across varying conditions.

[0016] The method efficiently selects and transmits appropriate distributed learning models from the model warehouse to the server and the clients based on the server's request and the provided characteristics (e.g., computing power, available data mode, and dataset size) of the clients to ensure low bandwidth usage while achieving optimal inference performance. The method involves sending the decoder and the joint structure description to the server and the marginal structure of each data mode to the corresponding client. By optimizing the learning model based on the clients' and server's capabilities and the specific dataset, the method can achieve higher inference quality, providing better performance for the given tasks.

[0017] Preferably, the joint structure is selected as a mixture of a plurality of products of distributions of each of the plurality of data modes. Optionally, the decoder is fixed for the clients and the server. Optionally, the request is in a predefined format. Preferably, the estimates of the marginal structure for each client is selected as a mixture of distributions. The mixture of distributions may include a Gaussian distribution. The plurality of data modes may include text, images and videos. Preferably, the clients and the server jointly train the encoders by imposing the corresponding marginal structure for the respective client, data available to each respective client, and the joint structure.

[0018] Preferably, the estimates of the joint structure for each of the outputs of the encoders for each of the respective clients is selected as a mixture- of distributions. The estimates of the marginal structure for each client may be selected as a mixture of distributions. Each distribution in the mixture-of distributions of the estimates of the joint structure is the product of a plurality of distributions. The product includes one distribution per client, and for each client, that one distribution is chosen from amongst the distributions in the mixture of distributions estimates of the marginal structure for each client.

[0019] According to a second aspect, there is provided a system comprising means adapted for carrying out all the steps of the method described above.

[0020] The system considers the processing capabilities and dataset sizes of each client, ensuring that the learning model is suitable for the computational resources available, leading to efficient learning model training. By transmitting only the necessary components of the learning model (i.e., the joint structure to the server and the marginal structures to the clients), the system significantly reduces the number of bits required for data transmission compared to existing solutions, leading to substantial bandwidth savings. The training and fine-tuning of the learning models become feasible even in resource-constrained environments, such as cellular networks, due to the efficient use of available computational resources and bandwidth. The system efficiently selects and transmits appropriate distributed learning models from the model warehouse to the server and the clients. This system leverages the server's request and the provided characteristics (e.g., computing power, available data mode, and dataset size) of the clients to ensure low bandwidth usage while achieving optimal inference performance. The system involves sending the decoder and the joint structure description to the server and the marginal structure of each data mode to the corresponding client.

[0021] According to a third aspect, there is provided a computer program including instructions for carrying out all the steps of the above-described method, when the computer program is executed on a computer system.

[0022] Therefore, in contradistinction to the existing solutions, by transmitting only the necessary components of the learning model (i.e., the joint structure to the server and the marginal structures to the clients), the system / method significantly reduces the number of bits required for data transmission compared to existing solutions, leading to substantial bandwidth savings. The system considers the processing capabilities and dataset sizes of each client, ensuring that the learning model is suitable for the computational resources available, leading to efficient learning model training.

[0023] These and other aspects of the disclosure will be apparent from the implementation s) described below.

[0024] BRIEF DESCRIPTION OF DRAWINGS

[0025] Implementations of the disclosure will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0026] FIG. 1 illustrates a typical multi-modal setup of a data processing system;

[0027] FIG. 2 is a block diagram that illustrates a system of training one or more learning models in a distributed client / server data processing system in accordance with an implementation of the disclosure; FIG. 3 illustrates an exemplary system for training one or more learning models in a distributed client / server data processing system in accordance with an implementation of the disclosure;

[0028] FIGS. 4A-4B are flow diagrams that illustrate a method of training one or more learning models in a distributed client / server data processing system in accordance with an implementation of the disclosure; and

[0029] FIG. 5 is an illustration of a computer system in which the various architectures and functionalities of the various previous implementations may be implemented.

[0030] DETAILED DESCRIPTION OF THE DRAWINGS

[0031] Implementations of the disclosure provide a method of training one or more learning models in a distributed client / server data processing system having one or more clients interacting with a server and with a model warehouse, over a network. Moreover, the disclosure relates to a system of training the learning models in the distributed client / server data processing system having the clients interacting with the server and with the model warehouse, over the network.

[0032] To make solutions of the disclosure more comprehensible for a person skilled in the art, the following implementations of the disclosure are described with reference to the accompanying drawings.

[0033] Terms such as "a first", "a second", "a third", and "a fourth" (if any) in the summary, claims, and foregoing accompanying drawings of the disclosure are used to distinguish between similar objects and are not necessarily used to describe a specific sequence or order. It should be understood that the terms so used are interchangeable under appropriate circumstances, so that the implementations of the disclosure described herein are, for example, capable of being implemented in sequences other than the sequences illustrated or described herein. Furthermore, the terms "include" and "have" and any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, a method, a system, a product, or a device that includes a series of steps or units, is not necessarily limited to expressly listed steps or units but may include other steps or units that are not expressly listed or that are inherent to such process, method, product, or device.

[0034] FIG. 2 is a block diagram that illustrates a system 200 of training one or more learning models in a distributed client / server data processing system 202 in accordance with an implementation of the disclosure. The distributed client / server data processing system 202 have one or more clients 204A-K interacting with a server 206 and with a model warehouse 208, over a network. The model warehouse 208 has access to the one or more learning models associated with a respective one or more pre-trained multi-modal encoders and decoders. In an embodiment, the learning model is a large learning model. The model warehouse 208 receives a request from the server 206. Optionally, the request is in a predefined format. The request includes an identifier for the server 206, an identifier for a task to be performed by the system 200, an identifier for each client 204A- K, a dataset size for each client 204A-K, a computational resource identifier for each client 204A-K, and a mode identifier for each client 204A-K. The mode identifier identifies a type of data mode, of one or more data modes, that the respective client has access to, and the computational resource identifier identifies a processing capability of the respective client. The data modes may include text, images and videos.

[0035] The model warehouse 208 selects a learning model corresponding to the task included in the request from the server 206. The model warehouse 208 derives an estimate of the marginal structure of the output of an encoder corresponding to the selected learning model, for each of the respective clients 204A-K, and the joint structure of the outputs of the encoders for each of the respective clients 204A-K, based on the dataset size for each client 204A-K and the processing capability for each client 204A- K. Preferably, the joint structure is selected as a mixture of one or more products of distributions of each of the data modes. In some embodiments, the joint structure further includes a distribution identifier, a multihead flag, a per class flag, and a data mode identifier. The model warehouse 208 sends the derived estimate of the joint structure and a decoder corresponding to the selected learning model to the server 206. Optionally, the decoder is fixed for the clients 204 A-K and the server 206. The model warehouse 208 sends the corresponding marginal structure specific to the client to each client 204A-K. Each client 204A-K selects an encoder based on the processing capability of the respective client. The clients 204 A-K and the server 206 jointly train the encoders by using the corresponding marginal structure for the respective client, data available to each respective client (e.g., 204A-K), and the joint structure. The clients 204A-K and the server 206 jointly train the encoders and decoder using the respective data at each of the clients 204A-K and the server 206. Preferably, the estimates of the marginal structure for each client 204A-K is selected as a mixture of distributions. The mixture of distributions may include a Gaussian distribution. In some embodiments, the marginal structure further includes a distribution identifier, a multihead flag and a per class flag.

[0036] Preferably, the clients 204A-K and the server 206 jointly train the encoders by imposing the corresponding marginal structure for the respective client, data available to each respective client, and the joint structure. Preferably, the estimates of the joint structure for each of the outputs of the encoders for each of the respective clients (e.g. 204A-K) is selected as a mixture- of distributions. The estimates of the marginal structure for each client 204A-K may be selected as a mixture of distributions. Each distribution in the mixture-of distributions of the estimates of the j oint structure is the product of one or more distributions. The product includes one distribution per client, and for each client 204A-K, that one distribution is chosen from amongst the distributions in the mixture of distributions estimates of the marginal structure for each client 204A-K

[0037] The system 200 considers the processing capabilities and dataset sizes of each client 204A-K, ensuring that the learning model is suitable for the computational resources available, leading to efficient learning model training. By transmitting only the necessary components of the learning model (i.e., the joint structure to the server and the marginal structures to the clients), the system 200 significantly reduces the number of bits required for data transmission compared to existing solutions, leading to substantial bandwidth savings. The training and fine-tuning of the learning models become feasible even in resource- constrained environments, such as cellular networks, due to the efficient use of available computational resources and bandwidth. The system 200 efficiently selects and transmits appropriate distributed learning models from the model warehouse 208 to the server 206 and the clients 204A-K. This system 200 leverages the server's request and the provided characteristics (e.g., computing power, available data mode, and dataset size) of the clients 204A-K to ensure low bandwidth usage while achieving optimal inference performance. The system 200 involves sending the decoder and the joint structural description to the server 206 and the marginal structure of each data mode to the corresponding client.

[0038] The learning model provided by the model warehouse 208 may be tailored to the specific request and capabilities of the clients 204A-K and the server 206. The learning model may include a fixed structure. The size of the learning model that is transmitted to the client (e.g., 204A-K) is significantly smaller compared to existing models, which reduces computation overhead. The structure of the learning model may dynamically adapt to the capabilities of the clients 204A-K and the server 206. The quality of the learning model is dependent on the precision of the provided structures, ensuring that the learning model is fine-tuned to the specific characteristics of the data and processing power available at each client 204A-K. Consequently, varying the capabilities of the learning model, while keeping the dataset fixed, leads to different levels of inference performance. The system 200 leverages 6G networks to enhance the collaborative learning process among the clients / nodes 204A-K with limited resources and data availability. The system 200 enables the efficient distribution of pre-trained learning models, optimization of data transmission, and implementation of robust fine-tuning techniques to achieve high inference accuracy.

[0039] FIG. 3 illustrates an exemplary system for training one or more learning models in a distributed client / server data processing system in accordance with an implementation of the disclosure. The distributed client / server data processing system have one or more clients 304A-K interacting with a server 306 and with a model warehouse 308, over a network. The model warehouse 308 has access to the one or more learning models associated with a respective one or more pre-trained multi-modal encoders and decoders. Initially, the server 306 sends a request to the model warehouse 308. This request includes the task details, descriptions of clients 304A-K and available data modes, the size of datasets, and processing capabilities of the clients 304A- K. The request may be in a predefined format of: (server_id, task_id, {client_id, mode_id, dataset_size, processor_id} ), where the server_id is an identifier for the server 306, the task_id is an identifier for a task (e.g., Qi) to be performed by the system, the client_id is an identifier for each client 304A-K, the dataset_size is a dataset size for each client 304A-K, the processor_id is a computational resource identifier for each client 304A-K, and the mode_id is a mode identifier for each client 304A-K. The mode identifier identifies a type of data mode of one or more data modes, accessible to each respective client. The data modes (SI-SK) may include text, images and videos. The computational resource identifier identifies a processing capability of the respective client.

[0040] The model warehouse 308 receives the request from the server 306. The model warehouse 308 selects an appropriate learning model based on the requested task, the capabilities of the clients 304A-K and the server 306, and the available dataset. The model warehouse 308 estimates the marginal structure (i.e., statistics) of the output of an encoder corresponding to the selected learning model, for each of the respective clients 304A-K. The model warehouse 308 estimates the joint structure (i.e., statistics) of the latent variables, considering the available dataset size for each client 304A-K and the processing capability for each client 304A-K. The model warehouse 308 sends the joint structure, Qj, and a decoder, Wd, (i.e., a warehouse_id, a task_id, Wd, QJ) corresponding to the selected learning model to the server 306. The model warehouse 308 sends the corresponding marginal structure (i.e., a warehouse_id, a task_id, Qk) to each client 304A-K. Each client 304A-K selects an encoder based on the processing capability of the respective client. Optionally, the decoder is fixed for the clients 304A-K and the server 306. Each client 304A-K may select a potentially smaller encoder while keeping the dimension of the latent variables fixed.

[0041] Preferably, the estimates of the marginal structure, Qk, for each client 304A-K is selected as a mixture of distributions. The mixture of distributions may include a Gaussian distribution. In some embodiments, the marginal structure further includes a distribution identifier (distribution_id), a multihead flag (multihead_flag) and a per class flag (per class_flag). The marginal structure, Qk, may be described as: (distribution_id, multihead_flag, per class_flag, N_components, coeff_components, param_component), where N_components is the number of components of the joint structure, and coeff_components is the coefficients of the joint structure.

[0042] Preferably, the joint structure, Qj, is selected as a mixture of one or more products of distributions of each of the data modes. In some embodiments, the joint structure further includes a distribution identifier (distribution_id), a multihead flag (multihead_flag), a per class flag (per class_flag), and a data mode identifier (mode_id). The joint structure, Qj, may be described as: (distribution_id, multihead_flag, per class_flag, N_components, coeff_components, {mode_id, component_id, param_component} ), where N_components is the number of components of the joint structure, and coeff_components is the coefficients of the joint structure. Each component is the product of the K data mode components.

[0043] The learning model is trained by the clients 304A-K and the server 306 jointly. Each client 304A-K imposes the structure of latent variables (i.e., the marginal structure, Qk, and the joint structure, Qj) on its encoder. The decoder may impose the joint structures of the encoders while keeping the decoder fixed. The encoders and the decoder fine-tune both the structures (i.e., the marginal structure, Qk, and the joint structure, Qj) and themselves. The clients 304A-K and the server 306 jointly train the encoders and the decoder using the respective data at each of the clients 304A-K and the server 306.

[0044] In an embodiment, the marginal structure of the latent variable space of each client 304A-K, optionally for every head h and optionally every class c, is estimated as a mixture distribution, for example, as a Gaussian-Mixture distribution, which is described as:

[0045] FIGS. 4A-4B are flow diagrams that illustrate a method of training one or more learning models in a distributed client / server data processing system in accordance with an implementation of the disclosure. The distributed client / server data processing system have one or more clients interacting with a server and with a model warehouse, over a network. The method is performed by the model warehouse which has access to the one or more learning models associated with a respective one or more pretrained multi-modal encoders and decoders. At a step 402, a request is received from the server. The request includes an identifier for the server, an identifier for a task to be performed by a system, an identifier for each client, a dataset size for each client, a computational resource identifier for each client, and a mode identifier for each client, where the mode identifier identifies a type of data mode, of one or more data modes, that the respective client has access to, and the computational resource identifier identifies a processing capability of the respective client. Optionally, the request is in a predefined format. At a step 404, a learning model is selected which corresponds to the task included in the request from the server. At a step 406, an estimate of the marginal structure of the output of an encoder corresponding to the selected learning model is derived, for each of the respective clients, and an estimate of the joint structure of the outputs of the encoders for each of the respective clients, based on the dataset size for each client and the processing capability for each client. At a step 408, the derived estimate of the joint structure and a decoder corresponding to the selected learning model is sent to the server. At a step 410, the corresponding marginal structure specific is sent to each client. Each client selects an encoder based on the processing capability of the respective client. The clients and the server jointly train the encoders by using the corresponding marginal structure for the respective client, data available to each respective client, and the joint structure. The clients and the server jointly train the encoders and decoder using the respective data at each of the clients and the server. Optionally, the clients and the server jointly train the encoders by imposing the corresponding marginal structure for the respective client, data available to each respective client, and the joint structure.

[0046] The method considers the processing capabilities and dataset sizes of each client, ensuring that the learning model is suitable for the computational resources available, leading to efficient learning model training. By transmitting only the necessary components of the learning model (i.e., the joint structure to the server and the marginal structures to the clients), the method significantly reduces the number of bits required for data transmission compared to existing solutions, leading to substantial bandwidth savings. The training and fine-tuning of the learning models become feasible even in resource-constrained environments, such as cellular networks, due to the efficient use of available computational resources and bandwidth. The learning models that is trained exhibits enhanced robustness to changes in data distribution, as it leverages a structure learned from a dataset to ensure more reliable performance across varying conditions.

[0047] The method efficiently selects and transmits appropriate distributed learning models from the model warehouse to the server and the clients. This method leverages the server's request and the provided characteristics (e.g., computing power, available data mode, and dataset size) of the clients to ensure low bandwidth usage while achieving optimal inference performance. The method involves sending the decoder and the joint structural description to the server and the marginal structure of each data mode to the corresponding client. By optimizing the learning model based on the clients' and server's capabilities and the specific dataset, the method can achieve higher inference quality, providing better performance for the given tasks.

[0048] Preferably, the joint structure is selected as a mixture of one or more products of distributions of each of the data modes. Optionally, the decoder is fixed for the clients and the server. In some embodiments, the joint structure further includes a distribution identifier, a multihead flag, a per class flag, and a data mode identifier.

[0049] Preferably, the estimates of the marginal structure for each client is selected as a mixture of distributions. The mixture of distributions may include a Gaussian distribution. In some embodiments, the marginal structure further includes a distribution identifier, a multihead flag and a per class flag. The data modes may include text, images and videos.

[0050] Preferably, the estimates of the joint structure for each of the outputs of the encoders for each of the respective clients is selected as a mixture- of distributions. The estimates of the marginal structure for each client may be selected as a mixture of distributions. Each distribution in the mixture-of distributions of the estimates of the joint structure is the product of a plurality of distributions. The product includes one distribution per client, and for each client, that one distribution is chosen from amongst the distributions in the mixture of distributions estimates of the marginal structure for each client.

[0051] FIG. 5 is an illustration of a computer system in which the various architectures and functionalities of the various previous implementations may be implemented. As shown, the computer system 500 includes at least one processor 504 that is connected to a bus 502, wherein the computer system 500 may be implemented using any suitable protocol, such as PCI (Peripheral Component Interconnect), PCI-Express, AGP (Accelerated Graphics Port), Hyper Transport, or any other bus or point-to-point communication protocol (s). The computer system 500 also includes a memory 506.

[0052] Control logic (software) and data are stored in the memory 506 which may take a form of random-access memory (RAM). In the disclosure, a single semiconductor platform may refer to a sole unitary semiconductor-based integrated circuit or chip. It should be noted that the term single semiconductor platform may also refer to multi-chip modules with increased connectivity which simulate on-chip modules with increased connectivity which simulate on-chip operation and make substantial improvements over utilizing a conventional central processing unit (CPU) and bus implementation. Of course, the various modules may also be situated separately or in various combinations of semiconductor platforms per the desires of the user.

[0053] The computer system 500 may also include a secondary storage 510. The secondary storage 510 includes, for example, a hard disk drive and a removable storage drive, representing a floppy disk drive, a magnetic tape drive, a compact disk drive, digital versatile disk (DVD) drive, recording device, universal serial bus (USB) flash memory. The removable storage drive at least one of reads from and writes to a removable storage unit in a well-known manner.

[0054] Computer programs, or computer control logic algorithms, may be stored in at least one of the memory 506 and the secondary storage 510. Such computer programs, when executed, enable the computer system 500 to perform various functions as described in the foregoing. The memory 506, the secondary storage 510, and any other storage are possible examples of computer-readable media. In an implementation, the architectures and functionalities depicted in the various previous figures may be implemented in the context of the processor 504, a graphics processor coupled to a communication interface 512, an integrated circuit (not shown) that is capable of at least a portion of the capabilities of both the processor 504 and a graphics processor, a chipset (namely, a group of integrated circuits designed to work and sold as a unit for performing related functions, and so forth).

[0055] Furthermore, the architectures and functionalities depicted in the various previous-described figures may be implemented in a context of a general computer system, a circuit board system, a game console system dedicated for entertainment purposes, an application-specific system. For example, the computer system 500 may take the form of a desktop computer, a laptop computer, a server, a workstation, a game console, an embedded system.

[0056] Furthermore, the computer system 500 may take the form of various other devices including, but not limited to a personal digital assistant (PDA) device, a mobile phone device, a smart phone, a television, and so forth. Additionally, although not shown, the computer system 500 may be coupled to a network (for example, a telecommunications network, a local area network (LAN), a wireless network, a wide area network (WAN) such as the Internet, a peer-to-peer network, a cable network, or the like) for communication purposes through an I / O interface 508.

[0057] It should be understood that the arrangement of components illustrated in the figures described are exemplary and that other arrangement may be possible. It should also be understood that the various system components (and means) defined by the claims, described below, and illustrated in the various block diagrams represent components in some systems configured according to the subject matter disclosed herein. For example, one or more of these system components (and means) may be realized, in whole or in part, by at least some of the components illustrated in the arrangements illustrated in the described figures.

[0058] In addition, while at least one of these components are implemented at least partially as an electronic hardware component, and therefore constitutes a machine, the other components may be implemented in software that when included in an execution environment constitutes a machine, hardware, or a combination of software and hardware.

[0059] Although the disclosure and its advantages have been described in detail, it should be understood that various changes, substitutions, and alterations can be made herein without departing from the spirit and scope of the disclosure as defined by the appended claims.

Claims

CLAIMS1. A method of training a plurality of learning models in a distributed client / server data processing system (202) having a plurality of clients (204A-K, 304A-K) interacting with a server (206, 306) and with a model warehouse (208, 308), over a network, comprising steps, performed by the model warehouse which has access to the plurality of learning models associated with a respective plurality of pre-trained multi-modal encoders and decoders, of: receiving, from the server, a request including an identifier for the server, an identifier for a task to be performed by the system, an identifier for each client, a dataset size for each client, a computational resource identifier for each client, and a mode identifier for each client, where the mode identifier identifies a type of data mode, of a plurality of data modes, that the respective client has access to, and the computational resource identifier identifies a processing capability of the respective client; selecting a learning model corresponding to the task included in the request from the server; deriving an estimate of the marginal structure of the output of an encoder corresponding to the selected learning model, for each of the respective clients, and the joint structure of the outputs of the encoders for each of the respective clients, based on the dataset size for each client and the processing capability for each client; sending, to the server, the derived estimate of the joint structure and a decoder corresponding to the selected learning model; and sending, to each client, the corresponding marginal structure specific to the client; wherein each client selects an encoder based on the processing capability of the respective client; and wherein the clients and the server jointly train the encoders by using the corresponding marginal structure for the respective client, data available to each respective client, and the joint structure; and wherein the clients and the server jointly train the encoders and decoder using the respective data at each of the clients and the server.

2. The method of claim 1, wherein the joint structure is selected as a mixture of a plurality of products of distributions of each of the plurality of data modes.

3. The method of claim 1, wherein the decoder is fixed for the clients and the server.

4. The method of claim 1, wherein the request is in a predefined format.

5. The method of claim 1 , wherein the estimates of the marginal structure for each client is selected as a mixture of distributions.

6. The method of claim 5, wherein the mixture of distributions includes a Gaussian distribution.

7. The method of claim 1, wherein the plurality of data modes includes text, images and videos.

8. The method of claim 1, wherein the clients and the server jointly train the encoders by imposing the corresponding marginal structure for the respective client, data available to each respective client, and the joint structure.

9. The method of claim 1, wherein the estimates of the joint structure for each of the outputs of the encoders for each of the respective clients is selected as a mixture- of distributions.

10. The method of claim 9, wherein the estimates of the marginal structure for each client is selected as a mixture of distributions, and wherein each distribution in the mixture-of distributions of the estimates of the joint structure is the product of a plurality of distributions, wherein the product comprises one distribution per client, and wherein for each client, that one distribution is chosen from amongst the distributions in the mixture of distributions estimates of the marginal structure for each client.

11. A system (200) comprising means adapted for carrying out all the steps of the method according to any preceding method claim.

12. A computer program comprising instructions for carrying out all the steps of the method according to any preceding method claim, when said computer program is executed on a computer system.

Images