Distributed training of machine learning models for personalization

Abstract

A method for distributed training of an artificial intelligence (AI) machine learning (ML) model, a system for distributed training of an artificial intelligence (AI) machine learning (ML) model, and a computer-readable medium having a computer program stored thereon are claimed, wherein the computer program, when executed by one or more processors, implements the method for distributed training of an ML model. The method comprises the following steps: collecting user-generated data through user input in each of one or more UEs during a data collection period; sending training data from a server to the one or more UEs; performing ML model training in each of the one or more UEs based on the collected data and the training data until a training stop criterion is met; and obtaining the trained ML model from the one or more UEs at the server.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular, to machine learning models for personalizing user devices. Background Art

[0002] Reference US 8,429,103 B1 (2012-06-22, Google Inc.) discloses a method for training a machine learning (ML) model executed in a user device (such as a mobile phone) when a data item is obtained from a mobile application or a network. The machine learning method may include: determining at least one feature based on the received data, and generating output data by performing a machine learning operation on the at least one feature. The output data may be provided to an application, a network, etc. A data aggregation and representation engine (DARE) may be provided that continuously receives and stores input data that may come from multiple sources. The stored input data may be aggregated to discover features within the data. For example, prior art machine adaptation techniques may use incremental learning algorithms that require limited or no historical information for training.

[0003] References H. Brendan McMahan et al. (2016) Communication-Efficient Learning of Deep Networks from Decentralized Data and Yujun Lin et al. (2018) Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training disclose a method for training so-called "deep learning" models in which training data is distributed across mobile devices and aggregating locally computed updates to learn a shared model. To improve communication in distributed stochastic gradient descent, several solutions are used: sending only sufficiently large weight updates during warm-up training, momentum correction, local gradient clipping, momentum factor masking, local gradient accumulation, and less aggressive gradient reduction. The method was tested in the context of image, speech, and text data processing.

[0004] The above-described methods may be considered the closest analogs of the claimed invention. Summary of the Invention

[0005] Technical issues

[0006] The solution of reference US8,429,103B1 (2012-06-22, Google Inc.) has disadvantages, such as the limitation of the application field to mobile phones, the need to collect personal data of users, and the risk of so-called "overfitting" of the model (an unwanted phenomenon that occurs when the error probability of the training algorithm on an entity of the test data set is significantly higher than the average error on the training data set).

[0007] The method of reference H. Brendan McMahan et al. (2016) has disadvantages, such as "overfitting" of the model on new data, requiring users to wait for training to complete until a better performing model is provided to the user, and the training method is constrained by stochastic gradient descent (SGD).

[0008] Problem Solution

[0009] This section discloses various aspects and embodiments of the claimed invention and is intended to present a brief summary of the features of the claimed subject matter and its embodiments. Detailed features of the technical means and methods for achieving the combinations of features of the claimed invention are provided below. Neither this summary of the invention nor the detailed description and accompanying drawings provided below should be considered to limit the scope of the claimed invention. The scope of legal protection for the claimed invention is defined solely by the appended claims.

[0010] Taking into account the above-mentioned drawbacks of the prior art, the object of the present invention is to provide a solution aimed at eliminating the above-mentioned disadvantages, reducing the risk of security breaches of users' personal data, and reducing the costs of data transmission over a network connection for the purpose of training machine learning models for personalized user devices. In addition, the claimed invention eliminates the risk of "overfitting" of the model (which in this case may also be called "forgetting"). In addition, the claimed solution makes it possible to group users according to the topics of their interest. The technical result achieved by the claimed invention is that the quality of training personalized artificial intelligence models is improved, while preventing their "overfitting" and reducing the costs of data transmission over a network connection.

[0011] To achieve the objective, according to one aspect, the present invention is directed to a method for distributed training of an artificial intelligence (AI) machine learning (ML) model, comprising the following steps: (a) initializing one or more machine learning (ML) models in a server; (b) distributing the one or more ML models in one or more user equipment (UEs) connected to the server via a communication network; (c) collecting user-generated data through user input in each of the one or more UEs during a data collection period; (d) sending training data from the server to the one or more UEs; (e) performing ML model training in each of the one or more UEs based on the collected data and the training data until a training stop criterion is met; (f) obtaining the trained ML model from the one or more UEs at the server; (g) updating the ML model at the server by aggregating the trained ML models obtained from the one or more user equipment; (h) sending the updated ML model to the one or more UEs; and (i) repeating steps (c)-(h) one or more times until an ML model that meets one or more ML model quality criteria is obtained.

[0012] In an embodiment of the present invention, the method may further include the following steps: identifying a personalized group of users for each of the one or more UEs based on user-generated data collected on each of the one or more UEs; grouping, at the server, ML models acquired from the one or more UEs into the personalized groups; and sending the updated ML models grouped into the personalized groups to the UEs included in the corresponding personalized groups.

[0013] In an embodiment, the ML model may be configured to predict words and phrases when a user enters a text message in a UE, wherein the user-generated data is the words and phrases entered by the user. In an embodiment, the ML model may be configured to recognize objects in images captured by one or more cameras of the UE, wherein the user-generated data is the images from the one or more cameras of the UE and / or labels assigned by the user to objects present in the images. In an embodiment, the ML model may be configured to recognize handwritten input received from the user via a touchscreen and / or a touchpad of the UE, wherein the user-generated data is the handwritten input and / or the user's selection of a variation of a character and / or word suggested by the ML model based on the handwritten input from the user. In an embodiment, the ML model may be configured to recognize voice input received from the user via one or more microphones of the UE, wherein the user-generated data is the voice input and / or the user's selection of a variation of a word and / or phrase suggested by the ML model based on the voice input from the user. In an embodiment, the ML model may be configured to: identify one or more characteristics of the UE's environment and / or one or more user actions, wherein the one or more characteristics of the UE's environment are one or more of the UE's time, date, weekday, illumination, temperature, geographic location, and spatial location, and wherein the user-generated data is user input into one or more program applications in the UE. In an embodiment, the training data may include a portion of publicly available training data from an initial dataset.

[0014] In an embodiment, the training stopping criterion is convergence of the ML model implemented in one or more UEs. In an embodiment, the training stopping criterion is a predetermined ML model quality characteristic value achieved by the ML model. In an embodiment, the training stopping criterion is a predetermined number of training periods achieved.

[0015] According to another aspect of the present invention, a system for distributed training of artificial intelligence (AI) machine learning (ML) models, the system comprising: a server; and one or more user equipment (UEs) connected to the server via a communication network; wherein the server is configured to: initialize one or more machine learning (ML) models; distribute one or more ML models in one or more user equipment (UEs); send training data to the one or more UEs; obtain trained ML models from the one or more UEs; update the ML models by aggregating the trained ML models obtained from the one or more UEs; and send the updated ML models to the one or more UEs; and wherein the one or more UEs are configured to: collect user-generated data through user input during a data collection period; receive training data from the server; and train the ML model based on the collected data and the training data until a training stop criterion is met.

[0016] In another aspect, the present invention provides a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by one or more processors, implements the distributed training method for an ML model according to the first aspect of the above aspects. The inventive concept of the present invention can be implemented in other subject matter (such as a computer program, a computer program product, a server, a user device, a wireless communication system, etc.).

[0017] Beneficial effects of the present invention

[0018] Thus, additional personalization of the trained ML model is achieved and the accuracy of the ML model for different user groups is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are provided here to facilitate understanding of the essence of the present invention. The accompanying drawings are schematic and not drawn to scale. The accompanying drawings are for illustration only and are not intended to limit the scope of the present invention.

[0020] Figure 1 A flowchart illustrating a method for distributed training of a machine learning (ML) model according to the first aspect of the present invention;

[0021] Figure 2 The present invention illustrates a process of training an ML model in a user equipment (UE) and collecting personalized ML models in a server.

[0022] Figure 3 Schematically illustrates ML model training in a UE according to the present invention.

[0023] Figure 4is a block diagram illustrating electronic devices (eg, one or more UEs) in a network environment according to various embodiments. DETAILED DESCRIPTION

[0024] Machine learning is a class of artificial intelligence methods that learn by applying solutions to multiple similar problems rather than solving the problem directly. In certain cases, a large number of machine learning methods are based on the use of neural networks, however there are other methods that use the concept of training data sets. In the context of the present invention, as non-limiting examples, machine learning methods can be used for the following purposes: object recognition (e.g., in images), word prediction (e.g., in various applications where a user enters a message or search query via an application interface in a user device), intelligent processing of super-resolution images, speech recognition (e.g., in applications that receive voice input from a user and convert the voice input data into text), handwritten text recognition (e.g., in applications that receive user input by writing letters and other characters on a touch screen of a user device with the help of a pen or the user's finger), and machine learning methods can be used in different software applications known as "intelligent assistants".

[0025] In the context of the present invention, it is assumed that a user device includes one or more artificial intelligence features, for example, implemented by software. Taking into account various characteristics of the user of the user device, a system including such artificial intelligence features is configured to "learn" with the aid of one or more machine learning methods to personalize the user device features implemented in the form of different media, services, software applications, etc. As non-limiting examples, personalization can be based on, for example, a user vocabulary (e.g., determined when the user composes a message in an instant messaging, email application, SMS, etc.), user topics of interest (e.g., determined based on the user's search queries in various search systems), information about the web pages browsed by the user, the frequency and duration of browsing of specific web pages, etc. However, the collection of user data and its transmission outside the user device are subject to various restrictions related to the security of the user's personal data, the protection of the user's privacy, etc.

[0026] Conventionally, artificial intelligence models are trained in one or more servers. However, this involves, in particular, the following problems: (1) the artificial intelligence system may not be able to adapt to the local conditions of a given user device; and (2) publicly available data may differ from real data. Usually, when object recognition or super-resolution image processing tasks are to be solved, adaptation to the local conditions of a given user device is achieved in the form of adaptation to the hardware parts of the device, in particular to the characteristics of the camera provided therein, or when speech recognition tasks are to be solved, adaptation to the local conditions of a given user device is achieved in the form of adaptation to the characteristics of one or more microphones included in the device. When speech recognition tasks are to be solved, adaptation to the user can be achieved based on recognized user interests (for example, in word prediction when a user types a message) or based on the speech of this specific user.

[0027] To address the aforementioned issues, adaptation of the AI system can be achieved by executing the training algorithm in the user's device. However, this solution, in turn, involves other problems, including insufficient data to adequately train the model within the user's device and the lack of the ability to collect user data for each given user in a remote server (particularly in view of the aforementioned concerns about the security and privacy of user personal data).

[0028] In turn, these problems are currently addressed in the aforementioned prior art by means of distributed “subsequent training” (which may also be characterized as a kind of “fine-tuning”, hereinafter referred to as training or subsequent training) of the AI model across multiple different user devices. However, as indicated above, prior art solutions in this area have the following associated problems: 1) such “subsequent training” of the AI model may result in “overfitting” or “forgetting” all the data originally included in the model as the model is adapted to a specific user; 2) users, their devices, and their environments may be too different to enable such distributed “subsequent training” of the model across multiple devices; and 3) this approach is expensive due to the high cost of data transmission over network connections.

[0029] The invention claimed for protection is generated in view of the above-mentioned problems in the prior art. The following methods are proposed to solve the above-mentioned problems in the prior art, which will be described in more detail in the following detailed description of the present invention.

[0030] 1) To prevent “overfitting” and ensure personal data security and user privacy, a small amount of initial training data is used in model training.

[0031] 2) Group users into different groups to obtain a new personalized model for each user group.

[0032] 3) During distributed model training, gradients are collected regarding the model trained in each user device as discussed above, rather than as in the closest prior art analogs discussed above.

[0033] Taking into account the above considerations, the object achieved by the claimed invention is to improve the quality of training of personalized artificial intelligence models and prevent their "overfitting" while reducing the cost of data transmission via a network connection. The invention basically aims to provide a way for continuously updating a machine learning model based on the user's data without collecting any personal data of the user, with low cost of data transmission via a network connection, improved model persistence and frequent updating thereof.

[0034] First, a small amount of initial training data is used in model training, which allows to prevent "overfitting" of the model ("forgetting" initial information) based on newly acquired data. Each user then trains the model on their own user device during several time periods and sends the updated machine learning model to the server, where the models acquired from the user devices are averaged. Thus, each end user continuously receives updates in the form of more accurate machine learning models adapted based on data generated by multiple users. As a result, the artificial intelligence features in the corresponding applications in each user device become more accurate. In addition, the security of each user's personal data stored in the form of, for example, photos, messages, text files, links to web pages, sound data (captured by the microphone of the user device), etc. is guaranteed. The trained model is prevented from "forgetting" the initial information obtained when training the model on publicly available data.

[0035] According to the present invention, an initial machine learning (ML) model for a software application including artificial intelligence (AI) features is trained in a server based on publicly available data. The initial ML model is provided by a user device or installed when the user device is in communication with a communication network in an initial training process. There is then a waiting period until a sufficient amount of data is generated by the user using the application including artificial intelligence features in the user device to enable adaptation of the machine learning model.

[0036] A machine learning model type is identified that is appropriate for the user and the user device based on the user-generated data and other accessible information, such as, for example, the make and model of the user device. By way of example and not limitation, for the purpose of machine learning model adaptation, a personalized group is formed based on the identified machine learning model type and / or the type, make, or model of the user device and / or user interests determined based on the user-generated data during the waiting period.

[0037] Based on the identified machine learning model type, the server sends the current version of the machine learning model to the user device. In this case, in a preferred embodiment, a specific version of the machine learning model is only sent to users within the corresponding personalized group.

[0038] To improve personal data security, a portion of publicly available data from the initial dataset is sent to the user for initial model training. This also prevents the machine learning model from "forgetting" the initial data by "overfitting" the model on a specific user's data. Model training is then performed on the user's device, using the ML model sent from the server as the initial model. Training is performed in this phase until model convergence is achieved across different user devices within a personalized group, for example, or until a certain predetermined maximum number of training iterations has been achieved.

[0039] Each user device that has completed ML model training sends its trained ML model to a server (such as a central server and / or a model aggregation server). Personalized models trained on different user devices (e.g., within a personalized group) are aggregated at the server. Aggregation is achieved, for example, by creating an average model. As a result of the aggregation, a new version of the model for a specific type is obtained. This new version of the model is sent to the user devices in the corresponding personalized group.

[0040] The aforementioned operation of sending a portion of the publicly available data from the initial dataset for initial model training to the user advantageously prevents the model from "overfitting" on new data in the user's device and protects user privacy by preventing a third party from identifying data representing the user's personality, for example, if the personalized ML model sent to the server is intercepted. A portion of the initial training data is sent to each user device, and the ML model training process is performed in each user device by combining the data collected in that user device with the initial data sent to the user device. Compared to the amount of initial training data, the ML model adaptation in the user device only involves a small portion of the available user data.

[0041] Conventional solutions that lack the ability to add a portion of the initial training data to the ML model training process result in the ML model being "overfitted" in a given user's device at a given time, characterized by the ML model "forgetting" all information previously stored in the ML model. Consequently, if the context of a message typed by a user differs from the frequent contexts previously collected in the user's device for training the personalized machine learning model, such an "overfitted" model may be unable to appropriately predict words based on user input, such as in a scenario where a "virtual keyboard" is used in a messaging application.

[0042] In a preferred embodiment of the claimed solution, the amount of data from the initial dataset and the amount of data from the user-generated dataset used for ML model training in a given user device are acquired in a 1:1 ratio. This provides an optimal balance between new data (i.e., data generated by the user of the given user device) and initial data (data obtained from the server) in ML model training. In this way, the ML model "acquires" new information without "forgetting" the initial information. If the ratio were, for example, 1:2, the balance would shift towards "new" data (user-generated data), which would result in "forgetting" the initial data. However, it should be understood that the scope of the present invention is not limited to the use of the ratio in the preferred embodiment of the present invention, and in other embodiments of the present invention, the ratio may be different for different users, for example, based on specific criteria that characterize the "behavior" of each given user. For example, in a specific embodiment of the present invention, different users may be assigned different coefficients based on the "contribution" of their generated data to ML model training (e.g., within a specific personalized group).

[0043] In order to obtain such a “combined” model based on both the data generated by the user of a given user device and the data of the initial training data set, any machine learning process known in the art may be used. As an example, the following references describe machine learning processes suitable for use in the context of the present invention:

[0044] Bishop, CM (2006) "Pattern recognition and Machine Learning", SpringerScience, pp. 232-272; Mozer, MC (1995). "A Focused Backpropagation Algorithm for Temporal Pattern Recognition". In Chauvin, Y.; Rumelhart, D. Backpropagation: Theory, architectures, and applications. ResearchGate. Hillsdale, NJ: Lawrence Erlbaum Associates, pp. 137-169.

[0045] Within a specific personalized group, in a preferred embodiment, ML model training is performed in the user device until a training stop condition is met in the user device (such as ML model convergence is achieved in the user device). Afterwards, the trained ML models are sent to the server, where they are aggregated (by averaging the ML models, as a non-limiting example).

[0046] Alternatively or additionally, the ML model training stopping criteria may include the ML model achieving a predetermined ML model quality feature value (which may be expressed in terms of prediction accuracy or task): thus, in a task of predicting the next word, the word prediction accuracy may be evaluated; in a task such as recognizing handwritten text, the text recognition accuracy or word accuracy may be evaluated. Based on the examples provided above, different methods for evaluating ML model quality may be apparent to those skilled in the art, depending on the task the model is intended to solve.

[0047] The model may be sent to the server only partially, but not completely: those model parameters whose changes relative to the previous iteration do not exceed a certain predetermined threshold may not be sent to the server. In this case, the average will use the parameter values from the previous iteration of the model. The threshold used to make the decision to send the ML model from the user device to the server can be determined, for example, based on a trade-off between the requirements for ML model accuracy and the limit on the amount of data transmitted over the network connection between the user device and the server.

[0048] The personalized model may be updated, for example, based on model averaging.

[0049] The present inventors propose to perform ML model training in the user device until any one of the predetermined training stopping criteria is met, rather than calculating and sending the gradient for stochastic gradient descent as in the case of the prior art analogs discussed above. As an example, the criteria can be the achievement of a predetermined maximum number of ML model training periods or the achievement of convergence of a specific model according to the optimization process. As an alternative to or in addition to the above criteria, other ML model training stopping criteria that can be envisioned by those skilled in the art upon reading the present disclosure are possible.

[0050] This reduces the need for data communication over the network connection between the user device and the server in order to implement the distributed ML model training process, thereby reducing the economic cost to the user.

[0051] In some embodiments of the present invention, distributed ML model training can further improve the trained model's efficiency in predicting rare words, events, or objects. This can be achieved by modifying the training criteria. This is due to the fact that rare categories (words, objects, etc.) occur relatively rarely in most user devices participating in distributed ML model training, causing the ML model training process to ignore them and, therefore, resulting in poor prediction results for these categories. Modifying the ML model training criteria can effectively overcome this problem if the new criteria are sensitive to these categories with low probability of occurrence.

[0052] As an example, in a standard training criterion, one can name, for example, the cross entropy loss function between the true class distribution (p) and the distribution assigned to the classes by a given model (q). This criterion can be illustrated by the following expression provided below:

[0053] [Mathematical formula 1]

[0054]

[0055] The present invention proposes to use a new criterion in training, wherein the new criterion is the sum of the cross entropy between p and q and the Kullback-Leibler distance between q and p:

[0056] [Mathematical formula 2]

[0057]

[0058] In Mathematical Formula 2, when the additional estimate of the true probability p(w) is significantly lower than q(w), a penalty factor is applied to the prediction of the q(w) model. The estimated value p(w) can be obtained from a discriminant algorithm trained to separate the true data from the data selected from the model q(w) using techniques known to those skilled in the art. The use of this method enables the prediction accuracy in the case of ratio classes to be improved by up to 1.5%, and leads to an overall improvement in prediction accuracy of up to 0.2%.

[0059] According to the above operations of the claimed method, users are grouped into various personalized groups based on the following criteria: the subject matter of text messages generated by the user, the user's geographic location, the user's age, the type of hardware running one or more software applications, and the type of hardware using one or more artificial intelligence features. It will be understood that the above criteria for grouping users into personalized groups are merely non-limiting examples. Moreover, those skilled in the art will understand that other criteria for grouping users into personalized groups are possible, in addition to or in lieu of the above criteria. In addition, users can be grouped into personalized groups based on, for example, the following:

[0060] - Technical specifications of the user's device: screen size, RAM size, processor type, etc.

[0061] - The geographic location of the user's device;

[0062] - User-generated data content, for example, on web pages (likes, comments, replies, posts, public, etc.);

[0063] - Demographic metadata (user gender, age, marital status, nationality).

[0064] According to the present invention, it is advantageous to have separate ML models for different groups of users or user devices. In order to identify the personalized group to which a user of a given user device should be assigned, a classification module can be implemented in the user device. At least one of the following (but not limited to) can be used as input data for the classification module:

[0065] - User-generated data in user devices;

[0066] - User equipment model;

[0067] - Demographic data about users;

[0068] - Geolocation tags, etc.

[0069] The number of personalized groups can be defined manually or by any suitable clustering method. Each personalized group corresponds to an ML model or an ML model type.

[0070] A model associated with a specific corresponding group will have a higher accuracy than a model common to all groups. Thus, as a non-limiting example, users discussing science and technology-related topics via text messaging in different applications on their mobile devices will receive more accurate word predictions in their topics when typing messages in their user devices because the personalized ML model in their user devices will be based only on data acquired from users with similar interests.

[0071] Aggregating multiple ML models from users combined in a common personalization group solves the problem of small ML model training dataset sizes. However, at the same time, the ML models remain personalized, in the sense that a separate ML model is generated and updated for each user personalization group. As a result, users within a given personalization group receive more accurate personalized ML models based on their interests, habits, details, and / or characteristics of their hardware and / or software.

[0072] In an embodiment of the present invention, models with different architectures (different machine learning algorithms) can be provided to different users, and the model with the best architecture can be identified based on the results of model training. For this purpose, additional units can be provided on the server side to generate new ML model architectures and hyperparameters for these models. If necessary, any AI system can also be expanded by including additional functions that allow new ML models to be tested on user-generated data.

[0073] The present invention is implemented in a standard wireless communication network architecture and includes hardware and / or software devices on the server side and hardware and / or software devices on the user equipment side. As a non-limiting example, the server-side device may include units and / or modules that perform the following operations: providing an initial ML model; initializing a machine learning (ML) model at the server; distributing (sending) the ML model to one or more user equipment (UEs) connected to the server via a communication network; sending training data of an initial sample from the server to the one or more UEs; receiving the ML model trained on the one or more UEs from the one or more UEs; and updating the personalized ML model at the server by averaging the trained ML models received from the one or more UEs. The above blocks and / or modules are configured to repeat the operations performed by them any number of times necessary based on the number of times the above operations must be repeated in order to obtain one or more personalized ML models with the desired accuracy and efficiency.

[0074] On the user equipment side, specific hardware and / or software models and / or units are also used to implement the present invention. As a non-limiting example, a user interface generation unit that provides a user interface that enables a user to control the UE may be provided. The UE may include various input / output tools, such as but not limited to a touch screen, one or more keys, one or more microphones, one or more cameras and / or video cameras, a positioning system signal receiver (such as GPS, GLONASS, GALILEO, etc.), one or more sensors for determining physical parameters of the user equipment and / or its environment (such as the spatial position, temperature, illumination level, etc. of the user equipment), and one or more speakers. It will be understood by those skilled in the art that the above list of I / O tools is provided only as an illustrative example and not a limiting example, and any suitable combination of the above and / or any other I / O tools may be provided depending on the specific implementation of the user equipment.

[0075] Furthermore, various units and / or modules may be provided in the user device for text recognition, handwriting input recognition, image analysis, object recognition in images, fingerprint recognition, converting speech input to text, grammatical and / or statistical analysis of natural language, generating text in natural language, converting text to speech output, etc. It should be understood that this list of possible units and / or modules allowing data input in the user device to be processed by I / O tools is not limiting, and that in addition to or as an alternative to the aforementioned tools, other tools for processing input data and / or processing data for output may be provided depending on the specific implementation of the claimed invention.

[0076] The data processing unit and / or module sends data derived from user input received via the I / O tool to one or more AI features, wherein the AI features implement one or more machine learning (ML) models in one or more software applications running on the user device. The one or more ML models receive the data from the data processing unit and / or module and use the data, in particular, to generate output in response to the data received from the user, as well as for their own training. Thus, for example, in response to user text input via an on-screen keyboard or one or more keys, the ML model can suggest predicted variations of the user input in the form of one or more variations of a word or phrase that the user may wish to type in a text message entry window. In an implementation where the ML model is used to identify objects in an image, in response to an image from a camera of the user device, the ML model can output text on the screen of the user device, wherein the text has one or more variations of the name of the object identified by the ML model in the input image. In an implementation that recognizes user voice input, the ML model can convert the voice input data into text in a natural language that can be further analyzed (e.g., parsed). Thereafter, if the user voice input is recognized as a search query for an application accessing one or more search engines and / or an application accessing a geographic map, a displayed user location, a generated navigation route, etc., the ML model outputs the data in the form of a text message output on the screen, wherein the text message repeats the user voice input, the search result output from the search engine on the screen of the user device, and / or the search result output on the geographic map. In response to user input in the form of one or more fingers or a pen moving on the touch screen surface, the ML model can output one or more suggested variations of the recognized characters, words, or sentences based on the user input.

[0077] It should be noted that one or more ML models may be implemented by software means (such as a computer program implemented in one or more programming languages or in the form of executable code and / or one or more computer program elements, computer program modules, computer program products, etc.). In addition, according to the present invention, one or more ML models may be implemented using different hardware devices (such as field programmable gate arrays (FPGAs), integrated circuits, etc.). Various specific examples of software and / or hardware devices suitable for implementing one or more ML models will be apparent to those skilled in the art depending on a given implementation of the claimed invention.

[0078] The communication between the server and the UE may be provided by one or more units known in the art that perform data transmission and reception, encoding and decoding, scrambling, encryption, conversion, etc. The communication between the UE and the server may be implemented by means of one or more communication networks based on any wireless communication technology known to those skilled in the art (such as GSM, 3GPP, LTE, LTE-A, CDMA, ZigBee, Wi-Fi, machine type communication (MTC), NFC, etc.) or based on any wired communication technology known to those skilled in the art. The apparatus for transmitting and receiving data between the server and the UE does not limit the scope of the present invention, and depending on a given implementation of the present invention, a combination of one or more apparatuses for transmitting and receiving data may be conceived by those skilled in the art.

[0079] In one or more embodiments of the present invention, an ML model evaluation module may also be provided. In particular, such a module may reside in a server. Based on the evaluation of the ML models received by the server from various user devices, the ML models from various user devices may be assigned different weights. Preferably, the quality of one or more ML models collected from the user devices is evaluated within each given personalized group to which the one or more ML models are collected. Based on the evaluation, weights may be assigned to the ML models, and based on the weights, the personalized ML models may be further updated in the server by averaging the ML models received from one or more UEs taking into account the assigned weights. In embodiments of the present invention, depending on the specific implementation of the claimed invention, averaging may not use all ML models collected from the user devices (e.g., within a given personalized group), but only use models whose weights are above a certain predetermined threshold, within a certain range defined by an upper threshold and a lower threshold, or closest to a certain predetermined value.

[0080] The operation of the present invention has been experimentally tested for the specific case of distributed subsequent training of a model for predicting the next word in an on-screen keyboard of a mobile phone. Text from the Wikipedia website was used as model data for training an initial model in the experiment. The initial model was trained in a virtual server (hereinafter referred to as VS). Messages from the Twitter dataset were used as model user data. Twitter text was randomly distributed between virtual nodes (hereinafter referred to as VN) representing mobile devices. The initial model was then sent to the VN along with a portion of the initial data from Wikipedia. The data portions from Twitter and Wikipedia were used in the VN in a 1:1 ratio (10 kilobytes each). A recursive neural network training algorithm was run on the resulting 20K bytes of text until convergence was achieved, after which the models trained on each VN were sent to the VS, where they were averaged. The model was updated in the VS and the process was repeated, wherein the Twitter data portion was updated in each VN to simulate a new set of messages typed by the user.

[0081] Tests showed that after 300 iterations of the algorithm, the quality of next-word predictions on Twitter text, as measured by the average number of keystrokes, improved by 8.5 percentage points. Meanwhile, the quality of predictions on Wikipedia text remained almost the same, indicating that “forgetfulness” was prevented.

[0082] Furthermore, the privacy level guaranteed by differential privacy measurement is experimentally tested. The experimental evaluation of the privacy level indicates that the probability of user data disclosure is low and at least no worse than that in the case of other similar distributed training methods.

[0083] The operation of the present invention will now be explained in the following illustrative examples which are provided by way of example only and not limitation.

[0084] The operational sequence of the method for distributed artificial intelligence (AI) machine learning (ML) model training according to the first aspect of the above-mentioned aspects of the present invention will now be discussed.

[0085] According to the method of the present invention, in step S1, one or more machine learning (ML) models are initialized in a server. Initialization may include training the one or more ML models based on an initial training dataset that is publicly available data.

[0086] Then, in step S2, the one or more initialized ML models are distributed among one or more user equipments (UEs) connected to the server via a communication network. Distribution can be achieved by transmitting data of the one or more ML models from the server to the one or more UEs using any method known in the art of wireless communication. Alternatively, the ML models can be distributed on a portable computer-readable medium, etc., by other means (particularly, via a wired network).

[0087] In step S3, user-generated data input by the user is collected in each of the one or more UEs. The data is generated by the user in the process of using one or more software applications installed in the UE and in the process of sending messages and making calls via one or more communication networks. As an example, the ML model to be trained can be configured to predict words and phrases when the user enters a text message in the UE. The user-generated data collected in step S3 can be, for example, words and phrases entered by the user when typing a text message, post, comment, etc. As another example, the ML model can be configured to recognize objects in images captured by one or more cameras of the UE. In this case, the user-generated data is images generated by the user using one or more cameras or video cameras provided in the UE, and labels assigned by the user to objects present in the images. In addition to images from one or more cameras of the UE, object recognition can also be performed by the ML model in images captured by the UE from other sources (for example, from other users via a communication network or by browsing a website).

[0088] In another example, the ML model may be configured to recognize handwriting input received from a user via a touch screen of the UE and / or a touch pad of the UE. In this case, the user-generated data may be handwriting input performed by the user on the touch screen and / or touch pad, for example, with the aid of one or more fingers or a pen, and user selection of variations of characters and / or words suggested by the ML model based on the handwriting input from the user, wherein the UE displays the suggested variations of characters and / or words on the screen when executing a corresponding software application.

[0089] In another example, the ML model may be configured to recognize voice input received from a user by means of one or more microphones provided in the UE, wherein the user-generated data is the voice input and / or a user selection of variations of words and / or phrases suggested by the ML model based on the voice input from the user, wherein the UE displays the suggested variations of words and / or phrases on a screen when executing a corresponding software application.

[0090] In yet another example, the ML model can be configured to identify one or more characteristics of the UE's environment and / or one or more user actions. Characteristics of the UE's environment may include, but are not limited to, time, date, weekday, illumination level, air temperature, air humidity level, UE's geographic location, or UE's spatial position. The user-generated data is user input to one or more software applications in the UE. In this example, the ML model can suggest different actions to the user, for example, for controlling different software applications in the UE, and / or automatically initiate specific actions in specific software applications.

[0091] The data generated by the user is collected in the UE during a predetermined data collection period. When the user data collected in the UE reaches a predetermined amount, the UE may send a message to the server that the necessary amount of data has been collected.

[0092] In step S4, the server sends training data to the UE. This training data is part of the initial dataset used in the initial training of the ML model in step S1. This data is publicly available and does not represent any specific user. Including the initial dataset in ML model training ensures the security of user personal data and prevents overfitting of the ML model in the UE.

[0093] Then, in step S5, the ML model is trained in each of the one or more UEs based on the collected data and the training data until a training stop criterion is met. As non-limiting examples, the training stop criterion may be when ML model convergence is achieved in the one or more UEs, or when a predetermined ML model quality characteristic value is achieved by the ML model, or when a predetermined number of ML model training periods are achieved.

[0094] In step S6, the server obtains the trained ML models from the one or more UEs. This operation includes sending the ML models trained in each UE to the server via a wireless communication network. The server collects the ML models trained in different UEs.

[0095] In step S7, the server updates the ML model by averaging the trained ML models obtained from one or more UEs. As a non-limiting example, the ML model update may involve aggregating the personalized ML models obtained from one or more UEs at the server. As a result of the aggregation, a new version of the ML model based on the personalized ML models trained in one or more UEs and collected at the server is provided.

[0096] In step S8, the server transmits the new version of the ML model provided by averaging to one or more UEs. As a non-limiting example, the transmission is performed by a well-known wireless communication network device.

[0097] Steps S3-S8 may be repeated one or more times (e.g., until an ML model that satisfies one or more ML model quality criteria is obtained). This generates a personalized ML model using "subsequent training" based on user-generated data from different UEs and the initial dataset used in the initial training of the ML model at the server.

[0098] In at least one embodiment of the present invention, the method may further include the following steps: identifying one or more personalized groups for users of each of the one or more UEs based on user-generated data collected in each of the one or more UEs. Furthermore, according to at least one embodiment, the method includes: grouping, at a server, ML models acquired from the one or more UEs into personalized groups; and sending updated ML models grouped into personalized groups only to UEs within a given personalized group. Thus, additional personalization of the trained ML model is achieved, and the accuracy of the ML model for different user groups is improved.

[0099] Figure 4 4 is a block diagram illustrating an electronic device (eg, one or more UEs) 401 in a network environment 400 according to various embodiments. Figure 4 , an electronic device 401 in a network environment 400 may communicate with an electronic device 402 via a first network 498 (e.g., a short-range wireless communication network), or may communicate with an electronic device 404 or a server 408 via a second network 499 (e.g., a long-range wireless communication network). Depending on an embodiment, the electronic device 401 may communicate with the electronic device 404 via the server 408. Depending on an embodiment, the electronic device 401 may include a processor 420, a memory 430, an input device 450, an audio output device 455, a display device 460, an audio module 470, a sensor module 476, an interface 477, a haptic module 479, a camera module 480, a power management module 488, a battery 489, a communication module 490, a subscriber identification module (SIM) 496, or an antenna module 497. In some embodiments, at least one of the above components (e.g., the display device 460 or the camera module 480) may be omitted from the electronic device 401, or one or more other components may be added to the electronic device 401. In some embodiments, some of the above components may be implemented as a single integrated circuit. For example, the sensor module 476 (eg, a fingerprint sensor, an iris sensor, or an illumination sensor) may be implemented as embedded in the display device 460 (eg, a display).

[0100] The processor 420 may run, for example, software (e.g., program 440) to control at least one other component of the electronic device 401 connected to the processor 420 (e.g., a hardware component or a software component), and may perform various data processing or calculations. According to one embodiment, as at least part of the data processing or calculation, the processor 420 may load commands or data received from another component (e.g., sensor module 476 or communication module 490) into the volatile memory 432, process the commands or data stored in the volatile memory 432, and store the resulting data in the non-volatile memory 434. According to an embodiment, the processor 420 may include a main processor 421 (e.g., a central processing unit (CPU) or an application processor (AP)) and an auxiliary processor 423 (e.g., a graphics processing unit (GPU), an image signal processor (ISP), a sensor hub processor, or a communication processor (CP)) that is operationally independent of or combined with the main processor 421. Additionally or alternatively, the auxiliary processor 423 may be adapted to consume less power than the main processor 421, or adapted to be specifically used for a designated function. The auxiliary processor 423 may be implemented separately from the main processor 421 or as part of the main processor 421 .

[0101] When the main processor 421 is in an inactive (e.g., sleep) state, the auxiliary processor 423 (rather than the main processor 421) may control at least some of the functions or states related to at least one component (e.g., the display device 460, the sensor module 476, or the communication module 490) among the components of the electronic device 401, or when the main processor 421 is in an active state (e.g., running an application), the auxiliary processor 423 may control at least some of the functions or states related to at least one component (e.g., the display device 460, the sensor module 476, or the communication module 490) together with the main processor 421. Depending on the embodiment, the auxiliary processor 423 (e.g., an image signal processor or a communication processor) may be implemented as part of another component (e.g., the camera module 480 or the communication module 490) that is functionally related to the auxiliary processor 423.

[0102] The memory 430 may store various data used by at least one component of the electronic device 401 (e.g., the processor 420 or the sensor module 476). The various data may include, for example, software (e.g., the program 440) and input data or output data for commands related thereto. The memory 430 may include a volatile memory 432 or a non-volatile memory 434.

[0103] The program 440 may be stored as software in the memory 430 , and may include, for example, an operating system (OS) 442 , middleware 444 , or applications 446 .

[0104] The input device 450 may receive commands or data from outside the electronic device 401 (e.g., a user) to be used by other components of the electronic device 401 (e.g., the processor 420). The input device 450 may include, for example, a microphone, a mouse, a keyboard, or a digital pen (e.g., a stylus).

[0105] The sound output device 455 can output sound signals to the outside of the electronic device 401. The sound output device 455 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as playing multimedia or records, while the receiver can be used for incoming calls. Depending on the embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0106] The display device 460 can visually provide information to the outside of the electronic device 401 (e.g., a user). The display device 460 may include, for example, a display, a holographic device, or a projector, and a control circuit for controlling a corresponding one of the display, the holographic device, and the projector. Depending on the embodiment, the display device 460 may include a touch circuit adapted to detect a touch or a sensor circuit adapted to measure the strength of the force caused by the touch (e.g., a pressure sensor).

[0107] The audio module 470 can convert sound into an electrical signal, and vice versa. Depending on the embodiment, the audio module 470 can obtain sound via the input device 450, or output sound via the sound output device 455 or an earphone of an external electronic device (e.g., electronic device 402) directly (e.g., wired) or wirelessly connected to the electronic device 401.

[0108] The sensor module 476 can detect the operating state (e.g., power or temperature) of the electronic device 401 or the environmental state (e.g., the state of the user) outside the electronic device 401, and then generate an electrical signal or data value corresponding to the detected state. Depending on the embodiment, the sensor module 476 may include, for example, a gesture sensor, a gyroscope sensor, an atmospheric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illumination sensor.

[0109] The interface 477 may support one or more specific protocols to be used to connect the electronic device 401 directly (e.g., wired) or wirelessly to an external electronic device (e.g., the electronic device 402). Depending on the embodiment, the interface 477 may include, for example, a High-Definition Multimedia Interface (HDMI), a Universal Serial Bus (USB) interface, a Secure Digital (SD) card interface, or an audio interface.

[0110] The connection end 478 may include a connector, wherein the electronic device 401 can be physically connected to an external electronic device (e.g., the electronic device 402) via the connector. Depending on the embodiment, the connection end 478 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0111] The haptic module 479 may convert the electrical signal into mechanical stimulation (eg, vibration or motion) or electrical stimulation that can be recognized by the user via his sense of touch or kinesthetic sense. According to an embodiment, the haptic module 479 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0112] The camera module 480 may capture still images or moving images. Depending on the embodiment, the camera module 480 may include one or more lenses, image sensors, image signal processors, or flashes.

[0113] The power management module 488 may manage power supply to the electronic device 401. According to an embodiment, the power management module 488 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0114] The battery 489 may power at least one component of the electronic device 401. According to an embodiment, the battery 489 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0115] The communication module 490 may support establishing a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device 401 and an external electronic device (e.g., electronic device 402, electronic device 404, or server 408), and perform communication via the established communication channel. The communication module 490 may include one or more communication processors capable of operating independently from the processor 420 (e.g., an application processor (AP)) and support direct (e.g., wired) communication or wireless communication. Depending on the embodiment, the communication module 490 may include a wireless communication module 492 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 494 (e.g., a local area network (LAN) communication module or a power line communication (PLC) module). A corresponding one of these communication modules can communicate with an external electronic device via a first network 498 (e.g., a short-range communication network such as Bluetooth, Wireless Fidelity (Wi-Fi) Direct, or Infrared Data Association (IrDA)) or a second network 499 (e.g., a long-range communication network such as a cellular network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules can be implemented as a single component (e.g., a single chip), or these various types of communication modules can be implemented as multiple components separated from each other (e.g., multiple chips). The wireless communication module 492 can identify and authenticate the electronic device 401 in a communication network (such as the first network 498 or the second network 499) using user information (e.g., an International Mobile Subscriber Identity (IMSI)) stored in the user identification module 496.

[0116] Antenna module 497 can transmit or receive signals or power to or from the outside of electronic device 401 (e.g., an external electronic device). Depending on the embodiment, antenna module 497 may include an antenna comprising a radiating element formed of a conductive material or conductive pattern formed in or on a substrate (e.g., a PCB). Depending on the embodiment, antenna module 497 may include multiple antennas. In this case, at least one antenna suitable for the communication scheme used in a communication network (such as first network 498 or second network 499) may be selected from the multiple antennas by, for example, communication module 490 (e.g., wireless communication module 492). Signals or power can then be transmitted or received between communication module 490 and the external electronic device via the selected at least one antenna. Depending on the embodiment, additional components other than the radiating element (e.g., a radio frequency integrated circuit (RFIC)) may also be formed as part of antenna module 497.

[0117] At least some of the above components can be connected to each other via an inter-peripheral communication scheme (e.g., a bus, general-purpose input output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and communicatively transmit signals (e.g., commands or data) therebetween.

[0118] According to an embodiment, commands or data may be transmitted or received between electronic device 401 and external electronic device 404 via server 408 connected to second network 499. Each of electronic device 402 and electronic device 404 may be a device of the same type as electronic device 401, or a device of a different type than electronic device 401. According to an embodiment, all or some operations to be executed on electronic device 401 may be executed on one or more of external electronic device 402, external electronic device 404, or server 408. For example, if electronic device 401 should automatically execute a function or service or should execute a function or service in response to a request from a user or another device, electronic device 401 may request one or more external electronic devices to execute at least part of the function or service instead of executing the function or service, or in addition to executing the function or service, electronic device 401 may request one or more external electronic devices to execute at least part of the function or service. The one or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or execute another function or service related to the request, and transmit the result of the execution to the electronic device 401. The electronic device 401 may provide the result as at least a partial response to the request, either by further processing the result or without further processing the result. To this end, for example, cloud computing technology, distributed computing technology, or client-server computing technology may be used.

[0119] The electronic device according to various embodiments may be one of various types of electronic devices. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a household appliance. According to an embodiment of the present disclosure, the electronic device is not limited to those described above.

[0120] It should be understood that the various embodiments of the present disclosure and the terms used therein are not intended to limit the technical features set forth herein to specific embodiments, but rather include various changes, equivalents or alternative forms for corresponding embodiments. For the description of the accompanying drawings, similar reference numerals may be used to refer to similar or related elements. It will be understood that the nouns in the singular form corresponding to the term may include one or more things, unless the relevant context clearly indicates otherwise. As used herein, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C" and "at least one of A, B or C" may include any one or all possible combinations of the items listed together with the corresponding phrase in the multiple phrases. As used herein, terms such as "1st" and "2nd" or "first" and "second" may be used to simply distinguish corresponding components from another component, and do not limit the components in other aspects (e.g., importance or order). It will be understood that if an element (e.g., a first element) is referred to as being “combined with another element (e.g., a second element)”, “combined to another element (e.g., a second element)”, “connected with another element (e.g., a second element)”, or “connected to another element (e.g., a second element)”, when the term “operably” or “communicatively” is used or when the term “operably” or “communicatively” is not used, it means that the element can be directly (e.g., wired) connected to the other element, wirelessly connected to the other element, or connected to the other element via a third element.

[0121] As used herein, the term "module" may include units implemented in hardware, software, or firmware, and may be used interchangeably with other terms (e.g., "logic," "logic block," "portion," or "circuit"). A module may be a single integrated component adapted to perform one or more functions or the smallest unit or portion of the single integrated component. For example, depending on an embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0122] The various embodiments described herein can be implemented as software (e.g., program 440) comprising one or more instructions stored in a storage medium (e.g., internal memory 436 or external memory 438) that can be read by a machine (e.g., electronic device 401). For example, under the control of a processor, a processor (e.g., processor 420) of the machine (e.g., electronic device 401) can call at least one of the one or more instructions stored in the storage medium and execute the at least one instruction with or without the use of one or more other components. This causes the machine to be operated to perform at least one function according to the called at least one instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The machine-readable storage medium can be provided in the form of a non-transitory storage medium. The term "non-transitory" only means that the storage medium is a tangible device and does not include signals (e.g., electromagnetic waves), but the term does not distinguish between data being semi-permanently stored in the storage medium and data being temporarily stored in the storage medium.

[0123] According to an embodiment, the method according to various embodiments of the present disclosure may be included and provided in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be released in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)), or may be downloaded via an application store (e.g., Play Store). TM ) The computer program product may be published online (e.g., downloaded or uploaded) or may be distributed (e.g., downloaded or uploaded) directly between two user devices (e.g., smartphones). If published online, at least part of the computer program product may be temporarily generated or at least part of the computer program product may be at least temporarily stored in a machine-readable storage medium (such as a memory of a manufacturer's server, an application store's server, or a forwarding server).

[0124] According to various embodiments, each component (for example, module or program) in the above-mentioned components may include a single entity or multiple entities. According to various embodiments, one or more components in the above-mentioned components may be omitted, or one or more other components may be added. Alternatively or additionally, multiple components (for example, module or program) may be integrated into a single component. In this case, according to various embodiments, the integrated component may still perform the one or more functions of each component in the multiple components in the same or similar manner as a corresponding component in the multiple components before integration. According to various embodiments, the operations performed by module, program or another component may be performed sequentially, in parallel, repeatedly or in a heuristic manner, or one or more operations in the operations may be run or omitted in different orders, or one or more other operations may be added.

Claims

1. A user equipment (UE), comprising: Communication circuits, memory to store instructions, and at least one processor, wherein the instructions, when executed by the at least one processor, cause the UE to perform the following operations: obtaining user-generated data based on user input of a user of the UE; receiving training data from a server connected to the UE via a communication circuit, wherein the training data comprises publicly available data for initializing a machine learning (ML) model in the server; training an ML model based on the user-generated data and the training data until a training stopping criterion is satisfied, wherein the training stopping criterion comprises achieving convergence of the ML model in one or more UEs including the UE; sending the trained ML model to a server via a communication circuit; identifying a personalized group for the user of the UE from among multiple personalized groups based on data generated by the user of the UE, wherein the multiple personalized groups are formed based on at least one of an ML model type, a type of the UE, a brand of the UE, and a model of the UE; and An updated ML model is received based on the one personalization group, wherein the updated ML model is sent only to UEs within the identified personalization group.

2. The UE according to claim 1, wherein: The ML model is configured to predict words and phrases for text input to the UE, wherein the user-generated data includes words and phrases input by the user.

3. The UE according to claim 1, wherein: The ML model is configured to: identify a first object in a first image acquired from one or more cameras of the UE, wherein the user-generated data includes a second image from the one or more cameras of the UE or a label assigned by a user of the UE to a second object present in the second image.

4. The UE according to claim 1, wherein: The ML model is configured to recognize a first handwriting input received from a user via a touch screen of the UE or a touchpad of the UE, wherein the user-generated data includes a second handwriting input of the user of the UE or a user selection of a variation of a character or word suggested by the ML model based on the second handwriting input from the user.

5. The UE according to claim 1, wherein: The ML model is configured to recognize a first voice input received by one or more microphones of the UE from a user of the UE, wherein the user-generated data comprises a second voice input or a user selection of a variation of a word or phrase suggested by the ML model based on the second voice input from the user.

6. The UE according to claim 1, wherein: The ML model is configured to: identify one or more characteristics of the environment of the UE or one or more user actions, wherein the one or more characteristics of the environment of the UE include one or more of the time, date, weekday, illumination, temperature, geographic location, and spatial location of the UE, and wherein the user-generated data is user input in one or more applications of the UE.

7. A method for performing distributed training of an artificial intelligence (AI) machine learning (ML) model by a user equipment (UE), the method comprising: obtaining user-generated data based on user input by a user of the UE; receiving training data from a server connected to the UE, wherein the training data comprises publicly available data for initializing an ML model in the server; training an ML model based on the user-generated data and the training data until a training stopping criterion is satisfied, wherein the training stopping criterion comprises achieving convergence of the ML model in one or more UEs including the UE; Send the trained ML model to the server; identifying a personalized group for the user of the UE from among multiple personalized groups based on data generated by the user of the UE, wherein the multiple personalized groups are formed based on at least one of an ML model type, a type of the UE, a brand of the UE, and a model of the UE; and An updated ML model is received based on the one personalization group, wherein the updated ML model is sent only to UEs within the identified personalization group.

8. The method of claim 7, wherein: The ML model is configured to predict words and phrases for text input to the UE, wherein the user-generated data includes words and phrases input by the user.

9. The method of claim 7, wherein: The ML model is configured to: identify a first object in a first image acquired from one or more cameras of the UE, wherein the user-generated data includes a second image from the one or more cameras of the UE or a label assigned by a user of the UE to a second object present in the second image.

10. The method of claim 7, wherein: The ML model is configured to recognize a first handwriting input received from a user via a touch screen of the UE or a touchpad of the UE, wherein the user-generated data includes a second handwriting input of the user of the UE or a user selection of a variation of a character or word suggested by the ML model based on the second handwriting input from the user.

11. The method according to claim 7, wherein: The ML model is configured to recognize a first voice input received by one or more microphones of the UE from a user of the UE, wherein the user-generated data comprises a second voice input or a user selection of a variation of a word or phrase suggested by the ML model based on the second voice input from the user.

12. The method of claim 7, wherein: The ML model is configured to: identify one or more characteristics of the environment of the UE or one or more user actions, wherein the one or more characteristics of the environment of the UE include one or more of the time, date, weekday, illumination, temperature, geographic location, and spatial location of the UE, and wherein the user-generated data is user input in one or more applications of the UE.

13. A computer-readable storage medium, wherein: The computer-readable storage medium stores instructions, wherein the instructions, when executed by one or more processors of a user equipment (UE), cause the UE to perform operations according to the method of any one of claims 1 to 6.

Images