Electronic device, server and operation method thereof for updating artificial intelligence model

Abstract

The present disclosure relates to providing an artificial intelligence (AI) system and its application that uses a machine learning algorithm such as deep learning to simulate the functions of the human brain (such as cognition and judgment). A method for updating a learning model in an electronic device is provided, the method comprising: detecting that information about the context of the electronic device to be used to update a local model has changed; based on the changed context information, obtaining a gradient for updating the local model; based on the gradient, updating the local model; sending the gradient to a server; receiving information about a global model updated based on the gradient from the server; and updating the local model based on the received information.

Description

Technical Field

[0001] The present disclosure relates to an electronic device and a server for refining an artificial intelligence (AI) model stored in an electronic device, and a method for refining the AI ​​model. Background Art

[0002] An artificial intelligence (AI) system is provided that uses machine learning and performs various operations based on various types of context information indicating various contexts of a user or an electronic device owned by a user (e.g., information about a state of the electronic device or information about a surrounding environment of the electronic device).

[0003] Since the context information of the electronic device may continue to change over time, the AI ​​model can be continuously refined according to the changed context information. Therefore, the AI ​​model can be continuously refined based on various types of context information collected in real time, so that the AI ​​model can perform appropriate operations according to the context information.

[0004] In terms of throughput and data size for learning, the refinement of the AI ​​model may be performed in a server having better performance than the electronic device owned by the user. For example, the AI ​​model stored in the electronic device may also be additionally refined based on the result of the refinement of the AI ​​model by the server. Therefore, in order to refine the AI ​​model, the electronic device may send the context information of the electronic device.

[0005] However, since the context information of the electronic device may include personal information including sensitive information of the user, biometric information, secret information, and financial information, when the context information is leaked during the transmission process, it may cause serious damage to the user. Summary of the invention

[0006] Embodiments of the present disclosure provide an electronic device and a server for refining an artificial intelligence (AI) model stored in an electronic device, and a method for refining the AI ​​model.

[0007] An embodiment of the present disclosure also provides a non-transitory computer-readable recording medium having recorded thereon a program for executing the method on a computer.

[0008] Additional features will be set forth in part in the description which follows and, in part, will be apparent from the description.

[0009] According to an example embodiment of the present disclosure, a method for refining an artificial intelligence (AI) model performed by an electronic device includes: detecting that information about a context of the electronic device used to refine a local model stored in the electronic device is changed; determining a gradient for refining the local model based on the changed information about the context; refining the local model based on the determined gradient; sending the gradient to a server; receiving information about a global model refined based on the gradient from the server; and refining the local model based on the received information.

[0010] According to another example embodiment of the present disclosure, an electronic device configured to refine an artificial intelligence (AI) model includes: a memory storing a local model; at least one processor configured to: detect that information about a context of the electronic device used to refine the local model is changed, determine a gradient for refining the local model based on the changed information about the context, and refine the local model based on the determined gradient; and a communicator including a communication circuit configured to: send the gradient to a server and receive information about a global model refined based on the gradient from the server, wherein the at least one processor is also configured to refine the local model based on the information received from the server.

[0011] According to another exemplary embodiment of the present disclosure, a non-transitory computer-readable recording medium has recorded thereon a program for implementing the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent through the following detailed description in conjunction with the accompanying drawings, in which:

[0013] Figure 1 is a diagram illustrating an example system for performing various operations using a local model of an electronic device according to an embodiment of the present disclosure;

[0014] Figure 2 is a diagram illustrating an example system for performing various operations using a local model according to an embodiment of the present disclosure;

[0015] Figure 3 is a diagram illustrating an example system for performing object recognition using a local model according to an embodiment of the present disclosure;

[0016] Figure 4 is a diagram illustrating an example system for performing speech recognition using a local model according to an embodiment of the present disclosure;

[0017] Figure 5 is a diagram showing an example of refining a local model and a global model according to an embodiment of the present disclosure;

[0018] Figure 6 is a diagram showing an example of a local model or a global model according to an embodiment of the present disclosure;

[0019] Figure 7 is a diagram illustrating an example operation of an electronic device according to an embodiment of the present disclosure;

[0020] Figure 8 is a diagram showing an example of refining a local model according to time according to an embodiment of the present disclosure;

[0021] Fig. 9 is a block diagram illustrating an example configuration of an example electronic device according to an embodiment of the present disclosure;

[0022] Fig.10 is a block diagram illustrating an example configuration of an example electronic device according to an embodiment of the present disclosure;

[0023] Fig.11 is a block diagram illustrating an example configuration of an example server according to an embodiment of the present disclosure;

[0024] Fig.12 is a flowchart illustrating an example method of refining a local model according to an embodiment of the present disclosure;

[0025] Fig.13 is a flow chart illustrating an example method of refining a global model performed by a server according to an embodiment of the present disclosure;

[0026] Fig.14 is a diagram illustrating an example system for refining local and global models according to an embodiment of the present disclosure;

[0027] Fig.15 is a diagram illustrating an example system for refining local models and global models via a relay device according to an embodiment of the present disclosure;

[0028] Fig.16 is a signal flow diagram illustrating an example method of refining a global model and a local model according to a gradient according to an embodiment of the present disclosure;

[0029] Fig.17 is a signal flow diagram illustrating an example method of refining a global model, a first local model of an electronic device, and a second local model of an external device according to a gradient according to an embodiment of the present disclosure; and

[0030] Fig.18 is a signal flow diagram illustrating an example method of refining a global model, a first local model of an electronic device, and a second local model of an external device according to a gradient according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] Various embodiments will be described in more detail below with reference to the accompanying drawings. However, the present disclosure may be implemented in various different forms and is not limited to the exemplary embodiments of the present disclosure described herein. In the accompanying drawings, parts not related to the specification may be omitted to clearly describe the present disclosure, and the same reference numerals throughout the specification may indicate the same elements.

[0032] Throughout this disclosure, the expression "at least one of a, b, or c" refers to only a, only b, only c, both a and b, both a and c, both b and c, all of a, b, and c, or variations thereof.

[0033] Throughout the present disclosure, when a component is "connected" to another component, the component may not only be "directly connected" to the other component but may also be "electrically connected" to the other component through another element in between. In addition, when a component "includes" a certain component, the component may further include another component instead of excluding other components unless otherwise specified.

[0034] According to the present disclosure, functions related to artificial intelligence (AI) can be operated via a processor and a memory. The processor may include one or more processors. In this case, the one or more processors may include, for example but not limited to: a general-purpose processor such as a central processing unit (CPU), an application processor (AP) or a digital signal processor (DSP); a dedicated graphics processor such as a graphics processing unit (GPU) and a vision processing unit (VPU); a dedicated AI processor such as a neural processing unit (NPU); and the like. The one or more processors may control the input data to be processed according to predefined operating rules or an AI model stored in a memory. When the one or more processors include a dedicated AI processor, the dedicated AI processor may be designed to have a hardware structure dedicated to processing a specific AI model.

[0035] The predefined operating rules or AI models may be generated via a training process. For example, this may refer to predefined operating rules or AI models that are executed based on generating desired characteristics (or purposes) by training a basic AI model using a learning algorithm using a large amount of training data. The training process may be performed by a device or a separate server and / or system for executing AI. Examples of learning algorithms may include, for example, but not limited to, supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning, etc., but are not limited thereto.

[0036] The AI ​​model may include multiple neural network layers. Each neural network layer may include multiple nodes and weight values, and the neural network arithmetic operation may be performed via the arithmetic operation result of the previous layer and the arithmetic operation between multiple weight values. The multiple weight values ​​in each neural network layer may be optimized by the result of training the AI ​​model. For example, multiple weight values ​​may be refined to reduce or minimize the loss or cost value obtained by the AI ​​model during the training process. In order to minimize and / or reduce the loss or cost value, multiple weight values ​​may be refined to minimize and / or reduce the gradient associated with the loss or cost value. The artificial neural network may include, for example, but not limited to, a deep neural network (DNN), and may include, for example, but not limited to, a convolutional neural network (CNN), a DNN, a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent DNN (BRDNN), a deep Q network (DQN), etc., but not limited thereto.

[0037] Hereinafter, various exemplary embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings.

[0038] Figure 1 is a diagram illustrating an example system for performing various operations using a partial model of an electronic device 1000 according to an embodiment of the present disclosure.

[0039] like Figure 1 As shown, the system according to an embodiment of the present disclosure may include an electronic device 1000 and a server 2000. The electronic device 1000 according to an embodiment of the present disclosure may refine a local model stored in the electronic device 1000 so that a result suitable for the context of the electronic device 1000 is output based on information about the context of the electronic device 1000 collected by the electronic device 1000, and the electronic device 1000 may perform various operations based on the refined local model.

[0040] The local model of the electronic device 1000 according to an embodiment of the present disclosure may include, for example, an artificial intelligence (AI) model, which may be continuously refined in the electronic device 1000 based on information about the context of the electronic device 1000 and used for various operations.

[0041] The electronic device 1000 according to an embodiment of the present disclosure may refine the local model by optimizing a plurality of weight values ​​in a plurality of neural network layers included in the local model based on information about the context. For example, the electronic device 1000 may refine the local model based on information about the context so that the difference between the prediction information output by the local model and the observation information corresponding to the prediction information is minimized and / or relatively low.

[0042] The observation information according to an embodiment of the present disclosure may include, for example, information indicating an answer to the prediction information, and may be determined based not only on the information about the context of the electronic device 1000 but also on various types of information related to the output result of the local model.

[0043] The information about the context of the electronic device 1000 may include context information collected from the electronic device 1000 , information about a state of the electronic device 1000 , information about a user of the electronic device 1000 , and information about a surrounding environment of the electronic device 1000 .

[0044] According to an embodiment of the present disclosure, the context information collected from the electronic device 1000 may include various types of information for refining the local model so as to output a result suitable for the current context of the electronic device 1000. The context information may include, for example but not limited to, information about the state of the electronic device 1000, information about the user of the electronic device 1000, information about the surrounding environment of the electronic device 1000, and the like.

[0045] According to an embodiment of the present disclosure, prediction information and corresponding observation information that can be output to the local model can be obtained based on context information. According to an embodiment of the present disclosure, the electronic device 1000 can refine the local model by modifying the values ​​included in the local model so that the difference between the observation information and the prediction information is minimized and / or relatively low.

[0046] According to an embodiment of the present disclosure, information about the status of the electronic device 1000 may include various types of information about the status of the electronic device 1000, such as, for example but not limited to, information about the operating status of the electronic device 1000, information about the network connection status of the electronic device 1000, information about the status of an operating system (OS) or application installed in the electronic device 1000, information about the location of the electronic device 1000, information about the motion status of the electronic device 1000, information about the status of hardware included in the electronic device 1000, and the like.

[0047] The information about the user of the electronic device 1000 according to an embodiment of the present disclosure may include, for example but not limited to, information about the characteristics of the user, various types of privacy information of the user (such as information about personal details of the user), information about an account of a service subscribed by the user, information about the life pattern of the user, etc. However, the information about the user is not limited thereto, and may include various types of information about the user.

[0048] According to an embodiment of the present disclosure, information about the surrounding environment of the electronic device 1000 may include, for example, various types of information about the surrounding environment of the electronic device 1000, such as, for example but not limited to, information about objects existing around the electronic device 1000, information about the lighting of the surrounding environment of the electronic device 1000, and the like.

[0049] The operations that can be performed based on the local model according to an embodiment of the present disclosure may include various operations that can be performed by the electronic device 1000. For example, but not limited to, the operation may include an operation of recommending an item, an operation of identifying an object around the electronic device 1000, an operation of performing voice recognition on an audio signal collected by the electronic device 1000, etc. However, according to an embodiment of the present disclosure, the operations that can be performed based on the local model are not limited thereto, and may include various operations that can be performed by the electronic device 1000 using the local model.

[0050] Based on the information about the context of the electronic device 1000, the electronic device 1000 according to an embodiment of the present disclosure can not only perform arithmetic operations for performing various operations using a local model, but also perform training for improving the accuracy of arithmetic operations using a local model. The electronic device 1000 according to an embodiment of the present disclosure can train a local model and derive an arithmetic operation optimized to the usage mode of the electronic device 1000 using the trained local model. According to an embodiment of the present disclosure, there may be at least one local model corresponding to each operation, and each operation may be performed according to at least one local model. For example, the electronic device 1000 may obtain at least one local model corresponding to an operation to be performed by the electronic device 1000 (e.g., item recommendation, object recognition, or speech recognition) in a plurality of local models, and perform the operation based on the obtained local model. However, the embodiments of the present disclosure are not limited thereto, and a plurality of different operations may be performed according to a single local model.

[0051] The electronic device 1000 according to an embodiment of the present disclosure may obtain at least one local model from a memory included in the electronic device 1000, or receive at least one local model from an external source. For example, the electronic device 1000 may obtain the local model by reading the memory. The electronic device 1000 may receive a global model from an external server 2000, and obtain the received global model as a local model of the electronic device 1000. However, the embodiments of the present disclosure are not limited thereto, and the electronic device 1000 may obtain the local model via any one of a variety of methods.

[0052] For example, when a local model for performing an operation is not stored in the electronic device 1000, the electronic device 1000 may receive information about a global model for performing an operation from the server 2000. The electronic device 1000 may obtain and store a local model based on the information about the global model received from the server 2000.

[0053] According to an embodiment of the present disclosure, the electronic device 1000 may request a global model for performing a certain operation from the server 2000, receive the global model from the server 2000, and store the received global model as a local model of the electronic device 1000. The local model stored in the electronic device 1000 may be continuously refined based on context information collected by the electronic device 1000. However, the embodiments of the present disclosure are not limited thereto, and the local model stored in the electronic device 1000 may be continuously refined based not only on the context collected by the electronic device 1000, but also on various types of data required to refine the local model.

[0054] The electronic device 1000 according to an embodiment of the present disclosure may include, for example, a device for performing various operations using a local model according to AI technology, and may be implemented in various forms. For example, the electronic device 1000 described herein may include, for example but not limited to, a digital camera, a smart phone, a laptop computer, a tablet personal computer (PC), an e-book terminal, a digital broadcast terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, an MP3 player, an AI speaker, a robot cleaner, a household appliance, etc., but not limited thereto.

[0055] The electronic device 1000 described herein may include a wearable device for a user. Examples of wearable devices include, for example but not limited to, accessory type devices (e.g., watches, rings, wristbands, ankle bands, necklaces, glasses, or contact lenses), head mounted devices (HMDs), textile or clothing integrated devices (e.g., electronic clothing), body-attachable devices (e.g., skin pads), bio-implantable devices (e.g., implantable circuits), etc., but are not limited thereto.

[0056] The refinement of the local model according to an embodiment of the present disclosure may be performed via multiple operations. For example, the refinement of the local model may include a first operation of refining the local model based on information about the context collected by the electronic device 1000. The refinement of the local model may include a second operation of refining the local model based on information received from the server 2000 when the gradient of the refined local model is sent to the server 2000.

[0057] According to an embodiment of the present disclosure, the gradient of the local model may include information for refining the local model. For example, the gradient may include information related to a situation where the difference between the observed information and the predicted information is minimal and / or relatively low.

[0058] For example, the electronic device 1000 may obtain a loss function indicating the difference between the observed information and the predicted information, and obtain a gradient indicating information related to a point where the value of the loss function is the smallest. For example, the minimum value of the loss function may be detected by finding a point where the slope of the loss function is the smallest. The slope value of the loss function according to an embodiment of the present disclosure may be obtained, for example, from a partial differential value of each variable included in the predicted information.

[0059] The electronic device 1000 according to an embodiment of the present disclosure can use the gradient to determine the prediction information of the loss function corresponding to the point where the loss function value is the smallest. The electronic device 1000 can refine the local model by modifying the parameters included in the local model (e.g., the weight value or bias of each node) so that the prediction information determined by the local model is output.

[0060] The server 2000 according to an embodiment of the present disclosure may refine the global model stored in the server 2000 based on the gradient sent from the electronic device 1000, and send information about the refined global model to the electronic device 1000. For example, the server 2000 may refine the global model so that prediction information corresponding to the slope information of the loss function included in the gradient is output by the global model. In this example, the observation information may include information indicating an answer to the prediction information of the global model and obtained based on various types of information related to the output result of the global model. Therefore, the electronic device 1000 according to an embodiment of the present disclosure may refine the local model again in the second operation based on the information about the refined global model received from the server 2000.

[0061] The global model according to an embodiment of the present disclosure can be refined based on various types of information collected by the server 2000 according to the relatively excellent learning ability and arithmetic operation ability of the server 2000. Before performing the operation of operating the local model, the global model according to an embodiment of the present disclosure can be continuously trained and stored based on various types of information collected by the server 2000 via various methods. However, the embodiments of the present disclosure are not limited thereto, and the global model can be trained and pre-stored in the server 2000 via various methods, and the global model stored in the server 2000 can be used to refine the local model according to an embodiment of the present disclosure.

[0062] The various types of information collected by the server 2000 may include, for example but not limited to, information collected by a plurality of external devices other than the electronic device 1000, information collected on the Internet, etc. However, the embodiments of the present disclosure are not limited thereto, and the various types of information collected by the server 2000 may include various types of information collected via various methods.

[0063] Therefore, the global model according to an embodiment of the present disclosure may show better performance than the local model when performing the operation according to an embodiment of the present disclosure. The electronic device 1000 according to an embodiment of the present disclosure can refine the local model again based on the global model refined by the server 2000 with relatively excellent learning ability and more information. For example, the server 2000 can not only use the data received from the electronic device 1000, but also use the data received from multiple other external devices to refine the global model. In addition, when the information about the context of the server 2000 is changed, the global model of the server 2000 according to an embodiment of the present disclosure can be refined. Therefore, according to an embodiment of the present disclosure, even if no data is received from an external source (for example, the gradient of the local model refined by the electronic device 1000 or another external device), the global model of the server 2000 can be refined based on the information about the context of the server 2000. Therefore, the electronic device 1000 according to an embodiment of the present disclosure can use the local model refined by the information about the global model to provide better services to users.

[0064] Considering that the global model refined by the server 2000 may include, for example, an AI model refined in consideration of not only the context information of the electronic device 1000 but also the context information of other external devices, the local model refined again based on the information about the refined global model according to the embodiment of the present disclosure may be refined. For example, the local model may be refined so that the result output by the local model does not conflict with the context of the electronic device 1000. Therefore, even if the local model according to the embodiment of the present disclosure is refined again based on the information about the global model, the local model may be refined so that appropriate operations are performed according to the separate context information of the electronic device 1000 that is different from the context of other external devices.

[0065] The electronic device 1000 according to an embodiment of the present disclosure may send a gradient instead of context information to the server 2000 to prevent and / or reduce the leakage of sensitive information of the user included in the context information. When the amount of data sent to the server 2000 is reduced, the time consumed to refine the local model may be reduced.

[0066] According to an embodiment of the present disclosure, the local model may be refined based on current context information collected by the electronic device 1000 , so that the local model outputs a result suitable for the current context of the electronic device 1000 .

[0067] According to an embodiment of the present disclosure, the context information collected from the electronic device 1000 may include various types of information for refining the local model so as to output a result suitable for the current context of the electronic device 1000. For example, but not limited to, the context information may include information about the state of the electronic device 1000, information about the user of the electronic device 1000, information about the surrounding environment of the electronic device 1000, feedback information about the user on the result output by the local model, and the like.

[0068] According to an embodiment of the present disclosure, a user may provide feedback information about an operation performed by a local model. The feedback information may include, for example but not limited to, information input by a user indicating whether a recommendation item, an object recognition result, a speech recognition result, etc. based on a local model is suitable. As another example, the feedback information may also include but is not limited to rating information about the recommended item, rating information input by a user, etc. However, the embodiments of the present disclosure are not limited thereto, and the feedback information may include various types of information, including the user's opinion on the operation performed by the local model.

[0069] Feedback information according to an embodiment of the present disclosure may be collected as context information collected by the electronic device 1000, which may be used to refine the local model, and the local model may be refined based on the context information of the electronic device 1000 including the feedback information. For example, when feedback information is obtained from a user, it may be determined that the context information is changed, and the local model may be refined according to the feedback information.

[0070] However, embodiments of the present disclosure are not limited thereto, and the context information collected by the electronic device 1000 may include various types of context information for refining the local model.

[0071] The context information of the electronic device 1000 according to an embodiment of the present disclosure may be continuously collected by the electronic device 1000. According to an embodiment of the present disclosure, the electronic device 1000 may continuously recognize the context of the electronic device 1000 and detect whether the context information of the electronic device 1000 is changed.

[0072] According to an embodiment of the present disclosure, when it is detected that the information about the context of the electronic device 1000 is changed, the electronic device 1000 can refine the local model based on the changed information. For example, when a specific type of information among the information about the context of the electronic device 1000 collected by the electronic device 1000 changes by at least one reference value, or when a new type of information about the context of the electronic device 1000 is obtained as context information, the electronic device 1000 can detect that the information about the context is changed.

[0073] For example, but not limited to, the context information of the electronic device 1000 may be changed when a new application is installed in the electronic device 1000, when a new user using the electronic device 1000 is recognized, when a new operation mode of the electronic device 1000 (not recognized before) is recognized, when a new form of an object is recognized around the electronic device 1000, when the surrounding environment of the electronic device 1000 changes, etc. However, the embodiments are not limited thereto, and when various types of information about the user or the electronic device 1000 that can be used to refine the local model are changed, the change in information about the context of the electronic device 1000 may be detected.

[0074] When the user's feedback information regarding the operation provided based on the local model according to an embodiment of the present disclosure changes, the context information of the electronic device 1000 may be changed.

[0075] According to an embodiment of the present disclosure, when a change in information about the context of the electronic device 1000 is detected, the electronic device 1000 may determine whether to perform an operation of refining the local model. For example, the local model may be refined based on at least one of, for example but not limited to, whether the electronic device 1000 is currently in an idle state, whether the memory space of the electronic device 1000 for refining the local model is sufficient, whether the battery of the electronic device 1000 is being charged, whether the current time is midnight, etc. However, the embodiments of the present disclosure are not limited thereto, and the electronic device 1000 may refine the local model based on the changed context information according to various preset conditions.

[0076] According to an embodiment of the present disclosure, when it is determined that the state of the electronic device 1000 is not suitable for the refined local model, the electronic device 1000 may perform an operation of refining the local model when it is later determined that the state of the electronic device 1000 is suitable for the refined local model.

[0077] According to an embodiment of the present disclosure, when refining a local model based on changed context information, the electronic device 1000 may transmit a gradient for refining the local model to the server 2000 .

[0078] The gradient according to an embodiment of the present disclosure may include, for example but not limited to: at least one value that may be generated based on the changed context information during the process of deriving the weight value; when the local model is refined based on the context information, deviation information of each node included in the local model; etc. For example, based on the information about the changed context, the gradient may include information about the point at which the loss function indicating the difference between the observed information and the predicted information of the local model is minimized.

[0079] When receiving the gradient from the electronic device 1000 via the network 100, the server 2000 according to an embodiment of the present disclosure may refine the global model included in the server 2000 according to the gradient. The global model according to an embodiment of the present disclosure may correspond to a local model of the electronic device 1000, but unlike the local model, the global model may be refined by a gradient received from an external device other than the electronic device 1000 included in the server 2000.

[0080] The server 2000 according to an embodiment of the present disclosure may transmit the gradient of the refined global model to the electronic device 1000 , and the electronic device 1000 may refine the local model again based on the gradient received from the server 2000 .

[0081] According to an embodiment of the present disclosure, a global model may include, for example, an AI model corresponding to a local model of the electronic device 1000, but the local model and the global model may have different data among various types of data (such as, for example but not limited to weight values, bias values, etc.), where each model is configured with the data when the operation of refining each model is performed by devices with different performance and specifications.

[0082] According to an embodiment of the present disclosure, the global model may be refined based on the global model received from the electronic device 1000. Considering that the gradient is obtained based on new context information collected by the electronic device 1000, the server 2000 according to an embodiment of the present disclosure may refine the global model so that the new context information is applied to the global model via the gradient.

[0083] The gradient of the refined global model may be sent to the electronic device 1000 to be used for refining the local model of the electronic device 1000. Fig.13 , 16 and 17 describe in more detail a method for refining a global model based on the gradient of the refined global model and a method for re-refining a local model.

[0084] According to an embodiment of the present disclosure, after re-refining the local model based on information about the refined global model, the electronic device 1000 may perform a certain operation (eg, item recommendation, object recognition, or speech recognition) according to the final refined local model.

[0085] According to an embodiment of the present disclosure, the electronic device 1000 may perform operations according to a local model based on on-device AI technology. For example, according to an embodiment of the present disclosure, after refining the local model based on information received from the server 2000, the electronic device 1000 may perform operations according to the refined local model based on the on-device AI technology. For example, the electronic device 1000 may continuously perform the refinement of the local model according to an embodiment of the present disclosure by sending data to the server 2000 or receiving data from the server 2000, and when the refinement of the local model according to an embodiment of the present disclosure is not performed, operations may be performed according to the on-device AI technology.

[0086] According to the on-device AI technology, the electronic device 1000 can perform an operation (e.g., item recommendation, object recognition, or voice recognition) based on the local model and output its result without sending data to or receiving data from the server 2000. The electronic device 1000 according to an embodiment of the present disclosure can perform operations using a local model based on various types of information collected by the electronic device 1000 in real time without using big data stored in the server 2000.

[0087] According to the on-device AI technology, the electronic device 1000 can make decisions on its own based on the local model trained according to the embodiment of the present disclosure and various types of data collected by itself. The electronic device 1000 through the on-device AI technology does not send the collected data to an external device, but operates autonomously, and is therefore advantageous in terms of user's private information protection and data processing speed.

[0088] For example, when the environment of network 100 is unstable, based on the local model of electronic device 1000, using information collected by electronic device 1000 instead of using big data received via network 100, electronic device 1000 can use on-device AI technology without having to connect to server 2000 to operate.

[0089] However, the electronic device 1000 is not limited to operating according to the on-device AI technology, and can perform operations according to the embodiments of the present disclosure by sending data to the server 2000 or an external device and receiving data from the server 2000 or an external device. For example, the electronic device 1000 can request the server 2000 to perform an operation according to the embodiments of the present disclosure instead of performing the operation using a local model. The server 2000 according to the embodiments of the present disclosure can perform the operation according to the embodiments of the present disclosure according to the global model based on the received request, and send the result thereof to the electronic device 1000. The electronic device 1000 can perform operations according to the embodiments of the present disclosure by combining the on-device AI technology and the method of sending data to the server 2000 and receiving data from the server 2000.

[0090] For example, when operation through server 2000 may be more advantageous in terms of data processing speed according to the network environment and the arithmetic operation capability of electronic device 1000, or when the method through server 2000 may be more advantageous than on-device AI technology (for example, when data that does not include the user's personal information is sent to server 2000), operations according to an embodiment of the present disclosure may be performed according to the method through server 2000.

[0091] Figure 2 is a diagram illustrating an example system for performing various operations using a local model according to an embodiment of the present disclosure.

[0092] like Figure 2 As shown, the system according to an embodiment of the present disclosure may include an electronic device 1000, a server 2000, a first external device 3000-1, and a second external device 3000-2. Figure 2 The electronic device 1000 and the server 2000 correspond to Figure 1 electronic device 1000 and server 2000.

[0093] Similar to the electronic device 1000, the first external device 3000-1 and the second external device 3000-2 according to an embodiment of the present disclosure can use the first local model and the second local model stored in the first external device 3000-1 and the second external device 3000-2 to perform various operations. The first local model and the second local model stored in the first external device 3000-1 and the second external device 3000-2 can be refined based on different context information collected by the first external device 3000-1 and the second external device 3000-2, respectively. Therefore, the first local model and the second local model can be AI models including different data, wherein the different data are suitable for the context of the first external device 3000-1 and the second external device 3000-2.

[0094] According to an embodiment of the present disclosure, users of the first external device 3000-1, the second external device 3000-2, and the electronic device 1000 may be different from each other. However, embodiments of the present disclosure are not limited thereto, and the first external device 3000-1, the second external device 3000-2, and the electronic device 1000 may be used by the same user.

[0095] According to an embodiment of the present disclosure, the server 2000 may receive at least one gradient obtained based on at least one of the first local model or the second local model refined by the corresponding device from at least one of the first external device 3000-1 or the second external device 3000-2. According to an embodiment of the present disclosure, when the context information collected by the first external device 3000-1 and the second external device 3000-2 changes, the first local model and the second local model refined by the corresponding device may be refined. According to an embodiment of the present disclosure, as an example of refining a global model based on a gradient received from the electronic device 1000, the server 2000 may refine the global model of the server 2000 based on at least one gradient received from the first external device 3000-1 and the second external device 3000-2.

[0096] According to an embodiment of the present disclosure, after refining the global model according to the gradient received from at least one of the first external device 3000-1 or the second external device 3000-2, the server 2000 may receive the gradient obtained based on the refined local model from the electronic device 1000. The server 2000 may refine the global model refined according to the gradient received from at least one of the first external device 3000-1 or the second external device 3000-2 again based on the gradient received from the electronic device 1000. The server 2000 may transmit information about the re-refined global model to the electronic device 1000.

[0097] According to an embodiment of the present disclosure, since a local model is refined based on a global model reflecting information collected from various devices, the electronic device 1000 may provide a better service to a user using the local model.

[0098] The server 2000 according to an embodiment of the present disclosure may refine the global model whenever receiving gradients from the electronic device 1000, the first external device 3000-1, and the second external device 3000-2. The server 2000 may transmit information about the refined global model to the device transmitting the gradient to the server 2000.

[0099] In addition, when the information about the context of the server 2000 is changed, the global model of the server 2000 according to an embodiment of the present disclosure may be refined. Therefore, according to an embodiment of the present disclosure, the global model of the server 2000 may be refined based on data not received from an external source (e.g., a gradient of a local model refined by the electronic device 1000 or another external device).

[0100] According to an embodiment of the present disclosure, the global model may be continuously refined based on gradients continuously received from a plurality of devices or based on changes in information about the context of the server 2000. However, the embodiments of the present disclosure are not limited thereto, and the global model may be continuously refined according to various methods based on various types of data. However, unlike the continuously refined global model, when the context information used to refine the local model does not change, the local model may not be refined within a certain period of time. Therefore, the difference between the local model and the global model may increase.

[0101] As the global model continues to be refined, the server 2000 according to an embodiment of the present disclosure may determine whether the difference between the gradient of the current global model and the gradient of the global model most recently sent from the server 2000 to the electronic device 1000 is equal to or greater than a reference value. The determination may be performed periodically by the server 2000.

[0102] When it is determined that the difference between the two gradients is equal to or greater than the reference value, the server 2000 may transmit the gradient of the most recently refined global model to the electronic device 1000. The electronic device 1000 may refine the local model based on the information about the global model received from the server 2000 (e.g., the gradient of the global model).

[0103] Therefore, the electronic device 1000 according to an embodiment of the present disclosure can receive the gradient of the most recently refined global model from the server 2000 without sending the gradient to the server 2000. The electronic device 1000 according to an embodiment of the present disclosure can continuously perform such an operation of refining the local model based on the global model when receiving the gradient of the most recently refined global model from the server 2000, although the context information collected from the electronic device 1000 has not changed.

[0104] However, the embodiments of the present disclosure are not limited thereto, and the server 2000 may transmit the gradient of the currently refined global model to the electronic device 1000 under various conditions. For example, when the server 2000 refines the global model at least a reference number of times when receiving the gradient from at least one external device other than the electronic device 1000 at least a reference number of times, the server 2000 may transmit the gradient of the currently refined global model to the electronic device 1000.

[0105] Figure 3 is a diagram illustrating an example system for performing object recognition using local models according to an embodiment of the present disclosure.

[0106] Reference Figure 3 According to an embodiment of the present disclosure, a system for performing object recognition using a local model may include a server 2000 and at least one electronic device, for example, electronic devices 3001, 3002, and 3003 (hereinafter may be referred to as 3001 to 3003). Figure 3 Each of the electronic devices 3001 to 3003 shown may correspond to Figure 1 and 2 An electronic device 1000 is shown.

[0107] The electronic devices 3001 to 3003 according to the embodiments of the present disclosure may be household appliances in different home environments and used by different users. The objects that the electronic devices 3001 to 3003 can recognize may be different from each other because the electronic devices 3001 to 3003 are in different home environments. The electronic devices 3001 to 3003 according to the embodiments of the present disclosure may respectively include a first local object recognition model to a third local object recognition model for performing object recognition, and the electronic devices 3001 to 3003 may perform object recognition by using the first local object recognition model to the third local object recognition model, respectively.

[0108] According to an embodiment of the present disclosure, the electronic devices 3001 to 3003 may refine at least one of the first to third local object recognition models for object recognition based on information about the surrounding environments of the electronic devices 3001 to 3003. Therefore, the first to third local object recognition models of the electronic devices 3001 to 3003 according to the embodiments of the present disclosure may be refined so that object recognition suitable for different surrounding environments of the electronic devices 3001 to 3003 is performed.

[0109] For example, the information about the surrounding environment used to refine the first local object recognition model to the third local object recognition model may include, for example but not limited to, state information, external information, shape information, etc. about objects recognizable by the electronic devices 3001 to 3003. However, the embodiment is not limited thereto, and the information about the surrounding environment may include various types of information that can be used to refine the first local object recognition model to the third local object recognition model for object recognition.

[0110] According to an embodiment of the present disclosure, the context information collected by the electronic device 1000 and related to the local object recognition model used for object recognition may include various types of information related to object recognition, such as, for example but not limited to, information about the state of objects existing around the electronic device 1000, information about objects newly recognized by the electronic device 1000 among the objects existing around the electronic device 1000, and the like.

[0111] By sending the gradient of the refined first local object recognition model to the third local object recognition model to the server 2000, the server 2000 can refine the global object recognition model according to the gradient of the first local object recognition model to the third local object recognition model refined by the electronic devices 3001 to 3003. The electronic devices 3001 to 3003 can refine the first local object recognition model to the third local object recognition model respectively stored therein based on the information about the refined global object recognition model received from the server 2000 (e.g., the gradient of the global object recognition model).

[0112] Figure 4 is a diagram illustrating an example system for performing speech recognition using a local model according to an embodiment of the present disclosure;

[0113] Reference Figure 4 According to an embodiment of the present disclosure, a system for performing speech recognition using a local model may include a server 2000 and at least one electronic device, such as electronic devices 4001, 4002, and 4003 (which may be referred to as 4001 to 4003 hereinafter). Figure 4Each of the electronic devices 4001 to 4003 shown may correspond to Figure 1 and 2 An electronic device 1000 is shown.

[0114] The server 2000 according to an embodiment of the present disclosure may include a global voice processing model in the global model that can be used to perform voice recognition operations. The electronic devices 4001 to 4003 according to an embodiment of the present disclosure may respectively include a first local voice processing model to a third local voice processing model as local models that can be used to perform voice recognition operations.

[0115] The electronic devices 4001 to 4003 according to the embodiments of the present disclosure may be electronic devices in different surrounding environments and used by different users. By being located in different surrounding environments, the electronic devices 4001 to 4003 may perform speech recognition based on noise information of different surrounding environments.

[0116] According to an embodiment of the present disclosure, the collected information related to the local model for speech recognition among the context information collected by the electronic device 1000 may include various types of information related to speech recognition, such as, for example but not limited to, information about noise generated in the surrounding environment of the electronic device 1000, information about the voice of the user recognizable by the electronic device 1000, and the like.

[0117] For example, the electronic devices 4001 to 4003 may be located in a house, an office, and a bus, respectively. By being located in different surrounding environments, the electronic devices 4001 to 4003 may have different noise characteristics included in the audio signal whose voice is recognized.

[0118] According to an embodiment of the present disclosure, electronic devices 4001 to 4003 can use the noise information of the surrounding environment of the electronic devices 4001 to 4003 to further effectively perform speech recognition according to the first local speech processing model to the third local speech processing model. For example, electronic devices 4001 to 4003 can further effectively perform speech recognition on the user's voice included in the audio signal based on the noise characteristics of the surrounding environment where speech recognition is to be performed and the voice characteristics of the user. For example, electronic devices 4001 to 4003 can use the noise information of the surrounding environment to obtain the relationship information between the user's voice and at least one type of noise, and refine the first local speech processing model to the third local speech processing model based on the obtained relationship information, so that speech recognition is effectively performed.

[0119] According to an embodiment of the present disclosure, the electronic devices 4001 to 4003 can refine the first to third local speech processing models for speech recognition based on information about the noise of the surrounding environment of the electronic devices 4001 to 4003. Therefore, the first to third local speech processing models of the electronic devices 4001 to 4003 according to the embodiment of the present disclosure can be refined so that speech recognition suitable for the noise of different surrounding environments of the electronic devices 4001 to 4003 is performed.

[0120] For example, the information about the noise of the surrounding environment used to refine the first to third local speech processing models may include, for example but not limited to, information about the characteristics of at least one type of noise detected by the electronic devices 4001 to 4003, relationship information between the user's voice and at least one type of noise, etc. However, the embodiment is not limited thereto, and the information about the noise of the surrounding environment may include various types of information that can be used to refine the first to third local speech processing models for speech recognition.

[0121] According to an embodiment of the present disclosure, when the information about the noise of the surrounding environment is changed, it is determined that the context information is changed, and therefore the first local speech processing model according to the embodiment of the present disclosure can be refined to the third local speech processing model. For example, according to an embodiment of the present disclosure, for example but not limited to, when a new type of noise appears, when new relationship information among the relationship information between the user's voice and at least one type of noise is obtained, etc., it can be determined that the context information is changed.

[0122] By sending the gradient of the refined first local speech processing model to the third local speech processing model to the server 2000, the server 2000 can refine the global speech processing model according to the first local speech processing model to the third local speech processing model refined by the electronic devices 4001 to 4003. The electronic devices 4001 to 4003 can refine the first local speech processing model to the third local speech processing model respectively stored therein based on the information about the refined global speech processing model received from the server 2000 (e.g., the gradient of the global speech processing model).

[0123] Figure 5 is a diagram showing an example of refining a local model and a global model according to an embodiment of the present disclosure.

[0124] Reference Figure 5 , the electronic device 1000 may include a local model 501 , and the server 2000 may include a global model 502 .

[0125] exist Figure 5In the example shown, arrows in the horizontal direction are associated with embodiments of the present disclosure in which operations are performed via corresponding AI models, and arrows in the vertical direction are associated with embodiments of the present disclosure in which corresponding AI models are refined.

[0126] According to an embodiment of the present disclosure, the electronic device 1000 may refine the local model 501 based on the collected various types of information, and obtain a gradient based on the refined local model 501 .

[0127] According to an embodiment of the present disclosure, information used to refine the local model 501 may include, for example but not limited to, changed context information about the electronic device 1000. For example, the information used for refinement may include changed context information based on feedback information about the result of an operation performed by the local model 501.

[0128] According to an embodiment of the present disclosure, the server 2000 may refine the global model 502 based on the information collected by the server 2000 for refining the global model 502 , and obtain a gradient of the refined global model 502 .

[0129] According to an embodiment of the present disclosure, when the server 2000 receives at least one gradient, information for refining the global model 502 may be obtained, wherein the at least one gradient is obtained when the local model 501 stored in the electronic device 1000 or the local model stored in the external device is received via the network 100. The server 2000 according to an embodiment of the present disclosure may refine the global model 502 based on the received gradient.

[0130] The server 2000 according to an embodiment of the present disclosure may send the gradient of the refined global model 502 to the electronic device 1000, wherein the electronic device 1000 sends the gradient of the local model 501 to the server 2000 or an external device. The electronic device 1000 or the external device that receives the information about the refined global model 502 from the server 2000 may refine the local model 501 based on the information about the refined global model 502.

[0131] When input information is input, the local model 501 and the global model 502 according to an embodiment of the present disclosure may output information corresponding to the input information.

[0132] According to an embodiment of the present disclosure, when operating in an on-device manner, based on the local model 501, the electronic device 1000 can perform, for example but not limited to, item recommendation, object recognition, voice recognition, etc.

[0133] According to an embodiment of the present disclosure, the input information input to the local model 501 may include, for example, but not limited to, information required to perform item recommendation, object recognition, speech recognition, etc. For example, the input information for item recommendation may include information about the type of item to be recommended. The input information for object recognition may include sensor information currently detected by the electronic device 1000, such as, for example, but not limited to, image information, infrared light detection information, etc. The input information for speech recognition information may include information about an audio signal currently received by the electronic device 1000. The input information may include contextual information for refining the local model 501. However, the embodiments of the present disclosure are not limited to this, and the input information may include various types of information for the electronic device 1000 to perform operations.

[0134] The output information output by the local model 501 according to an embodiment of the present disclosure may include, for example, but not limited to, information on the results of performing item recommendation, object recognition, speech recognition, etc. For example, the output information about item recommendation may include information about the recommended item and ranking information of the recommended item. The output information about object recognition may include, for example, but not limited to, identification information, state information, etc. of an object recognized based on sensor information. The output information about speech recognition may include text information obtained by performing speech recognition on an audio signal input as input information. However, the embodiments of the present disclosure are not limited to this, and the output information may include various types of information indicating the results of the electronic device 1000 performing an operation according to the local model 501.

[0135] According to an embodiment of the present disclosure, when the electronic device 1000 operates in a cloud manner rather than on a device, the server 2000 can use the global model 502 to perform various operations, such as item recommendation, object recognition, and speech recognition, according to a request of the electronic device 1000.

[0136] For example, when the electronic device 1000 operates in a cloud manner, the electronic device 1000 can request the server 2000 to perform the operation instead by sending the input information to be input to the local model 501 to the server 2000. Based on the request of the electronic device 1000, the server 2000 can perform the operation using the global model 502 and send its result to the electronic device 1000.

[0137] Figure 6 is a diagram illustrating an example of a local model or a global model according to an embodiment of the present disclosure.

[0138] The local model or global model according to an embodiment of the present disclosure may be configured as a neural network model that simulates a method such as a human brain recognition model. The electronic device 1000 or the server 2000 according to an embodiment of the present disclosure may provide various services to a user using a local model or a global model configured as a neural network model.

[0139] The neural network model according to one embodiment of the present disclosure may include, for example but not limited to, a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a bidirectional recurrent deep neural network (BRDNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a deep Q network, etc. However, the embodiments of the present disclosure are not limited thereto, and the local model or the global model according to an embodiment of the present disclosure may be one of various types of neural networks.

[0140] refer to Figure 6 , the neural network model according to an embodiment of the present disclosure may include at least one layer including at least one node N. For example, the neural network model may include a first layer as an input layer and a second layer as an output layer. The neural network model according to an embodiment of the present disclosure may further include at least one hidden layer (not shown) between the input layer and the output layer. Hereinafter, for ease of description, an example of a neural network including an input layer and an output layer but not including a hidden layer will be described in a non-limiting example manner.

[0141] According to an embodiment of the present disclosure, at least one input value may be input to the first layer of the neural network model. For example, the values ​​of I1 and I2 may be input to nodes N11 and N12 of the first layer, respectively. According to an embodiment of the present disclosure, when the input value is input to the first layer, nodes N11, N12, N21, and N22 included in the first and second layers of the neural network model may be processed.

[0142] The values ​​output from the nodes of each layer can be used as input values ​​of the next layer. For example, certain values ​​can be input to the nodes N21 and N22 of the second layer based on the values ​​obtained when processing the nodes N11 and N12 of the first layer. The values ​​output from the second layer can be output as output values ​​from the neural network model. For example, O1 and O2, which are the values ​​output from the second layer, can be output as output values ​​of the neural network model.

[0143] The electronic device 1000 according to an embodiment of the present disclosure may input the value of the input information to the input layer of the local model. The electronic device 1000 may provide various services (eg, item recommendation, object recognition, and speech recognition) to the user based on the value output from the output layer of the local model.

[0144] According to an embodiment of the present disclosure, when different weight values ​​are applied to a value output from a node, at least one edge data can be obtained from a node. Edge data is data that can be obtained when at least one weight value is applied to a value output from a node. Edge data can be obtained by applying multiple weight values ​​to a value. Therefore, the value output from each node of the first layer can be input to the node of the next layer after being converted into at least one edge data. For example, the edge data obtained by applying W11 and W12 as different weight values ​​to the value output from the node N11 can be respectively input to the nodes N21 and N22 of the second layer. The edge data obtained by applying W21 and W22 as different weight values ​​to the value output from the node N12 can be respectively input to the nodes N21 and N22 of the second layer.

[0145] According to an embodiment of the present disclosure, when the local model is refined, at least one weight value applicable to each node may be changed. The gradient according to an embodiment of the present disclosure may include information obtained during an arithmetic operation for deriving an optimal weight value applied to each node.

[0146] According to an embodiment of the present disclosure, a loss function or a cost function may be used to derive an optimal weight value W11 ′ (not shown) from the weight value W11 before the local model is refined.

[0147] According to an embodiment of the present disclosure, a loss value that can be determined via a loss function can be determined based on the difference between observation information that can be output by a local model and observation information indicating an answer to prediction information. According to an embodiment of the present disclosure, observation information can be determined as information corresponding to prediction information that can be output by a local model based on currently collected context information.

[0148] According to an embodiment of the present disclosure, the local model may be refined by replacing the weight value W11 of the local model with the weight value W11 ′ at which the loss value is minimized.

[0149] According to an embodiment of the present disclosure, a mean square error function or a cross entropy error function may be used as a loss function. However, the embodiment of the present disclosure is not limited thereto, and various types of functions may be used as a loss function.

[0150] The mean square error function and the cross entropy error function according to an embodiment of the present disclosure may be represented by the following Equation 1. In Equation 1, y and t respectively indicate a value of predicted information and a value of observed information.

[0151] [Equation 1]

[0152] (Mean Square Error Function)

[0153] (Cross Entropy Error Function)

[0154] The electronic device 1000 according to an embodiment of the present disclosure may determine a parameter (e.g., a weight value or a bias value of each node of a local model) that minimizes and / or reduces a loss value obtained by a loss function to derive an optimal weight value W11'. For example, in a loss function graph in which a first prediction value among a plurality of prediction values ​​of prediction information is a variable and other prediction values ​​are constants, when the loss value (e.g., a result value of the loss function) is minimum at a point where the slope is minimum, the electronic device 1000 may obtain the minimum value of the slope as a first gradient.

[0155] According to an embodiment of the present disclosure, when the loss value is the smallest, the second to nth gradients corresponding to the respective prediction values ​​can be obtained in the same manner as the second to nth prediction values. Therefore, the electronic device 1000 can obtain a gradient including the first to nth gradients as the gradient of the refined local model.

[0156] The electronic device 1000 according to an embodiment of the present disclosure may refine the local model based on a parameter corresponding to at least one predicted value when the loss value is minimized.

[0157] However, the embodiments of the present disclosure are not limited thereto, and the gradient may include various types of values ​​obtained in the process of deriving various types of information configuring the refined local model when refining the local model. For example, the gradient may include the difference between at least one weight value applicable to a node configuring the refined local model and a weight value before the local model is refined.

[0158] The gradient of the global model according to an embodiment of the present disclosure can be obtained in the same way as the gradient of the local model. For example, after the global model is refined, the slope information of the loss function corresponding to the prediction information obtainable by refining the global model can be obtained as the gradient of the refined global model. In addition, the gradient of the global model may include the difference between at least one weight value applicable to a node configuring the refined global model and the weight value before the global model is refined.

[0159] Figure 7 is a diagram illustrating an example operation of the electronic device 1000 according to an embodiment of the present disclosure.

[0160] Reference Figure 7 , the electronic device 1000 can use the local model 701 and the context information 702 to perform an operation of recommending items.

[0161] According to an embodiment of the present disclosure, the electronic device 1000 may perform an operation (e.g., item recommendation, object recognition, or speech recognition) via the local model 701, and may change the context information 702 based on the result of the operation. For example, the context information 702 may be changed based on user feedback information about the operation result.

[0162] According to an embodiment of the present disclosure, user feedback information regarding operations according to the local model 701 may be used to refine the local model 701 as the changed context information 702 .

[0163] Therefore, according to an embodiment of the present disclosure, when context information 702 is input into the local model 701, a specific operation (e.g., item recommendation, object recognition, or speech recognition) can be performed, but when the context information 702 is changed, the local model 701 can be refined based on the changed context information 702.

[0164] According to an embodiment of the present disclosure, performing an operation via the local model 701 , changing the context information 702 based on feedback information generated as a result of the operation, and sending the changed context information 702 to the local model 701 as information for refining the local model 701 may be considered as one step.

[0165] According to an embodiment of the present disclosure, at least one of refinement of the local model 701 or operations performed by the local model 701 may be performed each time a step is performed.

[0166] According to an embodiment of the present disclosure, after repeatedly performing a predetermined number of steps, a gradient may be generated based on the final refined local model 701. The electronic device 1000 according to an embodiment of the present disclosure may receive information about the global model by sending the generated gradient to the server 2000, and refine the local model 701 again based on the received information.

[0167] According to an embodiment of the present disclosure, the gradient of the local model or the global model may further include information obtained based on information about the policy or value.

[0168] The information about the strategy of the local model may include information about which operation can be selected as the output value according to the value input to the local model. For example, the information about the strategy (π) may include information for indicating at least one of the weight value or the bias value of each node. For example, the information about the strategy may be expressed as π(a t |s t ; θ), where a indicates an operation as an output value of the local model, s represents state information as an input value of the local model, t represents time, and θ represents a parameter.

[0169] The information about the value of the local model may include information about a value indicating how suitable the information about the strategy for performing an operation according to the local model is. For example, the information about the value (V) may be represented by V(St, θv).

[0170] For example, information on data of a policy of a local model (Lp) and information on data of a value (Lv) may be obtained according to the following Equations 2 and 4, respectively.

[0171] In Equation 2, H(π(θ)) represents an entropy function for improving the model exploration strategy, and β represents a parameter for adjusting the strength of the entropy regularization term.

[0172] [Equation 2]

[0173] L p =-log(π(s;θ))A(s,a;θ,θ v )-βH(π(θ)))

[0174] A(s, a; θ, θv) of Equation 2 can be obtained according to the following Equation 3. In Equation 3, k represents the total number of times the local model is refined, and γ represents a discount factor.

[0175] [Equation 3]

[0176]

[0177] In Equation 4, R represents the user's feedback information.

[0178] [Equation 4]

[0179] L v =∑(RV(s;θ)) 2

[0180] However, the embodiment is not limited thereto, and the gradient may include various types of information that can be used to determine a value related to at least one node configuring the local model.

[0181] Figure 8 is a diagram showing an example in which a local model is refined according to time according to an embodiment of the present disclosure.

[0182] Reference Figure 8, the electronic device 1000 can perform an operation at time point T1 by inputting context information 801 into the local model 802, and obtain user feedback information 803 about the performed operation. According to an embodiment of the present disclosure, the context information 801 of the electronic device 1000 is input as input information into the local model 802 so that a certain operation is performed. However, the embodiments of the present disclosure are not limited thereto, and in addition to the context information 801, various types of information required for performing the operation can also be input into the local model 802.

[0183] According to an embodiment of the present disclosure, the collected context information related to the local model for item recommendation in the context information collected by the electronic device 1000 may include various types of information related to item recommendation, such as, for example but not limited to, information about at least one item that can be recommended by the local model, information about the interaction between the user and at least one item, and the like.

[0184] Information about at least one item that can be recommended by the local model may include, for example, but not limited to, an application, a moving image, a still image, a text file, a web page, etc. The item may be link information or index information of an image connected to the item. The item may be configured in the form of an object for using a service. According to an embodiment of the present disclosure, an object for using a service may represent a user interface for using a service of a service provider. For example, an object for using a service may include, for example, but not limited to, an icon, text, a user interface including an image and link information, a description of the function of the object, etc., for using the service provided by a content provider.

[0185] Information about the interaction between the user and at least one item may include, for example but not limited to, information about the history of the user's use of the at least one item, feedback information from the user regarding items recommended by the local model, information about the method, time and place when the user used the at least one item, information about a signal input by the user for the at least one item, etc.

[0186] Feedback information 803 according to an embodiment of the present disclosure may be generated based on information output according to an operation performed via the local model 802. For example, the feedback information 803 may include information input by a user about the result of the operation performed according to the local model 802. For example, in the case of item recommendation, the feedback information 803 may include information about an item selected by a user among recommended items as output information of the local model 802, information about whether the user selected the recommended item, and ranking information of the recommended item input by the user. In the case of object recognition and speech recognition, the feedback information 803 may include information about the user's opinion on whether the result of object recognition or speech recognition as output information of the local model 802 is suitable.

[0187] At time point T2, context information 804 may be changed based on state 803-1 changed according to user feedback information 803. For example, electronic device 1000 may change information about the interaction between the user and the item in context information 804 based on user feedback information 803. Electronic device 1000 may change information about whether each item is recommended and ultimately selected by the user in context information 804 as feature information about each item based on the result of performing item recommendation. Electronic device 1000 may change information about the result of object recognition or speech recognition in context information 804 based on user feedback information 803 about the result of performing object recognition or speech recognition.

[0188] However, the embodiments of the present disclosure are not limited thereto, and the context information 804 may be changed via various methods based on the state 803 - 1 changed according to the feedback information 803 of the user.

[0189] In addition, at time point T2, the local model 805 may be refined based on the feedback information 803. The electronic device 1000 may refine the local model 805 by reflecting the feedback information 803 of the user.

[0190] At time point T2, when the context information 801 is changed and the local model 802 is refined, the electronic device 1000 can perform an operation by inputting its changed context information 804 into the local model 805, and obtain user feedback information 806 about the performed operation, as at time point T1.

[0191] At time point T3, context information 804 may be changed, and local model 805 may be refined in the same manner as at time points T1 and T2. For example, at time point T3, context information 807 may be changed based on state 806-1 changed according to user feedback information 806. At time point T3, when context information 804 is changed and local model 805 is refined, electronic device 1000 may perform an operation by inputting its changed context information 807 into local model 808, and obtain user feedback information 809 about the performed operation, as at time points T1 and T2.

[0192] Fig. 9 is a block diagram illustrating an example configuration of an example electronic device 1000 according to an embodiment of the present disclosure.

[0193] Fig.10 is a block diagram illustrating an example configuration of an example electronic device 1000 according to an embodiment of the present disclosure.

[0194] Reference Fig. 9, the electronic device 1000 may include a memory 1700, a processor (eg, including a processing circuit) 1300, and a communicator (eg, including a communication circuit) 1500. However, Fig. 9 The components shown in the figure are not all the basic components of the electronic device 1000. The electronic device 1000 may use Fig. 9 More components than shown may be used, or Fig. 9 The components shown may be implemented with fewer components.

[0195] For example, Fig.10 As shown, in addition to the memory 1700, the processor 1300 and the communicator 1500, the electronic device 1000 according to an embodiment of the present disclosure may also include a user inputter (inputter) (e.g., including an input circuit) 1100, an outputter (outputter) (e.g., including an output circuit) 1200, a sensor 1400 and an audio / video (A / V) inputter (e.g., including an A / V input circuit) 1600.

[0196] The user input 1100 may include various input circuits, and may be a unit that inputs data for a user to control the electronic device 1000. For example, the user input 1100 may include various input circuits, including, for example but not limited to, a keyboard, a dome switch, a touch pad (contact capacitance type, piezoresistive type, infrared detection type, surface ultrasonic wave conduction type, integral tension measurement type, piezoelectric effect type, etc.), a jog wheel, a jog switch, etc., but not limited thereto.

[0197] According to an embodiment of the present disclosure, the user input device 1100 may receive an input of a user requesting an operation (e.g., item recommendation, object recognition, or speech recognition) that may be performed according to a local model. In addition, the user input device 1100 may receive an input from a user, the input including feedback information about an operation performed via a local model of the electronic device 1000.

[0198] The outputter 1200 may include various output circuits and output an audio signal, a video signal, or a vibration signal, and the outputter 1200 may include various circuits, such as, but not limited to, a display 1210, a sound outputter (e.g., including a sound output circuit) 1220, and a vibration motor 1230. The outputter 1200 according to an embodiment of the present disclosure may output a result of an operation performed via a local model.

[0199] The display 1210 displays information processed by the electronic device 1000. According to an embodiment of the present disclosure, the display 1210 may display a result of an operation performed via a partial model according to a user's input.

[0200] When the display 1210 and the touch panel are configured as a touch screen by forming a layer structure, the display 1210 can also be used as an input device and an output device. The display 1210 may include, for example but not limited to, at least one of a liquid crystal display, a thin film transistor liquid crystal display, an organic light emitting diode, a flexible display, a three-dimensional (3D) display, an electrophoretic display, etc. Depending on the implementation of the electronic device 1000, the electronic device 1000 may include two or more displays 1210.

[0201] The sound outputter 1220 may include various sound output circuits and output audio data received from the communicator 1500 or stored in the memory 1700. The sound outputter 1220 according to an embodiment of the present disclosure may output audio data indicating a result of an operation performed via a partial model.

[0202] The vibration motor 1230 may output a vibration signal. When a touch is input on the touch screen, the vibration motor 1230 may output a vibration signal. The vibration motor 1230 according to an embodiment of the present disclosure may output a vibration signal indicating a result of an operation performed via a partial model.

[0203] The processor 1300 may include various processing circuits and control the overall operation of the electronic device 1000. For example, the processor 1300 may run a program stored in the memory 1700 to generally control the user input 1100, the output 1200, the sensor 1400, the communicator 1500, and the A / V input 1600. The electronic device 1000 may include at least one processor 1300.

[0204] The processor 1300 may be configured to process a command of a computer program by performing basic calculations, logic, and input or output arithmetic operations. The command may be provided to the processor 1300 from the memory 1700, or may be provided to the processor 1300 after being received via the communicator 1500. For example, the processor 1300 may be configured to execute a command according to a program code stored in a recording device such as a memory.

[0205] At least one processor 1300 according to an embodiment of the present disclosure may detect that information about the context of the electronic device 1000 used to refine the local model is changed. The information about the context may include, for example, context information such as information about the state of the electronic device 1000, information about the user of the electronic device 1000, and information about the surrounding environment of the electronic device 1000.

[0206] At least one processor 1300 may generate a gradient for refining the local model based on the changed context information. At least one processor 1300 according to an embodiment of the present disclosure may use the gradient to refine the local model.

[0207] At least one processor 1300 according to an embodiment of the present disclosure may refine the local model of the electronic device 1000 again based on the information about the global model received when the gradient is sent to the server. The information about the global model may be a gradient of the global model generated based on the gradient sent from the electronic device 1000. According to an embodiment of the present disclosure, the gradient of the global model is generated based on the gradient sent from the electronic device 1000, and the global model may be refined based on the gradient of the global model.

[0208] The sensor 1400 may detect a state of the electronic device 1000 and / or a state of a surrounding environment of the electronic device 1000 , and transmit the detected information to the processor 1300 .

[0209] According to an embodiment of the present disclosure, context information for refining a local model may be obtained based on various types of information detected by the sensor 1400. For example, information about a user or a surrounding environment in the context information may be obtained based on various types of sensing information related to the user or the surrounding environment detected by the sensor 1400.

[0210] Sensor 1400 may include, for example but not limited to, at least one of a magnetic sensor 1410, an acceleration sensor 1420, a temperature / humidity sensor 1430, an infrared sensor 1440, a gyroscope sensor 1450, a position sensor 1460 (e.g., a global positioning system), an atmospheric pressure sensor 1470, a proximity sensor 1480 and / or an RGB sensor 1490 (illuminance sensor), but is not limited thereto.

[0211] The communicator 1500 may include various communication circuits included in one or more components so that the electronic device 1000 can communicate with the server 2000 or an external device. For example, the communicator 1500 may include, for example but not limited to, a short-range wireless communicator 1510, a mobile communicator 1520, and a broadcast receiver 1530.

[0212] The short-range wireless communicator 1510 may include a Bluetooth communicator, a Bluetooth low energy (BLE) communicator, a near field communicator / radio frequency identification communicator (NFC / RFID), a wireless local area network (WLAN) (Wi-Fi) communicator, a Zigbee communicator, an infrared data association (IrDA) communicator (not shown), a Wi-Fi direct (WFD) communicator, an ultra wideband (UWB) communicator and / or an Ant+ communicator, but is not limited to these.

[0213] The mobile communicator 1520 may include various mobile communication circuits and transmit or receive wireless signals to or from at least one of a base station, an external terminal or a server on a mobile communication network. Here, the wireless signals may include various types of data according to the exchange of voice call signals, image call signals or text / multimedia messages.

[0214] The broadcast receiver 1530 may include various receiver circuits and receive a broadcast signal and / or broadcast-related information from an external source through a broadcast channel. The broadcast channel may include a satellite channel or a terrestrial channel. According to an embodiment, the electronic device 1000 may not include the broadcast receiver 1530.

[0215] The communicator 1500 according to an embodiment of the present disclosure may transmit a gradient generated by the electronic device 1000 to the server 2000 , and receive information of a global model refined based on the transmitted gradient from the server 2000 .

[0216] The A / V input device 1600 may include various A / V input circuits, may include a unit for inputting an audio signal or a video signal, and may include, for example, a camera 1610 and a microphone 1620. The camera 1610 may obtain an image frame of a still image or a moving image via an image sensor in a video call mode or a shooting mode. The image captured by the image sensor may be processed via the processor 1300 or a separate image processor (not shown). The microphone 1620 may receive an external sound signal and process the external sound signal into electrical voice data.

[0217] The electronic device 1000 according to an embodiment of the present disclosure may obtain context information based on an audio signal or a video signal received via the A / V inputter 1600. For example, information about a user or a surrounding environment in the context information may be obtained based on an audio signal or a video signal about the user or the surrounding environment obtained via the A / V inputter 1600.

[0218] The memory 1700 may store programs for processing and control of the processor 1300 , and store data input to or output from the electronic device 1000 .

[0219] The memory 1700 according to an embodiment of the present disclosure may store at least one instruction, and the at least one processor 1300 of the electronic device 1000 may perform an operation according to an embodiment of the present disclosure by executing the at least one instruction stored in the memory 1700 .

[0220] The memory 1700 according to an embodiment of the present disclosure may store at least one local model for performing at least one operation and context information of the electronic device 1000 for refining the local model.

[0221] The memory 1700 may include at least one type of storage medium selected from the group consisting of a flash memory type, a hard disk type, a multimedia card micro type, a card-type memory (e.g., a secure digital (SD) or an extreme digital (XD) memory), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a disk, or an optical disk.

[0222] The programs stored in the memory 1700 may be classified into a plurality of modules based on functions, and for example but not limited to, may be classified into a user interface (UI) module 1710 , a touch screen module 1720 , and a notification module 1730 .

[0223] The UI module 1710 may provide a dedicated UI or a graphical user interface (GUI) that interacts with the electronic device 1000 for each application. The touch screen module 1720 may detect a touch gesture of a user on the touch screen and send information about the touch gesture to the processor 1300. The touch screen module 1720 according to an embodiment of the present disclosure may recognize and analyze a touch code. The touch screen module 1720 may be configured as independent hardware including a controller.

[0224] Various sensors may be provided inside or near the touch screen to detect a touch or near touch on the touch screen. Examples of sensors for detecting a touch on the touch screen include tactile sensors. The tactile sensors may include sensors for detecting contact with a specific object, where the degree of contact with the specific object is the same as or greater than that which a person can feel. The tactile sensors may detect various types of information, such as the roughness of the contact surface, the rigidity of the contact object, and the temperature of the contact point.

[0225] The user's touch gesture may include, for example but not limited to, tapping, touching and holding, double-clicking, dragging, panning, flicking, dragging and dropping, sliding, etc.

[0226] The notification module 1730 may generate a signal for notifying the electronic device 1000 of occurrence of an event.

[0227] Fig.11 is a block diagram illustrating an example configuration of a server 2000 according to an embodiment of the present disclosure.

[0228] Reference Fig.11 According to an embodiment of the present disclosure, the server 2000 may include a memory 2100, a processor (eg, including a processing circuit) 2200, and a communicator (eg, including a communication circuit) 2300. Fig. 9 The components of the electronic device 1000 shown are the same.

[0229] However, Fig.11 The components shown in the figure are not all the basic components of the server 2000. The server 2000 can use Fig.11 may be implemented with more components than those shown in Fig.11 The components shown in the figure can be implemented with fewer components.

[0230] The processor 2200 may include various processing circuits and control the overall operation of the server 2000. For example, the processor 2200 may generally control the server 2000 by executing a program stored in the memory 2100. The server 2000 may include at least one processor 2200.

[0231] The processor 2200 may be configured to process a command of a computer program by performing basic calculations, logic, and input or output arithmetic operations. The command may be provided to the processor 2200 from the memory 2100, or may be provided to the processor 2200 after being received via the communicator 2300. For example, the processor 2200 may be configured to execute a command according to a program code stored in a recording device such as a memory.

[0232] The at least one processor 2200 according to an embodiment of the present disclosure may refine the global model based on the gradient received from the electronic device 1000. The at least one processor 2200 may generate information about the refined global model and transmit it to the electronic device 1000.

[0233] At least one processor 2200 according to an embodiment of the present disclosure may refine the global model based on the gradients received not only from the electronic device 1000 but also from a plurality of external devices. Therefore, the global model according to an embodiment of the present disclosure may be refined based not only on the gradients of the electronic device 1000 but also on the gradients of a plurality of external devices sent to the server 2000.

[0234] The communicator 2300 may include various communication circuits included in one or more components so that the server 2000 can communicate with the electronic device 1000 or an external device.

[0235] The communicator 2300 according to an embodiment of the present disclosure may receive a gradient from the electronic device 1000 and transmit information on a global model refined based on the received gradient to the electronic device 1000 .

[0236] The memory 2100 may store programs for processing and control of the processor 2200 , and store data input to or output from the server 2000 .

[0237] The memory 2100 according to an embodiment of the present disclosure may store at least one instruction, and the at least one processor 2200 of the server 2000 may perform an operation according to an embodiment of the present disclosure by executing the at least one instruction stored in the memory 2100 .

[0238] The memory 2100 of the server 2000 according to an embodiment of the present disclosure may store a global model for performing a certain operation (eg, item recommendation, object recognition, or speech recognition).

[0239] Fig.12 is a flowchart illustrating an example method of refining a local model according to an embodiment of the present disclosure.

[0240] Reference Fig.12 In operation 1201, the electronic device 1000 may detect that context information of the electronic device 1000 used to refine a local model is changed. The context information of the electronic device 1000 may be used to refine a local model included in the electronic device 1000 and may be input to the local model to be used to perform an operation (e.g., item recommendation, object recognition, or speech recognition) according to the local model.

[0241] For example, when a new item available by the electronic device 1000 is discovered, context information related to the available items of the electronic device 1000 may be changed. When the electronic device 1000 is moved to a new location, context information related to the location of the electronic device 1000 may be changed. When a new object appears in the surroundings of the electronic device 1000 or the shape of an existing object is changed, context information related to the surroundings of the electronic device 1000 may be changed. When a new utterer appears in the surroundings of the electronic device 1000, context information about the utterer for which speech recognition can be performed by the electronic device 1000 may be changed.

[0242] The context information may be changed based on user feedback information regarding the results of performing operations according to the local model.

[0243] However, embodiments of the present disclosure are not limited thereto, and the context information may include various types of information about the context of the electronic device 1000 , which may be used to refine the local model or perform an operation according to the local model.

[0244] In operation 1202, the electronic device 1000 may determine a gradient for refining the local model based on the changed context information. Based on the changed context information, the gradient according to an embodiment of the present disclosure may be obtained according to a loss function indicating the difference between the prediction information that may be output to the local model and the observation information indicating the answer to the prediction information. For example, the gradient may include information related to the point where the loss function is minimum, for example, the slope information of the loss function at the point.

[0245] In operation 1203 , the electronic device 1000 may refine the local model based on the gradient.

[0246] In operation 1204 , the electronic device 1000 may transmit the gradient to the server 2000 .

[0247] According to an embodiment of the present disclosure, the electronic device 1000 may send a gradient instead of the state information of the electronic device 1000 to the server 2000, and the state information may include the user's sensitive personal information. Therefore, according to an embodiment of the present disclosure, the risk of leakage of the user's sensitive personal information during transmission may be reduced.

[0248] According to an embodiment of the present disclosure, at least one of an operation of adding noise to a gradient or an operation of encrypting a gradient may be performed so that the gradient is protected during transmission. For example, an arbitrarily generated noise value may be added to at least one value included in the gradient indicating data modified by the refined AI model. The noise value may be negligible compared to the at least one value. Therefore, the electronic device 1000 or the server 2000 may refine a local model or a global model according to an embodiment of the present disclosure without having to identify or remove the added noise, even if the noise is added to the gradient.

[0249] Since the data value itself is changed when noise is added to the data, even in the case where the gradient is leaked, the data included in the leaked gradient can be prevented. Therefore, according to an embodiment of the present disclosure, since the gradient modified by adding noise or the encrypted gradient is sent instead of the gradient itself, the gradient can be protected from external attacks or prevented from leakage.

[0250] In operation 1205, the electronic device 1000 may receive information about the global model by sending the gradient to the server 2000. The global model of the server 2000 may be refined based on the gradient sent by the electronic device 1000, and the information about the refined global model may be sent to the electronic device 1000. For example, the information about the refined global model may be the gradient of the refined global model.

[0251] The information about the global model received in operation 1205 may be a gradient of the global model including information configuring the global model. In operation 1206, the electronic device 1000 may obtain a gradient of the refined global model of the server 2000 based on the gradient of the final refined global model, and refine the local model based on the obtained gradient of the global model.

[0252] The gradient of the global model according to an embodiment of the present disclosure can be obtained based on the difference between the weight values ​​of the global model of the server 2000 at fixed time intervals. The fixed time interval can be set based on the time interval for performing communication between the electronic device 1000 and the server 2000. For example, the fixed time interval can be set to the interval between the most recent time point at which communication is performed between the electronic device 1000 and the server 2000 and the time point before the most recent time point. However, the embodiments of the present disclosure are not limited thereto, and the gradient of the global model may include various types of information for refining the global model.

[0253] According to an embodiment of the present disclosure, the electronic device 1000 may perform operations according to a local model refined based on a gradient of a global model.

[0254] Fig.13 is a flow chart illustrating an example method of refining a global model performed by a server according to an embodiment of the present disclosure;

[0255] Reference Fig.13 , in operation 1301 , the server 2000 may receive a gradient from the electronic device 1000 .

[0256] According to an embodiment of the present disclosure, the electronic device 1000 may refine the local model based on the change of the information about the context of the electronic device 1000, wherein the information about the context of the electronic device 1000 is used to refine the local model of the electronic device 1000. The electronic device 1000 may send a gradient including information configuring the refined local model to the server 2000.

[0257] Therefore, when the local model is refined by the electronic device 1000 , the server 2000 may receive a gradient from the electronic device 1000 .

[0258] In operation 1302, the server 2000 may refine the global model based on the gradient. The global model according to an embodiment of the present disclosure is a model for performing a certain operation (e.g., item recommendation, object recognition, or speech recognition) and corresponding to the local model, and may be refined based on the gradient received from the electronic device 1000.

[0259] For example, the server 2000 can refine the global model by refining various types of information configuring the global model based on the gradient. For example, the server 2000 can refine the global model by refining the weight value and the bias value for each node configuring the global model.

[0260] In operation 1303 , the server 2000 may generate information about the refined global model, such as a gradient, and transmit the generated information about the global model to the electronic device 1000 in operation 1304 .

[0261] The electronic device 1000 according to an embodiment of the present disclosure may refine the local model of the electronic device 1000 based on the information about the global model received from the server 2000. The electronic device 1000 may perform a certain operation (eg, item recommendation, object recognition, or voice recognition) using the refined local model.

[0262] Hereinafter, according to an embodiment of the present disclosure, the gradient of the current global model stored in the server 2000 before being refined may be referred to as a first gradient, the gradient of the local model sent from the electronic device 1000 to the server 2000 may be referred to as a second gradient, and the gradient for refining the current global model obtainable based on the first gradient and the second gradient will be referred to as a third gradient.

[0263] The server 2000 according to an embodiment of the present disclosure may obtain a first gradient of a current global model and obtain a third gradient for refining the current global model based on the first gradient and the second gradient received from the electronic device 1000. The current global model may be refined based on the third gradient, and when the third gradient is sent as information about the refined global model, the local model of the electronic device 1000 may be refined again.

[0264] For example, at least one representative value of the corresponding information between the first gradient of the refined global model and the second gradient of the refined local model can be obtained. The third gradient according to an embodiment of the present disclosure can be configured in the at least one representative value obtained. The representative value can be, for example, an arithmetic mean or a weighted mean between the slope values ​​included in the first gradient and the second gradient. In addition, the representative value can be obtained based on a value obtained by simply adding the slope values ​​respectively included in the first gradient and the second gradient.

[0265] For example, the first gradient may include slope information of the loss function of the global model, and the second gradient may include slope information of the loss function of the local model. The slope information according to an embodiment of the present disclosure may include a value obtained by partially differentiating the loss function using each predicted value included in the prediction information as a variable. The slope information of the gradient may include a value corresponding to a point at which the value of the loss function is the smallest.

[0266] In this example, the slope information of the global model and the representative values ​​of the slope information of the local model can be obtained as the representative value of the third gradient. For example, the arithmetic mean (e.g., (a+b) / 2) or weighted mean (e.g., (a*a1+b*b1) / 2) of the first gradient (a) of the global model and the second gradient (b) of the local model can be obtained as the representative value of the third gradient.

[0267] According to an embodiment of the present disclosure, when the global model is refined based on the third gradient including at least one representative value, the global model based on the second gradient received from the electronic device 1000 may be refined.

[0268] When the representative value is a weighted average value between the values ​​included in the first gradient and the second gradient, the weight value applied to each value can be determined as the most appropriate value for refining the global model. For example, the weight value can be determined as the most appropriate value for refining the global model based on the result of performing machine learning to determine the weight value. As another example, the weight value of the weighted average value can be determined based on the gradient of the training result to be given a greater weight between the training result of the global model and the training result of the local model.

[0269] When refining the global model based on the gradient, according to an embodiment of the present disclosure, the server 2000 may obtain a third gradient used to refine the global model as information about the refined global model.

[0270] According to an embodiment of the present disclosure, the information about the refined global model generated by the server 2000 may include information required to refine the local model of the electronic device 1000. For example, the local model of the electronic device 1000 may be refined based on the information about the refined global model received from the server 2000 (e.g., the third gradient).

[0271] The local model according to an embodiment of the present disclosure may be refined according to the changed context information, and then refined based on the third gradient and the second gradient of the local model received from the server 2000. For example, a fourth gradient for refining the local model again may be obtained based on the third gradient and the second gradient of the local model received from the server 2000. The refinement operation of the local model in response to the reception of the third gradient may be performed based on the fourth gradient.

[0272] For example, at least one representative value of corresponding information between the information included in the third gradient and the information included in the second gradient may be obtained. The fourth gradient according to an embodiment of the present disclosure may be configured in the at least one representative value obtained. The representative value may be, for example, an arithmetic mean or a weighted mean between the values ​​included in the third gradient and the second gradient.

[0273] For example, the third gradient may include information related to the loss function of the refined global model, and the second gradient may include information related to the loss function of the local model. In this case, a representative value (e.g., an arithmetic mean or a weighted mean) of the first gradient of the refined global model and the second gradient of the local model may be obtained as a representative value of the fourth gradient.

[0274] According to an embodiment of the present disclosure, a local model based on the third gradient received from the electronic device 1000 can be refined based on a fourth gradient including at least one representative value. For example, when information related to the loss function of the local model is replaced based on the representative value included in the fourth gradient, the local model can be refined.

[0275] According to an embodiment, when the representative value is a weighted average value (e.g., (c*c1+d*d1) / 2) between the values ​​(e.g., c and d) included in the third gradient and the second gradient, the weight values ​​(e.g., c1 and d1) respectively applied to these values ​​may be determined as the most appropriate values ​​for refining the local model. For example, the weight value may be determined as the most appropriate value for refining the local model based on the result of performing machine learning to determine the weight value.

[0276] According to an embodiment of the present disclosure, machine learning for determining a weight value may be performed to determine a weight value that enables an operation of a local model refined according to the weight value to be appropriately performed without conflicting with a state of the electronic device 1000 or a surrounding context of the electronic device 1000. For example, the weight value may be learned based on information about the context of the electronic device 1000. However, the embodiments of the present disclosure are not limited thereto, and the weight value applied to each value may be determined as an appropriate value according to various methods.

[0277] According to an embodiment of the present disclosure, information about the refined global model is not limited to the third gradient, and may include various types of information required to refine the local model based on the most recently refined global model.

[0278] Fig.14 is a diagram illustrating an example system for refining local and global models according to an embodiment of the present disclosure.

[0279] like Fig.14 As shown, the system according to an embodiment of the present disclosure may include an electronic device 1000, a first server 2000-1, a second server 2000-2, a first external device 3000-1, and a second external device 3000-2.

[0280] Fig.14 The electronic device 1000, the first server 2000-1 and the second server 2000-2 correspond to Figure 1 electronic device 1000 and server 2000. Fig.14 The first external device 3000-1 and the second external device 3000-2 correspond to Figure 2 The first external device 3000 - 1 and the second external device 3000 - 2 .

[0281] Like the electronic device 1000 , the first and second external devices 3000 - 1 and 3000 - 2 according to an embodiment of the present disclosure may provide various services using partial models included in the first and second external devices 3000 - 1 and 3000 - 2 , respectively.

[0282] The electronic device 1000 according to an embodiment of the present disclosure can identify the first server 2000-1 and the second server 2000-2 that can receive information about the global model when sending the gradient of the local model. The electronic device 1000 can identify the first server 2000-1 and the second server 2000-2 by pre-receiving information about the first server 2000-1 and the second server 2000-2 from an external source. The electronic device 1000 can determine the appropriate server among the identified first server 2000-1 and the second server 2000-2 based on the information about each server. The electronic device 1000 can receive information about the global model refined based on the gradient by sending the gradient to the determined server.

[0283] For example, the electronic device 1000 may determine a server that is physically close to the electronic device 1000 as a server to which the gradient is to be transmitted among the identified first server 2000 - 1 and second server 2000 - 2 .

[0284] The electronic device 1000 may determine a server, which includes a global model having the same or similar characteristics as the local model, as a server to which the gradient is to be transmitted among the identified first server 2000-1 and second server 2000-2.

[0285] The global model of the server according to an embodiment of the present disclosure can be used to appropriately perform various operations under specific conditions according to various characteristics or locations of the server, such as, for example but not limited to, item recommendation, object recognition, voice recognition, etc. For example, the global model of a server located in the United Kingdom can be a model generated in consideration of the cultural characteristics of the United Kingdom, and the electronic device 1000 located in the United Kingdom can be appropriately used to perform item recommendation.

[0286] exist Fig.14 In the example shown, the electronic device 1000 may send the gradient to the first server 2000-1 when determining that it is appropriate to refine the local model via the global model of the first server 2000-1 of the first server 2000-1 and the second server 2000-2. Similar to the electronic device 1000, the first external device 3000-1 may send the gradient to the first server 2000-1 when determining that it is appropriate to refine the local model via the global model of the first server 2000-1.

[0287] On the other hand, the second external device 3000 - 2, when determining that it is appropriate to refine the local model via the global model of the second server 2000 - 2 , may transmit the gradient to the second server 2000 - 2 .

[0288] Fig.15 is a diagram illustrating an example system for refining local models and global models via a relay device according to an embodiment of the present disclosure.

[0289] like Fig.15 As shown, the system according to an embodiment of the present disclosure may include a first electronic device 1000 - 2 , a second electronic device 1000 - 3 , a third electronic device 1000 - 4 , and a server 2000 .

[0290] Fig.15 The server 2000 can correspond to Figure 1 The first electronic device 1000-2, the second electronic device 1000-3, and the third electronic device 1000-4 according to an embodiment of the present disclosure may provide various services (e.g., item recommendation, object recognition, and voice recognition) to a user using a local model provided in each of the first electronic device 1000-2, the second electronic device 1000-3, and the third electronic device 1000-4.

[0291] Reference Fig.15, the first electronic device 1000-2, the second electronic device 1000-3 and the third electronic device 1000-4 may be classified into a group 1000-1. For example, the first electronic device 1000-2, the second electronic device 1000-3 and the third electronic device 1000-4 may be classified into one group 1000-1 by being determined as devices used by the same user in the user's house. The first electronic device 1000-2, the second electronic device 1000-3 and the third electronic device 1000-4 may be classified into one group 1000-1, for example, based on various criteria such as whether they are geographically close to each other and whether they are used by the same account. The second electronic device 1000-3 and the third electronic device 1000-4 according to an embodiment of the present disclosure may determine a relay device, through which the gradient of the local model of each of the second electronic device 1000-3 and the third electronic device 1000-4 in the device of the group 1000-1 to which each device belongs is sent to the server 2000. The relay device according to an embodiment of the present disclosure may send the gradient of the local model refined according to the change of the context information to the server 2000 instead of the second electronic device 1000-3 and the third electronic device 1000-4. In addition, the relay device may receive the gradient of the global model from the server 2000 when sending the gradient of the refined local model, and send the received gradient of the global model to the second electronic device 1000-3 and the third electronic device 1000-4.

[0292] According to an embodiment of the present disclosure, the second electronic device 1000-3 and the third electronic device 1000-4 may determine the first electronic device 1000-2 as a relay device among the devices included in the group 1000-1.

[0293] The first electronic device 1000-2 determined as a relay device according to an embodiment of the present disclosure may be a device having better performance or network environment than the second electronic device 1000-3 and the third electronic device 1000-4. For example, the first electronic device 1000-2 may be a smart phone having better performance than the second electronic device 1000-3 and the third electronic device 1000-4. The second electronic device 1000-3 and the third electronic device 1000-4 may be, for example but not limited to, home appliances (e.g., robot cleaners, refrigerators, or smart televisions (TVs)) having lower performance than smart phones.

[0294] Therefore, when the gradient generated by each device is sent to the server 2000 through the first electronic device 1000-2, information about the refined global model can be obtained faster than when the gradient is sent directly from the second electronic device 1000-3 and the third electronic device 1000-4 to the server 2000.

[0295] According to an embodiment of the present disclosure, the first electronic device 1000-2 may directly process the gradients received from the second electronic device 1000-3 and the third electronic device 1000-4 via, for example, an edge computing method. For example, the first electronic device 1000-2 may refine the local model of the first electronic device 1000-2 based on the gradients received from the second electronic device 1000-3 or the third electronic device 1000-4, and send the gradients of the refined local model to the second electronic device 1000-3 or the third electronic device 1000-4. The first electronic device 1000-2 may send the gradients of the refined local model to the server 2000, and receive the gradients of the global model from the server 2000. The first electronic device 1000-2 may refine the local model again using the gradients of the received global model.

[0296] The first electronic device 1000 - 2 according to an embodiment of the present disclosure may transmit at least one of a gradient received from the second electronic device 1000 - 3 and the third electronic device 1000 - 4 or a gradient generated based on a local model of the first electronic device 1000 - 2 to the server 2000 .

[0297] For example, when the user's behavior pattern changes, the operation modes of the first electronic device 1000-2, the second electronic device 1000-3, and the third electronic device 1000-4 are likely to change. Therefore, when the local model is refined in the first electronic device 1000-2 in response to the change of the information about the user's behavior pattern in the context information according to an embodiment of the present disclosure, the local models of the second electronic device 1000-3 and the third electronic device 1000-4 can be refined, and a gradient can be generated by each device.

[0298] According to an embodiment of the present disclosure, when a gradient is received from at least one of the second electronic device 1000-3 or the third electronic device 1000-4 within a predetermined time after the gradient is generated based on the local model of the first electronic device 1000-2, the first electronic device 1000-2 can send the received gradient together with the gradient of the first electronic device 1000-2 to the server 2000.

[0299] The server 2000 according to an embodiment of the present disclosure may refine the global model based on at least one gradient received from the first electronic device 1000-2. The server 2000 according to an embodiment of the present disclosure may transmit information about the refined global model to the first electronic device 1000-2.

[0300] According to an embodiment of the present disclosure, the first electronic device 1000-2 may transmit information about the global model received from the server 2000 to the second electronic device 1000-3 and the third electronic device 1000-4. Each of the first electronic device 1000-2, the second electronic device 1000-3 and the third electronic device 1000-4 may refine the local model stored therein based on the information about the global model received from the server 2000.

[0301] Fig.16 is a signal flow diagram illustrating an example method of refining a global model and a local model according to a gradient according to an embodiment of the present disclosure.

[0302] Reference Fig.16 , the global model and the local model can be refined based on their gradients.

[0303] In the following, as referenced Figure 1 As described above, the gradient of the current global model stored in the server 2000 can be referred to as a first gradient, the gradient of the local model sent from the electronic device 1000 to the server 2000 can be referred to as a second gradient, and the gradient for refining the current global model that can be obtained based on the first gradient and the second gradient can be referred to as a third gradient.

[0304] According to an embodiment of the present disclosure, the electronic device 1000 may refine the local model upon detecting a change in context information of the electronic device 1000. When refining the local model based on the changed context information, the electronic device 1000 may generate a second gradient of the refined local model in operation 1601.

[0305] The second gradient according to an embodiment of the present disclosure may include information for refining the local model, and may include, for example, weight values ​​applicable to each node and information related to the loss function obtained to derive a deviation value (e.g., a slope value related to a point at which the loss function value is minimized).

[0306] In operation 1602, the electronic device 1000 may transmit the second gradient of the refined local model to the server 2000. The server 2000 according to an embodiment of the present disclosure may refine the global model based on the second gradient received from the electronic device 1000.

[0307] In operation 1603, the server 2000 may obtain a first gradient of the global model stored in the current server 2000 before refining the global model. In operation 1604, the server 2000 may generate a third gradient based on the first gradient and the second gradient. The third gradient according to an embodiment of the present disclosure may include at least one representative value of corresponding information between the information related to the loss function included in the first gradient and the information related to the loss function included in the second gradient. The representative value according to an embodiment of the present disclosure may be obtained based on an arithmetic mean, a weighted mean, or a simple addition value between the values ​​included in each gradient.

[0308] In operation 1605, the server 2000 may transmit the third gradient to the electronic device 1000. In operation 1607, the server 2000 may refine the global model based on the third gradient. For example, when the bias value and the weight value applicable to each node configuring the global model are modified based on the information related to the loss function included in the third gradient, the global model may be refined.

[0309] In operation 1606, the electronic device 1000 may refine the local model based on the third gradient received from the server 2000. The refinement of the local model in operation 1606 may be performed in the same manner as the operations of generating the third gradient in operation 1604 and refining the global model based on the third gradient in operation 1607.

[0310] For example, the local model can be refined when the bias value and the weight value applicable to each node configuring the local model are modified so that prediction information corresponding to at least one representative value of corresponding information between the information included in the second gradient and the information included in the third gradient is output by the local model.

[0311] According to an embodiment of the present disclosure, at least one representative value may be an arithmetic mean and a weighted mean between the values ​​included in the second gradient and the third gradient. In addition, the weight value applied to each value in the weighted mean may be determined as a value trained so that the operation according to the local model is suitable for the current context of the electronic device 1000. For example, the weight value applied to the value included in the gradient of the local model among the weight values ​​applied to the information in the weighted mean may be determined based on the context information of the electronic device 1000.

[0312] Fig.17 is a signal flow chart illustrating an example method of refining a global model, a first local model of the electronic device 1000 , and a second local model of the external device 3000 according to a gradient according to an embodiment of the present disclosure.

[0313] Reference Fig.17, the electronic device 1000 may refine the first partial model in response to a change in the context information of the electronic device 1000 , and the external device 3000 may refine the second partial model in response to a change in the context information of the external device 3000 .

[0314] Fig.17 The server 2000 can refine the global model based on the gradients of the refined local models received from the plurality of devices. The server 2000 can send the gradients of the refined global model to each device that sent the gradients of the local model to the server 2000.

[0315] In the following, as referenced Figure 1 and 16 As described above, the gradient of the current global model stored in the server 2000 can be referred to as a first gradient, the gradient of the local model sent from the external device 3000 to the server 2000 can be referred to as a second gradient, and the gradient for refining the current global model that can be obtained based on the first gradient and the second gradient can be referred to as a third gradient.

[0316] The gradient of the local model sent from the electronic device 1000 to the server 2000 may be referred to as the fourth gradient. When the gradient of the global model when the local model is refined is the third gradient, the gradient for refining the current global model obtained based on the third gradient and the fourth gradient may be referred to as the fifth gradient.

[0317] According to an embodiment of the present disclosure, Fig.16 In the refinement of the local model in operation 1701, the external device 3000 may refine the second local model of the external device 3000 upon detecting a change in the context information of the external device 3000. When refining the second local model based on the changed context information, the external device 3000 may generate a second gradient of the refined second local model in operation 1701.

[0318] In operation 1702, the external device 3000 may transmit the second gradient of the refined second local model to the server 2000. The server 2000 according to an embodiment of the present disclosure may refine the global model based on the second gradient received from the external device 3000.

[0319] In operation 1703, before refining the global model, the server 2000 may obtain a first gradient of the global model stored in the current server 2000. The first gradient may include information that is pre-generated when the global model is refined and has been stored in the server 2000.

[0320] In operation 1704 , the server 2000 may generate a third gradient based on the first gradient and the second gradient.

[0321] In operation 1705, the server 2000 may transmit the third gradient to the external device 3000. In operation 1707, the server 2000 may refine the global model according to the third gradient.

[0322] In operation 1706, the external device 3000 may refine the second local model based on the second gradient and the third gradient received from the server 2000. The refinement of the second local model in operation 1706 may be performed in the same manner as the operation of refining the global model according to the third gradient and the first gradient generated in operation 1704.

[0323] In operation 1708, when a change in the context information of the electronic device 1000 is detected, the electronic device 1000 may refine the first local model of the electronic device 1000 in the same manner as the refinement of the second local model in operation 1701. When refining the first local model based on the changed context information, the electronic device 1000 may generate a fourth gradient of the refined first local model in operation 1708.

[0324] In operation 1709, the electronic device 1000 may transmit the refined fourth gradient of the first local model to the server 2000. After refining the global model according to the third gradient in operation 1707, the fourth gradient may be received from the server 2000. The server 2000 according to an embodiment of the present disclosure may refine the global model refined in operation 1707 again based on the fourth gradient received from the electronic device 1000.

[0325] Therefore, the global model according to an embodiment of the present disclosure may be refined based on multiple gradients of local models refined by multiple devices.

[0326] In operation 1710, the server 2000 may obtain a third gradient of the current global model before refining the global model. In operation 1711, the server 2000 may generate a fifth gradient based on the third gradient and the fourth gradient. The fifth gradient according to an embodiment of the present disclosure may include at least one representative value of corresponding information between the third gradient and the fourth gradient. The representative value according to an embodiment of the present disclosure may be obtained based on an arithmetic mean, a weighted mean, or a simple addition value between the values ​​included in each gradient.

[0327] In operation 1712, the server 2000 may transmit the fifth gradient to the electronic device 1000. In operation 1714, the server 2000 may refine the global model based on the fifth gradient.

[0328] In operation 1713, the electronic device 1000 may refine the first partial model based on the fourth gradient and the fifth gradient received from the server 2000. The refinement of the first partial model in operation 1713 may be performed in the same manner as the refinement of the second partial model in operation 1706.

[0329] Fig.18 is a signal flow diagram illustrating an example method of refining a global model, a first local model of an electronic device, and a second local model of an external device according to a gradient according to an embodiment of the present disclosure.

[0330] Fig.18 The operation corresponds in part to Fig.17 operation, but with Fig.17 The difference between the methods is that the first local model, the second local model and the global model can be Fig.18 The example embodiments shown are refined.

[0331] According to an embodiment of the present disclosure, the external device 3000 may detect a change in the context information of the external device 3000 in a manner similar to Fig.16 and 17 The external device 3000 may refine the second partial model of the external device 3000 in a manner similar to the refinement of the partial model in the context information. When refining the second partial model based on the changed context information, the external device 3000 may generate a second gradient of the refined second partial model in operation 1801.

[0332] In operation 1802, the external device 3000 may transmit the second gradient of the refined second local model to the server 2000. The server 2000 according to an embodiment of the present disclosure may refine the global model based on the second gradient received from the external device 3000.

[0333] In operation 1803 , before refining the global model, the server 2000 may obtain a first gradient of the global model stored in the current server 2000 .

[0334] For example, the first gradient of the global model can be obtained based on the difference between the weight values ​​of the global model at fixed time intervals. The fixed time interval can be set to the interval between the most recent time point at which communication is performed between the server 2000 and the electronic device 1000 or other external device to refine the local model or the global model and the time point before the most recent time point.

[0335] In operation 1804 , the server 2000 may transmit the first gradient obtained in operation 1803 to the external device 3000 .

[0336] Then, in operation 1805, the server 2000 may refine the global model based on the second gradient. For example, the server 2000 may generate a third gradient based on the first gradient and the second gradient, and refine the global model according to the third gradient. For example, the third gradient may include at least one representative value of the corresponding information between the first gradient and the second gradient. However, the embodiments of the present disclosure are not limited thereto, and the server 2000 may refine the global model based on the second gradient according to various methods.

[0337] In operation 1806, the external device 3000 may refine the second local model based on the second gradient and the first gradient received from the server 2000. The refinement of the second local model in operation 1806 may be performed in the same or similar manner as generating the third gradient in operation 1805 and refining the global model according to the generated third gradient and the first gradient.

[0338] According to an embodiment of the present disclosure, a third gradient for refining the global model or the second local model based on the first gradient and the second gradient can be generated based on a representative value suitable for refining the global model or the second local model. For example, when the representative value suitable for refining the global model can be a simple addition value, the global model can be refined by generating a third gradient based on a value obtained by simply adding the corresponding information between the first gradient and the second gradient.

[0339] The embodiments of the present disclosure are not limited thereto, and the global model and the second local model may be refined according to various methods based on information obtained according to various methods from the first gradient and the second gradient.

[0340] In operation 1807, the electronic device 1000 may refine the first local model of the electronic device 1000 upon detecting a change in the context information of the electronic device 1000, similar to the refinement of the second local model in operation 1801. In operation 1807, when refining the first local model based on the changed context information, the electronic device 1000 may generate a fourth gradient of the refined first local model.

[0341] In operation 1808, the electronic device 1000 may transmit the fourth gradient of the refined first local model to the server 2000. After refining the global model in operation 1805, the server 2000 may receive the fourth gradient. The server 2000 according to an embodiment of the present disclosure may refine the global model refined in operation 1805 again based on the fourth gradient received from the electronic device 1000.

[0342] Therefore, the global model according to an embodiment of the present disclosure may be refined based on multiple gradients of local models refined by multiple devices.

[0343] In operation 1809 , before refining the global model, the server 2000 may obtain the fifth gradient of the current global model.

[0344] According to an embodiment of the present disclosure, considering that after the global model is refined in operation 1805, the global model may be refined due to various factors, a fifth gradient of the global model at the current time point may be obtained in operation 1809. For example, the fifth gradient may be obtained based on the difference between the weight values ​​of the global model at fixed time intervals. The fixed time interval may be set as an interval between the most recent time point at which communication is performed between the server 2000 and the external device 1000 or other external device to refine the local model or the global model and a time point before the most recent time point.

[0345] In operation 1810 , the server 2000 may transmit the fifth gradient obtained in operation 1809 to the electronic device 1000 , so that the first local model of the electronic device 1000 is refined according to the current global model.

[0346] In operation 1811, the electronic device 1000 may refine the first local model based on the fourth gradient and the fifth gradient received from the server 2000. The refinement of the first local model in operation 1811 may be performed in the same or similar manner as generating a sixth gradient and refining a global model according to the generated sixth gradient in the following operation 1812.

[0347] In operation 1812, the server 2000 may refine the global model based on the fourth gradient. For example, the server 2000 may generate a sixth gradient based on the fourth gradient and the fifth gradient, and refine the global model according to the generated sixth gradient. The sixth gradient according to an embodiment of the present disclosure may include at least one representative value of the corresponding information between the fourth gradient and the fifth gradient. The representative value according to an embodiment of the present disclosure may be obtained based on the arithmetic mean, weighted mean, or simple addition value between the values ​​included in each gradient. The embodiments of the present disclosure are not limited thereto, and the server 2000 may refine the global model according to various methods based on the fourth gradient.

[0348] According to an embodiment of the present disclosure, a sixth gradient for refining a global model or a first local model based on the fourth gradient and the fifth gradient can be generated based on a representative value suitable for refining the global model or the first local model. For example, when the representative value suitable for refining the global model is a simple addition value, the global model can be refined by generating a sixth gradient based on a value obtained by simply adding corresponding information between the fourth gradient and the fifth gradient.

[0349] The embodiments of the present disclosure are not limited thereto, and the global model and the first local model may be refined according to various methods based on information obtained according to various methods by the fourth gradient and the fifth gradient.

[0350] According to an embodiment of the present disclosure, because the gradient of the local model refined based on context information is sent to the server instead of information about the context of the electronic device (which may include sensitive information of the user), the risk of personal information being leaked during the sending process can be eliminated and / or reduced.

[0351] According to an embodiment of the present disclosure, since a gradient having a smaller data size than information about a context is sent to a server instead of information about a context to refine an AI model of an electronic device, a local model can be refined more quickly.

[0352] Embodiments of the present disclosure may also be implemented in the form of a recording medium, which includes instructions run by a computer, such as a program module run by a computer. A non-transitory computer-readable recording medium may be any available medium accessible by a computer, and examples thereof include all volatile and non-volatile media and detachable and non-detachable media. In addition, examples of computer-readable recording media may include computer storage media and communication media. Examples of computer storage media include all volatile and non-volatile media and detachable and non-detachable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules, and other data. Communication media typically include computer-readable instructions, data structures, or program modules, and include any information transmission media.

[0353] Furthermore, in the present disclosure, the term “unit” may include a hardware component such as a processor or a circuit and / or a software component executed by a hardware component such as a processor.

[0354] The above description of the present disclosure is provided for illustration, and those skilled in the art will appreciate that various changes in form and detail may be readily made without departing from the features and scope of the present disclosure, including the appended claims. Therefore, the above exemplary embodiments of the present disclosure are merely examples in all respects and are not limited. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

[0355] The scope of the present disclosure includes the appended claims, but is not limited to the specification, and all changes or modifications within the scope of the appended claims and their equivalents will be construed as being included in the scope of the present disclosure.

Claims

1. A method for refining a local artificial intelligence (AI) model performed by an electronic device, the method comprising: detecting a change in information about a context of the electronic device determined to be a relay device among the electronic devices classified into a group by being determined to belong to the same user, wherein the information about the context as a basis for refining a first partial AI model stored in the electronic device includes personal information of a user of the electronic device; Determining a first local gradient for refining the first local AI model based on the change in information about the context; In response to detecting a change in the information about the context of the electronic device, receiving, from at least one of the other electronic devices included in the group, a second local gradient for refining a second local model stored on at least one of the other electronic devices; Refining the first local AI model based on the first local gradient and the second local gradient; Sending at least one of the first local gradient and the second local gradient to a server having a global AI model; receiving, from the server, a global gradient of a global AI model refined based on at least one of the first local gradient and the second local gradient; and Based on the received global gradient of the refined global AI model, the refined first local AI model is further refined.

2. The method according to claim 1, wherein: The determination of the first local gradient includes: Based on the change in the information about the context, obtaining prediction information and observation information output by the first local AI model; obtaining a loss function indicating a difference between the observed information and the predicted information; and The first local gradient is determined as information related to a point where a value of the loss function is minimum.

3. The method according to claim 1, wherein: The global gradient of the global AI model includes information related to a point at which a value of a loss function of the global AI model used to refine the global AI model is minimized.

4. The method according to claim 3, wherein: Refining the first local AI model based on the global gradient includes: obtaining at least one representative value of corresponding information between information related to the first loss function included in the global gradient and information related to the second loss function included in the first local gradient of the refined first local AI model; and Based on the at least one representative value, the first local AI model is refined.

5. The method according to claim 4, wherein: At least one value of the at least one representative value includes at least one of an arithmetic mean or a weighted mean with respect to the corresponding information.

6. The method according to claim 5, wherein: The weighted average is obtained based on at least one weight value being applied to the corresponding information, and Based on the information about the context of the electronic device, a weight value applied to the information related to the loss function included in the gradient of the refined first partial AI model among the at least one weight value is determined.

7. The method of claim 1, comprising adding noise to at least one of the first local gradient and the second local gradient prior to transmission to the server.

8. The method according to claim 1, wherein: The first local gradient is transmitted to the server based on at least one of an operation of adding noise and an operation of performing encoding.

9. The method according to claim 1, wherein: The sending includes: identifying a relay device for transmitting the first local gradient of the first local AI model from the electronic device; and sending the first local gradient of the first local AI model to the server via the relay device, The relay device is configured to: receive at least one local gradient from at least one electronic device, and send the received at least one local gradient to the server.

10. The method according to claim 1, comprising: In response to the presence of a difference between the gradient of the current global AI model and the gradient of the global AI model previously sent from the server to the electronic device, wherein the difference is equal to or greater than a reference value, receiving the gradient of the current global AI model from the server; and Based on the received gradients, the first local AI model is refined.

11. The method according to claim 1, wherein: Based on detecting that information about the context of the electronic device has changed, the first local AI model is refined based on at least one of the following: the electronic device is currently in an idle state, the memory space of the electronic device for refining the first local AI model is sufficient, the battery of the electronic device is currently charging, or the current time is midnight.

12. An electronic device configured to refine a local artificial intelligence (AI) model, the electronic device comprising: A memory, storing a first local AI model; at least one processor configured to: detect a change in information about a context of the electronic device determined to be a relay device among electronic devices that are classified into a group by being determined to belong to the same user, wherein the information about the context as a basis for refining the first local AI model includes personal information of a user of the electronic device, determine a first local gradient for refining the first local AI model based on the change in the information about the context, and refine the first local AI model based on the determined gradient; and a communicator including a communication circuit, configured to: in response to detecting a change in the information about the context of the electronic device, receive a second gradient for refining a second local model stored on at least one of the other electronic devices included in the group, send at least one of the first local gradient and the second local gradient to a server having a global AI model, and receive from the server a global gradient of the global AI model refined based on at least one of the first local gradient and the second local gradient, The at least one processor is configured to refine the first local AI model based on the first local gradient and the second local gradient, and further refine the refined first local AI model based on the received global gradient of the refined global AI model.

13. The electronic device of claim 12, wherein the at least one processor is configured to: Based on the change of information about the context, prediction information and observation information output by the first local AI model are obtained, a loss function indicating the difference between the observation information and the prediction information is obtained, and the first local gradient is determined to be information related to the point where the value of the loss function is minimum.

14. The electronic device according to claim 12, wherein: The at least one processor is configured to, in response to the presence of a difference between the gradient of the current global AI model and the gradient of the global AI model previously sent from the server to the electronic device, wherein the difference is equal to or greater than a reference value, control the communicator to receive the gradient of the current global AI model from the server, and refine the first local AI model based on the received gradient.

15. A computer program product comprising a non-transitory computer-readable recording medium having recorded thereon instructions adapted to execute the method of claim 1.

Images