Device and method for data collection, machine learning model training, and machine learning model deployment

Abstract

This electronic device comprises: a communication circuit communicating with a network; at least one processor including a processing circuit; and a memory including one or more storage media storing instructions, wherein the instructions, when executed by the at least one processor, may instruct the electronic device to: transmit, to a server, reference data indicating a state of the electronic device; receive, from the server, information regarding a machine learning model corresponding to the state of the electronic device, the model being selected from among a plurality of candidate machine learning models on the basis of the reference data; and operate on the basis of an output of the machine learning model.

Description

Device and method for collecting data, training a machine learning model, and deploying a machine learning model

[0001] Below, a technique for collecting data, training a machine learning model, and deploying the trained machine learning model is disclosed.

[0002] Machine Learning Operations (MLOps) can refer to a service that aims to support the tools required throughout the entire machine learning lifecycle. To enable data engineers and / or machine learning experts to efficiently develop services that apply machine learning, MLOps can be supported in the form of a web or application.

[0003] The above information may be provided as background information to aid in understanding this document. None of the above is claimed to be prior art related to this document or can be used to determine prior art.

[0004] An electronic device includes a communication circuit that performs communication with a network; at least one processor including a processing circuit; and a memory including one or more storage media that store instructions, wherein when the instructions are executed by the at least one processor, the electronic device can transmit reference data indicating a state of the electronic device to a server, receive information about a machine learning model corresponding to the state of the electronic device selected based on the reference data from among a plurality of candidate machine learning models from the server, and operate based on an output of the machine learning model.

[0005] An electronic device comprises: a communication circuit for performing communication with a network; at least one processor including a processing circuit; and a memory including one or more storage media for storing instructions; wherein, when the instructions are executed by the at least one processor, the electronic device can cause the electronic device to collect data and metadata representing a state of the electronic device while collecting the data, and transmit the data and the metadata to a server, thereby causing the server to train a machine learning model selected from among a plurality of candidate machine learning models based on the metadata using the data.

[0006] The server comprises at least one processor including a processing circuit; and a memory including one or more storage media storing instructions, wherein when the instructions are executed by the at least one processor, the server can cause the server to receive, from a first electronic device, data collected from the first electronic device and metadata indicating that a state of the first electronic device is a first state while the data is being collected, determine a first machine learning model corresponding to the first state from among a plurality of candidate machine learning models based on the received metadata, train the first machine learning model based on at least one of the received data or the received metadata, receive reference data indicating that a state of the second electronic device is a second state, determine a second machine learning model corresponding to the second state from among the plurality of candidate machine learning models based on the reference data, and transmit information about the second machine learning model to the second electronic device.

[0007] A method performed by an electronic device may include: transmitting reference data indicating a state of the electronic device to a server; receiving information about a machine learning model corresponding to the state of the electronic device selected from among a plurality of candidate machine learning models based on the reference data from the server; and operating based on an output of the machine learning model.

[0008] A method performed by an electronic device may include: collecting data and metadata representing a state of the electronic device while collecting the data; and transmitting the data and the metadata to a server, thereby causing the server to train a machine learning model selected from among a plurality of candidate machine learning models based on the metadata using the data.

[0009] A method performed by a server may include: receiving, from a first electronic device, data collected from the first electronic device and metadata indicating that a state of the first electronic device is a first state while collecting the data; determining a first machine learning model corresponding to the first state from among a plurality of candidate machine learning models based on the received metadata; training the first machine learning model based on at least one of the received data or the received metadata; receiving, from the second electronic device, reference data indicating that a state of the second electronic device is a second state; determining a second machine learning model corresponding to the second state from among the plurality of candidate machine learning models based on the reference data; and transmitting information about the second machine learning model to the second electronic device.

[0010] FIG. 1 is a block diagram illustrating an exemplary configuration of an electronic device according to various embodiments.

[0011] FIG. 2 illustrates an example of a system for collecting training data, training a machine learning model, and deploying a machine learning model according to various embodiments.

[0012] FIG. 3 is a system diagram illustrating examples of operations of a first electronic device, a server, and a second electronic device according to various embodiments.

[0013] FIG. 4 is a block diagram illustrating an example configuration of an electronic device for collecting data and metadata according to various embodiments.

[0014] FIGS. 5A, 5B, and 5C are diagrams illustrating examples of forms of collecting data and metadata according to various embodiments.

[0015] FIG. 6 is a flowchart illustrating an example of an operation of an electronic device collecting data and metadata according to various embodiments.

[0016] FIG. 7 is a flowchart illustrating an example of an operation in which a server trains a machine learning model and provides the machine learning model to an electronic device according to various embodiments.

[0017] FIG. 8 is a flowchart illustrating an example of an operation of a server training a machine learning model according to various embodiments.

[0018] FIG. 9 is a block diagram illustrating an example configuration of an electronic device using a machine learning model according to various embodiments.

[0019] FIG. 10 is a flowchart illustrating an example of an operation of an electronic device obtaining an updated machine learning model according to various embodiments.

[0020] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are assigned the same reference numerals regardless of the drawing numbers, and redundant descriptions thereof will be omitted.

[0021] FIG. 1 is a block diagram illustrating an exemplary configuration of an electronic device according to various embodiments.

[0022] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108).

[0023] According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display (160), an audio module (170), a sensor (176), an interface (177), a connection terminal (178), a haptic module (179), a camera (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor (176), the camera (180), or the antenna module (197)) may be integrated into one component (e.g., the display (160)).

[0024] The processor (120) may be implemented as one or more IC (integrated circuit (or circuitry)) chips and may perform various data processing. The processor (120) may include at least one electrical circuit (e.g., processing circuit) and may individually or collectively perform distributed processing (e.g., execution) of instructions (or programs (140), data, etc.) stored in the memory (130). The processor (120) may include a processor assembly including one or more processing circuits. The processor (120) may include any processing circuit that is operative to control the performance and operations of one or more components (e.g., memory (130), display (160), camera (180), communication circuit, and / or sensor (176)) of the electronic device (101).

[0025] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculations, the processor (120) may store commands or data received from other components (e.g., a sensor (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or a secondary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor)) that can operate independently or together therewith. For example, if the electronic device (101) includes a main processor (121) and a secondary processor (123), the secondary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a specified function. The secondary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0026] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display (160), a sensor (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0027] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0028] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0029] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0030] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0031] A display (160) (e.g., a display) can visually provide information to an external device (e.g., a user) of the electronic device (101). The display (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0032] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0033] The sensor (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor (176) may include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor. For example, the sensor (176) may include an inertial measurement unit (IMU).

[0034] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0035] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0036] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0037] The camera (180) can capture still images and moving images. According to one embodiment, the camera (180) can include one or more lenses, one or more image sensors, one or more image signal processors, or one or more flashes.

[0038] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least a part of a power management integrated circuit (PMIC).

[0039] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0040] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication circuits. The communication module (190) may include one or more communication processors (CPs) that operate independently from the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data relation (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0041] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0042] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas by, for example, the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0043] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0044] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0045] According to one embodiment, commands or data may be transmitted or received between an electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199).

[0046] Each of the external electronic devices (102, 104) and the server (108) may be the same type of device as or different from the electronic device (101). According to one embodiment, all or part of the operations executed by the electronic device (101) may be executed by one or more of the external electronic devices (102, 104) or the server (108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of executing the function or service itself or in addition, request one or more external electronic devices to execute at least a part of the function or service. The one or more external electronic devices that receive the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a part of a response to the request.

[0047] FIG. 2 illustrates an example of a system for collecting training data, training a machine learning model, and deploying a machine learning model according to various embodiments.

[0048] According to one embodiment, the system (200) includes a first electronic device (201) (e.g., electronic device (101) of FIG. 1), a server (208) (e.g., server (108) of FIG. 1), and a second electronic device (204) (e.g., electronic device (104) of FIG. 1).

[0049] According to one embodiment, a first electronic device (201) may collect data regarding training data for training a machine learning model. The first electronic device (201) may refer to a device for collecting training data for a machine learning model. The first electronic device (201) may refer to a device selected for collecting training data from among devices that receive information regarding a machine learning model from a server (208). The server (208) may collect training data for training the machine learning model based on the first electronic device (201). According to various embodiments, the server (208) may provide information regarding a machine learning model to a plurality of devices, and select at least one device (e.g., the first electronic device (201)) among the plurality of devices as a device for collecting training data. The collection of data through the first electronic device (201) will be described in more detail below with reference to FIGS. 4 to 6 .

[0050] According to one embodiment, the server (208) can train a machine learning model using the collected training data. The server (208) can deploy the trained machine learning model to the second electronic device (204). In various embodiments of the present disclosure, the machine learning model may refer to a model generated and / or trained to output output data from input data. For example, the machine learning model may include a model based on a neural network, a tree, and / or a language model.

[0051] According to one embodiment, a server (208) can manage (e.g., create, store, train, and distribute) multiple machine learning models corresponding to a single function (or a single service). As will be described in more detail later with reference to FIGS. 7 to 10 , the multiple machine learning models corresponding to a single function can have inputs and outputs that include item values ​​of common items. For example, the inputs of the multiple machine learning models can all include RSRP values ​​of neighboring cells during a predetermined time interval. The outputs of the multiple machine learning models can include information regarding the occurrence of a raid link failure (RLF) (e.g., a score indicating the likelihood of an RLF occurring).

[0052] Multiple machine learning models corresponding to a single function may individually correspond to states of multiple electronic devices. Each machine learning model may refer to a model created and / or trained to output results tailored (e.g., specialized) to the device in the corresponding state. For example, the server (208) may train a machine learning model corresponding to the state of the first electronic device (201) using training data collected based on the first electronic device (201). The server (208) may distribute (e.g., transmit, provide) information about the trained machine learning model to the second electronic device (204). The training and distribution of the machine learning model via the server (208) will be described in more detail below with reference to FIGS. 7 and 8 .

[0053] Although not explicitly illustrated in FIG. 2, the server (208) may include a data server, a machine learning tools server, and a model server. The data server may support functions for managing data collected from the first electronic device. The data server is described in more detail below in FIG. 4. The machine learning model tools server may support functions for training a machine learning model. The model server may support functions for managing (e.g., updating, distributing) trained machine learning models. The model server is described in more detail below in FIG. 9.

[0054] According to one embodiment, the second electronic device (204) may operate using information regarding a machine learning model received from the server (208). For example, the second electronic device (204) may perform an inference operation of the machine learning model using information regarding the machine learning model received from the server (208). The second electronic device (204) may operate based on the output of the machine learning model obtained through the inference operation of the machine learning model. The distribution and use of the machine learning model through the second electronic device (204) will be described in more detail below with reference to FIGS. 9 and 10 .

[0055] In FIG. 2, the first electronic device (201) is depicted as a separate device from the second electronic device (204), but is not limited thereto. For example, the first electronic device (201) may be the same device as the second electronic device (204).

[0056] FIG. 3 is a system diagram illustrating examples of operations of a first electronic device, a server, and a second electronic device according to various embodiments.

[0057] In operation (S310), a first electronic device (301) according to an embodiment (e.g., the electronic device (101) of FIG. 1, the first electronic device (201) of FIG. 2) may collect data and metadata. The metadata may include data indicating the state of the first electronic device while collecting data. The data may include values ​​of various parameters generated from operations (e.g., communication) of the first electronic device (301). Examples of the data are described in more detail below in FIG. 4.

[0058] In various embodiments of the present disclosure, the state of an electronic device (e.g., the first electronic device (301), the second electronic device (304)) may indicate an operating environment of the electronic device. The state of the electronic device may be defined by information regarding hardware and / or software of the electronic device, or information regarding communication between the electronic device and a network. For example, the state of the electronic device may include at least one of information regarding subscriber identity module information (e.g., universal subscriber identity module information (USIM) information) used by the electronic device for communication, information regarding a chipset included in the electronic device, information regarding a location of the electronic device, information regarding a service provider providing communication to the electronic device, information regarding a standard type of radio access technology (RAT) used by the electronic device for communication, information regarding a version of software stored in the electronic device, or information regarding a product model of the electronic device. The state of the electronic device may include information regarding a device identifier of the electronic device or a user identifier of the electronic device. The state of an electronic device according to various embodiments of the present disclosure is not limited to the examples described above, and may include other parameters indicating an operating environment and / or a communication environment of the electronic device.

[0059] According to one embodiment, subscriber identity module information may include information indicating subscriber identity module information used to perform communication among a plurality of pieces of subscriber identity module information when an electronic device uses (e.g., stores, accesses) a plurality of pieces of subscriber identity module information. For example, an electronic device may communicate with a network. The electronic device may store first subscriber identity module information and second subscriber identity module information. When the electronic device performs communication (e.g., transmits a data packet, receives a data packet) with the network using one of the first subscriber identity module information or the second subscriber identity module information, the electronic device may collect data related to the communication. During the communication, the electronic device may collect, as metadata, information indicating subscriber identity module information used for the communication among the first subscriber identity module information or the second subscriber identity module information.

[0060] According to one embodiment, information about a chipset included in an electronic device may include information about a manufacturer of the chipset used to implement a component (e.g., a processor, an application processor, a communication processor) of the electronic device, information about a product model of the chipset (e.g., a product model identifier of the chipset), and / or information about a part of the chipset (e.g., a type and / or capacity of memory, the number of cores included in a processor).

[0061] In one embodiment, information regarding the location of an electronic device may refer to information regarding the geographical location of the electronic device. For example, information regarding the location of an electronic device may indicate the country (or city, or state) in which the electronic device is located, and / or the time zone in which the electronic device is located.

[0062] In one embodiment, information about a carrier providing communications to an electronic device may include information about a carrier managing a base station that provides a network and connection to the electronic device (e.g., a carrier identifier).

[0063] According to one embodiment, the information about the standard type of the wireless access technology that the electronic device uses for communication may indicate, for example, at least one of a 5G type (also expressed as 'NR type (new-radio type)' in various embodiments of the present disclosure) (e.g., 5G SA type (5th generation standalone), 5G NSA (5th generation non-standalone)), a 4G type (also expressed as 'LTE type (long-term evolution type)' in various embodiments of the present disclosure), an ENDC type (E-UTRAN new radio dual connectivity type), or a 3G type.

[0064] According to one embodiment, information about the version of software stored in the electronic device may mean information indicating the version of software (e.g., firmware) that controls the operation of the electronic device.

[0065] According to one embodiment, information about the product model of the electronic device may include an identifier (e.g., a product model identifier) ​​that distinguishes the product model of the electronic device. For example, the product model identifier of the electronic device may include a Galaxy S21, a Galaxy S22, a Galaxy S23, and / or a Galaxy S24 (or a product model identifier indicating the Galaxy S21, the Galaxy S22, the Galaxy S23, and / or the Galaxy S24).

[0066] For example, the first electronic device (301) may be in a first state while collecting data. The first electronic device (301) may collect metadata indicating that the first electronic device (301) is in the first state while collecting data.

[0067] In operation (S320), the first electronic device (301) according to one embodiment may transmit data and metadata to a server (308) (e.g., server (108) of FIG. 1, server (208) of FIG. 2). The server (308) may receive data and metadata from the first electronic device (301).

[0068] According to one embodiment, the first electronic device (301) may transmit data and metadata to the server (308) based on satisfying a predefined condition. The predefined condition may refer to a trigger condition regarding the transmission of the data and metadata. The predefined condition may, for example, include a predetermined time elapsed since the transmission of the data and metadata (e.g., a period elapsed).

[0069] In operation (S330), the server (308) according to one embodiment may determine a first machine learning model corresponding to the first state from among a plurality of candidate machine learning models based on metadata.

[0070] According to one embodiment, the state of an electronic device may be defined as one of a plurality of states. A plurality of candidate machine learning models may correspond to the plurality of states. For example, the plurality of candidate machine learning models may individually correspond (e.g., correspond one-to-one) to the plurality of states. However, the present invention is not limited thereto, and at least one of the plurality of candidate machine learning models may correspond to two or more states. The server (308) may select a first machine learning model corresponding to the state of the first electronic device (301) (e.g., the first state) from among the plurality of candidate machine learning models.

[0071] In operation S340, the server (308) according to one embodiment may train the first machine learning model based on at least one of data or metadata. For example, the server (308) may obtain a training pair based on at least one of data or metadata. The training pair may include a training input and a ground truth. The training input may include an item value of an item defined in the input of the first machine learning model and may have the format of an input of the first machine learning model. The ground truth may have the format of an output of the first machine learning model. The server (308) may train (e.g., adaptive learning) the first machine learning model by using the obtained training pair as training data of the first machine learning model.

[0072] Although not explicitly shown in FIG. 3, the state of the first electronic device (301) may change from the first state to the second state. For example, a change in the state of an electronic device (e.g., the first electronic device (301), the second electronic device (304)) may include a change in the network operator of the electronic device, a change in the geographical location of the electronic device, a change in the communication using the first SIM information and then a change in the communication using the second SIM information when the electronic device stores multiple SIM information, and / or a version of the software of the electronic device is updated.

[0073] For example, the first electronic device (301) may collect first data and first metadata indicating the first state while the state of the first electronic device (301) is the first state. When the server (308) receives the first data and the first metadata from the first electronic device (301), the server (308) may train a first machine learning model corresponding to the first state based on the first data and / or the first metadata. The state of the first electronic device (301) may change from the first state to a second state. When the first electronic device (301) is in the second state, the first electronic device (301) may collect second data and second metadata indicating the second state. When the server (308) receives the second data and the second metadata from the first electronic device (301), the server (308) may train a second machine learning model corresponding to the second state based on the second data and / or the second metadata.

[0074] In operation (S350), the second electronic device (304) according to one embodiment (e.g., the electronic device (104) of FIG. 1, the second electronic device (204) of FIG. 2) may transmit reference data to the server (308). The server (308) may receive the reference data from the second electronic device (304). The reference data may indicate a state of the second electronic device (304) corresponding to the time at which the reference data is acquired (e.g., generated) or transmitted. For example, the second electronic device (304) may be in a second state at the time at which the reference data is acquired. The reference data may indicate that the second electronic device (304) is in the second state.

[0075] In operation (S360), the server (308) according to one embodiment may determine a second machine learning model corresponding to a state (e.g., a second state) of the second electronic device (304) from among a plurality of candidate machine learning models based on reference data.

[0076] In operation (S370), the server (308) according to one embodiment may transmit information about the second machine learning model to the second electronic device (304). The second electronic device (304) may receive information about the second machine learning model from the server (308). Based on receiving the information about the second machine learning model, the second electronic device (304) may store instructions for an inference operation of the second machine learning model in the memory of the second electronic device (304).

[0077] In operation S380, the second electronic device (304) according to one embodiment may operate based on the output of the second machine learning model. For example, the second electronic device (304) may generate input data of the second machine learning model. The second electronic device (304) may obtain the output of the second machine learning model by applying the input data to the second machine learning model. The second electronic device (304) may control the operation of components (e.g., a processor, an application processor, a communication processor, a memory) of the second electronic device (304) based on the output of the second machine learning model.

[0078] FIG. 4 is a block diagram illustrating an example configuration of an electronic device for collecting data and metadata according to various embodiments.

[0079] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1, the first electronic device (201) of FIG. 2, the first electronic device (301) of FIG. 3) may include an application processor and a communication processor.

[0080] According to one embodiment, the electronic device (401) may acquire communication parameters related to communication collected by the communication processor (420) as at least a portion of the data. The communication parameters may include, for example, whether a radio link failure (RLF) has occurred and information about neighboring cells measured by the electronic device. The information about the neighboring cells may include a physical cell identifier of the neighboring cell, an interference intensity of the neighboring cell, and / or a signal quality of the neighboring cell. The signal quality of the neighboring cell may refer to a signal quality measured using a measurement signal received from the neighboring cell.

[0081] In various embodiments of the present disclosure, a measurement signal may refer to a signal measured by a terminal (e.g., an electronic device (401)) to obtain signal quality to be used for mobility, admission control, or radio resource management (RRM). For example, the measurement signal may be at least one of a synchronization signal (SS) (e.g., an SS block), a beam reference signal (BRS), a beam refinement reference signal (BRRS), a cell-specific reference signal (CRS), a channel state information-reference signal (CSI-RS), or a demodulation-reference signal (DM-RS). According to various embodiments, a base station may transmit not only one type of measurement signal through a cell, but also two or more types of measurement signals.

[0082] In various embodiments of the present disclosure, the signal quality may be, for example, at least one of RSRP (reference signal received power), BRSRP (beam reference signal received power), RSRQ (reference signal received quality), RSSI (received signal strength indicator), SINR (signal to interference and noise ratio), CINR (carrier to interference and noise ratio), SNR (signal to noise ratio), EVM (error vector magnitude), BER (bit error rate), or BLER (block error rate). In addition to the examples described above, it is to be understood that other terms having equivalent technical meanings or other metrics indicating channel quality may be used. Hereinafter, in the present disclosure, high signal quality means a case where a signal quality value related to a signal magnitude is large or a signal quality value related to an error rate is small. A higher signal quality may mean that a smooth wireless communication environment is guaranteed. In addition, an optimal beam may mean a beam with the highest signal quality among beams.

[0083] According to one embodiment, the communication processor (420) of the electronic device (401) may collect, when an RLF occurs, physical cell identifiers of neighboring cells and RSRP values ​​of neighboring cells accumulated during a time period including the time point at which the RLF occurred as data. The collected data may be generated and / or used as training data of a machine learning model that is trained to output output data indicating information about the occurrence of an RLF (e.g., the probability of occurrence of an RLF) from input data indicating characteristics (e.g., distribution characteristics) of RSRP values.

[0084] According to one embodiment, the communication processor (420) of the electronic device (401) may collect communication parameters related to communication as at least a portion of data. The communication processor (420) of the electronic device (401) may transmit the collected communication parameters to the application processor (410). At least one of the communication processor (420) or the application processor (410) of the electronic device (401) may collect information indicating the status of the electronic device (401) as at least a portion of metadata.

[0085] The application processor (410) can receive data and / or metadata collected by the communication processor (420) from the communication processor (420). The application processor (410) can gather the data (or metadata) collected by the application processor (410) and the data (or metadata) collected by the communication processor (420). The application processor (410) can transmit information indicating the status of the collected electronic device (401) together with the collected communication parameters to the server (408).

[0086] In various embodiments of the present disclosure, the communication processor (420) may not support a communication protocol (e.g., http), and the data and / or metadata collected by the communication processor (420) are mainly described as being transmitted to the server (408) via the application processor (410), but are not limited thereto. For example, in the case where a standard is defined in the wireless access technology performed by the communication processor (420), or in the case where a standard is defined according to an agreement between a network provider and a chipset manufacturer of the communication processor (420), the communication processor (420) may transmit the data and / or metadata collected by the communication processor (420) to the server (408) independently of the application processor (410) (e.g., without via the application processor).

[0087] According to one embodiment, the electronic device (401) may collect data related to the application processor (410) based on a data platform application (411) and / or an additional data application (e.g., a first additional data application (412-1), a second additional data application (412-2), a third additional data application (412-3)) among the stored (e.g., installed) applications. The data platform application (411) and the additional data application may be an example or a type of application (e.g., the application (146) of FIG. 1). The application may be a set of instructions executable by the application processor (410) and may be stored in the memory of the electronic device (401) when installed in the electronic device (401). The data platform application (411) and the additional data application will be described in more detail below.

[0088] For example, based on the data platform application (411) being installed in the electronic device (401), the application processor (410) of the electronic device (401) can collect basic data of the application processor (410) as at least a portion of the data based on the data platform application (411).

[0089] According to one embodiment, the data platform application (411) may refer to an application for collecting basic data of the application processor (410). The data platform application (411) may refer to an application that specifies parameters to be collected independently (e.g., irrespective of) from an installed application based on a user's selection. In various embodiments of the present disclosure, data collected based on the data platform application (411) may be expressed as basic data of the application processor (410).

[0090] In one embodiment, the base data may include parameter values ​​collected from all devices utilizing the machine learning model provided by the server (408), or from all devices designated by the server (408) as devices collecting data for training data of the machine learning model. The base data may include data generated from devices utilizing wireless access technology, independent of a specific application. In one embodiment, parameters that are important to the performance of communication and / or operation, or are likely to be utilized as at least a portion of the input data of the machine learning model, may be designated as base data.

[0091] As will be described later, according to one embodiment, the basic data may include basic data of the application processor (410) and basic data of the communication processor (420). The basic data of the application processor (410) may include values ​​of parameters (e.g., basic AP parameters) related to the operation of the application processor (410) collected based on the data platform application (411). As will be described later, the basic data of the communication processor (420) may include values ​​of parameters (e.g., basic CP parameters) related to the operation of the communication processor (420) collected based on the data platform task (421).

[0092] For example, based on an additional data application being installed on the electronic device (401), the application processor (410) of the electronic device (401) may collect additional data of the application processor (410) as at least a portion of the data based on the additional data application.

[0093] According to one embodiment, the additional data application may refer to an application for collecting additional data of the application processor (410). The additional data application may refer to an application installed based on a user's selection, and may refer to an application that specifies parameters to be additionally collected from the basic data of the application processor (410) for training and / or inference of a machine learning model corresponding to the additional data application. The machine learning model corresponding to the additional data application may include a machine learning model required for the operation of the additional data application. In various embodiments of the present disclosure, data collected based on the additional data application may be expressed as additional data of the application processor (410).

[0094] According to one embodiment, the application processor (410) of the electronic device (401) can transmit additional data of the application processor (410) to the server (408) through the data platform application (411). According to one embodiment, the application processor (410) of the electronic device (401) can reduce the risk in the operation of the electronic device (401) by unifying the data collection pipeline by transmitting the additional data of the application processor (410) to the server (408) through the data platform application (411). However, the present invention is not limited thereto, and the application processor (410) of the electronic device (401) can transmit the additional data of the application processor (410) to the server (408) independently from the data platform application (411) (e.g., through the additional data application, as a standalone additional data application).

[0095] Referring to FIG. 4, an additional data application according to one embodiment may include a data software development kit (SDK) (or instructions based on the data SDK) provided by a server (408) or a data platform application (411). When a data SDK is used, redundant development and / or redundant collection between additional data applications may be prevented. The data SDK may support collecting basic data of the application processor (410) when the application processor (410) of the electronic device (401) transmits additional data of the application processor (410) to the server (408) independently from the data platform application (411).

[0096] According to one embodiment, the additional data may include values ​​of parameters collected based on an application (e.g., an additional data application) installed on the electronic device (401) by a user's selection. The additional data may include values ​​of parameters collected based on a task (e.g., an additional data task) of the communication processor (420) that is added (e.g., generated) based on the additional data application or the state of the electronic device (401). As described below, the additional data may include additional data of the application processor (410) and additional data of the communication processor (420). The additional data of the application processor (410) may include values ​​of parameters (e.g., additional AP parameters) related to the operation of the application processor (410) collected based on the additional data application. As described below, the additional data of the communication processor (420) may include values ​​of parameters (e.g., additional CP parameters) related to the operation of the communication processor (420) collected based on the additional data task.

[0097] According to one embodiment, the electronic device (401) may collect data related to the communication processor (420) based on a data platform task (421) and / or an additional data task (e.g., a first additional data task (422-1), a second additional data task (422-2), a third additional data task (422-3)) via the communication processor (420). The data platform task (421) and the additional data task may be an example or a type of task. A task is a set of instructions executable by the communication processor (420), which, when generated, may be stored in the memory of the electronic device (401). The data platform task (421) and the additional data task are described in more detail below.

[0098] According to one embodiment, the application processor (410) and the communication processor (420) of the electronic device (401) may each be implemented as separate chipsets. Since the application processor (410) and the communication processor (420) implemented as separate chipsets can collect different data, the data collection of the communication processor (420) can be performed independently from the data collection of the application processor (410).

[0099] For example, the communication processor (420) of the electronic device (401) may collect basic data of the communication processor (420) based on the data platform task (421). The data platform task (421) may refer to a task for collecting basic data of the communication processor (420). The data platform task (421) may refer to a task that specifies parameters to be collected independently (e.g., regardless) of an installed application based on a user's selection. In various embodiments of the present disclosure, data collected based on the data platform task (421) may be expressed as basic data of the communication processor (420).

[0100] For example, the communication processor (420) of the electronic device (401) may collect additional data of the communication processor (420) based on an additional data task. The additional data task may refer to a task for collecting additional data of the communication processor (420). The additional data task may refer to a task for specifying parameters (e.g., additional CP parameters) to be additionally collected in addition to the basic data of the communication processor (420) for training and / or inference of a machine learning model corresponding to an application (e.g., an additional data application) installed based on a user's selection. For example, when an additional data application is installed in the electronic device (401), the additional data application may generate an additional data task for collecting additional data of the communication processor (420). In various embodiments of the present disclosure, data collected based on the additional data task may be expressed as additional data of the communication processor (420).

[0101] According to one embodiment, the communication processor (420) of the electronic device (401) can transmit basic data of the communication processor (420) and additional data of the communication processor (420) to the application processor (410). The application processor (410) of the electronic device (401) can collect the basic data of the communication processor (420) and the additional data of the communication processor (420) as at least a portion of the data through the data platform application (411). As described above, the data collected by the communication processor (420) can be transmitted to the server (408) through the application processor (410).

[0102] According to various embodiments of the present disclosure, the basic data can be collected independently of the additional data application and / or additional data task, as specified by the data platform application and / or data platform task. The additional data can be collected with parameters specified and / or defined based on the application and / or additional data task. Consequently, through the collection of the basic data and additional data, various parameters can be collected as training data for a machine learning model.

[0103] Table 1 describes examples of the types of parameters and parameter values ​​collected as basic data in a data platform application (411) according to various embodiments.

[0104] Type of parameter valueTimestamplong longrrc_stateenum (RRC_IDLE / RRD_CONNECTED)cell1PhysicalIdunsigned intcell1RSRPintcell1RSRQintcell1primaryCellboolean (True / False)cell1servingCellboolean (True / False)cell1bandwidthint

[0105] In Table 1, Timestamp may mean a timepoint corresponding to basic data (e.g., the time point at which the basic data was collected). rrc_state may mean a radio resource control (RRC) state, and rrc_state may have one of the values ​​of a state in which the electronic device (401) is not connected to a network but is ready to attempt a connection when necessary by monitoring a broadcast channel of the cell (RRC_IDLE), or a state in which the electronic device (401) is actively connected to the network and can transmit and receive data (RRC_CONNECTED). cell1PhysicalId may indicate a physical cell identity (PCI) of a first cell to which the electronic device (401) is connected. cell1RSRP may indicate a power level of a reference signal (e.g., reference signal received power (RSRP)) that the electronic device (401) receives from the first cell. cell1RSRQ may indicate the signal quality (e.g., RSRQ (reference signal received quality)) of the reference signal received by the electronic device (401) from the first cell. cell1primaryCell may indicate whether the first cell is a primary cell. cell1servingCell may indicate whether the first cell is a serving cell. cell1bandwidth may indicate the bandwidth of the first cell.

[0106] In various embodiments of the present disclosure, the electronic device (401) mainly describes, but is not limited to, collecting basic data and / or additional data of the application processor (410) based on the installation of the data platform application (411) and / or the additional data application. For example, the electronic device (401) may receive a configuration regarding data collection from the server (408). The configuration regarding data collection may specify whether to collect data related to the electronic device (401), and, when collecting data related to the electronic device (401), settings regarding an operation for collecting the data (e.g., a collection cycle, a transmission cycle, a trigger condition for data collection and / or data transmission). For example, the electronic device (401) may determine whether to collect data related to the electronic device (401) (e.g., basic data and / or additional data of the application processor (410) and the communication processor (420)) based on the configuration regarding data collection. The electronic device (401) may not collect data related to the electronic device (401) if it is determined not to collect data related to the electronic device (401) (e.g., if the configuration regarding data collection instructs not to collect data of the electronic device (401).

[0107] The electronic device (401) can transmit data and metadata to the server (408). The electronic device (401) can cause the server (408) to train a machine learning model (e.g., the first machine learning model of FIG. 3) selected from among a plurality of candidate machine learning models based on the metadata using the data.

[0108] Referring to FIG. 4, the server (408) may include a data server (450) and one or more collect servers (e.g., a first collect server (430) and a second collect server (440)). Each collect server may collect data by linking with an application installed on the electronic device (401) (e.g., a data platform application (411), an additional data application). According to one embodiment, the collect server may collect data based on a corresponding application. According to one embodiment, the collect server may collect data collected by a device having a corresponding state (e.g., a specific product model of the electronic device (401). The data server (450) may receive data collected by the collect server, and classify and store the collected data based on the format of the received data.

[0109] In FIG. 4, one or more collection servers are depicted as servers other than server (408), but are not limited thereto. For example, at least one of the one or more collection servers may be integrated into the data server (450) and / or server (408).

[0110] FIGS. 5A, 5B, and 5C are diagrams illustrating examples of forms of collecting data and metadata according to various embodiments.

[0111] An electronic device according to one embodiment (e.g., an electronic device (101) of FIG. 1, a first electronic device (201) of FIG. 2, a first electronic device (301) of FIG. 3, and an electronic device (401) of FIG. 4) may collect data and metadata. The electronic device may transmit the collected data and metadata to a server (e.g., a server (108) of FIG. 1, a server (208) of FIG. 2, a server (308) of FIG. 3, and a server (408) of FIG. 4). The data and / or metadata may be transmitted via a file. For example, the data and / or metadata may be transmitted together with information about a time series via a file in a json format or a yaml format, or may be transmitted as a file in a binary file format (e.g., a binary file).

[0112] Referring to FIG. 5a, the electronic device can transmit metadata to the server together with the message body (510a) of the packet when transmitting a file (520a) containing data.

[0113] Referring to FIG. 5b, the electronic device can transmit data and metadata to the server as a single file (520b).

[0114] Referring to FIG. 5c, the electronic device can transmit a file (521c) containing metadata and a file (522c) containing data to the server.

[0115] FIG. 6 is a flowchart illustrating an example of an operation of an electronic device collecting data and metadata according to various embodiments.

[0116] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1, the first electronic device (201) of FIG. 2, the first electronic device (301) of FIG. 3, and the electronic device (401) of FIG. 4) can collect data and metadata.

[0117] In operation (605), the electronic device may obtain settings related to data collection. In various embodiments of the present disclosure, the settings related to data collection may also be expressed as a collect configuration. The electronic device may receive the collection configuration from a server. The electronic device may transmit to the server a list of data platform applications and / or additional data applications installed on the electronic device. The electronic device may receive from the server, based on the list of additional data applications, a collection configuration regarding additional parameters (e.g., additional AP parameters, additional CP parameters) to be collected based on each additional data application. The collection configuration may indicate at least one of a collection cycle, a list of parameters to be collected, or a collect format.

[0118] In one embodiment, the collection settings may specify that data and / or metadata be collected based on a periodic timer. However, this is not limited to the collection settings, and data and / or metadata may be collected when a new additional data application is added (e.g., newly installed) or when a trigger condition is met.

[0119] In operation (610), the electronic device may collect basic data. The basic data may include basic data of the application processor and basic data of the communication processor, as described above. The basic data may include basic parameters (e.g., basic CP parameters, basic AP parameters) already defined based on the collection settings, data platform application, and / or data platform task.

[0120] In operation (615), the electronic device can check whether an additional data application is installed on the electronic device.

[0121] In operation (620), the electronic device may collect additional data based on the additional data application, if the additional data application is installed.

[0122] As described above in FIG. 4, an additional data task may be generated in response to an additional data application. Consequently, the application processor of the electronic device may collect additional data from the application processor based on the additional data application, and the communication processor of the electronic device may collect additional data from the communication processor based on the additional data task. In one embodiment, the electronic device may collect additional data using a registered function (e.g., a callback function).

[0123] In operation (625), the electronic device may skip collecting additional data if no additional data application is installed.

[0124] In operation (630), the electronic device may determine whether to immediately report the collected data to the server. For example, the electronic device may be configured to transmit the collected data and / or metadata immediately (e.g., within a threshold time from the completion of collection).

[0125] In operation (635), the electronic device may transmit the collected data to the server if it decides to immediately report the collected data to the server.

[0126] In operation (640), the electronic device may omit transmitting the collected data to the server if it decides not to immediately report the collected data to the server.

[0127] In operation (645), the electronic device may determine whether a request to stop data collection has been made. The electronic device may determine that a request to stop data collection has been made based on receiving a request to stop data collection from the server. If the electronic device does not receive a request to stop data collection, the electronic device may determine that a request to stop data collection has not been made.

[0128] In operation (650), the electronic device may wait for the next data collection if it determines that the data collection cessation is not requested. If a trigger condition for the next data collection (e.g., the passage of a predetermined time, the addition of a new additional data application) is met, the electronic device may perform operations for data collection (e.g., operations (610, 615, 620, 625, 630, 635, 640, 645)).

[0129] In operation (655), the electronic device may store the collected data if it determines that data collection has been requested to be discontinued. For example, the electronic device may store the collected data in the form of a file or database. The electronic device may transmit the stored data to a server at regular intervals or when a condition (e.g., the electronic device being connected to Wi-Fi) is met. The electronic device may transmit the data directly to the server or to the server via the web.

[0130] In Figure 6, the electronic device primarily describes storing collected data in response to a request to stop data collection, but this is not limited to this. For example, the electronic device may store collected data periodically, independently of the request to stop data collection.

[0131] FIG. 7 is a flowchart illustrating an example of an operation in which a server trains a machine learning model and provides the machine learning model to an electronic device according to various embodiments.

[0132] As described above in FIG. 3, a server according to one embodiment (e.g., server 108 of FIG. 1, server 208 of FIG. 2, server 308 of FIG. 3, server 408 of FIG. 4) may train a machine learning model based on data and / or metadata received from a first electronic device (e.g., electronic device 101 of FIG. 1, first electronic device 201 of FIG. 2, electronic device 301 of FIG. 3, electronic device 401 of FIG. 4) and transmit information about the machine learning model to a second electronic device (e.g., electronic device 104 of FIG. 1, second electronic device 204 of FIG. 2).

[0133] In operation (710), the server may receive data and metadata from the first electronic device. While the first electronic device collects data, the state of the first electronic device may be a first state. The metadata may indicate that the state of the first electronic device is the first state while collecting data.

[0134] In operation (720), the server may determine (e.g., select) a first machine learning model corresponding to a state (e.g., a first state) of the first electronic device from among a plurality of candidate machine learning models based on the received metadata.

[0135] In operation (730), the server may train a first machine learning model based on at least one of the received data or the received metadata.

[0136] A plurality of candidate machine learning models may have inputs and outputs that include item values ​​of a common item. This may refer to models trained with different training data. According to one embodiment, a server may obtain data and metadata collected from a plurality of first electronic devices. At least some of the plurality of first electronic devices may have different states. The server may classify the data and metadata collected from the plurality of first electronic devices based on the state of the first electronic device that collects the data while each piece of data is being collected, based on the metadata.

[0137] For example, data collected by a first electronic device in a first state may be classified as first training data. Data collected by a first electronic device in a second state may be classified as second training data. The server may train a first machine learning model using the first training data collected from the first electronic device in the first state. The server may train a second machine learning model using the second training data collected from the first electronic device in the second state.

[0138] In one embodiment, each machine learning model (e.g., each candidate machine learning model) may be trained using training data collected from a device having a state identical to or similar to the state corresponding to the corresponding machine learning model, among data collected from multiple devices. For example, the first machine learning model may refer to a model trained using training data collected from a device having a state identical to or similar to the first state.

[0139] In one embodiment, each machine learning model (e.g., each candidate machine learning model) may be tagged (e.g., labeled) with information (e.g., metadata) indicating the state of the electronic device to which the machine learning model corresponds. As described in more detail below, when the server transmits information about the machine learning models to the second electronic device, the server may use the information tagged to each candidate machine learning model to determine the machine learning model.

[0140] In operation (740), the server may receive reference data from the second electronic device. The reference data may refer to data indicating the state of the second electronic device. For example, when the second electronic device transmits the reference data, the state of the second electronic device may be the second state, and the reference data may indicate that the state of the second electronic device is the second state.

[0141] In operation (750), the server may determine (e.g., select) a second machine learning model corresponding to a state (e.g., a second state) of the second electronic device from among a plurality of candidate machine learning models based on reference data. At least one of the plurality of candidate machine learning models may be a machine learning model trained using communication parameters collected based on a communication processor of the device. As described above with reference to FIG. 4 , the first electronic device may collect communication parameters as at least a portion of data by the communication processor of the first electronic device. The server may train at least one of the plurality of candidate machine learning models by using the communication parameters collected by the communication processor of the first electronic device as training data.

[0142] In operation (760), the server may transmit information regarding the second machine learning model to the second electronic device. The second electronic device may receive information regarding the second machine learning model from the server. When the second electronic device receives information regarding the second machine learning model from the server, the second electronic device may store instructions corresponding to the inference operation of the second machine learning model. The operations performed by the second electronic device in response to receiving information regarding the machine learning model from the server are described in more detail below with reference to FIGS. 9 and 10 .

[0143] In various embodiments of the present disclosure, metadata is primarily described as being used to select one of multiple candidate machine learning models, but is not limited thereto. For example, at least a portion of the metadata may be used as input to the machine learning model.

[0144] FIG. 8 is a flowchart illustrating an example of an operation of a server training a machine learning model according to various embodiments.

[0145] A server according to one embodiment (e.g., server (108) of FIG. 1, server (208) of FIG. 2, server (308) of FIG. 3, server (408) of FIG. 4) can train a machine learning model based on training data collected based on one or more first electronic devices (e.g., electronic device (101) of FIG. 1, first electronic device (201) of FIG. 2, electronic device (301) of FIG. 3, electronic device (401) of FIG. 4).

[0146] In operation (810), the server may determine a machine learning model to be trained. For example, the server may determine a machine learning model to be trained from among a plurality of candidate machine learning models. For example, the server may collect training data in a plurality of states. Based on whether a learning start condition corresponding to a particular state of the electronic device is met, the server may determine a machine learning model corresponding to a particular state as the machine learning model to be trained.

[0147] For example, a learning start condition may include that the number of new data (e.g., training data) newly collected by an electronic device in a specific state is greater than or equal to a predetermined number. New data may refer to training data collected after the previous training of the machine learning model and not yet used for training. Existing data may refer to data already used for training the machine learning model. In response to the fulfillment of a learning start condition corresponding to a specific state, the new data and / or existing data used for training the machine learning model may refer to data collected by an electronic device in the specific state. For example, if a learning start condition corresponding to a standard type being a 5G type is fulfilled, the server may train the machine learning model based on at least a portion of the new data and / or at least a portion of the existing data collected by an electronic device in which the standard type is a 5G type.

[0148] For example, a training start condition may include the current time being a predetermined time (e.g., midnight). In other words, training of a machine learning model may be performed periodically, according to a predetermined cycle (e.g., every 24 hours).

[0149] Table 2 describes the learning start conditions of the machine learning model and examples of learning target data for each learning start condition.

[0150] Condition for starting learning Learning target data Product model is SM-S921N More than 10,000 new data New data and existing data Product model is SM-S926N More than 10,000 new data New data and existing data Product model is SM-S928N More than 10,000 new data New data and existing data Product model is SM-G991N More than 100,000 new data New data Product model is SM-G996N More than 100,000 new data New data Product model is SM-G998N More than 100,000 new data New data Standard type is 5G SA type Daily midnight data and existing data Standard type is 5G type Daily midnight data and existing data

[0151] The target data for learning can refer to the range of data to be used for training a machine learning model when the corresponding learning start conditions are met. For example, if the target data for learning is new data and existing data, the server can train the machine learning model using both the new and existing data. If the target data for learning is new data, the server can train the machine learning model using only the new data (e.g., excluding existing data).

[0152] In operation (820), the server may determine the values ​​of hyperparameters to be used for training. Hyperparameters to be used for training may include, for example, at least one of a learning rate, a batch size, a number of epochs, a regularization parameter, or a dropout rate.

[0153] In operation (830), the server may train the determined machine learning model based on the determined hyperparameter values. The server may train the machine learning model using the determined hyperparameter values ​​and the learning target data based on the learning start condition being met. The server may use only data collected in a state corresponding to the learning start condition for training the machine learning model, and may restrict the use of data collected in other states for training the machine learning model.

[0154] In one embodiment, the server may analyze the collected data and determine that the properties of data collected before a specific point in time have significantly changed from the properties of data collected after the specific point in time. The properties of the data may, for example, include distribution properties. If the server determines that the properties of the data have significantly changed from the specific point in time, it may use only the data collected after the specific point in time for training the machine learning model. The server may exclude data collected before the specific point in time from training the machine learning model. Excluding data collected before the specific point in time from training the machine learning model may also be referred to as data deprecation.

[0155] In operation (840), the server can store the trained machine learning model.

[0156] Although not explicitly illustrated in FIG. 8, a server according to various embodiments of the present disclosure may, after storing a trained machine learning model, wait for a learning start condition to be satisfied. Based on whether another learning start condition is satisfied, the server may determine a machine learning model corresponding to another learning start condition, determine the values ​​of hyperparameters, train the machine learning model, and store the trained machine learning model. In other words, in response to the satisfaction of the learning start condition, the server may perform operations (810, 820, 830, 840) again.

[0157] FIG. 9 is a block diagram illustrating an example configuration of an electronic device using a machine learning model according to various embodiments.

[0158] According to one embodiment, an electronic device (904) (e.g., an electronic device (104) of FIG. 1, a second electronic device (204) of FIG. 2, a second electronic device (304) of FIG. 3) may receive information about a machine learning model corresponding to a state of the electronic device (904) from a server (908) (e.g., a server (108) of FIG. 1, a server (208) of FIG. 2, a server (308) of FIG. 3, a server (408) of FIG. 4)) and operate based on an output of the machine learning model.

[0159] According to one embodiment, the electronic device (904) may transmit reference data indicating the status of the electronic device (904) to the server (908). For example, the electronic device (904) may manage one or more machine learning models used in the electronic device (904) based on a model platform application (911). The electronic device (904) may manage a machine learning model corresponding to an additional model application (e.g., a first additional model application (912-1), a second additional model application (912-2), a third additional model application (912-3)) installed in the electronic device (904) through the model platform application (911).

[0160] According to one embodiment, the server (908) and / or the electronic device (904) may perform distribution of at least one model through the model platform application (911) or the additional model application. The electronic device (904) may determine, through the model platform application (911), whether a machine learning model corresponding to the additional model application installed on the electronic device (904) requires an update. A model requiring an update may refer to a model that has been trained additionally to a machine learning model stored on the electronic device (904). If the electronic device (904) determines that a model requiring an update exists, the electronic device (904) may transmit a model update request to the server (908) that includes reference data indicating a state of the electronic device (904) (e.g., information regarding a product model of the electronic device (904), information regarding a version of software stored on the electronic device, USIM information of the electronic device, or information regarding a location of the electronic device). The electronic device (904) can check the need for an update of the machine learning model periodically based on the model platform application (911) or in response to a command received from the server (908).

[0161] Referring to FIG. 9, an additional model application according to one embodiment may include a model software development kit (SDK) (or instructions based on the model SDK) provided by a server (908) or a model platform application (911). For example, the electronic device (904) may utilize the SDK as a standalone or directly implement it to receive distribution of a model directly from the server (908) through the additional model application. The electronic device (904) may receive distribution of a model through the model platform application (911). When the electronic device (904) receives distribution of a model through the model platform application (911), the operational risk of the electronic device (904) may be reduced through unification of the model distribution pipeline. When the model SDK is utilized, redundant development and / or redundant collection between additional model applications may be prevented.

[0162] According to one embodiment, the server (908) may select a machine learning model corresponding to the state of the electronic device (904) from among a plurality of candidate machine learning models based on reference data (e.g., reference data included in a model update request). The server (908) may transmit information about the selected machine learning model to the electronic device (904). The electronic device (904) may receive information about the machine learning model from the server (908). The electronic device (904) may store instructions corresponding to the inference operation of the machine learning model.

[0163] According to one embodiment, the electronic device (904) may include an application processor (910) and a communication processor (920). The electronic device (904) may store instructions corresponding to the inference operation of the machine learning model, depending on whether the machine learning model is associated with the application processor (910) or the communication processor (920).

[0164] For example, based on the machine learning model being associated with the application processor (910), instructions corresponding to the inference operation of the machine learning model may be stored as at least a portion of an application executable by the application processor (910). According to various embodiments of the present disclosure, the machine learning model being associated with the application processor (910) may mean that the output of the machine learning model is used to control the operation of the application processor (910). For example, the electronic device (904) may control the operation of the application processor (910) based on the output of the machine learning model associated with the application processor (910). According to one embodiment, storing as at least a portion of the application may include storing in at least a portion of a memory accessible by the application processor (910). Instructions stored as at least a portion of the application may be executed by the application processor (910).

[0165] Referring to FIG. 9, the electronic device (904) can store instructions corresponding to the inference operation of the machine learning model as an additional model application in at least a portion of the memory accessible by the application processor (910).

[0166] According to one embodiment, the electronic device (904) may store instructions corresponding to the inference operation of the machine learning model as a task of the communication processor (920), based on the machine learning model being associated with the communication processor (920). According to various embodiments of the present disclosure, the machine learning model being associated with the communication processor (920) may mean that the output of the machine learning model is used to control the operation of the communication processor (920). For example, the electronic device (904) may control the operation of the communication processor (920) based on the output of the machine learning model being associated with the communication processor (920). According to one embodiment, storing as at least a part of the task may include storing in at least a part of a memory accessible by the communication processor (920). The instructions stored as at least a part of the task may be executed by the communication processor (920).

[0167] Referring to FIG. 9, the electronic device (904) may store instructions corresponding to the inference operation of the machine learning model as a model platform task (921) or an additional model task (e.g., a first additional model task (922-1), a second additional model task (922-2), a third additional model task (922-3)) in at least a portion of a memory accessible by the communication processor (920). According to one embodiment, the communication processor (920) of the electronic device (904) may obtain instructions corresponding to the inference operation of the machine learning model from the model platform task (921) based on the application processor (910) of the electronic device (904) executing the model platform application (911). The communication processor (920) of the electronic device (904) may generate and / or store instructions corresponding to the inference operation of the machine learning model as an additional model task based on the model platform task (921). Instructions stored as additional model tasks can be executed by the communication processor (920) by being stored in the internal memory of the communication processor (920) (or the chipset of the communication processor (920)).

[0168] Similar to the transmission of data collected by the communication processor (920) described in FIG. 4, in various embodiments of the present disclosure, the communication processor (920) may not support a communication protocol (e.g., http), and instructions corresponding to the inference operation of the machine learning model related to the communication processor (920) are mainly described as being transmitted to the communication processor (920) via the application processor (910), but are not limited thereto. For example, in the case where a wireless access technology performed by the communication processor (920) is defined as a standard, or in the case where a specification is defined according to an agreement between a network provider and a chipset manufacturer of the communication processor (920), the communication processor (920) may receive instructions from the server (908) independently of the application processor (910) (e.g., without going through the application processor (910).

[0169] In one embodiment, the electronic device (904) primarily describes receiving information about a machine learning model from a server (908) online through the server (908), but is not limited thereto. For example, a file containing information about a machine learning model may be stored in a specific location of the electronic device (904), and the machine learning model may be distributed in such a way that a changed file is detected based on an application (e.g., a model platform application (911), an additional model application).

[0170] Referring to FIG. 9, the server (908) may include a model server (950) and one or more deploy servers (e.g., a first deploy server (930) and a second deploy server (940). According to one embodiment, each deploy server may be linked to an application installed on an electronic device (904) (e.g., a model platform application (911), an additional model application) to deploy a machine learning model. The deploy server may receive a model update request from the electronic device (904). The deploy server may determine a machine learning model that requires an update based on reference data included in the model update request. According to one embodiment, model deployment based on a model update request may be required because using a model trained through data collected by devices in the same state as the state of the electronic device may exhibit higher performance.

[0171] In one embodiment, the distribution server may deploy a machine learning model corresponding to a corresponding application (e.g., an additional model application). In another embodiment, the distribution server may deploy a machine learning model corresponding to a corresponding state (e.g., a specific product model of an electronic device (904)). The model server (950) may transmit information about the machine learning model to the distribution server.

[0172] In FIG. 9, one or more distribution servers are depicted as servers other than server (908), but are not limited thereto. For example, at least one of the one or more distribution servers may be integrated into the model server (950) and / or server (908).

[0173] According to one embodiment, the state of the electronic device (904) can be changed from a first state to a second state. The electronic device (904) can transmit first reference data representing the first state to the server (908) while the state of the electronic device (904) is in the first state. The server (908) can determine a first machine learning model corresponding to the first state based on the first reference data from among a plurality of candidate machine learning models. The server (908) can transmit information about the first machine learning model to the electronic device (904). The electronic device (904) can receive information about the first machine learning model corresponding to the first state from the server (908). The electronic device (904) can operate based on the output of the first machine learning model while the state of the electronic device (904) is in the first state.

[0174] The electronic device (904) may transmit second reference data indicating the second state to the server (908) after the state of the electronic device (904) changes from the first state to the second state. The server (908) may select a second machine learning model corresponding to the second state from among a plurality of candidate machine learning models based on the second reference data. The server (908) may transmit information regarding the second machine learning model to the electronic device (904). The electronic device (904) may receive information regarding the second machine learning model corresponding to the second state from the server (908). The electronic device (904) may replace the first machine learning model with the second machine learning model. For example, the electronic device (904) may delete information regarding the first machine learning model from its memory based on receiving information regarding the second machine learning model. The electronic device (904) may operate based on the output of the second machine learning model.

[0175] FIG. 10 is a flowchart illustrating an example of an operation of an electronic device obtaining an updated machine learning model according to various embodiments.

[0176] According to one embodiment, an electronic device (e.g., an electronic device (104) of FIG. 1, a second electronic device (204) of FIG. 2, a second electronic device (304) of FIG. 3, and an electronic device (904) of FIG. 9) may receive information about a machine learning model corresponding to a state of the electronic device from a server (e.g., a server (108) of FIG. 1, a server (208) of FIG. 2, a server (308) of FIG. 3, a server (408) of FIG. 4, and a server (908) of FIG. 9)) and update the machine learning model using the received information about the machine learning model.

[0177] According to one embodiment, the electronic device may change a machine learning model that requires modification among the machine learning models based on a predetermined cycle (e.g., periodically), when a new additional model application is installed on the electronic device, or in response to a model change request (e.g., a model deployment request) received from a server. Below, examples of operations by which the electronic device changes a machine learning model to an additional machine learning model are described.

[0178] In operation (1010), the electronic device can identify additional model applications installed on the electronic device. For example, the electronic device can obtain a list of additional model applications through a model platform application (e.g., model platform application (911) of FIG. 9). The electronic device can identify a machine learning model corresponding to each additional model application.

[0179] In operation (1020), the electronic device may determine whether a machine learning model requires modification. For example, the electronic device may determine whether a machine learning model requiring modification exists among one or more machine learning models corresponding to the additional model application(s).

[0180] For example, the electronic device may determine whether a change (e.g., update) of the machine learning model is required based on a result of comparing version information of the machine learning model stored in the electronic device with version information of the machine learning model stored on a server.

[0181] For example, an electronic device may determine whether a change (e.g., replacement) of a machine learning model is necessary based on a comparison of a state of the electronic device corresponding to a machine learning model stored in the electronic device with a current state of the electronic device. The electronic device may determine that a change of the machine learning model is necessary based on the fact that the machine learning model stored in the electronic device corresponds to a first state and the state of the electronic device is a second state different from the first state. The electronic device may determine that a change of the machine learning model is not necessary based on the fact that the machine learning model stored in the electronic device corresponds to the first state and the state of the electronic device is the first state.

[0182] According to one embodiment, the electronic device first determines the need for a change in the machine learning model based on the state of the electronic device, and if it is determined that the change in the machine learning model is not necessary based on the state of the electronic device, the electronic device can determine again the need for a change in the machine learning model based on version information of the machine learning model.

[0183] In operation (1030), if the electronic device determines that a machine learning model requiring modification exists, the electronic device may modify the machine learning model. For example, the electronic device may transmit information regarding the machine learning model requiring modification (e.g., an identifier of the machine learning model, a model update request, and reference data) to a server. Based on the information regarding the machine learning model, the server may transmit information regarding an additional machine learning model to the electronic device. The additional machine learning model may be a new version of the machine learning model or a machine learning model corresponding to the current state of the electronic device. The electronic device may receive information regarding the additional machine learning model.

[0184] According to one embodiment, an electronic device may use information about an additional machine learning model to replace a machine learning model with the additional machine learning model. Replacing the machine learning model with the additional machine learning model may include deleting instructions corresponding to the inference operation of the machine learning model stored in the electronic device and storing instructions corresponding to the inference operation of the additional machine learning model.

[0185] In one embodiment, the electronic device can notify an additional model application (or a module associated with the additional model application) of a change in the machine learning model.

[0186] In FIG. 10, the main description is given of changing a machine learning model stored in an electronic device into an additional machine learning model, but is not limited thereto. According to one embodiment, the electronic device may not store a machine learning model corresponding to a newly installed additional model application. The electronic device may request information about a machine learning model corresponding to a newly added additional model application from a server. The server may receive information indicating the state of the electronic device (e.g., reference data) and information about the additional model application (or the machine learning model corresponding to the additional model application), and transmit information about the machine learning model corresponding to the state of the electronic device and the additional model application to the electronic device.

[0187] In various embodiments of the present disclosure, operations (1010, 1020, 1030, 1040) may be performed based on, but not limited to, a model platform application installed on an electronic device. At least some of operations (1010, 1020, 1030, 1040) may be performed based on an additional model application.

[0188] An electronic device (104; 204; 304; 904) includes a communication circuit for performing communication with a network; at least one processor including a processing circuit; and a memory including one or more storage media for storing instructions, wherein when the instructions are executed by the at least one processor, the electronic device (104; 204; 304; 904) can transmit reference data indicating a state of the electronic device (104; 204; 304; 904) to a server, receive information about a machine learning model corresponding to the state of the electronic device (104; 204; 304; 904) selected based on the reference data from among a plurality of candidate machine learning models from the server, and operate based on an output of the machine learning model.

[0189] The status of the electronic device (104; 204; 304; 904) may include subscriber identification module information used by the electronic device (104; 204; 304; 904) for the communication, information about a chipset included in the electronic device (104; 204; 304; 904), information about a location of the electronic device (104; 204; 304; 904), information about a carrier providing communication to the electronic device (104; 204; 304; 904), information about a standard type of radio access technology (RAT) used by the electronic device (104; 204; 304; 904) for the communication, information about a version of software stored in the electronic device (104; 204; 304; 904), or information about the electronic It may include at least one piece of information about the product model of the device (104; 204; 304; 904).

[0190] When the instructions are executed by the at least one processor, the electronic device (104; 204; 304; 904) is configured to transmit first reference data representing the first state to a server while the state of the electronic device (104; 204; 304; 904) is in the first state, receive information about a first machine learning model corresponding to the first state selected based on the first reference data from among a plurality of candidate machine learning models from the server, operate based on an output of the first machine learning model, and, after the state of the electronic device (104; 204; 304; 904) changes from the first state to the second state, transmit second reference data representing the second state to the server, receive information about a second machine learning model corresponding to the second state selected based on the second reference data from among the plurality of candidate machine learning models, and replace the first machine learning model with the second machine learning model. Can be.

[0191] The instructions, when executed by the at least one processor, may cause the electronic device (104; 204; 304; 904) to delete information about a first machine learning model from the memory based on receiving information about a second machine learning model, and to operate based on an output of the second machine learning model.

[0192] The above machine learning model may include a machine learning model trained using training data collected based on a device having a state that is the same as or similar to the state of the electronic device (104; 204; 304; 904).

[0193] The plurality of candidate machine learning models have inputs and outputs including item values ​​of common items, and each of the plurality of candidate machine learning models can be trained using training data collected by a device having a state that is the same as or similar to a corresponding state among data collected based on the plurality of devices.

[0194] At least one of the plurality of candidate machine learning models can be trained using communication parameters collected based on a communication processor (CP) (420) of the device (101; 201; 301; 401).

[0195] The at least one processor may include an application processor (AP) (910) and a communication processor (CP) (920), and the instructions, when executed by the at least one processor, may cause the electronic device (104; 204; 304; 904) to store instructions corresponding to an inference operation of the machine learning model as at least a part of an application executable by the application processor (910), based on the machine learning model being related to the application processor (910), and to store instructions corresponding to an inference operation of the machine learning model as a task of the communication processor (920), based on the machine learning model being related to the communication processor (920).

[0196] An electronic device (101; 201; 301; 401) comprises: a communication circuit for performing communication with a network; at least one processor (120) including a processing circuit; and a memory (130) including one or more storage media storing instructions, wherein when the instructions are executed by the at least one processor (120), the electronic device (101; 201; 301; 401) collects data and metadata representing a state of the electronic device (101; 201; 301; 401) while collecting the data, and transmits the data and the metadata to a server (108; 208; 308; 408; 908), thereby causing the server (108; 208; 308; 408; 908) to train a machine learning model selected from among a plurality of candidate machine learning models based on the metadata using the data.

[0197] The state of the electronic device (101; 201; 301; 401) may include at least one of subscriber identification module information used by the electronic device (101; 201; 301; 401) for the communication, information about a chipset included in the electronic device (101; 201; 301; 401), information about a location of the electronic device (101; 201; 301; 401), information about a business operator providing the communication, information about a standard type of wireless access technology used by the electronic device (101; 201; 301; 401) for the communication, information about a version of software stored in the electronic device (101; 201; 301; 401), or information about a product model of the electronic device (101; 201; 301; 401). Can be.

[0198] The at least one processor (120) includes an application processor (AP) (410) and a communication processor (CP) (420), and when the instructions are executed by the at least one processor (120), the electronic device (101; 201; 301; 401) is caused to cause the communication processor (420) to collect communication parameters related to the communication as at least a part of the data, the communication processor (420) to transmit the collected communication parameters to the application processor (410), and at least one of the communication processor (420) or the application processor (410) collects information representing the state of the electronic device (101; 201; 301; 401) as at least a part of the metadata, and transmits the collected information representing the state of the electronic device (101; 201; 301; 401) together with the collected communication parameters to the server (108; You can send it to 208; 308; 408; 908).

[0199] The at least one processor (120) may include an application processor (410), and when the instructions are executed by the at least one processor (120), the application processor (410) may cause the data platform application (411) to be installed in the electronic device (101; 201; 301; 401), to collect basic data of the application processor (410) based on the data platform application as at least a part of the data, and the application processor (410) may cause the additional data application (412-1; 412-2; 412-3) to be installed in the electronic device (101; 201; 301; 401), to collect additional data of the application processor (410) based on the additional data application (412-1; 412-2; 412-3).

[0200] The at least one processor (120) includes an application processor (410) and a communication processor (420), and when the instructions are executed by the at least one processor (120), the communication processor (420) collects basic data of the communication processor (420) based on a data platform task (421), causes the communication processor (420) to collect additional data of the communication processor (420) based on an additional data task (422-1; 422-2; 422-3), causes the communication processor (420) to transmit the basic data of the communication processor (420) and the additional data of the communication processor (420) to the application processor (410), and causes the application processor (410) to transmit the basic data of the communication processor (420) and the additional data of the communication processor (420) through a data platform application (411). It may be possible to collect at least some of the above data.

[0201] A server (108; 208; 308; 408; 908) comprises at least one processor comprising a processing circuit; And a memory including one or more storage media storing instructions, wherein when the instructions are executed by the at least one processor, the server (108; 208; 308; 408; 908) receives, from the first electronic device (101; 201; 301; 401), data collected in the first electronic device (101; 201; 301; 401) and metadata indicating that the state of the first electronic device (101; 201; 301; 401) is a first state while collecting the data, determines a first machine learning model corresponding to the first state among a plurality of candidate machine learning models based on the received metadata, trains the first machine learning model based on at least one of the received data or the received metadata, and from the second electronic device (104; 204; 304; 904), The device (104; 204; 304; 904) may receive reference data indicating that the state is a second state, determine a second machine learning model corresponding to the second state among the plurality of candidate machine learning models based on the reference data, and transmit information about the second machine learning model to the second electronic device (104; 204; 304; 904).

[0202] The status of the electronic device may include at least one of subscriber identification module information used by the electronic device for communication, information about a chipset included in the electronic device, information about a location of the electronic device, information about a carrier providing communication to the electronic device, information about a standard type of wireless access technology used by the electronic device for communication, information about a version of software stored in the electronic device, or information about a product model of the electronic device.

[0203] The instructions, when executed by the at least one processor, cause the server (108; 208; 308; 408; 908) to receive, from the first electronic device (101; 201; 301; 401), communication parameters collected by the communication processor (420) of the first electronic device (101; 201; 301; 401) as at least a portion of the data, and to receive, from the first electronic device (101; 201; 301; 401), communication parameters collected by at least one of the communication processor (420) of the first electronic device (101; 201; 301; 401) or the application processor (410) of the first electronic device (101; 201; 301; 401). 401) can be received as at least part of the metadata.

[0204] The data includes basic data of an application processor (410) of the first electronic device (101; 201; 301; 401) and additional data of the application processor (410) of the first electronic device (101; 201; 301; 401), and the basic data of the application processor (410) is collected by the application processor (410) of the first electronic device (101; 201; 301; 401) based on a data platform application (411) installed in the first electronic device (101; 201; 301; 401), and the additional data of the application processor (410) is collected by an additional data application (412-1; 412-2; 412-3) of the first electronic device (101; 201; Based on what is installed in the first electronic device (101; 201; 301; 401), the additional data application (412-1; 412-2; 412-3) can be collected by the application processor (410) of the first electronic device (101; 201; 301; 401).

[0205] The data includes basic data of the communication processor (420) of the first electronic device (101; 201; 301; 401) and additional data of the communication processor (420) of the first electronic device (101; 201; 301; 401), and the basic data of the communication processor (420) is collected by the communication processor (420) of the first electronic device (101; 201; 301; 401) based on a data platform task (421), and the additional data of the communication processor (420) is collected by the communication processor (420) of the first electronic device (101; 201; 301; 401) based on an additional data task (422-1; 422-2; 422-3), and the basic data of the communication processor (420) The data and the additional data of the communication processor (420) are transmitted from the communication processor (420) of the first electronic device (101; 201; 301; 401) to the application processor (410) of the first electronic device (101; 201; 301; 401), and can be collected as at least a part of the data through a data platform application (411) by the application processor (410) of the first electronic device (101; 201; 301; 401).

[0206] The above first machine learning model can be trained using training data collected based on a device having a state identical to or similar to the first state.

[0207] The plurality of candidate machine learning models have inputs and outputs including item values ​​of common items, and each of the plurality of candidate machine learning models can be trained using training data collected by a device having a state that is the same as or similar to a corresponding state among data collected based on the plurality of devices.

[0208] Electronic devices according to various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments of this document are not limited to the aforementioned devices.

[0209] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0210] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0211] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0212] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0213] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0214] The embodiments described above may be implemented using hardware components, software components, and / or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and software applications running on the OS. Furthermore, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone; however, one of ordinary skill in the art will recognize that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing unit may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

[0215] Software may include computer programs, codes, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may independently or collectively command the processing device. The software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, or computer storage medium or device for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed over networked computer systems and stored or executed in a distributed manner. The software and data may be stored on a computer-readable recording medium.

[0216] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination, and the program commands recorded on the medium may be those specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands such as ROMs, RAMs, and flash memories. Examples of program commands include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc.

[0217] The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiment, and vice versa.

Claims

1. In electronic devices (104; 204; 304; 904), A communication circuit that performs communication with a network; At least one processor comprising a processing circuit; and A memory comprising one or more storage media for storing instructions, When the above instructions are executed by the at least one processor, the electronic device (104; 204; 304; 904) causes: Transmit reference data indicating the status of the above electronic device (104; 204; 304; 904) to the server, Receive information about a machine learning model corresponding to the state of the electronic device (104; 204; 304; 904) selected from among a plurality of candidate machine learning models from the server based on the reference data, Action based on the output of the above machine learning model To do, Electronic devices (104; 204; 304; 904).

2. In paragraph 1, The above state of the above electronic device (104; 204; 304; 904) is, Information about the subscriber identification module used by the electronic device (104; 204; 304; 904) for the communication, information about the chipset included in the electronic device (104; 204; 304; 904), information about the location of the electronic device (104; 204; 304; 904), information about the operator providing communication to the electronic device (104; 204; 304; 904), information about the standard type of radio access technology (RAT) used by the electronic device (104; 204; 304; 904) for the communication, information about the version of the software stored in the electronic device (104; 204; 304; 904), or information about the product model of the electronic device (104; 204; 304; 904). Containing at least one of the information, Electronic devices (104; 204; 304; 904).

3. In any one of paragraphs 1 and 2, When the above instructions are executed by the at least one processor, the electronic device (104; 204; 304; 904) causes: While the state of the electronic device (104; 204; 304; 904) is the first state, first reference data indicating the first state is transmitted to the server, Receive information about a first machine learning model corresponding to the first state selected based on the first reference data from among a plurality of candidate machine learning models from the server, It operates based on the output of the first machine learning model, After the state of the electronic device (104; 204; 304; 904) changes from the first state to the second state, second reference data indicating the second state is transmitted to the server, Receive information about a second machine learning model corresponding to the second state selected based on the second reference data from among the plurality of candidate machine learning models from the server, Replace the above first machine learning model with the above second machine learning model. To do, Electronic devices (104; 204; 304; 904).

4. In any one of paragraphs 1 to 3, The above instructions, when executed by the at least one processor, cause the electronic device (104; 204; 304; 904) to: Based on receiving information about the second machine learning model, information about the first machine learning model is deleted from the memory, Action based on the output of the second machine learning model To do, Electronic devices (104; 204; 304; 904).

5. In any one of paragraphs 1 to 4, The above machine learning model is, A machine learning model trained using training data collected based on a device having a state identical to or similar to the state of the electronic device (104; 204; 304; 904), Electronic devices (104; 204; 304; 904).

6. In any one of paragraphs 1 to 5, The above multiple candidate machine learning models are: Having inputs and outputs that contain the item values ​​of common items, Each of the above multiple candidate machine learning models, Trained using training data collected by a device having a state identical to or similar to the corresponding state among data collected based on multiple devices. Electronic devices (104; 204; 304; 904).

7. In any one of paragraphs 1 to 6, At least one of the above multiple candidate machine learning models, Trained using communication parameters collected based on the communication processor (CP) (420) of the device (101; 201; 301; 401). Electronic devices (104; 204; 304; 904).

8. In any one of paragraphs 1 to 7, At least one processor, It includes an application processor (AP) (910) and a communication processor (CP) (920), The above instructions, when executed by the at least one processor, cause the electronic device (104; 204; 304; 904) to: Based on the machine learning model being related to the application processor (910), storing instructions corresponding to the inference operation of the machine learning model as at least a part of an application executable by the application processor (910), Based on the machine learning model being related to the communication processor (920), instructions corresponding to the inference operation of the machine learning model are stored as tasks of the communication processor (920). To do, Electronic devices (104; 204; 304; 904).

9. In a method performed by a server (108; 208; 308; 408; 908), An operation of receiving, from a first electronic device (101; 201; 301; 401), data collected from said first electronic device (101; 201; 301; 401) and metadata indicating that the state of said first electronic device (101; 201; 301; 401) is a first state while collecting said data; An operation of determining a first machine learning model corresponding to the first state among a plurality of candidate machine learning models based on the received metadata; An operation of training the first machine learning model based on at least one of the received data or the received metadata; An operation of receiving reference data from a second electronic device (104; 204; 304; 904) indicating that the state of the second electronic device (104; 204; 304; 904) is a second state; An operation of determining a second machine learning model corresponding to the second state among the plurality of candidate machine learning models based on the reference data; and An operation of transmitting information about the second machine learning model to the second electronic device (104; 204; 304; 904), method.

10. In paragraph 9, The status of the electronic device is, The electronic device includes at least one of subscriber identification module information used for communication, information about a chipset included in the electronic device, information about a location of the electronic device, information about a business operator providing communication to the electronic device, information about a standard type of wireless access technology used for communication by the electronic device, information about a version of software stored in the electronic device, or information about a product model of the electronic device. method.

11. In any one of paragraphs 9 to 10, The action of receiving the above metadata is: An operation of receiving communication parameters collected by a communication processor (420) of the first electronic device (101; 201; 301; 401) from the first electronic device (101; 201; 301; 401) as at least a part of the data; and An operation of receiving information representing a state of the first electronic device (101; 201; 301; 401) collected by at least one of the communication processor (420) of the first electronic device (101; 201; 301; 401) or the application processor (410) of the first electronic device (101; 201; 301; 401) from the first electronic device (101; 201; 301; 401) as at least a part of the metadata, method.

12. In any one of paragraphs 9 to 11, The above data is, Contains basic data of the application processor (410) of the first electronic device (101; 201; 301; 401) and additional data of the application processor (410) of the first electronic device (101; 201; 301; 401), The basic data of the above application processor (410) is: Based on the data platform application (411) being installed in the first electronic device (101; 201; 301; 401), the data platform application is collected by the application processor (410) of the first electronic device (101; 201; 301; 401), The additional data of the above application processor (410) is: Based on the additional data application (412-1; 412-2; 412-3) being installed in the first electronic device (101; 201; 301; 401), the additional data application (412-1; 412-2; 412-3) is collected by the application processor (410) of the first electronic device (101; 201; 301; 401). method.

13. In any one of paragraphs 9 to 12, The above data is, Contains basic data of the communication processor (420) of the first electronic device (101; 201; 301; 401) and additional data of the communication processor (420) of the first electronic device (101; 201; 301; 401), The basic data of the above communication processor (420) is: Collected by the communication processor (420) of the first electronic device (101; 201; 301; 401) based on the data platform task (421), The additional data of the above communication processor (420) is, Collected by the communication processor (420) of the first electronic device (101; 201; 301; 401) based on an additional data task (422-1; 422-2; 422-3), The basic data of the communication processor (420) and the additional data of the communication processor (420) are Transmitted from the communication processor (420) of the first electronic device (101; 201; 301; 401) to the application processor (410) of the first electronic device (101; 201; 301; 401), By the application processor (410) of the first electronic device (101; 201; 301; 401), at least a portion of the data is collected through the data platform application (411). Server (108; 208; 308; 408; 908).

14. In any one of paragraphs 9 to 13, The above first machine learning model is, Trained using training data collected based on a device having a state identical to or similar to the first state, Server (108; 208; 308; 408; 908).

15. In any one of paragraphs 9 to 14, The above multiple candidate machine learning models are: Having inputs and outputs that contain the item values ​​of common items, Each of the above multiple candidate machine learning models, Trained using training data collected by a device having a state identical to or similar to the corresponding state among data collected based on multiple devices. Server (108; 208; 308; 408; 908).

Images