Electronic device, method, and non-transitory storage medium for generating personalized metadata

Abstract

An electronic device is disclosed. This electronic device may comprise: a display; a memory storing instructions; and at least one processor including processing circuitry. The instructions, when executed individually or collectively by the at least one processor, may instruct the electronic device to: display an execution screen of an application on the display; display content selected on the basis of a user input on the execution screen of the application; acquire information inputted to the application during a preset time or during a time in which at least a part of the content is displayed on the execution screen of the application; generate personalized metadata for the content on the basis of the acquired information; and map the personalized metadata to the content and store same in the electronic device.

Description

Electronic device, method, and non-transient storage medium for generating personalized metadata

[0001] The present disclosure relates to an electronic device, a method, and a non-transient storage medium for generating personalized metadata.

[0002] Electronic devices can provide various content, such as images and videos, to users using a display. In addition, electronic devices can provide various functions related to the content by utilizing the metadata of the content.

[0003] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to the present disclosure.

[0004] An electronic device according to one embodiment includes at least one processor comprising a display, a memory for storing instructions, and a processing circuit, wherein when the instructions are executed individually or collectively by the at least one processor, the electronic device may display an execution screen of an application on the display, display content selected based on user input on the execution screen of the application, acquire information input to the application for a preset time or for a time during which at least a part of the content is displayed on the execution screen of the application, generate personalized metadata for the content based on the acquired information, and map the personalized metadata to the content and store it in the electronic device.

[0005] A method for generating personalized metadata for an electronic device including a display according to one embodiment may include: displaying an execution screen of an application on the display; displaying content selected based on user input on the execution screen of the application; acquiring information input to the application for a preset time or for a time during which at least a portion of the content is displayed on the execution screen of the application; generating personalized metadata for the content based on the acquired information; and mapping the personalized metadata to the content and storing it in the electronic device.

[0006] In a non-transient computer-readable medium storing computer instructions that cause an electronic device to perform an operation when executed by at least one processor of an electronic device including a display according to one embodiment, the operation may include an operation of displaying an execution screen of an application on the display; an operation of displaying content selected based on user input on the execution screen of the application; an operation of acquiring information input to the application for a preset time or for a time during which at least a portion of the content is displayed on the execution screen of the application; an operation of generating personalized metadata for the content based on the acquired information; and an operation of mapping the personalized metadata to the content and storing it in the electronic device.

[0007] In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0008] FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments.

[0009] FIGS. 2A and 2B illustrate an example of a block diagram of an electronic device according to one embodiment.

[0010] FIG. 3 is a flowchart illustrating operations for an electronic device to generate personalized metadata according to one embodiment.

[0011] FIG. 4 is a flowchart illustrating operations for displaying content on an application execution screen by an electronic device according to one embodiment.

[0012] FIG. 5a illustrates an example of an operation in which an electronic device according to one embodiment displays content on the execution screen of a social networking service (SNS) application.

[0013] FIG. 5b illustrates an example of an operation in which an electronic device according to one embodiment displays content on the execution screen of a message application.

[0014] FIG. 6 is a diagram illustrating operations for an electronic device to generate personalized metadata according to one embodiment.

[0015] FIGS. 7, FIGS. 8, FIGS. 9, FIGS. 10, FIGS. 11 and FIGS. 12 illustrate an example in which an electronic device generates personalized metadata according to one embodiment.

[0016] FIG. 13 illustrates an example of an interface screen for editing personalized metadata according to one embodiment.

[0017] FIGS. 14, 15, and 16 illustrate an example in which an electronic device generates personalized metadata according to one embodiment.

[0018] FIGS. 17a, FIGS. 17b, FIGS. 17c, and FIGS. 17d illustrate examples of an operation in which an electronic device performs a search using metadata according to one embodiment.

[0019] FIGS. 18a and FIGS. 18b illustrate an example in which an electronic device generates content using personalized metadata according to one embodiment.

[0020] FIG. 19 illustrates an example in which an electronic device generates personalized metadata according to one embodiment.

[0021] Hereinafter, embodiments of the present disclosure are described in detail with reference to the drawings so that those skilled in the art can easily practice them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein. In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and brevity.

[0022] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this disclosure is not limited to the devices described above.

[0023] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as "coupled" or "connected" to another (e.g., 2nd) component, with or without the terms "functionally" or "communicationly," it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0024] Throughout the specification, singular expressions include plural expressions unless the context clearly indicates otherwise. Wherever a part of the specification states that it "includes" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0025] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0026] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments.

[0027] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0028] The processor (120) can control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., a program (140)), and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0029] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0030] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0031] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0032] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0033] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0034] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0035] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0036] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0037] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0038] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0039] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that can be perceived by the user through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0040] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0041] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0042] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0043] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0044] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) may support a Peak data rate (e.g., 20 Gbps or more) for eMBB realization, loss coverage (e.g., 164 dB or less) for mMTC realization, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for URLLC realization.

[0045] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0046] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0047] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0048] According to one embodiment, commands or data may be transmitted or received between an electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used.

[0049] The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199). The electronic device (101) may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0050] FIGS. 2a and 2b illustrate an example of a block diagram of an electronic device according to one embodiment. The electronic device (101) of FIG. 1 may correspond to the electronic device (200) of FIGS. 2a and 2b. The electronic device (200) of FIGS. 2a and 2b may include the electronic device (101) of FIG. 1. The electronic device (200) of FIGS. 2a and 2b may include at least one component among the components of the electronic device (101) shown in FIG. 1. The components, their relationships, and their functions shown in FIGS. 2a and 2b are merely exemplary and are not intended to limit the implementations described or claimed herein.

[0051] Referring to FIG. 2a, the electronic device (200) may include at least one processor (210) (hereinafter referred to as processor (210)), at least one memory (220) (hereinafter referred to as memory (220)), and at least one display (230) (hereinafter referred to as display (230)). Referring to FIG. 2b, the electronic device (200) may include a processor (210), memory (220), display (230), and at least one communication circuit (240) (hereinafter referred to as communication circuit (240)). The processor (210), memory (220), and display (230) of FIG. 2a are each implemented substantially identically to the processor (120), memory (130), and display module (160) of FIG. 1, so that they can perform the same functions. The processor (210), memory (220), display (230), and communication circuit (240) of FIG. 2b are each implemented substantially identically to the processor (120), memory (130), display module (160), and communication module (190) of FIG. 1, so that they can perform the same functions.

[0052] The components illustrated in FIGS. 2a and 2b are merely exemplary. For example, the electronic device (200) may include other components (e.g., at least one image sensor, at least one sensor, a PMIC (power management integrated circuit), an audio processing circuit, an antenna, a rechargeable battery, and / or an input / output interface). For example, some components may be omitted from the electronic device (200). For example, some components may be integrated into a single component.

[0053] The processor (210) can control the overall operations of the electronic device (200). For example, the processor (210) can control the operation of the electronic device (200) by being operatively connected to the memory (220), the display (230), and the communication circuit (240). Additionally, the processor (210) can control the operation of the electronic device (200) according to the present disclosure by executing one or more instructions stored in the memory (220). The processor (210) may be composed of one or more processors.

[0054] The processor (210) may be implemented as one or more IC (integrated circuit (or circuitry)) chips and may perform various data processing operations. The processor (210) may include at least one electrical circuit and may process instructions (or programs, data) stored in memory (220) individually or collectively in a distributed manner. The processor (210) may include a processor assembly comprising one or more processing circuits. The processor (210) may include any processing circuit that is operative to control the performance and operations of one or more components of the electronic device (200) (e.g., memory (220), display (230), communication circuit). For example, the processor (210) (e.g., application processor (AP)) may be implemented as a system on chip (SoC) (e.g., a single chip or a chipset). For example, the processor (210) may be implemented as multiple cores (or at least one core circuit), multiple chips, or multiple chipsets. For example, the processor (210) may include one or more processing circuits. For example, the processor (210) may include one or more processing circuits configured to perform the various functions of the present disclosure individually and / or collectively. As an example without limitation, at least a portion of the processor (210) may be included in a first chip of the electronic device (200), and at least another portion of the processor (210) may be included in a second chip of the electronic device (200) different from the first chip of the electronic device (200).

[0055] For example, the processor (210) may include a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), an image signal processor (ISP), a display controller, a memory controller, a storage controller, a communication processor (CP), and / or a sensor interface. These components of the processor (210) are merely exemplary. For example, the processor (210) may include other components. For example, some components of the processor (210) may be omitted from the processor (210). For example, some components of the processor (210) may be included as separate components of the electronic device (200) outside the processor (210). For example, some components of the processor (210) (e.g., a memory controller) may be included within other components (e.g., at least a portion of the memory (220), an interface (e.g., available for connection to at least one component of the electronic device (200)), a display (230)).

[0056] The processor (210) may cause other components of the electronic device (200) to perform various operations by executing instructions stored in memory (220). The CPU (or central processing circuit) may be configured to control the components of the processor (210) based on the execution of instructions stored in memory (220) (e.g., volatile memory and / or non-volatile memory). The GPU (or graphics processing circuit) may be configured to execute parallel operations (e.g., rendering). The NPU (or neural processing circuit, or AI (artificial intelligence) chip) may be configured to execute operations for an artificial intelligence model (e.g., convolution computation). The ISP (or image signal processing circuit) may be configured to process a raw image acquired through an image sensor into a format suitable for components within the electronic device (200) or components of the processor (210). A display controller (or display control circuit, or DPU (display processing unit)) may be configured to process an image obtained from a CPU, GPU, ISP, or memory (220) (e.g., volatile memory) into a format suitable for display (230). A memory controller (or memory control circuit) may be configured to control reading data from volatile memory and writing data to volatile memory. A storage controller (or storage control circuit) may be configured to control reading data from non-volatile memory and writing data to non-volatile memory. A CP (communication processing circuit) may be configured to process data obtained from a component of the processor (210) into a format suitable for transmitting to another electronic device via a communication circuit, or to process data obtained from another electronic device via a communication circuit into a format suitable for processing by a component of the processor (210).For example, the communication circuit may include one or more communication circuits. The sensor interface (or sensing data processing circuit, sensor hub) may be configured to process data regarding the state of the electronic device (200) and / or the state around the electronic device (200), obtained through the sensor, into a format suitable for the components of the processor (210).

[0057] For example, the processor (210) can perform the method according to an embodiment of the present disclosure by executing one or more instructions stored in memory (220).

[0058] When a method according to one embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by a single processor or by a plurality of processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to one embodiment, the first operation, the second operation, and the third operation may all be performed by a first processor, or the first operation and the second operation may be performed by a first processor (e.g., a general-purpose processor) and the third operation may be performed by a second processor (e.g., an artificial intelligence dedicated processor).

[0059] The processor (210) may be implemented as a single-core processor including one core, or as one or more multicore processors including multiple cores (e.g., homogeneous multicore or heterogeneous multicore). When the processor (210) is implemented as a multicore processor, each of the multiple cores included in the multicore processor may include internal processor memory such as cache memory or on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. Additionally, each of the multiple cores included in the multicore processor (or some of the multiple cores) may independently read and execute program instructions for implementing a method according to one embodiment of the present disclosure, or all (or some) of the multiple cores may be linked together to read and execute program instructions for implementing a method according to one embodiment of the present disclosure.

[0060] When a method according to one embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one of the plurality of cores included in a multi-core processor, or may be performed by a plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to one embodiment, the first operation, the second operation, and the third operation may all be performed by a first core included in a multi-core processor, or the first operation and the second operation may be performed by a first core included in a multi-core processor and the third operation may be performed by a second core included in a multi-core processor.

[0061] In the embodiments of the present disclosure, a processor may mean a system-on-chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, GPU, APU (accelerated processing unit), MIC (many integrated core), DSP (digital signal processor), NPU, hardware accelerator, or machine learning accelerator, but the embodiments of the present disclosure are not limited thereto.

[0062] Memory (220) may include one or more storage media (or one or more storage devices). For example, memory (220) may include a memory assembly comprising one or more storage media. For example, the one or more storage media may include a hard drive, flash memory, permanent memory (e.g., non-volatile memory) such as ROM (read-only memory), semi-permanent memory (e.g., volatile memory) such as RAM (random access memory), any other suitable type of storage (or storage assembly), or any combination thereof. Memory (220) may include a cache memory, which is one or more different types of memory used to temporarily store data for a function or feature of the electronic device (200). As an example, but not limited to, the cache memory may be included within the processor (210). The memory (220) can be fixedly embedded in the electronic device (200) or incorporated into one or more suitable types of components (e.g., a SIM (subscriber identity module) card and / or an SD (secure digital) card) that can be repeatedly inserted into and removed from the electronic device (200).

[0063] For example, memory (220) may store one or more software applications, such as an operating system (or system) software application (e.g., application (146) of FIG. 1), a firmware software application, a driver software application, a plugin (e.g., add-in, add-on, and / or applet) software application, and / or any other suitable software application. For example, the one or more software applications may include instructions executable by the processor (210). For example, memory (220) may store instructions that can be called by an application programming interface (API). For example, memory (220) may store instructions within a library.

[0064] The display (230) is controlled by the processor (210) and can output visualized information to the user. The visualized information may include visual objects displayed on the display (230). For example, the visual objects may include screens, images, icons, GUI (graphic user interface), and UI (user interface) elements.

[0065] For example, the display (230) can be implemented as a flat panel display (FPD), a curved display, or a flexible display. For example, the display (230) can be implemented as various types of displays such as a liquid crystal display (LCD), an active matrix organic light emitting diodes (AMOLED) display, an LED (light emitting diodes), a micro LED, or a mini LED.

[0066] The display (230) may include a touch detection circuit configured to detect touch. The touch detection circuit may acquire user input that touches (or selects) a visual object displayed on the display (230). For example, the touch detection circuit may detect an input (e.g., touch input or hovering input) on a specific location by measuring a change in a signal (e.g., voltage, light intensity, resistance, or charge) for a specific location on the display (230), and provide information regarding the detected input to the processor (210).

[0067] The communication circuit (240) can perform data communication with an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)) under the control of the processor (210). For example, the communication circuit (240) can communicate with an external electronic device through a network. For example, the processor (210) can receive data from an external electronic device through the communication circuit (240) and transmit data to an external electronic device through the communication circuit (240).

[0068] The communication circuit (240) may include hardware components to support the transmission and / or reception of electrical signals between the electronic device (200) and an external electronic device. For example, the communication circuit (240) may support the transmission and / or reception of electrical signals based on various types of protocols such as Ethernet, LAN (local area network), WAN (wide area network), Wi-Fi (wireless fidelity), Bluetooth, BLE (bluetooth low energy), ZigBee, LTE (long term evolution), 5G NR (new radio) and / or 6G.

[0069] FIG. 3 is a diagram illustrating operations for an electronic device to generate personalized metadata according to one embodiment. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. A processor (210) may perform at least one of the operations of FIG. 3. Instructions stored in memory (220) may cause the electronic device (200) to perform the operations of FIG. 3 when executed individually or collectively by the processor (210).

[0070] According to one embodiment, in operation 310, the electronic device (200) can display the execution screen of the application on the display (230).

[0071] In one embodiment, the electronic device (200) may display a plurality of icons corresponding to a plurality of applications on the display (230). For example, the electronic device (200) may display a home screen including a plurality of icons corresponding to a plurality of applications. The applications may include applications installed on the electronic device (200). In response to user input touching an icon among the plurality of icons, the electronic device (200) may execute an application corresponding to the selected icon and display the execution screen of the application on the display (230). The execution screen of the application may be a screen including visual information provided by the application.

[0072] According to one embodiment, in operation 320, the electronic device (200) can display selected content on the execution screen of the application based on user input.

[0073] In one embodiment, the content may include media content such as images, videos, and audio.

[0074] In one embodiment, displaying content on the execution screen of an application may include content being displayed on the execution screen of an application by an application for displaying content (e.g., a viewer application) and / or content selected by a picker application being displayed on the execution screen of an application that called the picker application (e.g., a messaging application or a social media application). The viewer application may be an application for executing a function of displaying content and / or files to visually provide content and / or files to a user. Additionally, the picker application may be a function (e.g., a function of an operating system or a function of an application) for providing a user interface necessary for a user to select content and / or files.

[0075] In one embodiment, displaying content on the execution screen of an application may include the electronic device (200) transmitting content through the application to the counterparty's device via an external electronic device (e.g., the server (108) of FIG. 1), thereby displaying the content on the execution screen of the application. For example, the application may be a message application. The message application may execute a function that provides a conversation service through a server. The message application may display messages and / or content transmitted and received between the user and the counterparty on the execution screen. The counterparty may include one or more counterparties. The message application may select content using a picker application.

[0076] In one embodiment, displaying content on the execution screen of an application may include the electronic device (200) uploading content to an external electronic device (e.g., the server (108) of FIG. 1) through the application, thereby displaying the content on the execution screen of the application. For example, the application may be a social networking service (SNS) application. An SNS application may perform functions that connect users with each other and allow sharing various content. An SNS application may display content on the execution screen during the process of uploading a post containing content to the server or uploading to the user's account. An SNS application may select content using a picker application.

[0077] FIG. 4 is a diagram illustrating operations for displaying content on an application execution screen by an electronic device according to one embodiment. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. A processor (210) may perform at least one of the operations of FIG. 4. When instructions stored in memory (220) are executed individually or collectively by the processor (210), the electronic device (200) may be made to perform the operations of FIG. 4.

[0078] According to one embodiment, in operation 410, the electronic device (200) can receive user input for displaying a plurality of contents stored in the electronic device (200).

[0079] In one embodiment, the electronic device (200) displays an interface element (e.g., a button displayed on the execution screen) for displaying a plurality of contents stored in the media storage (e.g., an image database) of the electronic device (200) on the execution screen of an application, and can receive touch input for the interface element.

[0080] According to one embodiment, in operation 410-N, if user input for displaying a plurality of contents is not received, the electronic device (200) may not proceed with subsequent operations. For example, the electronic device (200) may not display a plurality of contents stored in the electronic device (200).

[0081] According to one embodiment, in operation 410-Y, 420, when a user input for displaying a plurality of contents is received by the electronic device (200), the plurality of contents stored in the electronic device (200) can be displayed on the display (230).

[0082] In one embodiment, when user input touching an interface element is received, the application may call a picker application (e.g., an image picker). The picker application may provide an interface screen for the user to select an image stored in the media storage. The interface screen may include a plurality of contents in the media storage. For example, a plurality of contents stored in the media storage may be visually displayed on the display (230). The contents stored in the media storage may include content created directly by the user through shooting and / or content obtained by the user from an external source (e.g., content downloaded from the public domain and content received from another user).

[0083] In one embodiment, content stored in a media storage may include common metadata. Common metadata may include the title of the content, the date and time of shooting, and the location of shooting. Additionally, common metadata may include a tag list. Tags included in the tag list may include information obtained through content analysis. For example, if the content is an image or a video, the tags may include information obtained through image analysis. For example, the tags may be information obtained by classifying images or by detecting objects (e.g., people, objects, backgrounds, landscapes) in images and performing object recognition. For example, if the content is audio, the tags may include singer, title, genre, composer, or lyrics. The tag list may be automatically generated when the content is created or stored. Common metadata may be stored in the content file. Alternatively, common metadata may be stored in the media storage associated with the content separately from the content file. For example, the electronic device (200) may use a database or use the name or hash value of the content to map and store the common metadata to the content.

[0084] According to one embodiment, in operation 430, the electronic device (200) may receive user input selecting content among a plurality of contents. In one embodiment, the electronic device (200) may receive touch input for content among a plurality of contents displayed on the display (230).

[0085] According to one embodiment, in operation 430-N, if the electronic device (200) does not receive user input selecting content among a plurality of contents, it may not proceed with subsequent operations. For example, the electronic device (200) may not display the content selected based on user input on the execution screen of the application.

[0086] According to one embodiment, in operation 430-Y, 440, when the electronic device (200) receives user input selecting content among a plurality of contents, it can display the selected content on the execution screen of the application based on the user input.

[0087] In one embodiment, when a picker application receives user input selecting at least one of a plurality of contents, it can transmit the selected content to the application. For example, the picker application can provide the uniform resource identifier (URI) of the selected content to the application.

[0088] An application can display content received from a picker application on the application's execution screen. For example, the application's execution screen may include interface elements for transmitting content to a server. When user input is received by touching an interface element displayed on the execution screen, the application can retrieve content from a media storage using a URI and display the content on the application's execution screen. For example, a messaging application can transmit content to a server and display the content on the application's execution screen to send the content to the recipient's device. For example, a social media application can upload content to a server and display the content on the application's execution screen.

[0089] In one embodiment, the content may be displayed on the application's execution screen in various forms. For example, if the content is an image, the image may be displayed on the application's execution screen. The image may be displayed in a preview form. For example, if the content is a video, a thumbnail and a play icon of the video may be displayed on the application's execution screen. The thumbnail may be the first image frame of the video or an image frame of a specific scene in the video. The play icon may be displayed on the thumbnail. For example, if the content is audio, a progress bar and a play icon may be displayed on the application's execution screen. The progress bar may indicate the length of the audio. The play icon may be displayed around the progress bar. However, it is not limited to the examples described above.

[0090] FIG. 5a illustrates an example of an operation in which an electronic device according to one embodiment displays content on the execution screen of an SNS application.

[0091] Referring to 501 and 502 in FIG. 5a, when the electronic device (200) receives user input touching a button (511) displayed on the execution screen (510) of an SNS application, it can display images (521, 522, 523) stored in a media storage. The button (511) may be a button for uploading a new post. At least some of the images stored in the media storage may be displayed, and the remaining images may be displayed according to touch input (e.g., scrolling).

[0092] Referring to 502 and 503 in FIG. 5a, when the electronic device (200) receives user input in which an image (521) among images (521, 522, 523) is touched and a share button is touched, it can transmit a post containing the image (521) to a server via a communication circuit (240) to upload the post to the user's account. The uploaded post may be displayed on the execution screen (510) of the SNS application. The post may include the image (521) and text (531) written by the user when uploading the image (521).

[0093] FIG. 5b illustrates an example of an operation in which an electronic device according to one embodiment displays content on the execution screen of a message application.

[0094] Referring to 541 and 542 in FIG. 5b, when the electronic device (200) receives user input touching a gallery button (551) displayed on the execution screen (550) of a message application, it can display images (561, 562, 563) stored in a media storage. At least some of the images stored in the media storage may be displayed, and the remaining images may be displayed according to touch input (e.g., scrolling).

[0095] Referring to 542 and 543 in FIG. 5b, when the electronic device (200) receives user input in which an image (561) among images (561, 562, 563) is touched and a transmit button (552) is touched, the device can transmit the image (561) to the other party's device via a server using a communication circuit (240) and display the image (561) on an execution screen (550). For example, the image (561) can be displayed in a chat window (553). In the chat window (553), messages (e.g., text, content) that the user exchanges with the other party through a message application can be displayed in chronological order.

[0096] FIG. 6 is a diagram illustrating operations for an electronic device to generate personalized metadata according to one embodiment. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. A processor (210) may perform at least one of the operations of FIG. 6. Instructions stored in memory (220) may cause the electronic device (200) to perform the operations of FIG. 6 when executed individually or collectively by the processor (210).

[0097] According to one embodiment, in operation 610, the electronic device (200) can select content using an application. For example, the content may be selected by a viewer application or a picker application.

[0098] According to one embodiment, in operation 610-N, the electronic device (200) may not proceed with subsequent operations if content is not selected. For example, if content is not selected, the electronic device (200) may not log the operation of the electronic device (200).

[0099] According to one embodiment, in operation 610-Y, 620, the electronic device (200) can log the operation of the electronic device (200) when content is selected.

[0100] For example, logging the operation of the electronic device (200) may include recording the operation of the electronic device (200) and / or the usage history of the application based on a certain period of time after content is selected or based on a certain task after content is selected, in order to generate metadata. The application may be a viewer application or may include another application that called the picker application (e.g., a message application or an SNS application).

[0101] According to one embodiment, in operation 630, the electronic device (200) can generate metadata. For example, the electronic device (200) can generate metadata by analyzing the operation of the electronic device (200) using logged data or by analyzing the execution screen of an application. According to an embodiment, an artificial intelligence model may be used when generating metadata. The artificial intelligence model may include a model trained to generate metadata by analyzing text and / or images. Meanwhile, a specific method for the electronic device (200) to generate metadata will be described later.

[0102] Returning to FIG. 3, according to one embodiment, in operation 330, the electronic device (200) can acquire input information entered into the application for a preset time or for a time during which at least part of the content is displayed on the application's execution screen.

[0103] In one embodiment, the preset time may include a preset time interval from a first time point to a second time point. The electronic device (300) may acquire information input into the application during the preset time interval from the first time point to the second time point. The first time point may include a time point when the content is displayed on the execution screen of the application. Alternatively, the first time point may be a time point prior to the time point when the content is displayed on the execution screen of the application. The preset time may be set at the time of installation of the application, or may be set and changed by user input.

[0104] In one embodiment, displaying at least a portion of the content on the execution screen of the application may include at least a portion of the content being visually output on the execution screen of the application. The electronic device (200) may track the execution state of the application or analyze the screen displayed on the display (230) to identify whether at least a portion of the content is displayed on the execution screen of the application.

[0105] For example, if the application is terminated, or if another screen (e.g., another execution screen provided by the application, an execution screen provided by another application, a home screen, or a lock screen) is displayed on the display (230) instead of the execution screen of the application where the content is displayed, or if the display (230) is deactivated, the display of the execution screen where the content is displayed may be stopped. The electronic device (200) can acquire information input into the application during the time interval from when the content is displayed on the execution screen of the application to when the display of the execution screen is stopped.

[0106] For example, when information is displayed sequentially on the execution screen of an application, the execution screen may be automatically scrolled. As scrolling, the content displayed on the execution screen moves, and the content may gradually disappear visually from the display (230). The electronic device (200) can acquire input information entered into the application during the time interval from the point when the content is displayed on the execution screen of the application until the point when at least a part of the content is displayed on the execution screen. If the entire content disappears visually from the execution screen, it cannot be considered that at least a part of the content is displayed on the execution screen. Therefore, the fact that at least a part of the content is displayed on the execution screen may include the time interval from when the content is displayed on the execution screen until the entire content disappears visually from the execution screen by scrolling.

[0107] In one embodiment, the information entered into the application may include information entered into the application by a user using an electronic device (200). The user may enter information into the application by using a keyboard interface displayed on the display (230), or through gesture input or voice input. The entered information may be displayed on the execution screen of the application.

[0108] For example, a user can input messages and / or content to be sent to a recipient into a message application. The messages may include text, emoticons, and emojis. The content may include images, videos, audio, and documents. The electronic device (200) can transmit the messages and / or content entered into the message application to the recipient's device via a server and display the messages and / or content on the execution screen of the message application.

[0109] For example, a user can input text about a post uploaded to their account into a social media application. The text may include a post or comment about the post, or a comment about a comment written by another person about the post. The electronic device (200) can upload the text entered into the social media application to a server and display the uploaded text on the execution screen of the social media application.

[0110] In one embodiment, the information input into the application may include information received by the application from an external electronic device (e.g., the server (108) of FIG. 1). The input information may be displayed on the execution screen of the application.

[0111] For example, the other party may input messages and / or content to be sent to the user into a message application on the other party's device (e.g., the electronic device (104) of FIG. 1). The messages may include text, emoticons, and emojis. The content may include images, videos, audio, and documents. The electronic device (200) may receive messages and / or content from the other party's device via a server and display the messages and / or content on the execution screen of the message application.

[0112] For example, another person can write a comment on a post uploaded to the user's account by entering text into their SNS application. The electronic device (200) can connect to a server through the SNS application and display an execution screen of the SNS application that includes the comment.

[0113] In one embodiment, the electronic device (200) can acquire information input into the application. The information input into the application can be displayed on the execution screen of the application. The electronic device (200) can acquire information input into the application during the time interval by analyzing the execution screen of the application.

[0114] For example, the electronic device (200) can capture the execution screen of an application at regular time intervals from the time when content is displayed on the execution screen, thereby capturing the execution screen within the said time interval. The electronic device (200) can extract text from the captured execution screen through text extraction (e.g., OCR (optical character recognition)). Additionally, the electronic device (200) can obtain a text description of the content by analyzing the image displayed on the captured execution screen through an image analysis technique (e.g., image captioning, image classification, object recognition). According to an embodiment, the electronic device (200) can capture the execution screen of another application and extract text from the captured execution screen through text extraction. For example, a user can execute another application and input information provided by the other application into the application. The electronic device (200) can capture the execution screen of another application and extract text from the execution screen using OCR.

[0115] For example, information entered into an application can be stored in an electronic device (200). The electronic device (200) transmits information entered into an application to an external electronic device (e.g., the server (108) of FIG. 1), and can store the data transmitted to the external electronic device in the electronic device (200). For example, the electronic device (200) can store messages and content exchanged between a user and another party through a message application in a memory (220). The electronic device (200) can obtain information entered into the application during the time interval using the data stored in the memory (220).

[0116] According to the present disclosure, the electronic device (200) can acquire information input into the application during the time interval. For example, when a user sends content to another party through a message application, the user and the other party are likely to have a conversation related to the content and exchange other content related to the content. Similarly, when a user shares content through an SNS application, people are likely to write posts related to the content and share other content related to the content. In this way, information input into the application for a certain period of time after the content is displayed on the application's execution screen is likely to be information related to the content. Therefore, when the content is displayed on the application's execution screen, the electronic device (200) can acquire information input into the application for a preset period of time or for a period during which at least a part of the content is displayed on the application's execution screen.

[0117] In one embodiment, the electronic device (200) can identify at least some of the acquired information.

[0118] For example, the electronic device (200) can identify at least some of the texts based on the context of the texts input into the application. The electronic device (200) can analyze the texts input into the application using an artificial intelligence model (e.g., a natural language processing (NLP) model). The electronic device (200) can analyze the texts using a pre-trained language model and identify the meaning and context of the texts to identify texts that have substantially the same context (or topic) among the texts. For example, there may be cases where users write texts unrelated to the content after the content has been transmitted or shared through the application. In the present disclosure, the electronic device (200) can identify information related to the content among the acquired information by considering the context or topic of the texts.

[0119] In the present disclosure, the artificial intelligence model may be implemented in the form of an on-device device included in the electronic device (200). The electronic device (200) may input texts into the artificial intelligence model stored in memory (220) to obtain information about texts having substantially the same context from the artificial intelligence model. As another example, the artificial intelligence model may be stored in an external electronic device (e.g., the server (108) of FIG. 1) connected to the electronic device (200). When the artificial intelligence model is stored in the server, the electronic device (200) may transmit texts to the external electronic device through a communication circuit (240) and receive information about the texts from the server.

[0120] In the example described above, input information entered into the application is acquired during the time when at least a portion of the content is displayed on the application's execution screen, but the present disclosure is not limited thereto. The electronic device (200) may acquire information even when the content is not displayed. For example, when a picker application is executed, the screen displayed on the electronic device (200) may be switched to another screen. Due to this screen switching, some operations may be performed while the content is not displayed. For example, the electronic device (200) may acquire the information by identifying the operation of the electronic device (200) and / or the usage history of the application while the content is not visually displayed on the electronic device (200) or while the application is operating in the background.

[0121] According to one embodiment, in operation 340, 350, the electronic device (200) can generate personalized metadata for content based on acquired information and map the personalized metadata to the content and store it in the electronic device (200).

[0122] In one embodiment, the electronic device (200) can extract at least one keyword from information input into the application for a preset time or for a time during which at least a portion of the content is displayed on the application's execution screen, and generate personalized metadata including the extracted keyword.

[0123] For example, the electronic device (200) can obtain texts from information input to the application during the time interval. The texts may include text input to the application during the time interval and / or text obtained from content input to the application during the time interval. Text obtained from content may be obtained through an image analysis technique.

[0124] The electronic device (200) can extract at least one keyword from the texts.

[0125] Keywords may include words representing the core content of the text. Keywords may correspond to tags in personalized metadata. For example, an electronic device (200) may analyze the text to decompose it into multiple components and extract at least one keyword from among the multiple components. The electronic device (200) may extract at least one keyword from the text using an artificial intelligence model (e.g., an NLP model). For example, the electronic device (200) may analyze the text using a pre-trained language model, identify the meaning and context of words within the text, and extract important words within the text to obtain at least one keyword. However, the present disclosure is not limited thereto, and the electronic device (200) may extract keywords using various text extraction techniques.

[0126] In one embodiment, the electronic device (200) can generate personalized metadata containing keywords. For example, the electronic device (200) can tag keywords as personalized metadata. The personalized metadata may include a tag list. Tags included in the tag list may correspond to keywords.

[0127] In one embodiment, the electronic device (200) can generate personalized metadata using an artificial intelligence model. For example, the electronic device (200) can input an image (e.g., the aforementioned captured execution screen) into the artificial intelligence model to obtain personalized metadata from the artificial intelligence model. The artificial intelligence model can be trained to obtain context related to the image. For example, the artificial intelligence model can obtain text from the image and analyze the text to generate personalized metadata containing keywords.

[0128] In one embodiment, when generating personalized metadata, the electronic device (200) can use an artificial intelligence model to edit keywords and generate personalized metadata including the edited keywords.

[0129] As described above, when the electronic device (200) generates personalized metadata, keywords can be extracted from information input into the application. The electronic device (200) can use an artificial intelligence model to analyze the meaning of the keywords and identify whether there are keywords with the same or similar meanings. If the electronic device (200) identifies that there are keywords with the same or similar meanings, it can integrate the identified keywords into a single keyword. Integrating keywords may include selecting one keyword from among the keywords with the same or similar meanings. The selected keyword may be a keyword that represents the keywords with the same or similar meanings. The electronic device (200) can generate personalized metadata using the integrated keyword and the remaining keywords.

[0130] In one embodiment, the electronic device (200) may store personalized metadata by mapping it to content. For example, the personalized metadata may be stored in a content file. For example, the personalized metadata may be stored separately from the content file in the media storage of the electronic device (200) in association with the content. The electronic device (200) may store personalized metadata by mapping it to content using a database or by using the name or hash value of the content. For example, the personalized metadata may be stored in an external electronic device (e.g., the server (108) of FIG. 1). The external electronic device may be a cloud server linked to a user account. For example, the personalized metadata may be stored in a secure area of ​​the electronic device (200). The secure area is an area for protecting user data and may be an area where biometric information and encryption keys are stored in memory (220). For example, the electronic device (200) may store the personalized metadata by encrypting it. For example, personalized metadata may be stored as metadata of content in a metadata format (e.g., exif) or stored in a metadata management unit within an electronic device (200).

[0131] Meanwhile, although the above example describes generating personalized metadata using keywords obtained from text, the present disclosure is not limited thereto. For example, an electronic device (200) may summarize text to obtain summarized text (e.g., summarized natural language) and generate metadata containing the summarized text. The summarization of text may be performed by a language model.

[0132] Below, with reference to the attached drawings, an example of an electronic device (200) generating personalized metadata is described.

[0133] Referring to FIG. 7, a user can send an image (721) to another party through a message application. An electronic device (200) can acquire information entered into the message application for a preset time period from the time when the image (721) is displayed in the chat window (711). For example, if the messages (731, 732, 733, 734) displayed in the chat window (711) are messages entered into the application during the time interval, the electronic device (200) can acquire the messages (731, 732, 733, 734), as shown in 741 of FIG. 7. The electronic device (200) can extract keywords (751) such as "Hayoung", "Barcelona", "Travel", "May", and "Born District" from the messages (741) and generate personalized metadata (761) containing the extracted keywords (751).

[0134] Metadata associated with the image (721) may include personalized metadata (761) and general metadata (762). General metadata (762) may include tags (person, fountain, tree, building) obtained from the image (721) through image analysis. Personalized metadata (761) may include tags (Hayoung, Barcelona, ​​travel, May, Born district) obtained through analysis of the execution screen (711) on which the image (721) is displayed.

[0135] Referring to FIG. 8, a user can transmit an image (821) to another party through a message application. An electronic device (200) can obtain information entered into the message application during the time when at least a portion of the image (821) is displayed in the chat window (811). For example, if the messages (831, 832, 833, 834, 835, 836, 837, 838) displayed in the chat window (811) are messages entered into the application during the time interval, the electronic device (200) can obtain the messages (831, 832, 833, 834, 835, 836, 837, 838), as shown in 841 of FIG. 8. The electronic device (200) can extract keywords (851) such as "Hayoung", "Barcelona", "travel", "May", "Born district", "Santa Maria del Mar Cathedral", and "boyfriend" from messages (841), and generate personalized metadata (861) containing the extracted keywords (851).

[0136] Metadata associated with the image (821) may include personalized metadata (861) and general metadata (862). General metadata (862) may include tags (person, fountain, tree, building) obtained from the image (821) through image analysis. Personalized metadata (861) may include tags (Hayoung, Barcelona, ​​travel, May, Born district, Santa Maria del Mar Cathedral, boyfriend) obtained through analysis of the execution screen (811) on which the image (821) is displayed.

[0137] Referring to FIG. 9, a user can upload a post containing an image (921) and text (931) through an SNS application. The uploaded post can be displayed on the execution screen (911) of the SNS application. The electronic device (200) can acquire information entered into the SNS application for a preset time period starting from the time the image (921) is displayed. For example, if the text (931) displayed on the execution screen (911) is text entered into the application during the time interval, the electronic device (200) can acquire the text (941), as in 941 of FIG. 9. The electronic device (200) can extract keywords (951) such as "spring," "Spain," "Barcelona," "Gaudi tour," and "European travel" from the text (941) and generate personalized metadata (961) containing the extracted keywords (951).

[0138] Metadata associated with the image (921) may include personalized metadata (961) and general metadata (962). General metadata (962) may include tags (person, fountain, tree, building) obtained from the image (921) through image analysis. Personalized metadata (961) may include tags (spring, Spain, Barcelona, ​​Gaudi tour, Europe travel) obtained through analysis of the execution screen (911) on which the image (921) is displayed.

[0139] Referring to FIG. 10, a user can transmit an image (1021) to another party through a message application. An electronic device (200) can obtain information entered into the message application during the time when at least a portion of the image (1021) is displayed in the chat window (1011). For example, messages (1031, 1032, 1033, 1034, 1035, 1036, 1037, 1038, 1039) displayed in the chat window (1011) may be messages entered into the application during the time interval. The electronic device (200) can analyze the context of messages (1031, 1032, 1033, 1034, 1035, 1036, 1037, 1038, 1039) to identify messages (1031, 1032, 1033, 1034, 1035, 1038, 1039) related to the image (1021), and acquire messages (1041) as shown in 1041 of FIG. 10. The electronic device (200) can extract keywords (1051) such as "Hayoung", "Barcelona", "travel", "May", "Born district", and "Picasso Museum" from the messages (1041), and generate personalized metadata (1061) containing the extracted keywords (1051).

[0140] Metadata associated with the image (1021) may include personalized metadata (1061) and general metadata (1062). General metadata (1062) may include tags (person, fountain, tree, building) obtained from the image (1021) through image analysis. Personalized metadata (1061) may include tags (Hayoung, Barcelona, ​​travel, May, Born district, Picasso Museum) obtained through analysis of the execution screen (1011) on which the image (1021) is displayed.

[0141] In one embodiment, tags included in personalized metadata may have priority. For example, when the electronic device (200) generates a list of tags for personalized metadata using keywords, it may identify the count for each tag and add the count information to the tag list. The priority of the tags may be determined based on the number of tags. For example, tags may have a higher priority in order of the number of tags.

[0142] For example, assume that the keywords obtained from information entered into the application are "Spain", "Travel", "Barcelona", "May", "Born District", "Travel", "Born District", and "Travel". The electronic device (200) can generate personalized metadata using the keywords. The tag list of the personalized metadata may include tags and the number of each tag, such as "Travel (3), Born District (2), Spain, Barcelona, ​​May". For example, since three keywords of "Travel" are obtained from information entered into the application, the number of "Travel" in the tag list may be 3. Also, since two keywords of "Born District" are obtained from information entered into the application, the number of "Born District" in the tag list may be 2. Among the tags, "Travel", which has the highest number, may have the highest priority, and "Born District," which has the next highest number, may have the next highest priority after "Travel".

[0143] In one embodiment, the electronic device (200) may display content on the execution screens of a plurality of applications, extract at least one keyword from information input to each of the plurality of applications, and generate personalized metadata including the extracted keyword. The electronic device (200) may obtain keywords from information input to the plurality of applications and integrate the keywords to generate personalized metadata. Since the operation of the electronic device (200) extracting keywords using information input to the applications is the same as previously described, a repetitive description is omitted for brevity.

[0144] For example, referring to FIG. 11, when an image (1101) is displayed on the execution screen (1111) of a message application, the electronic device (200) can obtain keywords (1121) such as "Hayoung," "Barcelona," "Travel," "May," and "Born District" from the messages displayed on the execution screen (1111). When an image (1101) is displayed on the execution screen (1112) of a social media application, the electronic device (200) can obtain keywords (1122) such as "Spring," "Spain," "Barcelona," "Gaudi Tour," and "European Travel" from the texts displayed on the execution screen (1112). The electronic device (200) can generate personalized metadata (1131) using the keywords (1121) and keywords (1122).

[0145] Metadata associated with an image (1101) may include personalized metadata (1131) and general metadata (1132). General metadata (1132) may include tags such as "person, fountain, tree, building". Personalized metadata (1131) may include tags such as "Barcelona (2), Hayoung, travel, May, Born district, spring, Spain, Gaudi tour, Europe travel".

[0146] In one embodiment, the electronic device (200) may display content on a plurality of execution screens of an application, extract at least one keyword from information input to the application, and generate personalized metadata including the extracted keyword. The electronic device (200) may obtain keywords from information input to different execution screens of the application and integrate the keywords to generate personalized metadata. Since the operation of the electronic device (200) extracting keywords using information input to the application is the same as previously described, a repetitive description is omitted for brevity.

[0147] For example, referring to FIG. 12, when an image (1201) is displayed on a first execution screen (1211) of a message application, the electronic device (200) can obtain keywords (1221) such as "Hayoung," "Barcelona," "Travel," "May," and "Born District" from the messages displayed on the first execution screen (1211). Additionally, when an image (1201) is displayed on a second execution screen (1212) of a message application, the electronic device (200) can obtain keywords (1222) such as "Born District," "European Travel," and "Spain" from the messages displayed on the second execution screen (1212). The second execution screen (1212) may be a different execution screen from the first execution screen (1211). For example, the person with whom the user converses through the conversation window of the first execution screen (1211) may be different from the person with whom the user converses through the conversation window of the second execution screen (1212). The electronic device (200) can generate personalized metadata (1231) using keywords (1221) and keywords (1222).

[0148] Metadata associated with an image (1201) may include personalized metadata (1231) and general metadata (1232). General metadata (1232) may include tags such as "person, fountain, tree, building". Personalized metadata (1231) may include tags such as "Born district (2), Hayoung, Barcelona, ​​travel, May, Europe travel, Spain".

[0149] In one embodiment, the electronic device (200) may display an interface screen for editing personalized metadata on a display (230).

[0150] FIG. 13 illustrates an example of an interface screen for editing personalized metadata according to one embodiment.

[0151] Referring to 1301 in FIG. 13, the interface screen may include various visual elements. For example, the interface screen (1311) may display content (e.g., an image) (1321), tags (1331) included in general metadata associated with the content (1321), and tags (1332) included in personalized metadata associated with the content (1321).

[0152] The electronic device (200) may receive user input by touching an interface element (1341) for editing tags (1332) included in personalized metadata. Referring to 1302 in FIG. 13, the electronic device (200) may display an interface screen (1351) for editing tags included in personalized metadata on a display (230) based on the user input. The interface screen (1351) may include a keyword input field (1361) and tags (1362) included in personalized metadata.

[0153] In one embodiment, when the electronic device (200) receives user input to add a keyword through an interface screen, it can add a keyword corresponding to the user input to personalized metadata. For example, the user can use a keyboard interface or voice input to enter a keyword into a keyword input field (1361) and touch an add button (1363). The electronic device (200) can add the keyword entered into the keyword input field (1361) to a tag list of personalized metadata.

[0154] In one embodiment, the electronic device (200) may provide recommended keywords to the user. Recommended keywords may be generated by an artificial intelligence model. The electronic device (200) may display recommended keywords on an interface screen, and when user input selecting at least one keyword among the recommended keywords is received, the selected keyword may be added to personalized metadata.

[0155] In one embodiment, when the electronic device (200) receives user input to delete a keyword through an interface screen, it can delete a keyword corresponding to the user input from at least one keyword included in the personalized metadata. For example, the user can touch a delete button (1364) displayed around "Born District" to delete "Born District". The electronic device (200) can delete the tag of "Born District" from the personalized metadata.

[0156] In one embodiment, the priority of a tag added based on user input may be higher than the priority of other tags. For example, assume a case where "Spain" is added by a user among the tags "Born District (2), Spain, Barcelona, ​​May" included in personalized metadata. Among the tags, "Spain" may have the highest priority. Among the remaining tags, "Born District," which has the most occurrences, may have the next highest priority after "Spain."

[0157] In one embodiment, the electronic device (200) can acquire information related to an application running in the background and generate personalized metadata for content based on the acquired information.

[0158] An application running in the background means that the execution screen provided by the application is not visually displayed on the display (230), but the application may be in a state where it can perform some tasks using resources allocated by the operating system (e.g., the operating system (142) of FIG. 1). For example, an application running in the background may include an application running in the background before content is displayed on the application's execution screen and / or an application running in the background after content is displayed on the application's execution screen. The electronic device (200) can identify an application running in the background when content is displayed on the application's execution screen.

[0159] In one embodiment, information related to an application running in the background may include at least one of the application's identification information and information stored through the application.

[0160] For example, the identification information of an application may include the name of the application or keywords related to the name. Keywords related to the name may be pre-set based on the attributes of the application. For example, keywords corresponding to a health application may include "health," "exercising," and "exercise completed." As another example, keywords corresponding to a schedule management application may include "schedule" and "travel." Keywords related to the name may be set and changed by user input.

[0161] For example, information stored through an application may include information stored by the application in an electronic device (200) or an external electronic device (e.g., the server (108) of FIG. 1) based on user input. For example, a user may store exercise records through a health application and schedules through a schedule management application. Information entered by the user may be stored in memory (220) or a cloud server.

[0162] In one embodiment, when content is displayed on the execution screen of an application, the electronic device (200) can identify an application running in the background and obtain identification information of the application or information stored through the application. The electronic device (200) can obtain keywords from the obtained information and generate personalized metadata containing the keywords. Since the operation of the electronic device (200) extracting keywords and generating personalized metadata is the same as described above, a repetitive description is omitted for brevity.

[0163] For example, referring to FIG. 14, when an image (1401) is displayed on the execution screen (1411) of a message application, the electronic device (200) can obtain keywords (1421) such as "weekend," "exercise," "XX gym," and "squat" from the messages displayed on the execution screen (1411). When the electronic device (200) identifies that the application running in the background is a health application (1431), it can obtain keywords (1432) such as "exercising" and "health." The electronic device (200) can generate personalized metadata (1441) using the keywords (1421) and keywords (1432).

[0164] Metadata associated with an image (1401) may include personalized metadata (1441) and general metadata (1442). General metadata (1442) may include tags such as "person, exercise equipment". Personalized metadata (1441) may include tags such as "exercising, fitness, weekend, workout, XX gym, squat".

[0165] For example, referring to FIG. 15, a user can take an image using a camera application. An electronic device (200) can display an image (1501) on the execution screen (1511) of the camera application. When the electronic device (200) receives user input for image capture, it can capture the image (1501) and identify the application running in the background. For example, if the electronic device (200) identifies that the application running in the background is a health application (1521), it can obtain keywords (1522) such as "exercising" and "health". The electronic device (200) can generate personalized metadata (1531) using the keywords (1522).

[0166] Metadata associated with an image (1501) may include personalized metadata (1531) and general metadata (1532). General metadata (1532) may include tags such as "person, exercise equipment". Personalized metadata (1531) may include tags such as "exercising, fitness".

[0167] For example, referring to FIG. 16, a user can take an image using a camera application. An electronic device (200) can display an image (1601) on the execution screen (1611) of the camera application. When the electronic device (200) receives user input for image capture, it can capture the image (1601) and identify an application running in the background. For example, if the electronic device (200) identifies that the application running in the background is a schedule management application (1621) (or a calendar application), it can obtain schedule information (e.g., name of the schedule, time) corresponding to the time of image capture. The schedule information corresponding to the time of image capture may include schedule information between a first time point and a second time point. The first time point is a time point prior to a preset time from the time of image capture, and the second time point may be a time point after a preset time from the time of image capture. The electronic device (200) can obtain keywords (1622) such as "travel," "Jeju Island," "June," and "2 nights and 3 days" from the acquired schedule information. The electronic device (200) can generate personalized metadata (1631) using the keywords (1622).

[0168] Metadata associated with an image (1601) may include personalized metadata (1631) and general metadata (1632). General metadata (1632) may include tags such as "person, sea". Personalized metadata (1631) may include tags such as "travel, Jeju Island, June, 2 nights 3 days".

[0169] In one embodiment, the electronic device (200) can search for content stored in the electronic device (200) using metadata. For example, the electronic device (200) can display an interface screen for searching for content stored in the electronic device (200) on a display (230). A plurality of contents may be content stored in the media storage of the electronic device (200).

[0170] When a search term is entered through an interface screen, the electronic device (200) can search for at least one content among multiple contents that corresponds to the search term. For example, the electronic device (200) can search for at least one content that matches the search term. Alternatively, the electronic device (200) can search for at least one content among multiple contents stored in the electronic device (200) by referring to the search term. For example, the electronic device (200) can search for at least one content that matches a word related to the search term (e.g., a synonym, a similar word).

[0171] The electronic device (200) can search for at least one content corresponding to a search term among a plurality of contents. For example, the electronic device (200) can search for content corresponding to a search term by using the metadata of the content. The metadata may include general metadata and personalized metadata. The electronic device (200) can search for content corresponding to a search term entered through an interface screen based on general metadata and personalized metadata corresponding to each of the plurality of contents stored in the electronic device (200). For example, the electronic device (200) can search for content tagged with tags matching the search term or words related to the search term based on a tag list of personalized metadata corresponding to each of the plurality of contents. Additionally, the electronic device (200) can search for content tagged with tags matching the search term or words related to the search term based on a tag list of general metadata corresponding to each of the plurality of contents.

[0172] The electronic device (200) can display at least one piece of content retrieved based on personalized metadata and at least one piece of content retrieved based on general metadata on the display (230).

[0173] For example, at least one piece of content retrieved based on personalized metadata may be displayed separately from at least one piece of content retrieved based on general metadata. For example, an electronic device (200) may display at least one piece of content retrieved based on personalized metadata in a first area of ​​an interface screen and at least one piece of content retrieved based on general metadata in a second area of ​​an interface screen. The second area may be a different area from the first area.

[0174] In one embodiment, at least one piece of content retrieved based on general metadata may be displayed sorted by the time or date the content was created, or may be displayed separated by the location where the content was taken.

[0175] In one embodiment, at least one piece of content retrieved based on personalized metadata may be displayed sorted by the time or date the content was created, or may be displayed separated by the location where the content was captured.

[0176] In one embodiment, at least one piece of content retrieved based on personalized metadata may be displayed sorted based on the priority of the tags. For example, the electronic device (200) may identify the priority of tags that match the search term. As previously described, the priority may be determined based on the number of tags. The electronic device (200) may display the retrieved content sorted in order of the priority of the tags. For example, assume a case where "Born District (2)" is tagged in the personalized metadata of the first image and "Born District" is tagged in the personalized metadata of the second image. The electronic device (200) may retrieve the first image and the second image corresponding to the search term "Born District" and display the first image and the second image. Given that the number of "Born District" tagged in the first image is 2 and the number of "Born District" tagged in the second image is 1, the first image may have a higher priority than the second image with respect to "Born District". In this case, the first image and the second image may be displayed in that order.

[0177] Referring to FIG. 17a, the electronic device (200) may display an interface screen (1711) for receiving a search term. The interface screen (1711) may be a screen provided by a gallery application. An interface element (1712) for searching images may be displayed on the interface screen (1711). When the electronic device (200) receives user input touching the interface element (1712), it may display a screen including a search term input field (1721). The user may input a search term into the search term input field (1721) using a keyboard interface or voice input, and input a search command into the gallery application. For example, referring to FIG. 17a, the search term “travel” may be entered into the search term input field (1721).

[0178] Referring to FIG. 17b, the electronic device (200) can display images found for the search term "travel". For example, the electronic device (200) can display images (1731) found based on tags of personalized metadata for the search term "travel" and images (1732) found based on tags of general metadata. When user input is received by touching the accuracy button (1733), the electronic device (200) can display images (1741) sorted based on the priority of "travel".

[0179] In the example described above, search results based on general metadata and search results based on personalized metadata are displayed together, but the present disclosure is not limited thereto. For example, referring to FIG. 17c, an electronic device (200) may display an interface element (1751) for selecting a searched image. In the interface element (1751), "All images" may be an interface element for displaying search results based on general metadata and personalized metadata, "General images" may be an interface element for displaying search results based on general metadata, and "Personalized images" may be an interface element for displaying search results based on personalized metadata. When "Personalized images" is selected and a search term is entered, the electronic device (200) may display images (1761) searched based on tags of personalized metadata.

[0180] As another example, referring to FIG. 17d, the electronic device (200) can display images found for the search term “photos I boasted about to Hayoung.” For example, the electronic device (200) can display images (1771) found based on tags of personalized metadata for the search term “photos I boasted about to Hayoung.” If there are no images found in tags of general metadata, “no results” (1772) may be displayed in the area where search results based on general metadata are displayed.

[0181] As such, according to the present disclosure, personalized metadata is not merely obtained through image analysis of the content, but can be generated by recognizing the context of the execution screen where the content is displayed. The information tagged in the personalized metadata may vary depending on the user utilizing the content and / or the application utilizing the content. Accordingly, when performing a search using personalized metadata, user-customized search results may be provided, and the user experience may be enhanced.

[0182] In one embodiment, the electronic device (200) can learn an artificial intelligence model using personalized metadata and acquire content using the learned artificial intelligence model.

[0183] For example, an artificial intelligence model can be a generative model. Generative AI can refer to AI technology capable of generating new text or images in response to input data. A generative model can refer to a neural network model that implements generative AI technology. By learning the patterns and structures of training data, a generative model can generate new data with characteristics similar to the input data or new data corresponding to the input data.

[0184] In one embodiment, an artificial intelligence model may be trained based on personalized metadata to generate an image that reflects the personalized metadata. The artificial intelligence model may be trained to generate an image that includes the features of tags tagged in the personalized metadata. For example, low-rank adaptation (LoRA) may be used to train the artificial intelligence model, and the training may be performed on an electronic device (200) or an external electronic device (e.g., the server (108) of FIG. 1).

[0185] For example, the electronic device (200) can input content into a learned artificial intelligence model to obtain content that reflects the characteristics of personalized metadata. Referring to FIG. 18a, the electronic device (200) can display an image (1811) through an application (e.g., a viewer application). The execution screen of the application may include a button (1821) for creating an image (hereinafter referred to as the image creation button), a button (1822) for sharing an image (hereinafter referred to as the share button), and a button (1832) for deleting an image (hereinafter referred to as the delete button).

[0186] When an electronic device (200) receives a touch input for an image creation button (1821), it may display tags included in the personalized metadata of the image. For example, referring to FIG. 18b, when an electronic device (200) receives a touch input for an image creation button (1821), it may display a pop-up window (1831) containing "birthday" and "gift". "Birthday" and "gift" may be tags included in the personalized metadata of the image (1811). When the electronic device (200) receives user input to select a tag from among the tags included in the personalized metadata, it may display a pop-up window (1841) and, using a trained artificial intelligence model, generate an image that reflects the features of the selected tag. For example, the pop-up window (1841) may include a notification message indicating that an image is being created.

[0187] For example, referring to FIG. 18b, personalized metadata (1851) mapped to an image (1811) may include tags such as "birthday" and "gift". An artificial intelligence model (1861) may be trained to generate an image that reflects the personalized metadata. An electronic device (200) may input the image (1811) and a tag selected by user input (e.g., "birthday") into the artificial intelligence model (1861) to obtain an image (1871) generated by the artificial intelligence model (1861) and display the image (1871) using an application. The background of the image (1871) may be depicted as a place where a birthday party is taking place. Meanwhile, although the above example describes the electronic device (200) displaying tags included in the personalized metadata, the present disclosure is not limited thereto. For example, if there are two or more tags included in the personalized metadata, the electronic device (200) displays the tags, and if there is only one tag included in the personalized metadata, it may generate an image using the tag included in the personalized metadata without user selection.

[0188] In one embodiment, when the electronic device (200) receives user input for storing content displayed on the execution screen of an application in the electronic device (200), the electronic device (200) can acquire information entered into the application. Based on the acquired information, the electronic device (200) can generate personalized metadata for the content and map the personalized metadata to the content and store it in the electronic device (200).

[0189] For example, when the electronic device (200) receives user input for storing content displayed on the execution screen of an application in the electronic device (200), it can analyze the execution screen of the application to obtain information entered into the application during a time interval. For example, the content displayed on the execution screen of the application may be content received from an external electronic device through a message application. The time interval may include a preset time interval or a time interval in which at least a portion of the content is displayed on the execution screen of the application. Meanwhile, since the operation of the electronic device (200) analyzing the execution screen of the application to obtain and store personalized metadata is the same as described above, a repetitive description is omitted for brevity.

[0190] For example, referring to FIG. 19, an electronic device (200) may receive user input to download an image (1911) displayed on the execution screen (1901) of a message application. The image (1911) may be an image transmitted to the electronic device (200) by the other party's device. The electronic device (200) may obtain keywords (1921) such as "birthday party," "gift," "perfume," and "birthday" from the messages displayed on the execution screen (1901). The electronic device (200) may generate personalized metadata (1931) using the keywords (1921).

[0191] Metadata associated with the image (1911) may include personalized metadata (1931) and general metadata (1932). General metadata (1932) may include tags such as "person, cake". Personalized metadata (1931) may include tags such as "birthday party, gift, perfume, birthday".

[0192] In one embodiment, the electronic device (200) can transmit content to an external electronic device (e.g., the electronic device (102), the electronic device (104), or the server (108) of FIG. 1).

[0193] For example, when the electronic device (200) transmits content to an external electronic device, it may transmit the content without transmitting personalized metadata mapped to the content, or transmit the content and general metadata mapped to the content.

[0194] For example, when an electronic device (200) transmits content to an external electronic device, it may transmit personalized metadata mapped to the content. If the device to which the content is to be transmitted is a device registered with the same user account as the user account of the electronic device (200), the electronic device (200) may transmit the content and personalized metadata mapped to the content.

[0195] As previously mentioned, since personalized metadata can be obtained from conversations between individuals, if personalized metadata is shared with any person, privacy or security-related issues may arise. Therefore, depending on the device to which the content is to be shared, the electronic device (200) may transmit personalized metadata along with the content, or transmit only the content without transmitting personalized metadata.

[0196] In the embodiments described above, the electronic device (200) is described as generating personalized metadata using information input into the application, but the present disclosure is not limited thereto. For example, when content is displayed on the execution screen of the application, the electronic device (200) may acquire context information regarding the surrounding situation and generate personalized metadata using the acquired information.

[0197] For example, context information may include at least one of biometric information, voice information, environment information, and object information within the user's field of view (FOV).

[0198] Biometric information may include various data representing the user's physical or physiological state. For example, biometric information may include heart rate (HR), oxygen saturation (SpO2), blood pressure, blood volume, and stress index measured by a wearable device (e.g., smartwatch, smart ring) worn by the user. When content is displayed on the execution screen of an application, the electronic device (200) may transmit a request for biometric information to a wearable device (e.g., the electronic device (102) or electronic device (104) of FIG. 1) via a communication circuit (240). The electronic device (200) may receive biometric information measured by the wearable device from the wearable device via the communication circuit (240) and obtain at least one keyword from the biometric information. Alternatively, biometric information may be received from the wearable device periodically or in real time.

[0199] Voice information may include information obtained from voice input. When content is displayed on the execution screen of an application, the electronic device (200) may receive voice input through an input module (e.g., input module (150) of FIG. 1) (e.g., microphone). The electronic device (200) may convert the voice input into text using automatic speech recognition (ASR) and analyze the text to obtain at least one keyword.

[0200] Environmental information may include information regarding temperature, illuminance, and atmospheric pressure. For example, a sensor module of an electronic device (200) (e.g., sensor module (176) of FIG. 1) may include a temperature sensor, an illuminance sensor, and / or an atmospheric pressure sensor. When content is displayed on the execution screen of an application, the electronic device (200) may use the sensor module to detect the body temperature, the brightness of external light, and / or atmospheric pressure, and obtain at least one keyword from the detected information.

[0201] Object information may include information about objects within the field of view of a user wearing the electronic device (200). For example, the electronic device (200) is a device that can be worn on the user's head, such as a head-mounted display (HMD) device, and can perform functions related to video see-through (VST).

[0202] For example, the electronic device (200) acquires an image including ambient light of the electronic device (200), and the display (230) displays the image so that the user can perceive the surroundings (or front) of the electronic device (200) even while wearing the electronic device (200). The electronic device (200) can composite a virtual object onto the image displayed through the display (230) to allow the user to perceive real objects and virtual objects located around (or front) the electronic device (200). The display (230) can display a virtual object floating in the user's FOV. A virtual object may refer to an object that can be deployed within the screen for the transmission of information and / or interaction, such as text, images, icons, buttons, and / or progress bars. Additionally, the virtual object may include a virtual screen. An execution screen provided by an application may be displayed on the virtual screen. When content is displayed on the execution screen of an application, the electronic device (200) can identify a virtual object within the user's field of view, obtain information about the virtual object, and obtain keywords from the obtained information. For example, the electronic device (200) can obtain at least one keyword from text included in the virtual object, or analyze an image included in the virtual object through image analysis to obtain text and obtain at least one keyword from the text.

[0203] In these embodiments, the electronic device (200) can generate personalized metadata using the acquired keywords. Since the operation of the electronic device (200) generating personalized metadata using the keywords is the same as previously described, a repetitive description is omitted for brevity.

[0204] An electronic device according to one embodiment as described above may include a display (e.g., the display (230) of FIG. 2a and / or FIG. 2b), a memory (e.g., the memory (220) of FIG. 2a and / or FIG. 2b) for storing instructions, and at least one processor (e.g., the processor (210) of FIG. 2a and / or FIG. 2b) including a processing circuit. When the instructions are executed individually or collectively by the at least one processor, the electronic device may display an application execution screen on the display, display content selected based on user input on the application execution screen, acquire information input to the application for a preset time or for a time during which at least part of the content is displayed on the application execution screen, generate personalized metadata for the content based on the acquired information, and map the personalized metadata to the content and store it in the electronic device.

[0205] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may acquire texts entered into the application during the preset time or during the time when at least a portion of the content is displayed on the execution screen of the application, extract at least one keyword from the acquired texts, and generate the personalized metadata including the extracted keyword.

[0206] For example, when the instructions are executed individually or collectively by the at least one processor, the electronic device may identify at least some of the acquired texts based on the context of the acquired texts, extract at least one keyword from the identified texts, and generate the personalized metadata including the extracted keyword.

[0207] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may display the content on the execution screens of a plurality of applications, extract at least one keyword from information input to each of the plurality of applications, and generate the personalized metadata including the extracted keyword.

[0208] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may display an interface screen for editing the personalized metadata on the display, and when a first user input for adding a keyword is received through the interface screen, add a keyword corresponding to the first user input to the personalized metadata, and when a second user input for deleting a keyword is received through the interface screen, delete a keyword corresponding to the second user input from among at least one keyword included in the personalized metadata.

[0209] For example, when the instructions are executed individually or collectively by the at least one processor, the electronic device may acquire information related to an application running in the background and generate personalized metadata including at least one keyword acquired from the information. The information related to the application may include at least one of identification information of the application and information stored through the application.

[0210] For example, the electronic device may store general metadata mapped to the content. The general metadata may include information obtained through image analysis of the content.

[0211] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may display an interface screen for searching content stored in the electronic device on the display, search for content corresponding to a search term entered through the interface screen based on general metadata and personalized metadata corresponding to each of the plurality of contents stored in the electronic device, and display at least one content searched based on the personalized metadata and at least one content searched based on the general metadata on the display.

[0212] For example, at least one piece of content retrieved based on the personalized metadata may be displayed in a first area of ​​the interface screen, and at least one piece of content retrieved based on the general metadata may be displayed in a second area of ​​the interface screen different from the first area.

[0213] A method for generating personalized metadata for an electronic device including a display according to one embodiment as described above may include: displaying an execution screen of an application on the display; displaying content selected based on user input on the execution screen of the application; acquiring information input to the application for a preset time or for a time during which at least a portion of the content is displayed on the execution screen of the application; generating personalized metadata for the content based on the acquired information; and mapping the personalized metadata to the content and storing it in the electronic device.

[0214] For example, the acquiring operation may include acquiring texts entered into the application during the preset time or during the time when at least a portion of the content is displayed on the execution screen of the application. The generating operation may include extracting at least one keyword from the acquired texts and generating personalized metadata including the extracted keyword.

[0215] For example, the acquiring operation may include an operation of identifying at least some of the acquired texts based on the context of the acquired texts. The generating operation may include an operation of extracting at least one keyword from the identified texts and an operation of generating personalized metadata including the extracted keyword.

[0216] For example, the metadata generation method may further include the operation of displaying the content on the execution screens of a plurality of applications, the operation of extracting at least one keyword from information input to each of the plurality of applications, and the operation of generating the personalized metadata including the extracted keyword.

[0217] For example, the metadata generation method may further include the operation of displaying an interface screen for editing the personalized metadata on the display, the operation of adding a keyword corresponding to the first user input to the personalized metadata when a first user input for adding a keyword is received through the interface screen, and the operation of deleting a keyword corresponding to the second user input from at least one keyword included in the personalized metadata when a second user input for deleting a keyword is received through the interface screen.

[0218] For example, the metadata generation method may further include the operation of obtaining information related to an application running in the background and the operation of generating the personalized metadata including at least one keyword obtained from said information. The information related to said application may include at least one of identification information of said application and information stored through said application.

[0219] For example, the electronic device may store general metadata mapped to the content. The general metadata may include information obtained through image analysis of the content.

[0220] For example, the metadata generation method may further include the operation of displaying an interface screen for searching content stored in the electronic device on the display, the operation of searching for content corresponding to a search term entered through the interface screen based on general metadata and personalized metadata corresponding to each of a plurality of contents stored in the electronic device, and the operation of displaying at least one content searched based on the personalized metadata and at least one content searched based on the general metadata on the display.

[0221] For example, at least one piece of content retrieved based on the personalized metadata may be displayed in a first area of ​​the interface screen, and at least one piece of content retrieved based on the general metadata may be displayed in a second area of ​​the interface screen different from the first area.

[0222] Various embodiments of this document may be implemented as software comprising one or more instructions stored in a storage medium (e.g., memory (620)) readable by a machine (e.g., electronic device (600)). For example, a processor (e.g., processor (610)) of the machine (e.g., electronic device (600)) may call at least one of the one or more instructions stored from the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0223] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). ™It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0224] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0225] Although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure.

Claims

1. In an electronic device (200), Display (230); Memory (220) for storing instructions; and It includes at least one processor (210) including a processing circuitry; and When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Display the execution screen of the application on the above display, and Displays selected content based on user input on the execution screen of the above application, and Acquire information input into the application during a preset time or during the time when at least a portion of the content is displayed on the execution screen of the application, and Based on the information obtained above, personalized metadata for the content is generated, and An electronic device that maps the personalized metadata to the content and stores it in the electronic device.

2. In Paragraph 1, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Acquire texts entered into the application during the above preset time or during the time when at least a portion of the content is displayed on the execution screen of the application, and Extract at least one keyword from the above-mentioned acquired texts, and An electronic device that generates the personalized metadata including the extracted keywords.

3. In Paragraph 2, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Identify at least some of the acquired texts based on the context of the acquired texts, and Extract at least one keyword from the above-mentioned identified texts, and An electronic device that generates the personalized metadata including the extracted keywords.

4. In any one of paragraphs 1 through 3, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, The above content is displayed on the execution screens of multiple applications, respectively, and Extract at least one keyword from the information input into each of the above plurality of applications, and An electronic device that generates the personalized metadata including the extracted keywords.

5. In any one of paragraphs 1 through 4, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, An interface screen for editing the personalized metadata is displayed on the display, and When a first user input for adding a keyword is received through the interface screen above, a keyword corresponding to the first user input is added to the personalized metadata, and An electronic device that, when a second user input for deleting a keyword is received through the interface screen, deletes a keyword corresponding to the second user input among at least one keyword included in the personalized metadata.

6. In any one of paragraphs 1 through 5, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Obtain information related to applications running in the background, and Generating the personalized metadata including at least one keyword obtained from the above information, Information related to the above application is, An electronic device comprising at least one of identification information of the above application and information stored through the above application.

7. In any one of paragraphs 1 through 6, The electronic device stores common metadata mapped to the content, and The above general metadata is an electronic device comprising information obtained through image analysis of the content.

8. In Paragraph 7, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, An interface screen for searching content stored in the electronic device is displayed on the display, and Based on general metadata and personalized metadata corresponding to each of the plurality of contents stored in the electronic device, content corresponding to a search term entered through the interface screen is searched, and An electronic device that displays at least one piece of content retrieved based on the personalized metadata and at least one piece of content retrieved based on the general metadata on the display.

9. In Paragraph 8, At least one piece of content retrieved based on the above personalized metadata is displayed in a first area of ​​the interface screen, and An electronic device in which at least one piece of content retrieved based on the above general metadata is displayed in a second area different from the first area on the interface screen.

10. A method for generating personalized metadata for an electronic device including a display, An operation (310) of displaying the execution screen of an application on the above display; An operation (320) of displaying selected content on the execution screen of the application based on user input; An operation (330) to acquire information input into the application during a preset time or during the time when at least a portion of the content is displayed on the execution screen of the application; An operation (340) of generating personalized metadata for the content based on the information obtained above; and A method for generating personalized metadata, comprising: an operation (350) of mapping the personalized metadata to the content and storing it in the electronic device.

11. In Paragraph 10, The above-mentioned acquisition operation is, The operation of acquiring texts entered into the application during the above-mentioned preset time or during the time when at least a portion of the content is displayed on the execution screen of the application; The operation of generating the above is, The operation of extracting at least one keyword from the above-mentioned acquired texts; and A method for generating personalized metadata, comprising: an operation of generating personalized metadata including the extracted keywords.

12. In Paragraph 11, The above-mentioned acquisition operation is, The operation of identifying at least some of the acquired texts based on the context of the acquired texts; is included, The operation of generating the above is, The operation of extracting at least one keyword from the identified texts; and A method for generating personalized metadata, comprising: an operation of generating personalized metadata including the extracted keywords.

13. In any one of paragraphs 10 through 12, The operation of displaying the above content on the execution screens of a plurality of applications, respectively; The operation of extracting at least one keyword from information input to each of the plurality of applications; and A method for generating personalized metadata, further comprising the operation of generating the personalized metadata including the extracted keywords.

14. In any one of paragraphs 10 through 13, The operation of displaying an interface screen for editing the personalized metadata on the display; When a first user input for adding a keyword is received through the interface screen, the operation of adding a keyword corresponding to the first user input to the personalized metadata; and A method for generating personalized metadata, further comprising: an operation of deleting a keyword corresponding to the second user input among at least one keyword included in the personalized metadata when a second user input for deleting a keyword is received through the interface screen; 15. In any one of paragraphs 10 through 14, An action of obtaining information related to an application running in the background; and The operation of generating the personalized metadata including at least one keyword obtained from the above information is further included, Information related to the above application is, A method for generating personalized metadata comprising at least one of identification information of the above application and information stored through the above application.

Images