Electronic device, method, and non-transitory computer-readable recording medium for assigning tag to content

Abstract

An electronic device is disclosed. The electronic device may identify one or more first images to which a tag defined by a user input is assigned. The electronic device may display multiple second images among multiple images via a display on the basis of characteristics of the one or more first images. The electronic device may assign the tag to the one or more first images and at least one third image selected by another user input among the multiple second images.

Description

Electronic device, method, and non-transitory computer-readable recording medium for tagging content

[0001] The following descriptions relate to electronic devices, methods, and non-transitory computer-readable recording media for tagging content.

[0002] Users can create or collect various content through electronic devices. Consequently, electronic devices may end up storing a large amount of content. Depending on this usage, electronic devices need to manage content through technology that categorizes or groups the content.

[0003] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0004] An electronic device is disclosed. The electronic device may include a display. The electronic device may include at least one processor comprising a processing circuit. The electronic device may include a memory storing instructions and including one or more storage media. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to identify one or more first images for assigning a tag defined by a user input. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display a plurality of second images from among a plurality of images through the display based on characteristics of the one or more first images. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to assign the tag to the one or more first images and at least one third image selected from among the plurality of second images by another user input.

[0005] A method is disclosed. It can be performed by an electronic device including a display. The method can include an operation of identifying one or more first images for assigning a tag defined by a user input. The method can include an operation of displaying a plurality of second images from among a plurality of images through the display based on characteristics of the one or more first images. The method can include an operation of assigning the tag to at least one third image selected by another user input from among the plurality of second images and the one or more first images.

[0006] A non-transitory computer-readable storage medium is disclosed. The non-transitory computer-readable storage medium may store a program including instructions. The instructions, when individually or collectively executed by at least one processor including a processing circuit of an electronic device including a display, may cause the electronic device to identify one or more first images for assigning a tag defined by a user input. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display a plurality of second images from among a plurality of images through the display based on characteristics of the one or more first images. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to assign the tag to the one or more first images and at least one third image selected from among the plurality of second images by another user input.

[0007] In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

[0008] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0009] FIG. 2A is a block diagram of an electronic device according to one embodiment.

[0010] FIG. 2b is a block diagram of a program of an electronic device according to one embodiment.

[0011] Figure 3 is a graph showing the relationships between objects.

[0012] FIG. 4A is a diagram illustrating an example of a screen of an electronic device according to one embodiment.

[0013] FIG. 4b is a diagram illustrating an example of a screen of an electronic device according to one embodiment.

[0014] FIG. 5 is a diagram illustrating an example of a screen of an electronic device according to one embodiment.

[0015] FIG. 6A is a diagram illustrating an example of a screen of an electronic device according to one embodiment.

[0016] FIG. 6b is a diagram illustrating an example of a screen of an electronic device according to one embodiment.

[0017] FIG. 6c is a diagram illustrating an example of a screen of an electronic device according to one embodiment.

[0018] FIG. 6d is a diagram illustrating an example of a screen of an electronic device according to one embodiment.

[0019] FIG. 7A is a diagram illustrating an example of a screen of an electronic device according to one embodiment.

[0020] FIG. 7b is a diagram illustrating an example of a screen of an electronic device according to one embodiment.

[0021] Figure 8 is a flowchart illustrating the operation of an electronic device according to one embodiment.

[0022] FIG. 9 is a flowchart illustrating the operation of an electronic device according to one embodiment.

[0023] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0024] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments.

[0025] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0026] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0027] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0028] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0029] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0030] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0031] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0032] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0033] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0034] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0035] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0036] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0037] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0038] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0039] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0040] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0041] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0042] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for realizing eMBB, a loss coverage (e.g., 664 dB or less) for realizing mMTC, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 6 ms or less for round trip) for realizing URLLC.

[0043] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0044] According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0045] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0046] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0047] FIG. 2A is a block diagram of an electronic device according to one embodiment.

[0048] In one embodiment, the electronic device (101) of FIG. 2A may correspond to the electronic device (101) of FIG. 1. Referring to FIG. 2A, the electronic device (101) may include a processor (120), a memory (130), an input module (150), a display (260), and a communication circuit (290).

[0049] In one embodiment, the processor (120) may be used to execute the operations of the electronic device (101) exemplified in the descriptions of FIGS. 8 and 9. For example, the processor (120) may include at least a portion of the processor (120) of FIG. 1 or may correspond to at least a portion of the processor (120) of FIG. 1. For example, the processor (120) may include one or more processors, including an application processor (AP) and / or a communication processor (CP). For example, the processor (120) may be implemented as a single chip, such as a system on chip (SoC), or may be implemented as multiple chips. For example, the processor (120) may be implemented as a single integrated circuit or may be implemented as multiple integrated circuits. For example, the processor (120) may be distributedly arranged within the electronic device (101). For example, the processor (120) may correspond to multiple processors that collectively perform multiple operations by dividing them among the processors.

[0050] In one embodiment, the memory (130) may (at least temporarily) store instructions for executing operations of the electronic device (101) exemplified in the descriptions of FIGS. 8 and 9. The instructions may be executed by the processor (120). The instructions may be included in one or more programs (140) stored in the memory (130). For example, the memory (130) may include at least a portion of the memory (130) of FIG. 1 (or at least a portion of the non-volatile memory (134)) or may correspond to at least a portion of the memory (130) of FIG. 1 (or at least a portion of the non-volatile memory (134)). For example, the memory (130) may include a main memory (e.g., random access memory (RAM) and / or read only memory (ROM)) within the electronic device (101), a register for the processor (120), a cache for the processor (120), a register for the communication circuit (290), a buffer (or soft buffer) for the communication circuit (290), and / or an auxiliary memory (e.g., a hard disk drive (HDD), a solid state drive (SSD)) of the electronic device (101). For example, the memory (130) may be implemented as a single chip or may be implemented as multiple chips. For example, the memory (130) may be implemented as one integrated circuit or may be implemented as multiple integrated circuits. For example, the memory (130) may be distributedly arranged within the electronic device (101).

[0051] In one embodiment, the input module (150) may include at least one physical key (e.g., buttons) for obtaining user input. For example, the input module (150) may correspond to at least a portion of the input module (150) of FIG. 1.

[0052] In one embodiment, the display (260) may display visual content (e.g., an image). For example, the display (260) may include at least a portion of the display module (160) of FIG. 1 or may correspond to at least a portion of the display module (160) of FIG. 1.

[0053] In one embodiment, the communication circuit (290) may be used for a communication connection between the electronic device (101) and another device (e.g., the electronic device (102), the server (108)). For example, the communication circuit (290) may include at least a portion of the communication module (190) (or the wireless communication module (192)) of FIG. 1, or may correspond to at least a portion of the communication module (190) (or the wireless communication module (192)) of FIG. 1. For example, the communication circuit (290) may include a communication circuit for a long-distance communication network. For example, the communication circuit (290) may be used to establish a communication link. For example, the communication circuit (290) may be implemented as a single chip or may be implemented as multiple chips. For example, the communication circuit (290) may be implemented as a single integrated circuit or may be implemented as multiple integrated circuits. For example, the communication circuit (290) may be distributed within the electronic device (101).

[0054] FIG. 2b is a block diagram of a program of an electronic device according to one embodiment.

[0055] Figure 2b can be explained with reference to Figures 1 and 2a.

[0056] Referring to FIG. 2b, the program (140) may include a plurality of applications (APPs) (211, 215), a plurality of tagging modules (221, 225), and a plurality of databases (DBs) (231, 233, 235, 237, 239). In one embodiment, the program (140) may include an image analysis module (241), a data collection module (243), an intent analysis module (245), and a UD tag interface (250).

[0057] In one embodiment, the plurality of tagging modules (221, 225) may include a predefined (PD) tag tagging module (221) and a user-defined (UD) tag tagging module (225). In one embodiment, the plurality of DBs (231, 233, 235, 237, 239) may include a PD tag DB (231), a user profile DB (233), a meta information DB (235), a UD tag profile DB (237), and a UD tag DB (239). In one embodiment, the predefined tag may be a tag that defines features that can be classified through content analysis and / or meta data analysis in the electronic device (101). In one embodiment, the user-defined tag may be distinguished from a tag based on features that can be classified in the electronic device (101). For example, user-defined tags may be tags entered by the user to define characteristics of content and classify content based on those characteristics. For example, at least some of the user-defined tags entered by the user may overlap with predefined tags.

[0058] Hereinafter, with reference to FIGS. 1, 2a, 2b, 3, 4a, 4b, 5, 6a, 6b, 6c, 6d, 7a, and 7b, an operation of an electronic device (101) attaching (or tagging) a PD tag to an image, an operation of collecting data for a UD tag based on data related to an application, and an operation of attaching (or tagging) a UD tag to an image according to a user's intention will be described. Herein, attaching (or tagging) a tag (e.g., a UD tag and / or a PD tag) to an image may be referred to as assigning (or designating) (or assigning) a tag (e.g., a UD tag and / or a PD tag) to an image.

[0059] Tagging of PD tags

[0060] In one embodiment, the image analysis module (241) may extract image features from images. In one embodiment, the image analysis module (241) may extract image features from images accessible through the first APP (211). In one embodiment, the first APP (211) may be a gallery application, but is not limited thereto. The first APP (211) may be any application (e.g., a web browser) capable of accessing images stored in the memory (130) of the electronic device (101) or the server (108). In one embodiment, the images may be images stored in the memory (130), but is not limited thereto. For example, the images may be images stored in the server (108) in association with a user account. In one embodiment, the image features may represent at least one object included in the image. In one embodiment, image features may represent a situation (e.g., travel, wedding, exercise, holiday, anniversary, specific person, specific place) depicted by the image and at least one object (e.g., a person (e.g., a user), an animal, an object) included in the image.

[0061] In one embodiment, the image analysis module (241) may extract image features from images using an image encoder for extracting image features. However, the present invention is not limited thereto. In one embodiment, the image analysis module (241) may extract image features from images using an artificial intelligence (AI) model (e.g., vision transformer (ViT)) trained to extract image features (or trained based on contrastive language-image pretraining (CLIP)). In one embodiment, CLIP may be a learning technique that quantifies the correlation between text and images. For example, through CLIP, a text encoder of an AI model may convert text into a text-encoded vector. For example, through CLIP, an image encoder of an AI model may convert an image into an image-encoded vector. For example, through CLIP, an AI model may calculate similarity (or cosine similarity) between vectors. In one embodiment, the AI ​​model used by the image analysis module (241) may be a pre-trained AI model. In one embodiment, the AI ​​model used by the image analysis module (241) may include a plurality of parameters related to a neural network having a structure based on an encoder and decoder, such as a transformer.

[0062] In one embodiment, the AI ​​model may be included within the electronic device (101) or may be included in an external device (e.g., a server (108)) and connected to the electronic device (101) via a network.

[0063] In one embodiment, the PD tag tagging module (221) may attach (or tag) PD tags to images based on image features extracted by the image analysis module (241). In one embodiment, the PD tag tagging module (221) may attach (or tag) words representing image features extracted by the image analysis module (241) as PD tags to images. In one embodiment, attaching (or tagging) a PD tag to an image may include the PD tag tagging module (221) generating PD tag information associated with the image. For example, associating PD tag information with an image may include storing the PD tag information as meta information of the image. For example, associating PD tag information with an image may include storing the PD tag information in a PD tag DB (231).

[0064] In one embodiment, the PD tag tagging module (221) may store PD tag information associated with an image in a PD tag DB (231). In one embodiment, the PD tag DB (231) may be implemented with a designated database management system (e.g., SQLlite). However, the present invention is not limited thereto. In one embodiment, the PD tag tagging module (221) may store PD tag information associated with an image in the meta information of the image. In one embodiment, the PD tag information may be stored as separate data dependent on the image and managed (e.g., stored, deleted, updated, transmitted) together with the image.

[0065] Data collection for UD tags

[0066] In one embodiment, the data collection module (243) can obtain information related to images through a plurality of APPs (211, 215). In one embodiment, the data collection module (243) can store information related to a user profile among information related to images through a plurality of APPs (211, 215) in a user profile DB (233). In one embodiment, the data collection module (243) can store image meta information among information related to images through a plurality of APPs (211, 215) in a meta information DB (235). In one embodiment, the user profile DB (233) and the meta information DB (235) can be implemented as a designated database management system (e.g., SQLlite). In one embodiment, the user profile DB (233) can store data in a format (e.g., knowledge graph) that represents associations between objects.

[0067] In one embodiment, information related to a user profile may include information related to a user's usage history of an APP (e.g., frequency of use of an APP, category (or type) of frequently used APP (e.g., shopping APP, social network service (SNS) APP, financial service APP, gallery APP), and / or usage history of an APP (e.g., in the case of a web browser, internet search history, in the case of an APP for multimedia playback, multimedia search history, multimedia viewing history, or type (or category) of viewed multimedia, and / or in the case of a gallery APP, image search history or image viewing history). In one embodiment, information related to a user profile may include information on PD tags attached to images stored in a PD tag DB (231). For example, information related to a user profile may include PD tags of images (e.g., PD tags indicating objects included in the image (e.g., people, animals, objects), and / or PD tags indicating situations depicted by the image (e.g., travel, wedding, exercise, holiday, anniversary)). For example, information related to a user profile may include information about images to which the same PD tag is attached (or, The number of images (or tags) attached to the PD tag, and / or the frequency with which images with the same PD tag are captured (e.g., average image capture cycle).

[0068] In one embodiment, the data collection module (243) may generate data indicating associations (or relationships) between one or more objects (or entities) (or nodes) based on information related to a user profile. In one embodiment, the data collection module (243) may generate data having a structure in which one or more objects (or entities) (or nodes) are connected by edges based on information related to a user profile. Hereinafter, data indicating associations (or relationships) will be described with reference to FIG. 3.

[0069] Figure 3 is a graph showing relationships between objects. Referring to Figure 3, a knowledge graph (300) may show a connection relationship between one or more objects (320, 330, 340, 350, 360, 370, and 380) related to a user profile (310).

[0070] For example, a user may frequently (or most frequently) take pictures of a dog (320). In this case, the relationship (311) between the user and the dog (320) may represent the subject of which the user frequently (or most frequently) takes pictures. For example, the most frequently (or most frequently) photographed breed of dog (320) included in the pictures may be a Pomeranian (330). In this case, the relationship (321) between the dog (320) and the Pomeranian (330) may represent the breed of which the picture is frequently taken. For example, the Pomeranian (330) included in the pictures may be defined with the highest probability as a companion dog (340). In this case, the relationship (331) between the Pomeranian (330) and the companion dog (340) may represent the role of the Pomeranian (330). For example, a pet dog (340) included in a photo may be frequently (or most frequently) photographed at a house (350) and / or Hangang Park (360). In this case, the relationship (341, 345) between the pet dog (340) and the house (350) and / or Hangang Park (360) may indicate the location where the pet dog (340) is most frequently photographed.

[0071] For example, a user may frequently (or most frequently) visit Hangang Park (360). In this case, the relationship (313) between the user and Hangang Park (360) may indicate the location that the user frequently (or most frequently) visits. Although FIG. 3 illustrates that there is no association between Hangang Park (360) and other objects, this is merely an example. In an embodiment, data indicating an association between Hangang Park (360) and one or more objects (e.g., food, friends) not illustrated in FIG. 3 may be stored in the user profile DB (233).

[0072] For example, a user may frequently (or most frequently) run Samsung Health (370). In this case, the relationship (315) between the user and Samsung Health (370) may indicate the application that the user frequently (or most frequently) runs. For example, Samsung Health (370) may be most frequently (or most frequently) run in Hangang Park (360). In this case, the relationship (371) between Samsung Health (370) and Hangang Park (360) may indicate the location where Samsung Health (370) is frequently run.

[0073] For example, a user may frequently (or most frequently) take pictures for exercise (380). Or, for example, a user may frequently (or most frequently) take pictures during exercise (380). In this case, the relationship (317) between the user and exercise (380) may represent the user's most frequent state at the time of taking pictures. For example, Samsung Health (370) may be executed most frequently (or most frequently) during exercise (380). In this case, the relationship (381) between exercise (380) and Samsung Health (370) may represent an application that was frequently executed during exercise (380).

[0074] In one embodiment, the data collection module (243) may store information about a knowledge graph (300) representing a connection relationship between one or more objects (320, 330, 340, 350, 360, 370, and 380) related to a user profile (310) in a user profile DB (233).

[0075] In one embodiment, the metadata of the image may include information about the photo being taken (e.g., the date and time taken, the location taken, the audio recording at the time of taking the photo (e.g., the type of audio being played, the conversation), the schedule at the time of taking the photo (e.g., the schedule recorded in a calendar application)).

[0076] In one embodiment, the meta information of the image may include creation information of the image (e.g., in the case of a screenshot image, screenshot-related information (e.g., the name of the APP on which the screenshot was taken (e.g., the name of the SNS (social network service) APP, the name of the web browser), the type (or purpose) of the APP on which the screenshot was taken (e.g., SNS, shopping, web browsing), information included in the screenshot (e.g., if the APP on which the screenshot was taken is an APP for web browsing, a website address, and / or if the APP on which the screenshot was taken is an APP for multimedia playback, a title of the video, or a category of the video)).

[0077] In one embodiment, the meta information of the image may include image sharing information (e.g., the application (e.g., SNS APP) and / or function (e.g., Quick Share) through which sharing was performed), information about the sharer (e.g., whether the person is saved in the contact list, and if so, information about the relationship with the user).

[0078] In one embodiment, the meta information of an image may include information related to the storage of the image (e.g., whether the image is stored in a path other than the default storage path of the image, the path in which the image is stored (e.g., the folder name), and commonalities between other images stored in the path in which the image is stored).

[0079] In one embodiment, the meta information of an image may include other information related to the image, such as whether the image has been edited (or corrected), and / or the number of times the image has been viewed.

[0080] In one embodiment, the data collection module (243) may encode each of one or more objects (or entities) (or nodes) into a vector (e.g., a text encoded vector). In one embodiment, the data collection module (243) may encode each of one or more objects (or entities) (or nodes) into a vector (e.g., a text encoded vector) through a text encoder (or a text encoder of an AI model trained through CLIP). In one embodiment, the data collection module (243) may assign a vector to each of one or more objects (or entities) (or nodes).

[0081] Tagging of UD tags

[0082] In one embodiment, the UD tag interface (250) may be a user interface (UI) (or graphical user interface (GUI)) displayed through a display (260). In one embodiment, the UD tag interface (250) may be a UI (or GUI) for creating (or modifying) and / or attaching (or tagging) a UD tag. In one embodiment, the UD tag interface (250) may be displayed on the display (260) based on the gallery application being executed. In one embodiment, the UD tag interface (250) may be displayed on the display (260) based on the acquisition of an input for an item (or an image object) (or a text object) (or an executable object) (or a UI) included in the screen of the gallery application. Hereinafter, an operation for displaying the UD tag interface (250) on the screen of the gallery application may be described with reference to FIG. 4A.

[0083] FIG. 4A is a diagram illustrating an example of a screen of an electronic device according to one embodiment.

[0084] Referring to FIG. 4A, as the gallery application is executed, a screen (401) may be displayed on the display (260) of the electronic device (101). An item (411) (or an image object) (or a text object) (or an executable object) (or a UI) for AI tagging (or a UD tag) may be displayed on the screen (401).

[0085] In one embodiment, in response to a user input (415) selecting one image (413) from among one or more images displayed through the screen (401), a screen (403) may be displayed on the display (260) of the electronic device (101) to display the image (413). The screen (403) may display an item (431) (or an image object) (or a text object) (or an executable object) (or a UI) for AI tagging (or a UD tag).

[0086] In one embodiment, the electronic device (101) may display a UD tag interface (250) on the display (260) in response to a user input for AI tagging (411) on the screen (401) or AI tagging (431) on the screen (403).

[0087] In one embodiment, the UD tag interface (250) may include an item for entering a name of a UD tag, an item for adding an image to which a UD tag will be attached (or tagged), and / or an item for recommending an image to which a UD tag will be attached (or tagged). Hereinafter, the UD tag interface (250) may be described with reference to FIG. 4B.

[0088] FIG. 4b is a diagram illustrating an example of a screen of an electronic device according to one embodiment.

[0089] Referring to FIG. 4B, a screen (405) representing a UD tag interface (250) may be displayed on a display (260) of an electronic device (101). An item (451) (or an image object) (or a text object) (or an executable object) (or a UI) for inputting a tag name (or a name of a UD tag) may be displayed on the screen (405). In one embodiment, a UD tag name may be obtained through a user input for the item (451) for inputting a tag name (or a name of a UD tag). For example, based on a user input for selecting the item (451) (e.g., a touch input for the item (451)), a UI (user interface) for inputting a tag name (or a name of a UD tag) may be displayed on the screen (405). Based on other user input to the UI for entering a tag name (or the name of a UD tag), the screen (405) may display the UD tag name in the item (451). In one embodiment, other UD tags, PD tags, or tags designated or generated by an AI model may be displayed on or adjacent to the item (451).

[0090] On the screen (405), an item (453) (or an image object) (or a text object) (or an executable object) (or a UI) for adding a photo to which a UD tag will be attached may be displayed. On the screen (405), an item (455) (or an image object) (or a text object) (or an executable object) (or a UI) for recommending a photo to which a UD tag will be attached may be displayed in an inactive state. In one embodiment, the item (455) for recommending a photo may be displayed in an activated state when a specified condition is met (e.g., one or more photos to which a UD tag will be attached (or tagged) is selected). However, the present invention is not limited thereto. On the screen (405), an item (455) (or an image object) (or a text object) (or an executable object) (or a UI) for recommending a photo to which a UD tag will be attached may not be displayed. An item (455) for recommending photos to which a UD tag will be attached may be displayed when a specified condition is met (e.g., one or more photos to which a UD tag will be attached (or tagged) are selected). In some embodiments, the UD tag interface (250) may be included as part of a gallery application. For example, the inclusion of the UD tag interface (250) as part of a gallery application may refer to the UD tag interface (250) being displayed on a display (260) as part of a screen that represents the execution result of the gallery application.

[0091] In one embodiment, in response to a user input selecting an item (453) on the screen (405), a screen (407) may be displayed. The screen (407) may display an item (471) (or an image object) (or a text object) (or an executable object) (or a UI) for querying a selection of a method for adding photos. In one embodiment, the item (471) for querying a selection of a method for adding photos may include an item (473) for finding in a gallery and an item (475) for finding with a prompt input. In one embodiment, the prompt may include data for guiding the selection of images for attaching (or tagging) a UD tag. In one embodiment, the prompt may be a task instruction for a generative artificial intelligence (AI) model (not shown). In one embodiment, the prompt may be a set of words (or a sentence including words) for selecting images for attaching (or tagging) a UD tag via the generative AI model. In one embodiment, the generative AI model may be a pre-trained AI model. In one embodiment, the generative AI model may include a plurality of parameters associated with a neural network having a structure based on an encoder and decoder, such as a transformer.

[0092] In one embodiment, in response to a user input selecting a Find in Gallery item (473), a list of images stored on the electronic device (101) or images associated with the user's account (e.g., images stored on a server) may be displayed on the display (260). The user may select an image (e.g., a photo, a video) from the list for which they wish to assign a tag (e.g., a UD tag).

[0093] In one embodiment, when the Add Photo item (453) is selected, instead of displaying an item (471) that asks for a selection of a method for adding photos, a list of images stored on the electronic device (101) or images linked to the user's account (e.g., images stored on a server) may be displayed on the display (260). The user may select an image (e.g., a photo, a video) for which he or she wishes to assign a tag (e.g., a UD tag) from the list. The Find with Prompt Input item (475) may be provided as a separate menu. In one embodiment, the electronic device (101) may display the images initially selected by the user through the UD tag interface (250).

[0094] In one embodiment, the intent analysis module (245) can identify the user's intent. In one embodiment, the intent analysis module (245) can identify the user's intent based on the user's input for the item (455) for recommending photos to which UD tags will be attached. For example, the user's intent may be related to a criterion for attaching a UD tag, but is not limited thereto. The intent analysis module (245) can identify the user's intent based on the user's input for the item (453) for adding photos to which UD tags will be attached. For example, the intent analysis module (245) can identify the user's intent based on some photos selected by the user from among photos displayed based on the user's input for the item (453) for adding photos to which UD tags will be attached. In one embodiment, the screen for selecting photos may be a screen for displaying a list of photos stored in the electronic device (101) (or linked to the account of the user of the electronic device (101)) as thumbnails.

[0095] In one embodiment, the intent analysis module (245) may select (or identify) one or more candidate images (or recommended images) for identifying the user's intent. In one embodiment, the intent analysis module (245) may select (or identify) one or more candidate images (or recommended images) based on characteristics (e.g., commonalities) of the images selected by the user input. In one embodiment, the intent analysis module (245) may select (or identify) one or more candidate images (or recommended images) that have commonalities (e.g., visual commonalities and / or non-visual commonalities) with the images selected by the user input. Hereinafter, the candidate images may be referred to as recommended images.

[0096] In one embodiment, the one or more candidate images may include at least one candidate image having visual commonality with the images selected by the user input. In one embodiment, the intent analysis module (245) may select (or identify) an image including at least one PD tag attached to a specified number or more of the PD tags of the selected images as a candidate image having visual commonality. For example, if the number of PD tags 'dog' among the PD tags of the selected images is a specified number or more, the intent analysis module (245) may identify an image to which the PD tag 'dog' is attached (or tagged) as a candidate image. In one embodiment, the intent analysis module (245) may select (or identify) an image having a similarity with each of the selected images that is equal to or greater than a reference similarity as a candidate image having visual commonality. In one embodiment, the similarity between the images may be identified based on the distance (or cosine similarity) between vectors (e.g., image encoded vectors) of the images. In one embodiment, the image encoded vector can be obtained by an image encoder of an AI model via CLIP.

[0097] In one embodiment, the one or more candidate images may include at least one candidate image that has non-visual commonality with the images selected by the user input.

[0098] In one embodiment, the intent analysis module (245) may identify (or select) at least one candidate image having non-visual commonality with the selected images based on data stored in the user profile DB (233). For example, the intent analysis module (245) may select (or identify) an image including an object having a commonality with the objects included in the selected images as a candidate image having non-visual commonality based on data indicating a commonality between objects (e.g., a knowledge graph). For example, the intent analysis module (245) may identify an image to which another PD tag (e.g., a park) having a commonality with the PD tag 'dog' of the selected images is attached (or tagged) as a candidate image.

[0099] For example, the intent analysis module (245) may select (or identify) an image having an image encoded vector having a high similarity to the text encoded vector of words (e.g., 'dog', 'Pomeranian', 'Hangang Park', 'Samsung Health') that are related to objects included in the selected images, based on data (e.g., a knowledge graph) indicating a relationship between objects, as a candidate image having non-visual commonalities. For example, the fact that a word is related to objects included in the selected images may refer to indicating a node connected to the objects included in the selected images through the knowledge graph (300).

[0100] In one embodiment, the intent analysis module (245) can identify (or select) at least one candidate image that has non-visual commonality with the selected images based on data stored in the meta information DB (235).

[0101] In one embodiment, the intent analysis module (245) may select (or identify) an image having the same (or similar) meta information as the meta information of the selected images as a candidate image having non-visual commonalities, based on the meta information of the images. For example, the intent analysis module (245) may select (or identify) an image having the same photo shooting information as the selected images, where at least some of the photo shooting information (e.g., shooting date, shooting time, and / or shooting location) of the selected images is the same, as a candidate image having non-visual commonalities. For example, the intent analysis module (245) may select (or identify) an image having the same image sharing information as the selected images, where at least some of the image sharing information of the selected images is the same, as a candidate image having non-visual commonalities. However, the present invention is not limited thereto.

[0102] In one embodiment, the intent analysis module (245) may present one or more candidate images to the user for identifying the user's intent. In one embodiment, the intent analysis module (245) may display one or more candidate images for identifying the user's intent through the display (260). In one embodiment, the intent analysis module (245) may display a UI for requesting selection of an image to which a UD tag is to be attached (or tagged) among the one or more candidate images through the display (260). Hereinafter, with reference to FIGS. 5 and 6A, an operation for displaying a UI for requesting selection of an image to which a UD tag is to be attached (or tagged) among the one or more candidate images on the screen of the gallery application may be described.

[0103] FIG. 5 is a diagram illustrating an example of a screen of an electronic device according to one embodiment.

[0104] Referring to FIG. 5, a screen (501) may be displayed on a display (260) of an electronic device (101) as a user selects one or more photos (510) to which a UD tag is to be attached (or tagged). In one embodiment, a screen (503) may be displayed in response to a user input selecting an item (455) on the screen (501).

[0105] On the screen (503), candidate photos (520, 530) selected according to different criteria can be displayed.

[0106] For example, the candidate photos (520) may include photos (521, 523, 525, 527) that have something in common with one or more photos (510). For example, the candidate photos (520) may include photos (521, 523, 525, 527) that have something in common visually (e.g., PD tag 'food') with one or more photos (510). For example, the candidate photos (520) may include photos (521, 523, 525, 527) that have something in common visually (e.g., PD tag 'food') and something in common non-visually (e.g., shooting location 'Nampo-dong') with one or more photos (510).

[0107] For example, the candidate photos (530) may include photos (531, 533, 535, 537) that have something in common with one or more photos (510). For example, the candidate photos (530) may include photos (531, 533, 535, 537) that have a visual commonality (e.g., PD tag 'food') with one or more photos (510). For example, the candidate photos (530) may include photos (531, 533, 535, 537) that have a visual commonality (e.g., PD tag 'food') and a non-visual commonality (e.g., PD tag (or object) 'recipe' that has an association) with one or more photos (510).

[0108] FIG. 6A is a diagram illustrating an example of a screen of an electronic device according to one embodiment. FIG. 6B is a diagram illustrating an example of a screen of an electronic device according to one embodiment.

[0109] Referring to FIG. 6A, a screen (601) may be displayed on a display (260) of an electronic device (101) as a user selects one or more photos (610) to which a UD tag is to be attached (or tagged). In one embodiment, a UD tag name indicated by an item (451) displayed on the screen (601) may be “Walk with Momorang.” In one embodiment, the UD tag name may be input by the user selecting the item (451) before selecting the photos (610). In one embodiment, the UD tag name may be input by the user selecting the item (451) before selecting the photo recommendation item (455) after selecting the photos (610). In one embodiment, the UD tag name may be input by automatically displaying a user interface (UI) for UD tag names after the user selects the photos (610). For example, when a user selects photos (610), a UI may be provided that includes words and / or dictionary tags recommended as user-defined tags based on information about the selected photos (610). For example, the user may create and / or designate one or more user-defined tags using the recommended words or dictionary tags, and the user may also designate additional dictionary tags in addition to the user-defined tags created by the user.

[0110] In one embodiment, the photos (610) may have a PD tag of 'dog' (or 'Pomeranian') (or 'companion dog'). In one embodiment, in response to a user input of selecting an item (455) on the screen (601), a screen (603) may be displayed. Candidate photos (620, 630, 640) selected according to different criteria may be displayed on the screen (603). For example, information indicating a correlation with one or more photos (610) may be displayed in an area where candidate photos (620, 630, 640) are displayed (e.g., 'Would you like to also add a photo of your companion dog below?', 'Would you like to also add a photo taken with your companion dog at Hangang Park below?', 'Would you like to add a Samsung Health photo below?'). For example, the candidate photos (620, 630, 640) may be displayed sequentially. For example, candidate photos (620) may be displayed based on the selection of one or more photos (610). For example, candidate photos (630) may be displayed based on the selection of at least one photo among the candidate photos (620). For example, candidate photos (640) may be displayed based on the selection of at least one photo among the candidate photos (630). The order in which the candidate photos (620, 630, 640) are listed on the screen may be based on a priority according to the association between each of the candidate photos (620, 630, 640) and one or more photos (610). For example, the presentation order of the candidate photos (620, 630, 640) may be determined in order of proximity in the knowledge graph (300). However, the present invention is not limited thereto.

[0111] For example, the candidate photos (620) may include photos (621, 623, 625, 627) that have something in common with one or more of the photos (610). For example, the candidate photos (620) may include photos (621, 623, 625, 627) that have a visual commonality (e.g., PD tag 'companion dog') with one or more of the photos (610). For example, a photo that includes an object included in two or more of the photos (610) may be included as the candidate photos (620). For example, a photo that includes a PD tag related to an object included in two or more of the photos (610) may be included as the candidate photos (620). In one embodiment, there may be at least one difference between one or more photos (610) and photos (621, 623, 625, 627) that have a visual commonality (e.g., PD tag 'pet'). For example, there may be no commonality between photos (621, 623, 625, 627) other than the commonality with one or more photos (610).

[0112] For example, the candidate photos (630) may include photos (631, 633, 635, 637) that have a visual commonality (e.g., PD tag 'companion dog') with one or more photos (610). For example, the candidate photos (630) may include photos (631, 633, 635, 637) that have a visual commonality (e.g., PD tag 'companion dog') and a non-visual commonality (e.g., shooting location 'Hangang Park') with one or more photos (610). In one embodiment, there may be at least one difference between the photos (631, 633, 635, 637) that have a visual commonality (e.g., PD tag 'companion dog'). For example, there may be no commonality between the photos (631, 633, 635, 637) other than what they have in common with one or more photos (610).

[0113] For example, the candidate photos (640) may include photos (641, 643, 645, 647) that have a non-visual commonality (e.g., a PD tag (or object) 'Samsung Health' that has an association) with one or more photos (610). In one embodiment, there may be at least one difference between the photos (641, 643, 645, 647) that have a visual commonality (e.g., a PD tag 'companion dog') with one or more photos (610). For example, there may be no commonality between the photos (641, 643, 645, 647) other than the commonality with one or more photos (610).

[0114] In one embodiment, the candidate photos (640) may include photos that have a shared history with other users who have shared one or more of the photos (610). In one embodiment, the candidate photos (640) may include photos that have a shared or created history through the same application as the application that shared or created one or more of the photos (610).

[0115] In one embodiment, the candidate photos (640) may include photos that have a PD tag that matches at least a portion of the content of a UD tag (e.g., a walk with Momorang) entered by the user (e.g., a photo that has a PD tag that includes 'exercise', which is similar to 'walk' included in the UD tag).

[0116] In one embodiment, the candidate photos (640) may include photos that have a connection relationship with one or more photos (610) according to the knowledge graph (300). For example, an application (e.g., an exercise log application) associated with exercise, which is an event that occurred at a time or place while walking a dog, may be identified, and an image containing a screen of the application may be included as one of the candidate photos (640).

[0117] In one embodiment, an image file may include attribute information (e.g., metadata) including the shooting date, location, lens type, camera manufacturer, aperture value, sensitivity, shutter speed, brightness, resolution, compression method, etc. The attribute information may include, for example, information supported by the exchangeable image file format (EXIF). For example, the candidate photos (640) may include one or more photos (610) and at least a specified number (e.g., two) of photos with the same attribute information.

[0118] In one embodiment, UD tags, PD tags, and / or AI-specified tags may be added to an image file as attribute information.

[0119] In one embodiment, on the screen (603), some photos (650) among candidate photos (620, 630, 640) selected according to different criteria based on user input may be selected. According to the embodiment, based on the completion of selection of some photos (650) among the selected candidate photos (620, 630, 640) (or, user input indicating the completion of selection is obtained), the electronic device (101) may display the screen (605) according to FIG. 6B. For example, referring to FIG. 6B, the screen (605) may display one or more photos (610) initially selected by the user, and additionally selected photos (631, 633, 635, 637, 641, 643, 645, 647) among the candidate photos (620, 630, 640). One or more photos and additional selected photos (650) may be assigned a UD tag (e.g., Walking with Momorang).

[0120] In one embodiment, one or more photos (610) and additional selected photos (650) may be additionally assigned a tag automatically determined by the electronic device (101) or an AI model in addition to a UD tag. For example, the screen (605) may display a tag automatically determined by the AI ​​model adjacent to the item (451).

[0121] In one embodiment, one or more photos (610) and additional selected photos (650) may be assigned additional UD tags.

[0122] In one embodiment, one or more photos (610) and additional selected photos (650) may be assigned a common PD tag.

[0123] In one embodiment, one or more of the photos (610) and additional selected photos (650) may be assigned a common UD tag (e.g., Walking with Momorang), and only some of the photos may be assigned other common UD tags and / or PD tags.

[0124] In Figures 6a and 6b, some of the candidate photos are shown as additionally selected, but this is merely an example. Depending on the embodiment, not all of the candidate photos may be additionally selected.

[0125] In one embodiment, the intent analysis module (245) may identify images selected from one or more candidate images as images to which a UD tag is attached (or tagged) through a UI requesting selection of an image.

[0126] In one embodiment, the intent analysis module (245) can identify a UD tagging criterion (hereinafter, UD tagging criterion) (or a profile for a UD tag) with respect to an arbitrary UD tag. In one embodiment, the UD tagging criterion (or a profile for a UD tag) can be identified based on images initially selected by the user, images additionally selected by the user from among candidate images, and / or images not selected by the user from among candidate images. For example, the intent analysis module (245) can identify a UD tagging criterion with respect to a UD tag based on commonalities (e.g., visual commonalities and / or non-visual commonalities) between images initially selected by the user. For example, the intent analysis module (245) can identify a UD tagging criterion with respect to a UD tag based on commonalities (e.g., visual commonalities and / or non-visual commonalities) between images initially selected by the user and / or images additionally selected. For example, the intent analysis module (245) may identify UD tagging criteria in relation to a UD tag based on commonalities among images initially selected by the user, additionally selected images, and / or unselected images. In one embodiment, commonalities among images initially selected by the user, additionally selected images, and / or unselected images may be related to criteria for excluding images in relation to a UD tag.

[0127] In one embodiment, the intent analysis module (245) can determine (or set) UD tagging criteria with respect to any UD tag as one or more conditions. In one embodiment, the intent analysis module (245) can determine (or set) UD tagging criteria with respect to any UD tag based on one or more vectors (e.g., text encoded vectors and / or image encoded vectors). In one embodiment, the one or more vectors can be referred to as vectors for UD tags. In one embodiment, the one or more vectors can be vectors representing visual commonalities of images to which UD tags are attached. In one embodiment, the one or more vectors can be vectors (or multimodal vectors (e.g., text encoded vectors and / or image encoded vectors)) that include vectors representing visual commonalities and vectors representing non-visual commonalities.

[0128] In one embodiment, the intent analysis module (245) may ask the user whether to store UD tagging criteria in relation to any UD tag.

[0129] FIG. 6C is a diagram illustrating an example of a screen of an electronic device according to an embodiment. For example, the intent analysis module (245) of FIG. 6C may display a menu (607) via the display (260) that prompts the user to store UD tagging criteria in relation to an arbitrary UD tag.

[0130] In one embodiment, the intent analysis module (245) may store UD tagging criteria in the UD tag profile DB (237) in relation to any UD tag. In one embodiment, the intent analysis module (245) may store UD tagging criteria (or UD tag profile) in the UD tag profile DB (237) in response to a user input (e.g., a user input of 'Yes') selecting to store UD tagging criteria in relation to any UD tag for the menu (607).

[0131] In one embodiment, the UD tag tagging module (225) may attach (or tag) UD tags to images stored in the memory (130) of the electronic device (101) or the server (108) based on conditions (or criteria) for attaching (or tagging) UD tags.

[0132] For example, the UD tag tagging module (225) can attach (or tag) a UD tag to an image that satisfies one or more conditions. For example, the UD tag tagging module (225) can attach (or tag) a UD tag to an image if the similarity between a vector of the image and a vector for a UD tag (e.g., a text encoded vector and / or an image encoded vector) is greater than or equal to a reference similarity. For example, the UD tag tagging module (225) can attach (or tag) a UD tag to an image if the similarity between a vector of the image and a vector for a UD tag is greater than or equal to a reference similarity, and the image has non-visual commonalities with an image to which a UD tag is attached.

[0133] In one embodiment, the UD tag tagging module (225) may store UD tag information associated with an image in a UD tag DB (239). In one embodiment, the UD tag DB (239) may be implemented with a designated database management system (e.g., SQLlite). However, the present invention is not limited thereto. In one embodiment, the UD tag tagging module (225) may store UD tag information associated with an image in the meta information of the image. In one embodiment, the UD tag information may be stored as separate data dependent on the image and managed (e.g., stored, deleted, updated, transmitted) together with the image. In FIG. 2B, the UD tag DB (239) and the PD tag DB (231) are illustrated as different DBs, but this is merely an example. According to one embodiment, the electronic device (101) may manage data stored in the UD tag DB (239) and data stored in the PD tag DB (231) in a single DB. Hereinafter, with reference to FIG. 6d, an operation of the UD tag tagging module (225) to tag a new image with a UD tag based on the UD tagging criteria can be described.

[0134] FIG. 6d is a diagram illustrating an example of a screen of an electronic device according to one embodiment.

[0135] Referring to FIG. 6d, as new photos are added (or taken), the UD tag tagging module (225) of the electronic device (101) can identify (or determine) whether the new photos are tagged with UD tags.

[0136] For example, the UD tag tagging module (225) can determine whether to tag new photos with UD tags using UD tagging criteria based on one or more photos (610) initially selected by the user and additionally selected photos (631, 633, 635, 637, 641, 643, 645, 647). For example, the UD tag tagging module (225) can determine whether to tag new photos with UD tags using UD tagging criteria based on one or more photos (610) initially selected by the user, additionally selected photos (631, 633, 635, 637, 641, 643, 645, 647), and unselected photos (621, 623, 625, 627).

[0137] In one embodiment, referring to screen (609), photos (660) selected based on UD tagging criteria may be attached (or tagged) with the UD tag 'Walking with Momorang'. For example, screen (609) may display one or more photos (610) initially selected by the user, additionally selected photos (631, 633, 635, 637, 641, 643, 645, 647), and automatically added photos (661, 663, 665).

[0138] As described above, the electronic device (101) can tag UD tags based on user input. Accordingly, the electronic device (101) can attach phrases (or concepts) that are difficult to attach as PD tags to images as UD tags, depending on the user's intention.

[0139] Hereinafter, with reference to FIGS. 7a and 7b, other user inputs (e.g., prompt inputs) for tagging UD tags and UD tagging criteria stored in the UD tag profile DB (237) based on other user inputs are described.

[0140] FIG. 7A is a diagram illustrating an example of a screen of an electronic device according to one embodiment. FIG. 7B is a diagram illustrating an example of a screen of an electronic device according to one embodiment.

[0141] In one embodiment, the electronic device (101) may display a UD tag interface (250) on a display (260). In one embodiment, the UD tag interface (250) may include an item for entering a name of a UD tag, an item for adding an image to which a UD tag will be attached (or tagged), and / or an item for recommending an image to which a UD tag will be attached (or tagged).

[0142] In one embodiment, a screen (e.g., screen 405 of FIG. 4B) representing a UD tag interface (250) may be displayed on a display (260) of an electronic device (101). The screen (405) may display an item (451) (or an image object) (or a text object) (or an executable object) (or a UI) for entering a tag name (or a name of a UD tag). The screen (405) may display an item (453) (or an image object) (or a text object) (or an executable object) (or a UI) for adding a photo to which a UD tag will be attached. The screen (405) may display an item (455) (or an image object) (or a text object) (or an executable object) (or a UI) for recommending a photo to which a UD tag will be attached.

[0143] In one embodiment, in response to a user input selecting an item (453) on the screen (405), a screen (407) may be displayed. Referring to FIG. 7A, the screen (407) may display an item (471) (or an image object) (or a text object) (or an executable object) (or a UI) for querying a selection of a method for adding photos. In one embodiment, the item (471) for querying a selection of a method for adding photos may include an item (473) for finding in a gallery and an item (475) for finding with a prompt input. In one embodiment, the prompt may include data for guiding the selection of images for attaching (or tagging) a UD tag. In one embodiment, the prompt may be a task instruction for a generative AI model (not shown). In one embodiment, the prompt may be a set of words (or a sentence including words) for selecting images for attaching (or tagging) a UD tag via the generative AI model. In one embodiment, the generative AI model may be a pre-trained AI model. In one embodiment, the generative AI model may include a plurality of parameters associated with a neural network having a structure based on an encoder and decoder, such as a transformer.

[0144] In one embodiment, the electronic device (101) may display a screen (701) on the display (260) based on an input for an item (475) to be found as a prompt input. In one embodiment, the screen (701) may include an item (710) for entering a prompt (e.g., 'Insert pictures related to my exercise'). In one embodiment, the prompt may be obtained through a touch input on the display (260). In one embodiment, the prompt may be obtained through an input (e.g., keyboard input, gesture input, and / or voice input) via the input module (150).

[0145] In one embodiment, the electronic device (101) can perform one or more tasks by inputting a prompt to a generative AI model. In one embodiment, the electronic device (101) can perform a task to access one or more photos stored in the electronic device (101) (or linked to a user account of the electronic device (101)) by inputting the prompt to the generative AI model. In one embodiment, the electronic device (101) can perform a task to identify (or select) some photos that have something in common with the one or more accessed photos and the prompt (e.g., 'Insert photos related to my exercise') by inputting the prompt to the generative AI model. In one embodiment, the electronic device (101) can perform a task to display some of the identified (or selected) photos via the UD tag interface (250) by inputting the prompt to the generative AI model.

[0146] In one embodiment, the generative AI model can access the user's photos by exchanging specific data with the electronic device (101) (e.g., processor (120)). For example, the generative AI model that has received the prompt can request the electronic device (101) for information necessary to access the user's photos (e.g., storage location, account information, etc.) and access the user's photos using the information necessary to access the user's photos provided by the electronic device (101). In one embodiment, the generative AI model that has received the prompt can be pre-allowed to access the user's photos, and thus can access photos in a pre-designated location and perform an action according to the prompt.

[0147] In one embodiment, the generative AI model may inform (or provide) the electronic device (101) of identifiers (e.g., storage locations, file names, EXIF ​​properties, PD tags, etc.) for the photos (e.g., 610, 620, 630, 640, 720, 730, or 740) to be displayed on the user's screen.

[0148] In one embodiment, the electronic device (101) may display images identified (or selected) based on a prompt via the UD tag interface (250).

[0149] In one embodiment, the intent analysis module (245) can identify the user's intent. In one embodiment, the intent analysis module (245) can identify the user's intent based on the user's input regarding an item for recommending a photo to which a UD tag will be attached. For example, the user's intent may be related to a criterion for attaching a UD tag.

[0150] In one embodiment, the intent analysis module (245) may select (or identify) one or more candidate images for identifying the user's intent. In one embodiment, the intent analysis module (245) may select (or identify) one or more candidate images that have commonalities (e.g., visual commonalities and / or non-visual commonalities) with the images selected by the user input.

[0151] In one embodiment, the one or more candidate images may include at least one candidate image having visual commonality with the images selected by the user input. In one embodiment, the intent analysis module (245) may select (or identify) an image that includes at least one PD tag attached to a specified number or more of the PD tags of the selected images as a candidate image having visual commonality. For example, the intent analysis module (245) may identify an image to which the PD tag 'dog' is attached (or tagged) as a candidate image if the number of PD tags 'dog' among the PD tags of the selected images is a specified number or more. In one embodiment, the intent analysis module (245) may select (or identify) an image having a similarity with each of the selected images that is equal to or greater than a reference similarity as a candidate image having visual commonality. In one embodiment, the similarity between the images may be identified based on the distance (or cosine similarity) between vectors (e.g., image encoded vectors) of the images.

[0152] In one embodiment, the one or more candidate images may include at least one candidate image that has non-visual commonality with the images selected by the user input.

[0153] In one embodiment, the intent analysis module (245) may identify (or select) at least one candidate image having non-visual commonality with the selected images based on data stored in the user profile DB (233). For example, the intent analysis module (245) may select (or identify) an image including an object having a commonality with the objects included in the selected images as a candidate image having non-visual commonality based on data indicating a commonality between objects (e.g., a knowledge graph). For example, the intent analysis module (245) may identify an image to which another PD tag (e.g., a park) having a commonality with the PD tag 'dog' of the selected images is attached (or tagged) as a candidate image.

[0154] In one embodiment, the intent analysis module (245) can identify (or select) at least one candidate image that has non-visual commonality with the selected images based on data stored in the meta information DB (235).

[0155] In one embodiment, the intent analysis module (245) may present one or more candidate images to the user for identifying the user's intent. In one embodiment, the intent analysis module (245) may display one or more candidate images for identifying the user's intent through the display (260).

[0156] Referring to FIG. 7A, one or more photos (720) selected according to a prompt may be displayed on a screen (703). In one embodiment, the UD tag name indicated by the item (451) displayed on the screen (703) may be "wod" (workout of the day). In one embodiment, the photos (720) may have a PD tag of "workout" (or "Samsung Health").

[0157] In one embodiment, a user may first input "wod" in the UD tag entry (451) and then input "Insert my workout related photos" as a prompt. The electronic device (101) may transmit the prompt including "wod" to the AI ​​model. The AI ​​model may select photos that match "wod" among the user's workout related photos (e.g., a verification photo taken once a day after completing a similar type of workout that is repeated periodically). In one embodiment, the user may input the UD tag entry (451) after selecting photos (e.g., 720, 730, or 740).

[0158] In one embodiment, the screen (703) may display candidate photos (730, 740) selected according to different criteria.

[0159] For example, candidate photos (730, 740) may include photos that have a visual commonality (e.g., PD tag 'exercise') with one or more photos (720). For example, candidate photos (730) may include photos that have a visual commonality (e.g., PD tag 'exercise') and a non-visual commonality (e.g., PD tag 'Samsung Health' according to association) with one or more photos (710). For example, candidate photos (740) may include photos that have a visual commonality (e.g., PD tag 'exercise') and a non-visual commonality (e.g., PD tag (or object) 'person' according to association) with one or more photos (720).

[0160] In one embodiment, the electronic device (101) can input a prompt (e.g., 'Remove the photo with another person in the background') through an item (715) for inputting a prompt. In one embodiment, the prompt can be obtained through a touch input on the display (260). In one embodiment, the prompt can be obtained through an input (e.g., keyboard input, gesture input, and / or voice input) through the input module (150).

[0161] Referring to FIG. 7B, the electronic device (101) may display a screen (705) in which a prompt (750) is input. In one embodiment, referring to the screen (707), the electronic device (101) may select, based on the prompt (750), photos (771, 773, 775, 777) among photos with the UD tag 'wod' attached that meet the conditions according to the prompt (e.g., photos with another person in the background). In one embodiment, referring to the screen (707), the electronic device (101) may display a menu (760) on the screen (707) for asking whether to remove the selected photos from the tag.

[0162] In one embodiment, the electronic device (101) may delete (e.g., de-designate) the UD tag 'wod' of photos (771, 773, 775, 777) based on a user input for a menu (760). In one embodiment, the electronic device (101) may delete the UD tag 'wod' of photos (771, 773, 775, 777) based on a user input for some items of the menu (760) (e.g., apply again next time, just this once). In one embodiment, the intent analysis module (245) may update the UD tagging criteria for the user's intention to delete the UD tag 'wod' of photos (771, 773, 775, 777) (i.e., the intention to 'exclude photos with other people in the background'). For example, the electronic device (101) may update the UD tagging criteria for the user's intention to delete the UD tag 'wod' of photos (771, 773, 775, 777) (i.e., the intention to 'exclude photos with other people in the back') based on user input for some items of the menu (760) (e.g., also applies to the following).

[0163] According to an embodiment, the electronic device (101) may receive an image with a tag attached from another electronic device of another user via the communication module (190). The electronic device (101) may transmit (or suggest to transmit) (or recommend to transmit) another image with a UD tag attached, related to a tag already attached to the image received from the other electronic device of the other user, to the electronic device of the other user via the communication module (190). In one embodiment, the electronic device (101) may change the UD tag to another tag (e.g., a tag already attached to the image received from the other electronic device, or another third tag) and then transmit the image to the other electronic device. In one embodiment, the change (or conversion) of the tag may take into consideration universality and privacy. For example, the third tag to be changed may be a universal tag. If the UD tag is determined to have high privacy, the electronic device (101) may change the UD tag to another tag (e.g., a tag already attached to an image received from another electronic device, or another third-party tag) and then transmit the image to the other electronic device. According to an embodiment, when displaying an image (or thumbnail image) for previewing an image with a tag attached from another electronic device of another user through the communication module (190), the electronic device (101) may display the UD tag to be assigned to the image together with the image (or thumbnail image) for previewing.

[0164] In one embodiment, the electronic device (101) may display different types of tags (e.g., UD tags, PD tags, tags automatically assigned by AI) in a gallery application so as to be distinguished from each other (e.g., by applying different colors and / or fonts).

[0165] As described above, the electronic device (101) has been exemplified as tagging an image (or photograph) with a UD tag, but this is merely an example. According to an embodiment, the electronic device (101) may attach UD tags to, in addition to images, videos, audio (e.g., voice, music), documents, web pages, contacts, and / or objects recognized by an XR (extended reality) device.

[0166] According to an embodiment, the electronic device (101) may include audio (e.g., voice, music) as content to be tagged. The electronic device (101) may recommend audio to a user to be tagged (e.g., PD tag, UD tag) or automatically assign tags using information included in the audio (e.g., content of the voice, speaker, recording location, lyrics, genre, artist, number of plays, playback location, playback time).

[0167] According to an embodiment, the electronic device (101) may include a document as content to be tagged. The electronic device (101) may recommend a document to be tagged (e.g., PD tag, UD tag) to a user or automatically assign a tag using information included in the document (e.g., content of the text, attached images, type of text, author, writing location, writing time, sharing history, number of views).

[0168] According to an embodiment, the electronic device (101) may include an XR object (e.g., a person or an object recognized by an XR device) or an XR scene (e.g., a scene recognized by an XR device) as content to be tagged. The electronic device (101) may recommend an object or scene to be tagged (e.g., a PD tag or a UD tag) to a user or automatically assign a tag using information collected through the XR device (e.g., an object, movement of an object, background, location, time).

[0169] Figure 8 is a flowchart illustrating the operation of an electronic device according to one embodiment.

[0170] FIG. 8 can be described with reference to FIG. 1, FIG. 2a, FIG. 2b, FIG. 3, FIG. 4a, FIG. 4b, FIG. 5, FIG. 6a, FIG. 6b, FIG. 6c, FIG. 6d, FIG. 7a, and FIG. 7b.

[0171] In the following examples, the operations may be performed sequentially, but are not necessarily sequential. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0172] Referring to FIG. 8, in operation 810, according to an embodiment, the electronic device (101) may obtain an input for a user-defined (UD) tag. For example, the electronic device (101) may obtain an input for a UD tag (e.g., user input for AI tagging (411) or AI tagging (431)) through the UD tag interface (250). For example, the electronic device (101) may obtain an input for a UD tag (e.g., user input for an item to find in a gallery (473) and / or an item to find with a prompt input (475)) through the UD tag interface (250).

[0173] In operation 820, according to one embodiment, the electronic device (101) may identify an image to be tagged with a UD tag. For example, the electronic device (101) may identify an image to be tagged with a UD tag among a plurality of images based on a user input.

[0174] For example, the user input may be an input for selecting an image on a screen for selecting images displayed on the display (260) by selecting an item (473) to find in the gallery. For example, the electronic device (101) may identify the images selected by the user as images to be tagged with a UD tag.

[0175] For example, the user input may be input of a prompt after selecting an item (475) to search for with the prompt input. For example, the electronic device (101) may identify an image to be tagged with a UD tag based on the similarity between the text encoded vector for the prompt and the image encoded vector of each of the plurality of images.

[0176] In operation 830, according to one embodiment, the electronic device (101) may display a plurality of recommended images associated with the identified image.

[0177] In one embodiment, the electronic device (101) may select (or identify) one or more recommended images that have commonalities (e.g., visual commonalities and / or non-visual commonalities) with images selected by a user input. In one embodiment, the one or more recommended images may include at least one recommended image that has visual commonalities with the selected images. In one embodiment, the electronic device (101) may select (or identify) an image that includes at least one PD tag attached to a specified number or more of the PD tags of the selected images as a recommended image that has visual commonalities. In one embodiment, the electronic device (101) may select (or identify) an image that has a similarity with each of the selected images that is equal to or greater than a reference similarity as a recommended image that has visual commonalities. In one embodiment, the similarity between the images may be identified based on the distance (or cosine similarity) between vectors (e.g., image encoded vectors) of the images. In one embodiment, the image encoded vector can be obtained by an image encoder of an AI model via CLIP.

[0178] In one embodiment, the electronic device (101) may identify (or select) at least one recommended image having a non-visual commonality with the selected images based on data stored in the user profile DB (233). For example, the electronic device (101) may select (or identify) an image including an object having a commonality with the objects included in the selected images as a recommended image having a non-visual commonality based on data indicating a commonality between objects (e.g., a knowledge graph). For example, the electronic device (101) may identify an image to which another PD tag (e.g., a park) having a commonality with the PD tag 'dog' of the selected images is attached (or tagged) as a recommended image.

[0179] For example, the electronic device (101) may select (or identify) an image having an image encoded vector having a high similarity with text encoded vectors of words (e.g., 'dog', 'Pomeranian', 'Hangang Park', 'Samsung Health') representing objects included in the selected images, based on data (e.g., a knowledge graph) representing associations between objects, as a recommended image having non-visual commonalities.

[0180] In one embodiment, the electronic device (101) can identify (or select) at least one recommended image that has non-visual commonality with the selected images based on data stored in the meta information DB (235).

[0181] In one embodiment, the electronic device (101) may select (or identify) an image having the same (or similar) meta information as the meta information of the selected images, as a recommended image having non-visual commonalities, based on the meta information of the images. For example, the electronic device (101) may select (or identify) an image having the same photo shooting information as the selected images, where at least some of the photo shooting information (e.g., shooting date, shooting time, and / or shooting location) of the selected images is the same, as a recommended image having non-visual commonalities. For example, the electronic device (101) may select (or identify) an image having the same image sharing information as the selected images, where at least some of the image sharing information of the selected images is the same, as a recommended image having non-visual commonalities. However, the present invention is not limited thereto.

[0182] In one embodiment, the electronic device (101) may present one or more recommended images to the user for identifying the user's intention. In one embodiment, the electronic device (101) may display one or more recommended images for identifying the user's intention through the display (260). In one embodiment, the electronic device (101) may display a UI for requesting selection of an image to which a UD tag is to be attached (or tagged) among the one or more recommended images through the display (260).

[0183] In operation 840, according to one embodiment, the electronic device (101) may assign a user-defined tag based on a selected image and an identified image among a plurality of recommended images.

[0184] In one embodiment, the electronic device (101) can identify a profile for a UD tag based on selected images and identified images from among a plurality of recommended images assigned user-defined tags. In one embodiment, the profile for a UD tag can be identified based on images initially selected by the user, images additionally selected by the user from among candidate images, and / or images not selected by the user from among candidate images. For example, the electronic device (101) can identify a UD tagging criterion in relation to a UD tag based on commonalities (e.g., visual commonalities and / or non-visual commonalities) between images initially selected by the user. For example, the electronic device (101) can identify a UD tagging criterion in relation to a UD tag based on commonalities (e.g., visual commonalities and / or non-visual commonalities) between images initially selected by the user and / or images additionally selected. For example, the electronic device (101) may identify UD tagging criteria in relation to a UD tag based on commonalities among images initially selected by the user, additionally selected images, and / or unselected images. In one embodiment, the commonalities among images initially selected by the user, additionally selected images, and / or unselected images may be related to criteria for excluding images in relation to a UD tag.

[0185] In one embodiment, the electronic device (101) may determine (or set) UD tagging criteria with respect to an arbitrary UD tag based on one or more conditions. In one embodiment, the electronic device (101) may determine (or set) UD tagging criteria with respect to an arbitrary UD tag based on one or more vectors (e.g., text encoded vectors and / or image encoded vectors). In one embodiment, the one or more vectors may be referred to as vectors for UD tags. In one embodiment, the one or more vectors may be vectors representing visual commonalities of images to which UD tags are attached. In one embodiment, the one or more vectors may be vectors (or multimodal vectors (e.g., text encoded vectors and / or image encoded vectors)) that include vectors representing visual commonalities and vectors representing non-visual commonalities.

[0186] FIG. 9 is a flowchart illustrating the operation of an electronic device according to one embodiment.

[0187] FIG. 9 can be explained with reference to FIG. 1, FIG. 2a, FIG. 2b, FIG. 3, FIG. 4a, FIG. 4b, FIG. 5, FIG. 6a, FIG. 6b, FIG. 6c, FIG. 6d, FIG. 7a, and FIG. 7b.

[0188] In the following examples, the operations may be performed sequentially, but are not necessarily sequential. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0189] The operations of FIG. 9 may be performed after operation 840 of FIG. 8.

[0190] Referring to FIG. 9, in operation 910, according to one embodiment, the electronic device (101) may identify a new image. In one embodiment, the electronic device (101) may identify that a new image is captured through the camera module (180). In one embodiment, the electronic device (101) may identify that a new image is shared through the communication module (190).

[0191] In operation 920, according to one embodiment, the electronic device (101) may tag a new image with a UD tag based on a profile for a UD tag. The electronic device (101) may tag a new image with a UD tag based on a UD tagging criterion (or UD tag profile). For example, the electronic device (101) may attach (or tag) a UD tag to new images stored in the memory (130) of the electronic device (101) based on a condition (or criterion) for attaching (or tagging) a UD tag.

[0192] For example, the electronic device (101) can attach (or tag) a UD tag to an image that satisfies one or more conditions. For example, the electronic device (101) can attach (or tag) a UD tag to an image if the similarity between a vector of the image and a vector for a UD tag is greater than or equal to a reference similarity. For example, the electronic device (101) can attach (or tag) a UD tag to an image if the similarity between a vector of the image and a vector for a UD tag is greater than or equal to a reference similarity and the image has non-visual commonalities with an image to which a UD tag is attached. However, the present invention is not limited thereto.

[0193] As described above, the electronic device (101) may include a display (260). The electronic device (101) may include at least one processor (120) including a processing circuit. The electronic device (101) may include a memory (130) storing instructions and including one or more storage media. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to identify one or more first images (610) for assigning a tag defined by a user input. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display, through the display (260), a plurality of second images (620, 630, 640) from among a plurality of images based on characteristics of the one or more first images (610). The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to assign the tag to at least one third image (631, 633, 635, 637, 641, 643, 645, 647) selected by another user input from among the plurality of second images (620, 630, 640) and the one or more first images (610).

[0194] The above plurality of second images (620, 630, 640) may have at least one non-visual commonality or visual commonality with one or more of the first images (610).

[0195] Among the plurality of second images (620, 630, 640), some of the second images having visual commonality with one or more of the first images (610) may include the first object included in the one or more of the first images (610).

[0196] Among the plurality of second images (620, 630, 640), a tag that is distinct from the tag of one or more of the first images (610) and other second images having the non-visual commonality, and / or a part of the meta information may correspond to a tag that is distinct from the tag of one or more of the first images (610), and / or a part of the meta information.

[0197] The above instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to determine whether to assign the tag to a fourth image acquired through the camera based on commonalities between the one or more first images (610) and the at least one third image (631, 633, 635, 637, 641, 643, 645, 647).

[0198] The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to identify whether to de-tag at least one of the at least one third image to which the tag has been assigned and at least one of the one or more first images. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to determine whether to de-tag a fifth image acquired through the camera based on a difference between the at least one third image to which the tag has been assigned and the one or more first images and the at least one image to which the tag has been de-tagned.

[0199] The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to determine whether to assign the tag to an image based on commonalities between the one or more first images (610) and the remaining images of the plurality of second images (620, 630, 640) excluding the third image (631, 633, 635, 637, 641, 643, 645, 647).

[0200] The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to generate a prompt based on the user input. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to identify the one or more first images (610) by performing the prompt based on an artificial intelligence (AI) model.

[0201] The plurality of second images (620, 630, 640) may be displayed in different areas of the display (260) for different categories. Each of the different areas may include a text object representing a different one of the different categories.

[0202] As described above, the method may be performed by an electronic device (101) including a display (260). The method may include an operation of identifying one or more first images (610) for assigning a tag defined by a user input. The method may include an operation of displaying a plurality of second images (620, 630, 640) from among a plurality of images through the display (260) based on characteristics of the one or more first images (610). The method may include an operation of assigning the tag to at least one third image (631, 633, 635, 637, 641, 643, 645, 647) selected from among the plurality of second images (620, 630, 640) by another user input and the one or more first images (610).

[0203] The above plurality of second images (620, 630, 640) may have at least one non-visual commonality or visual commonality with one or more of the first images (610).

[0204] Among the plurality of second images (620, 630, 640), some of the second images having visual commonality with one or more of the first images (610) may include the first object included in the one or more of the first images (610).

[0205] Among the plurality of second images (620, 630, 640), a tag that is distinct from the tag of one or more of the first images (610) and other second images having the non-visual commonality, and / or a part of the meta information may correspond to a tag that is distinct from the tag of one or more of the first images (610), and / or a part of the meta information.

[0206] The method may include an operation of determining whether to assign a tag to a fourth image acquired through a camera based on commonalities between the one or more first images (610) and the at least one third image (631, 633, 635, 637, 641, 643, 645, 647).

[0207] The method may include an operation of identifying whether to release the tag from at least one of the at least one third image to which the tag has been assigned and the at least one first image. The method may include an operation of determining whether to release the tag from a fifth image acquired through the camera based on a difference between the at least one third image to which the tag has been assigned and the at least one first image and the at least one image to which the tag has been released.

[0208] The method may include an operation of determining whether to assign the tag to an image based on commonalities between the one or more first images (610) and the remaining images excluding the third image (631, 633, 635, 637, 641, 643, 645, 647) among the plurality of second images (620, 630, 640).

[0209] The method may include an action of generating a prompt based on the user input. The method may include an action of identifying the one or more first images (610) by performing the prompt based on an artificial intelligence (AI) model.

[0210] The plurality of second images (620, 630, 640) may be displayed in different areas of the display (260) for different categories. Each of the different areas may include a text object representing a different one of the different categories.

[0211] The operation of displaying the plurality of second images (620, 630, 640) through the display (260) may include an operation of determining an image that meets a specified criterion in relation to at least one of the user profile information as at least one of the plurality of second images. The user profile information may include a shared history, application usage information, or user interest information.

[0212] The above user interest information may include information about applications related to the above tag.

[0213] The operation of displaying the plurality of second images (620, 630, 640) through the display (260) may include an operation of determining an image including data generated from the application as one of the plurality of images.

[0214] The act of identifying the one or more first images (610) may include displaying a user interface (UI) (471) including an object (473) for adding the one or more images for assigning the tag and another object (475) for inputting a prompt, based on other user input for assigning the tag.

[0215] The method may include an operation of changing the tag of the requested one or more first images (610) to another tag based on receiving a request to transmit the one or more first images (610).

[0216] As described above, a non-transitory computer readable storage medium can store a program including instructions. The instructions, when individually or collectively executed by at least one processor (120) including a processing circuit of an electronic device (101) including a display (260), can cause the electronic device (101) to identify one or more first images (610) for assigning tags defined by a user input. The instructions, when individually or collectively executed by the at least one processor (120), can cause the electronic device (101) to display a plurality of second images (620, 630, 640) among a plurality of images through the display (260) based on characteristics of the one or more first images (610). The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to assign the tag to at least one third image (631, 633, 635, 637, 641, 643, 645, 647) selected by another user input from among the plurality of second images (620, 630, 640) and the one or more first images (610).

[0217] The above plurality of second images (620, 630, 640) may have at least one non-visual commonality or visual commonality with one or more of the first images (610).

[0218] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0219] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0220] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0221] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0222] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., by download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0223] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and arranged in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (101), Display (260), At least one processor (120) comprising a processing circuit; and A memory (130) storing instructions and including one or more storage media, wherein the instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: Identifying one or more first images (610) to which tags are assigned as defined by user input, Based on the characteristics of one or more of the first images (610), a plurality of second images (620, 630, 640) among the plurality of images are displayed through the display (260), Causing to assign the tag to at least one third image (631, 633, 635, 637, 641, 643, 645, 647) selected by another user input from among the plurality of second images (620, 630, 640) and one or more of the first images (610). Electronic devices.

2. In claim 1, The above plurality of second images (620, 630, 640) have at least one non-visual commonality or visual commonality with one or more of the first images (610). Electronic devices.

3. In claim 2, Among the plurality of second images (620, 630, 640), some of the second images having visual commonalities with one or more of the first images (610) include a first object included in one or more of the first images (610). Electronic devices.

4. In claim 2, Among the plurality of second images (620, 630, 640), a tag that is distinct from the tag of one or more of the first images (610) and other second images having the non-visual commonality, and / or a part of the meta information corresponds to a tag that is distinct from the tag of one or more of the first images (610), and / or a part of the meta information. Electronic devices.

5. In any one of claims 1 to 4, Including a camera, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: Based on the commonality between the one or more first images (610) and the at least one third image (631, 633, 635, 637, 641, 643, 645, 647), causing a decision on whether to assign the tag to the fourth image acquired through the camera. Electronic devices.

6. In any one of claims 1 to 5, Including a camera, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: Identifying the release of the tag from at least one of the at least one third image and at least one of the at least one first image to which the tag has been assigned; Causing a determination to be made as to whether to assign the tag to the fifth image acquired through the camera based on the difference between the at least one third image to which the tag is assigned, and the one or more first images and the at least one image from which the tag is de-assigned. Electronic devices.

7. In any one of claims 1 to 6, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: Based on the commonality between the one or more first images (610) and the remaining images excluding the third image (631, 633, 635, 637, 641, 643, 645, 647) among the plurality of second images (620, 630, 640), causing a decision on whether to assign the tag to the image. Electronic devices.

8. In any one of claims 1 to 7, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: Generate a prompt based on the above user input, By performing the above prompt based on an AI (artificial intelligence) model, causing the identification of one or more first images (610), Electronic devices.

9. In any one of claims 1 to 8, Some of the above plurality of second images (620, 630, 640) are displayed in the first area for the first category of the display (260), Another part of the above plurality of second images (620, 630, 640) is displayed in a second area for a second category distinct from the first category of the display (260), The first area includes a text object representing the first category, and the second area includes another text object representing the second category. Electronic devices.

10. In a method of an electronic device (101) including a display (260), An action of identifying one or more first images (610) to which a tag defined by user input is assigned; An operation of displaying a plurality of second images (620, 630, 640) among a plurality of images through the display (260) based on the characteristics of one or more of the first images (610), and An operation of assigning a tag to at least one third image (631, 633, 635, 637, 641, 643, 645, 647) selected by another user input from among the plurality of second images (620, 630, 640) and one or more of the first images (610). method.

11. In claim 10, Among the plurality of second images (620, 630, 640), some of the second images having visual commonality with one or more of the first images (610) include a first object included in one or more of the first images (610). method.

12. In claim 10 or 11, Among the plurality of second images (620, 630, 640), a tag distinct from the tag of another part of the second image having non-visual commonality with one or more of the first images (610), and / or a part of the meta information corresponding to a tag distinct from the tag of one or more of the first images (610), and / or a part of the meta information method.

13. In any one of claims 10 to 12, An operation of determining whether to assign the tag to a fourth image acquired through a camera based on commonalities between the one or more first images (610) and the at least one third image (631, 633, 635, 637, 641, 643, 645, 647). method.

14. In any one of claims 10 to 13, An operation for identifying the removal of the tag from at least one of the at least one third image and at least one of the at least one first image to which the tag has been assigned, and An operation of determining whether to assign a tag to a fifth image acquired through a camera based on a difference between the at least one third image to which the tag is assigned, and the one or more first images and the at least one image from which the tag is de-tagged. method.

15. In a non-transitory computer readable storage medium, Save a program containing instructions, The above instructions, when executed individually or collectively by at least one processor (120) including a processing circuit of an electronic device (101) including a display (260), cause the electronic device (101) to: Identifying one or more first images (610) to which tags are assigned as defined by user input, Based on the characteristics of one or more of the first images (610), a plurality of second images (620, 630, 640) among the plurality of images are displayed through the display (260), Causing to assign the tag to at least one third image (631, 633, 635, 637, 641, 643, 645, 647) selected by another user input from among the plurality of second images (620, 630, 640) and one or more of the first images (610). A non-transitory computer-readable recording medium.

Images