Electronic device for outputting answer to question of user by using artificial intelligence model and operating method thereof

Abstract

According to an embodiment, this electronic device may comprise: a camera; at least one processor (220) including a processing circuit; and a memory (230) including one or more storage media and storing instructions. According to an embodiment, the instructions, when collectively or individually executed by the at least one processor, may cause the electronic device to receive a question from a user while capturing images at a designated period through the camera. According to an embodiment, the instructions, when collectively or individually executed by the at least one processor, may cause the electronic device to obtain a first keyword of the question on the basis of providing the question to an artificial intelligence model. According to an embodiment, the instructions, when collectively or individually executed by the at least one processor, may cause the electronic device to, on the basis of identifying that the first keyword matches with at least one keyword among keywords stored in the memory, obtain location information indicating a location at which at least one image corresponding to the at least one keyword is stored in an external database. According to an embodiment, the instructions, when collectively or individually executed by the at least one processor, may cause the electronic device to identify at least one first image most recently captured from a time point at which the question is received. According to an embodiment, the instructions, when collectively or individually executed by the at least one processor, may cause the electronic device to obtain an answer to the question on the basis of providing the at least one first image, the location information, and the question-related information to the artificial intelligence model. According to an embodiment, the instructions, when collectively or individually executed by the at least one processor, may cause the electronic device to output the answer as at least one of visual information or auditory information.

Description

Electronic device that outputs an answer to a user's question using an artificial intelligence model and the method of operation thereof

[0001] The present disclosure relates to an electronic device that outputs an answer to a user's question using an artificial intelligence model and a method of operating the same.

[0002] Thanks to the remarkable advancements in information and communication technology and semiconductor technology, the distribution and use of various electronic devices are increasing rapidly. Electronic devices are being developed to allow users to carry them around and communicate. The term "electronic device" may refer to a device that performs specific functions according to an installed program, such as mobile communication terminals, tablet PCs, wearable electronic devices, video / audio devices, desktop / laptop computers, or vehicle navigation systems.

[0003] Electronic devices may utilize artificial intelligence (AI) models to provide services for specific purposes. For example, AI models are used in various fields such as content streaming, translation, photo editing, finance, new drug development, law, and the military. At least some of the various AI models for specific services may be implemented as generative AI models. Depending on the implementation, the AI ​​model may operate in a connected form.

[0004] According to one embodiment, the electronic device may include at least one processor (220) comprising a camera and a processing circuit, and a memory (230) for storing instructions, comprising one or more storage media. According to one embodiment, when the instructions are executed collectively or individually by the at least one processor, the electronic device may cause the electronic device to receive a user's question while capturing images at a specified interval through the camera. According to one embodiment, when the instructions are executed collectively or individually by the at least one processor, the electronic device may cause the electronic device to obtain a first keyword of the question based on providing the question to an artificial intelligence model. According to one embodiment, when the instructions are executed collectively or individually by the at least one processor, the electronic device may cause the electronic device to obtain location information indicating the location where at least one image corresponding to the at least one keyword is stored in a database outside the electronic device, based on confirming that the first keyword matches at least one keyword among the keywords stored in the memory. According to one embodiment, when the instructions are executed collectively or individually by the at least one processor, the electronic device may be caused to identify at least one first image most recently captured from the time of receiving the question. According to one embodiment, when the instructions are executed collectively or individually by the at least one processor, the electronic device may be caused to obtain an answer to the question based on providing the at least one first image, the location information, and information regarding the question to the artificial intelligence model.According to one embodiment, when the instructions are executed collectively or individually by the at least one processor, the electronic device may cause the answer to be output as at least one of visual information or auditory information.

[0005] According to one embodiment, the method of operation of an electronic device may include receiving a user's question while capturing images at a specified interval through a camera included in the electronic device. According to one embodiment, the method of operation of the electronic device may include obtaining a first keyword of the question based on providing the question to an artificial intelligence model. According to one embodiment, the method of operation of the electronic device may include obtaining location information indicating the location where at least one image corresponding to the at least one keyword is stored in a database outside the electronic device, based on confirming that the first keyword matches at least one keyword among the keywords stored in the electronic device. According to one embodiment, the method of operation of the electronic device may include checking at least one first image most recently captured from the time of receiving the question. According to one embodiment, the method of operation of the electronic device may include obtaining an answer to the question based on providing the at least one first image, the location information, and information regarding the question to the artificial intelligence model. According to one embodiment, the method of operation of the electronic device may include an operation of outputting the answer as at least one of visual information or auditory information.

[0006] According to one embodiment, in a computer-readable non-transient storage medium (130, 230) that stores instructions, the instructions may cause an electronic device (201), when executed individually or collectively by at least one processor (220), to receive a user's question while capturing images at a specified period through a camera included in the electronic device, and based on providing the question to an artificial intelligence model, to obtain a first keyword of the question, and based on confirming that the first keyword matches at least one keyword among the keywords stored in the electronic device, to obtain location information indicating the location where at least one image corresponding to the at least one keyword is stored in a database outside the electronic device, and based on confirming at least one first image most recently captured from the time the question was received, and based on providing the at least one first image, the location information, and information about the question to the artificial intelligence model, to obtain an answer to the question and output the answer as at least one of visual information or auditory information.

[0007] FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments.

[0008] FIG. 2a is a block diagram showing the schematic configuration of an electronic device according to one embodiment.

[0009] FIG. 2b is a diagram showing the configuration of a history database (DB) of an electronic device according to one embodiment.

[0010] FIG. 3 is a flowchart illustrating a method for an electronic device to store location information and object analysis information for an image captured through a camera, according to one embodiment.

[0011] FIG. 4 is a diagram illustrating a method for an electronic device to store location information and object analysis information for an image captured through a camera, according to one embodiment.

[0012] FIG. 5a is a drawing of a prompt for requesting an artificial intelligence model to analyze an image according to one embodiment.

[0013] FIG. 5b is a diagram illustrating object analysis information output by an artificial intelligence model according to one embodiment.

[0014] FIG. 6 is a flowchart illustrating a method for an electronic device according to one embodiment to output an answer to a user's query using an artificial intelligence model.

[0015] FIG. 7 is a diagram illustrating a method for an electronic device according to one embodiment to output an answer to a user's query using an artificial intelligence model.

[0016] FIG. 8 is a diagram of a first prompt for requesting an artificial intelligence model to extract keywords of a question according to one embodiment.

[0017] FIGS. 9a and 9b are drawings illustrating a second prompt for requesting an answer to a question from an artificial intelligence model according to one embodiment.

[0018] FIGS. 10a and FIGS. 10b are drawings illustrating a method for an electronic device to output an answer to a user's query using an artificial intelligence model.

[0019] FIGS. 11a and FIGS. 11b are drawings illustrating a method for an electronic device to output an answer to a user's query using an artificial intelligence model.

[0020] FIGS. 12a and FIGS. 12b are drawings illustrating a method for an electronic device to add various information as a watermark to an image.

[0021] FIGS. 13a and FIGS. 13b are drawings illustrating a method for an electronic device to output an answer to a user's query using an artificial intelligence model.

[0022] FIG. 14 is a flowchart illustrating a method for an electronic device according to one embodiment to output an answer to a user's query using an artificial intelligence model.

[0023] FIG. 15 is a diagram illustrating a generative artificial intelligence system according to one embodiment.

[0024] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or may communicate with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0025] The processor (120) can control at least one other component (e.g., hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., program (140)), for example, and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., sensor module (176) or communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., central processing unit or application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., graphics processing unit, neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0026] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence is performed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0027] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0028] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0029] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0030] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0031] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0032] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0033] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0034] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0035] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0036] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0037] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0038] The power management module (188) can manage the power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0039] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0040] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0041] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0042] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally created as part of the antenna module (197).

[0043] According to one embodiment, the antenna module (197) can create a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0044] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0045] According to one embodiment, commands or data may be transmitted or received between an electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0046] Functions related to artificial intelligence according to the present disclosure are operated through a processor and memory. The processor may be composed of one or more processors. In this case, the one or more processors may be general-purpose processors such as CPUs, APs, and DSPs (Digital Signal Processors), graphics-dedicated processors such as GPUs and VPUs (Vision Processing Units), or artificial intelligence-dedicated processors such as NPUs. The one or more processors control the processing of input data according to predefined operation rules or artificial intelligence models stored in memory. Alternatively, if the one or more processors are artificial intelligence-dedicated processors, the artificial intelligence-dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0047] The predefined rules of operation or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that a predefined rules of operation or artificial intelligence models configured to perform desired characteristics (or objectives) are created by a basic artificial intelligence model being trained using multiple learning data by a learning algorithm. Such learning may be performed on the device itself where the artificial intelligence according to the present disclosure is executed, or it may be performed through a separate server and / or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples described above.

[0048] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values ​​and performs neural network operations through operations between the results of previous layers and the multiple weights. The multiple weights possessed by the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated so that the loss value or cost value obtained from the artificial intelligence model during the learning process is reduced or minimized. Artificial neural networks may include deep neural networks (DNNs), such as Convolutional Neural Networks (CNNs), Deep Neural Networks (DNNs), Recurrent Neural Networks (RNNs), Restricted Boltzmann Machines (RBMs), Deep Belief Networks (DBNs), Bidirectional Recurrent Deep Neural Networks (BRDNNs), or Deep Q-Networks, but are not limited to the examples mentioned above.

[0049] Meanwhile, terms related to 'identify' in the present disclosure may be replaced with 'detect', 'recognize', 'determine', and / or 'sense'.

[0050] FIG. 2a is a block diagram showing the schematic configuration of an electronic device according to one embodiment. FIG. 2b is a diagram showing the configuration of a history database (DB) of an electronic device according to one embodiment.

[0051] Referring to FIG. 2a, the electronic system may include an electronic device (201), a database (202), and a server (203). Meanwhile, although FIG. 2a depicts the database (202) as being placed outside the electronic device (201), the technical features of the present invention may not be limited thereto. For example, the database (202) may be placed inside or outside the electronic device (201). For example, the database (202) may be included in the electronic device (201). Additionally, the artificial intelligence model (290) may also be stored in the electronic device (201) rather than the server (203).

[0052] For convenience of explanation, the database (202) and the artificial intelligence model (290) will be described as being placed outside the electronic device (201). However, the technical features of the present invention may not be limited thereto.

[0053] According to one embodiment, the electronic device (201) can transmit an image to the database (202) so that an image captured using the camera (210) is stored in the database (202). The database (202) may represent a storage space outside the electronic device (201). The database (202) may store the image and may provide or transmit the stored image to the server (203). The server (203) may include an artificial intelligence model (290). The server (203) may provide or transmit result data output through the artificial intelligence model (290) to the electronic device (201) in response to a request from the electronic device (201).

[0054] According to one embodiment, an electronic device (201) (e.g., the electronic device (101) of FIG. 1) may include a camera (210) (e.g., the camera module (180) of FIG. 1), a processor (220) (e.g., the processor (120) of FIG. 1), a memory (230) (e.g., the memory (130) of FIG. 1), a communication circuit (250), a display (260) (e.g., the display module (160) of FIG. 1), and a speaker (265) (e.g., the sound output module (155) of FIG. 1). According to one embodiment, the electronic device (201) may omit at least one of the components or additionally include another component (e.g., the input module (150)).

[0055] According to one embodiment, the processor (220) can control the overall operation of the electronic device (201). For example, the processor (220) may be implemented identically or similarly to the processor (120) of FIG. 1. According to one embodiment, the processor (220) can control at least one other component (e.g., hardware or software component) of the electronic device (201) connected to the processor (220) by executing software (e.g., program (140) of FIG. 1), and can perform data processing or operations based on instructions. According to one embodiment, the instructions may include instructions composed of machine language that can be processed by the electronic device (201) or the processor (220). For example, the instructions may include instructions corresponding to operation instructions used in the program.

[0056] Meanwhile, although FIG. 2a illustrates that the electronic device (201) includes one processor (220), this is exemplary and the technical concept of the present invention may not be limited thereto. For example, the electronic device (201) may include at least one processor (e.g., CPU, DSP, GPU, and / or NPU). For example, the processor (220) may be implemented as at least one processor.

[0057] According to one embodiment, the memory (230) (e.g., the memory (130) of FIG. 1) may store at least one instruction (or instruction) that causes at least one operation of the electronic device (201). When the at least one instruction is executed collectively or individually by the processor (220), it may cause the electronic device (201) to perform the corresponding operation.

[0058] According to one embodiment, the processor (220) can obtain an answer (or response) to a user's question (or inquiry) using an artificial intelligence model (290). For example, the question (or inquiry) can be received as voice through a microphone of an electronic device (201) or as text through a display (260) (e.g., a touchscreen). The processor (220) can output information about the answer (or response) obtained using the artificial intelligence model (290) as visual information or auditory information through at least one of the display (260) or a speaker (265).

[0059] According to one embodiment, the processor (220) may obtain an answer (or response) suitable for the user's intention by providing various data stored in memory (230) to an artificial intelligence model (290). For example, the various data stored in memory (230) may include data related to images captured using a camera (210) prior to receiving a question. For example, the data may include location information indicating the location (or path) where the images are stored in an external database (202), analysis information analyzing the images (or objects included in the images) (e.g., object analysis information and surrounding context information), and / or timestamp information indicating the time the images were taken. For example, the object analysis information may include keywords of the object included in the image and information describing the object (e.g., name, designation, shape, use, color, location, and / or correlation with the surroundings). For example, the surrounding context information may include information describing the surrounding context other than the object (e.g., place, location, background information, and / or correlation between the object and the surroundings).

[0060] According to one embodiment, the artificial intelligence model (290) may include a neural network model trained to derive a result value for a command (e.g., a prompt) requested by an electronic device (201). For example, the artificial intelligence model (290) may be trained to output a result of analyzing an object included in a corresponding image based on an analysis request for an image (or an object included in an image). For example, the artificial intelligence model (290) may be trained to output a result of extracting keywords of a corresponding question based on an analysis request for a user's question (e.g., a keyword extraction request). For example, the artificial intelligence model (290) may be trained to output an answer to a corresponding question based on a request to derive an answer to a user's question. For example, the artificial intelligence model (290) may be stored in memory (230) or a server (203). Meanwhile, in FIG. 2, the artificial intelligence model (290) is depicted as being included in the server (203), but this is exemplary and the technical features of the present invention may not be limited thereto.

[0061] According to one embodiment, for convenience of explanation, the following description will focus on an embodiment in which the artificial intelligence model (290) is stored in a server (203). However, the technical features of the present invention may not be limited thereto, and the artificial intelligence model (290) may be stored in memory (230).

[0062] According to one embodiment, the processor (220) can collect and store data to obtain an answer to a user's question using an artificial intelligence model (290) based on capturing images through a camera (210).

[0063] According to one embodiment, the processor (220) can capture images through the camera (210) at a specified period (e.g., 1 to 10 seconds). For example, the processor (220) can capture images through the camera (210) at a specified period when the electronic device (201) is turned on. For example, the specified period may be changed by the user's settings or by the processor (220). For example, the specified period may be changed based on the degree of movement (e.g., acceleration) of the electronic device (201) sensed through a sensor included in the electronic device (201) (e.g., sensor module (176) of FIG. 1), the complexity of the scene of the image captured through the camera (210), and / or the illumination outside the electronic device (201).

[0064] According to one embodiment, the processor (220) can transmit the captured images to a database (202) via a communication circuit (250) so that the captured images are stored in a database (e.g., bucket) (202) outside the electronic device (201). After the images captured through the camera (210) are stored in the database (202) outside the electronic device (201), the processor (220) can delete the images from the memory (230). That is, since the storage space of the memory (230) may become insufficient if the captured images are continuously stored in the memory (230), the processor (220) can store only the data obtained by analyzing the captured images in the memory (230).

[0065] According to one embodiment, the processor (220) may add information indicating the time each image was taken as a watermark to each corresponding image so that images with information indicating the time each image was taken are stored in the database (202). According to one embodiment, since the artificial intelligence model (290) (e.g., AI analysis engine) is a model or engine configured to analyze only images, it may be difficult to obtain information such as timestamps included in the metadata of the image. Therefore, the electronic device (201) may add the timestamp at which the image was acquired (e.g., time of capture) as a watermark to the image before storing the image acquired through the camera (210) in the database (202).

[0066] According to one embodiment, the processor (220) can obtain location information from the database (202) indicating the location (or path) where the images are stored in the database (202) based on transmitting the images to the database (202).

[0067] According to one embodiment, the processor (220) may provide location information to the artificial intelligence model (or server (203)) to analyze images captured through the camera (210) using the artificial intelligence model (290). For example, the processor (220) may provide the artificial intelligence model (290) with a prompt requesting analysis of the images along with location information of the images. The artificial intelligence model (290) may acquire images corresponding to location information stored in the database (202). For example, the server (203) may acquire images corresponding to location information stored in the database (202) and provide the acquired images to the artificial intelligence model (290). Based on analyzing each image, the artificial intelligence model (290) may output or derive object analysis information that analyzes the objects included in each image.

[0068] According to one embodiment, the processor (220) can obtain object analysis information including keywords related to objects included in each image through a communication circuit (250). For example, the object analysis information may include keywords for objects included in each image, a description of said objects, and information about the time each image was taken (e.g., a timestamp). The processor (220) can store the object analysis information in memory (250) in association with location information of the corresponding image.

[0069] Referring to FIG. 2b, according to one embodiment, the processor (220) may store location information of images and object analysis information of images in association in a history DB (270) contained in memory (230). For example, the history DB (270) may represent a storage space configured to store data for obtaining answers to user questions using an artificial intelligence model (290). For example, the history DB (270) may be a logically distinct storage space. For example, the processor (220) may store in the history DB (270) in association with information (271) about an image, such as a keyword (e.g., keyword of an image), location information (e.g., information indicating the location where an image is stored in the database (202) (URL)), a description (e.g., a description of an object included in an image), and / or a timestamp (e.g., the time when an image was captured). The processor (220) may store in the history DB (270) associated with information (272) about another image, such as keywords (e.g., keywords of another image), location information (e.g., information indicating the location where another image is stored in the database (202) (URL)), description (e.g., description of an object included in another image), and / or timestamp (e.g., time when another image was captured). The processor (220) may store in the history DB (270) associated with information (273) about another image, such as keywords (e.g., keywords of another image), location information (e.g., information indicating the location where another image is stored in the database (202) (URL)), description (e.g., description of an object included in another image), and / or timestamp (e.g., time when another image was captured).

[0070] According to one embodiment, the processor (220) may receive a question from a user while capturing images at a specified interval through the camera (210).

[0071] According to one embodiment, the processor (220) can obtain a first keyword of a question based on providing a question to an artificial intelligence model (290). For example, the processor (220) can provide the artificial intelligence model (290) with a prompt requesting keyword extraction of the question along with the question.

[0072] According to one embodiment, the processor (220) can identify or search for a keyword that matches the first keyword among the keywords stored in memory (230) (or history DB (270)). If the processor (220) identifies that the first keyword matches at least one keyword among the keywords stored in memory (230) (or history DB (270)), it can obtain location information (e.g., URL) indicating the location (or path) where at least one image corresponding to the at least one keyword is stored in the database (202).

[0073] According to one embodiment, prior to this, the processor (220) may generate a prompt to request an answer to a user's question. The processor (220) may provide the generated prompt to the artificial intelligence model (290). The processor (220) may identify at least one first image captured most recently (or at the time the question was received) from the time the question is received, along with the prompt. The processor (220) may provide at least one first image, location information, and / or information about the question (e.g., information for understanding the question and the context of the question) to the artificial intelligence model (290). According to one embodiment, when the processor (220) requests an answer to the user's question from the artificial intelligence model (290), the processor (220) may provide the artificial intelligence model (290) with at least one of a description and / or timestamp of at least one corresponding image stored in the history DB (270), as well as location information. For example, when a question is received as text input, information for understanding the context of the question may include the remaining text included in the dialogue screen in addition to the text corresponding to the question.

[0074] According to one embodiment, the processor (220) may obtain an answer to a question derived from an artificial intelligence model (290). The processor (220) may output the answer as at least one of visual information or auditory information through a display (260) and / or a speaker (265). Additionally, the processor (220) may generate or obtain information about at least one first image (e.g., keywords, description of the image, and / or timestamp) as an output of the artificial intelligence model (290), and may store the information about at least one first image in a history DB (230).

[0075] Based on the method described above, the electronic device (201) can automatically provide an answer to the user that matches the user's query intent by using an artificial intelligence model (290).

[0076] At least some of the operations of the electronic device described below may be performed by the processor (220). However, for the convenience of explanation, said operations will be described as being performed by the electronic device (201).

[0077] FIG. 3 is a flowchart illustrating a method for an electronic device to store location information and object analysis information for an image captured through a camera, according to one embodiment.

[0078] Referring to FIG. 3, according to one embodiment, in operation 301, an electronic device (e.g., electronic device (201) of FIG. 2a) can transmit images to a database (202) so that the captured images are stored in a database (e.g., database (202) of FIG. 2a) based on capturing images at a specified interval through a camera (e.g., camera (210) of FIG. 2a).

[0079] According to one embodiment, in operation 303, the electronic device (201) can obtain location information (e.g., URL information) from the database (202) indicating the location (or stored path) where images are stored in the database (202).

[0080] According to one embodiment, in operation 305, the electronic device (201) may provide location information of captured images to the artificial intelligence model (290) in order to analyze the images using the artificial intelligence model (e.g., the artificial intelligence model (290) of FIG. 2a). For example, the electronic device (201) may generate a prompt requesting the artificial intelligence model (290) to analyze the images and provide the generated prompt to the artificial intelligence model (290) along with location information.

[0081] According to one embodiment, in operation 307, the electronic device (201) may obtain object analysis information including keywords related to objects included in each image from an artificial intelligence model (290). For example, the object analysis information may include keywords for objects included in the image, descriptions of said objects, and / or information about the time when said images were taken.

[0082] According to one embodiment, in operation 309, the electronic device (201) may store object analysis information in memory (e.g., memory (230) of FIG. 2b) (or history DB (270)) in association with the location information of the corresponding image. The electronic device (201) may use the location information and object analysis information stored in memory (230) as data to obtain an answer to a user's question.

[0083] FIG. 4 is a diagram illustrating a method for an electronic device to store location information and object analysis information for an image captured through a camera, according to one embodiment.

[0084] Referring to FIG. 4, according to one embodiment, in operation 401, the electronic device (201) can capture images at specified intervals through a camera (e.g., the camera (210) of FIG. 2a).

[0085] According to one embodiment, in operation 403, the electronic device (201) may request that the captured image be stored in the database (202). In operation 405, the database (202) may store the image received from the electronic device (201) and check the location information where the image is stored.

[0086] According to one embodiment, in operation 407, the electronic device (201) may receive and store location information (e.g., URL information) indicating the location where the image is stored from the database (202).

[0087] According to one embodiment, in operation 409, the electronic device (201) may request the artificial intelligence model (290) to analyze the image. For example, the electronic device (201) may provide the artificial intelligence model (290) with location information of the image and a prompt requesting analysis.

[0088] According to one embodiment, in operation 411, the artificial intelligence model (290) can analyze an object included in an image and output object analysis information as an analysis result. For example, the artificial intelligence model (290) can analyze an image obtained from location information of the image (e.g., URL) to extract keywords, descriptions (e.g., image descriptions), and / or timestamps (e.g., time the image was taken) of the image. The artificial intelligence model (290) can transmit the extracted information as an analysis result (e.g., object analysis information) to the electronic device (201). In operation 413, the electronic device (201) can receive object analysis information from the artificial intelligence model (290).

[0089] According to one embodiment, in operation 415, the electronic device (201) may store object analysis information and location information for one image in association in memory (e.g., memory of FIG. 2b) or history DB (e.g., history DB of FIG. 2b (270)). The electronic device (201) may use the location information and object analysis information stored in memory (230) or history DB (270) as data to obtain an answer to a user's question.

[0090] FIG. 5a is a drawing of a prompt for requesting image analysis from an artificial intelligence model according to one embodiment. FIG. 5b is a drawing for explaining object analysis information output by an artificial intelligence model according to one embodiment.

[0091] Referring to FIG. 5a, according to one embodiment, an electronic device (e.g., the electronic device (201) of FIG. 2a) may provide the artificial intelligence model (e.g., the artificial intelligence model (290) of FIG. 2a) with a prompt (520) containing location information (510) of the image or texts requesting analysis of the image when requesting analysis of the image from the artificial intelligence model (290). For example, the location information (510) may include URL information where the image is stored. The prompt (520) may include texts requesting analysis of the image (or an object included in the image) according to a specified format (e.g., name, designation, quantity, color, use, mutual location, and / or written characters).

[0092] Meanwhile, the prompt format illustrated in FIG. 5a is exemplary, and the technical features of the present invention may not be limited thereto.

[0093] Referring to (a) of FIG. 5b, a specific image (530) may be stored in a database (202). An artificial intelligence model (290) may obtain a specific image (530) from the database (202) based on a prompt received from an electronic device (201) (e.g., a prompt requesting analysis of a specific image (530)) and location information of the specific image (530) (e.g., a URL). The artificial intelligence model (290) may analyze the specific image (530) in response to a request from the electronic device (201). For example, the artificial intelligence model (290) may analyze a watermark (e.g., 20224086104942) of the specific image (530) to determine the timestamp (e.g., time of capture) of the specific image (530).

[0094] Referring to (b) of FIG. 5b, the artificial intelligence model (290) can output object analysis information (540) for a specific image (530) as an analysis result. The electronic device (201) can obtain object analysis information (540) for a specific image (530) from the artificial intelligence model (290). For example, the object analysis information (540) may include a description of an object included in the specific image (530) and a timestamp indicating the time the specific image (530) was captured. For example, a name or designation included in the object analysis information (540) may be used as a keyword for the specific image (530).

[0095] Based on the method described above, the electronic device (201) can use location information and object analysis information stored in memory (230) or history DB (270) as data to obtain an answer to a user's question.

[0096] Below, a method for obtaining an answer to a user's question will be described based on the electronic device (201) providing data stored in memory (230) or history DB (270) to an artificial intelligence model.

[0097] FIG. 6 is a flowchart illustrating a method for an electronic device according to one embodiment to output an answer to a user's query using an artificial intelligence model.

[0098] Referring to FIG. 6, according to one embodiment, in operation 601, an electronic device (e.g., the electronic device (201) of FIG. 2a) may receive or obtain a question from a user while capturing images at a specified interval through a camera (e.g., the camera (210) of FIG. 2a). For example, the electronic device (201) may obtain the question in voice or text.

[0099] According to one embodiment, in operation 603, the electronic device (201) may obtain a first keyword of a question based on providing a question to an artificial intelligence model (e.g., the artificial intelligence model (290) of FIG. 2a). For example, the electronic device (201) may request keyword extraction for the question while providing a question to the artificial intelligence model (290). For example, the electronic device (201) may provide a first prompt to the artificial intelligence model (290) requesting keyword extraction for the question. The electronic device (201) may obtain a first keyword of the question from the artificial intelligence model. For example, the first keyword may include one or more keywords.

[0100] According to one embodiment, in operation 605, the electronic device (201) can obtain location information of at least one image corresponding to at least one keyword based on confirming that the first keyword matches at least one keyword among the keywords stored in memory (e.g., memory (230) of FIG. 2b) (or history DB (270)).

[0101] According to one embodiment, in operation 607, the electronic device (201) can check at least one first image that was most recently captured from the time the question was received (or captured at the time the question was received).

[0102] According to one embodiment, in operation 609, the electronic device (201) can obtain an answer to a question based on providing at least one first image, location information, and information about the question to the artificial intelligence model (290). For example, the electronic device (201) can provide a second prompt requesting an answer to the question to the artificial intelligence model (290).

[0103] According to one embodiment, in operation 611, the electronic device (201) may output an answer as at least one of visual information or auditory information. For example, the electronic device (201) may display text corresponding to the answer on a display (e.g., display (260) of FIG. 2a) or output a voice corresponding to the answer through a speaker (e.g., speaker (265) of FIG. 2a).

[0104] FIG. 7 is a diagram illustrating a method for an electronic device according to one embodiment to output an answer to a user's query using an artificial intelligence model.

[0105] Referring to FIG. 7, according to one embodiment, in operation 701, the electronic device (201) can capture a first image through a camera (e.g., the camera (210) of FIG. 2a). In operation 703, the electronic device (201) can receive a user's question. For example, the first image may be the image most recently captured at the time the question is received or at the time the question is received.

[0106] According to one embodiment, in operation 705, the electronic device (201) may request the artificial intelligence model (290) to extract keywords of the question. The electronic device (201) may transmit a first prompt requesting the extraction of keywords of the question to the artificial intelligence model. For example, the electronic device (201) may convert speech related to the user query into text using a speech-to-text (stt) program and provide the converted text to the artificial intelligence model (290) along with the first prompt. For example, the first prompt may include content requesting the extraction of keywords from the text related to the user query.

[0107] According to one embodiment, in operation 707, the artificial intelligence model (290) may analyze the question based on the question and / or the first prompt and output the first keyword of the question as an analysis result. In operation 709, the electronic device (201) may receive or obtain the first keyword from the artificial intelligence model (290).

[0108] According to one embodiment, in operation 711, the electronic device (201) can check whether there is a keyword matching the first keyword in the history DB (e.g., the history DB (270) of FIG. 2b).

[0109] According to one embodiment, operations 713 to 715 and operations 717 to 722 may be optionally performed depending on whether a keyword matching the first keyword is found in the history DB (270).

[0110] According to one embodiment, when a keyword matching the first keyword is not found in the history DB (270) (No in operation 711), in operation 713, the electronic device (201) may request an answer to the question while transmitting or providing the question and the first image to the artificial intelligence model (290). That is, the electronic device (201) may not be able to provide the data stored in the history DB (270) to the artificial intelligence model (290). In operation 715, the artificial intelligence model (290) may output or derive an answer to the question. However, the artificial intelligence model (290) may derive an answer to the question without considering the data stored in the history DB (270).

[0111] According to one embodiment, when a keyword matching the first keyword is identified in the history DB (270) (e.g., operation 711), in operation 717, the electronic device (201) can obtain location information related to the keyword matching the first keyword. In operation 719, the electronic device (201) can request an answer to the question while transmitting or providing the question, location information, and the first image to the artificial intelligence model (290). For example, the electronic device (201) can provide the artificial intelligence model (290) with a second prompt requesting an answer to the question along with the question, location information, and the first image. In addition, the electronic device (201) can further provide user query history (e.g., previous user queries) to the artificial intelligence model (290).

[0112] According to one embodiment, in operation 720, the artificial intelligence model (290) may request the database (202) to transmit an image corresponding to location information. In operation 721, the artificial intelligence model (290) may receive an image corresponding to location information from the database (202). In operation 722, the artificial intelligence model (290) may obtain or output an answer to a question based on the question, the image corresponding to location information, and the first image.

[0113] According to one embodiment, in operation 723, the electronic device (201) may receive information about an answer to a question from an artificial intelligence model (290). In operation 725, the electronic device (201) may output information about the answer as text and / or voice.

[0114] FIG. 8 is a diagram of a first prompt for requesting an artificial intelligence model to extract keywords of a question according to one embodiment.

[0115] Referring to FIG. 8, according to one embodiment, an electronic device (e.g., the electronic device (201) of FIG. 2a) may provide a first prompt (810) to an artificial intelligence model (e.g., the artificial intelligence model (290) of FIG. 2a) when the artificial intelligence model (290) requests keyword extraction of a question. For example, the first prompt (810) may include texts requesting keyword analysis related to the question based on a specified format (e.g., name, designation, quantity, color, use, mutual location, and / or written characters). For example, the first prompt (810) may include texts requesting the extraction of at least one keyword related to the question. For example, the first prompt (810) may include content requesting the extraction of keyword 1, keyword 2, keyword 3, keyword 4, and keyword 5 related to the question. Meanwhile, the prompt format illustrated in FIG. 8 is exemplary, and the technical features of the present invention may not be limited thereto.

[0116] FIGS. 9a and 9b are drawings illustrating a second prompt for requesting an answer to a question from an artificial intelligence model according to one embodiment.

[0117] Referring to FIG. 9a, according to one embodiment, an electronic device (e.g., the electronic device (201) of FIG. 2a) may provide the artificial intelligence model (e.g., the artificial intelligence model (290) of FIG. 2a) with a second prompt (940) including a question (910), location information (920) (e.g., URL information), a first image (930) captured at the time the question is received, and texts requesting an answer to the question when requesting an answer to the question.

[0118] Referring to FIG. 9b, according to one embodiment, the second prompt (940) may include texts requesting an answer to a question. For example, the second prompt (940) may include texts regarding rules for deriving an answer related to a question based on a specified format (e.g., a definition of an image, a definition of a question type, and / or a method for processing each question type). For example, the second prompt (940) may include texts regarding how the artificial intelligence model (290) determines the question type.

[0119] According to one embodiment, a second prompt can be generated based on a user query. For example, the electronic device (201) can determine the user query to be of any one of types A through G based on analyzing what type of question the user query is. The electronic device (201) can generate a prompt corresponding to the type based on the determined type. For example, the content of the question included in the prompt may vary depending on the type.

[0120] Meanwhile, the prompt format illustrated in FIG. 9b is exemplary, and the technical features of the present invention may not be limited thereto.

[0121] FIGS. 10a and FIGS. 10b are drawings illustrating a method for an electronic device to output an answer to a user's query using an artificial intelligence model.

[0122] Referring to FIG. 10a, according to one embodiment, a plurality of images (1011, 1012, 1013, 1014, 1015) may be stored in a database (e.g., the database (202) of FIG. 2a). For example, the plurality of images (1011, 1012, 1013, 1014, 1015) may be images of people located in the same place. In this case, the people included in each image may be somewhat different. Location information indicating the location where the plurality of images (1011, 1012, 1013, 1014, 1015) are stored in the database (202) may be stored in the memory (e.g., the memory (230) of FIG. 2b) or the history DB (e.g., the history DB (270) of FIG. 2b) of an electronic device (e.g., the electronic device (201) of FIG. 2a).

[0123] According to one embodiment, there may be 6 people (A1, A2, A3, A4, A5, and A6) at a place (e.g., a conference room) where a plurality of images (1011, 1012, 1013, 1014, 1015) are taken. Alternatively, there may be 7 people (user and A1, A2, A3, A4, A5, and A6) including a user of the electronic device (201) at a place (e.g., a conference room) where a plurality of images (1011, 1012, 1013, 1014, 1015) are taken. For example, images (1011) may include people corresponding to A1, A2, and A3, images (1012) may include people corresponding to A1, A2, A3, and A4, images (1013) may include people corresponding to A2, A3, and A4, images (1014) may include people corresponding to A3, A4, A5, and A6, and images (1015) may include people corresponding to A4, A5, and A6.

[0124] Referring to FIG. 10b, according to one embodiment, an electronic device (201) may receive a user's question (1020) while capturing at least one first image through a camera (210). For example, the question (1020) may be "How many people are in the conference room?". The electronic device (201) may request keyword extraction of the question (1020) based on providing the question (1020) and a first prompt to an artificial intelligence model (290). The electronic device (201) may obtain a first keyword (1030) corresponding to the keyword of the question (1020) from the artificial intelligence model (290). For example, the first keyword (1030) may include "people," "conference room," and "how many."

[0125] According to one embodiment, the electronic device (201) can check or search for whether there is a keyword matching the first keyword (1030) in the history DB (270). The electronic device (201) can check location information (e.g., 5 URL information) of at least one keyword matching the first keyword (1030). For example, when there are multiple keywords matching the keyword (1030), the electronic device (201) can extract multiple location information. Alternatively, when there are more than a specified number of keywords matching the first keyword (1030) (e.g., 5), the electronic device (201) may extract only the specified number of location information that was recently captured.

[0126] According to one embodiment, the electronic device (201) may request an answer to a question (1020) based on providing the artificial intelligence model (290) with first information (1040) including a question, location information, at least one first image (e.g., at least one image taken most recently since the time the question (1020) is received and / or said time), and a second prompt. Depending on the implementation, when the number of images taken most recently since the time the question (1020) is received is greater than a specified number (e.g., 5), the electronic device (201) may provide the artificial intelligence model (290) with only the specified number of recently taken images.

[0127] According to one embodiment, the artificial intelligence model (290) may request the database (202) to provide at least one image corresponding to location information (1050). The artificial intelligence model (290) may obtain at least one image (1060) corresponding to location information (1050). Subsequently, the artificial intelligence model (290) may derive or output an answer (1070) to a question based on a question (1020), at least one image (1060), at least one first image, and / or a second prompt. For example, the second prompt may be generated using at least one question (1020), at least one image (1060), at least one first image, and / or additional information (e.g., environmental information or other information stored in the electronic device (201).

[0128] According to one embodiment, the artificial intelligence model (290) can determine that there is one duplicate person in at least one image (1060). Additionally, if the artificial intelligence model (290) determines that a user not included in at least one image (1060) is located in that location (e.g., a meeting room), the user can be counted as a person located in that location. Furthermore, since there may be people who leave the meeting room or newly enter the meeting room during the meeting, the artificial intelligence model (290) can derive a more accurate answer if all images taken during the meeting are obtained from the history DB (270).

[0129] According to one embodiment, the electronic device (201) may obtain information about the answer (1070) from the artificial intelligence model (290). For example, the answer (1070) may be, "There are 7 people in the conference room, including you." For example, if it is determined that the user of the electronic device (201) is not located in the conference room, the answer may be, "There are 6 people in the conference room." Or, if it is determined that someone has left the conference room during the meeting, the answer may be, "There are currently 7 people, but a gentleman in gray clothes left in the middle, so the final number of attendees is determined to be 8." The electronic device (201) may output information about the answer (1070) as text or voice. Depending on the implementation, the electronic device (201) may use a generative AI model to generate an image containing 6 (or 7) people attending the meeting and store the generated image.

[0130] Based on the method described above, the electronic device (201) can obtain an appropriate answer to a question using an artificial intelligence model (290).

[0131] Meanwhile, what is described in FIGS. 10a and FIGS. 10b is merely an example of the present invention, and the technical concept of the present invention may not be limited thereto.

[0132] FIGS. 11a and FIGS. 11b are drawings illustrating a method for an electronic device to output an answer to a user's query using an artificial intelligence model.

[0133] Referring to FIG. 11a, according to one embodiment, a plurality of images (1111, 1112) may be stored in a database (e.g., the database (202) of FIG. 2a). For example, the plurality of images (1111, 1112) may be images of different glasses located in different places. For example, the plurality of images (1111, 1112) may be images taken discontinuously in different places. Location information indicating the location where the plurality of images (1111, 1112) are stored in the database (202) may be stored in the memory (e.g., the memory (230) of FIG. 2b) or the history DB (e.g., the history DB (270) of FIG. 2b) of an electronic device (e.g., the electronic device (201) of FIG. 2a).

[0134] Referring to FIG. 11b, according to one embodiment, an electronic device (201) may receive a user's question (1120) while capturing at least one first image through a camera (210). For example, the question (1020) may be "Where are the glasses?". The electronic device (201) may request keyword extraction of the question (1120) based on providing the question (1120) and a first prompt to an artificial intelligence model (290). The electronic device (201) may obtain a first keyword (1130) corresponding to the keyword of the question (1120) from the artificial intelligence model (290). For example, the first keyword (1130) may include "glasses" and "where".

[0135] According to one embodiment, the electronic device (201) can check or search for whether there is a keyword matching the first keyword (1130) in the history DB (270). The electronic device (201) can check location information (e.g., two URL information) of at least one keyword matching the first keyword (1130).

[0136] According to one embodiment, the electronic device (201) may request an answer to a question (1120) based on providing first information (1140) to an artificial intelligence model (290), which includes a question, location information, at least one first image (e.g., at least one image taken most recently from the time when the question (1120) is received and / or said time), and a second prompt.

[0137] According to one embodiment, the artificial intelligence model (290) may request the database (202) to provide at least one image corresponding to location information (1150). The artificial intelligence model (290) may obtain at least one image (1160) corresponding to location information (1150). Subsequently, the artificial intelligence model (290) may derive or output an answer (1170) to a question based on a question (1120), at least one image (1160), at least one first image, and / or a second prompt.

[0138] According to one embodiment, the artificial intelligence model (290) can derive an answer by considering that the first pair of glasses and the second pair of glasses are located in different places and are included in different images. For example, if the first pair of glasses is included in other images, the model can determine that the first pair of glasses included in the other images is the same pair of glasses by referring to the descriptions of the images. For example, the artificial intelligence model (290) can use information about surrounding objects (e.g., information about books included in image (1111)) to determine that the glasses included in multiple images (1111, 1112) are the same pair of glasses. For example, when the description of the surrounding objects in image (1111) is determined to be similar to the description of the surrounding objects included in other images (e.g., glasses placed on the same books) at a certain level or higher, the model can determine that the first pair of glasses is the same pair of glasses included in other images. For example, because the number (and type) of books included in image (1111) and image (1112) are different, the artificial intelligence model (290) can determine that the two glasses are different. According to one embodiment, the electronic device (201) can obtain information about the answer (1170) from the artificial intelligence model (290). For example, the answer (1170) may be, "The first glasses are at position A, and the second glasses are at position B."

[0139] Based on the method described above, the electronic device (201) can obtain an appropriate answer to a question using an artificial intelligence model (290).

[0140] Meanwhile, what is described in FIGS. 11a and FIGS. 11b is merely an example of the present invention, and the technical concept of the present invention may not be limited thereto.

[0141] FIGS. 12a and FIGS. 12b are drawings illustrating a method for an electronic device to add various information as a watermark to an image.

[0142] Referring to FIG. 12a and FIG. 12b, according to one embodiment, an electronic device (e.g., the electronic device (201) of FIG. 2a) may add a watermark containing specific information to an image before storing the image in a database (202). That is, the electronic device (201) may store an image containing a watermark containing specific information in the database (202).

[0143] Through the method described above, the electronic device (201) can enable an artificial intelligence model (e.g., the artificial intelligence model (290) of FIG. 2a) to analyze the image by referring to the specific information based on adding a watermark containing specific information to the image.

[0144] Referring to FIG. 12a, according to one embodiment, when capturing an image (1210), if voice (1220) is input together with the image (1210), the electronic device (201) can identify information about the voice (1220) (e.g., keywords of the voice or the name of the object pointed to by the voice). The electronic device (201) can add or display information about the voice (1220) (e.g., keywords of the voice or the name of the object pointed to by the voice) as a watermark (1230) on the image (1210). The electronic device (201) can store the image (1210) with the watermark (1230) added or displayed in a database (202).

[0145] Referring to FIG. 12b, according to one embodiment, an electronic device (201) may add or display information (or timestamp) about the time at which the image (1260) was taken as a watermark (1270 or 1280) of the image (1260). For example, referring to FIG. 12b (a), the electronic device (201) may add or display a watermark (1270) by overlaying or overlapping it on the image (1260). For example, referring to FIG. 12b (b), the electronic device (201) may add or display a watermark (1280) in a separate area that does not include the image (1260). The electronic device (201) may also add a watermark containing specific information through various other methods.

[0146] FIGS. 13a and FIGS. 13b are drawings illustrating a method for an electronic device to output an answer to a user's query using an artificial intelligence model.

[0147] Referring to FIGS. 13a and 13b, according to one embodiment, an electronic device (e.g., the electronic device (201) of FIG. 2a) can receive a user's question via voice or text. Additionally, the electronic device (201) can output an answer to the user's question via voice or text in various UI environments.

[0148] Referring to FIG. 13a, according to one embodiment, an electronic device (201) may display a dialog-type user interface (UI) (1310) on a display (e.g., the display (260) of FIG. 2a). For example, the UI (1310) may be configured to input a user's question or to display an input question and an answer thereto. For example, the user's question may be input via a touchscreen or voice.

[0149] According to one embodiment, the electronic device (201) can display text (1320) corresponding to a question entered into the UI (1310). The electronic device (201) can display text (1330) corresponding to an answer to the question (1320) in the UI (1320). For example, an answer to a question can be obtained using an artificial intelligence model (e.g., the artificial intelligence model (290) of FIG. 2a) based on the method described above.

[0150] Referring to FIG. 13b, according to one embodiment, when an electronic device (e.g., the electronic device (201) of FIG. 2) is implemented as a wearable electronic device (1350) that does not include a display (e.g., a glasses-type wearable electronic device), the wearable electronic device (1350) can output an answer to a question as voice. For example, the wearable electronic device (1350) can receive a user's question (1370) via voice. The wearable electronic device (1350) can output a voice (1380) corresponding to an answer to the question (1370). For example, an answer to a question can be obtained using an artificial intelligence model (e.g., the artificial intelligence model (290) of FIG. 2a) based on the method described above.

[0151] According to another embodiment, an electronic device (e.g., the electronic device (201) of FIG. 2) can provide answers to various types of user questions. For example, the electronic device (201) can display a summary of the user's daily routine on the display (260) as at least one of text, images, or video, based on images captured and saved on that day before the user goes to sleep. Additionally, the electronic device (201) can output a summary of the user's daily routine as voice, based on images captured and saved on that day before the user goes to sleep. For example, the electronic device (201) can receive a question such as "Tell me about today's daily routine" in response to the above question. For example, the electronic device (201) can output an answer containing content such as, "Today, I attended meetings at 11:00 AM, 2:00 PM, and 4:00 PM. The 11:00 AM meeting was held with X1 at location A, the 2:00 PM meeting with X2 at location B, and the 4:00 PM meeting was conducted as an online meeting. I ate a hamburger for dinner and updated the food data consumed today in the health data." For example, when a user wears another wearable electronic device (e.g., an earphone-type wearable electronic device) on their ear while wearing the wearable electronic device (1350), the wearable electronic device (1350) can transmit information about the answer to the other wearable electronic device so that the answer can be output as voice from the other wearable electronic device.

[0152] Based on the above, the electronic device (201) can provide answers to questions to the user in various ways.

[0153] In FIG. 14 below, an embodiment will be described in which an electronic device (201) obtains an answer to a user's question without using a database (202).

[0154] FIG. 14 is a flowchart illustrating a method for an electronic device according to one embodiment to output an answer to a user's query using an artificial intelligence model.

[0155] Referring to FIG. 14, according to one embodiment, in operation 1401, an electronic device (e.g., the electronic device (201) of FIG. 2a) may receive a question from a user while capturing images at a specified interval through a camera (e.g., the camera (210) of FIG. 2a).

[0156] According to one embodiment, in operation 1403, the electronic device (201) can obtain a first keyword of a question based on providing a question to an artificial intelligence model (e.g., the artificial intelligence model (290) of FIG. 2a). For example, the artificial intelligence model (290) may be stored in a server (e.g., the server (203) of FIG. 2a) or the electronic device (201).

[0157] According to one embodiment, in operation 1405, the electronic device (201) can obtain object analysis information corresponding to at least one keyword based on confirming that the first keyword matches at least one keyword among the keywords stored in memory (e.g., memory (230) of FIG. 2a). For example, the object analysis information may be stored in memory (230) and may include information on analyzing objects included in images previously captured through the camera (210). For example, the object analysis information may include information on the name, designation, quantity, color, use, and relative position of the object relative to its surroundings.

[0158] According to one embodiment, in operation 1407, the electronic device (201) can check at least one first image that was most recently captured from the time the question was received (or captured at the time the question was received).

[0159] According to one embodiment, in operation 1409, the electronic device (201) can obtain an answer to a question based on providing at least one first image, object analysis information, and information about the question to the artificial intelligence model (290). For example, unlike operation 609 of FIG. 6, the electronic device (201) may not provide location information to the artificial intelligence model (290) because it does not use a database (202). That is, instead of location information, the electronic device (201) may provide object analysis information obtained by analyzing the corresponding image to the artificial intelligence model (290).

[0160] According to one embodiment, in operation 1411, the electronic device (201) can output the obtained answer as at least one of visual information or auditory information.

[0161] Based on the method described above, the electronic device (201) can obtain an answer to a user's question using an artificial intelligence model (290) without using a database (202). The method described above can also be utilized when the artificial intelligence model (290) is stored in the electronic device (201).

[0162] Meanwhile, the artificial intelligence model described above can be implemented in the same or similar manner as the generative artificial intelligence model of FIG. 15 described below.

[0163] FIG. 15 is a diagram illustrating a generative artificial intelligence system according to one embodiment.

[0164] According to one embodiment, a user query / response interface (1510) may receive user input. The user input may be in the form of natural language, images, and / or videos, but is not limited thereto. Additionally, context information may be transmitted along with the user input. The context information may include various additional information at the time of user input. For example, the additional information may include information about the application currently being used by the user or the user's location information. Furthermore, the user input may be in a mixed form of the aforementioned natural language, images, sounds, and context information. Additionally, the user input may be in a non-natural language form, such as selecting a menu. The user query / response interface (1510) may output results from a generative artificial intelligence system to the user. The output may be in the form of natural language or specific content, and may also be provided in the form of actions requested by the user. The user query / response interface (1510) may output results from a generative artificial intelligence system to the user. The output can be in the form of natural language or specific content, and it may also be provided in the form of actions requested by the user.

[0165] The AI ​​framework (1540) can receive input from the user and coordinate and control each component necessary to perform the user's intent based on the user's query.

[0166] User input received from the user query / response interface (1510) can be transmitted to a prompt design component (1541). The prompt design component (1541) can be used to generate prompts suitable for inputting user input into a large language model (LLM) or a large multimodal model (LMM). The prompt design component (1541) may be an AI component that uses machine learning algorithms or neural networks to develop better prompts over time. The prompt design component (1541) can generate prompts by accessing a knowledge component containing user preference data, a prompt library, and prompt examples based on user input, and can transmit the generated prompts to the LLM or LMM.

[0167] The API / Plug-in management component (1542) can perform the role of communicating with external information when there is a request for additional information when user input is passed as input to a generative model. The API / Plug-in management component (1542) establishes a channel to communicate with the outside of the AI ​​Interface via the API, and through the established channel, it can enable access to various data sources (e.g., knowledge repository (1520)). Additionally, if the API / Plug-in management component (1542) needs to perform an action that executes the user input as a final step rather than an intermediate result in an application or service, it can request that action from the application / service component (1530) via the API. The information obtained from the outside can be used to generate a prompt in the prompt design component (1541) along with the user input, or it can be passed as input to the generative model.

[0168] The output modification component (or refiner component) (1543) can fine-tune the output of the generative model. For example, the output modification component (1543) can verify whether the content generated through the LLM and / or LMM is irrelevant, contains biased content, or contains harmful content. Additionally, the output modification component (1543) can determine the extent to which the output matches the desired result and, if additional processing is required, proceed with that process. Furthermore, the output modification component (1543) can configure and provide hints to the user to avoid unwanted output.

[0169] A generative AI model (1560) generally refers to an artificial intelligence neural network that generates new forms of data based on user input information. A generative AI model (1560) may include a model that generates images and / or a model that generates language. Models that generate images include, but are not limited to, GANs (generative adversarial networks) and VAEs (variational autoencoders), and examples include Diffusion-based generative models that use VAEs and Transformer structures. Models that generate language are models trained to output the most statistically appropriate output value based on input values, and examples include models such as CHAT-GPT 3 and CHAT-GPT 4. There are also LMMs (large multimodal models) that can recognize various forms of data input, such as text, images, and voice, and generate new data corresponding to them.

[0170]

[0171] According to one embodiment, the electronic device may include at least one processor (220) comprising a camera and a processing circuit, and a memory (230) for storing instructions, comprising one or more storage media. According to one embodiment, when the instructions are executed collectively or individually by the at least one processor, the electronic device may cause the electronic device to receive a user's question while capturing images at a specified interval through the camera. According to one embodiment, when the instructions are executed collectively or individually by the at least one processor, the electronic device may cause the electronic device to obtain a first keyword of the question based on providing the question to an artificial intelligence model. According to one embodiment, when the instructions are executed collectively or individually by the at least one processor, the electronic device may cause the electronic device to obtain location information indicating the location where at least one image corresponding to the at least one keyword is stored in a database outside the electronic device, based on confirming that the first keyword matches at least one keyword among the keywords stored in the memory. According to one embodiment, when the instructions are executed collectively or individually by the at least one processor, the electronic device may be caused to identify at least one first image most recently captured from the time of receiving the question. According to one embodiment, when the instructions are executed collectively or individually by the at least one processor, the electronic device may be caused to obtain an answer to the question based on providing the at least one first image, the location information, and information regarding the question to the artificial intelligence model.According to one embodiment, when the instructions are executed collectively or individually by the at least one processor, the electronic device may cause the answer to be output as at least one of visual information or auditory information.

[0172] According to one embodiment, when the instructions are executed collectively or individually by the at least one processor, the electronic device may cause the first keyword to be obtained based on providing the artificial intelligence model with the question and a first prompt requesting keyword extraction of the question.

[0173] According to one embodiment, when the instructions are executed collectively or individually by the at least one processor, the electronic device may cause the answer to be obtained by providing the artificial intelligence model with a second prompt requesting an answer to the question, along with the at least one first image, the location information, and information regarding the question.

[0174] According to one embodiment, when the instructions are executed collectively or individually by the at least one processor, the electronic device may cause the answer to the question to be obtained based on further providing the artificial intelligence model with information regarding a description of an object included in the at least one image and the time at least one image was taken.

[0175] According to one embodiment, when the instructions are executed collectively or individually by the at least one processor, the electronic device may cause the images to be transmitted to the database so that the images are stored in the database, based on capturing the images at the specified interval through the camera. According to one embodiment, when the instructions are executed collectively or individually by the at least one processor, the electronic device may cause location information indicating the location where the images are stored in the database, based on transmitting the images to the database. According to one embodiment, when the instructions are executed collectively or individually by the at least one processor, the electronic device may cause the location information to be provided to the artificial intelligence model to analyze the images using the artificial intelligence model. According to one embodiment, when the instructions are executed collectively or individually by the at least one processor, the electronic device may cause object analysis information including keywords related to objects included in each image to be obtained, based on providing the location information to the artificial intelligence model. According to one embodiment, when the instructions are executed collectively or individually by the at least one processor, the electronic device may cause the object analysis information to be stored in the memory in association with the location information of the corresponding image.

[0176] According to one embodiment, the object analysis information may include the keyword for the object, the description of the object, and information about the time when the image was taken.

[0177] According to one embodiment, when the instructions are executed collectively or individually by the at least one processor, the electronic device may cause information about the time each image was taken to be added as a watermark to the corresponding image.

[0178] According to one embodiment, when the instructions are executed collectively or individually by the at least one processor, the electronic device may cause the artificial intelligence model to obtain object analysis information for each image based on providing the artificial intelligence model with a prompt requesting location information of the images and analysis of the images.

[0179] According to one embodiment, when the instructions are executed collectively or individually by the at least one processor, the electronic device may cause the answer to the question to be obtained based on providing the artificial intelligence model with object analysis information of the at least one first image stored in the memory instead of the location information.

[0180] According to one embodiment, when the instructions are executed collectively or individually by the at least one processor, the electronic device may cause information about the answer to be displayed as text through a display included in the electronic device or information about the answer to be output as voice through a speaker included in the electronic device.

[0181] According to one embodiment, the method of operation of an electronic device may include receiving a user's question while capturing images at a specified interval through a camera included in the electronic device. According to one embodiment, the method of operation of the electronic device may include obtaining a first keyword of the question based on providing the question to an artificial intelligence model. According to one embodiment, the method of operation of the electronic device may include obtaining location information indicating the location where at least one image corresponding to the at least one keyword is stored in a database outside the electronic device, based on confirming that the first keyword matches at least one keyword among the keywords stored in the electronic device. According to one embodiment, the method of operation of the electronic device may include checking at least one first image most recently captured from the time of receiving the question. According to one embodiment, the method of operation of the electronic device may include obtaining an answer to the question based on providing the at least one first image, the location information, and information regarding the question to the artificial intelligence model. According to one embodiment, the method of operation of the electronic device may include an operation of outputting the answer as at least one of visual information or auditory information.

[0182] According to one embodiment, the operation of obtaining the first keyword may include the operation of obtaining the first keyword based on providing the artificial intelligence model with the question and a first prompt requesting keyword extraction of the question.

[0183] According to one embodiment, the operation of obtaining the answer may include the operation of obtaining the answer based on providing the artificial intelligence model with the at least one first image, the location information, and information about the question, and a second prompt requesting an answer to the question.

[0184] According to one embodiment, the operation of obtaining the answer may include the operation of obtaining the answer to the question based on further providing the artificial intelligence model with information regarding an object included in the at least one image and the time at which the at least one image was taken.

[0185] According to one embodiment, the method of operation of the electronic device may further include the operation of transmitting the images to the database so that the images are stored in the database, based on capturing the images at the specified interval through the camera. According to one embodiment, the method of operation of the electronic device may further include the operation of obtaining location information indicating the location where the images are stored in the database, based on transmitting the images to the database. According to one embodiment, the method of operation of the electronic device may further include the operation of providing the location information to the artificial intelligence model to analyze the images using the artificial intelligence model. According to one embodiment, the method of operation of the electronic device may further include the operation of obtaining object analysis information including keywords related to objects included in each image, based on providing the location information to the artificial intelligence model. According to one embodiment, the method of operation of the electronic device may further include the operation of storing the object analysis information in the memory in association with the location information of the corresponding image.

[0186] According to one embodiment, the object analysis information may include the keyword for the object, the description of the object, and information about the time when the image was taken.

[0187] According to one embodiment, the method of operation of the electronic device may further include the operation of adding information about the time each image was captured as a watermark of the corresponding image.

[0188] According to one embodiment, the operation of acquiring object analysis information may include the operation of acquiring object analysis information for each image based on providing the artificial intelligence model with a prompt requesting location information of the images and analysis of the images.

[0189] According to one embodiment, the operation of obtaining the answer may include the operation of obtaining the answer to the question based on providing object analysis information of the at least one first image stored in the memory to the artificial intelligence model instead of the location information.

[0190] According to one embodiment, in a computer-readable non-transient storage medium (130, 230) that stores instructions, the instructions may cause an electronic device (201), when executed individually or collectively by at least one processor (220), to receive a user's question while capturing images at a specified interval through a camera included in the electronic device, and based on providing the question to an artificial intelligence model, to obtain a first keyword of the question, and based on confirming that the first keyword matches at least one keyword among the keywords stored in the electronic device, to obtain location information indicating the location where at least one image corresponding to the at least one keyword is stored in a database outside the electronic device, and based on confirming at least one first image most recently captured from the time the question was received, and based on providing the at least one first image, the location information, and information about the question to the artificial intelligence model, to obtain an answer to the question and output the answer as at least one of visual information or auditory information.

[0191]

[0192] The embodiment(s) and the terms used in this document are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or any combination thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as "coupled" or "connected" to another (e.g., 2nd) component, with or without the terms "functionally" or "communicationly," it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0193] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0194] Various embodiments of this document may be implemented as software (e.g., a program) comprising one or more instructions stored in a storage medium (e.g., internal memory or external memory) readable by a machine (e.g., an electronic device). For example, a processor (e.g., a processor) of the machine (e.g., an electronic device) may call at least one of the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, "non-transitory" simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily in the storage medium.

[0195] According to one embodiment, the method according to various embodiments of the present disclosure may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TM It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0196] According to the embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to the embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to the integration. According to the embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (201), Camera (210); At least one processor (220) including a processing circuit; and The electronic device comprises one or more storage media and includes a memory (230) for storing instructions, and when the instructions are executed collectively or individually by the at least one processor, the electronic device, While capturing images at a specified interval through the above camera, receiving a user's question, and Based on providing the above question to the artificial intelligence model (290), the first keyword of the above question is obtained, and Based on confirming that the first keyword matches at least one keyword among the keywords stored in the memory, location information indicating the location where at least one image corresponding to the at least one keyword is stored in a database (202) outside the electronic device is obtained, and Check at least one first image most recently captured from the time the above question was received, and Based on providing the above at least one first image, the above location information, and information regarding the above question to the artificial intelligence model, an answer to the above question is obtained, and An electronic device that causes the above answer to be output as at least one of visual information or auditory information.

2. In paragraph 1, when the instructions are executed collectively or individually by the at least one processor, the electronic device, An electronic device that causes the acquisition of the first keyword based on providing the artificial intelligence model with the above question and a first prompt requesting keyword extraction of the above question.

3. In any one of paragraphs 1 to 2, when the instructions are executed collectively or individually by the at least one processor, the electronic device, An electronic device that causes the artificial intelligence model to obtain the answer based on providing the artificial intelligence model with at least one first image, the location information, and information regarding the question, along with a second prompt requesting an answer to the question.

4. In any one of paragraphs 1 to 3, when the instructions are executed collectively or individually by the at least one processor, the electronic device, An electronic device that causes to obtain the answer to the question based on providing the artificial intelligence model with further information regarding the description of an object included in the at least one image and the time at least one image was taken.

5. In any one of claims 1 to 4, when the instructions are executed collectively or individually by the at least one processor, the electronic device, Based on capturing the images at the specified interval through the camera, the images are transmitted to the database so that the images are stored in the database, and Based on transmitting the above images to the database, location information indicating the location where the above images are stored in the database is obtained, and To analyze the images using the artificial intelligence model, the location information is provided to the artificial intelligence model, and Based on providing the above location information to the above artificial intelligence model, object analysis information including keywords related to objects included in each image is obtained, and An electronic device that causes the above object analysis information to be stored in the memory in association with the location information of the corresponding image.

6. In any one of paragraphs 1 through 5, The above object analysis information is an electronic device comprising the above keyword for the object, the description of the object, and information about the time when the image was taken.

7. In any one of claims 1 to 6, when the instructions are executed collectively or individually by the at least one processor, the electronic device, An electronic device that causes information about the time each image was taken to be added as a watermark to the said image.

8. In any one of claims 1 to 7, when the instructions are executed collectively or individually by the at least one processor, the electronic device, An electronic device that causes the artificial intelligence model to acquire object analysis information for each image based on providing the artificial intelligence model with a prompt requesting location information of the images and analysis of the images.

9. In any one of claims 1 through 8, when the instructions are executed collectively or individually by the at least one processor, the electronic device, An electronic device that causes to obtain the answer to the question based on providing the artificial intelligence model with object analysis information of at least one first image stored in the memory instead of the location information.

10. In any one of claims 1 to 9, when the instructions are executed collectively or individually by the at least one processor, the electronic device, An electronic device that causes information about the answer to be displayed as text through a display (260) included in the electronic device or to be output as voice through a speaker (265) included in the electronic device.

11. In the method of operating the electronic device (201), The operation of receiving a user's question while capturing images at a specified interval through a camera (210) included in the electronic device; An operation to obtain a first keyword of the question based on providing the question to an artificial intelligence model (290); Based on confirming that the first keyword matches at least one keyword among the keywords stored in the electronic device, the operation of obtaining location information indicating the location where at least one image corresponding to the at least one keyword is stored in a database (202) outside the electronic device; An operation to verify at least one first image most recently captured from the time the above question was received; An operation of obtaining an answer to the question based on providing the above at least one first image, the above location information, and information regarding the above question to the artificial intelligence model; and A method of operation of an electronic device comprising the operation of outputting the above answer as at least one of visual information or auditory information.

12. In Clause 11, the operation of acquiring the first keyword is, A method of operation of an electronic device comprising the operation of acquiring the first keyword based on providing the artificial intelligence model with the above question and a first prompt requesting keyword extraction of the above question.

13. In any one of paragraphs 11 to 12, the operation of obtaining the above answer is, A method of operation of an electronic device comprising the operation of obtaining the answer based on providing the artificial intelligence model with at least one first image, the location information, and the second prompt requesting an answer to the question along with information regarding the question.

14. In any one of paragraphs 11 through 13, the operation of obtaining the answer is, A method of operation of an electronic device comprising the operation of obtaining the answer to the question based on providing the artificial intelligence model with information regarding an object included in the at least one image and the time at least one image was taken.

15. In a computer-readable non-transient storage medium (130, 230) for storing instructions, When the above instructions are executed individually or collectively by at least one processor (220), the electronic device (201) causes, While capturing images at a specified interval through the camera (210) included in the electronic device, a user's question is received, and Based on providing the above question to the artificial intelligence model (290), the first keyword of the above question is obtained, and Based on confirming that the first keyword matches at least one keyword among the keywords stored in the electronic device, location information indicating the location where at least one image corresponding to the at least one keyword is stored in a database (202) outside the electronic device is obtained, and Check at least one first image most recently captured from the time the above question was received, and Based on providing the above at least one first image, the above location information, and information regarding the above question to the artificial intelligence model, an answer to the above question is obtained, and A storage medium that causes the above answer to be output as at least one of visual information or auditory information.

Images