Method for providing summary of content, electronic device supporting same, and storage medium

Abstract

An electronic device according to an embodiment may comprise a display, at least one processor including processing circuitry, and a memory storing instructions. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to: obtain a user input for summarizing content; identify, on the basis of user data and the user input, a range of content for summarization; and on the basis of identifying whether a plurality of candidate items within the range of content match a user intention identified on the basis of the user input, identify a plurality of target items matching the user intention from among the plurality of candidate items.

Description

A method for providing a summary of content, an electronic device supporting the same, and a storage medium

[0001] The present disclosure relates to a method for providing a summary of content, an electronic device supporting the same, and a storage medium.

[0002] Driven by the remarkable advancements in information and communication technology and semiconductor technology, the distribution and use of various electronic devices are increasing rapidly. Electronic devices are being developed to enable users to carry them around and communicate. The term "electronic device" may refer to a device that performs specific functions according to an installed program, such as mobile communication terminals, tablet PCs, video / audio devices, desktop / laptop computers, or in-vehicle navigation systems.

[0003] The electronic device provides text or images upon a user's request. The provided text or images include content stored on the electronic device or content stored on an application server.

[0004] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to the present disclosure.

[0005] An electronic device according to one embodiment may include a display, at least one processor including a processing circuit, and a memory for storing instructions. The instructions may cause the electronic device to obtain user input for summarizing content when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to identify a range of content for summarization based on user data and the user input when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to identify a plurality of target items that match the user intent among the plurality of candidate items based on whether the plurality of candidate items within the range of content match the user intent identified based on the user input when executed individually or collectively by the at least one processor. Each of the candidate items may include content. The above instructions may cause the electronic device to obtain a summary result of the plurality of target items when executed individually or collectively by the at least one processor. The above instructions may cause the electronic device to display the summary result through the display when executed individually or collectively by the at least one processor.

[0006] A method according to one embodiment may include an operation of obtaining user input for summarizing content. The method may include an operation of determining the range of content for summarization based on user data and the user input. The method may include an operation of determining a plurality of target items that match the user intent among the plurality of candidate items, based on determining whether a plurality of candidate items within the range of content match the user intent determined based on the user input. Each of the candidate items may include content. The method may include an operation of obtaining a summary result of the plurality of target items. The method may include an operation of displaying the summary result through a display of an electronic device.

[0007] A storage medium recording computer-executable instructions according to one embodiment may be provided. When the computer-executable instructions are executed individually or collectively by at least one processor of an electronic device, the electronic device may cause the electronic device to obtain user input for summarizing content. When the computer-executable instructions are executed individually or collectively by at least one processor of an electronic device, the electronic device may cause the electronic device to determine a range of content for summarization based on user data and the user input. When the computer-executable instructions are executed individually or collectively by at least one processor of an electronic device, the electronic device may cause the electronic device to determine a plurality of target items that match the user intent among the plurality of candidate items based on determining whether the plurality of candidate items within the range of content match the user intent identified based on the user input. Each of the candidate items may include content. When the above computer-executable instructions are executed individually or collectively by at least one processor of the electronic device, the electronic device may cause the electronic device to obtain a summary result of the plurality of target items. When the above computer-executable instructions are executed individually or collectively by at least one processor of the electronic device, the electronic device may cause the summary result to be displayed through a display of the electronic device.

[0008] In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0009] FIG. 1 is a block diagram of an electronic device in a network environment according to one embodiment.

[0010] FIG. 2 is a block diagram of an electronic device according to one embodiment.

[0011] FIG. 3 is a block diagram for illustrating an electronic device according to one embodiment.

[0012] FIG. 4a is an illustrative diagram for explaining a method for identifying target items of an electronic device according to one embodiment.

[0013] FIG. 4b is an illustrative diagram for explaining a method of obtaining a summary result of an electronic device according to one embodiment.

[0014] FIG. 5 is a flowchart illustrating a method for providing summary results of an electronic device according to one embodiment.

[0015] FIGS. 6a, FIGS. 6b, FIGS. 6c, FIGS. 6d, FIGS. 6e, FIGS. 6f, FIGS. 6g, and FIGS. 6h are illustrative diagrams for explaining a method of providing a summary result of an electronic device according to one embodiment.

[0016] FIG. 7 is a flowchart illustrating a method for setting the format of a summary result of an electronic device according to one embodiment.

[0017] FIG. 8 is an illustrative diagram for explaining a method of setting the format of a summary result of an electronic device according to one embodiment.

[0018] FIG. 9 is a flowchart illustrating a method for providing summary results based on the scope of the content of an electronic device according to one embodiment.

[0019] FIG. 10 is a flowchart illustrating a method for providing an integrated summary document of an electronic device according to one embodiment.

[0020] FIGS. 11a and FIGS. 11b are illustrative diagrams for explaining a method of displaying an object associated with a source of an electronic device according to one embodiment.

[0021] FIG. 12 is a flowchart illustrating a method for providing summary results of a wearable electronic device according to one embodiment.

[0022] FIGS. 13 and FIGS. 14 are illustrative diagrams for explaining a method for providing summary results of a wearable electronic device according to one embodiment.

[0023] FIG. 15 is a diagram illustrating a generative artificial intelligence model according to one embodiment.

[0024] Hereinafter, embodiments of the present disclosure are described in detail with reference to the drawings so that those skilled in the art can easily practice them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein. In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and brevity.

[0025] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to one embodiment.

[0026] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0027] The processor (120) can control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., a program (140)), and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0028] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0029] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0030] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0031] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0032] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0033] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0034] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0035] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0036] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0037] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0038] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that can be perceived by the user through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0039] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0040] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0041] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0042] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0043] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) may support a Peak data rate (e.g., 20 Gbps or more) for eMBB realization, loss coverage (e.g., 164 dB or less) for mMTC realization, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for URLLC realization.

[0044] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0045] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0046] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0047] According to one embodiment, commands or data may be transmitted or received between an electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0048] FIG. 2 is a block diagram of an electronic device according to one embodiment.

[0049] In one embodiment, the electronic device (201) may be the electronic device (101) of FIG. 1.

[0050] Referring to FIG. 2, in one embodiment, the electronic device (201) may include a display (210), a memory (220), and / or a processor (230).

[0051] In one embodiment, the display (210) may be included in the display module (160) of FIG. 1.

[0052] In one embodiment, the display (210) may display a screen containing summary results. For example, the display (210) may display a screen containing summary results of content based on a control signal of the processor (230). The screen displayed by the display (210) will be described in detail later in FIGS. 3 to 15.

[0053] In one embodiment, the memory (220) may be included in the memory (130) of FIG. 1.

[0054] In one embodiment, the memory (220) may store information for providing a summary result.

[0055] In one embodiment, the memory (220) may store a generative AI (artificial intelligence) model. The generative AI model may include an artificial intelligence model trained to output content similar to the input content based on receiving content such as text, audio, and / or images. The generative AI model may learn, for example, patterns of content included in training data. The type of content output by the generative AI model may include at least one of images, audio, or text associated with a description of the output content. In one embodiment, the operation of acquiring content using the generative model may be referred to as the "operation of generating content." In one embodiment, the generative AI model may be stored in an external electronic device.

[0056] In one embodiment, the processor (230) may be included in the processor (120) of FIG. 1.

[0057] In one embodiment, the processor (230) may control the overall operation of providing a summary result. The processor (230) may include one or more processors for performing the operation of providing a summary result. For example, the processor (230) may correspond to a plurality of processors that collectively perform a plurality of operations by dividing them among the processors. The operation of providing a summary result performed by the processor (230) will be described in detail with reference to FIGS. 3 through 15.

[0058] In one embodiment, the processor (230) may include a neural processing unit (NPU) for performing a content summarization operation. For example, the processor (230) may include an NPU capable of performing a content summarization operation using an artificial intelligence model when a content summarization operation is performed, but is not limited thereto. For example, the processor (230) may include a graphic processing unit (GPU) capable of performing a content summarization operation using a specified algorithm when a content summarization operation is performed using a specified algorithm.

[0059] In FIG. 2, the electronic device (201) is illustrated as including a display (210), memory (220), and / or a processor (230), but is not limited thereto. In one embodiment, the electronic device (201) may further include at least one of the components included in the electronic device (101) of FIG. 1. For example, the electronic device (201) may further include at least one of an input module (150) (e.g., microphone), an audio output module (155) (e.g., speaker), or a camera module (180).

[0060] FIG. 3 is a block diagram for explaining an electronic device according to one embodiment. The embodiment of FIG. 3 will be explained with reference to FIG. 4a and FIG. 4b. FIG. 4a is an illustrative diagram for explaining a method for identifying target items of an electronic device according to one embodiment. FIG. 4b is an illustrative diagram for explaining a method for obtaining a summary result of an electronic device according to one embodiment.

[0061] In one embodiment, referring to FIG. 3, an electronic device (201) (e.g., the electronic device (101) of FIG. 1 and / or the electronic device (201) of FIG. 2) may include a user input module (310), a content selection module (320), a content processing module (330), and / or an output module (340). The electronic device (201) may provide an integrated summary of a plurality of items (e.g., each of the items includes at least one content among text, still images, or video).

[0062] In one embodiment, modules (e.g., at least one of a user input module (310), a content selection module (320), a content processing module (330), or an output module (340)) implemented in the electronic device (201) (or stored in memory (220)) may be implemented in the form of a program, computer code, instructions, routines, processes, software, applications, firmware, or a combination of at least two of these, which are executable by a processor (e.g., the processor (230) of FIG. 2). For example, when at least one of the modules is executed, the processor (230) may perform an operation corresponding to at least one module. The description "a specific module performs an operation" below may be understood as "the processor (230) performs an operation corresponding to a specific module as the specific module is executed." In one embodiment, at least some of the modules may include a plurality of programs, but are not limited to those described. Meanwhile, at least some of the modules may be implemented in a hardware form. For example, at least some of the user input module (310), content selection module (320), content processing module (330), or output module (340) may be implemented as a processing circuit (not shown). In one embodiment, the modules may be implemented as a service or application when running on an Android operating system.

[0063] In one embodiment, the modules (e.g., at least one of the user input module (310), content selection module (320), content processing module (330), or output module (340)) may output a summary result of the content based on user input. In one embodiment, the content selection module (320) and / or the content processing module (330) may include a generative AI model (327, 334). The generative AI model (327, 334) may be an AI model trained to output information associated with the summary result of the content (or information associated with selected content) based on receiving data input. The content may include at least one of a moving image or video, a still image or video, text, audio (e.g., a recorded file), or a website link (e.g., a uniform resource identifier (URI)). The content may include a combination of at least two of the moving image, photo, text, audio, or website link. For example, the electronic device (201) may output a short text summary by considering trends (e.g., distribution and / or pattern) of data associated with a user of the electronic device (201) (e.g., a user identified based on a user account or identification information) based on receiving a long text input. The electronic device (201) may provide (or extract) a transcript, summary, and / or thumbnail of the video (e.g., a representative image of the video) by considering trends in data associated with a user of the electronic device (201) based on receiving an image (e.g., a still image and / or video input). The device (201) can provide a text description corresponding to an image by considering the trends of data associated with the user of the electronic device (201) based on receiving an image input.The electronic device (201) can provide a description of an object or scene identified within an image based on the analysis of an input image. The electronic device (201) can provide a summary of the content using a level of detail corresponding to the expertise identified based on user data. If a high level of expertise is identified based on user data, the electronic device (201) can provide a summary of the image description at a more detailed level. After providing the summary of the content, the electronic device (201) can perform additional actions. The electronic device (201) may be implemented, for example, as a retrieval augmented generation (RAG). If an additional query is entered by a user based on the summary of the image description, the electronic device (201) can provide an additional answer through an external interface (e.g., an external knowledge source). If a recipe query regarding the description of food identified within an image is entered, the electronic device (201) can provide an additional answer related to recipe recommendations through an external interface. The electronic device (201) can provide a summary corresponding to the content connected by the website link by considering the trends of data associated with the user of the electronic device (201) based on receiving a website link (e.g., a hyperlink of a video provided by a video platform). The user of the electronic device (201) can obtain a summary corresponding to the content (e.g., a text script of the video converted via STT (speech-to-text)) without consuming (or watching) the entire content of the website (e.g., a video). The summary corresponding to the content may provide a link to a segment (or frame) of the video mapped to the text summary. The user of the electronic device (201) can understand the content in detail by playing the video of the segment corresponding to the link.In one embodiment, the electronic device (201) may be implemented as conversational artificial intelligence. The conversational artificial intelligence may provide a response corresponding to the user's intent identified by the user input, based on receiving user input (e.g., user utterance). The electronic device (201) may provide a summary corresponding to the user's purpose of consuming the content. The user's user experience may be enhanced as the electronic device (201) provides a content summary based on the user's data (or intent). For example, the user of the electronic device (201) may quickly preview the content based on a summary of a vast amount of content without having to read, view, or listen to all the content.

[0064] In one embodiment, the electronic device (201) may provide an integrated summary based on integrating various content. The electronic device (201), for example, upon receiving a request for a summary of multiple documents, may go beyond simply providing a summary in which the amount of text is reduced based on high-frequency tokens within the text, and may provide a summary of target content corresponding to the user's intent (e.g., the user's perspective on the content or the subject of the content). The electronic device (201) may provide a summary of target content corresponding to the user's purpose of the summary based on receiving a request for a summary of various types of composite content including videos, photos, and text. In addition to options for the amount of summary content, the electronic device (201) may provide summary content at a detail level corresponding to expertise identified based on user input and / or user data. The electronic device (201) may identify that the expertise for summarizing the content is relatively high based on an input requesting a summary of the sales revenue of a specific product over a specific period from the sales data of a product family. The electronic device (201) can provide summary content with a high level of detail based on verifying expertise, even when the amount of input content is not large.

[0065] In one embodiment, the user input module (310) may receive user input (or user command). User input may be referred to as a "prompt." The prompt may include at least one of information, for example, text, an image, or audio. If the prompt includes audio information, the user input, which includes voice input based on natural language, may be referred to as a "user utterance." The user utterance may be a command that includes a request for a specific task or a request for a summary of content, such as "plan a trip based on recent search history" or "organize the social media data I watched today." The user input module (310) may convert the user utterance into text information using STT, for example, when the user utterance is input. In one embodiment, the user input may include touch input to an object (for example, an icon associated with the execution of a summary function) displayed through a touch interface (for example, a display (210)). Touch input may include sequential inputs including inputs associated with the designation of a summary target (e.g., designation of a block of summary targets) and inputs associated with a batch summary request for the summary targets.

[0066] In one embodiment, the content selection module (320) may select data for performing content summarization based on receiving a prompt (e.g., user utterance and / or text input). The content selection module (320) may select data for performing content summarization based on user data stored in an electronic device (201) (e.g., memory (220)). The data for performing content summarization may include application data if an application programming interface (API) of the application is provided. The content selection module (320) may include a content determination module (321), a user data database (322), an input prompt history management module (323), a preferred content management module (324), a search history management module (325), and a stored content management module (326).

[0067] In one embodiment, the content determination module (321) can identify (or determine) the scope of the content based on a prompt using user data in the user data DB (322). For example, the scope of the content may include at least one of the period of the data, the type of the data, the type of the application, or the subject of the summary. The content determination module (321) can select (or extract) the summary target data identified from the user utterance based on local data and / or application data.

[0068] In one embodiment, the content determination module (321) can select data based on the confirmed period of data when the user utterance includes information that can verify the period of data. For example, when a user utterance such as "Organize the SNS content I watched today" is input, the content determination module (321) can verify the period of data and obtain SNS application data corresponding to the confirmed period. In one embodiment, when the user utterance does not include information that can verify the period of data, the content determination module (321) can obtain data to be summarized based on conditions for selecting data to be summarized. When a user utterance such as "Plan a trip based on recently watched content" is input, where the specific period of data cannot be determined, the content determination module (321) can verify the conditions set for the period of data. The content determination module (321) can verify the range from data obtained 7 days ago to data obtained currently based on the condition specified by the system corresponding to 7 days. In one embodiment, the content determination module (321) may determine the period of data based on the category of content identified by user utterance. When user utterance that cannot specify a specific period of data is input, such as "plan a trip based on recently watched content," the content determination module (321) may identify "travel" as the category of content to be summarized. The content determination module (321) may determine the point in time when data in the travel category began to be acquired intensively (or consumed by the user of the electronic device (201)). For example, the content determination module (321) may determine the range from data acquired 5 days ago to data acquired now based on confirming that travel-related content began to be acquired 5 days ago.

[0069] In one embodiment, the content determination module (321) can select data based on the identified application type when the user utterance includes information that can identify the application type. For example, when a user utterance such as "Organize the SNS content I saw today" is input, the content determination module (321) can identify that the application type is an SNS application and obtain SNS application data. In one embodiment, when the user utterance does not include information that can identify the application type, the content determination module (321) can obtain data to be summarized based on conditions for selecting data to be summarized. When a user utterance such as "Plan a trip based on recent searches" is input, where the application type cannot be specified, the content determination module (321) can check the conditions set for the application type. The content determination module (321) can identify an application related to search as the application type based on the word "search" among the tokens identified by the user utterance. The content determination module (321) can acquire (or extract) user data of, for example, "circle to search", voice assistant, web browser application, and third-party search application.

[0070] In one embodiment, the content determination module (321) can determine the range of data to be summarized based on the topic of the summary identified based on user input when the user utterance includes information that can identify the topic of the summary. For example, when a user utterance such as "plan a trip based on recent searches" is input, the content determination module (321) can identify that the topic of the summary is "travel." The content determination module (321) can acquire (or output) data related to the topic of the summary (e.g., travel) among the data of a specific application acquired during a specific period.

[0071] In one embodiment, the content determination module (321) may prioritize selecting data of relatively high interest to the user when the range of data is not confirmed by user utterance (e.g., input of ambiguous user utterance). The content determination module (321) may obtain data of high interest to the user based on data obtained by at least one of, for example, an input prompt history management module (323), a preferred content management module (324), a search history management module (325), or a stored content management module (326).

[0072] In one embodiment, data obtained by the input prompt history management module (323) may include prompts entered by a user. The history of prompts entered by a user may include the history of utterances entered when the user uses an AI agent (e.g., LLM). The history of prompts entered by a user may also include the history of settings for the AI ​​agent. For example, the history of prompts may include prompts such as "Plan a trip based on recent search history," "Organize the social media data I watched today," "Summarize the content of media streaming applications and social media applications," "Summarize the product information I was interested in," "Summarize the dress products I checked today so I can compare them," or "Summarize the styling methods for the dresses I looked up." In one embodiment, data obtained by the search history management module (325) may include keywords and / or search results searched by a user of the electronic device (201). The content determination module (321) can estimate the context associated with the user's interest based on the data provided by the search history management module (325). Based on the fact that data consistency can be associated with the user's interest, the content determination module (321) can select the data provided by the search history management module (325) when conditions for verifying data consistency are satisfied. Conditions for verifying data consistency may include performing additional related searches after a search and / or performing searches repeatedly in adjacent time intervals, but there are no limitations on examples of conditions.

[0073] In one embodiment, the data obtained by the favorite content management module (324) may include data designated as favorite content using a function (e.g., 'save' or 'like') used in the content platform. The favorite content management module (324) can identify the trend (or ratio) of data identified within the favorite content based on the fact that the purpose of the content consumed by the user may be associated with the user's preference. The favorite content management module (324) can identify an overall direction related to the user's daily interests based on data classified at the content platform level (e.g., posts (feed), videos, and / or images).

[0074] In one embodiment, the data obtained by the stored content management module (326) may include content collected by the user based on a specific purpose (e.g., content saved (or scraped) by the user using a capture or text saving function). The content determination module (321) may identify (or infer) major category keywords associated with the user's interest based on identifying similarities within the stored content (e.g., identifying duplicate content or high-frequency content).

[0075] In one embodiment, the content determination module (321) may prioritize acquiring (or extracting) data of high user interest among the acquired application user data as data to be summarized. For example, after confirming the range of data, the content determination module (321) may finely adjust the range of content based on the user's interest. The content determination module (321) may select (or extract) high-priority data among the entire user data as data to be summarized. For example, when a user utterance such as "Organize the SNS data I watched today" is acquired, the content determination module (321) may select data of high user interest among the data of a specific application acquired during a specific period. The content determination module (321) may select content related to "cat" when content related to "cat" is repeated among the data of the SNS application.

[0076] In one embodiment, the content processing module (330) can identify items that match the user intent based on comparing the selected content provided by the content selection module (320) with the user intent. The content processing module (330) can obtain a summary of each of the items that match the user intent. The content processing module (330) can obtain a summary corresponding to the user intent identified based on user utterance and / or user data. The content processing module (330) can obtain a summary at the detail level identified based on user utterance and / or user data. The content processing module (330) may include a user intent and content analysis module (331), an integrated content generation module (332), and / or a link generation module (333).

[0077] Referring to FIG. 4a, in one embodiment, the user intent and content analysis module (331) can identify the user intent based on the user utterance. Referring to reference numerals 410 and 430, the user intent and content analysis module (331) can identify target items (411, 414, 416, 417, 418) that match the user intent based on comparing (421) the candidate items (411, 412, 413, 414, 415, 416, 417, 418) (or major category content) provided by the content selection module (320) with the user intent. The user intent and content analysis module (331) can identify, for example, items of similar context and / or items with high repetitiveness among candidate items (411, 412, 413, 414, 415, 416, 417, 418) as target items (411, 414, 416, 417, 418). In one embodiment, the target items (411, 414, 416, 417, 418) may be selected based on whether the candidate items match the user intent, regardless of the type of content (e.g., text, image, and / or video) included in each of the candidate items (411, 412, 413, 414, 415, 416, 417, 418). When the content selection module (320) selects candidate items (411, 412, 413, 414, 415, 416, 417, 418) using data from a video platform (or, video streaming application), the target items (411, 414, 416, 417, 418) may include video content of a single media type.When the content selection module (320) selects candidate items (411, 412, 413, 414, 415, 416, 417, 418) using various types of application data including image applications, text applications, and media streaming applications, the target items (411, 414, 416, 417, 418) may include content of various media types (e.g., text, images, maps, videos, and / or audio). The target items may be referred to as "interest target areas." In one embodiment, a summary of the remaining candidate items (412, 413, 415), excluding the target items (411, 414, 416, 417, 418), may be provided in the form of footnotes within the summary document provided at the end.

[0078] Referring to reference numeral 440, in one embodiment, the user intent and content analysis module (331) may obtain summaries (441, 442, 443, 444, 445) corresponding to each of the target items (411, 414, 416, 417, 418). Each of the summaries (441, 442, 443, 444, 445) corresponding to the target items (411, 414, 416, 417, 418) may be referred to as a summary document. For example, the first target item (411) may include content of a pure text type. The content of the target item may be converted into a summary document. The user intent and content analysis module (331) can obtain a summary document (441) composed of sequential paragraphs including a title, subheading, and body, based on summarizing (431) the first target item (411) using a generative AI model (e.g., generative AI model (334)). The second target item (414) may include video type content. The user intent and content analysis module (331) can obtain a summary document (442) of video content converted into text based on summarizing (432) the second target item (414) using a generative AI model. The third target item (416) may include image type content. The user intent and content analysis module (331) can obtain a summary document (443) of image content converted into text based on summarizing (433) the third target item (416) using a generative AI model. The fourth target item (417) may include composite content of image and text types. The user intent and content analysis module (331) can obtain a summary document (444) of composite content converted into text based on summarizing (434) the fourth target item (417) using a generative AI model.The fifth target item (418) may include composite content of text, image, and video types. The user intent and content analysis module (331) may obtain a summary document (445) of the composite content converted into text based on summarizing (435) the fifth target item (418) using a generative AI model. The operation of the user intent and content analysis module (331) obtaining summaries (441, 442, 443, 444, 445) corresponding to each of the target items (411, 414, 416, 417, 418) may be referred to as the "operation of obtaining (or executing) a primary summary." The summaries (441, 442, 443, 444, 445) corresponding to each of the target items (411, 414, 416, 417, 418) may include duplicate content.

[0079] In one embodiment, the integrated content generation module (332) can integrate summaries (441, 442, 443, 444, 445) corresponding to each of the target items (411, 414, 416, 417, 418). Referring to reference numeral 460 in FIG. 4b, the integrated content generation module (332) can obtain an integrated summary (461) of the target items based on integrating (451) the summaries (441, 442, 443, 444, 445) corresponding to each of the target items. The operation of integrating the target items may be referred to as the operation of merging the summarized documents. The merged summary document may include duplicate content and / or repeated sentences. The integrated content generation module (332) can identify content that matches the user's intent. For example, the integrated content generation module (332) can identify sections connecting major topics, duplicate content, and / or summarized content within the integrated summary (461). Referring to reference numeral 470 in FIG. 4b, the integrated content generation module (332) can obtain a summary result of content (471) based on categorizing multiple summary documents (463). The summary result of content (471) may be a summary document composed of sequential paragraphs including a title, a subtitle, and a body text. The summary result of content (471) may include images, text, and / or hyperlinks (e.g., URIs) associated with the original source. In one embodiment, the paragraph composition of the summary result of content (471) may be set based on contextual information that corresponds to the user's intent. For example, if the summary purpose corresponding to the input prompt is "travel planning," a paragraph composition corresponding to travel planning may be set. The summary result for the travel plan may include the name of the travel destination, a map showing the location of the travel destination, or information associated with at least one of the activities (e.g., text, images, and / or link information).If the purpose of the summary corresponding to the input prompt is "organizing consumed SNS content," a paragraph structure corresponding to the organization of SNS content may be set. The summary result for organizing SNS content may include a summary of content with high importance (e.g., frequency) among the content consumed by the user.

[0080] In one embodiment, contextual information corresponding to user intent may be obtained based on a pattern (or distribution) identified based on user data of the electronic device (201). User data (e.g., consumed content or preferred content) may reflect the interests of the user of the electronic device (201). The user's interests may be identified based on a pattern (e.g., directionality or algorithm) associated with the user's consumed content and / or the user's actions (e.g., user's reaction to writing or commenting by the user). The electronic device (201) may identify user intent based on at least one of search history, preferred content, or stored content. For example, first contextual information corresponding to user data related to artwork or art history (or a user with high interest in art) may differ from second contextual information corresponding to user data related to world history or world culture (or a user with high interest in history). The electronic device (201) can check contextual information based on user data when a prompt such as "plan a trip" is entered. The electronic device (201) can provide summary results of different configurations and / or tables of contents based on different contextual information even when the same prompt is entered. For example, the electronic device (201) can provide summary results recommending places significant in art history within the "History" category based on first contextual information. The electronic device (201) can provide summary results recommending cultural properties significant in world history within the "History" category based on second contextual information.

[0081] In one embodiment, the link generation module (333) can generate a link (e.g., a URI) to the original source of the content included in the summary result.

[0082] In one embodiment, the output module (340) can output a summary result provided by the content processing module (330). For example, the output module (340) can display the summary result through a display (e.g., a display (210)).

[0083] FIG. 5 is a flowchart illustrating a method for providing a summary result of an electronic device according to one embodiment. The embodiment of FIG. 5 is described with reference to FIG. 6a, 6b, 6c, 6d, 6e, 6f, 6g, and 6h. FIG. 6a, 6b, 6c, 6d, 6e, 6f, 6g, and 6h are illustrative diagrams illustrating a method for providing a summary result of an electronic device according to one embodiment.

[0084] In one embodiment, the operations illustrated in FIG. 5 may be performed in various orders, not limited to the order illustrated. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. According to one embodiment, more operations may be performed than those illustrated in FIG. 5, or at least one fewer operation may be performed.

[0085] Referring to FIG. 5, in operation 501, in one embodiment, an electronic device (201) (e.g., electronic device (201)) may obtain user input for summarizing content. The user input may be speech and / or text input, but is not limited thereto. Referring to FIG. 6a, in one embodiment, the electronic device (201) may execute (or execute on foreground) an application for summarizing based on acknowledging user input (603) (e.g., touch input) for at least a portion area (601) of the display (210). Referring to FIG. 6b, in one embodiment, the electronic device (201) may display a guide message (611) for prompt input through the display (210). The electronic device (201) may display a window for prompt input based on user input (615) (e.g., touch input) for an object (613) (e.g., an object for executing a summary) displayed through the display (210). Referring to FIG. 6c, in one embodiment, the electronic device (201) may display text (621) corresponding to user input on the window (620) for prompt input. Based on confirming (624) user input (623) (e.g., touch input) for an object (622) (e.g., an object for executing a summary) displayed through the display (210), a guide message (625) indicating that a summary is being executed may be displayed through the display (210). The electronic device (201) may run an application for summarization in the background based on checking user input (625) (e.g., touch input) for at least some area (601) of the display (210) while summarizing the content is being performed.

[0086] In operation 503, in one embodiment, the electronic device (201) can determine the scope of content for summary based on user data and the user input. Information associated with the scope of content may include information associated with the purpose of the summary, the type of user data, and the duration of the user data. The type of user data may include text, still images, and video. The user data may include local data stored in the electronic device (201). For example, the local data may include image data of a gallery application or text data of a memo application. The purpose of the summary may include travel plans, cooking recipes, or shopping suggestions, but is not limited thereto. The user data may include text data, still image data, and video data associated with the execution history of at least one application. At least one application may include at least one of a preloaded application or an application provided by an application platform.

[0087] In one embodiment, the electronic device (201) can determine the scope of content based on application data when the topic of the summary and the type of application are identified based on user input. When a user command such as "summarize the content of the media streaming application and the content of the SNS application" is entered, the electronic device (201) can determine the topic of the summary based on data of the media streaming application (e.g., images and / or videos) and data of the SNS application (e.g., images, videos, and / or text). The electronic device (201) can determine content related to aerobic exercise, weight loss habits, diet recipes, and sportswear as the scope of content based on, for example, confirming that content related to "weight loss" is repeated.

[0088] In operation 505, in one embodiment, the electronic device (201) can determine whether a plurality of candidate items within the range of content (e.g., a plurality of candidate items (411, 412, 413, 414, 415, 416, 417, 418)) match a user intent identified based on user input. Each of the candidate items may include content (e.g., text, still image, video, or a combination thereof).

[0089] In one embodiment, based on confirming that at least some of the candidate items match the user's intent (operation 505-e.g.), the electronic device (201) can, in operation 507, identify a plurality of target items (e.g., a plurality of target items (411, 414, 416, 417, 418)) that match the user's intent among the candidate items. The electronic device (201) can identify target items that include content that is repeated or identified as having high user interest among the SNS content collected over the day, based on an input prompt such as "Summarize the SNS feed I watched today."

[0090] In operation 509, in one embodiment, the electronic device (201) can obtain summary results of a plurality of target items. The electronic device (201) can, for example, identify content that is repeated or emphasized among the plurality of target items as content of high importance. The electronic device (201) can identify a template of the summary results based on the importance of the content included in the summary results. In one embodiment, the electronic device (201) can identify information related to the ratio of video content in the summary results. The electronic device (201) can set the format of the summary results to a text format based on confirming that the ratio of video content is below a threshold ratio. The electronic device (201) can, for example, provide a summary document in a pure text format. The electronic device (201) can set the format of the summary results to a video format based on confirming that the ratio of video content exceeds a threshold ratio. The electronic device (201) may provide a summary result of composite content including, for example, an image (e.g., a representative image of a still image or video) and text. In one embodiment, based on confirming that at least some of the plurality of candidate items do not match the user's intent (operation 505-No), the electronic device (201) may, in operation 511, provide an auxiliary summary result of the remaining items excluding the plurality of target items. Data that is not selected as content to be summarized may be preserved as data in a brief format. The preserved data may be provided auxiliaryly in addition to the summary data that is finally provided.

[0091] In operation 513, in one embodiment, the electronic device (201) may display summary results through a display (e.g., display (210)). Referring to FIG. 6d, in one embodiment, the electronic device (201) may display a guide message (631) indicating that the summary is complete through the display (210). The electronic device (201) may display a guide message (631) indicating that a summary task performed in the background is complete, for example, while another application is running in the foreground. The electronic device (201) may display a window containing summary results through the display (210) based on user input (633) for at least a portion area (632) of the display (210). Referring to FIG. 6e, in one embodiment, the electronic device (201) may display a window (640) containing summary results through the display (210). The electronic device (201) may display an object (641) associated with the summary result and an object (642) associated with the summary history on a window (640) containing the summary result. The electronic device (201) may display a detailed summary result through a display (210) based on user input (644) for the object (641) associated with the summary result. Referring to FIG. 6f, in one embodiment, the electronic device (201) may display a detailed summary result (650) through a display (210). For example, when a prompt such as "plan a trip" is entered, the detailed summary result (650) may include a map image (651) showing the approximate location of the travel destination, text associated with a list of travel destinations, and text associated with activities. The detailed summary result (650) may further include a link associated with the source of the summary content.The electronic device (201) may display a window for providing original content (e.g., referred to as a "source finder") based on user input (652) (e.g., touch input) for providing original content. Referring to FIG. 6g, in one embodiment, the electronic device (201) may display a window (661) for providing original content over a detailed summary result (660). The electronic device (201) may display original link information of the summary content. The electronic device (201) may display original content through a display (210) based on user input (662) for the window (661) for providing original content. Referring to FIG. 6h, in one embodiment, the electronic device (201) may display an object (670) associated with original content through a display (210).

[0092] FIG. 7 is a flowchart illustrating a method for setting the format of a summary result of an electronic device according to one embodiment. The embodiment of FIG. 7 will be described with reference to FIG. 8. FIG. 8 is an illustrative diagram illustrating a method for setting the format of a summary result of an electronic device according to one embodiment.

[0093] In one embodiment, the operations illustrated in FIG. 7 may be performed in various orders, not limited to the order illustrated. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. According to one embodiment, more operations may be performed than those illustrated in FIG. 7, or at least one fewer operation may be performed.

[0094] Referring to FIG. 7, in operation 701, in one embodiment, an electronic device (201) (e.g., electronic device (201)) may obtain user input for summarizing content. Operation 701 may be at least partially identical or similar to operation 501, and descriptions that overlap with operation 501 may not be repeated herein.

[0095] In operation 703, in one embodiment, the electronic device (201) can determine the range of content for summary based on user data and user input. Operation 703 may be at least partially identical or similar to operation 503, and descriptions that overlap with operation 503 may not be repeated here.

[0096] In operation 705, in one embodiment, the electronic device (201) can identify a plurality of target items that match the user's intention among a plurality of items within the range of content. Operation 705 may be at least partially identical or similar to operation 507, and descriptions that overlap with operation 507 may not be repeated here.

[0097] In operation 707, in one embodiment, the electronic device (201) may obtain summary results of a plurality of target items. In one embodiment, user intent may be identified based on the distribution (or directionality) of user data. For example, user intent may be identified based on search history or content consumption patterns. The electronic device (201) may set a high importance for content that matches user intent based on comparing data primarily selected for summary from user data with user intent. The importance of content that matches user intent may be set higher than the importance of content that does not match user intent. In one embodiment, the electronic device (201) may set a high importance for content that is repeated or emphasized within the data primarily selected for summary (e.g., objects identified within an image or words identified within text). The electronic device (201) can set different levels of importance corresponding to target items based on the frequency of content consumption in response to a prompt such as "summarize recent travel-related search history." For example, if it is confirmed based on user data that the frequency of consumption of content related to food or cooking is high, the importance of content related to food or cooking can be set high. Based on the frequency of content that is repeated or emphasized in each of the multiple target items, a higher level of importance can be set sequentially for the content that is repeated or emphasized. Operation 707 may be at least partially identical or similar to operation 509, and descriptions that overlap with operation 509 may not be repeated here.

[0098] In operation 709, in one embodiment, the electronic device (201) can determine whether the ratio of video content exceeds a threshold ratio. The electronic device (201) can determine information related to the ratio of video content in the summary results. The electronic device (201) can provide summary results of different formats based on user input and / or the type of content. Based on determining that the ratio of video content exceeds a threshold ratio (operation 709-e), in one embodiment, the electronic device (201) can set the format of the summary results to a video format in operation 711. For example, the summary results may include text (e.g., lists and / or descriptions), images (e.g., tables, maps, photos, and / or drawings), and webpage links. The ratios of text, images, and webpage links within the summary results may be set differently. The electronic device (201) can provide summary results of composite content including, for example, images (e.g., still images or representative images of video) and text. The electronic device (201) may provide product images corresponding to recurring content identified from user data as at least part of the summary result based on a prompt such as, "Summarize the product information I was interested in." The electronic device (201) may provide a summary result of composite content including images of dresses, text associated with the price and reviews of the dresses, and links to the products' web pages based on receiving a prompt such as, "Summarize the dress products I looked up today so I can compare them." The electronic device (201) may provide a summary result of composite content including images of outfits associated with the searched dresses and text descriptions associated with styling based on receiving a prompt such as, "Summarize how to style the dresses I looked up."The electronic device (201) can improve the user experience by providing a summary result of complex content so that the user can easily check the summary result, in contrast to providing a summary result that includes only text.

[0099] In one embodiment, based on confirming that the ratio of video content is below a threshold ratio (operation 709-No), the electronic device (201) may set the format of the summary result to a text format in operation 713. The electronic device (201) may, for example, provide a summary document in a pure text format. Referring to FIG. 8, in one embodiment, the electronic device (201) may provide an integrated summary document (810) composed of multiple tables of contents (811, 812, 813, 814, 815, 816, 817). The integrated summary document (810) may be composed of a table of contents including, for example, a main section (811), a section (812), a subsection (813), a paragraph (814), subparagraphs (815), a conclusion (816), and references (817). The electronic device (201) may provide summary results in different formats based on the proportion of the types of content identified within the summary results.

[0100] FIG. 9 is a flowchart illustrating a method for providing summary results based on the scope of the content of an electronic device according to one embodiment.

[0101] In one embodiment, the operations illustrated in FIG. 9 may be performed in various orders, not limited to the order illustrated. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. According to one embodiment, more operations may be performed than those illustrated in FIG. 9, or at least one fewer operation may be performed.

[0102] Referring to FIG. 9, in operation 901, in one embodiment, an electronic device (201) (e.g., electronic device (201)) may obtain user input for summarizing content. Operation 901 may be at least partially identical or similar to operation 501, and descriptions that overlap with operation 501 may not be repeated herein.

[0103] In operation 903, in one embodiment, the electronic device (201) can determine whether the user input includes information related to the scope of the content. For example, the electronic device (201) can determine whether information related to the purpose of the summary, the target of the summary, or the duration of the content is determined based on the user input (utterance and / or text).

[0104] In one embodiment, based on confirming that the user input includes information related to the scope of the content (operation 903-e), the electronic device (201) can, in operation 905, confirm the scope of the content among the user data. The electronic device (201) can primarily select candidate items for summarization based on the information related to the scope of the content.

[0105] In one embodiment, based on confirming that user input does not include information associated with the scope of content (operation 903-No), the electronic device (201) may, in operation 907, determine the scope of content based on information associated with search history, input prompt history, preferred content, and stored content. The electronic device (201) may, based on user data, determine the scope of content for summary based on that the scope of content is not specified by user input.

[0106] In one embodiment, the electronic device (201) may provide a message guiding the input of information related to the scope of the content when the scope of the content is not specified by user input. For example, the guide message may include a request for input of information to specify the purpose of the summary, the target of the summary, or the duration of the content. When a prompt such as "summarize it" is entered, the electronic device (201) may provide a message guiding the input of information related to the target of the summary or the method of summary.

[0107] In operation 909, in one embodiment, the electronic device (201) can identify a plurality of target items that match the user's intention among a plurality of items within the range of content. Operation 909 may be at least partially identical or similar to operation 507, and descriptions that overlap with operation 507 may not be repeated here.

[0108] In operation 911, in one embodiment, the electronic device (201) may obtain a summary result of a plurality of target items. Operation 911 may be at least partially identical or similar to operation 509, and descriptions that overlap with operation 509 may not be repeated herein.

[0109] In operation 913, in one embodiment, the electronic device (201) may provide a summary result. For example, the electronic device (201) may display the summary result through a display (e.g., a display (210)).

[0110] FIG. 10 is a flowchart illustrating a method for providing an integrated summary document of an electronic device according to one embodiment. The embodiment of FIG. 10 is described with reference to FIG. 11a and FIG. 11b. FIG. 11a and FIG. 11b are illustrative diagrams illustrating a method for displaying an object associated with a source of an electronic device according to one embodiment.

[0111] In one embodiment, the operations illustrated in FIG. 10 may be performed in various orders, not limited to the order illustrated. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. According to one embodiment, more operations may be performed than those illustrated in FIG. 10, or at least one fewer operation may be performed.

[0112] Referring to FIG. 10, in operation 1001, in one embodiment, an electronic device (201) (e.g., electronic device (201)) may obtain user input for summarizing content. Operation 1001 may be at least partially identical or similar to operation 501, and descriptions that overlap with operation 501 may not be repeated herein.

[0113] In operation 1003, in one embodiment, the electronic device (201) can determine the range of content for summary based on user data and user input. Operation 1003 may be at least partially identical or similar to operation 503, and descriptions that overlap with operation 503 may not be repeated here.

[0114] In operation 1005, in one embodiment, the electronic device (201) can identify a plurality of target items that match the user's intention among a plurality of items within the range of content. Operation 1005 may be at least partially identical or similar to operation 507, and descriptions that overlap with operation 507 may not be repeated here.

[0115] In operation 1007, in one embodiment, the electronic device (201) may respectively acquire a plurality of summarized documents corresponding to a plurality of target items. The electronic device (201) may acquire summary results of a plurality of target items based on a detail level. The detail level may indicate the expertise of the content within the summary result. The electronic device (201) may determine the detail level of the content for summary based on user input and user data. The electronic device (201) may respectively acquire a plurality of summarized documents corresponding to a plurality of target items based on the determined detail level.

[0116] In operation 1009, in one embodiment, the electronic device (201) may obtain an integrated summary document based on integrating a plurality of summarized documents. The integrated summary document may include at least one object associated with the source of the summary result. In operation 1011, in one embodiment, the electronic device (201) may provide the integrated summary document as a summary result. The summary result may include, for example, an image, text with links, and text without links. Text with links may be distinguished from text without links by color. Highlighting effects may be applied to text with links. Text with links may be distinguished from text without links by an underline and / or bold. Based on user input associated with the selection of text with links (for example, the electronic device (201) may display a window associated with the source of the text (for example, referred to as a "source finder") by overlaying it on the summary result.

[0117] Referring to FIG. 11a, in one embodiment, an electronic device (201) may display a summary result (1110) including a source of the summary result and an object (1111). The electronic device (201) may display a window (1121) associated with the source of the summary result on the summary result (1120) based on confirming (1113) user input (e.g., touch input) for the source of the summary result and the object (1111). The electronic device (201) may provide a summary result (1130) including original content (1131) based on confirming (1123) user input for at least a portion (1122) of the window (1121) associated with the source of the summary result.

[0118] Referring to FIG. 11b, in one embodiment, an electronic device (201) may display a window (1117) associated with additional features based on user input (1116) (e.g., long press input) for at least a portion (1115) (e.g., text without links) of a summary result (1110). The electronic device (201) may provide a tracking function for the original content for the text without links based on the window (1117) associated with additional features. The window (1117) associated with additional features may include objects associated with search, image search, and / or source search. The electronic device (201) may display a window (1141) associated with the source of the summary result on the summary result (1140) based on confirming user input (1119) for the window (1117) associated with additional features. The window (1141) associated with the source of the summary result may include a description of the original content and / or the original content. The electronic device (201) can display a summary result (1150) containing original content (1151) based on checking (1143) user input (1142) for a window (1141) associated with the source of the summary result.

[0119] FIG. 12 is a flowchart illustrating a method for providing summary results of a wearable electronic device according to one embodiment. The embodiment of FIG. 12 will be described with reference to FIG. 13 and FIG. 14. FIG. 13 and FIG. 14 are illustrative diagrams illustrating a method for providing summary results of a wearable electronic device according to one embodiment.

[0120] In one embodiment, the operations illustrated in FIG. 12 may be performed in various orders, not limited to the order illustrated. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. According to one embodiment, more operations may be performed than those illustrated in FIG. 12, or at least one fewer operation may be performed.

[0121] Referring to FIG. 12, in operation 1201, in one embodiment, an electronic device (201) (e.g., electronic device (201)) may obtain user input for summarizing content. In one embodiment, the electronic device (201) may be a wearable electronic device (e.g., augmented reality (AR) glasses). Referring to FIG. 13, the wearable electronic device (1301) may be an electronic device worn on at least a part of the body of a user (1310). The wearable electronic device (1301) may establish a connection with an external electronic device (1320) based on a user account. The wearable electronic device (1301) may include a plurality of sensors (e.g., sensor module (176)). The wearable electronic device (1301) can display text corresponding to user input in real time through a display based on user input (e.g., voice command). Operation 1201 may be at least partially identical or similar to operation 501, and descriptions that overlap with operation 501 may not be repeated here.

[0122] In operation 1203, in one embodiment, the electronic device (201) may determine the range of content for summary based on user data and user input. Operation 1203 may be at least partially identical or similar to operation 503, and descriptions that overlap with operation 503 may not be repeated herein. User data may include, for example, visual data collected by a wearable electronic device.

[0123] In operation 1205, in one embodiment, the electronic device (201) can identify a plurality of target items that match the user's intention among a plurality of items within the range of content. Operation 1205 may be at least partially identical or similar to operation 507, and descriptions that overlap with operation 507 may not be repeated here.

[0124] In operation 1207, in one embodiment, the electronic device (201) can acquire sensing information. The electronic device (201) can acquire sensing information including a point of interest (POI) and a region of interest (ROI) by using a plurality of sensors included in the electronic device (201). Information associated with the POI may indicate a preferred location (or characteristics of the preferred location) of a user (e.g., user (1310) of the wearable electronic device (1301)). Information associated with the ROI may indicate a preferred object of the user. Referring to FIG. 14, an image (1410) acquired by the wearable electronic device (e.g., wearable electronic device (1301)) may include a plurality of objects (1411, 1412). The wearable electronic device may identify an ROI (1420) within the image (1410) based on tracking the user's gaze.

[0125] In operation 1209, in one embodiment, the electronic device (201) can determine the level of detail of the content for summary based on user data and sensing information. The electronic device (201) can determine the level of detail associated with the user's expertise based on information associated with user data, POI, and ROI. In operation 1211, in one embodiment, the electronic device (201) can obtain summary results of a plurality of target items based on the determined level of detail. In operation 1213, in one embodiment, the electronic device (201) can provide summary results. For example, the electronic device (201) can display the summary results through a display (e.g., display (210)).

[0126] FIG. 15 is a diagram illustrating a generative artificial intelligence model according to one embodiment.

[0127] Referring to FIG. 15, a user query / response interface (1510), an application / service component (1530), a knowledge repository (1520), an AI framework (1540), and a generative AI model (1560) (e.g., the generative AI model (340) of FIG. 3) may be stored in memory (220) (e.g., the memory (220) of FIG. 2) or on a separate server. At least some of the user query / response interface (1510), the application / service component (1530), the knowledge repository (1520), the AI ​​framework (1540), or the generative AI model (1560) may be implemented in software or in hardware.

[0128] According to one embodiment, a user query / response interface (1510) may receive user input. The user input may be in the form of natural language, images, and / or videos, but is not limited thereto. Additionally, context information may be transmitted along with the user input. The context information may include various additional information at the time of user input. For example, the additional information may include information about the application currently being used by the user or the user's location information. Furthermore, the user input may be in a mixed form of the aforementioned natural language, images, sounds, and context information. Additionally, the user input may be in a non-natural language form, such as selecting a menu. The user query / response interface (1510) may output results from a generative artificial intelligence system to the user. The output may be in the form of natural language or specific content, and may also be provided in the form of an action requested by the user. The AI ​​framework (1540) can receive input from the user and coordinate and control each component necessary to perform the user's intent based on the user's query.

[0129] User input received from the user query / response interface (1510) can be transmitted to a prompt design component (1541). The prompt design component (1541) can be used to generate prompts suitable for inputting user input into a large language model (LLM), a large vision model (LVM), or a large multimodal model (LMM). The prompt design component (1541) may be an AI component that uses machine learning algorithms or neural networks to develop better prompts over time. Based on user input, the prompt design component (1541) can generate prompts by accessing a knowledge component containing user preference data, a prompt library, and prompt examples, and can transmit the generated prompts to the large language model (LLM) or large multimodal model (LMM).

[0130] The API / Plug-in management component (1542) can perform the role of communicating with external information when there is a request for additional information when user input is passed as input to a generative model. The API / Plug-in management component (1542) establishes a channel to communicate with the outside of the AI ​​Interface via the API, and through the established channel, it can enable access to various data sources (e.g., knowledge repository (1520)). Additionally, if the API / Plug-in management component (1542) needs to perform an action that executes the user input as a final step rather than an intermediate result in an application or service, it can request that action from the application / service component (1530) via the API. The information obtained from the outside may be used to generate a prompt in the prompt design component (1541) along with the user input, or it may be passed as input to the generative model.

[0131] The output modification component (or refiner component) (1543) can fine-tune the output of the generative model. For example, the output modification component (1543) can verify whether the content generated through the LLM and / or LMM is irrelevant, contains biased content, or contains harmful content. Additionally, the output modification component (1543) can determine the extent to which the output matches the desired result and, if necessary, proceed with the additional process. Furthermore, the output modification component (1543) can configure and provide hints to the user to avoid unwanted output.

[0132] A generative AI model (1560) generally refers to an artificial intelligence neural network that generates new forms of data based on user input information. A generative AI model (1560) may include models that generate images and / or models that generate language. Models that generate images include, but are not limited to, GANs (generative adversarial networks) and VAEs (variational autoencoders), and examples include Diffusion-based generative models that use VAEs and Transformer structures. Models that generate language are models trained to output the most statistically appropriate output value based on input values, and examples include models such as CHAT-GPT 3 and CHAT-GPT 4. There are also LMMs (large multimodal models) that can recognize various forms of data input, such as text, images, and audio, and generate new data corresponding to them.

[0133] An electronic device according to one embodiment (e.g., electronic device (201)) may include a display (e.g., display (210)), at least one processor (e.g., processor (230)) including a processing circuit, and a memory (e.g., memory (220)) for storing instructions. The instructions may cause the electronic device (201) to obtain user input for summarizing content when executed individually or collectively by the at least one processor (230). The instructions may cause the electronic device (201) to determine the range of content for summarization based on user data and the user input when executed individually or collectively by the at least one processor (230). The above instructions may cause the electronic device (201), when executed individually or collectively by the at least one processor (230), to identify a plurality of target items that match the user intent among the plurality of candidate items, based on whether the plurality of candidate items within the range of the content match the user intent identified based on the user input. Each of the candidate items may include content. The above instructions may cause the electronic device (201), when executed individually or collectively by the at least one processor (230), to obtain a summary result of the plurality of target items. The above instructions may cause the electronic device (201), when executed individually or collectively by the at least one processor (230), to display the summary result through the display (210).

[0134] In one embodiment, the instructions may cause the electronic device (201) to check information related to the ratio of video content in the summary result when executed individually or collectively by the at least one processor (230). In one embodiment, the instructions may cause the electronic device (201) to set the format of the summary result to a text format based on confirming that the ratio of the video content is less than a threshold ratio when executed individually or collectively by the at least one processor (230). In one embodiment, the instructions may cause the electronic device (201) to set the format of the summary result to a video format based on confirming that the ratio of the video content exceeds a threshold ratio when executed individually or collectively by the at least one processor (230).

[0135] In one embodiment, the instructions may cause the electronic device (201) to check whether the user input contains information associated with the range of the content when executed individually or collectively by the at least one processor (230). The instructions may cause the electronic device (201) to check the range of the content among the user data based on the confirmation that the user input contains information associated with the range of the content when executed individually or collectively by the at least one processor (230).

[0136] In one embodiment, the instructions may cause the electronic device (201), when executed individually or collectively by the at least one processor (230), to determine the range of the content based on information associated with search history, input prompt history, preferred content, and stored content, based on confirming that the user input does not include information associated with the range of the content.

[0137] In one embodiment, information associated with the scope of the content may include information associated with the purpose of the summary, the type of user data, and the period of the user data. The type of user data may include text, still images, and video. The user data may include local data stored in the electronic device.

[0138] In one embodiment, the user data may include text data, still image data, and video data associated with the execution history of at least one application.

[0139] In one embodiment, the instructions may cause the electronic device (201), when executed individually or collectively by the at least one processor (230), to determine the level of detail of the content for summary based on the user input and the user data. The level of detail may indicate the expertise of the content within the summary result. The instructions may cause the electronic device (201), when executed individually or collectively by the at least one processor (230), to obtain the summary result of the plurality of target items based on the determined level of detail.

[0140] In one embodiment, the instructions may cause the electronic device (201) to acquire, respectively, a plurality of summarized documents corresponding to the plurality of target items when executed individually or collectively by the at least one processor (230). The instructions may cause the electronic device (201) to acquire an integrated summary document based on integrating the plurality of summarized documents when executed individually or collectively by the at least one processor (230). The instructions may cause the electronic device (201) to provide the integrated summary document as a summary result when executed individually or collectively by the at least one processor (230).

[0141] In one embodiment, the integrated summary document may include at least one object associated with the source of the summary result. The instructions may cause the electronic device (201) to display at least one object associated with the source of the summary result through the display (210) when executed individually or collectively by the at least one processor (230).

[0142] In one embodiment, the electronic device (201) may be a wearable electronic device. The instructions may cause the electronic device (201), when executed individually or collectively by the at least one processor (230), to acquire sensing information including a point of interest (POI) and a region of interest (ROI) using a plurality of sensors. The instructions may cause the electronic device (201), when executed individually or collectively by the at least one processor (230), to determine a detail level of content for summary based on the user data and the sensing information. The instructions may cause the electronic device (201), when executed individually or collectively by the at least one processor (230), to acquire a summary result of the plurality of target items based on the determined detail level.

[0143] A method according to one embodiment may include an operation of obtaining user input for summarizing content. The method may include an operation of determining the range of content for summarization based on user data and the user input. The method may include an operation of determining a plurality of target items that match the user intent among the plurality of candidate items based on determining whether a plurality of candidate items within the range of content match the user intent determined based on the user input. Each of the candidate items may include content. The method may include an operation of obtaining a summary result of the plurality of target items. The method may include an operation of displaying the summary result through a display (210) of an electronic device (201).

[0144] In one embodiment, the method may further include an operation of checking information related to the ratio of video content among the summary results. The method may further include an operation of setting the format of the summary result to a text format based on confirming that the ratio of the video content is less than a threshold ratio. The method may further include an operation of setting the format of the summary result to a video format based on confirming that the ratio of the video content exceeds a threshold ratio.

[0145] In one embodiment, the operation of determining the range of content for summarization based on the user data and the user input may include an operation of determining whether the user input contains information associated with the range of content. The operation of determining the range of content for summarization based on the user data and the user input may include an operation of determining the range of content among the user data based on determining that the user input contains information associated with the range of content.

[0146] In one embodiment, the operation of determining the range of content for summarization based on the user data and the user input may include determining the range of content based on information associated with search history, input prompt history, preferred content, and stored content, based on confirming that the user input does not include information associated with the range of content.

[0147] In one embodiment, information associated with the scope of the content may include information associated with the purpose of the summary, the type of user data, and the period of the user data. The type of user data may include text, still images, and video. The user data may include local data stored in the electronic device.

[0148] In one embodiment, the user data may include text data, still image data, and video data associated with the execution history of at least one application.

[0149] In one embodiment, the operation of obtaining the summary result of the plurality of target items may include the operation of checking the detail level of the content for the summary based on the user input and the user data. The detail level may indicate the expertise of the content within the summary result. The operation of obtaining the summary result of the plurality of target items may include the operation of obtaining the summary result of the plurality of target items based on the checked detail level.

[0150] In one embodiment, the method may further include the operation of acquiring a plurality of summarized documents corresponding to the plurality of target items respectively. The method may further include the operation of acquiring an integrated summary document based on integrating the plurality of summarized documents. The method may further include the operation of providing the integrated summary document as a summary result.

[0151] In one embodiment, the integrated summary document may include at least one object associated with the source of the summary result. The operation of displaying the summary result may include the operation of displaying at least one object associated with the source of the summary result through the display (210).

[0152] According to one embodiment, a non-transient storage medium recording computer-executable instructions may be provided. When executed by a processor (230) of an electronic device (201), the computer-executable instructions may cause the electronic device (201) to obtain user input for summarizing content. When executed by a processor (230) of an electronic device (201), the computer-executable instructions may cause the electronic device (201) to identify a range of content for summarization based on user data and the user input. When executed by a processor (230) of an electronic device (201), the computer-executable instructions may cause the electronic device (201) to identify a plurality of target items that match the user intent among the plurality of candidate items based on whether the plurality of candidate items within the range of content match the user intent identified based on the user input. Each of the candidate items may include content. When the above computer-executable instructions are executed by the processor (230) of the electronic device (201), the electronic device (201) may cause the electronic device (201) to obtain a summary result of the plurality of target items. When the above computer-executable instructions are executed by the processor (230) of the electronic device (201), the electronic device (201) may cause the electronic device (201) to display the summary result through the display (210) of the electronic device (201).

[0153] An electronic device according to one embodiment disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiment of this document is not limited to the aforementioned devices.

[0154] The embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0155] As used in one embodiment of this document, the term “module” may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0156] One embodiment of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, "non-transient" simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0157] In addition, the structure of the data used in the embodiments of the present document described above may be recorded on a computer-readable recording medium through various means. The computer-readable recording medium includes storage media such as magnetic storage media (e.g., ROM, floppy disk, hard disk) and optical reading media (e.g., CD-ROM, DVD).

[0158] According to one embodiment, the method according to one embodiment disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0159] According to one embodiment, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to one embodiment, one or more of the components or operations among the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to one embodiment, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (201), Display (210); At least one processor (230) including a processing circuit; and The electronic device (201) includes a memory (220) for storing instructions, and the instructions, when executed individually or collectively by the at least one processor (230): Obtain user input for summarizing the content, and Based on user data and the above user input, determine the scope of content for the summary, and Based on determining whether multiple candidate items within the scope of the above content match the user intent identified based on the user input, among the multiple candidate items, multiple target items that match the user intent are identified, - each of the above candidate items includes content -, Obtain a summary result of the above multiple target items, and An electronic device (201) that causes the summary result to be displayed through the display (210).

2. In Paragraph 1, The above instructions, when executed individually or collectively by the at least one processor (230), cause the electronic device (201): Check the information related to the ratio of video content among the summary results above, and Based on confirming that the ratio of the above video content is below a threshold ratio, the format of the above summary result is set to a text format, and An electronic device (201) that causes the format of the summary result to be set to a video format based on confirming that the ratio of the video content exceeds a threshold ratio.

3. In Paragraph 1 or 2, The above instructions, when executed individually or collectively by the at least one processor (230), cause the electronic device (201): Check whether the above user input includes information related to the scope of the above content, and An electronic device (201) that causes the range of the content to be identified among the user data based on confirming that the user input includes information related to the range of the content.

4. In any one of paragraphs 1 to 3, The above instructions, when executed individually or collectively by the at least one processor (230), cause the electronic device (201) to determine the range of the content based on information associated with search history, input prompt history, preferred content, and stored content, based on confirming that the user input does not include information associated with the range of the content.

5. In any one of paragraphs 1 to 4, Information related to the scope of the above content includes information related to the purpose of the summary, the type of user data, and the period of the user data, and The types of the above user data include text, still images, and videos, and The above user data includes local data stored in the electronic device, an electronic device (201).

6. In any one of paragraphs 1 to 5, The above user data includes text data, still image data, and video data associated with the execution history of at least one application, in an electronic device (201).

7. In any one of paragraphs 1 through 6, The above instructions, when executed individually or collectively by the at least one processor (230), cause the electronic device (201): Based on the above user input and the above user data, determine the detail level of the content for the summary—the above detail level indicates the expertise of the content within the summary result—, An electronic device (201) that causes to obtain a summary result of the plurality of target items based on the above-mentioned confirmed detail level.

8. In any one of paragraphs 1 through 7, The above instructions, when executed individually or collectively by the at least one processor (230), cause the electronic device (201): Respectively acquire multiple summarized documents corresponding to the above multiple target items, and Based on integrating the above multiple summarized documents, an integrated summary document is obtained, and An electronic device (201) that causes the above integrated summary document to be provided as a summary result.

9. In any one of paragraphs 1 through 8, The above integrated summary document includes at least one object associated with the source of the summary result, and The above instructions, when executed individually or collectively by the at least one processor (230), cause the electronic device (201) to display at least one object associated with the source of the summary result through the display (210), the electronic device (201).

10. In any one of paragraphs 1 through 9, The above electronic device (201) is a wearable electronic device, and The above instructions, when executed individually or collectively by the at least one processor (230), cause the electronic device (201), Using multiple sensors, sensing information including POI (point of interest) and ROI (region of interest) is acquired, and Based on the above user data and the above sensing information, determine the detail level of the content for summary, and An electronic device (201) that causes to obtain a summary result of the plurality of target items based on the above-mentioned confirmed detail level.

11. Regarding the method, An action of obtaining user input for summarizing content; An operation to determine the scope of content for summarization based on user data and the above user input; An operation of identifying multiple target items that match the user intent among the multiple candidate items, based on checking whether multiple candidate items within the scope of the above content match the user intent confirmed based on the user input—each of the above candidate items includes content—; The operation of obtaining a summary result of the plurality of target items above; and Operation of displaying the summary result through the display (210) of the electronic device (201). A method including 12. In Paragraph 11, An action of verifying information related to the ratio of video content among the above summary results; An operation to set the format of the summary result to a text format based on confirming that the ratio of the above video content is less than a threshold ratio; and A method further comprising the operation of setting the format of the summary result to a video format based on confirming that the ratio of the video content exceeds a threshold ratio.

13. In Paragraph 11 or 12, Based on the above user data and the above user input, the operation of determining the above range of content for summarization is, An operation to check whether the above user input includes information related to the scope of the above content; and A method comprising an operation to determine the range of the content among the user data based on confirming that the user input includes information related to the range of the content.

14. In any one of paragraphs 11 through 13, A method comprising, based on the above user data and the above user input, determining the range of content for summarization, determining the range of content based on information associated with search history, input prompt history, preferred content, and stored content, based on confirming that the above user input does not include information associated with the range of content.

15. In a non-transient storage medium storing computer-executable instructions, said computer-executable instructions, when executed by a processor (230) of an electronic device (201), cause said electronic device (201): Obtain user input for summarizing the content, and Based on user data and the above user input, determine the scope of content for the summary, and Based on determining whether multiple candidate items within the scope of the above content match the user intent identified based on the user input, among the multiple candidate items, multiple target items that match the user intent are identified—each of the candidate items includes content—, Obtain a summary result of the above multiple target items, and A storage medium that causes the summary result to be displayed through the display (210) of the electronic device (201).

Images