Electronic device using artificial intelligence model and operation method of electronic device

Abstract

The present disclosure may provide an electronic device using an AI model. The electronic device may: display a screen corresponding to an incoming call through a display, wherein the screen includes a first graphic object associated with a call connection of the incoming call; acquire a first user input for the first graphic object; modify the first graphic object on the basis of identifying that the first user input satisfies a first condition, and display the modified first graphic object through the display, wherein the modified first graphic object is associated with the call connection using an artificial intelligence (AI) model; generate, using the AI model, a first message including information related to a call intention of a caller on the basis of at least one application, wherein the at least one application includes data associated with the caller; display the first message through the display; while the first message is being displayed, acquire a second user input for the modified first graphic object; perform the call connection with the caller on the basis of the second user input; and on the basis of identifying that the first user input satisfies a second condition, perform the call connection with the caller without displaying the first message.

Description

Electronic devices and methods of operating electronic devices using artificial intelligence models

[0001] The present disclosure relates to an electronic device using an artificial intelligence model and a method of operating the electronic device.

[0002] Generative AI refers to an AI system that can generate new content based on input data. Generative AI can create content in a variety of formats, including text, images, audio, and video. For example, generative AI can learn from vast amounts of data to understand specific patterns and structures, and then create new content based on this knowledge. Generative AI can generate content in a variety of styles and themes tailored to user needs.

[0003] This type of generative AI could be utilized in a variety of applications. For example, generative AI could be used to assist users in using electronic devices.

[0004] The above information may be provided as background information to aid in understanding this document. None of the above is claimed to be prior art related to this document or can be used to determine prior art.

[0005] According to one embodiment, an electronic device may be provided. The electronic device may include a display; a communication circuit; at least one processor including a processing circuit; and a memory including at least one storage medium storing instructions, wherein the instructions, when individually and / or collectively executed by the at least one processor, cause the electronic device to perform at least one operation. The at least one operation may include displaying a screen corresponding to an incoming call through the display. The screen may include a first graphical object associated with a call connection of the incoming call. The at least one operation may include obtaining a first user input for the first graphical object. The at least one operation may include modifying the first graphical object based on identifying that the first user input satisfies a first condition, and displaying the modified first graphical object through the display. The modified first graphical object may be associated with a call connection using an artificial intelligence (AI) model. The at least one operation may include generating a first message including information related to the caller's call intent based on at least one application using the AI ​​model. The at least one application may include data associated with the caller. The at least one operation may include displaying the first message through the display. The at least one operation may include obtaining a second user input for the modified first graphic object while the first message is being displayed.The at least one action may include an action of establishing a call connection with the caller based on the second user input. The at least one action may include an action of establishing a call connection with the caller without displaying the first message based on identifying that the first user input satisfies a second condition.

[0006] According to one embodiment, a method of an electronic device may be provided. The method of the electronic device may perform at least one operation. The at least one operation may include displaying a screen corresponding to an incoming call through the display. The at least one operation may include obtaining a first user input for the first graphical object. The at least one operation may include modifying the first graphical object based on identifying that the first user input satisfies a first condition, and displaying the modified first graphical object through the display. The modified first graphical object may be associated with a call connection using an artificial intelligence (AI) model. The at least one operation may include generating a first message including information related to a caller's call intent based on at least one application using the AI ​​model. The at least one application may include data associated with the caller. The at least one operation may include displaying the first message through the display. The at least one operation may include obtaining a second user input for the modified first graphic object while the first message is displayed. The at least one operation may include establishing a call connection with the sender based on the second user input. The at least one operation may include establishing a call connection with the sender without displaying the first message based on identifying that the first user input satisfies a second condition.

[0007] According to one embodiment, a storage medium storing at least one computer-readable instruction may be provided. The at least one instruction, when executed by at least a portion of at least one processor of an electronic device, may cause the electronic device to perform at least one operation. The at least one operation may include displaying a screen corresponding to an incoming call through the display. The at least one operation may include obtaining a first user input for the first graphical object. The at least one operation may include modifying the first graphical object based on identifying that the first user input satisfies a first condition, and displaying the modified first graphical object through the display. The modified first graphical object may be associated with a call connection using an artificial intelligence (AI) model. The at least one operation may include generating a first message including information related to a caller's call intent based on at least one application using the AI ​​model. The at least one application may include data associated with the caller. The at least one operation may include displaying the first message through the display. The at least one operation may include obtaining a second user input for the modified first graphic object while the first message is being displayed. The at least one operation may include establishing a call connection with the caller based on the second user input. The at least one operation may include establishing a call connection with the caller without displaying the first message based on identifying that the first user input satisfies a second condition.

[0008] In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

[0009] FIG. 1A is a block diagram of an electronic device within a network environment according to various embodiments of the present disclosure.

[0010] FIG. 1b is a diagram illustrating a generative artificial intelligence system according to one embodiment of the present disclosure.

[0011] FIG. 2A illustrates an exemplary screen displayed on an electronic device depending on the status of a call connection, according to one embodiment of the present disclosure.

[0012] FIG. 2b illustrates an exemplary screen displayed on an electronic device depending on the status of a call connection, according to one embodiment of the present disclosure.

[0013] FIG. 3 is a flowchart illustrating a method for an electronic device to perform an operation according to a state of a call connection, according to one embodiment of the present disclosure.

[0014] FIG. 4 is a diagram illustrating a method for an electronic device to perform an operation according to a state of a call connection using an AI model, according to one embodiment of the present disclosure.

[0015] FIG. 5 is a flowchart illustrating an exemplary operation for an electronic device to select a call mode according to one embodiment of the present disclosure.

[0016] FIG. 6 illustrates an exemplary screen provided by an electronic device to perform a call connection according to a general call mode, according to one embodiment of the present disclosure.

[0017] FIGS. 7A and 7B illustrate exemplary screens provided by an electronic device to perform a call connection according to an AI assistant mode, according to one embodiment of the present disclosure.

[0018] FIG. 8 is a flowchart illustrating operations before a call connection of an electronic device according to one embodiment of the present disclosure.

[0019] FIG. 9A illustrates user registration information and applications used to select a representative application according to one embodiment of the present disclosure.

[0020] FIG. 9b illustrates correlation map information between user registration information and applications used to select a representative application according to one embodiment of the present disclosure.

[0021] FIGS. 10A and 10B illustrate representative applications according to one embodiment of the present disclosure.

[0022] FIGS. 11A and 11B are diagrams illustrating an operation of an electronic device providing a first message in a pre-call connection state based on a representative application according to one embodiment of the present disclosure.

[0023] FIG. 12 is a diagram illustrating an operation of an electronic device providing sender information based on a representative application according to one embodiment of the present disclosure.

[0024] FIG. 13 is a diagram for explaining an operation of an electronic device providing a first message in a pre-call connection state based on a representative application and a call reception time according to one embodiment of the present disclosure.

[0025] FIG. 14 is a diagram for explaining an operation of an electronic device providing a first message in various ways in a state before a call connection, according to one embodiment of the present disclosure.

[0026] FIG. 15 is a flowchart illustrating operations during a call connection of an electronic device according to one embodiment of the present disclosure.

[0027] FIG. 16 illustrates an exemplary screen including a second message in a call connection state according to one embodiment of the present disclosure.

[0028] FIG. 17 is a flowchart illustrating operations after a call connection is terminated in an electronic device according to one embodiment of the present disclosure.

[0029] FIG. 18 illustrates an exemplary screen including a third message in a call connection termination state according to one embodiment of the present disclosure.

[0030] FIG. 19 is a diagram for explaining an operation of updating class information of a sender according to one embodiment of the present disclosure.

[0031] FIG. 20 is a diagram illustrating an operation of providing a second message and additional information associated with the second message in a call connection state of an electronic device according to one embodiment of the present disclosure.

[0032] FIG. 21 is a diagram illustrating an operation of providing a second message and additional information associated with the second message in a call connection state of an electronic device according to one embodiment of the present disclosure.

[0033] FIG. 22 is a flowchart illustrating a method of operating an electronic device according to one embodiment of the present disclosure.

[0034] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0035] FIG. 1A is a block diagram of an electronic device within a network environment according to various embodiments of the present disclosure.

[0036] Referring to FIG. 1A, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0037] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0038] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, in the electronic device (101) itself where artificial intelligence is performed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0039] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0040] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0041] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0042] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0043] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0044] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0045] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0046] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0047] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0048] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0049] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0050] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0051] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0052] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a communication module (192) (e.g., a cellular communication module, a short-range communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0053] The communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The communication module (192) may support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The communication module (192) may support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the communication module (192) may support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0054] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0055] According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0056] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface).

[0057] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In one embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0058] FIG. 1b is a diagram illustrating a generative artificial intelligence system according to one embodiment of the present disclosure.

[0059] According to one embodiment, a user query / response interface (110b) may receive input (e.g., user input or data acquired or generated by an electronic device (e.g., electronic device (101) of FIG. 1A). The data acquired or generated by the electronic device may include, for example, image or video data generated using a processor (e.g., processor (120) of FIG. 1A), values ​​transmitted via a sensor (e.g., sensor module (176) of FIG. 1A) or sensor hub (e.g., external illuminance, angle of the electronic device, temperature of the display (e.g., display module (160) of FIG. 1A) or electronic device, display size or expansion / reduction information, captured images of an image sensor). The user input may be in the form of natural language, touch coordinates or stylus coordinates acquired through a touch panel or digitizer included in the display, images, and / or videos, but is not limited thereto. In addition, context information may also be transmitted when transmitting the user input. The context information may be user input. It can include various additional information at the time point. For example, the additional information can include information on the application currently being used by the user or the user's location information. In addition, the user input can also be in a mixed form of natural language, images, sounds, and context information as described above. In addition, the user input can also be in a non-natural language form, such as selecting a menu. The user query / response interface (110b) can output the results of the generative artificial intelligence system and / or the results of analyzing the input to the user. The output can be in the form of natural language or a specific content, and can also be provided in the form of an action requested by the user. The user query / response interface (110b) can output the results of the generative artificial intelligence system to the user. The output can be in the form of natural language or a specific content, and can also be provided in the form of an action requested by the user.

[0060] The AI ​​framework (140b) can receive user input and coordinate and control each component necessary to perform the user's intention based on the user's query.

[0061] User input received from the user query / response interface (110b) can be transmitted to a prompt design component (141b). The prompt design component (141b) can be used to generate prompts suitable for inputting user input into a large language model (LLM) or a large multimodal model (LMM). The prompt design component (141b) can be an AI component that uses a machine learning algorithm or a neural network to develop better prompts over time. The prompt design component (141b) can access a knowledge component including user preference data, a prompt library, and prompt examples based on the user input to generate prompts and transmit the generated prompts to the LLM or LMM.

[0062] The API / Plug-in management component (142b) can communicate with external information when there is a request for additional information when passing user input as input to the generative model. The API / Plug-in management component (142b) can establish a channel for communicating with the outside of the AI ​​Interface through the API, and can enable access to various data sources (e.g., knowledge storage (120b)) through the established channel. In addition, the API / Plug-in management component (142b) can request the application / service component (130b) through the API for an action that ultimately performs the user input, rather than an intermediate result, when the action needs to be performed in the application or service. Information obtained from the outside can be used to generate a prompt in the prompt design component (141b) together with the user input, or can be passed as input to the generative model.

[0063] The output modification component (also called a refiner component) (143b) can fine-tune the output from the generative model. For example, the output modification component (143b) can verify whether the content generated through the LLM and / or LMM is irrelevant, biased, or harmful. In addition, the output modification component (143b) can determine to what extent it matches the result desired by the user and, if necessary, perform additional processing. The output modification component (143b) can additionally configure and provide the user with hints to avoid unwanted output.

[0064] A generative AI model (160b) can generally refer to an artificial intelligence neural network that creates new types of data based on user input information. A generative AI model (160b) can include an image-generating model and / or a language-generating model. Representative models for generating images include a generative adversarial network (GAN) and a variational autoencoder (VAE), and examples include a VAE and a Diffusion-based generative model that uses a Transformer structure. A language-generating model is a model trained to statistically output the most appropriate output based on input values, and representative examples include models such as CHAT-GPT 3 and CHAT-GPT 4. In addition, there are also LMMs (large multimodal models) that can recognize various types of data input, such as text, images, and voice, and generate new data corresponding to them.

[0065] FIG. 2A illustrates an exemplary screen displayed on an electronic device depending on the status of a call connection, according to one embodiment of the present disclosure.

[0066] According to one embodiment, each screen of the embodiment of FIG. 2A may be a screen (or an interface corresponding to a call application) displayed when, for example, an electronic device (e.g., the electronic device (101) of FIG. 1A) operates in a normal call mode. According to one embodiment, the normal call mode may correspond to a mode that provides a call (or call connection) without using an AI model. In the present disclosure, the normal call mode may also be referred to as a normal mode, a first mode, or a first call mode.

[0067] According to one embodiment, part (a) of FIG. 2A illustrates an exemplary screen (210a) displayed on an electronic device before a call connection is performed. Referring to part (a) of FIG. 2A, the electronic device may display a screen (210a) including caller information (20a) about a caller of an incoming call in a state before a call connection is performed (pre-call state). If the caller is a caller registered with the electronic device (hereinafter referred to as a registered caller), the caller information (20a) may include name information indicating the name of the registered caller, phone number information indicating the phone number of the registered caller, and / or image information including an image (e.g., a photo) of the registered caller. If the caller is a caller not registered with the electronic device (hereinafter referred to as an unregistered caller), the caller information (20a) may include only phone number information indicating the phone number of the unregistered caller.

[0068] According to one embodiment, part (b) of FIG. 2A illustrates an exemplary screen (220a) displayed on an electronic device from after a call connection is established until the call connection is terminated. Referring to part (b) of FIG. 2A, the electronic device can display a screen (220a) including caller information (20a) about the caller even in a state (in-call state) after a call connection is established but before the call connection is terminated, similar to the state before the call connection is established.

[0069] According to one embodiment, part (c) of FIG. 2A illustrates an exemplary screen (230a) displayed on an electronic device after a call connection has been terminated. Referring to part (c) of FIG. 2A, the electronic device can display a screen (230a) including caller information (20a) about the caller, similar to previous states, even in a state where a call connection has been terminated (a call termination state).

[0070] According to the embodiment of FIG. 2a described above, the electronic device can provide the user with only the same caller information (20a) for a registered caller as information associated with the caller through the screens (210a, 220a, 230a), regardless of changes in the call connection status. Through these screens, the user can check basic information (e.g., name information, phone number information, image information) about the registered caller, but cannot check other information associated with the caller (e.g., previous call history with the caller). Furthermore, the user cannot check any information other than the phone number information for unregistered callers.

[0071] FIG. 2b illustrates an exemplary screen displayed on an electronic device depending on the status of a call connection, according to one embodiment of the present disclosure.

[0072] According to one embodiment, each screen of the embodiment of FIG. 2B may be a screen (or an interface corresponding to a call application) displayed when, for example, an electronic device (e.g., the electronic device (101) of FIG. 1A) operates in an AI assistant call mode. According to one embodiment, the AI ​​assistant call mode may correspond to a mode that provides a call (or a call connection) using an AI model. In the AI ​​assistant call mode, the electronic device may provide information for assisting a call (or a call connection) generated using the AI ​​model (e.g., the first message (31b), the second message (32b), the third message (33b) of FIG. 2B) and / or information about the caller (e.g., the caller information (20b) of FIG. 2B). In the present disclosure, the AI ​​assistant call mode may also be referred to as an AI assistant mode, an AI call mode, an AI mode, a second mode, or a second call mode. In the present disclosure, the first message (31b) may be referred to as a first auxiliary message, a first call auxiliary message, or a first brief message, the second message (32b) may be referred to as a second auxiliary message, a second call auxiliary message, or a second brief message, and the third message (33b) may be referred to as a third auxiliary message, a third call auxiliary message, or a third brief message.

[0073] According to one embodiment, part (a) of FIG. 2b illustrates an exemplary screen (210b) displayed on an electronic device before a call connection is made.

[0074] According to one embodiment, referring to part (a) of FIG. 2b, the electronic device may display a screen (210b) including caller information (20b) about the caller of the incoming call and / or a first message (31b) including information related to the incoming call, in a state before a call connection is made (pre-call state). For example, the electronic device may display a screen (210b) including caller information (20b) and / or a first message (31b) from after the incoming call is detected (or identified) until before the call connection is made.

[0075] According to one embodiment, the sender information (20b) may include name information indicating the name of the sender, phone number information indicating the phone number of the sender, and / or image information including an image (e.g., a photo) of the sender. If the sender is a registered sender, the electronic device may obtain the sender information (20b) using the registration information (e.g., name, phone number, image, address) of the registered sender registered in the electronic device. If the sender is an unregistered sender, the electronic device may obtain the information (e.g., name, image) of the unregistered sender through an external electronic device (e.g., a server) and obtain the sender information (20b) using the obtained information.

[0076] According to one embodiment, the first message (31b) may include information associated with the incoming call (e.g., information associated with the caller's intent to make the call). For example, the first message (31b) may include, but is not limited to, information indicating the caller's intent to make the call (e.g., predicted intent or predicted purpose of the call) and / or additional information associated with the call intent. The first message (31b) may be used to assist a user (e.g., the called party) in deciding whether to connect the call to the incoming call. Through the first message (31b), the user can check in advance, prior to connecting the call, the caller's intent to make the call and / or additional information associated with the intent to make the call, and based on the checked information, determine whether to connect the call.

[0077] According to one embodiment, the electronic device may generate (or obtain) the first message (31b) using an AI model (e.g., the generative AI model (160b) of FIG. 1b).

[0078] According to one embodiment, part (b) of FIG. 2B illustrates an exemplary screen (220b) displayed on an electronic device from after a call connection is made until the call connection is terminated.

[0079] According to one embodiment, referring to part (b) of FIG. 2B, the electronic device may display a screen (220b) including caller information (20a) and / or a second message (32b) including information related to the content of a conversation during a call, in a state (during a call) before the call connection is terminated after the call connection is established. The caller information (20b) displayed in the during a call state may be identical to the caller information (20b) displayed in the pre-call state.

[0080] In one embodiment, the second message (32b) may include information related to the content of the conversation during the call. For example, the second message (32b) may include, but is not limited to, information regarding the topic of the conversation (e.g., the current conversation) between the recipient and the caller during the call, and / or additional information related to the topic of the conversation. The second message (32b) may be used to help a user (e.g., the recipient) conduct a smooth conversation during the call. Through the second message (32b), the user can confirm the topic of the conversation and / or related information with the caller during the call, thereby conducting a smooth conversation with the caller.

[0081] In one embodiment, the electronic device may generate (or obtain) a second message (32b) using the AI ​​model.

[0082] According to one embodiment, part (c) of FIG. 2B illustrates an exemplary screen (230b) displayed on an electronic device after a call connection is terminated. Referring to part (c) of FIG. 2B, the electronic device may display a screen (230b) including caller information (20b) and / or a third message (33b) including information related to the entire contents of a call when the call connection is terminated (call terminated state). The caller information (20b) displayed in the call terminated state may be identical to the caller information (20b) displayed before and / or during a call.

[0083] In one embodiment, the third message (33b) may include information related to the entire content of the call. For example, the third message (33b) may include summary information that summarizes the entire content of the call by topic of conversation and / or selectable items (or selection options) related to each topic of conversation included in the summary information, but is not limited thereto. The third message (33b) may be used to help a user (e.g., the recipient) reconstruct and save only the necessary portions of the call content. Through the third message (33b), the user can check the topics and related information of conversations with the caller during the call and select only the topics of the conversations that need to be saved. In this case, the electronic device can reconstruct the content of the call using only the topics of the conversation selected by the user and store it on the electronic device.

[0084] In one embodiment, the electronic device may generate (or obtain) a third message (33b) using an AI model.

[0085] According to the embodiment of FIG. 2b described above, the electronic device can provide the user with not only caller information (20b) regarding the caller registered in the electronic device through the screen (210b, 220b, 230b), but also additional information (or, assistant information for reporting the call) (e.g., first message (31b), second message (32b), third message (33b)) according to each status of the call connection. Through this screen, the user can confirm the basic information of the caller and additionally confirm information through each message, thereby making the call smooth and efficient.

[0086] FIG. 3 is a flowchart illustrating a method for an electronic device to perform an operation according to a state of a call connection, according to one embodiment of the present disclosure.

[0087] Referring to FIG. 3, according to one embodiment, in operation 310, an electronic device (e.g., the electronic device (101) of FIG. 1A) may receive a call. For example, the electronic device may detect (or identify) the incoming call. In response to detecting the incoming call, the electronic device may display a screen (or interface) corresponding to the incoming call (e.g., the first screen (210b) of FIG. 2B).

[0088] According to one embodiment, at operation 320, the electronic device may identify (or select) an AI assistant call mode or a regular call mode. If the AI ​​assistant call mode is identified, operation 330 may be performed. If the regular call mode is identified, operation 340 may be performed.

[0089] According to one embodiment, the electronic device can select either an AI assistant call mode or a regular call mode based on user input. Selection of a call mode based on user input is described below with reference to FIGS. 5, 6, 7A, and 7B.

[0090] According to one embodiment, the electronic device may select either the AI ​​assistant call mode or the normal call mode based on default call mode setting information or the caller's class information (e.g., the first class information (1910) or the second class information (1920) of FIG. 19).

[0091] According to one embodiment, the default call mode setting information may include information indicating the default call mode. For example, if the default call mode setting information indicates that the AI ​​assistant call mode is the default call mode, in response to identifying an incoming call, the electronic device may set the call mode for the incoming call to the AI ​​assistant call mode without separate user input.

[0092] In one embodiment, default call mode setting information may be set for each caller. For example, if the default call mode setting information indicates that the AI ​​assistant call mode for a specific caller is the default call mode, then in response to identifying an incoming call from that caller, the electronic device may set the call mode for that incoming call to the AI ​​assistant call mode without separate user input.

[0093] In one embodiment, the caller's class information may include default call mode setting information for the caller. For example, if the caller's class information includes default call mode setting information indicating that the default call mode for the caller is AI assistant call mode, the electronic device may set the call mode for the incoming call to AI assistant call mode without separate user input.

[0094] According to one embodiment, in operation 330, in response to identifying that an AI assistant call mode is selected, the electronic device may perform at least one operation for pre-call connection processing. In the present disclosure, the operation for pre-call connection processing may also be referred to as a pre-call processing operation.

[0095] According to one embodiment, at least one pre-call processing action may include a first action (e.g., first action (411) of pre-call state (410) of FIG. 4) that identifies (or selects) at least one application (e.g., a representative application) associated with the caller of the incoming call. For example, the electronic device may select at least one representative application associated with the sender from a plurality of applications installed in the electronic device based on the sender's registration information (or user registration information (e.g., user registration information (910) of FIG. 9A). For example, the electronic device may select at least one representative application associated with the sender from a plurality of applications installed in the electronic device based on the sender's class information. For example, the electronic device may select at least one representative application associated with the sender from a plurality of applications installed in the electronic device using an AI model based on the sender's class information. The representative application may be, for example, an application used by the electronic device to assist a call (or call connection) using the AI ​​model. The sender's class information may include, for example, at least one of information about the relationship with the sender, information about the intimacy with the sender, information about the sender's main interests, information about the most relevant applications, information about the conversation responsiveness, information about the frequency of contact, information about the sender's tendencies, or information about the adoption rate of AI assistant conversations. The sender's class information may include information about the sender's It may include additional information about the default call mode settings.

[0096] According to one embodiment, the at least one pre-call processing operation may include a second operation (e.g., the second operation (412) of the pre-call state (410) of FIG. 4) for obtaining (or generating) information about the caller's intent to make the incoming call. For example, the electronic device may generate information about the caller's intent to make the call using an AI model based on the class information of the caller and / or at least one application (or data associated with the at least one application) identified through the first operation. For example, the electronic device may analyze the caller's intent to make the call using an AI model based on the class information of the caller and data associated with the at least one application identified through the first operation, thereby generating information about the caller's intent. The at least one application (or data associated with the at least one application) may include, but is not limited to, data associated with the caller, such as data received from the caller of the incoming call via the at least one application.

[0097] According to one embodiment, the at least one pre-call processing operation may include a third operation (e.g., the third operation (413) of the pre-call state (410) of FIG. 4) that obtains (or generates) a first message (e.g., the first message (31b) of FIG. 2b) that includes information associated with the incoming call in the pre-call state (e.g., information associated with the call intent of the caller of the incoming call). For example, the electronic device may generate the first message using an AI model based on the information about the call intent of the caller obtained through the second operation. For example, the electronic device may generate the first message using an AI model based on at least one application (or data associated with at least one application) identified through the first operation.

[0098] According to one embodiment, at least one pre-call processing operation may include a fourth operation that provides the generated first message. For example, as illustrated in part (a) of FIG. 2B , the electronic device may display a screen including the first message (e.g., the first message (31b) of FIG. 2B ) via a display.

[0099] In one embodiment, in operation 331, the electronic device may perform a call connection. In one embodiment, the electronic device may perform a call connection in an AI assistant call mode based on user input. After the call connection is performed, the recipient and the caller may proceed with a call (or conversation) via the electronic device.

[0100] According to one embodiment, in operation 332, the electronic device may perform at least one operation for processing during a call connection after a call connection is established. In the present disclosure, the operation for processing during a call connection may also be referred to as an operation for processing during a call.

[0101] According to one embodiment, at least one of the in-call processing operations may include a first operation analyzing the content of a conversation during the call (e.g., the first operation (421) of the in-call state (420) of FIG. 4 ). For example, the electronic device may analyze the content of a conversation during the call in real time using an AI model, thereby generating (or obtaining) information about the analyzed conversation content.

[0102] According to one embodiment, at least one of the in-call processing operations may include a second operation (e.g., the second operation (422) of the in-call state (420) of FIG. 4 ) that generates a second message (e.g., the second message (32b) of FIG. 2B ) that includes information related to the content of the in-call conversation. For example, the electronic device may analyze the content of the in-call conversation in real time using an AI model to generate the second message.

[0103] In one embodiment, at least one of the call processing operations may include a third operation that provides the generated second message. For example, as illustrated in part (b) of FIG. 2B , the electronic device may display a screen including the second message (e.g., the second message (32b) of FIG. 2B ) via a display.

[0104] In one embodiment, in operation 333, the electronic device may terminate the call connection. In one embodiment, the electronic device may terminate the call connection in an AI assistant call mode based on user input. After the call connection is terminated, the recipient and the caller can no longer continue the call (or conversation) via the electronic device.

[0105] According to one embodiment, in operation 334, the electronic device may perform at least one operation for post-call processing after the call connection is terminated. In the present disclosure, the operation for post-call processing may also be referred to as a post-call processing operation.

[0106] According to one embodiment, at least one post-call processing operation may include a first operation (e.g., a first operation (431) of a call end state (430) of FIG. 4) that analyzes at least a portion of the content (e.g., the entire content or a portion of the content) of the call and generates a third message (e.g., a third message (33b) of FIG. 2B) that includes information related to the content of the call. For example, the electronic device may analyze at least a portion of the content (e.g., the entire content) of the call using an AI model and generate the third message. The electronic device may provide a screen including the generated third message. For example, as illustrated in part (c) of FIG. 2B, the electronic device may display a screen including the third message (e.g., the third message (33b) of FIG. 2B) through a display.

[0107] According to one embodiment, the at least one post-call processing operation may include a second operation (e.g., the second operation (432) of the call end state (430) of FIG. 4 ) of reconstructing and storing the content of the call based on the topic of at least one conversation selected via the third message. For example, the electronic device may perform an operation of reconstructing and storing the content of the call based on the topic of at least one conversation selected via the third message using an AI model.

[0108] According to one embodiment, at least one post-call processing operation may include a third operation (e.g., the third operation (433) of the call end state (430) of FIG. 4 ) that generates or updates class information about the caller. For example, the electronic device may use the AI ​​model to generate or update class information about the caller based on the entire content of the call or the reconstructed content of the call. For example, if there is no class information about the caller already generated, the electronic device may use the AI ​​model to generate class information about the caller based on at least a portion of the content (e.g., the entire content) or the reconstructed content of the call. For example, if there is class information about the caller already generated, the electronic device may use the AI ​​model to update class information about the caller based on at least a portion of the content (e.g., the entire content) or the reconstructed content of the call.

[0109] According to one embodiment, in operation 335, the electronic device may terminate the AI ​​assistant call mode.

[0110] According to one embodiment, in operation 340, the electronic device may perform a call connection in a normal call mode. According to one embodiment, while the call connection is performed in the normal call mode, the electronic device may switch the call mode to an AI assistant call mode based on a user input (or selection). In this case, the electronic device may perform all or part of the operations corresponding to operation 322 described above.

[0111] According to one embodiment, in operation 341, the electronic device may terminate a call connection in a normal call mode. As illustrated in FIG. 2A, when the normal call mode is applied, the electronic device may display a screen (e.g., screens 210a, 220a, 230a of FIG. 2A) containing the same caller information (e.g., caller information (20a) of FIG. 2A) regardless of the state of the call connection.

[0112] FIG. 4 is a diagram illustrating a method for an electronic device to perform an operation according to a state of a call connection using an AI model, according to one embodiment of the present disclosure.

[0113] Referring to FIG. 4, according to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1A) may utilize an AI model (400) (e.g., the generative AI model (160b) of FIG. 1B). The AI ​​model (400) may be stored in the electronic device, or may be an AI model stored in an external electronic device (e.g., the server (108) of FIG. 1A). The AI ​​model (400) may be an AI model that has been trained in advance using training data. In the present disclosure, the AI ​​model (400) may also be referred to as an analysis model, an AI analysis model, or an LLM analysis model.

[0114] According to one embodiment, the operation of the electronic device inputting input data into the AI ​​model (400) may include, for example, an operation of processing the input data using a user query / response interface (e.g., the user query / response interface (110b) of FIG. 1B) and / or an AI framework (e.g., the AI ​​framework (140b) of FIG. 1B) and an operation of inputting the processed input data (e.g., prompt data) into the AI ​​model (e.g., the generative AI model (160b) of FIG. 1B). According to one embodiment, the operation of the electronic device obtaining output data from the AI ​​model (400) may include, for example, an operation of processing output data output from the AI ​​model (e.g., the generative AI model (160b) of FIG. 1B) using an AI framework (e.g., the AI ​​framework (140b) of FIG. 1B) and / or a user query / response interface (e.g., the user query / response interface (110b) of FIG. 1B) and an operation of obtaining the processed output data. Processing in the user query / response interface, AI framework, and AI model can refer to the description of Fig. 1b described above.

[0115] According to one embodiment, in response to a call event (401) (e.g., an incoming call) being identified (or detected), the electronic device may enter a pre-call state (410).

[0116] According to one embodiment, in a pre-call state (410), the electronic device may perform at least one pre-call processing operation (411, 412, 413) using the AI ​​model (400).

[0117] According to one embodiment, at least one pre-call processing operation may include a first operation (411) for selecting (or identifying) at least one representative application, a second operation (412) for analyzing a caller's call intent to obtain call intent information, and / or a third operation (413) for generating a first message (e.g., the first message (31b) of FIG. 2b). The first operation (411) may include, for example, an operation for selecting (or identifying) at least one application (e.g., a representative application) associated with the caller of the incoming call. The second operation (412) may include, for example, an operation for obtaining information about the call intent of the caller of the incoming call. The third operation (412) may include, for example, an operation for obtaining (or generating) a first message including information associated with the incoming call in a pre-call state.

[0118] According to one embodiment, the electronic device may input input data (e.g., prompt data) into an AI model (400) and obtain output data associated with a specific pre-call processing operation from the AI ​​model (400). The input data may include, for example, first input data (402) and / or second input data (e.g., a command or request) associated with the pre-call processing operation. The first input data (402) may include on-device data and / or network data.

[0119] For example, the electronic device may input input data including information about the sender (e.g., class information about the sender) into the first AI model (400), and obtain information about at least one representative application from the AI ​​model (400) as output data associated with the first operation (411). According to one embodiment, the representative application may be, for example, an application that is mainly used with the sender, but is not limited thereto. According to one embodiment, the electronic device may select an application having a higher relevance to the sender than a specified standard (e.g., the application having the highest relevance to the sender) as the representative application based on information stored during user registration and / or information acquired during application use. The information stored during user registration may include, for example, the user's name, nickname, phone number, email address, and / or work address, but is not limited thereto. Details regarding the selection of the representative application will be exemplarily described below with reference to FIG. 9B.

[0120] For example, the electronic device may input input data (e.g., prompt data) including information about the caller (e.g., class information about the caller) and / or data associated with at least one representative application obtained through the first operation (411) into the AI ​​model (400), and obtain information about the caller's call intent from the AI ​​model (400) as output data associated with the second operation (412).

[0121] For example, the electronic device may input input data (e.g., prompt data) including information about the caller (e.g., class information of the caller) and / or information about the caller's intent obtained through the second operation (412) into the AI ​​model (400), and obtain the first message from the AI ​​model (400) as output data associated with the third operation (413).

[0122] According to one embodiment, in a call state (420), the electronic device may perform at least one call processing operation (421, 422) using the AI ​​model (400).

[0123] According to one embodiment, at least one of the in-call processing operations may include a first operation (421) for analyzing the content of a conversation during the call and / or a second operation (422) for generating a second message (e.g., the second message (32b) of FIG. 2b). The first operation (421) may, for example, include analyzing the content of a conversation during the call (e.g., analyzing in real time). The second operation (421) may, for example, include generating a second message including information related to the content of the conversation during the call based on information about the analyzed conversation content.

[0124] According to one embodiment, the electronic device may input input data (e.g., prompt data) into an AI model (400) and obtain output data associated with a specific call processing operation from the AI ​​model (400). The input data may include, for example, first input data (403) and / or second input data (e.g., a command or request) associated with the call processing operation. The first input data (403) may include speech data of a conversation during a call.

[0125] For example, the electronic device may input input data (e.g., prompt data) including speech data into the first AI model (400), and obtain information on the analyzed conversation content from the AI ​​model (400) as output data associated with the first action (421).

[0126] For example, the electronic device may input input data (e.g., prompt data) containing information about the analyzed conversation content obtained through the first operation (421) into the AI ​​model (400), and obtain a second message from the AI ​​model (400) as output data associated with the second operation (422).

[0127] According to one embodiment, in a call end state (430), the electronic device may perform at least one post-call processing operation (431, 432, 433) using the AI ​​model (400).

[0128] According to one embodiment, at least one post-call processing operation may include a first operation (431) for analyzing at least a portion of the content of the call (e.g., the entire content or a portion of the content) to generate a third message (e.g., the third message (33b) of FIG. 3b), a second operation (432) for reconstructing and storing the content of the call, and / or a third operation (433) for generating or updating class information about the caller. The first operation (421) may include, for example, analyzing at least a portion of the content of the call (e.g., the entire content) to generate a third message that includes information associated with at least a portion of the content of the call (e.g., the entire content). The second operation (432) may include, for example, reconstructing and storing the content of the call based on at least one conversation topic selected via the third message. The third operation (433) may include, for example, generating or updating class information about the caller.

[0129] In one embodiment, the electronic device may input input data (e.g., prompt data) into an AI model (400) and obtain output data associated with a specific call processing action from the AI ​​model (400). The input data may include input data (e.g., a command or request) associated with the post-call processing action.

[0130] For example, the electronic device may input input data (e.g., prompt data) containing information about at least a portion of the content (e.g., the entire content) of a call into the AI ​​model (400), and obtain a third message from the AI ​​model (400) as output data associated with the first action (431).

[0131] For example, the electronic device may input input data (e.g., prompt data) including information about the topic of at least one conversation selected through a third message and information about the entire content of the call into the AI ​​model (400), and obtain information about the reconstructed content of the call from the AI ​​model (400) as output data associated with the second operation (432).

[0132] For example, the electronic device may input input data (e.g., prompt data) containing at least a portion of the content of the call (e.g., the entire content) or information about reconstructed call content into the AI ​​model (400), and obtain the class information of the caller or the updated class information of the caller from the AI ​​model (400) as output data associated with the third operation (433).

[0133] FIG. 5 is a flowchart illustrating an exemplary operation for an electronic device to select a call mode according to one embodiment of the present disclosure.

[0134] Referring to FIG. 5, according to one embodiment, in operation 510, an electronic device (e.g., the electronic device (101) of FIG. 1A) may display a first screen corresponding to an incoming call. For example, the electronic device may display a screen corresponding to an incoming call in response to identifying (or detecting) an incoming call. The screen corresponding to an incoming call may include, for example, the first screen (601) of FIG. 6 and the first screen (701) of FIG. 7A, which will be described below.

[0135] According to one embodiment, in operation 520, the electronic device may obtain a first user input for a first graphical object associated with a call connection of an incoming call included in the first screen. The first graphical object may include, for example, the first graphical object (610) of FIG. 6 or the first graphical object (710) of FIG. 7A, which will be described below.

[0136] According to one embodiment, in operation 530, the electronic device may select a call mode as either a regular call mode or an AI assistant call mode based on a first user input. For example, the electronic device may select the AI ​​assistant call mode as the call mode based on identifying that the first user input satisfies a first condition. For example, the electronic device may select the regular call mode as the call mode based on identifying that the first user input satisfies a second condition.

[0137] Hereinafter, with reference to FIGS. 6, 7a and 7b, a screen (or user interface) provided for selecting a call mode and a screen (or user interface) provided differently depending on the selection of each call mode are exemplarily described.

[0138] FIG. 6 illustrates an exemplary screen provided by an electronic device to perform a call connection according to a general call mode, according to one embodiment of the present disclosure.

[0139] FIGS. 7A and 7B illustrate exemplary screens provided by an electronic device to perform a call connection according to an AI assistant mode, according to one embodiment of the present disclosure.

[0140] According to one embodiment, each of the screens displayed in FIGS. 6, 7a, and 7b corresponds to screens displayed before a call connection is performed (e.g., screens displayed in a state before a call connection).

[0141] Referring to part (a) of FIG. 6 and part (a) of FIG. 7A, according to one embodiment, an electronic device (e.g., electronic device (101) of FIG. 1A) may display a first screen (601, 701) corresponding to an incoming call. According to one embodiment, the electronic device may display the first screen (601, 701) in response to an incoming call being identified (or detected).

[0142] According to one embodiment, the first screen (601, 701) may include caller information (640, 740) (e.g., caller information (20b) of FIG. 2b), a first graphical object (610, 710) associated with a call connection, and / or a second graphical object (620, 720) associated with a call termination. The second graphical object (620, 720) may be displayed in the same manner as that displayed on the first screen (601, 701) without change in each screen of FIGS. 6, 7a, and 7b.

[0143] According to one embodiment, the first graphical object (610, 710) may include a first icon (61, 71) indicating a call connection and / or a second icon (62, 72) indicating an AI assistant call mode. The first icon (61, 71) may be displayed within a first area (611, 711) of the first graphical object (610, 710), and the second icon (62, 72) may be displayed within a second area (612, 712) of the first graphical object (610, 710) surrounding (or including) the first area (611, 711).

[0144] According to one embodiment, the electronic device may select a call mode as either a normal call mode or an AI assistant call mode based on a first user input (U1) to a first graphic object (610, 710).

[0145] Referring to part (b) of FIG. 7A, according to one embodiment, based on identifying that a first user input (U1) satisfies a first condition, the electronic device may select a call mode as an AI assistant call mode. The first condition may be, for example, a condition that must be satisfied in order for the call mode to be selected as the AI ​​assistant call mode, and various types of conditions may be set as the first condition depending on the setting. According to one embodiment, the electronic device may identify that the first user input (U1) satisfies the first condition when it is identified that the first user input (U1) corresponds to a swipe input in a first direction (D1). In the present disclosure, a swipe input in the first direction (D1) may be referred to as a first swipe input. The first swipe input may include a swipe input in a direction (D1) from outside the first area (711) of the first graphical object (710) to inside the first area (711), as illustrated in the second screen (702) of FIG. 7A. For example, the first swipe input may include a swipe input from, for example, a point outside the first area (711) of the first graphical object (710) and inside the second area (712) to a point within the first area (711). According to one embodiment, the electronic device may provide an animation effect that gradually reduces the size of the second area (712) within the second screen (702) while a swipe input in the first direction (D1) (e.g., in the direction of the first area (711)) is received from the second area (712).

[0146] Referring to part (b) of FIG. 6, according to one embodiment, based on identifying that the first user input (U1) satisfies the second condition, the electronic device may select the call mode as the normal call mode. The second condition is, for example, a condition that must be satisfied in order for the call mode to be selected as the normal call mode, and various types of conditions may be set as the second condition depending on the setting. According to one embodiment, the electronic device may identify that the first user input (U1) satisfies the second condition when it is identified that the first user input (U1) corresponds to a swipe input in the second direction (D2). In the present disclosure, a swipe input in the second direction (D2) may be referred to as a second swipe input. The second swipe input may include a swipe input in a direction (D2) from the inside of the second area (612) of the first graphical object (610) or the first area (611) to the outside of the second area (612), as illustrated in the second screen (602) of FIG. 6. For example, the second swipe input may include a swipe input from, for example, a point within the first area (611) or the second area (612) of the first graphical object (610) to a point outside the second area (612) or to a point within the area of ​​the second graphical object (620). Based on identifying that the general call mode is selected, the electronic device may perform a call connection (general call connection) according to the general call mode. While the general call connection is performed, the electronic device may display, for example, a third screen (603) as illustrated in part (c) of FIG. 6. The third screen (603) may include a modified first graphic object (610a) that indicates a call connection according to the general call mode.

[0147] Referring to part (c) of FIG. 7a, the electronic device may display a third screen (703) based on identifying that the AI ​​assistant call mode is selected. The third screen (703) may include caller information (740), a modified first graphic object (710a), a second graphic object (720), a third graphic object (730) associated with a timer, and / or a first message (740a, 740b) including information associated with an incoming call (e.g., the first message (31b) of FIG. 2b).

[0148] According to one embodiment, the electronic device may modify the first graphical object (710) and display the modified first graphical object (710a) in place of the first graphical object (710) within the third screen (703). As illustrated in part (c) of FIG. 7A, the modified first graphical object (710a) may include a first icon (71) and a second icon (72) displayed within a first area (711). Unlike the first graphical object (710) in which the second icon (72) is displayed outside the first area (711) or within the second area (712), the modified first graphical object (710a) may display the second icon (72) together with the first icon (71) within the first area (711). The user may confirm that the AI ​​assistant call mode is selected as the call mode through the modified first graphical object (710a).

[0149] According to one embodiment, the third graphic object (730) may include a timer value. The timer value may indicate the remaining time until a call connection according to the AI ​​assistant call mode is automatically (or forcibly) performed without a user input (e.g., a second user input (U2)) (e.g., 10 seconds remaining on the third screen (703) and 9 seconds remaining on the fourth screen (704). When the timer value expires, the electronic device may automatically perform a call connection according to the AI ​​assistant call mode even without a separate user input.

[0150] In one embodiment, the electronic device may set a timer value based on the urgency of the call connection associated with the caller's intent to call. For example, the electronic device may set a lower timer value as the urgency of the call connection increases.

[0151] According to one embodiment, the first message (740a, 740b) may include, but is not limited to, information indicating the caller's call intent (e.g., predicted call intent) of the incoming call, and / or additional information associated with the call intent. For example, as illustrated in the third screen (703) and the fourth screen (704), the first message (740a, 740b) may include information about the caller's predicted call intent (e.g., to check the progress of Project A) and a probability value (e.g., 82%) for the predicted call intent, such as "There is an 82% chance that you are calling to check the progress of Project A.", and additional information associated with the predicted call intent, such as "Regarding Project A, schedule for meeting with the vendor: ... Sample delivery date for demo production: ... Key issues with the sample production vendor: ..." For example, as illustrated in the third screen (703) and the fourth screen (704), if the amount of information included in the first message (740a, 740b) is too much to be displayed at once within the message display area, the user may use a scroll to completely check the information included in the first message (740a, 740b).

[0152] Meanwhile, unlike the embodiment of FIG. 7A described above, in the embodiment of FIG. 7B, in response to an incoming call being identified (or detected), a screen corresponding to the third screen (703) of FIG. 7A can be displayed immediately, as illustrated in part (a) of FIG. 7B, regardless of the first user input (U1). For example, as illustrated in part (a) of FIG. 7B, in the embodiment of FIG. 7B, in response to an incoming call being detected, a screen corresponding to the third screen (703) of FIG. 7A can be displayed immediately, without displaying screens such as the first screen (701) and the second screen (702) of FIG. 7A. The operation of the embodiment of FIG. 7B can be performed, for example, when the default call mode is set to the AI ​​assistant call mode in the default call setting information or the caller's class information.

[0153] Referring to part (d) of FIG. 7a and part (b) of FIG. 7b, the electronic device may perform a call connection according to the AI ​​assistant call mode based on the second user input (U2) for the modified first graphic object (710a). For example, the electronic device may perform a call connection according to the AI ​​assistant call mode based on identifying that the second user input (U2) satisfies the third condition. According to one embodiment, the electronic device may identify that the first user input (U2) satisfies the third condition when the second user input (U2) is identified as corresponding to a swipe input in a third direction (D3). In the present disclosure, the swipe input in the third direction (D3) may be referred to as a third swipe input. The third swipe input may include a swipe input in a direction (D3) from the inside to the outside of the modified first graphical object (710a), as illustrated in the screen (704) of portion (d) of FIG. 7a and portion (b) of FIG. For example, the third swipe input may include a swipe input from, for example, a point within the inside area of ​​the modified first graphical object (710a) to a point within the outside area of ​​the modified first graphical object (710a) or to a point within the inside area of ​​the second graphical object (720).

[0154] FIG. 8 is a flowchart illustrating operations before a call connection of an electronic device according to one embodiment of the present disclosure.

[0155] The pre-call connection operation of the embodiment of FIG. 8 may be an operation performed by an electronic device (e.g., the electronic device (101) of FIG. 1a) to generate and provide a first message (e.g., the first message (31b) of FIG. 2b, the first messages (740a, 740b) of FIGS. 7a and 7b) before a call connection is performed, for example, when the call mode is selected as the AI ​​assistant call mode.

[0156] Referring to FIG. 8, in operation 810, the electronic device may select (or identify) at least one representative application associated with the sender of the incoming call. The description of operation 810 may refer to the related descriptions of operation 411 of FIG. 4 and operation 330 of FIG. 3. Therefore, duplicate descriptions are omitted. According to one embodiment, the electronic device may select at least one application from a plurality of applications based on information about the sender and correlation information (e.g., correlation map information of FIG. 9b) between a plurality of applications installed on the electronic device using an AI model. The operation of selecting a representative application is described below with reference to, for example, FIGS. 9a and 9b. Examples of selected representative applications are described below with reference to FIGS. 10a and 10b.

[0157] In operation 820, the electronic device may obtain information about the caller's call intent based on data associated with at least one representative application. For a description of operation 820, reference may be made to the relevant descriptions of operation 412 of FIG. 4 and operation 330 of FIG. 3 . Therefore, any duplicate description will be omitted.

[0158] In operation 830, the electronic device may generate a first message (e.g., the first message (31b) of FIG. 2B , the first messages (740a, 740b) of FIGS. 7A and 7B ) including information associated with the received call (e.g., information associated with the caller's call intent) based on information about the caller's call intent. For a description of operation 830, reference may be made to the relevant descriptions of operation 413 of FIG. 4 and operation 330 of FIG. 3 . Therefore, any duplicate description will be omitted.

[0159] In operation 840, the electronic device may display a screen including the generated first message. For a description of operation 840, reference may be made to the related description of operation 330 of FIG. 3 . Therefore, a duplicate description will be omitted. The operation of acquiring the caller's call intent based on data associated with the representative application and generating and displaying the first message including the call intent will be described below with reference to, for example, FIGS. 11A, 11B, and 13 .

[0160] FIG. 9A illustrates user registration information and applications used to select a representative application according to one embodiment of the present disclosure.

[0161] FIG. 9b illustrates correlation map information between user registration information and applications used to select a representative application according to one embodiment of the present disclosure.

[0162] According to one embodiment, as illustrated in FIG. 9A, the sender's user registration information (910) may include at least one of the sender's registration information (e.g., name, ID, phone number, email, address, workplace, relationship).

[0163] According to one embodiment, as illustrated in FIG. 9b, the plurality of applications (920) may include, but are not limited to, a phone application, a mail application, a messenger application, a calendar application, a call recording application, and / or a gallery application.

[0164] According to one embodiment, as illustrated in FIG. 9b, the correlation map information may provide values ​​of the correlation between each installed application (e.g., APP1 to APP6) (variable 2) installed on the electronic device and each user registration information (e.g., name, ID, phone number, email, job, relationship) (variable 1).

[0165] According to one embodiment, the electronic device may select at least one representative application from a plurality of applications (920) installed in the electronic device based on the sender's user registration information (910) (or the sender's class information). For example, the electronic device may select at least one representative application from the plurality of applications (920) using correlation map information that provides information on the correlation between the sender's user registration information (910) and the plurality of applications (920) installed in the electronic device. For example, the electronic device may select at least one application among the plurality of applications (920) having a correlation value greater than or equal to a reference value (e.g., 0.5) as the representative application using the user registration information (910) and the correlation map information. For example, the electronic device may input input data (e.g., prompt data) generated based on the user registration information (910), information on the plurality of applications (920) installed in the electronic device, and / or the correlation map information into an AI model (e.g., the AI ​​model (400) of FIG. 4), and obtain information on the representative application as output data from the AI ​​model.

[0166] FIGS. 10A and 10B illustrate representative applications according to one embodiment of the present disclosure.

[0167] Referring to FIG. 10A, the representative application may be a call application (1000a) (or a call recording application) that includes call recording data. In one embodiment, the call application (1000a) may include call recording data associated with a caller. For example, the call application (1000a) may record and store call recording data associated with a caller, along with the date and time of the call connection. The call recording data associated with a caller may, for example, include data recording the content of each call made with the caller.

[0168] According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1A) may analyze call content associated with a caller based on call recording data associated with the caller included in a call application (1000a) using an AI model (e.g., the AI ​​model (400) of FIG. 4), thereby identifying at least one major call content (or topic) (e.g., a project progress report). For example, the electronic device may analyze recent call content associated with the caller based on call recording data associated with the caller using the AI ​​model, thereby obtaining at least one major call content. The recent call content may include, for example, content of calls made within a specified period (e.g., one month) from the current point in time.

[0169] Referring to FIG. 10b, the representative application may be a mail application (1000b) that includes mail data. In one embodiment, the mail application (1000b) may include mail data associated with a sender. For example, the mail application (1000b) may record and store mail data associated with a sender along with the time the mail was received or sent. The mail data associated with a sender may include, for example, data of mail (1010b) in which the sender or recipient of the mail is the sender of a call, and / or data of mail (1020b) in which the sender or recipient of the mail is not the sender of a call and includes content associated with the sender (or the sender's intent to make the call).

[0170] According to one embodiment, the electronic device may analyze mail content associated with the sender based on mail data associated with the sender included in the mail application (1000b) using an AI model to identify at least one major mail content (or topic) (e.g., a report on the progress of Project A) and / or content related to the at least one major mail content (e.g., matters related to the progress of Project A, e.g., a schedule for a company meeting, a sample delivery date for demo production, a major issue of a sample production company). For example, the electronic device may analyze recent mail content associated with the sender based on mail data associated with the sender using an AI model to obtain at least one major mail content and content related to the at least one major mail content. The recent mail content may include, for example, content of mail within a specified period (e.g., one month) from the current time.

[0171] FIGS. 11A and 11B are diagrams illustrating an operation of an electronic device providing a first message in a pre-call connection state based on a representative application according to one embodiment of the present disclosure.

[0172] Referring to FIG. 11a, the representative application may be a messenger application (1100a). The messenger application (1100a) may include message data associated with the sender (e.g., data of a message exchanged between the sender and recipient via the messenger application (1100a). For example, as illustrated in FIG. 11a, the message data may include multiple main message contents (or topics) (e.g., contents related to Jeju Island travel and accommodations (1110a) and contents related to bag purchases (1120a)).

[0173] Referring to FIG. 11b, an electronic device (e.g., the electronic device (101) of FIG. 1a) may analyze message content associated with a sender based on message data associated with the sender included in a messenger application (1100a) using an AI model (e.g., the AI ​​model (400) of FIG. 4), obtain the call intent of the sender associated with at least one main message content, and generate a first message (1110b, 1120b) (e.g., the first message (31b) of FIG. 2b)) based on the call intent of the sender. For example, as illustrated in the first screen (1101a) of part (a) of FIG. 11b, the first message (1110b) may include information on multiple call intents (e.g., a call intent related to a Jeju Island travel itinerary with a probability of 64% and a call intent related to a bag purchase with a probability of 32%). For example, as illustrated in the second screen (1102b) of part (b) of FIG. 11b, the first message (1120b) may further include additional information for each of the plurality of call intentions (e.g., additional information related to Jeju Island travel (e.g., information on good restaurants, information on recommended travel destinations) and / or additional information related to bag purchase (e.g., recommendations for seasonal trendy colors, guidance on lowest price purchase sites)).

[0174] As described above, if there are multiple main topics associated with a caller obtained based on the data associated with the caller included in the representative application, the first message provided in the pre-call state may also provide information about multiple call intents and additional information for each of the multiple call intents.

[0175] FIG. 12 is a diagram illustrating an operation of an electronic device providing sender information based on a representative application according to one embodiment of the present disclosure.

[0176] Referring to FIG. 12, the representative application may be a gallery application (1200). The gallery application (1200) may include image data associated with the sender (e.g., data of a photo including the sender). For example, as illustrated in FIG. 12, the image data may include data of a representative image (1210) associated with the sender (e.g., a photo taken together of the sender and recipient).

[0177] According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1A) may analyze image content associated with a sender based on image data associated with the sender included in a gallery application (1200) using an AI model (e.g., the AI ​​model (400) of FIG. 4), to obtain at least one representative image, and generate or change sender information (1220) (e.g., sender information (20b) of FIG. 2B) based on the at least one representative image. For example, as illustrated in a screen (1201) of part (b) of FIG. 12, the sender information (1220) may include a representative image (1220) of the sender (e.g., a photo taken together of the sender and the recipient) to replace the basic image of the sender (e.g., an image displayed in the sender information (20b) of FIG. 2B).

[0178] As described above, the electronic device can change and provide the image of the sender included in the sender information by using the representative image of the sender acquired based on the sender and image data included in the representative application.

[0179] FIG. 13 is a diagram for explaining an operation of an electronic device providing a first message in a pre-call connection state based on a representative application and a call reception time according to one embodiment of the present disclosure.

[0180] Referring to FIG. 13, the representative application may be a calendar application (1300a). The calendar application (1300a) may include schedule data (e.g., data regarding a lunch appointment with the sender, a birthday of the recipient, a travel schedule with the sender, and a meeting schedule with the sender). For example, as illustrated in part (a) of FIG. 13, the schedule data may include multiple major schedule contents (or topics) (e.g., a private schedule (e.g., a birthday of the recipient) and a public schedule (e.g., a business meeting schedule with the sender)).

[0181] According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1A) may obtain (or analyze) a caller's call intent based on schedule data and call reception time included in a calendar application (1300a) using an AI model (e.g., the AI ​​model (400) of FIG. 4), and generate a first message (1310, 1320) (e.g., the first message (31b) of FIG. 2B)) based on the caller's call intent. For example, as illustrated in the first screen (1301) of part (b) of FIG. 13, if the call reception time is in the morning, the electronic device may assign a higher weight to a public schedule than a private schedule among the main schedule contents, and generate a first message (1310) including information about a call intent related to a public schedule (e.g., a call intent related to a work meeting tomorrow with a probability of 64%). For example, as illustrated in the second screen (1302) of part (c) of FIG. 13, if the call reception time is in the afternoon, the electronic device may generate a first message (1320) that includes information about a call intention related to a private schedule (e.g., a call intention related to a birthday celebration tomorrow with a probability of 64%) by giving a higher weight to a private schedule than to a public schedule among the main schedule contents.

[0182] As described above, if there are multiple main topics obtained based on data included in the representative application, the electronic device can provide a first message including information about a call intent with a higher probability by giving a higher weight to one of the multiple topics (e.g., a private topic and a public topic) based on the call reception time.

[0183] FIG. 14 is a diagram for explaining an operation of an electronic device providing a first message in various ways in a state before a call connection, according to one embodiment of the present disclosure.

[0184] Referring to FIG. 14, an electronic device (e.g., the electronic device (101) of FIG. 1a) may provide a first message (e.g., the first message (31b) of FIG. 2b) in various ways before a call is connected, using call reception-related information. The call reception-related information may include, but is not limited to, information on the urgency of the call, information on the caller's disposition, and / or information on the call reception preference. Information on the caller's disposition may be included, for example, in the caller's class information.

[0185] For example, as illustrated in the first screen (1401) of part (a) of FIG. 14, if the urgency of the call is identified as being higher than the reference urgency, the electronic device may provide a first message (1410) briefly indicating information about the intent of the call.

[0186] For example, as illustrated in the second screen (1402) of part (b) of FIG. 14, if the urgency of the call is identified as being lower than the reference urgency, the electronic device may provide a first message (1420) detailing information about the call intent.

[0187] For example, as illustrated in the third screen (1403) of part (c) of FIG. 14, if the call is identified as a call that the recipient does not wish to receive (e.g., a call with a preference for receiving calls lower than a reference preference), the electronic device may provide a first message (1430) that includes a selection option (or item) that allows the recipient to reject the call connection or perform another method of call connection (e.g., a selection option to receive the call via text, or a selection option to leave a missed message).

[0188] FIG. 15 is a flowchart illustrating operations during a call connection of an electronic device according to one embodiment of the present disclosure.

[0189] FIG. 16 illustrates an exemplary screen including a second message in a call connection state according to one embodiment of the present disclosure.

[0190] According to one embodiment, the call connection operation of the embodiment of FIG. 15 may be an operation performed by an electronic device (e.g., the electronic device (101) of FIG. 1a) to generate and provide a second message (e.g., the second message (32b) of FIG. 2b) during a call connection, for example, when the call mode is selected as an AI assistant call mode.

[0191] Referring to FIG. 15, according to one embodiment, in operation 1510, the electronic device may analyze the content of a conversation during a call. According to one embodiment, after connecting the call, the electronic device may analyze the content of the conversation using an AI model. For a description of operation 1510, reference may be made to the related descriptions of operation 421 of FIG. 4 and operation 332 of FIG. 3 . Therefore, any duplicate description will be omitted.

[0192] According to one embodiment, in operation 1520, the electronic device may generate a second message containing information related to the content of the conversation during the call based on the results of analyzing the content of the conversation during the call. For a description of operation 1520, reference may be made to the relevant descriptions of operation 422 of FIG. 4 and operation 332 of FIG. 3 . Therefore, any duplicate description will be omitted.

[0193] According to one embodiment, in operation 1530, the electronic device may display the generated second message (or a screen including the second message). For a description of operation 1530, reference may be made to the relevant description of operation 330 of FIG. 3 . Therefore, any duplicate description will be omitted.

[0194] According to one embodiment, as illustrated in the first screen (1601) of part (a) of FIG. 16, the second message (1610) may include information (1611) about the topic of the conversation during the call (e.g., conversation about a defect in Project A), summary information (1612) about the conversation content (e.g., link information about an email related to a defect in Project A, a self-defect rate value, details of a company's countermeasures), additional information (1613) related to the conversation content (e.g., description of ways to improve the defect rate), and / or additional prompts (1614). The additional information (1613) may be obtained, for example, based on internal information and / or external information (e.g., network information) stored in the electronic device.

[0195] According to one embodiment, as illustrated in the second screen (1602) of part (b) of FIG. 16, the second message (1620) may include information (1621) about the topic of conversation during the call (e.g., a conversation about pets), additional conversation topic recommendation information (1622) related to the conversation topic (e.g., additional conversation topic recommendations such as things your pet likes, sharing pictures of your pet), and / or additional prompts (1623). The additional conversation topic recommendation information (1622) may be obtained based on external information (e.g., network information), for example.

[0196] These second messages (1610, 1620) may be used to help the recipient conduct a smooth conversation with the caller during a call.

[0197] FIG. 17 is a flowchart illustrating operations after a call connection is terminated in an electronic device according to one embodiment of the present disclosure.

[0198] FIG. 18 illustrates an exemplary screen including a third message in a call connection termination state according to one embodiment of the present disclosure.

[0199] FIG. 19 is a diagram for explaining an operation of updating class information of a sender according to one embodiment of the present disclosure.

[0200] According to one embodiment, the post-call termination operation of the embodiment of FIG. 17 may be an operation performed by an electronic device (e.g., the electronic device (101) of FIG. 1A) to generate and provide a third message (e.g., the third message (33b) of FIG. 2B) in a call connection termination state, for example, when the call mode is selected as an AI assistant call mode.

[0201] Referring to FIG. 17, according to one embodiment, in operation 1710, the electronic device may analyze at least a portion (e.g., the entire content) of the call, and generate and provide a third message including information related to the content of the call. According to one embodiment, the electronic device may terminate the call connection, and based on the termination of the call connection, may analyze the entire content of the call using an AI model. The description of operation 1710 may refer to the related descriptions of operation 431 of FIG. 4 and operation 334 of FIG. 3 . Therefore, any duplicate description will be omitted.

[0202] According to one embodiment, at operation 1720, the electronic device may reconstruct and store the content of the call based on the topic of at least one conversation selected via the third message. For a description of operation 1720, reference may be made to the relevant descriptions of operation 432 of FIG. 4 and operation 334 of FIG. 3 . Therefore, any duplicate description will be omitted.

[0203] According to one embodiment, as illustrated in the first screen (1801) of FIG. 18, the third message (1810) may include summary information (1811a, 1811b, 1811c) including information summarizing the entire content of the call by topic of the conversation and / or selectable items (1812a, 1812b, 1813c) associated with each topic of the conversation included in the summary information.

[0204] According to one embodiment, the electronic device may select at least one conversation topic based on user input for selectable items (1811b, 1812b, 1813b) within the third message (1810). For example, as illustrated in the first screen (1801) of FIG. 18, if it is determined that a first selectable item (1812a) associated with summary information (1811a) regarding a first conversation topic (e.g., related to defect rate of Project A) and a second selectable item (1812b) associated with summary information (1811b) regarding a second conversation topic (e.g., related to lunch appointments tomorrow) are selected based on the user input, and a third selectable item (1812c) associated with summary information (1811c) regarding a third conversation topic (e.g., related to summer vacation plans) is not selected, the electronic device may identify that the first conversation topic and the second conversation topic are selected.

[0205] According to one embodiment, the electronic device may store the selected topic content or exit without saving it based on user input for the first selection button (1813) (e.g., a button for saving the selection) and the second selection button (1814) (e.g., a button for exiting without saving). For example, if the first selection button (1813) is selected, the electronic device may reconstruct the call content based on the content of the selected first and second conversation topics, excluding the content of the unselected third conversation topic, and store the reconstructed call content.

[0206] In operation 1730, the electronic device may create or update class information for the sender. A description of operation 1730 may refer to the related description of operation 330 in FIG. 3 . Therefore, any duplicate description will be omitted.

[0207] According to one embodiment, after a call connection is terminated, the electronic device may update class information about the caller based on at least a portion of the call (e.g., the entire call) or reconstructed call content. For example, as illustrated in FIG. 19 , the electronic device may update the class information from the first class information (1910), which is the current class information of the caller in FIG. 19 , to the second class information (1920). For example, the electronic device may update the class information to the second class information (1920) by modifying at least one piece of information (e.g., intimacy information, conversation responsiveness information, contact frequency, assistant conversation adoption rate information) included in the first class information (1910).

[0208] FIG. 20 is a diagram illustrating an operation of providing a second message and additional information associated with the second message in a call connection state of an electronic device according to one embodiment of the present disclosure.

[0209] Referring to FIG. 20, according to one embodiment, an electronic device (e.g., electronic device (101) of FIG. 1A) may be an electronic device (e.g., a foldable display device) that includes a display (e.g., a foldable display) that includes a plurality of display areas.

[0210] According to one embodiment, when the electronic device is a foldable display device, the electronic device may display a second message (2010) (e.g., the second message (1610) of FIG. 16) through a first display area (2001) in a call connection state, and display additional information (2020) associated with the second message (2010) (e.g., basic data (e.g., raw data) used to generate information included in the second message (2010)) through a second display area (2002). In this way, in the embodiment of FIG. 20, by providing the second message (2010) and the additional information (2020) together using multiple display areas, a larger amount of information may be provided to the user compared to providing the second message (2010) and the additional information (2020) through a single display area.

[0211] FIG. 21 is a diagram illustrating an operation of providing a second message and additional information associated with the second message in a call connection state of an electronic device according to one embodiment of the present disclosure.

[0212] Referring to FIG. 21, according to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1A) may display a second message (2010) (e.g., the second message (2010) of FIG. 20) and / or additional information associated with the second message (2010) (e.g., the additional information (2020) of FIG. 20) through an external electronic device (2100) (e.g., a wearable device (e.g., a smart watch)) connected to the electronic device during a call connection state. During a call connection state, a user of the electronic device typically conducts a call while the electronic device is positioned close to the ear. In this case, it is difficult for the user to check the display of the electronic device during the call. Accordingly, for example, as illustrated in parts 2101, 2102, and 2103 of FIG. 21, the electronic device can enable the user to easily check the second message (2010) by processing (e.g., transmitting data of the second message (2010) to the external electronic device (2100)) so that the second message (2010) is displayed on an external electronic device (2100) whose display can be easily checked during a call.

[0213] FIG. 22 is a flowchart illustrating a method of operating an electronic device according to one embodiment of the present disclosure.

[0214] For a description of the embodiment of Fig. 22, reference may be made to the descriptions given above in Figs. 1 to 21. Therefore, any duplicate description will be omitted.

[0215] Referring to FIG. 22, in operation 2210, an electronic device (e.g., the electronic device (101) of FIG. 1A) may display a screen corresponding to an incoming call (e.g., the first screen (701) of FIG. 7A) through a display. According to one embodiment, the screen may include a first graphic object (e.g., the first graphic object (710) of FIG. 7A) associated with a call connection of the incoming call.

[0216] In operation 2220, the electronic device may obtain a first user input for a first graphic object (e.g., the first user input (U1) of FIG. 7A).

[0217] At operation 2230, the electronic device can identify whether the first user input satisfies the first condition or the second condition.

[0218] According to one embodiment, the electronic device may identify that the first user input satisfies the first condition when the first user input is identified as corresponding to a swipe input in a first direction (e.g., the first direction (D1) of FIG. 7A). In the present disclosure, a swipe input in the first direction may be referred to as a first swipe input. The first swipe input may include, for example, a swipe input in a direction (D1) from outside a first area (711) of a first graphical object (710) to inside the first area (711), as illustrated in the second screen (702) of FIG. 7A.

[0219] According to one embodiment, the electronic device may identify that the first user input satisfies the second condition when the first user input is identified as corresponding to a swipe input in a second direction (e.g., the second direction (D2) of FIG. 6 ). In the present disclosure, the swipe input in the second direction may be referred to as a second swipe input. The second swipe input may include, for example, a swipe input in the direction (D2) from the second area (612) of the first graphical object (610) or from the inside of the first area (611) to the outside of the second area (612), as illustrated in the second screen (602) of FIG. 6 .

[0220] In operation 2241, based on identifying that the first user input satisfies the first condition, the electronic device may modify the first graphical object and display the modified first graphical object (e.g., the modified first graphical object (710a) of FIG. 7A) through the display. In one embodiment, the modified first graphical object may be associated with a call connection using an artificial intelligence model (e.g., the AI ​​model (400) of FIG. 4).

[0221] In operation 2242, the electronic device may use the AI ​​model to generate a first message (e.g., the first message (31b) of FIG. 2b) containing information related to the caller's intent to make a call based on at least one application (e.g., a representative application). In one embodiment, the at least one application may contain data associated with the caller.

[0222] In operation 2243, the electronic device can display the first message through the display.

[0223] In operation 2244, the electronic device may obtain a second user input (e.g., second user input (U2) of FIG. 7A) for the modified first graphic object while the first message is displayed.

[0224] In operation 2245, the electronic device may establish a call connection with the caller based on the second user input. The call connection in operation 2245 may be, for example, a call connection according to an AI assistant call mode.

[0225] In operation 2245, the electronic device can establish a call connection with the caller without displaying the first message based on identifying that the first user input satisfies the second condition.

[0226] According to one embodiment, the information related to the call intent may include: call intent information indicating the caller's call intent and probability information indicating a probability value of the call intent.

[0227] According to one embodiment, the electronic device can identify the time of reception of the incoming call, and analyze the caller's call intent using the AI ​​model based on the time of reception of the incoming call and data associated with the caller included in the at least one application.

[0228] According to one embodiment, the first graphic object and the modified first graphic object may include an icon indicating a call connection using the AI ​​model.

[0229] In one embodiment, the screen may include a second graphical object associated with a timer for performing a call connection using the AI ​​model. In response to the timer expiring, the electronic device may cause the call connection to be performed without the second user input.

[0230] According to one embodiment, the electronic device may set the value of the timer based on the urgency of the call connection associated with the caller's calling intent.

[0231] According to one embodiment, the electronic device may use the AI ​​model to select at least one application from the plurality of applications based on correlation information between the sender's information and the plurality of applications installed on the electronic device.

[0232] According to one embodiment, after performing the call connection, the electronic device may analyze the content of the conversation using the AI ​​model, generate a second message including information related to the content of the conversation using the AI ​​model, and display the second message through the display.

[0233] According to one embodiment, the electronic device may terminate the call connection, analyze the entire content of the call using the AI ​​model based on the termination of the call connection, display a third message providing selection options for topics of multiple conversations during the call through the display, obtain a third user input for the third message, reconstruct the content of the call based on the topic of at least one conversation selected based on the third user input, and store data of the content of the reconstructed call in the memory.

[0234] According to one embodiment, the electronic device may generate class information of the caller based on identifying that the call connection has been terminated.

[0235] According to one embodiment, the class information may include at least one of information about the relationship with the sender, information about the intimacy with the sender, information about the main interests of the sender, information about the most relevant applications, information about the conversation response rate, information about the frequency of contact, information about the tendencies of the sender, or information about the adoption rate of AI assistant conversation.

[0236] The embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0237] The term "module" used in the embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0238] One embodiment of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0239] According to one embodiment, the method according to one embodiment disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0240] According to one embodiment, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and arranged in other components. According to one embodiment, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to one embodiment, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In electronic devices, display; communication circuit; At least one processor comprising a processing circuit; and A memory comprising at least one storage medium storing instructions, said instructions, when individually or collectively executed by said at least one processor, causing said electronic device to: A screen corresponding to an incoming call is displayed through the display, and the screen includes a first graphical object associated with the call connection of the incoming call. Obtaining a first user input for the first graphic object, Based on identifying that the first user input satisfies the first condition: By modifying the first graphic object, the modified first graphic object is displayed through the display, and the modified first graphic object is associated with a call connection using an artificial intelligence (AI) model. Using the above AI model, a first message is generated that includes information related to the caller's call intent based on at least one application, wherein the at least one application includes data associated with the caller, Displaying the first message through the above display, While the first message is displayed, a second user input for the modified first graphic object is obtained, Connecting a call with the caller based on the second user input, An electronic device that causes a call connection to be made with the caller without displaying the first message, based on identifying that the first user input satisfies the second condition.

2. In paragraph 1, information related to the intent of the call is: An electronic device comprising call intent information indicating the caller's call intent and probability information indicating a probability value of the call intent.

3. In paragraph 1 or 2, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Identify the time of receipt of the above incoming call, An electronic device that causes the caller's call intent to be analyzed using the AI ​​model based on the reception time of the incoming call and data associated with the caller included in the at least one application.

4. An electronic device according to claim 1, wherein the first graphic object and the modified first graphic object include an icon indicating a call connection using the AI ​​model.

5. In any one of paragraphs 1 to 4, the screen includes a second graphic object associated with a timer for performing a call connection using the AI ​​model, The above instructions, when individually and / or collectively executed by the at least one processor, cause the electronic device to: An electronic device that, in response to the expiration of said timer, causes said call connection to be performed without said second user input.

6. In the fifth paragraph, the instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An electronic device that causes the value of the timer to be set based on the urgency of the call connection associated with the caller's call intent.

7. In any one of paragraphs 1 to 6, the instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An electronic device that causes at least one application to be selected from a plurality of applications based on correlation information between the sender's information and a plurality of applications installed on the electronic device using the AI ​​model.

8. In any one of paragraphs 1 to 7, the instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: After performing the above call connection, the content of the conversation is analyzed using the AI ​​model, Using the AI ​​model, a second message is generated that includes information related to the content of the conversation, An electronic device that causes the second message to be displayed through the display.

9. In any one of paragraphs 1 to 8, the instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: End the above call connection, Based on the termination of the call connection, the entire content of the call is analyzed using the AI ​​model, Displaying a third message providing selection options for the topics of multiple conversations during the call through the above display, Obtaining a third user input for the third message, Reconstructing the content of the call based on the topic of at least one conversation selected based on the third user input; An electronic device that causes data of the content of the reconstructed call to be stored in the memory.

10. In the 9th paragraph, the instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Based on identifying that the call connection has been terminated, causing the class information of the caller to be generated, The above class information is, An electronic device comprising at least one of information about a relationship with the sender, information about a level of intimacy with the sender, information about the sender's main interests, information about top-relevant applications, information about conversation responsiveness, information about contact frequency, information about the sender's tendencies, or information about the adoption rate of AI assistant conversations.

11. In the method of operating an electronic device, An action of displaying a screen corresponding to an incoming call through a display, the screen including a first graphic object associated with a call connection of the incoming call; An operation of obtaining a first user input for the first graphic object; Based on identifying that the first user input satisfies the first condition: An operation of modifying the first graphic object and displaying the modified first graphic object through the display, wherein the modified first graphic object is associated with a call connection using an artificial intelligence (AI) model, An operation of generating a first message including information related to the caller's call intent based on at least one application using the AI ​​model, wherein the at least one application includes data associated with the caller, An action of displaying the first message through the display; An action of obtaining a second user input for the modified first graphic object while the first message is displayed; An operation of connecting a call with the caller based on the second user input, and A method comprising: an action of establishing a call connection with the caller without displaying the first message, based on identifying that the first user input satisfies the second condition.

12. In paragraph 11, the information related to the call intent is: A method comprising call intent information indicating the caller's call intent and probability information indicating a probability value of the call intent.

13. In the 11th or 12th paragraph, the method: An action for identifying the reception time of the above incoming call; and A method comprising an operation of analyzing the caller's call intention using the AI ​​model based on the reception time of the incoming call and data associated with the caller included in the at least one application.

14. A method according to claim 11, wherein the first graphic object and the modified first graphic object include an icon indicating a call connection using the AI ​​model.

15. In any one of paragraphs 11 to 14, the screen includes a second graphic object associated with a timer for performing a call connection using the AI ​​model, and the method comprises: A method comprising, in response to the expiration of the timer, performing the call connection without the second user input.

Images