Electronic device, method, and non-transitory storage medium for executing automatic call using artificial intelligence model

Abstract

The present document relates to an electronic device, a method, and a non-transitory storage medium for executing an automatic call using an artificial intelligence model. According to one embodiment, the electronic device may comprise a display, a communication circuit, at least one processor, and a memory for storing instructions. The instructions, when executed individually or collectively by the at least one processor, may instruct the electronic device to: analyze data related to a user interaction by using a first artificial intelligence model; on the basis of the analysis result, acquire message information related to the purpose of making a call with a designated recipient by using a generative artificial intelligence model; establish a call connection with an external electronic device of the recipient through the communication circuit by executing an automatic call function using the generative artificial intelligence model; transmit, to the external electronic device through the communication circuit, a first message generated on the basis of the message information by using the generative artificial intelligence model without user intervention; and transmit, to the external electronic device, a second message to reply, which is generated on the basis of the message received from the external electronic device and the message information, by using the generative artificial intelligence model without user intervention. Various other embodiments are also possible.

Description

Electronic device, method, and non-temporary storage medium for executing automatic telephone calls using an artificial intelligence model

[0001] The present disclosure relates to an electronic device, a method, and a non-transient storage medium for executing an automatic telephone using an artificial intelligence model.

[0002] With the advancement of digital technology, electronic devices are being provided in various forms, such as smartphones, tablet PCs, and PDAs. Electronic devices are also being developed in wearable forms to enhance portability and user accessibility. Electronic devices can be configured in various forms to be worn on parts of the user's body, and as technology advances, technologies are being developed to provide real-world spaces that correspond to the actual external environment (e.g., virtual reality, augmented reality, or mixed reality).

[0003] Meanwhile, the electronic device may utilize artificial intelligence (AI) models to provide various services. At least some of the various AI models for various services may be implemented as generative AI models. Depending on the implementation, the AI ​​models may operate in a form where multiple AI models are connected.

[0004] With increasing interest in artificial intelligence models, the field of artificial intelligence is experiencing rapid growth, and electronic devices are utilizing AI models to provide automated telephone services that automatically make calls to commercial facilities, service centers, or public institutions.

[0005] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to the present disclosure.

[0006] According to one embodiment of the present disclosure, an electronic device may include a display, a communication circuit, at least one processor, and a memory for storing instructions.

[0007] When the above instructions are executed individually or collectively by the at least one processor, the electronic device is configured to analyze data related to user interaction using a first artificial intelligence model based on identifying telephone-related interactions.

[0008] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device is enabled to obtain message information related to the purpose of communicating with the recipient of the phone using a generative artificial intelligence model based on the analysis results.

[0009] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device is configured to establish a call connection with the recipient's external electronic device through the communication circuit using the generative artificial intelligence model.

[0010] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device transmits a first message generated based on the message information using the generative artificial intelligence model to the external electronic device through the communication circuit without user intervention.

[0011] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device transmits to the external electronic device a second message to be answered, generated based on a message received from the external electronic device and the message information, using the generative artificial intelligence model without user intervention.

[0012] According to one embodiment, the method of operation in an electronic device includes the operation of analyzing data related to user interaction using a first artificial intelligence model based on identifying interactions related to a telephone.

[0013] According to one embodiment, the method includes the operation of obtaining message information related to the purpose of making a call with the recipient of the phone using a generative artificial intelligence model based on the analysis results.

[0014] According to one embodiment, the method includes the operation of establishing a call connection with the recipient's external electronic device through the communication circuit using the generative artificial intelligence model.

[0015] According to one embodiment, the method includes the operation of transmitting a first message generated based on message information using the generative artificial intelligence model without user intervention to the external electronic device through the communication circuit.

[0016] According to one embodiment, the method includes the operation of transmitting to the external electronic device a message received from the external electronic device and a second message to be answered, generated based on the message information, using the generative artificial intelligence model without the intervention of the user.

[0017] According to one embodiment, in a non-transient storage medium storing one or more programs, the program includes a command that, when executed by at least one processor of an electronic device, causes the electronic device to execute an operation of analyzing data related to user interaction using a first artificial intelligence model based on identifying an interaction related to a telephone.

[0018] According to one embodiment, the program includes a command that, when executed by at least one processor of an electronic device, causes the electronic device to execute an operation of obtaining message information related to the purpose of making a call with the recipient of the phone using a generative artificial intelligence model based on an analysis result.

[0019] According to one embodiment, the program includes a command that, when executed by at least one processor of the electronic device, causes the electronic device to execute an operation of establishing a call connection with the recipient's external electronic device through the communication circuit using the generative artificial intelligence model.

[0020] According to one embodiment, the program includes a command that, when executed by at least one processor of an electronic device, causes the electronic device to perform an operation of transmitting a first message generated based on message information using the generative artificial intelligence model to the external electronic device through the communication circuit without user intervention.

[0021] According to one embodiment, the program includes a command that, when executed by at least one processor of an electronic device, causes the electronic device to execute an operation of transmitting to the external electronic device a second message to be answered, generated based on a message received from the external electronic device and the message information, using the generative artificial intelligence model without the intervention of the user.

[0022] FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments.

[0023] FIG. 2 is a drawing showing an example of the configuration of an electronic device according to one embodiment.

[0024] FIG. 3 is a diagram illustrating an example of an artificial intelligence model for performing automatic telephone calls according to one embodiment.

[0025] FIGS. 4a, FIGS. 4b, FIGS. 4c, and FIGS. 4d are drawings illustrating an example of performing an automatic telephone call using an artificial intelligence model according to one embodiment.

[0026] FIG. 5 is a diagram illustrating an example of performing automatic telephone calls using an artificial intelligence model according to one embodiment.

[0027] FIG. 6 is a diagram illustrating an example of performing automatic telephone calls using an artificial intelligence model according to one embodiment.

[0028] FIG. 7 is a diagram illustrating an example of performing automatic telephone calls using an artificial intelligence model according to one embodiment.

[0029] FIGS. 8A and FIGS. 8B are drawings illustrating an example of performing an automatic telephone call using an artificial intelligence model according to one embodiment.

[0030] FIG. 9 is a drawing showing an example of an operation method in an electronic device according to one embodiment.

[0031] FIGS. 10a, FIGS. 10b, and FIGS. 10c are drawings illustrating an example of performing an automatic telephone call using an artificial intelligence model according to one embodiment.

[0032] FIG. 11 is a diagram illustrating a generative artificial intelligence system according to one embodiment.

[0033] In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0034] Hereinafter, embodiments of the present disclosure are described in detail with reference to the drawings so that those skilled in the art can easily implement them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein. In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and brevity. The term "user" as used in the embodiments of the present disclosure may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0035] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or may communicate with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0036] The processor (120) can control at least one other component (e.g., hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., program (140)), for example, and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., sensor module (176) or communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., central processing unit or application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., graphics processing unit, neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0037] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0038] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0039] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0040] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0041] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0042] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0043] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0044] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0045] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0046] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0047] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0048] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0049] The power management module (188) can manage the power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0050] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0051] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0052] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0053] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0054] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0055] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0056] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0057] FIG. 2 is a diagram showing an example of the configuration of an electronic device according to one embodiment, FIG. 3 is a diagram illustrating an example of an artificial intelligence model for performing automatic telephone calls according to one embodiment, and FIG. 4a, FIG. 4b, FIG. 4c and FIG. 4d are diagrams illustrating examples of performing automatic telephone calls using an artificial intelligence model according to one embodiment.

[0058] Referring to FIGS. 2, FIGS. 3, FIGS. 4a, FIGS. 4b, FIGS. 4c and FIGS. 4d, an electronic device (201) according to one embodiment (e.g., the electronic device (101) of FIG. 1) may include a processor (210), memory (220), display (230), camera circuit (240), and communication circuit (250). An electronic device (201) according to one embodiment may include a first artificial intelligence model (310) and a second artificial intelligence model (320). Here, the first artificial intelligence model (310) may be a preprocessing model for performing data collection and analysis and determining whether to make an automatic phone call based on identifying phone-related interactions. The second artificial intelligence model (320) is an artificial intelligence model for performing automatic calls, and may be a generative artificial intelligence model (e.g., LLM (large language model) or LMM (large multimodal model)) for generating message information (e.g., text) (321) related to the purpose of calling the recipient of the call, based on an analysis result (e.g., first data (311)) input from the first artificial intelligence model (310). The second artificial intelligence model (320) may be a different model configured separately from the first artificial intelligence model (310).

[0059] According to one embodiment, a processor (210) of an electronic device (201) (e.g., processor (120) of FIG. 1) may collect (e.g., obtain) data related to user interaction (e.g., raw data) in relation to the execution of the second application and the third application in order to determine whether to execute an automatic phone function when a call is requested through the execution screen (413, 415) of a deep link (e.g., a third application such as search applications) while a phone application (hereinafter referred to as the first application) and another application (e.g., a messenger application) (hereinafter referred to as the second application) are being executed. The data related to user interaction may include information collected in relation to the execution of the second application and the execution of the third application (e.g., conversation messages, search information) and / or information stored in relation to the user's behavior prior to the call request.

[0060] According to one embodiment, the processor (210) provides data related to collected user interactions to the first artificial intelligence model (310), and analyzes the collected data by the first artificial intelligence model (320) to extract (e.g., obtain or confirm) data highly relevant to the purpose of the call (hereinafter referred to as the first data (311)). For example, the first data (311) may include information (421) about conversation messages collected through the execution screen (411) of the first application (e.g., "I want to go to the rooftop," "Schedule near Cheongdam?", "Okay, I'll ask if corkage is allowed," "I guess valet parking will be available~"), images (422) shared during the execution of the first application, and / or detailed information (423) retrieved through the execution screens (413, 415) of the third application (e.g., [Seat-style wine bar], [Public parking lot]).

[0061] According to one embodiment, the processor (210) may provide the extracted first data (311) as input to the second artificial intelligence model (320) and obtain message information (321) related to the purpose of a call generated by the second artificial intelligence model (320) based on the first data. As illustrated in FIG. 4b, the processor (210) may display a screen (e.g., pop-up window or user interface) (440) containing message information (321) (e.g., "Booking for 4 people at 6 PM tomorrow, checking if corkage and valet parking are available, booking at the terrace") and information (443) for executing an auto call (e.g., "Would you like to execute an auto call?" and an execution button (e.g., menu)) on the execution screen (431) of the first application (e.g., call application).

[0062] According to one embodiment, the processor (210) may display the execution screen (431) of the first application through the display (230) as shown in FIG. 4b, and may display message information (321) related to the purpose of the call and a menu (443) for automatic call execution (e.g., an auto call button) on the execution screen (431). According to one embodiment, the processor (210) may modify the message information (321) related to the purpose of the call based on a request (401) for modification of the message information (321) related to the purpose of the call.

[0063] According to one embodiment, the processor (210) can execute an automatic call function based on identifying an automatic call request (401) by inputting a menu (443) (e.g., an auto call button) for the call execution as shown in FIG. 4b, establish a call connection to the recipient's electronic device (hereinafter referred to as an external electronic device) through a communication circuit (250), and display an execution screen (432) for the automatic call function on a display (230). According to one embodiment, when the processor (210) executes the automatic call, it can display message information (321) in a first area (432a) of the execution screen (432) of the automatic call and display conversation content with the recipient (e.g., the transmitted message (452) and received messages (451, 453, 454) of FIG. 4b) in a second area (432b).

[0064] According to one embodiment, when a call to an automatic phone is established between the recipient and the processor (210), the processor (210) can generate a message to be transmitted to the recipient's external electronic device using a second artificial intelligence model (320) without user intervention, based on message information (321) related to the purpose, and transmit the generated message to the external electronic device. According to one embodiment, as illustrated in FIG. 4b, when the processor (210) receives a first received message (e.g., "Yes, this is Terrace.") (451) from an external electronic device, it can provide the message information (321) related to the purpose and the first received message (451) to the second artificial intelligence model (320). The second artificial intelligence model (320) can generate a first transmitted message (452) to be answered (e.g., "Hello, is it possible to make a reservation for 4 people tomorrow evening at 6 o'clock?") based on the message information (321) related to the purpose and the first received message (451) provided to the second artificial intelligence model (320). The processor (210) can transmit a first transmission message (452) generated by the second artificial intelligence model (320) to an external electronic device. Subsequently, when the processor (210) receives a second reception message (453) (e.g., "Yes, it is possible. Please tell me your name and contact information") from the recipient's external electronic device, it can provide the purpose-related message information (321) and the second reception message (453) to the second artificial intelligence model (320). The second artificial intelligence model (320) can generate a second transmission message (454) (e.g., "I am Lee**, and my number is 010-****-****") based on the purpose-related message information (321) and the second reception message (453). When the second artificial intelligence model (320) generates the second transmission message (454), if the second reception message (453) contains a question that is not in the message information (321) related to the purpose, the model can generate the second transmission message (454) that fits the context by using information learned from the context database stored in memory (430) (e.g., user's personalized data).The processor (210) can transmit the second transmission message (454) generated by the second artificial intelligence model (320) to an external electronic device through the communication circuit (250).

[0065] According to one embodiment, the processor (210) can continue to converse with the recipient using the second artificial intelligence model (320) without user intervention until the call ends. If the third received message (455) contains a question that is not in the message information (321) related to the purpose, the second artificial intelligence model (320) can generate a third transmitted message (456) as a context-appropriate answer based on learned data by analyzing the image (422) included in the extracted first data (311). The processor (210) can transmit the third transmitted message (456) generated by the second artificial intelligence model (320) to an external electronic device through the communication circuit (250). According to one embodiment, the second artificial intelligence model (320) can generate a transmission message (e.g., a first transmission message (452), a second transmission message (454) and / or a third transmission message (456)) based on the first data (311), message information (321) displayed in the first area (432a), and the content of a conversation with a recipient displayed in the second area (432b) (e.g., a first reception message (451), a second reception message (453) and / or a third reception message (455)).

[0066] According to one embodiment, the processor (210) may output a notification (471) (e.g., a ring or vibration notification) for a user confirmation request when the processor deviates from the purpose of the call or is unable to answer by the second artificial intelligence model (320) while transmitting and receiving conversation messages based on message information (321) related to the purpose of the call through the second artificial intelligence model (320). According to one embodiment, as shown in FIG. 4c, when the processor (210) deviates from the purpose of the call or is unable to answer by the second artificial intelligence model (320) while executing an automatic call, the processor (210) may acquire a transmission message (458) generated by the answer specified by the second artificial intelligence model (320) (e.g., "Yes, just a moment") and transmit the transmission message (458) to an external electronic device.

[0067] According to one embodiment, the processor (120) can transmit voice information entered by the user through the voice call function (473) or text information entered by the user through the input window of the execution screen (432) as a response (e.g., a sent message) to the recipient's external electronic device. According to one embodiment, when the voice call function (473) is executed, the processor (120) can display a message (e.g., "Switched to voice call") on the execution screen (432) indicating that the voice call has been switched. According to one embodiment, the processor (120) can convert voice information (e.g., conversation content) transmitted and received through the voice call function (473) into text information and display it in the second area (432b) on the execution screen (432) following the previous conversation message, or store it in memory (220).

[0068] According to one embodiment, when an object (481) for checking previously called content via automatic phone is selected on the third execution screen (433) of the first application after the call is ended, as illustrated in FIG. 4d, the processor (120) can display on the display (230) through the fourth execution screen (434) information (491) that requires user verification later (e.g., "notifying the type of parking vehicle, notifying the type of corkage alcohol, selecting the menu in advance") and call summary information (493) set based on transmitted or received messages.

[0069] According to one embodiment, the processor (120) runs the first application in the background without displaying the execution screens (431 or 432) of the first application on the display (230) based on the connection of a call for automatic phone, and can display the execution screens (431 or 432) of the first application on the display (230) in response to a user's request.

[0070] According to one embodiment, the processor (210) may be a hardware component (function) or a software element (program) comprising at least one component provided in the electronic device (201), such as a hardware module or a software module (e.g., an application program). According to one embodiment, the processor (210) may include, for example, one or more combinations of hardware, software, or firmware. The processor (210) may be configured to omit at least some of the components or to include additional components for performing image processing operations in addition to the components.

[0071] According to one embodiment, the memory (220) (e.g., the memory (130) of FIG. 1) can store applications. For example, the memory (220) can store applications related to calls (functions or programs), applications related to messengers, applications related to searches, or applications related to artificial intelligence models (320). The memory (220) can store messages transmitted and received and collected data (e.g., user interaction data), personalized data, and information resulting from the execution of other applications while making calls through an external electronic device and an automatic phone function.

[0072] According to one embodiment, the memory (220) may store various data generated during the execution of the program (140), including a program used for functional operation (e.g., the program (140) of FIG. 1). For example, the memory (220) may include a program (140) area and a data area (not shown). The program (140) area may store related program information for operating the electronic device (201), such as an operating system (OS) (e.g., the operating system (142) of FIG. 1) that boots the electronic device (201). The data area (not shown) may store transmitted and / or received data and generated data according to various embodiments. Additionally, the memory (220) may be configured to include at least one storage medium among flash memory, hard disk, multimedia card micro type memory (e.g., secure digital (SD) or extreme digital (XD) memory), RAM, and ROM.

[0073] According to one embodiment, a display (230) (e.g., the display module (160) of FIG. 1) can display an execution screen of an application being executed. The display (230) can display an execution screen containing conversation content (e.g., conversation messages) and message information related to the purpose of the call generated by a generative artificial intelligence model while performing a call with a recipient through an automatic call function, under the control of a processor (210). According to one embodiment, the display (230) can be implemented in the form of a touch screen. When the display (230) is implemented in the form of a touch screen together with an input module, it can display various information generated according to the user's touch actions. According to one embodiment, the display (230) can be composed of at least one of an LCD (liquid crystal display), TFT-LCD (thin film transistor LCD), OLED (organic light emitting diodes), LED, AMOLED (active matrix organic LED), flexible display, and 3-dimensional display. Additionally, some of these displays may be configured to be transparent or light-transmitting so that the outside can be seen through them. This may be configured in the form of a transparent display including a TOLED (transparent OLED). According to one embodiment, other display modules mounted in addition to the display (230) may be further included (e.g., an extended display, a flexible display, or a display placed on the rear (e.g., the rear relative to the front where the display (230) is placed).

[0074] According to one embodiment, the camera circuit (240) (e.g., the camera module (180) of FIG. 1) includes at least one camera and can capture information necessary to collect data for automatic telephone using at least one camera.

[0075] According to one embodiment, a communication circuit (250) (e.g., the communication module (190) of FIG. 1) can communicate with an external electronic device (e.g., the electronic device (102, 104) of FIG. 1, the server (108) of FIG. 1, or another user's electronic device). For example, the communication circuit (250) can transmit or receive messages while running an automatic call, and can transmit or receive messages related to the application being run. The communication circuit (250) can receive content for the automatic call service from an external electronic device or server. When running an automatic call, the communication circuit (250) can transmit real-time chat information and real-time voice information to or from an external electronic device. According to one embodiment, the communication circuit (250) may include a cellular module, a Wi-Fi (wireless-fidelity) module, a Bluetooth module, or a near field communication (NFC) module.

[0076] FIG. 5 is a diagram illustrating an example of performing automatic telephone calls using an artificial intelligence model according to one embodiment.

[0077] Referring to FIGS. 2, 3, and 5, a processor (210) of an electronic device (201) according to one embodiment (e.g., the electronic device (101) of FIG. 1) can perform editing of information (321) related to the purpose of a call displayed on the execution screen (431) of a first application (e.g., a call application). Editing of information (321) related to the purpose of a call can be performed automatically by a second artificial intelligence model (320) as in FIG. 5 (a), or manually by a user's request (e.g., user's gesture input (501)) as in FIG. 5 (b). According to one embodiment, when performing automatic editing as in FIG. 5 (a), the processor (210) may obtain editing recommendation information (511, 513) generated by analyzing information (321) related to the purpose of a call by the second artificial intelligence model (320), and display the obtained editing recommendation information (511, 513) on an execution screen (431). The processor (210) may display a graphic object (521) on a part of the information (321) related to the purpose of a call that requires user verification. According to one embodiment, as in FIG. 5 (b), when a part of the information (321) related to the purpose of a call is selected by a user's gesture input (501), the processor (210) may manually edit the selected part.

[0078] FIG. 6 is a diagram illustrating an example of performing automatic telephone calls using an artificial intelligence model according to one embodiment. FIG. 7 is a diagram illustrating an example of performing automatic telephone calls using an artificial intelligence model according to one embodiment.

[0079] Referring to FIGS. 2, FIGS. 3, FIGS. 6 and FIGS. 7, when a processor (210) of an electronic device (201) (e.g., the electronic device (101) of FIG. 1) identifies a call request (e.g., identifies the execution of a call application), it collects data from execution screens (611) through a first artificial intelligence model (310) and, based on the collected data, can automatically determine whether to execute an automatic phone function without user intervention. According to one embodiment, if a mechanical conversation or process occurs before the automatic phone function is executed, the processor (210) analyzes information related to the mechanical conversation or process (e.g., screen analysis or voice analysis), and based on the analysis result data, can determine whether to execute an automatic phone function (e.g., an automatic phone function such as an ARS (automated response system)) by the first artificial intelligence model (310). For example, the first artificial intelligence model (310) can identify a word inferred for automatic phone connection in the execution screens (611) or confirm the execution of the automatic phone function when mechanical voice information is received after a call request. When the execution of the automatic phone function is confirmed, the first artificial intelligence model (310) can transmit the collected data to the generative artificial intelligence model (320). The generative artificial intelligence model (320) can generate message information (621) related to the purpose of the call based on the collected data. The collected data (e.g., automatic phone content) may include data to be used for the automatic phone function based on at least one of information related to the user's previous behavior (e.g., data related to user interaction such as keyboard input, voice input, gaze input and / or gesture input), screen capture information, search result information, or information related to the previous execution screen.

[0080] According to one embodiment, the processor (210) can display message information (621) related to the purpose of a call on the execution screen (620) of the first application (e.g., a call application).

[0081] According to one embodiment, when the processor (210) receives an edit request (701) for message information (621) related to the purpose of a call, it can edit the message information (621) related to the purpose of a call (e.g., add or delete information) by user input.

[0082] According to one embodiment, when a call connection of an automatic telephone is established, the processor (210) can obtain a response message automatically generated by a second artificial intelligence model (320) based on message information (621) related to the purpose of the call, and transmit the obtained response message to an external electronic device of the recipient through a communication circuit (250).

[0083] According to one embodiment, the processor (210) can automatically perform a process related to the call of an automatic phone using a second artificial intelligence model (320) until it identifies that user intervention is required while performing the automatic phone call, proceed with the call of the automatic phone in the background without displaying the execution screen (620), and display an object (731) indicating the background execution on the currently displayed screen (710). The processor (210) can display the execution screen (620) again based on a user's request.

[0084] According to one embodiment, when the processor (210) identifies that human voice information other than mechanical voice is received, it may output notification information (630) (e.g., vibration and / or bell) to the user.

[0085] FIGS. 8A and FIGS. 8B are drawings illustrating an example of performing an automatic telephone call using an artificial intelligence model according to one embodiment.

[0086] Referring to FIGS. 2, FIGS. 3 and FIGS. 8a and FIGS. 8b, a processor (210) of an electronic device (201) according to one embodiment (e.g., electronic device (101) of FIG. 1) can execute a voice call function (813 or 825) when an object (811 or 823) related to a call is selected by a user on a current screen (810 or 820) after notification information is output. The processor (210) can use a generative artificial intelligence model (320) to display messages transmitted or received on an execution screen (820) while the user is having a conversation with a recipient via an automated call, based on information (821) related to the purpose of the call. The processor (210) can obtain an automatically generated answer message based on personalized data obtained through another application (e.g., a payment-related application (831) or a schedule-related application (833)) to generate an answer message while executing an automatic call using a generative artificial intelligence model (320).

[0087] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1 and / or the electronic device (201) of FIG. 2) may implement a software module related to a call (e.g., the program (140) of FIG. 1). The memory of the electronic device (e.g., the memory (130) of FIG. 1 and / or the memory (220) of FIG. 2) may store instructions (e.g., instructions) to implement the software module. At least one processor (e.g., processor (120) of FIG. 1 and / or processor (210) of FIG. 2) can execute instructions stored in memory to implement a software module and can control hardware associated with the function of the software module (e.g., sensor module (176) of FIG. 1, camera module (180), communication module (190) of FIG. 1 and / or communication circuit (250) of FIG. 2, display module (160) of FIG. 1 and / or display (230) of FIG. 2).

[0088] A software module of an electronic device (101, 201) according to one embodiment may be configured to include a kernel (or HAL), a framework (e.g., middleware (144) of FIG. 1), and an application (e.g., application (146) of FIG. 1). At least some of the software modules may be preloaded onto the electronic device (101, 201) or downloadable from a server (e.g., server (108)).

[0089] According to one embodiment, the kernel may include, for example, a system resource manager or a device driver, but may be configured to include other modules, not limited thereto. The system resource manager may perform control, allocation, or reclamation of system resources. The device driver may include, for example, a display driver, a camera driver, a Bluetooth driver, a shared memory driver, a USB driver, a keypad driver, a WIFI driver, an audio driver, or an IPC (inter-process communication) driver.

[0090] According to one embodiment, the framework may provide functions commonly required by the application, or provide various functions to the application through an application programming interface (API) (not shown) so that the application can efficiently use limited system resources within the electronic device (101, 201). The framework may include modules that form combinations of various functions of the components. The framework may provide modules specialized for each type of operating system to provide differentiated functions. The framework may dynamically delete some existing components or add new components.

[0091] According to one embodiment, the application may be configured to include an application (e.g., a module, a manager, or a program) related to a video streaming service. The application may include an application received from an external electronic device (e.g., the server (108) or electronic devices (102, 104) of FIG. 1 or the external electronic device (601) of FIG. 6). According to one embodiment, the application may include a preloaded application or a third-party application downloadable from the server. The components of the software module and the names of the components according to the illustrated embodiments may vary depending on the type of operating system. According to one embodiment, at least a portion of the software module may be implemented as software, firmware, hardware, or a combination of at least two of these. At least a portion of the software module may be implemented (e.g., executed) by a processor (e.g., AP). At least a portion of the software module may include, for example, a module, a program, a routine, a set of instructions, or a process for performing at least one function.

[0092] As such, in one embodiment, the main components of an electronic device have been described through the electronic device (101, 201) of FIGS. 1 and 2. However, in various embodiments, not all components illustrated in FIGS. 1 and 2 are essential components, and the electronic device (101, 201) may be implemented with more components than those illustrated, or with fewer components. Additionally, the positions of the main components of the electronic device (101, 201) described above in FIGS. 1 and 2 may be changed according to various embodiments.

[0093] According to one embodiment, an electronic device (e.g., electronic device (101) of FIG. 1, electronic device (201) of FIG. 2)) may include a display (e.g., display module (160) of FIG. 1 or display (230) of FIG. 2), a communication circuit (e.g., communication module (190) of FIG. 1 or communication circuit (250) of FIG. 2), at least one processor (e.g., processor (120) of FIG. 1 or processor (210) of FIG. 2)), and a memory for storing instructions (e.g., memory (130) of FIG. 1 or memory (220) of FIG. 2).

[0094] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be enabled to analyze data related to user interaction using a first artificial intelligence model (e.g., the first artificial intelligence model (310) of FIG. 3) based on identifying interactions related to telephone.

[0095] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be enabled to obtain message information related to the purpose of making a call to the recipient of the phone using a generative artificial intelligence model (e.g., generative artificial intelligence model (320) of FIG. 3) based on the analysis results.

[0096] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be enabled to establish a call connection with the recipient's external electronic device through the communication circuit using the generative artificial intelligence model.

[0097] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be enabled to transmit a first message generated based on the message information using the generative artificial intelligence model to the external electronic device through the communication circuit without user intervention.

[0098] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be enabled to transmit to the external electronic device a second message to be answered, generated based on a message received from the external electronic device and the message information, using the generative artificial intelligence model without user intervention.

[0099] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be configured to output notification information for a user confirmation request based on the identification by the generative artificial intelligence model that a response to a message received from the external electronic device is impossible or that confirmation by the user is required, and to transmit a third message input by the user to the external electronic device through the communication circuit.

[0100] According to one embodiment, the third message is a voice message or a text message, and the notification information may be a vibration or a ringtone.

[0101] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be enabled to execute the automatic telephone function using the first artificial intelligence model based on identifying information related to the execution of the automatic telephone function in the analysis result by the first artificial intelligence model.

[0102] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be configured to generate the second message based on the received message and the message information, based on identifying that the correlation value between the received message and the message information is greater than a specified value using the generative artificial intelligence model.

[0103] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be configured to edit a portion of the message information based on the user's edit request and display the edited message information through the display.

[0104] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be configured to obtain editing recommendation information for a portion of the message information generated by the generative artificial intelligence model, display the editing recommendation information on the message information through the display, and transmit the editing recommendation information to the external electronic device when transmitting the message information using the generative artificial intelligence model without user intervention.

[0105] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be able to display an execution screen including messages transmitted and received during the execution of the automatic telephone and message information through the display.

[0106] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be configured to obtain information requiring confirmation from the user generated based on messages transmitted or received during the execution of the automatic call, based on identifying that the execution of the automatic call has ended using the generative artificial intelligence model, store the information requiring confirmation in the memory, and display call summary information and the information requiring confirmation set based on the transmitted or received messages through the display, based on identifying a request for confirmation of call content by the user.

[0107] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may execute the function of the automatic call in the background without displaying the execution screen on the display while the automatic call is being executed, and may display the execution screen on the display in response to a request from the user.

[0108] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be enabled to obtain a fourth message generated by the generative artificial intelligence model without user intervention, based on at least one of the personalized data or collected data stored in the memory, based on identifying that the correlation value between the message received from the external electronic device and the message information is less than or equal to a specified value.

[0109] FIG. 9 is a diagram illustrating an example of an operation method in an electronic device according to one embodiment. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0110] Referring to FIG. 9, an electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1 and the electronic device (201) of FIG. 2) can identify a call request through a deep link while running a first application (e.g., a call application) and a second application (e.g., a messenger application or a search-related application) different from the first application (e.g., a call application) in operation 901.

[0111] In operation 903, the electronic device can collect and analyze data related to user interactions (e.g., phone-related interactions) using a first artificial intelligence model (e.g., the first artificial intelligence model (310) of FIG. 3) based on identifying phone-related interactions. Based on the results analyzed by the first artificial intelligence model, the electronic device can extract first data to be used when executing an automatic phone call using a generative artificial intelligence model (e.g., the second artificial intelligence model (320) of FIG. 3) from the collected data. Here, the first artificial intelligence model may be a preprocessing model for performing data collection and analysis when there is a phone-related interaction and determining whether to make an automatic phone call. The generative artificial intelligence model is an artificial intelligence model for performing automatic calls, and may be a generative artificial intelligence model (e.g., LLM (large language model) or LMM (large multimodal model)) for generating message information (e.g., message information (321) or text) related to the purpose of calling the recipient of the call based on an analysis result input from the first artificial intelligence model (e.g., the first data (311) in FIG. 3). The second artificial intelligence model may be a different model configured separately from the first artificial intelligence model.

[0112] In operation 905, the electronic device can obtain (e.g., generate) message information related to the purpose of making a call to a designated recipient based on the first data by a generative artificial intelligence model.

[0113] In operation 907, the electronic device performs the function of an automatic telephone using a generative artificial intelligence model and can establish a call connection with the electronic device of a designated recipient (hereinafter referred to as an external electronic device) through a communication circuit.

[0114] In operation 909, the electronic device, based on the establishment of a call connection, can use a generative artificial intelligence model to initiate a call via an automatic phone function without user intervention and display a first execution screen containing message information through a display. The electronic device can display the first execution screen of the first application through a display (e.g., the display module (160) of FIG. 1 or the display (230) of FIG. 2). The electronic device can display message information related to the purpose of the call on the first execution screen. The electronic device can automatically edit the message information related to the purpose of the call displayed on the first execution screen by the generative artificial intelligence model or manually by the user (e.g., modifying words included in the message information, adding or deleting content).

[0115] In a 911 operation, the electronic device can use a generative artificial intelligence model without user intervention to obtain a message to be transmitted to an external electronic device based on message information related to the purpose of the call, and transmit the obtained message to the external electronic device. When the electronic device receives a message (hereinafter referred to as the received message) from an external electronic device, it can obtain a response message generated by a generative artificial intelligence model based on the message information and the received message, and transmit the obtained response message to the external electronic device using a generative artificial intelligence model without user intervention. If the electronic device identifies that the degree of correlation between the received message and information related to the purpose of the call is less than or equal to a specified value, it can obtain a response message generated by a generative artificial intelligence model based on personalized data stored in memory and / or collected data.

[0116] In a 913 operation, the electronic device can check whether the call on the automatic telephone has ended. If the check reveals that the call has not ended, the electronic device continues to perform the 911 operation, and if the call has ended, the electronic device can terminate the operation.

[0117] When performing the operation method of FIG. 9 as described above, in the 911 operation, if the electronic device identifies, based on message information, that a response to a received message is impossible or user confirmation is required by an artificial intelligence model while executing an automatic call, it may output notification information (e.g., vibration or ringtone) for requesting user confirmation. When the notification information is output, the electronic device may transmit a response message entered by the user to an external electronic device through a communication circuit. When the voice call function is executed by the user, the electronic device may transmit voice information to an external electronic device as a response message. The electronic device may transmit a text message entered by the user through a first execution screen to an external electronic device as a response message.

[0118] According to one embodiment, an electronic device may use a generative artificial intelligence model to identify that the execution of an automatic call has ended, and based on this, acquire information requiring user confirmation generated based on messages transmitted or received during the execution of the automatic call, and store the information requiring confirmation in memory. When the electronic device identifies a request to confirm call content, it may display call summary information and information requiring confirmation, which are set based on messages transmitted or received during the execution of the previous automatic call, through a display.

[0119] FIGS. 10a, FIGS. 10b, and FIGS. 10c are drawings illustrating an example of performing an automatic telephone call using an artificial intelligence model according to one embodiment.

[0120] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1 or the electronic device (201) of FIG. 2) can execute an automatic calling function using an artificial intelligence model when, for example, when calling a restaurant to change a reservation during a conversation through a messenger application as shown in FIG. 10a, and can execute an automatic calling for a reservation change based on message information (1011) related to the purpose of the call generated based on conversation messages displayed on an execution screen (1010) displayed on a display (230) using an artificial intelligence model.

[0121] An electronic device according to one embodiment, as illustrated in FIG. 10b, can execute an automatic call function using an artificial intelligence model when making a call by viewing a text message included in an execution screen (1020), and can execute an automatic call for a reservation change based on message information (1021) related to the purpose of the call generated based on conversation messages displayed on the execution screen (1020) using an artificial intelligence model.

[0122] An electronic device according to one embodiment, as illustrated in FIG. 10c, can execute an automatic call function using an artificial intelligence model when making a call with search information through a search-related application, and can execute an automatic call for a reservation change based on message information (1031) related to the purpose of the call generated based on the search information included in the execution screen (1030) using the artificial intelligence model.

[0123] FIG. 11 is a diagram illustrating a generative artificial intelligence system according to one embodiment.

[0124] Referring to FIG. 11, in a generative artificial intelligence system (800) according to one embodiment, a user query / response interface (810) (e.g., an input module (150) or a display module (160) of FIG. 1 or a display (230) of FIG. 2) can receive user input. The user input may be in the form of natural language, images and / or videos, but is not limited thereto. Additionally, context information may be transmitted along with the user input. The context information may include various additional information at the time of user input. For example, the additional information may include information about the application currently being used by the user or the user's location information. Furthermore, the user input may be in a mixed form of the aforementioned natural language, images, sounds, and context information. Additionally, the user input may be in a non-natural language form, such as selecting a menu. The user query / response interface (1110) can output the results of the generative artificial intelligence system to the user. The output can be in the form of natural language or specific content, and can also be provided in the form of actions requested by the user. The user query / response interface (1110) can output the results of the generative artificial intelligence system to the user. The output can be in the form of natural language or specific content, and can also be provided in the form of actions requested by the user.

[0125] An artificial intelligence framework (840) (e.g., the processor (120) of FIG. 1 or the processor (210) of FIG. 2) can receive input from a user and coordinate and control each component necessary to perform the user's intent based on the user's query.

[0126] User input received from the user query / response interface (1110) can be transmitted to a prompt design component (1141) (e.g., the processor (120) of FIG. 1 or the processor (210) of FIG. 2). The prompt design component (1141) can be used to generate prompts suitable for inputting user input into a large language model (LLM), a large vision model (LVM), or a large multimodal model (LMM). The prompt design component (1141) may be an artificial intelligence component that uses machine learning algorithms or neural networks to develop better prompts over time. The prompt design component (1141) can generate prompts by accessing a knowledge component containing user preference data, a prompt library, and prompt examples based on user input, and can transmit the generated prompts to the LLM, LVM, or LMM.

[0127] An API / Plug-in management component (1142) (e.g., the processor (120) of FIG. 1 or the processor (210) of FIG. 2) can perform the role of communicating with external information when there is a request for additional information when user input is passed as input to a generative model (e.g., the generative artificial intelligence (AI) model (320) of FIG. 3 or the cloud artificial intelligence (AI) model). The API / Plug-in management component (1142) establishes a channel to communicate with the outside of the artificial intelligence framework (1140) via an API, and through the established channel, it can access various data sources (e.g., a knowledge store (1120)) (e.g., the memory (130) of FIG. 1 or the memory (220) of FIG. 2). Additionally, the API / plugin management component (1142) may request the application / service component (1130) (e.g., the processor (120) of FIG. 1 or the processor (210) of FIG. 2) via the API when the application or service needs to perform an action that ultimately performs user input rather than an intermediate result. Information obtained from the outside may be used to generate a prompt in the prompt design component (1141) along with user input, or it may be passed as input to a generative artificial intelligence model (1160) (e.g., the generative artificial intelligence model (320) of FIG. 3 or a cloud artificial intelligence model).

[0128] An output modification component (or refiner component) (1143) (e.g., the processor (120) of FIG. 1 or the processor (210) of FIG. 2) can finely tune the output of a generative artificial intelligence model (1160) (e.g., the generative artificial intelligence model (320) of FIG. 3 or a cloud artificial intelligence model). For example, the output modification component (1143) can verify whether the content generated through LLM, LVM, and / or LMM is irrelevant, contains biased content, or contains harmful content. Additionally, the output modification component (1143) can determine the extent to which the output matches the desired result and, if additional processing is required, proceed with that process. Furthermore, the output modification component (1143) can configure and provide hints to the user to avoid unwanted output.

[0129] A generative AI model (1160) (e.g., the AI ​​model (320) of FIG. 3 or a cloud AI model) can generally refer to an artificial intelligence neural network that generates new forms of data based on user input information. A generative AI model (1160) may include a model that generates images and / or a model that generates language. Models that generate images include, but are not limited to, GANs (generative adversarial networks) and VAEs (variational autoencoders), and examples include diffusion-based generative models that use VAEs and transformer structures. Models that generate language are models trained to output the most statistically appropriate output value based on input values, and examples include models such as CHAT-GPT 3 and CHAT-GPT 4. Additionally, there are LMMs (large multimodal models) that can recognize various forms of data input, such as text, images, and voice, and generate new data corresponding to them.

[0130] In one embodiment, the artificial intelligence framework (1140) and / or generative artificial intelligence model (1160) may be included within an artificial intelligence module (e.g., including a processing circuit) within the electronic device. For example, the artificial intelligence module may be operatively coupled with at least one processor of the electronic device (e.g., at least one processor (120) of FIG. 1 or processor (210) of FIG. 2). For example, the artificial intelligence module may be operatively coupled with a sensor hub of the electronic device for one or more sensors within the electronic device.

[0131] According to one embodiment, a method of operation in an electronic device (e.g., the electronic device (101) of FIG. 1 or the electronic device (201) of FIG. 2) may include an operation of analyzing data related to user interaction using a first artificial intelligence model (e.g., the first artificial intelligence model (310) of FIG. 3) based on identifying interactions related to telephone.

[0132] According to one embodiment, the method may include the operation of obtaining message information related to the purpose of making a call with the recipient of the phone using a generative artificial intelligence model (e.g., the generative artificial intelligence model (320) of FIG. 3) based on the analysis results.

[0133] According to one embodiment, the method may include the operation of establishing a call connection with the recipient's external electronic device through the communication circuit of the electronic device (e.g., the communication module (190) of FIG. 1 or the communication circuit (250) of FIG. 2) using the generative artificial intelligence model.

[0134] According to one embodiment, the method may include the operation of transmitting a first message generated based on message information using the generative artificial intelligence model without user intervention to the external electronic device through the communication circuit.

[0135] According to one embodiment, the method may include the operation of transmitting to the external electronic device a message received from the external electronic device and a second message to be answered generated based on the message information using the generative artificial intelligence model without the intervention of the user.

[0136] According to one embodiment, the method may further include the operation of outputting notification information for a user confirmation request based on identifying by the generative artificial intelligence model that a response to a message received from the external electronic device is impossible or that confirmation by the user is required, and the operation of transmitting a third message input by the user to the external electronic device through the communication circuit.

[0137] According to one embodiment, the third message is a voice message or a text message, and the notification information may be a vibration or a ringtone.

[0138] According to one embodiment, the operation of executing the automatic phone function may include the operation of executing the automatic phone function using the first artificial intelligence model based on identifying information related to the execution of the automatic phone function in the analysis result by the first artificial intelligence model.

[0139] According to one embodiment, the method may further include an operation of generating the second message based on the received message and the message information, based on identifying that the correlation value between the received message and the message information is greater than a specified value using the generative artificial intelligence model.

[0140] According to one embodiment, the method may further include an operation of editing a part of the message information based on the user's editing request and an operation of displaying the edited message information through the display.

[0141] According to one embodiment, the method may further include the operation of obtaining editing recommendation information for a portion of the message information generated by the generative artificial intelligence model, the operation of displaying the editing recommendation information on the message information through the display, and the operation of transmitting the editing recommendation information to the external electronic device when transmitting the message information using the generative artificial intelligence model without user intervention.

[0142] According to one embodiment, the method may further include an operation of displaying an execution screen containing messages transmitted and received during the execution of the automatic telephone and message information through the display.

[0143] According to one embodiment, the method may further include: an operation of obtaining information requiring confirmation by the user generated based on messages transmitted or received during the execution of the automatic call, based on identifying that the execution of the automatic call has ended using the generative artificial intelligence model; an operation of storing the information requiring confirmation in the memory; and an operation of displaying call summary information set based on the transmitted or received messages and the information requiring confirmation through the display, based on identifying a request for confirmation of call content by the user.

[0144] According to one embodiment, the method may further include an operation of executing the function of the automatic call in the background without displaying the execution screen on the display while the automatic call is being executed, and an operation of displaying the execution screen through the display in response to a request from the user.

[0145] According to one embodiment, the method may further include the operation of obtaining a fourth message generated by the generative artificial intelligence model without user intervention, based on at least one of the personalized data stored in the memory or the collected data, based on identifying that the correlation value between the message received from the external electronic device and the message information is less than or equal to a specified value.

[0146] According to one embodiment, in a non-transient storage medium storing one or more programs, the one or more programs may include a command that, when executed by at least one processor (e.g., processor (120) of FIG. 1 or processor (210) of FIG. 2) of an electronic device (e.g., electronic device (101) of FIG. 1 or electronic device (201) of FIG. 2), causes the electronic device to execute an operation to analyze data related to user interaction using a first artificial intelligence model (e.g., first artificial intelligence model (310) of FIG. 3) based on identifying an interaction related to a telephone.

[0147] According to one embodiment, the program may include a command that, when executed by at least one processor of the electronic device, causes the electronic device to execute an operation to obtain message information related to the purpose of making a call with the recipient of the phone using a generative artificial intelligence model (e.g., generative artificial intelligence model (320) of FIG. 3) based on the analysis results.

[0148] According to one embodiment, the program may include a command that, when executed by at least one processor of the electronic device, causes the electronic device to execute an operation to establish a call connection with the recipient's external electronic device through the communication circuit of the electronic device (e.g., the communication module (190) of FIG. 1 or the communication circuit (250) of FIG. 2) using the generative artificial intelligence model.

[0149] According to one embodiment, the program may include a command that, when executed by at least one processor of an electronic device, causes the electronic device to perform an operation of transmitting a first message generated based on message information using the generative artificial intelligence model to the external electronic device through the communication circuit without user intervention.

[0150] According to one embodiment, the program may include a command that, when executed by at least one processor of an electronic device, causes the electronic device to execute an operation of transmitting to the external electronic device a second message to be answered, generated based on a message received from the external electronic device and the message information, using the generative artificial intelligence model without the intervention of the user.

[0151] An electronic device according to one embodiment of the present disclosure can, when making a call through an automatic telephone function, use an artificial intelligence model to identify the purpose of the call and automatically proceed with the conversation using the artificial intelligence model without user intervention until the user directly performs the call, and can provide a notification to the user to perform the call when user intervention is required. In addition, various effects that can be identified directly or indirectly through the present disclosure may be provided. The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure belongs from the description below.

[0152] Furthermore, the embodiments disclosed in this document are presented for the purpose of explaining and understanding the disclosed technical content and are not intended to limit the scope of the technology described in this document. Accordingly, the scope of this document should be interpreted to include all modifications or various other embodiments based on the technical concept of this document.

[0153] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0154] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0155] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0156] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0157] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TM It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0158] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (101, 201), Display(160, 230); Communication circuit (190, 250); At least one processor (120, 210); and It includes memory (130, 220) for storing instructions, When the above instructions are executed individually or collectively by the at least one processor, the electronic device: Based on identifying phone-related interactions, data related to user interactions is analyzed using the first artificial intelligence model (310), and Based on the analysis results, message information related to the purpose of talking to the recipient of the phone is generated using a generative artificial intelligence model (320), and Using the above generative artificial intelligence model, a call connection is established with the recipient's external electronic device through the above communication circuit, and A first message generated based on the message information using the generative artificial intelligence model without user intervention is transmitted to the external electronic device through the communication circuit, and An electronic device that transmits to the external electronic device a second message to be answered, generated based on a message received from the external electronic device and the message information, using the generative artificial intelligence model without the intervention of the user.

2. In paragraph 1, when the instructions are executed individually or collectively by the at least one processor, the electronic device: Based on identifying by the above generative artificial intelligence model that a response to a message received from the external electronic device is impossible or that confirmation by the user is required, notification information for a user confirmation request is output, and A third message input by the above user is transmitted to the external electronic device through the communication circuit, and The above third message is a voice message or a text message, and The above notification information is an electronic device, which is a vibration or a ring sound.

3. In paragraph 1 or 2, when the instructions are executed individually or collectively by the at least one processor, the electronic device: Based on identifying information related to the execution of an automatic phone function in the analysis result by the first artificial intelligence model, the automatic phone function is executed using the first artificial intelligence model, and An electronic device that generates a second message based on the received message and the message information, based on identifying that the correlation value between the received message and the message information is greater than a specified value using the generative artificial intelligence model.

4. In any one of claims 1 to 3, when the instructions are executed individually or collectively by the at least one processor, the electronic device: Based on identifying the user's edit request, a portion of the message information is edited, and An electronic device that displays the above-mentioned edited message information through the above-mentioned display.

5. In any one of claims 1 to 4, when the instructions are executed individually or collectively by the at least one processor, the electronic device: Obtaining editing recommendation information for a portion of the message information generated by the above generative artificial intelligence model, and The above message information is to display the above editing recommendation information through the above display, and An electronic device that transmits the editing recommendation information to the external electronic device when transmitting the message information using the generative artificial intelligence model without the intervention of the user.

6. In any one of claims 1 to 5, when the instructions are executed individually or collectively by the at least one processor, the electronic device: An execution screen including messages transmitted and received and message information transmitted during the execution of the above automatic telephone is displayed through the above display, and Based on identifying that the execution of the automatic phone call has ended using the above generative artificial intelligence model, information requiring confirmation from the user is obtained based on the transmitted or received messages during the execution of the automatic phone call, and The information requiring verification is stored in the memory, and An electronic device that, based on identifying a request to verify call content by the user, displays call summary information set based on the transmitted or received messages and information requiring verification through the display.

7. In any one of claims 1 through 6, when the instructions are executed individually or collectively by the at least one processor, the electronic device: While the above automatic call is being executed, the above execution screen is not displayed on the above display, and the above automatic call function is executed in the background, and An electronic device that displays the execution screen through the display in response to the request of the above user.

8. In any one of claims 1 through 7, when the instructions are executed individually or collectively by the at least one processor, the electronic device: An electronic device that obtains a fourth message generated by the generative artificial intelligence model without user intervention, based on at least one of the personalized data stored in the memory or the collected data, based on identifying that the correlation value between the message received from the external electronic device and the message information is less than or equal to a specified value.

9. A method of operation in an electronic device (101, 201), An operation to analyze data related to user interaction using a first artificial intelligence model (310) based on identifying interactions related to phone calls; Based on the analysis results, the operation of obtaining message information related to the purpose of communicating with the recipient of the phone using a generative artificial intelligence model (320); An operation of establishing a call connection with the recipient's external electronic device through the communication circuit (190, 250) of the electronic device using the generative artificial intelligence model; The operation of transmitting a first message generated based on the message information using the generative artificial intelligence model without user intervention to the external electronic device through the communication circuit; and An electronic device comprising the operation of transmitting to the external electronic device a second message to be answered, generated based on a message received from the external electronic device and the message information, using the generative artificial intelligence model without the intervention of the user.

10. In paragraph 9, the above method is, An operation of outputting notification information for a user confirmation request based on identifying by the generative artificial intelligence model that a response to a message received from the external electronic device is impossible or that confirmation by the user is required; and It further includes the operation of transmitting a third message input by the above user to the external electronic device through the communication circuit, and The above third message is a voice message or a text message, and The above notification information is a method that is a vibration or a ring sound.

11. In paragraph 9, the above method is, An operation of outputting notification information for a user confirmation request based on identifying by the generative artificial intelligence model that a response to a message received from the external electronic device is impossible or that confirmation by the user is required; and It further includes the operation of transmitting a third message input by the above user to the external electronic device through the communication circuit, and The above third message is a voice message or a text message, and The above notification information is a method that is a vibration or a ring sound.

12. In any one of paragraphs 9 to 11, the above method is, An operation to edit a part of the message information based on the user's editing request; An operation of displaying the above-mentioned edited message information through the display (160, 230) of the electronic device; An operation to obtain editing recommendation information for a portion of the message information generated by the above generative artificial intelligence model; The operation of displaying the above editing recommendation information through the above display in the above message information; and A method comprising further including the operation of transmitting the editing recommendation information to the external electronic device when transmitting the message information using the generative artificial intelligence model without the intervention of the user.

13. In any one of paragraphs 9 through 12, the above method is, An operation of displaying an execution screen through the display that includes messages transmitted and received and message information during the execution of the above automatic telephone; An operation of obtaining information requiring user confirmation generated based on the transmitted or received messages during the execution of the automatic call, based on identifying that the execution of the automatic call has ended using the generative artificial intelligence model above; The operation of storing the information requiring verification in the memory; and A method further comprising the operation of displaying call summary information and information requiring verification through the display based on the transmitted or received messages, based on identifying a request to verify call content by the user.

14. In any one of paragraphs 9 to 13, the above method is, An operation to execute the automatic call function in the background without displaying the execution screen on the display while executing the automatic call; An operation to display the execution screen through the display in response to the request of the above user; and A method further comprising the operation of obtaining a fourth message generated by the generative artificial intelligence model without user intervention, based on at least one of the personalized data stored in the memory or the collected data, based on identifying that the correlation value between the message received from the external electronic device and the message information is less than or equal to a specified value.

15. In a non-transient storage medium storing one or more programs, the program, when executed by at least one processor (120, 210) of an electronic device (101, 201), causes the electronic device: An operation to analyze data related to user interaction using a first artificial intelligence model (310) based on identifying interactions related to phone calls; Based on the analysis results, the operation of obtaining message information related to the purpose of communicating with the recipient of the phone using a generative artificial intelligence model (320); An operation of establishing a call connection with the recipient's external electronic device through the communication circuit (190, 250) of the electronic device using the generative artificial intelligence model; The operation of transmitting a first message generated based on the message information using the generative artificial intelligence model without user intervention to the external electronic device through the communication circuit; and A non-transient storage medium comprising commands that execute an operation to transmit to the external electronic device a second message to be answered, generated based on a message received from the external electronic device and the message information, using the generative artificial intelligence model without the intervention of the user.

Images