Electronic device for generating audio signal notifying input of text and method thereof

Abstract

A processor of an electronic device according to an embodiment may establish a call connection in response to a first input based on text-to-speech (TTS). The processor may display a screen including a first area capable of displaying at least one first text message corresponding to at least a portion of a first voice signal to be provided by the call connection. The processor may determine whether or not to transmit a second voice signal prior to completion of a second input, on the basis of a period of the second input indicating a second text message to be converted by the TTS on the basis of a second area of the screen. The processor may execute, on the basis of determining transmission of the second voice signal, a function for transmitting, through the call connection, the second voice signal notifying reception of the second input. The processor may display, on the basis of the execution of the function, a third text message corresponding to the second voice signal within the first area of the screen.

Description

Electronic device and method for generating an audio signal indicating the input of text

[0001] The present disclosure relates to an electronic device and method for generating an audio signal indicating the input of text.

[0002] The number of functions supported by electronic devices (e.g., smartphones) that support voice calls is increasing with technological advancements. Electronic devices may include a display for visualizing information. Electronic devices can visualize various information processed by the electronic device on the display.

[0003] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0004] According to an embodiment, an electronic device may include a display, a communication circuit, a memory storing instructions including one or more storage media, and at least one processor. The instructions, when executed by the at least one processor, may cause the electronic device to receive a first input for establishing a call connection based on a text-to-speech (TTS) based on a first screen for a call connection. The instructions, when executed by the at least one processor, may cause the electronic device to control the communication circuit in response to the first input to establish the call connection. The instructions, when executed by the at least one processor, may cause the electronic device to display a second screen including a first area capable of displaying at least one first text message corresponding to at least a portion of a first voice signal to be provided by the call connection on the display. The instructions, when executed by the at least one processor, may cause the electronic device to determine, based on a duration of the second input while receiving a second input indicating a second text message to be converted by the TTS based on a second area of ​​the second screen, whether to transmit a second voice signal prior to completion of the second input. The instructions, when executed by the at least one processor, may cause the electronic device to execute a function for transmitting, over the call connection, the second voice signal indicating receipt of the second input based on the determination to transmit the second voice signal.The instructions, when executed by the at least one processor, may cause the electronic device to display a third text message corresponding to the second voice signal within the first area of ​​the second screen based on execution of the function.

[0005] In one embodiment, a method of an electronic device including a display, a communication circuit, and a processor is provided. The method may include receiving, based on a first screen for a call connection, a first input for establishing the call connection based on a TTS. The method may include controlling, in response to the first input, the communication circuit to establish the call connection, and displaying, on the display, a second screen including a first area capable of displaying at least one first text message corresponding to at least a portion of a first voice signal to be provided by the call connection. The method may include receiving, based on a second area of ​​the second screen, a second input indicating a second text message to be converted by the TTS, while determining, based on a duration of the second input, whether to transmit a second voice signal prior to completion of the second input. The method may include executing, based on the determination of transmission of the second voice signal, a function for transmitting the second voice signal, which notifies receipt of the second input, through the call connection. The method may include an operation of displaying a third text message corresponding to the second voice signal within the first area of ​​the second screen based on the execution of the function.

[0006] In one embodiment, a non-transitory computer-readable storage medium storing instructions is provided. The instructions, when executed by a processor of an electronic device including a display, communication circuitry, and a processor, may cause the electronic device to receive a first input for establishing a call connection based on a text-to-speech (TTS) based on a first screen for a call connection. The instructions, when executed by the processor, may cause the electronic device to, in response to the first input, control the communication circuitry to establish the call connection and display a second screen on the display, the second screen including a first area capable of displaying at least one first text message corresponding to at least a portion of a first voice signal to be provided by the call connection. The instructions, when executed by the processor, may cause the electronic device to determine, based on a duration of a second input while receiving a second input indicating a second text message to be converted by the TTS based on the second area of ​​the second screen, whether to transmit a second voice signal prior to completing the second input. The instructions, when executed by the processor, may cause the electronic device to execute a function for transmitting the second voice signal via the call connection, the second voice signal indicating receipt of the second input, based on the determining of transmission of the second voice signal. The instructions, when executed by the processor, may cause the electronic device to display, based on the execution of the function, a third text message corresponding to the second voice signal within the first area of ​​the second screen.

[0007] FIG. 1 illustrates one embodiment of an electronic device that performs functions related to call connection.

[0008] FIG. 2 illustrates an example block diagram of an electronic device according to one embodiment.

[0009] FIG. 3 illustrates a flow diagram of an electronic device according to one embodiment.

[0010] FIG. 4 illustrates a flow diagram of an electronic device according to one embodiment.

[0011] FIG. 5 illustrates an example of a user interface (UI) displayed by an electronic device that has received a signal for establishing a call.

[0012] Figures 6a, 6b, 6c, 6d and 6e illustrate different states of an electronic device associated with a call connection.

[0013] Figure 7 illustrates an exemplary operation of an electronic device that recognizes utterances exchanged between users through a call connection.

[0014] FIG. 8 illustrates a flowchart of an electronic device according to one embodiment.

[0015] FIG. 9 is a block diagram of an electronic device within a network environment according to various embodiments.

[0016] Hereinafter, various embodiments of this document are described with reference to the attached drawings. Substantially identical components may have the same reference numbers in each of the attached drawings.

[0017] FIG. 1 illustrates an embodiment of an electronic device (101) that performs functions related to call connection. In terms of being owned by a user, the electronic device (101) may be referred to as a terminal (or user terminal). Terminals may include, for example, personal computers (PCs) such as laptops and desktops. Terminals may include, for example, smartphones, smartpads, and / or tablet PCs. Terminals may include smart accessories such as smartwatches and / or head-mounted devices (HMDs). Embodiments are not limited thereto, and terminals may include home appliances such as refrigerators, televisions, and air conditioners, and / or intelligent speakers.

[0018] In one embodiment, the electronic device (101) can wirelessly transmit or receive a signal for call connection and / or call establishment. Based on the signal, the electronic device (101) can perform a voice call and / or a video call. The electronic device (101) performing the voice call can wirelessly transmit a signal in which the user's speech of the electronic device (101) is recorded to a counterpart of the call connection (e.g., an external electronic device (120) different from the electronic device (101) and / or a user (122) of the external electronic device). The electronic device (101) performing the voice call can output the signal received through the call connection through one or more speakers. The one or more speakers can be included in the electronic device (101), included in earphones connected to the electronic device (101) by a wire, or included in earbuds connected wirelessly (e.g., paired) to the electronic device (101). For example, the electronic device (101) can activate a speaker and / or a microphone to support a voice call based on call establishment.

[0019] In one embodiment, the electronic device (101) may, instead of activating the speaker and / or microphone, utilize the display (110) to at least partially visualize the voice call. The functionality of visualizing information and / or signals associated with the voice call may be referred to as a text call. Referring to FIG. 1 , an exemplary state of the electronic device (101) is illustrated while connected to an external electronic device (120) based on a call connection, wherein the electronic device (101) at least partially visualizes a voice signal associated with the call connection. Within the exemplary state of FIG. 1 , the electronic device (101) and the external electronic device (120) associated with the call connection may be directly connected within a network, or indirectly connected via an access point (AP) and / or a router. For example, a server (130) that provides the functionality of at least partially visualizing the voice call may be connected to the electronic device (101) and / or the external electronic device (120).

[0020] Referring to FIG. 1, a screen of an electronic device (101) related to a text call is exemplarily illustrated. The operation of the electronic device (101) switching from another screen for confirming call establishment to the exemplary screen of FIG. 1 is described with reference to FIG. 5. The screen displayed on the display (110) may be divided into a first area (or region) (140), a second area (150), and a third area (160). Within the first area (140), the electronic device (101) may display information related to the call connection. For example, the electronic device (101) may display text corresponding to the other party of the call connection (e.g., a name and / or phone number based on contact information, such as "Mr. Sam") within the first area (140). The electronic device (101) may display visual objects (141, 142) for controlling the call connection within the first area (140). For example, through a visual object (141) in the form of a button, the electronic device (101) can receive an input to switch from a text call to a voice call based on a speaker and / or microphone. For example, through a visual object (142) in the form of a button, the electronic device (101) can receive an input to terminate the call connection.

[0021] Referring to FIG. 1, within the second area (150), the electronic device (101) may display at least one text message (e.g., text messages (151, 152)) corresponding to at least a portion of a voice signal associated with a call connection. Within the third area (160), the electronic device (101) may display a visual object for receiving an input for generating a voice signal to be provided to the other party of the call connection. For example, within the third area (160), the electronic device (101) may display a keyboard (e.g., a software keyboard and / or keypad) for inputting text. Within the third area (160), the electronic device (101) may display a text box (161) for displaying one or more characters combined by the keyboard. Within the third area (160), the electronic device (101) may display a visual object (162) having the form of a button. Through the visual object (162), the electronic device (101) can receive an input indicating completion of input of text based on the text box (161) and / or the keyboard. In response to the input indicating selection of the visual object (162), the electronic device (101) can execute a function for transmitting a voice signal corresponding to the text contained in the text box (161) through a call connection. The function can cause execution of TTS (text-to-speech) by the electronic device (101) and / or the server (130).

[0022] In the exemplary state of FIG. 1, a speech of a user (122) of an external electronic device (120) may be recognized by the external electronic device (120), the server (130), and / or the electronic device (101). For example, based on speech-to-text (STT), the electronic device (101) and / or the server (130) may obtain or generate a signal and / or information representing the speech of the user (122). For example, the external electronic device (120) may transmit an audio signal including the speech of the user (122) to the electronic device (101) and / or the server (130) through a call connection. The electronic device (101) that receives the audio signal may obtain a text (or one or more characters) corresponding to the speech of the user (122) included in the audio signal based on STT. Hereinafter, text may mean a binary code matched to a character based on a standard such as Unicode, ASCII (American Standard Code for Information Interchange), and / or EUC (Extended Unix Code). The text may include a set of binary codes corresponding to each of a plurality of characters. The text may be referred to as text information, text data, a character set, a character string, and / or a character stream. The hardware configuration and / or one or more programs of an electronic device (101) and / or a server (130) that perform operations related to STT are described with reference to FIG. 2.

[0023] Referring to FIG. 1, in an exemplary state where an electronic device (101) and an external electronic device (120) are connected to each other through a call connection, while a user (112) speaks a natural language such as “hello?”, the external electronic device (120) can obtain an audio signal corresponding to the natural language. The audio signal can be transmitted to the electronic device (101) and / or the server (130) through the call connection. The electronic device (101) and / or the server (130) can recognize the natural language by using characteristic information of the audio signal (e.g., information related to a waveform and / or frequency of the audio signal). The electronic device (101) and / or the server (130) can obtain or generate a text corresponding to the audio signal and / or the natural language based on the recognition of the audio signal.

[0024] Referring to FIG. 1, in an exemplary state where a call connection is established based on a text call, the electronic device (101) may visualize a text obtained based on the audio signal on the display (110). For example, the electronic device (101) may display a text message (151) including the text on the display (110). The text message (151) may have a location on the display (110) corresponding to a speaker of the text message (151) (e.g., a user (122) of an external electronic device (120). The text message (151) may have a form to be differentiated from other text messages (e.g., other utterances uttered at a different timing from the utterance corresponding to the text message (151). Based on the form, the text message (151) may be referred to as a speech bubble or bubble.

[0025] In one embodiment, while a call connection is established based on a text call, user utterances may be accumulated within the display (110) of the electronic device (101). The electronic device (101) may display text messages (151, 152) corresponding to each of the accumulated utterances using a user interface (UI) having a layout similar to a messenger application (e.g., a screen displayed on the display (110) of FIG. 1). For example, if a user (122) who has uttered “hello” additionally utters “are you there?”, the electronic device (101) and / or the server (130) may recognize text corresponding to the additional utterance uttered by the user (122). The electronic device (101) may display a text message (152) corresponding to the recognized text on the display (110). Text messages (151, 152) corresponding to utterances (e.g., “Hello?”, “Are you there?”) spoken by the user (122) may be displayed relative to one edge (e.g., the left edge) of the display (110) set to align the utterances of the other party of the call connection.

[0026] Referring to FIG. 1, in an exemplary state where a call connection based on a text call is established, the electronic device (101) may receive, through the third area (160), an input representing a text message to be converted by TTS. The input may include gestures (e.g., tap gestures) sequentially performed on a keyboard displayed in the third area (160). In order to mitigate silence of the user of the electronic device (101) (e.g., a case where only the user (122) of the external electronic device (120) speaks) that occurs while the electronic device (101) receives the input, the electronic device (101) may execute a function for transmitting a voice signal guiding the input to the other party of the call connection. The operation of the electronic device (101) for determining a condition for executing the function is described with reference to FIGS. 3 to 4 and / or 8.

[0027] For example, the electronic device (101) may execute a function for transmitting a voice signal (e.g., a voice signal including a designated natural language sentence such as “wait a second”) notifying the reception of the input through the call connection. The function may cause the execution of the TTS of the electronic device (101) and / or the server (130). Based on the function, while the user of the electronic device (101) inputs text (e.g., “I’m in a meeting, I received a nice gift. I’ll use it well, how’s the weather there?”) through the third area (160), the external electronic device (120) may output the voice signal. The user (122) who hears the voice signal may be relieved of anxiety caused by silence (e.g., anxiety caused by the thought that the call connection is not normal) and may wait for the completion of the input. Based on the execution of the above function, the operation of the electronic device (101) and / or server (130) that provides a voice signal to the other party of the call connection is described with reference to FIGS. 6a to 6e.

[0028] In one embodiment, in order to perform a text call based on TTS and / or STT, the electronic device (101) and / or the server (130) may run a mathematical model based on the nervous system of a living organism. The mathematical model may be referred to as a model, a neural network, a deep-learning model, and / or an artificial neural network (ANN). The model of the electronic device (101) and / or the server (130) may include software, hardware (e.g., a central processing unit (CPU), a graphic processing unit (GPU), and / or a neural processing unit (NPU)) or any combination thereof for performing the computations included in the model. Hereinafter, an exemplary configuration of the electronic device (101) and / or the server (130) including the mathematical model will be described with reference to FIG. 2.

[0029] FIG. 2 illustrates an example of a block diagram of an electronic device (101), according to one embodiment. The electronic device (101) of FIG. 1 may include hardware of the electronic device (101) described with reference to FIG. 2 . Referring to FIG. 2 , the electronic device (101) according to one embodiment may include at least one of a processor (210), a memory (215), a display (220), a microphone (225), a speaker (230), or a communication circuit (235). The processor (210), the memory (215), the display (220), the microphone (225), the speaker (230), and the communication circuit (235) may be electronically and / or operably coupled with each other by an electronic component, such as a communication bus (202). In one embodiment, the hardware components of the electronic device (101) being operatively coupled may mean that a direct connection, or an indirect connection, is established between the hardware components, either wired or wireless, such that a second hardware component is controlled by a first hardware component among the hardware components.

[0030] Although illustrated based on different blocks, the embodiment is not limited thereto, and some of the hardware of FIG. 2 (e.g., at least a portion of the processor (210), the memory (215), and / or the communication circuit (235)) may be included in a single integrated circuit such as a system on a chip (SoC). The type and / or number of hardware components included in the electronic device (101) is not limited to those illustrated in FIG. 2. For example, the electronic device (101) may include only some of the hardware components illustrated based on the blocks of FIG. 2.

[0031] According to one embodiment, the processor (210) of the electronic device (101) may include hardware and / or circuitry for processing data based on one or more instructions. The hardware and / or circuitry for processing data may include, for example, an arithmetic and logic unit (ALU), a floating point unit (FPU), a field programmable gate array (FPGA), a central processing unit (CPU), and / or an application processor (AP). The number of processors (210) may be one or more. For example, the processor (210) may have a multi-core processor structure such as a dual core, a quad core, a hexa core, or an octa core.

[0032] According to one embodiment, the memory (215) of the electronic device (101) may include hardware components for storing data and / or instructions input and / or output to the processor (210). The memory (215) may include, for example, volatile memory such as random-access memory (RAM) and / or non-volatile memory such as read-only memory (ROM). The volatile memory may include, for example, at least one of dynamic RAM (DRAM), static RAM (SRAM), cache RAM, and pseudo SRAM (PSRAM). The non-volatile memory may include, for example, at least one of programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, hard disk, compact disc, solid state drive (SSD), and embedded multi-media card (eMMC).

[0033] According to one embodiment, a display (220) of an electronic device (101) may output visualized information (e.g., screens of FIGS. 1, 5, 6A to 6E, and / or 7) to a user. For example, the display (220) may be controlled by a controller, such as a graphic processing unit (GPU) and / or a processor (210), to output visualized information to a user. The display (220) may include a liquid crystal display (LCD), a plasma display panel (PDP), and / or one or more light emitting diodes (LEDs). The LEDs may include organic LEDs (OLEDs). The display (220) may include electronic paper. The display (220) may have an at least partially curved shape and / or a deformable shape (e.g., a flexible display). Hereinafter, the active area (or active region) and / or displaying area (or displaying region) of the display (220) may correspond to at least a portion of the display (220) that includes one or more enabled or activated pixels. The display (220) of FIG. 2 may include the display (110) of FIG. 1.

[0034] According to one embodiment, the display (220) of the electronic device (101) may include a sensor (e.g., a touch sensor panel (TSP)) for detecting an external object (e.g., a user's finger) on the display (220). For example, based on the TSP, the electronic device (101) may detect an external object (e.g., a user's finger) that is in contact with the display (220) or floating on the display (220). In response to detecting the external object, the electronic device (101) may execute a function associated with a specific visual object corresponding to a location of the external object on the display (220) among visual objects displayed within the display (220).

[0035] According to one embodiment, the electronic device (101) may include a microphone (225) that outputs an electrical signal indicating vibration of the atmosphere. For example, the microphone (225) of the electronic device (101) may output an audio signal including a user's speech. The user's speech included in the audio signal may be converted into information in a format recognizable by the electronic device (101) (e.g., the processor (210)) based on a speech recognition model and / or a natural language understanding model. For example, the electronic device (101) may recognize the user's speech and execute one or more functions among a plurality of functions supported by the electronic device (101).

[0036] An electronic device (101) according to one embodiment may include an output means for outputting information in a form other than a visualized form. For example, the electronic device (101) may include a speaker (230) for outputting an audio signal (or, an acoustic signal). The number of speakers (230) included in the electronic device (101) may be one or more. Although not illustrated, the electronic device (101) may include another output means for outputting information in a form other than a visual or auditory form. For example, the electronic device (101) may include a motor for providing haptic feedback based on vibration.

[0037] A communication circuit (235) of an electronic device (101) according to one embodiment may include hardware (or circuitry) for supporting transmission and / or reception of electrical signals between the electronic device (101) and an external electronic device (e.g., the external electronic device (120) of FIG. 1 and / or the server (250) of FIG. 2). The communication circuit (235) may include, for example, at least one of a modem (MODEM), an antenna, and an optical / electronic (O / E) converter. The communication circuit (235) may support transmission and / or reception of electrical signals based on various types of protocols, such as Ethernet, a local area network (LAN), a wide area network (WAN), wireless fidelity (WiFi), near field communication (NFC), Bluetooth, Bluetooth low energy (BLE), ZigBee, long term evolution (LTE), fifth generation (5G), new radio (NR), sixth generation (6G), and / or above-6G.

[0038] According to one embodiment, the processor (210) of the electronic device (101) may control the communication circuit (235) to perform operations related to a call connection. The operations related to the call connection may include operations of transmitting or receiving a signal for establishing a call. The operations related to the call connection may include operations of transmitting or receiving an audio signal (or audio stream) based on the establishment of the call connection. The call connection may be established for a voice call based on voice over LTE (VoLTE) and / or voice over NR (VoNR). The embodiment is not limited thereto, and the call connection may be established for a video call based on video streaming through a network such as the Internet. While the call connection is established, the electronic device (101) may transmit an audio signal acquired through the microphone (225) to the other party of the call connection through the communication circuit (235). While a call connection is established, the electronic device (101) may output an audio signal received through the communication circuit (235) through the speaker (230). The embodiment is not limited thereto, and the electronic device (101) may support an operation (e.g., a text call) of performing a call connection based on a screen displayed on the display (220) (e.g., a screen illustrated with reference to FIG. 1).

[0039] Referring to FIG. 2, exemplary hardware included in a server (250) connected to an electronic device (101) via a communication circuit (235) is illustrated. The server (250) may include a processor (260), a memory (265), and / or a communication circuit (270). The processor (260), the memory (265), and the communication circuit (270) are illustrated as being electrically and / or operatively connected to each other by a communication bus (204), but the embodiment is not limited thereto. The server (250) of FIG. 2 may include the server (130) of FIG. 1. At least a portion of the hardware included in the server (250) of FIG. 2 may operate substantially similarly to at least a portion of the hardware included in the electronic device (101) of FIG. 1. For example, the processor (260), the memory (265), and / or the communication circuit (270) of the server (250) may operate substantially similarly to the processor (210), the memory (215), and / or the communication circuit (235) of the electronic device (101), respectively. To reduce repetition of description, descriptions of the processor (260), the memory (265), and / or the communication circuit (270) of the server (250) that overlap with the descriptions of the processor (210), the memory (215), and / or the communication circuit (235) of the electronic device (101) may be omitted.

[0040] According to one embodiment, within the memory (215) of the electronic device (101), one or more instructions (or commands) representing operations and / or actions to be performed by the processor (210) on data may be stored. Similarly, within the memory (265) of the server (250), one or more instructions (or commands) representing operations and / or actions to be executed by the processor (260) may be stored. A set of the one or more instructions stored in any one of the memories (215, 265) may be referred to as firmware, an operating system, a process, a routine, a sub-routine, and / or a software application (hereinafter, an application).

[0041] For example, the electronic device (101) and / or the processor (210) may perform at least one of the operations of FIGS. 3 to 4 and / or 8 when a set of a plurality of instructions distributed in the form of an operating system, firmware, driver, and / or application is executed. Hereinafter, the fact that an application is installed in the electronic device (101) may mean that one or more instructions provided in the form of an application are stored in the memory (215) of the electronic device (101), and that the one or more applications are stored in a format executable by the processor (210) of the electronic device (101) (e.g., a file having an extension designated by the operating system of the electronic device (101). Similarly, one or more instructions included in the application may be stored in the memory (265).

[0042] Referring to FIG. 2, programs stored in the memories (215, 265) of the electronic device (101) and / or the server (250) are illustrated as different blocks. In one embodiment of FIG. 2, a call application (240), a keyboard application (242), a TTS application (246), and / or an automatic speech recognition (ASR) client application (248) may be installed in the memory (215) of the electronic device (101). A response determiner (282) (or decision maker), an event detector (284), a generative language model (286), and / or an ASR server application (280) may be installed in the memory (265) of the server (250).

[0043] In one embodiment, based on the execution of the call application (240), the processor (210) of the electronic device (101) may display a screen related to call connection and / or call establishment. While the call application (240) is running, the processor (210) may control the display (220) to display the screen, and may control the communication circuit (235) to transmit or receive a signal related to the call connection and / or call establishment. The processor (210) may acquire or output an audio signal by activating at least one of the microphone (225) and / or the speaker (230). The screens of FIGS. 1, 5, 6A to 6E, and / or 7 may be displayed based on the execution of the call application (240).

[0044] In one embodiment, based on the execution of the keyboard application (242), the processor (210) of the electronic device (101) can visualize a keyboard organized by graphical objects on at least a portion of the display (220). In response to an event detected by a specific application (e.g., a call application (240)), the processor (210) can execute the keyboard application (242). The event can be generated by a request (of a user and / or of the specific application) to display a keyboard for entering text and / or a call or invoke of an application programming interface (API) corresponding to the request. Based on the execution of the keyboard application (242), the processor (210) can display a keyboard occupying at least a portion of the display area of ​​the display (220). The keyboard can be displayed as an overlay on a screen provided by the specific application in which the event occurred. In response to a touch input performed on at least a portion of the display area occupied by the keyboard application (242), the processor (210) may provide text comprising one or more characters to the particular application.

[0045] In one embodiment, based on the execution of the TTS application (246), the processor (210) of the electronic device (101) may generate or synthesize a voice signal including an utterance corresponding to text that was input based on the execution of the keyboard application (242). The TTS application (246) may include a model for generating a voice signal. The processor (210) may obtain a voice signal corresponding to the text from the model in which the text is input. The TTS application (246) may provide an API for returning a voice signal included in a request from another application (e.g., a call application (240)).

[0046] In one embodiment, based on the execution of the ASR client application (248), the processor (210) of the electronic device (101) may process an audio signal acquired via the microphone (225). For example, the processor (210) may detect a natural language-based utterance from the audio signal. If the utterance indicates control (e.g., execution and / or termination) of a specific function of the electronic device (101), the processor (210) may perform an operation related to the specific function based on the execution of the ASR client application (248). The processor (210) may communicate with the server (250) based on the execution of the ASR client application (248) to cause an operation of the electronic device (101) and / or the server (250) corresponding to the user's intention included in the utterance.

[0047] In one embodiment, based on the execution of the ASR server application (280), the processor (260) of the server (250) can provide a service based on voice recognition to an external electronic device (e.g., a client device including the electronic device (101)) connected to the server (250) via the communication circuit (270). For example, a signal transmitted from the electronic device (101) based on the execution of the ASR client application (248) can be processed by the processor (260) executing the ASR server application (280). The ASR server application (280) can be executed by the processor (260) of the server (250) to support functions different from those that can be processed by the ASR client application (248).

[0048] In one embodiment, based on the execution of the event detector (284), the processor (260) of the server (250) may detect an event of the electronic device (101) related to a text call. The event may include an input performed by a user of the electronic device (101) to input text. The event may be related to a call connection established between the electronic device (101) and an external electronic device (e.g., the external electronic device (120) of FIG. 1) by a call application (240). The event may be related to a user's utterance of the external electronic device, identified from an audio signal transmitted from the external electronic device to the electronic device (101) by the call connection.

[0049] In one embodiment, based on the execution of the response determiner (282), the processor (260) of the server (250) may determine whether to transmit a voice signal to an external electronic device connected to the electronic device (101) based on the call connection to notify an action of a user of the electronic device (101) (e.g., an action of entering text). For example, based on the execution of the response determiner (282), the processor (260) of the server (250) may determine whether to transmit a voice signal including a utterance guiding the maintenance of the call connection to the user of the external electronic device (e.g., the user (122) of FIG. 1). Based on the execution of the response determiner (282), the processor (260) of the server (250) may analyze the situations of users conversing through the call connection. Based on the above analysis, to reduce silence caused by text calls, the processor (260) may generate or recommend a speech signal and / or a voice signal including the speech on behalf of the user of the electronic device (101) making the text call.

[0050] In one embodiment, based on determining whether to transmit a voice signal, the processor (260) of the server (250) may generate or synthesize the voice signal by driving a generative language model (286). The text for generating the voice signal may be provided based on the execution of the response determiner (282), or may be selected from a set of designated texts. Based on the voice signal output from the generative language model (286) into which the text is input, the processor (260) of the server (250) may transmit the voice signal to the other party of the call connection through the communication circuit (270).

[0051] In one embodiment, the generative language model (286) may include a neural network (e.g., a transformer model such as bidirectional encoder representations from transformers (BERT) and / or a recurrent neural network (RNN)) for generating sentences based on natural language and / or speech signals for the sentences. The generative language model (286) may include a large language model such as a generative pre-trained transformer (GPT). Based on the operation of the generative language model (286), the processor (260) of the server (250) may synthesize a speech signal from the input natural language based on unit data such as tokens. A model for synthesizing a speech signal, such as the generative language model (286), may be referred to as an agent.

[0052] In one embodiment, the processor (260) may adjust the voice characteristics (e.g., frequency, rate, and / or amplitude) of a speech signal to be synthesized from natural language based on the generative language model (286). For example, the processor (260) may control the generative language model (286) based on a first voice characteristic of a designated speaker to synthesize a speech signal including utterances similar to those uttered by the designated speaker. The embodiment is not limited thereto, and the processor (260) may control the generative language model (286) based on a second voice characteristic of a user of the electronic device (101) to synthesize a speech signal including utterances similar to those uttered by the user.

[0053] Although an embodiment is illustrated in which a call application (240), a keyboard application (242), a TTS application (246), and an ASR client application (248) are installed in the memory (215) of the electronic device (101), and a response determiner (282), an event detector (284), a generative language model (286), and an ASR server application (280) are installed in the memory (265) of the server (250), the embodiment is not limited thereto. For example, programs illustrated as being divided into different blocks in the memories (215, 265) of FIG. 2 may be integrated in the memory (215) of the electronic device (101).

[0054] The programs depicted as being divided into different blocks within the memories (215, 265) of FIG. 2 may be executed by distinct models or by an integrated model (e.g., a large language model). The model driven by the event detector (284) to recognize the context of a call connection and / or the model driven by the response determiner (282) to synthesize a speech signal may be different or substantially the same. The model and / or neural network may represent interconnected nodes for transmitting signals and / or information, and computations based on the nodes. The nodes may be interconnected by pre-tuned weights. The connections between the nodes may have a structure (or topology) based on a recurrent neural network (RNN), a feed-forward neural network (FNN), a convolutional neural network (CNN), and / or a long-short term memory (LSTM).

[0055] As described above, according to one embodiment, the electronic device (101) and / or the server (250) may include hardware and / or one or more programs for text calling. While a call connection based on a text call is established, the electronic device (101) and / or the server (250) may obtain a text to be transmitted through the call connection from the user of the electronic device (101). While obtaining the text, the electronic device (101) and / or the server (250) may provide a speech and / or voice signal to the other party of the call connection (e.g., an external electronic device connected to the electronic device (101) via the communication circuit (235)) indicating a refrain from interrupting the call connection.

[0056] Hereinafter, with reference to FIG. 3 and / or FIG. 4, the operation of an electronic device (101) and / or a server (250) that guides the other party to maintain a call connection is described.

[0057] FIG. 3 illustrates a flow diagram of an electronic device according to one embodiment. The electronic device (101) of FIG. 1 and / or FIG. 2 may include the electronic device of FIG. 3.

[0058] In the embodiments below, the operations may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. For example, the electronic device (101) of FIGS. 1 and 2 and / or the processor (210) of FIG. 2 may perform at least one of the operations of FIG. 3.

[0059] Referring to FIG. 3, in operation (310), according to one embodiment, a processor of an electronic device may establish a call connection based on text-to-speech (TTS). The processor of the electronic device, which receives a signal for call establishment and / or call setup through a communication circuit (e.g., the communication circuit (235) of FIG. 2), may display a first screen for establishing a call connection. Based on the first screen for the call connection, the processor may receive a first input for establishing a call connection based on TTS. In response to the first input, the processor may establish the call connection in operation (310). For example, the processor may control the communication circuit to establish the call connection.

[0060] In one embodiment, in response to the first input, the processor may display a second screen on a display (e.g., display (110) of FIG. 1 and / or display (220) of FIG. 2), the second screen being different from the first screen. The second screen may include a first area (e.g., second area (150) of FIG. 1) capable of displaying at least one first text message corresponding to at least a portion of a first voice signal to be provided by the call connection, such as the screen exemplarily illustrated in FIG. 1. The processor may display, on the display, the second screen including a visual object for executing TTS (e.g., visual object (162) of FIG. 1). The processor may display, on the display, the second screen including a second area (e.g., third area (160) of FIG. 1) for obtaining text to be transmitted via the call connection based on the TTS.

[0061] According to one embodiment, the processor of the electronic device may receive a second input related to a voice signal to be provided to the other party of the call connection through the second screen. The text received based on the second input may be completed based on a third input indicating selection of a visual object included in the second screen. In response to the third input indicating selection of the visual object, the processor of the electronic device may execute a function for transmitting a voice signal representing a text message including the text input based on the second input through the call connection. Execution of the function may cause the electronic device and / or a server (e.g., server 130 of FIG. 1 and / or server 250 of FIG. 2) to perform TTS. Execution of the function may cause the processor of the electronic device to display the text message on a display. Within the time period during which the second input is received, the other party of the call connection (e.g., external electronic device 120 and / or user 122 of FIG. 1) may wait for transmission of a voice signal from the electronic device. The above time interval may be referred to as a waiting time.

[0062] Referring to FIG. 3 , in operation (320), according to one embodiment, a processor of an electronic device may determine whether a wait time associated with a call connection exceeds a threshold (e.g., about 3 seconds and / or about 5 seconds). The threshold of operation (320) may be determined heuristically based on a user's tolerance for text calls. The threshold of operation (320) may be determined based on call logs collected by a communication service associated with the text call. The threshold of operation (320) may be personalized to the other party of the call connection based on profile information of the other party. The profile information may be related to contact information of the electronic device. The profile information may include a threshold of operation (320) determined based on at least one of a user's gender, age, and / or a call log associated with the call connection.

[0063] Referring to FIG. 3, if the waiting time is less than or equal to a threshold (320 - No), the processor may not perform other operations of FIG. 3 (e.g., operations (330, 340, 350, 360)). If the waiting time exceeds the threshold (320 - Yes), the processor may perform operation (330). For example, based on a duration of the second input, the processor may determine whether to transmit a voice signal before completing the second input. If the duration of the second input (e.g., the waiting time) exceeds the threshold, the processor may determine to transmit the voice signal. Based on determining to transmit the voice signal, the processor may execute a function for transmitting the voice signal, which notifies the reception of the second input, via a call connection. For example, based on the duration exceeding a designated duration (e.g., the threshold of operation (320)), the processor may initiate execution of the function for transmitting the voice signal.

[0064] Referring to FIG. 3, in operation (330), according to one embodiment, a processor of an electronic device may obtain information related to text input from a user. The information may include a period of time during which the text was input based on a second input. The information may include the number of natural language sentences included in the text. The information may include profile information about a counterparty to whom the text is to be transmitted (e.g., a counterparty of a call connection). The information may include a level of completion of the natural language sentences included in the text. To calculate the number and / or level of completion of natural language sentences, the processor may run a model related to NLP. For example, at least a portion of the information of operation (330) may be generated based on the running of the NLP model.

[0065] Referring to FIG. 3 , in operation (340), according to one embodiment, a processor of an electronic device may determine, based on information obtained by performing operation (330), whether to transmit a voice signal to the other party of a call connection before completion of a text. The processor may determine whether to transmit the voice signal of operation (340) by comparing numerical values ​​and / or parameters included in the information of operation (330) with a corresponding threshold. The processor may generate a flag, a variable, and / or a parameter indicating whether to transmit the voice signal of operation (340). An example of an operation for determining whether to transmit the voice signal of operation (340) is described with reference to FIG. 4 . If it is determined not to transmit a voice signal, the processor may not transmit any voice signal. For example, the processor may refrain from executing a function for synthesizing a voice signal until a third input indicating completion of the text input by the second input. If it is decided to transmit a voice signal, the processor may perform action (350).

[0066] Referring to FIG. 3, in operation (350), according to one embodiment, a processor of an electronic device may execute a function for transmitting a voice signal to a call partner based on whether to transmit a voice signal before the completion of a text. The function may include an operation for determining text to be included in the voice signal. For example, the processor may execute the function based on a specified text to obtain or generate a voice signal (or data for outputting the voice signal) corresponding to the specified text. The specified text may be referred to as a filler word (or filler sentence). For example, the processor may execute the function based on at least a portion of a text obtained from a user by the second input. For example, the processor may determine the text based on profile information of the call partner.

[0067] In one embodiment, the processor may execute the function of operation (350) using a model of an electronic device and / or a server. For example, the processor may execute the function of operation (350) using a model of a TTS application (246) of FIG. 2. In the example, the processor may obtain or generate the speech signal of operation (350) and / or data for outputting the speech signal without a server. In one embodiment, the processor may request generation of the speech signal of operation (350) from a server for TTS (e.g., server (250) of FIG. 2). The processor of the server (e.g., processor (260) of FIG. 2) may, in response to the request, obtain or generate the speech signal and / or data for outputting the speech signal from the generative language model (286) of FIG. 2.

[0068] In one embodiment, a processor executing the function of operation (350) may transmit, to the other party of a call connection, a voice signal obtained based on the function and / or data for outputting the voice signal. The voice signal may include an electrical signal to be transmitted to a speaker (e.g., a speaker of an external electronic device) and / or data in a digital format (e.g., a digital format such as mp3) for generating the electrical signal. Based on operation (350), a voice signal may be transmitted from an electronic device and / or a server to an external electronic device (e.g., the external electronic device (120) of FIG. 1), which is the other party of the call connection. The external electronic device, which receives the voice signal, may output the received voice signal to guide a user of the external electronic device (e.g., the user (122) of FIG. 1) to input text based on a second input.

[0069] In one embodiment, a processor executing the function of operation (350) may guide the user of the electronic device to provide a voice signal based on the function. For example, based on the execution of the function, the processor may display a text message corresponding to the voice signal provided to the other party based on operation (350) within the first area of ​​the second screen displayed based on operation (310).

[0070] Referring to FIG. 3, while a call connection based on a text call is maintained, the processor of the electronic device may additionally perform operations (320, 330, 340, 350) of FIG. 3. While the call connection is maintained, the processor may generate log information including one or more text messages transmitted or received based on the call connection. In operation (360), according to one embodiment, the processor of the electronic device may update at least one model associated with the generation of a voice signal based on the log information associated with the call connection.

[0071] In one embodiment, the model of operation (360) may include a model used to execute the function of operation (350) (e.g., a model installed on the electronic device based on the TTS application (246) of FIG. 2 and / or a generative language model (286) of the server (250). In one embodiment, the model of operation (360) may include a model that determines a context related to a call connection to determine whether to transmit the speech signal of operation (340) (e.g., a model installed on the server (250) for driving the event detector (284) of FIG. 2). Based on operation (360), a processor of the electronic device and / or the server (e.g., at least one of the processors (210, 260) of FIG. 2) may change or tune at least one parameter (e.g., a weight included in the model) associated with the at least one model so that the at least one model synthesizes the speech signal of operation (350) more naturally.

[0072] Hereinafter, with reference to FIG. 4, exemplary operations of an electronic device for comparing one or more conditions and / or checking one or more conditions to perform an operation (340) are described.

[0073] FIG. 4 illustrates a flow diagram of an electronic device according to one embodiment. The electronic device (101) of FIG. 1 and / or FIG. 2 may include the electronic device of FIG. 4.

[0074] In the embodiments below, the operations may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. For example, the electronic device (101) of FIGS. 1 and 2 and / or the processor (210) of FIG. 2 may perform at least one of the operations of FIG. 4. The embodiment is not limited thereto, and the operations of FIG. 4 may be performed by the server (250) and / or the processor (260) executing the event detector (284) of FIG. 2. The operations of FIG. 4 may be related to or included in operation (340) of FIG. 3.

[0075] Referring to FIG. 4 , in operation (410), according to one embodiment, the processor may determine whether a waiting time exceeds a reference time. The waiting time of operation (410) may include a time period during which no voice signal is provided to the other party of the call connection related to the text call while the electronic device receives a text based on the text call. The reference time may be set to a time of several seconds, including about 3 seconds. The reference time may be adaptively changed or set based on a context related to the call connection (e.g., at least one of a text received by the electronic device, a speech received by the other party, an external electronic device, and / or profile information corresponding to the other party of the call connection), similar to the threshold of FIG. 3 . If the waiting time of operation (410) exceeds the reference time (410-Yes), the processor may perform operation (465). If the waiting time is less than the reference time (410-No), the processor may additionally check a condition different from the condition of operation (410) to determine whether to execute the function of operation (350) of FIG. 3.

[0076] Referring to FIG. 4, within operation (420), according to one embodiment, the processor may determine whether a keyboard-based user input has been received. The keyboard of operation (420) may be displayed on a display of the electronic device (e.g., display (110) of FIG. 1 and / or display (220) of FIG. 2) based on execution of the keyboard application (242) of FIG. 2. The user input of operation (420) may include one or more gestures performed on the keyboard to compose text to be transmitted from the electronic device to the other party of a call connection based on a text call. Within a state of receiving the user input of operation (420-Yes), the processor may perform operation (465). If the user input of operation (420) is not received, or before receiving the user input of operation (420) (420-No), the processor may additionally check a condition different from the condition of operation (420) to determine whether to transmit a voice signal indicating the user input.

[0077] Referring to FIG. 4, in operation (430), according to one embodiment, the processor of the electronic device may determine whether a first natural language sentence has been completed by a user input. For example, the processor may determine whether at least one natural language sentence has been completed based on the user input of operation (420). The processor may determine the number of completed sentences in the text obtained from the user by the user input. If the number increases, the processor may determine whether the first natural language sentence has been completed in operation (430). The operation of determining whether the first natural language sentence has been completed is not limited to the operations described above. For example, the processor may determine whether the first natural language sentence has been completed based on a user input indicating selection of an emoticon or input of an emoticon. For example, based on the convention of combining an emoticon after a natural language sentence, the processor may determine whether the first natural language sentence has been completed based on a user input for inputting the emoticon. In one embodiment, an emoticon-based TTS may include an action of performing TTS on an utterance to describe the shape of the emoticon.

[0078] Referring to FIG. 4, if the first natural language sentence of operation (430) is completed (430-Yes), the processor can perform operation (465). Before the first natural language sentence of operation (430) is completed (430-No), the processor can additionally check a different condition from operation (430) to determine whether to transmit a voice signal notifying a state of receiving natural language from a user.

[0079] Referring to FIG. 4, in operation (440), according to one embodiment, a processor of an electronic device may determine whether a user input for a second natural language sentence following a first natural language sentence has been initiated. For example, the processor may detect a user input for inputting a second natural language sentence next to the first natural language sentence based on operation (440). The user input for inputting the second natural language sentence may indicate completion of the first natural language sentence. If the user input for the second natural language sentence of operation (440) has been initiated (440-Yes), the processor may perform operation (465). Before the second natural language sentence of operation (440) is input (440-No), the processor may additionally determine a different condition from operation (440) to determine whether to transmit a voice signal for guiding a state of a user inputting a natural language sentence.

[0080] Referring to FIG. 4, in operation (450), according to one embodiment, a processor of an electronic device may determine whether a natural language sentence corresponding to a voice signal provided through a call connection has been changed by a user input. For example, the processor may perform at least one of the operations of FIG. 3 to provide a portion of a text received by a user input to the other party of the call connection based on TTS before receiving another input indicating transmission of the text based on TTS. In the example, the processor may detect a user input for changing the portion of the text provided to the other party (e.g., the natural language sentence of operation (450)). If the natural language sentence of operation (450) is changed by the user input (450 - Yes), the processor may perform operation (465). If the natural language sentence of operation (450) is not changed by the user input (450 - No), the processor may perform operation (460).

[0081] Referring to FIG. 4, if none of the conditions of operations (410, 420, 430, 440, 450) are satisfied, the processor may perform operation (460). In operation (460), the processor of the electronic device according to one embodiment may receive a user input for entering text independently of a voice signal. The processor may not perform any function for transmitting the voice signal. Based on operation (460), the processor may maintain the state of the electronic device receiving the user input. After performing operation (460), the processor may repeatedly perform operations (410, 420, 430, 440, 450) to determine whether to provide a voice signal for maintaining a call connection to the other party independently of the user input of operation (460).

[0082] Referring to FIG. 4, if at least one of the conditions of operations (410, 420, 430, 440, 450) is satisfied, the processor may perform at least one of operations (465, 470). In operation (465), the processor of the electronic device according to one embodiment may determine whether a user input for transmitting a voice signal has been received. For example, the processor may display a visual object indicating whether to transmit the voice signal prior to completion of text that is updated based on the user input, or may output a designated audio signal (e.g., “I will notify the other party that text has been entered”) that guides transmission of the voice signal. In the example, if a user input related to the visual object is received, or an utterance instructing transmission of the voice signal is identified (465-Yes), the processor may perform operation (470). If the user input of operation (465) is not received, or if another user input is received that stops transmitting the voice signal (465-No), the processor may perform operation (460). Referring to FIG. 4, an embodiment in which operations (465, 470) are performed sequentially is illustrated, but the embodiment is not limited thereto. For example, if at least one of the conditions of operations (410, 420, 430, 440, 450) is satisfied, the processor may perform operation (470) independently of the user input of operation (465).

[0083] Referring to FIG. 4, in operation (470), according to one embodiment, the processor of the electronic device may determine to transmit a speech signal prior to completion of the text. In one embodiment, based on the determination of operation (470), the processor may perform operation (350) of FIG. 3. For example, the processor may determine to synthesize and / or transmit the speech signal of operation (470) to mitigate the probability of a call connection being interrupted due to a delay caused by the input of the text.

[0084] In one embodiment, the speech signal of operation (470) may include utterances, such as uttering at least a portion of text obtained from a user of the electronic device. The embodiment is not limited thereto, and the utterances included in the speech signal of operation (470) may include natural language related to profile information and / or contact information corresponding to the other party of the call connection. The utterances included in the speech signal of operation (470) may include at least one of the filler words listed in Table 1.

[0085] Number Filler Words 1 Well (or hmm) 2 You know 3 Actually 4 So 5 That's it 6 Wait 7 Please wait 8 I'm thinking about how to say it 9 My answer to your question is 10 Please don't interrupt me

[0086] In one embodiment, the filler words that the electronic device can select are not limited to Table 1. The electronic device can select a filler word to be expressed by a voice signal from among the filler words in Table 1 based on a relationship (e.g., a relationship indicated by profile information and / or contact information) between a first user of the electronic device and a second user of an external electronic device that is a call connection counterpart. For example, if the age of the second user is higher than that of the first user, the processor can perform TTS for filler word number 7 (e.g., “please wait”) from among the filler words in Table 1 to generate a voice signal to be transmitted to the call connection counterpart. For example, if the age of the second user is lower than that of the first user, the processor can perform TTS based on filler word number 6 (e.g., “wait”) from among the filler words in Table 1 to obtain a voice signal expressing an utterance that includes the 6th filler word.

[0087] In one embodiment, the utterance to be expressed by the voice signal of operation (470) may be determined based on the context of the call connection. For example, if it is determined to transmit the voice signal based on operation (440), the processor may determine the natural language to be included in the voice signal to be provided to the other party of the call connection based on the content (or context) of the second natural language sentence and / or the first natural language sentence of operation (440).

[0088] As described above, according to one embodiment, the processor of the electronic device may generate a voice signal to be provided to an external electronic device connected to the electronic device through a call connection, based on the operations described with reference to FIGS. 3 and 4. The voice signal may be automatically generated and / or transmitted by the electronic device and / or a server for TTS without an explicit input for completing a text being input into the electronic device by a user of the electronic device and / or transmitting the text to the external electronic device based on TTS. Hereinafter, exemplary operations of the electronic device based on an exemplary call connection will be described with reference to FIGS. 5 and 6A to 6E.

[0089] FIG. 5 illustrates an example of a user interface (UI) displayed by an electronic device (101) that has received a signal for establishing a call. The electronic device (101) of FIG. 1 and / or FIG. 2 may include the electronic device (101) of FIG. 5. For example, the electronic device (101) of FIG. 1 and FIG. 2 and / or the processor (210) of FIG. 2 may perform the operations of the electronic device (101) described with reference to FIG. 5. The operations of the electronic device (101) of FIG. 5 may be related to at least one of the operations of FIG. 3 and / or FIG. 4.

[0090] Referring to FIG. 5, exemplary states (501, 502, 503, 504) of an electronic device (101) displaying a screen for establishing a call are illustrated. Within state (501), the electronic device (101) that has received a signal for establishing a call may display a screen provided from the call application (240) of FIG. 2. The screen displayed on the display (110) within state (501) may be displayed to notify of an incoming call. Within state (501), the electronic device (101) may display text indicating the other party of the incoming call (e.g., a name identified through contact information, such as “Anh”) and / or a phone number uniquely assigned to the other party within the screen displayed on the display (110).

[0091] Within the exemplary state (501) of FIG. 5, the electronic device (101) may display visual objects (511, 512, 513) corresponding to different operations for controlling call establishment, respectively. In response to an input related to the visual object (511), which includes a color (e.g., red), shape, and / or icon for guiding the termination of the call, the electronic device (101) may terminate the call establishment (or call setup). In response to an input to the visual object (511), the electronic device (101) may execute a function for refusing an incoming call. The electronic device (101) executing the function may replace the screen displayed on the display (110) with another screen (e.g., a home screen and / or a launcher screen) based on the state (501).

[0092] Within the exemplary state (501) of FIG. 5, the electronic device (101) may display a visual object (512) including a color (e.g., green), shape, and / or icon to guide the establishment of a call. For example, the electronic device (101) may display a screen including a visual object (512) for establishing a voice call. In response to an input related to the visual object (512), the electronic device (101) may switch from state (501) to state (502). Based on the input, the electronic device (101) may activate the microphone (225) and / or the speaker (230) of FIG. 2. Based on the input, the electronic device (101) may establish a call connection based on a voice call (and / or a video call). Within state (502), the electronic device (101) may display a screen for controlling the call connection. Through the screen displayed based on the status (502), the electronic device (101) can receive input for terminating a call connection or controlling a microphone and / or speaker for a call connection.

[0093] In an exemplary state (501) of FIG. 5, the electronic device (101) may display a visual object (513) for establishing a call connection based on TTS. A visual object (513) in the form of a button containing a designated text, such as "text call," is exemplarily illustrated, but the embodiment is not limited thereto. In a state (501) displaying a screen containing the visual object (513), the electronic device (101) that receives an input indicating selection of the visual object (513) may switch to the state (503) of FIG. 5. The electronic device (101) that receives the input may establish a call connection based on TTS. The electronic device (101) that receives the input may display a screen (e.g., a screen displayed on the display (110) in the state (503)) for supporting a text call.

[0094] Referring to FIG. 5, the electronic device (101) that receives an input related to a visual object (513) may switch to a state (503) related to a call connection based on a text call. Within the state (503), which is distinct from the state (502) in which the speaker and / or microphone are activated, the electronic device (101) may not change the state of the speaker and / or the microphone. For example, within the state (503), the speaker and / or the microphone may be deactivated or may remain in an inactive state. Within the state (503), the electronic device (101) may display a screen including the first area (140), the second area (150), and the third area (160) of FIG. 1 on the display (110). Within the first area (140), the electronic device (101) can display information about a call connection established within the state (503) and / or visual objects (141, 142) for controlling the call connection. Within the second area (150), the electronic device (101) can display text messages corresponding to utterances exchanged between a user of the electronic device (101) and a user of an external electronic device that is the other party to the call connection, based on the call connection. Through the third area (160), the electronic device (101) can obtain or receive text to be used for synthesizing a voice signal to be provided to the other party to the call connection.

[0095] In the exemplary state (503) of FIG. 5, based on the establishment of a call connection based on a text call, the electronic device (101) may execute a function for transmitting a voice signal including a specified text to the other party of the call connection. The function may be executed based on a model included in the electronic device (101) (e.g., a model provided by the TTS application (246) of FIG. 2). The embodiment is not limited thereto, and the function may be executed based on a model included in a server (e.g., the server (130) of FIG. 1 and / or the server (250) of FIG. 2) connected to the electronic device (101) (e.g., the generative language model (286) of FIG. 2).

[0096] Within the exemplary state (503) of FIG. 5, the electronic device (101) may display a text message (530) corresponding to the specified text. While displaying the text message (530), the electronic device (101) may highlight a portion of the text message (530) that is currently being provided to the other party of the call connection (or is currently being played by the other party, which is an external electronic device). The highlighting may include an action of adjusting the color, size, and / or visual effects (e.g., underlining and / or italics) of the text.

[0097] Referring to FIG. 5, the text provided to the other party of the call connection in state (503) may be predetermined by the user of the electronic device (101). For example, in state (504) prior to state (503), the electronic device (101) may display a screen for receiving the text to be provided to the other party by establishing a call connection based on a text call. The electronic device (101) may switch to state (504) based on the execution of a designated software application for changing the setting value of the electronic device (101), such as a settings application.

[0098] In an exemplary state (504) of FIG. 5, the electronic device (101) may display a text box (540) for obtaining text. Together with the text box (540), the electronic device (101) may display a visual object (542) for storing the text included in the text box (540). The electronic device (101) may display a visual object (541) for switching to a state different from the state (504) without storing the text included in the text box (540). Referring to FIG. 5, in the state (503), a voice signal played by an external electronic device, which is the other party of a call connection, may include a natural language sentence that was obtained through the text box (540) in the state (504) prior to the state (503).

[0099] The embodiment is not limited thereto, and based on the establishment of a call connection based on a text call, the electronic device (101) may provide a voice signal including an utterance representing at least a portion of the context of the user of the electronic device (101) to the other party of the call connection. The user context represented by the utterance may include at least one of schedule information of the user of the electronic device (101) (e.g., schedule information registered or stored by a calendar application) and / or the current location of the user. For example, the electronic device (101) may provide a natural language sentence, such as “I am currently in a meeting,” to the other party of the call connection in the form of a voice signal together with other natural language sentences included in a text message (530). The natural language sentence, such as “I am currently in a meeting,” may be synthesized by a model (e.g., a model provided by the TTS application (246) of FIG. 2 and / or a generative language model (286)) into which schedule information corresponding to the current time is input.

[0100] As described above, according to one embodiment, the electronic device (101) may receive an input for establishing a call connection based on either a text call or a voice call through a screen for establishing a call (e.g., a screen displayed in a state (501)). In a state (502) in which a call connection is established based on a voice call, the electronic device (101) may display a screen for controlling hardware related to the voice call (e.g., a microphone (225) and / or a speaker (230) of FIG. 2). In a state (503) in which a call connection is established based on a text call, the electronic device (101) may display a screen for obtaining text based on a touch input. In order to guide that the call connection is established based on a text call, the electronic device (101) may, in response to establishing the call connection, provide a voice signal including an utterance related to the text call (e.g., an utterance included in a text message (530)) to the other party of the call connection.

[0101] Hereinafter, exemplary operations of an electronic device (101) based on a call connection are described with reference to FIGS. 6a to 6e.

[0102] FIGS. 6A, 6B, 6C, 6D, and 6E illustrate different states of an electronic device (101) related to a call connection. The electronic device (101) of FIGS. 1 and / or 2 may include the electronic device (101) of FIGS. 6A to 6E. For example, the electronic device (101) of FIGS. 1 and 2 and / or the processor (210) of FIG. 2 may perform the operations of the electronic device (101) described with reference to FIGS. 6A to 6E. The operations of the electronic device (101) of FIGS. 6A to 6E may be related to at least one of the operations of FIGS. 3 and / or 4.

[0103] Referring to FIGS. 6A to 6E , exemplary states (601, 602, 603, 604, 605, 606, 607, 608, 609) of an electronic device (101) associated with a call connection based on a text call are illustrated. The states of the electronic device (101) may be sequentially switched between the states (601, 602, 603, 604, 605, 606, 607, 608, 609) within the exemplary call connection.

[0104] Referring to FIG. 6A, an exemplary state (601) after the state (503) of FIG. 5 is illustrated. In the state (601) in which a call connection based on a text call is established, the electronic device (101) can obtain a voice signal provided from the other party of the call connection through a communication circuit. In the state (601) in which the voice signal is identified, the electronic device (101) can display a visual object (610) on the display (110) to guide the identification of the voice signal. An embodiment in which the electronic device (101) displays a visual object (610) in the form of a speech bubble on a second area (150) of a screen displayed on the display (110) is illustrated, but the embodiment is not limited thereto. Through the visual object (610), the electronic device (101) can display text, an image, an icon, and / or an animation that notifies the identification and / or reception of the voice signal.

[0105] In a state (601) of receiving a voice signal from the other party of a call connection, the electronic device (101) can recognize an utterance included in the voice signal. To recognize the utterance, the electronic device (101) can run a model provided by the ASR client application (248) of FIG. 2 or communicate with a server (250). A visual object (610) can be displayed while executing a function for recognizing the utterance. Referring to FIG. 6A, the electronic device (101) that has recognized an utterance in a voice signal can switch from a state (601) of displaying a visual object (610) to a state (602). In a state (602) after stopping displaying the visual object (610), the electronic device (101) can display a text message (620) including the recognized utterance in a second area (150) of a screen displayed on the display (110).

[0106] Referring to FIG. 6A, in an exemplary state (602) in which a text message (530) corresponding to a voice signal provided to the other party of a call connection is displayed based on the right edge of the display (110), the electronic device (101) may display a text message (620) corresponding to a voice signal provided from the other party of the call connection based on the left edge of the display (110). In one embodiment of FIG. 6A, at least one text message provided from the electronic device (101) to the other party through the call connection may be aligned at the right edge of the second region (150). In one embodiment of FIG. 6A, at least one text message provided from the other party to the electronic device (101) through the call connection may be aligned at the left edge of the second region (150). The embodiment is not limited thereto.

[0107] Referring to FIG. 6B , an exemplary state (603) following state (602) of FIG. 6A is illustrated. Within state (603), the electronic device (101) may recommend utterances to be included in a voice signal to be provided to the other party of the call connection. For example, the electronic device (101) may display visual objects (631, 632) on the display (110) for recommending utterances. Each of the visual objects (631, 632) may include a candidate utterance generated by the electronic device (101) (or a server connected to the electronic device (101)) based on a context related to the call connection.

[0108] For example, the visual object (631) may include an utterance generated based on schedule information corresponding to the user of the electronic device (101), such as "I'm in a meeting." In a state (603), if the schedule information related to the meeting matches the current time, the electronic device (101) may recommend an utterance to guide the user's state related to the meeting using the visual object (631). The embodiment is not limited thereto, and the electronic device (101) may recommend different utterances depending on the type of schedule information, as shown in Table 2.

[0109] Types (or categories) of information: I'm on vacation. I'm on a train. I'm at the library. I'm taking a seminar. I'm in class.

[0110] The utterances recommended by the electronic device (101) are not limited to those listed in Table 2. For example, the electronic device (101) may recommend other utterances related to the utterance (e.g., a question) contained in a voice signal provided by the other party of the call connection. For example, the electronic device (101) may adjust the formality, tone, speed, and / or volume of the utterance based on the relationship between the user of the electronic device (101) and the other user connected through the call connection, as indicated by the contact information. The relationship may be detected or determined by the electronic device (101) based on at least one of the contact information, call log, and / or message log stored in the electronic device (101).

[0111] For example, based on the relationship between the other party of the call connection indicated by the contact information and the user of the electronic device (101), the electronic device (101) may display a visual object (632) on the display (110) for recommending a text to be included in a voice signal to be provided to the other party. In a state (603) of establishing a call connection with an external electronic device of another user registered as a friend in the contact information, the electronic device (101) may display a visual object (632) corresponding to a utterance (e.g., “Hello, my friend”) based on the contact information.

[0112] Referring to FIG. 6B, in a state (603) of recommending a utterance to be provided to the other party based on visual objects (631, 632), the electronic device (101) may receive an input indicating a selection of any one of the visual objects (631, 632). For example, in response to an input for selecting a visual object (631), the electronic device (101) may execute a function for transmitting a voice signal including a utterance corresponding to the visual object (631) to the other party of the call connection. For example, in response to an input indicating a selection of a visual object (632), the electronic device (101) may switch from state (603) of FIG. 6B to state (604).

[0113] Referring to FIG. 6B, within state (604), the electronic device (101) may execute a function for transmitting a voice signal (e.g., a voice signal including an utterance corresponding to the text “Hello, friend”) associated with a visual object (632) to the other party of the call connection. The function may be performed exclusively by the electronic device (101) or may be performed by a server connected to the electronic device (101) (e.g., the server (130) of FIG. 1 and / or the server (250) of FIG. 2). Based on the execution of the function, the other party, which is an external electronic device, may output the voice signal. Based on the execution of the function, the electronic device (101) may display a text message (640) corresponding to the voice signal within the second area (150).

[0114] Referring to FIG. 6B, within states (603, 604), the electronic device (101) may receive or identify user input indicating text input through the third region (160). Referring to FIG. 6C, while switching from state (603) to state (604) and state (605), the electronic device (101) may continuously receive the user input. Within state (605) of FIG. 6C, the electronic device (101) may determine whether to provide a voice signal to the other party of the call connection based on the duration of receiving the user input. The voice signal may be provided to the other party prior to completion of the text based on the user input, based on the determination of the electronic device (101).

[0115] Referring to FIG. 6c, in a state (605) where the period of time for which a user input is received through the third region (160) exceeds a threshold, the electronic device (101) may execute a function for transmitting a voice signal guiding the user input. For example, the function may include an operation of providing a filler word, such as “wait a second,” to the other party of a call connection based on TTS. In the state (605) of FIG. 6c, based on the execution of the function, the electronic device (101) may generate a voice signal for reproducing a utterance including the filler word. The generated voice signal may be transmitted to the other party, which is an external electronic device, through the call connection. The external electronic device may output the voice signal through a speaker. In a state (605) in which the above function is executed, the electronic device (101) can display a text message (650) corresponding to the voice signal provided to the other party based on the above function within the second area (150) of the screen displayed on the display (110).

[0116] Referring to FIG. 6c, in a state (605) where a voice signal related to a filler word is provided to the other party of a call connection prior to completion of a text based on user input, the electronic device (101) may periodically (or repeatedly) transmit the voice signal. For example, the electronic device (101) may periodically transmit the voice signal prior to completion of the text based on the user input. The electronic device (101) may display text indicating the periodic transmission of the voice signal (e.g., “Repeated transmission after 3 seconds”, located adjacent to a visual object (650)) on the display (110).

[0117] Referring to FIG. 6C, a text displayed in a text box (161) of a third area (160) may be gradually completed by a user input received through a third area (160). Referring to FIG. 6C, a state (606) is illustrated after a state (605) in which a first natural language sentence (e.g., “oh, how have you been?”) is completed. The electronic device (101) may confirm completion of the first natural language sentence based on NLP. After confirming completion of the first natural language sentence, the electronic device (101) may receive at least a portion of a second natural language sentence following the first natural language sentence. In an exemplary state (606) of FIG. 6C, the electronic device (101) may receive at least a portion of the second natural language sentence (e.g., “meal”).

[0118] Referring to FIG. 6c, in a state (606) where at least a portion of the second natural language sentence is received, the electronic device (101) may execute a function for transmitting a voice signal expressing the first natural language sentence through a call connection based on TTS. For example, if the function for automatically transmitting at least a portion of a text input by a user based on a setting value set by a setting application is permitted, the electronic device (101) may convert the first natural language sentence into a voice signal based on the function before the entire text including the first natural language sentence is completed. The electronic device (101) may provide the converted voice signal to the other party of the call connection.

[0119] Referring to FIG. 6C, in a state (606) where a voice signal corresponding to a first natural language sentence is provided to the other party of a call connection, the electronic device (101) may display a text message (660) related to the voice signal in a second area (150) of the screen. In a state (606) where a voice signal corresponding to the first natural language sentence input through the third area (160) is transmitted, the text message (660) displayed by the electronic device (101) may include the first natural language sentence. The electronic device (101) may highlight the first natural language sentence corresponding to the voice signal provided to the other party of the call connection in a text box (161) of the third area (160) based on a visual effect such as an underline. The visual effect applied to the first natural language sentence is not limited to the underline, and may include other visual effects such as italics, changes in color, and / or size.

[0120] Referring to FIG. 6C, based on a user input received through the third area (160), the electronic device (101) can add, change, or delete one or more characters in the text box (161). The electronic device (101) can display a cursor (661) within the text box (161) indicating a point where a character selected by the keyboard is input. The cursor (661) can be moved within the text box (161) based on a gesture (e.g., a drag gesture) performed on a portion of the display (110) where the text box (161) is displayed.

[0121] In the exemplary state (606) of FIG. 6C, since a first natural language sentence corresponding to a voice signal provided to the other party of the call connection is included in the text box (161), the electronic device (101) can receive a gesture for editing the first natural language sentence. For example, based on the first gesture for moving the cursor (661), the electronic device (101) can move the cursor (661) into the first natural language sentence. In the example, based on the cursor (661) moved into the first natural language sentence, the electronic device (101) can receive one or more second gestures for editing the first natural language sentence.

[0122] Referring to FIG. 6D , an exemplary state (607) of the electronic device (101) that has received the first gesture and the one or more second gestures after the state (606) of FIG. 6C is illustrated. Within the state (607), a first natural language sentence may be changed (e.g., “Oh, how are you?”) based on the first gesture and the one or more second gestures. The electronic device (101), upon confirming a change in the first natural language sentence corresponding to a voice signal provided to the other party of the call connection, may execute a function for transmitting a voice signal notifying the change. Based on the execution of the function, the electronic device (101) may provide a voice signal corresponding to a natural language sentence notifying the change in the first natural language sentence (e.g., “The sentence previously spoken is being edited”) to the other party of the call connection. Based on the execution of the above function, the electronic device (101) may display a text message (670) corresponding to the natural language sentence notifying a change in the first natural language sentence within the second area (150). For example, the electronic device (101) may display a text message (670) indicating that the change in the first natural language sentence has been notified to the other party of the call connection.

[0123] Referring to FIG. 6d, in a state (607) where a first natural language sentence is changed by a user input, the electronic device (101) may receive a third gesture for moving the cursor (661) out of the first natural language sentence. The electronic device (101) that has received the third gesture may switch from state (607) of FIG. 6d to state (608). For example, the electronic device (101) that has identified the cursor (661) that has moved to a second natural language sentence following the first natural language sentence may execute a function for transmitting a voice signal related to the changed first natural language sentence through a call connection. In state (608), based on the execution of the function, the electronic device (101) may provide a voice signal expressing the changed first natural language sentence to the other party of the call connection. Within state (608), the electronic device (101) may display a text message (680) corresponding to the modified first natural language sentence within the second area (150). Within state (608), the electronic device (101) may apply a visual effect (e.g., underlining) within a text box (161) to indicate that a voice signal corresponding to the modified first natural language sentence has been provided to the other party of the call connection.

[0124] As described above, according to one embodiment, the electronic device (101) may provide a voice signal related to the text in the text box (161) to the external electronic device, which is the other party of the call connection, while the text is being updated by a user input. By providing the voice signal before the completion of the text, the electronic device (101) may prevent the user of the external electronic device from misunderstanding the silence that occurs before the completion of the text (e.g., recognizing that the call connection has been interrupted).

[0125] Although an embodiment in which the electronic device (101) automatically executes the function of providing the voice signal has been described, the embodiment is not limited thereto. For example, the electronic device (101) may determine whether to transmit the first natural language sentence based on TTS within a state (606) in which the completion of the first natural language sentence has been confirmed. For example, within a state (605) in which the period of time for which a user input has been received exceeds a threshold, the electronic device (101) may display a pop-up window for determining whether to transmit a filler word. Based on the input associated with the pop-up window, the electronic device (101) may display a text message (650) described within the state (605) and provide a voice signal corresponding to the text message (650) to the other party of the call connection.

[0126] Referring to FIG. 6e, a state (609) subsequent to the state (608) of FIG. 6d is illustrated. After receiving a text (e.g., "Oh, how are you? Let's have dinner~") through a text box (161), in response to an input indicating selection of a visual object (162), the electronic device (101) may switch to the state (609). In response to the input, the electronic device (101) may execute a function for transmitting at least a portion of the text based on TTS. In the state (608) of FIG. 6d, since a voice signal corresponding to a first natural language sentence (e.g., "Oh, how are you?") among the text has been transmitted, the electronic device (101) may provide another portion of the text, which is distinct from the first natural language sentence, to the other party of the call connection based on TTS.

[0127] Referring to FIG. 6e, the electronic device (101) may provide a voice signal corresponding to the other part (e.g., "Let's eat~") to the other party of the call connection. Within the state (609) of FIG. 6e, the electronic device (101) may further display a text message (690) corresponding to the voice signal provided to the other party within the second area (150). Based on an input indicating selection of a visual object (162), the electronic device (101) may remove the text displayed within the text box (161).

[0128] Referring to FIG. 6E, based on utterances of users (e.g., a user of the electronic device (101) and a user of an external electronic device) based on a call connection, text messages (530, 620, 640, 650, 660, 670, 680, 690) corresponding to the utterances may be accumulated within the second area (150). Within the scrollable second area (150), the electronic device (101) may display at least one text message corresponding to at least one recently uttered utterance among the accumulated text messages (530, 620, 640, 650, 660, 670, 680, 690). Based on a drag gesture performed on the second area (150) (e.g., a drag gesture performed along the vertical direction of a screen displayed on the display (110), the electronic device (101) can change a text message viewable through the second area (150). For example, based on the drag gesture, the electronic device (101) can move or shift text messages (530, 620, 640, 650, 660, 670, 680, 690) sequentially arranged in a time domain within the second area (150) in a direction corresponding to the drag gesture.

[0129] As described above, according to one embodiment, the electronic device (101) can synthesize a first voice signal representing text provided by a user of the electronic device (101), as well as a second voice signal including other text describing a motion and / or action of the user (e.g., the user's action of entering the text). While an embodiment of providing a voice signal including a filler word such as "Just a moment" has been described within state (605) of FIG. 6C , the embodiment is not limited thereto. For example, the electronic device (101) can generate or synthesize the second voice signal including utterances describing the actions of a previous user from a third-person perspective, such as "User A is typing a text with a keyboard," "User A has almost completed the first sentence and would like to reply," and / or "User A is viewing a list of emoticons." Within the above example, “User A” may include text (e.g., name and / or ID) representing a user logged into the electronic device (101).

[0130] Referring to FIG. 6E, the electronic device (101) may generate or synthesize utterances that appear to be uttered by different speakers based on one or more voice characteristics. For example, the electronic device (101) may synthesize voice signals corresponding to at least a portion of text based on user input (e.g., voice signals corresponding to text messages (640, 660, 680, 690)) based on a first voice characteristic of a user of the electronic device (101). The first voice characteristic may be obtained based on user utterances recorded by the electronic device (101). The voice signals synthesized based on the first voice characteristic may include utterances having the same rate, volume, and / or tone as those uttered by the user of the electronic device (101).

[0131] In the above example, voice signals for notifying the user input (e.g., voice signals corresponding to text messages (650, 670)), such as filler words, may be synthesized based on a second voice characteristic of a designated user (or narrator). The voice signals generated based on the second voice characteristic may include utterances having the same rate, volume, and / or tone as if spoken by the designated user. The embodiment is not limited thereto, and the electronic device (101) may synthesize or generate all voice signals to be provided to the other party of the call connection based on either the first voice characteristic or the second voice characteristic.

[0132] Referring to FIG. 6E, according to one embodiment, the electronic device (101) may display a different form (e.g., size, color, and / or position) of a text message based on whether the text message corresponds to a user input. For example, the form of text messages (640, 660, 680, 690) corresponding to at least a portion of the text entered based on the user input may be different from the form of text messages (650, 670) corresponding to a voice signal indicating the user input. The embodiment is not limited thereto, and the electronic device (101) may display an icon and / or image indicating that the text messages (650, 670) among the text messages (640, 650, 660, 670, 680, 690) having a matching form (e.g., color) have been synthesized to notify the other party of the call connection of the user input by overlapping the text messages.

[0133] With reference to FIGS. 1 to 5 and FIGS. 6A to 6E, an embodiment of an electronic device (101) that provides a voice signal guiding the reception of user input based on the context of a call connection has been described, but the embodiment is not limited thereto. Below, an embodiment of an electronic device (101) that performs an operation different from TTS based on the context is described.

[0134] FIG. 7 illustrates an exemplary operation of an electronic device (101) that recognizes utterances exchanged between users through a call connection. The electronic device (101) of FIG. 1 and / or FIG. 2 may include the electronic device (101) of FIG. 7. For example, the electronic device (101) of FIG. 1 and FIG. 2 and / or the processor (210) of FIG. 2 may perform the operation of the electronic device (101) described with reference to FIG. 7. The operation of the electronic device (101) of FIG. 7 may be related to at least one of the operations of FIG. 3 and / or FIG. 4.

[0135] Referring to FIG. 7, exemplary states (701) of an electronic device (101) related to the call connection of FIGS. 6A to 6E are illustrated. Within a state (701) subsequent to a state (609) of FIG. 6E, the electronic device (101) may display a text message (711) corresponding to a voice signal provided from the other party of the call connection subsequent to the state (609) (e.g., a text message (711) including text such as “GOOD! How about 27th?”), and, following the text message (711), may display a text message (712) input from a user of the electronic device (101) (e.g., a text message (712) including text such as “Ok, shall we meet in Seoul?”). The text message (711) may include a text obtained by performing STT on a voice signal provided from the other party of the call connection. A text message (712) may include text entered by a user of the electronic device (101) based on a text box (161).

[0136] Referring to FIG. 7, in a state (701) after the state (609) of FIG. 6e, when a natural language input based on a text box (161) is changed by a user, the electronic device (101) may display a text message (713) notifying the change of the natural language, similar to the text message (670) of FIG. 6d. At the timing of generating the text message (713), the electronic device (101) may provide a voice signal corresponding to the natural language included in the text message (713) (e.g., “I said it wrong. I’ll talk to you again.”) to the other party of the call connection. The timing of generating the text message (713) and / or the timing of providing the voice signal to the other party may be before a user input (e.g., a user input indicating selection of a visual object (162)) notifying the completion of the input of the natural language after the electronic device (101) identifies the change of the natural language. Referring to FIG. 7, after displaying a text message (713), the electronic device (101) can display a text message (714) corresponding to a natural language sentence input by the user (e.g., “How about the next day, at Seoul?”).

[0137] Referring to FIG. 7, according to one embodiment, an electronic device (101) can recognize the context of a call connection related to a text call. For example, the electronic device (101) can recognize the context based on at least one of text messages (530, 620, 640, 650, 660, 670, 680, 690, 711, 712, 713, 714). For example, when users have determined the schedule and / or content of a meeting through a call connection, such as text messages (711, 712, 714), the electronic device (101) can display a visual object (715) for adding and / or registering the schedule. The visual object (715) can have the form of a button including a specified text, such as "Add Schedule." The embodiment is not limited thereto.

[0138] Within the exemplary state (701) of FIG. 7, in response to an input indicating selection of a visual object (715), the electronic device (101) may display a pop-up window (716) for adding a schedule. The pop-up window (716) may be displayed as an overlay on the screen that was being displayed within the state (701). Within the pop-up window (716), the electronic device (101) may display metadata (e.g., date, location, and / or invitee) to be added to the schedule based on context recognition. Referring to FIG. 7, the metadata may include a date (e.g., "2023.08.28") based on date-related natural language included in text messages (711, 714). Referring to FIG. 7, the metadata may include a location (e.g., "Seoul") based on words included in text messages (714). The attendees included in the above metadata may include the user of the external electronic device that is the other party to the call connection (e.g., “Anh”).

[0139] Referring to FIG. 7, within a pop-up window (716) displayed during a call connection, the electronic device (101) may display a visual object (719) (e.g., a visual object (719) including designated text such as “Save”) for generating and / or storing a schedule including metadata of the pop-up window (716). In response to an input indicating selection of the visual object (719), the electronic device (101) may generate or store a schedule including metadata of the pop-up window (716).

[0140] In one embodiment, within a pop-up window (716), the electronic device (101) may display a visual object (718) (e.g., a visual object (718) containing designated text such as “Cancel”) to stop displaying the pop-up window (716) without generating the schedule.

[0141] According to one embodiment, within the pop-up window (716), the electronic device (101) may display a visual object (717) (e.g., a visual object (717) including specified text such as “Invite”) that causes transmission of a text message (e.g., short message service (SMS), long message service (LMS), and / or email) to an external electronic device of an attendee specified by the metadata (e.g., an external electronic device that is the other party to a call connection) to notify the schedule.

[0142] Referring to FIG. 7, in response to an input indicating selection of a visual object (717), the electronic device (101) may perform an action based on metadata displayed within a pop-up window (716). For example, similar to the visual object (719), the electronic device (101) may add a schedule including metadata of the pop-up window (716). In response to an input indicating selection of the visual object (717), the electronic device (101) may transmit information to the other party of the call connection to create and / or add a schedule including metadata of the pop-up window (716). The information may be transmitted to the other party, which is an external electronic device, via e-mail, SMS, and / or LMS. The external electronic device may display a pop-up window for adding the schedule based on the information.

[0143] Referring to FIG. 7, an embodiment of an electronic device (101) that recommends adding a schedule based on context is described, but the embodiment is not limited thereto. Actions that may be recommended based on context recognition may include actions such as setting an alarm, performing a payment, and / or sending an e-mail.

[0144] As described above, according to one embodiment, the electronic device (101) can recognize the context of utterances between users based on a call connection. Based on the result of recognizing the context, the electronic device (101) can recommend another action that is distinct from the call connection. In one embodiment, the context recognition can be performed after the call connection. For example, after the call connection is terminated, the electronic device (101) can recognize the context of text messages (e.g., text messages (530, 620, 640, 650, 660, 670, 680, 690, 711, 712, 713, 714)) corresponding to utterances transmitted between users during the call connection, and determine one or more topics of the call connection. The electronic device (101) can display one or more topics corresponding to the call connection together with the call connection within a screen including a call history.

[0145] Hereinafter, with reference to FIG. 8, an exemplary operation of the electronic device (101) described with reference to FIGS. 1 to 7 is described.

[0146] FIG. 8 illustrates a flowchart of an electronic device according to one embodiment. The electronic device (101) of FIG. 1 and / or FIG. 2 may include the electronic device of FIG. 8. For example, the electronic device (101) of FIG. 1 and FIG. 2 and / or the processor (210) of FIG. 2 may perform the operations of the electronic device (101) described with reference to FIG. 8. The operations of the electronic device of FIG. 8 may be related to at least one of the operations of FIG. 3 and / or FIG. 4.

[0147] In the following examples, the operations may be performed sequentially, but are not necessarily sequential. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0148] Referring to FIG. 8, in operation (810), according to one embodiment, a processor of an electronic device may receive a first input for establishing a call connection using TTS through a first screen for call connection. The first screen of operation (810) may include a screen displayed in state (501) of FIG. 5. The first input of operation (810) may include an input indicating selection of a visual object (513) of FIG. 5. Operation (810) may be performed based on the first screen displayed by the processor that has received a signal for establishing a call through a communication circuit (e.g., the communication circuit (235) of FIG. 2).

[0149] Referring to FIG. 8, in operation (820), according to an embodiment, a processor of an electronic device may display a second screen corresponding to a first input. Based on receiving the first input, the processor may perform operation (820). The second screen of operation (820) may include a screen displayed in state (503) of FIG. 5. The processor may establish a call connection based on operation (820). Based on the established call connection, the processor may execute a function for a text call (e.g., a function of displaying the second screen). Through the second screen, the processor may receive a second input representing a text message to be converted by TTS. While displaying the second screen, based on a voice signal provided from the other party of the call connection, the processor may display at least one text message (e.g., text message (620) of FIG. 6A) on the second screen, the text including the converted text from the voice signal.

[0150] Referring to FIG. 8, in operation (830), a processor of an electronic device according to one embodiment may determine, while receiving a second input representing a text message to be converted by TTS, whether to transmit a voice signal before completion of the second input. The processor may perform operation (830) of FIG. 8 similarly to at least one of operation (340) of FIG. 3 and / or operations of FIG. 4. The voice signal of operation (830) may be provided before completion of the second input to prevent call termination and / or call release by the other party of the call connection.

[0151] Referring to FIG. 8, in operation (840), according to one embodiment, a processor of an electronic device may execute a function for transmitting a voice signal notifying receipt of the second input through a call connection based on determining transmission of a voice signal in operation (830). The processor may perform operation (840) of FIG. 8 similarly to operation (350) of FIG. 3. Based on operation (840), a user of an external electronic device connected to the electronic device based on the call connection may recognize that the user of the electronic device is inputting text.

[0152] Referring to FIG. 8, in operation (850), according to one embodiment, the processor of the electronic device may determine whether a third input indicating completion of a second input has been received. The third input may include an input indicating selection of a visual object (162) of FIGS. 6A to 6E. If the third input has not been received, or before receiving the third input (850-No), the processor may perform operations (830, 840, 850) of FIG. 8. For example, before receiving the third input, based on operation (840), an audio signal indicating receipt of the second input may be repeatedly transmitted through a call connection. If the third input has been received (850-Yes), the processor may perform operation (860).

[0153] Referring to FIG. 8, in operation (860), according to one embodiment, a processor of an electronic device may execute a function for transmitting a voice signal related to a text message of a second input through a call connection. The processor may acquire, synthesize, or generate a voice signal corresponding to the text message related to the second input based on TTS. The processor may provide the voice signal to the other party through the call connection. An external electronic device connected to the electronic device through the call connection may output the voice signal through a speaker. Based on operation (860), the processor may display the text message of the second input on the second screen of operation (820). After operation (860), before the call connection is released, the processor may additionally receive a second input for inputting a text message. The processor, which additionally receives the second input, may perform operations (830, 840, 850, and 860) of FIG. 8 again.

[0154] As described above, according to one embodiment, an electronic device can visualize a voice call and support the synthesis and / or transmission of a voice signal based on TTS. To maintain a conversation (e.g., a conversation between users connected via a call connection) that is interrupted while acquiring text (e.g., at least one natural language sentence) to be included in the voice signal based on a touch input, the electronic device can provide an audio signal to the other party of the voice call, indicating the receipt of the touch input.

[0155] FIG. 9 is a block diagram of an electronic device (901) within a network environment (900) according to various embodiments. Referring to FIG. 9, in the network environment (900), the electronic device (901) may communicate with the electronic device (902) via a first network (998) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (904) or the server (908) via a second network (999) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (901) may communicate with the electronic device (904) via the server (908). According to one embodiment, the electronic device (901) may include a processor (920), a memory (930), an input module (950), an audio output module (955), a display module (960), an audio module (970), a sensor module (976), an interface (977), a connection terminal (978), a haptic module (979), a camera module (980), a power management module (988), a battery (989), a communication module (990), a subscriber identification module (996), or an antenna module (997). In some embodiments, the electronic device (901) may omit at least one of these components (e.g., the connection terminal (978)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (976), the camera module (980), or the antenna module (997)) may be integrated into one component (e.g., the display module (960)).

[0156] The processor (920) may, for example, execute software (e.g., a program (940)) to control at least one other component (e.g., a hardware or software component) of the electronic device (901) connected to the processor (920) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (920) may store commands or data received from other components (e.g., a sensor module (976) or a communication module (990)) in a volatile memory (932), process the commands or data stored in the volatile memory (932), and store result data in a non-volatile memory (934). According to one embodiment, the processor (920) may include a main processor (921) (e.g., a central processing unit or an application processor) or an auxiliary processor (923) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (921). For example, when the electronic device (901) includes the main processor (921) and the auxiliary processor (923), the auxiliary processor (923) may be configured to use less power than the main processor (921) or to be specialized for a given function. The auxiliary processor (923) may be implemented separately from the main processor (921) or as a part thereof.

[0157] The auxiliary processor (923) may control at least a portion of functions or states associated with at least one component (e.g., a display module (960), a sensor module (976), or a communication module (990)) of the electronic device (901), for example, on behalf of the main processor (921) while the main processor (921) is in an inactive (e.g., sleep) state, or together with the main processor (921) while the main processor (921) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (923) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (980) or a communication module (990)). In one embodiment, the auxiliary processor (923) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (901) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (908)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0158] The memory (930) can store various data used by at least one component (e.g., the processor (920) or the sensor module (976)) of the electronic device (901). The data can include, for example, software (e.g., the program (940)) and input data or output data for commands related thereto. The memory (930) can include a volatile memory (932) or a non-volatile memory (934).

[0159] The program (940) may be stored as software in the memory (930) and may include, for example, an operating system (942), middleware (944), or an application (946).

[0160] The input module (950) can receive commands or data to be used in a component of the electronic device (901) (e.g., a processor (920)) from an external source (e.g., a user) of the electronic device (901). The input module (950) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0161] The audio output module (955) can output audio signals to the outside of the electronic device (901). The audio output module (955) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0162] The display module (960) can visually provide information to an external party (e.g., a user) of the electronic device (901). The display module (960) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (960) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0163] The audio module (970) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (970) can acquire sound through the input module (950), output sound through the sound output module (955), or an external electronic device (e.g., electronic device (902)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (901).

[0164] The sensor module (976) can detect the operating status (e.g., power or temperature) of the electronic device (901) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (976) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0165] The interface (977) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (901) with an external electronic device (e.g., the electronic device (902)). In one embodiment, the interface (977) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0166] The connection terminal (978) may include a connector through which the electronic device (901) may be physically connected to an external electronic device (e.g., the electronic device (902)). In one embodiment, the connection terminal (978) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0167] The haptic module (979) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (979) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0168] The camera module (980) can capture still images and videos. According to one embodiment, the camera module (980) may include one or more lenses, image sensors, image signal processors, or flashes.

[0169] The power management module (988) can manage the power supplied to the electronic device (901). According to one embodiment, the power management module (988) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0170] A battery (989) may power at least one component of the electronic device (901). In one embodiment, the battery (989) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0171] The communication module (990) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (901) and an external electronic device (e.g., electronic device (902), electronic device (904), or server (908)), and the performance of communication through the established communication channel. The communication module (990) may operate independently from the processor (920) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (990) may include a wireless communication module (992) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (994) (e.g., a local area network (LAN) communication module, or a power line communication module). Any of these communication modules may communicate with an external electronic device (904) via a first network (998) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (999) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (992) may use subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (996) to verify or authenticate the electronic device (901) within a communication network such as the first network (998) or the second network (999).

[0172] The wireless communication module (992) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency communications (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (992) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (992) may support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (992) may support various requirements specified in the electronic device (901), an external electronic device (e.g., the electronic device (904)), or a network system (e.g., the second network (999)). According to one embodiment, the wireless communication module (992) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0173] The antenna module (997) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (997) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (997) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (998) or the second network (999), may be selected from the plurality of antennas, for example, by the communication module (990). A signal or power may be transmitted or received between the communication module (990) and the external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (997).

[0174] According to various embodiments, the antenna module (997) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0175] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0176] According to one embodiment, commands or data may be transmitted or received between the electronic device (901) and an external electronic device (904) via a server (908) connected to a second network (999). Each of the external electronic devices (902, or 1504) may be the same or a different type of device as the electronic device (901). According to one embodiment, all or part of the operations executed in the electronic device (901) may be executed in one or more of the external electronic devices (902, 1504, or 1508). For example, when the electronic device (901) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (901) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (901). The electronic device (901) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (901) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (904) may include an Internet of Things (IoT) device. The server (908) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (904) or the server (908) may be included in the second network (999).The electronic device (901) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0177] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0178] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0179] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0180] Various embodiments of the present document may be implemented as software (e.g., a program (940)) including one or more instructions stored in a storage medium (e.g., an internal memory (936) or an external memory (938)) readable by a machine (e.g., an electronic device (901)). For example, a processor (e.g., a processor (920)) of the machine (e.g., an electronic device (901)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0181] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0182] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separately arranged in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, a program, or another component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added. The electronic device (901) of FIG. 9 may be an example of the electronic device described with reference to FIGS. 1 to 8.

[0183] In one embodiment, a method may be required for conducting a voice call using a voice signal converted from text based on text-to-speech (TTS). In one embodiment, a method may be required for preventing unintended interruption of a voice call while receiving text as a source of the voice signal. As described above, according to one embodiment, an electronic device (e.g., the electronic device (101) of FIGS. 1 to 7 and / or the electronic device (901) of FIG. 9) may include a display (e.g., the display (110) of FIG. 1 and / or the display (220) of FIG. 2), a communication circuit (e.g., the communication circuit (235) of FIG. 2), and a processor (e.g., the processor (210) of FIG. 2). The processor may be configured to receive a first input for establishing a call connection based on text-to-speech (TTS) based on a first screen for call connection. The processor may be configured to, in response to the first input, control the communication circuitry to establish the call connection, and display a second screen including a first area (e.g., the second area (150) of FIG. 1 ) capable of displaying at least one first text message corresponding to at least a portion of a first voice signal to be provided by the call connection on the display. The processor may be configured to, while receiving a second input representing a second text message to be converted by the TTS based on the second area (e.g., the third area (160) of FIG. 1 ) of the second screen, determine whether to transmit a second voice signal prior to completing the second input based on a duration of the second input.The processor may be configured to execute a function for transmitting the second voice signal, indicating receipt of the second input, through the call connection based on determining transmission of the second voice signal. The processor may be configured to display a third text message (e.g., text message (650) of FIG. 6C) corresponding to the second voice signal within the first area of ​​the second screen based on execution of the function.

[0184] For example, the processor may be configured to, within a state in which the first natural language sentence is recognized from the second text message based on natural language processing (NLP), receive at least a portion of a second natural language sentence next to the first natural language sentence based on the second input. The processor may be configured to, based on receiving the at least a portion of the second natural language sentence, execute the function for transmitting, through the call connection, the second voice signal representing the first natural language sentence based on the TTS.

[0185] For example, the processor may be configured to display the third text message including the first natural language sentence within the first region based on execution of the function.

[0186] For example, the processor may be configured to execute another function for transmitting, through the call connection, a third voice signal for notifying the change in the first natural language sentence output by the second voice signal based on detecting a change in the first natural language sentence displayed in the second area after executing the function.

[0187] For example, the processor may be configured to determine the third text message to be represented by the second voice signal based on contact information for the counterpart of the call connection.

[0188] For example, the processor may be configured to, while receiving the second input, display at least one visual object on the second screen for recommending text to be included in the second voice signal based on a relationship between the other party and the user of the electronic device indicated by the contact information.

[0189] For example, the processor may be configured to, based on determining transmission of the second voice signal, communicate with a server for the TTS (e.g., server (130) of FIG. 1 and / or server (250) of FIG. 2) to cause the server to transmit the second voice signal to an external electronic device that is a counterpart of the call connection.

[0190] For example, the processor may be configured to initiate execution of the function for transmitting the second voice signal based on the period exceeding the specified period.

[0191] For example, the processor may be configured to repeatedly transmit the second voice signal based on the specified period of time prior to completion of the second input, after displaying the third text message, based on execution of the function.

[0192] For example, the processor may be configured to display a visual object indicating repetitive transmission of the second speech signal prior to completion of the second input.

[0193] For example, the processor may be configured to display, on the display, the second screen including a visual object for executing the TTS. The processor may be configured to execute, in response to a third input indicating selection of the visual object, a second function for transmitting, through the call connection, a third voice signal representing the second text message input based on the second input. The processor may be configured to display, based on execution of the second function, the second text message within the first area of ​​the second screen.

[0194] For example, the processor may be configured to execute a first function for transmitting the second voice signal based on a first voice characteristic of a designated speaker through the call connection. The processor may be configured, in response to the third input, to execute the second function for transmitting the third voice signal based on a second voice characteristic of a user of the electronic device through the call connection.

[0195] For example, the processor may be configured to display, within the first region, the third text message having a first designated color corresponding to the first function based on the execution of the first function. The processor may be configured to display, within the first region, the second text message having a second designated color based on the execution of the second function.

[0196] For example, the processor may be configured to display the first screen, which includes a first visual object for displaying the second screen, and a second visual object for establishing a voice call for the call connection using a microphone and a speaker of the electronic device. The processor may be configured to control the communication circuitry to establish the call connection in response to the first input indicating selection of the first visual object.

[0197] According to one embodiment of the present invention, a method of an electronic device including a display, a communication circuit, and a processor as described above may include an operation of receiving, based on a first screen for a call connection, a first input for establishing the call connection based on TTS (e.g., operation 810 of FIG. 8 ). The method may include, in response to the first input, an operation of controlling the communication circuit to establish the call connection, and displaying, on the display, a second screen including a first area capable of displaying at least one first text message corresponding to at least a portion of a first voice signal to be provided by the call connection (e.g., operation 820 of FIG. 8 ). The method may include an operation of receiving, based on a second area of ​​the second screen, a second input indicating a second text message to be converted by TTS, while determining, based on a duration of the second input, whether to transmit a second voice signal before completion of the second input (e.g., operation 830 of FIG. 8 ). The method may include an operation (e.g., operation (840) of FIG. 8) of executing a function for transmitting the second voice signal, which notifies receipt of the second input, through the call connection based on determining transmission of the second voice signal. The method may include an operation of displaying a third text message corresponding to the second voice signal within the first area of ​​the second screen based on execution of the function.

[0198] For example, the determining operation may include an operation of receiving, based on the second input, at least a portion of a second natural language sentence following the first natural language sentence, within a state in which the first natural language sentence has been recognized from the second text message based on natural language processing (NLP). The executing operation may include an operation of executing a function for transmitting, through the call connection, the second voice signal expressing the first natural language sentence based on the TTS, based on receiving the at least a portion of the second natural language sentence.

[0199] For example, the action of displaying the third text message may include an action of displaying the third text message including the first natural language sentence within the first area based on the execution of the function.

[0200] For example, the method may include an operation of executing another function for transmitting, through the call connection, a third voice signal for notifying the change in the first natural language sentence output by the second voice signal based on detecting a change in the first natural language sentence displayed in the second area after executing the function.

[0201] For example, the action of displaying the third text message may include an action of determining the third text message to be represented by the second voice signal based on contact information about the other party of the call connection.

[0202] For example, the method may include, while receiving the second input, displaying at least one visual object within the second screen for recommending text to be included in the voice signal based on a relationship between the other party indicated by the contact information and the user of the electronic device.

[0203] For example, the executing action may be configured to cause the server for the TTS to transmit the second voice signal to an external electronic device that is the other party of the call connection, by communicating with the server based on determining transmission of the second voice signal.

[0204] For example, the executing action may include an action of initiating execution of the function for transmitting the second voice signal based on the period exceeding the specified period.

[0205] For example, the method may include, based on execution of the function, repeatedly transmitting the second voice signal based on the specified period of time before completion of the second input after displaying the third text message.

[0206] For example, the method may include, prior to completion of the second input, displaying a visual object indicating repetitive transmission of the second speech signal.

[0207] For example, the operation of displaying the second screen may include an operation of displaying the second screen including a visual object for executing the TTS on the display. The method may include an operation of executing a second function for transmitting, through the call connection, a third voice signal representing the second text message input based on the second input in response to a third input indicating selection of the visual object. The method may include an operation of displaying the second text message within the first area of ​​the second screen based on the execution of the second function.

[0208] For example, the act of executing the first function may include an act of executing the first function for transmitting the second voice signal based on the first voice characteristic of the designated speaker through the call connection. The method may include an act of executing the second function for transmitting the third voice signal based on the second voice characteristic of the user of the electronic device through the call connection in response to the third input.

[0209] For example, the action of displaying the third text message may include an action of displaying the third text message having a first designated color corresponding to the first function within the first area based on the execution of the first function. The method may include an action of displaying the second text message having a second designated color within the first area based on the execution of the second function.

[0210] For example, the receiving operation may include displaying the first screen, which includes a first visual object for displaying the second screen, and a second visual object for establishing a voice call for the call connection using a microphone and a speaker of the electronic device. The method may further include controlling the communication circuit to establish the call connection in response to the first input indicating selection of the first visual object.

[0211] In one embodiment, a non-transitory computer-readable storage medium including instructions as described above, wherein the instructions, when executed by a processor of an electronic device including a display, communication circuitry, and a processor, cause the electronic device to receive a first input for establishing a call connection based on a first screen for a call connection, based on text-to-speech (TTS). The instructions, when executed by the processor, cause the electronic device to, in response to the first input, control the communication circuitry to establish the call connection, and display a second screen including a first area capable of displaying at least one first text message corresponding to at least a portion of a first voice signal to be provided by the call connection. The instructions, when executed by the processor, may cause the electronic device to determine, based on a duration of a second input while receiving a second input indicating a second text message to be converted by the TTS based on a second area of ​​the second screen, whether to transmit a second voice signal prior to completing the second input. The instructions, when executed by the processor, may cause the electronic device to execute a function for transmitting the second voice signal via the call connection, the second voice signal indicating receipt of the second input, based on the determining of transmission of the second voice signal. The instructions, when executed by the processor, may cause the electronic device to display, based on the execution of the function, a third text message corresponding to the second voice signal within the first area of ​​the second screen.

[0212] The devices described above may be implemented as hardware components, software components, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone; however, one of ordinary skill in the art will recognize that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing unit may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

[0213] Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may independently or collectively command the processing device. The software and / or data may be embodied in any type of machine, component, physical device, computer storage medium, or device for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed over networked computer systems and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

[0214] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. In this case, the medium may be one that continuously stores a computer-executable program or one that temporarily stores it for execution or download. In addition, the medium may be various recording or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, but may also be distributed over a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to store program commands, including ROM, RAM, and flash memory. In addition, examples of other media may include recording or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc.

[0215] Although the embodiments described above have been described by way of limited examples and drawings, those skilled in the art will appreciate that various modifications and variations can be made based on the above teachings. For example, appropriate results can still be achieved even if the described techniques are performed in a different order than described, and / or components of the described systems, structures, devices, circuits, etc. are combined or combined in a different manner than described, or are replaced or substituted with other components or equivalents.

[0216] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims described below.

Claims

1. In an electronic device (101; 901), display(110; 220); Communication circuit (235); A memory (215) storing instructions, including one or more storage media; and An electronic device comprising at least one processor (210), wherein the instructions, when executed by the at least one processor, Based on a first screen for call connection, a first input for establishing the call connection is received based on TTS (text-to-speech); In response to said first input, controlling said communication circuit to establish said call connection, and displaying a second screen including a first area (150) capable of displaying at least one first text message corresponding to at least a portion of a first voice signal to be provided by said call connection on said display; While receiving a second input representing a second text message to be converted by the TTS based on the second area (160) of the second screen, determining whether to transmit a second voice signal prior to completing the second input based on the duration of the second input; Based on determining transmission of the second voice signal, executing a function for transmitting the second voice signal, which notifies reception of the second input, through the call connection; and Based on the execution of the above function, causing a third text message (650) corresponding to the second voice signal to be displayed within the first area of ​​the second screen. Electronic devices.

2. In claim 1, the instructions, when executed by the at least one processor, cause the electronic device to: Recognizing a first natural language sentence from the second text message based on natural language processing (NLP), receiving at least a part of a second natural language sentence next to the first natural language sentence based on the second input; causing said function to be executed to transmit said second voice signal representing said first natural language sentence based on said TTS through said call connection, based on receiving said at least a part of said second natural language sentence; Electronic devices.

3. In claims 1 and 2, the instructions, when executed by the at least one processor, cause the electronic device to: Based on the execution of the above function, causing the third text message including the first natural language sentence to be displayed within the first area, Electronic devices.

4. In claims 1 to 3, the instructions, when executed by the at least one processor, cause the electronic device to: After executing the above function, based on detecting a change in the first natural language sentence displayed in the second area, causing another function to be executed for transmitting a third voice signal for notifying the change in the first natural language sentence output by the second voice signal through the call connection. Electronic devices.

5. In claims 1 to 4, the instructions, when executed by the at least one processor, cause the electronic device to: causing the third text message to be represented by the second voice signal to be determined based on contact information about the counterpart of the call connection; Electronic devices.

6. In claims 1 to 5, the instructions, when executed by the at least one processor, cause the electronic device to: While receiving said second input, causing at least one visual object to be displayed on said second screen for recommending text to be included in said second voice signal, based on a relationship between said counterpart indicated by said contact information and a user of said electronic device. Electronic devices.

7. In claims 1 to 6, the instructions, when executed by the at least one processor, cause the electronic device to: Based on determining the transmission of the second voice signal, causing the server (130; 250) for the TTS to communicate with the server to transmit the second voice signal to an external electronic device which is a counterpart of the call connection. Electronic devices.

8. In claims 1 to 7, the instructions, when executed by the at least one processor, cause the electronic device to: causing the execution of the function for transmitting the second voice signal to be initiated based on the period exceeding the specified period; Electronic devices.

9. In claims 1 to 8, the instructions, when executed by the at least one processor, cause the electronic device to: Based on the execution of the above function, causing the second voice signal to be repeatedly transmitted based on the specified period of time before the completion of the second input after displaying the third text message. Electronic devices.

10. In claims 1 to 9, the instructions, when executed by the at least one processor, cause the electronic device to: causing a visual object to be displayed prior to completion of said second input, indicating a repetitive transmission of said second speech signal; Electronic devices.

11. In claims 1 to 10, the instructions, when executed by the at least one processor, cause the electronic device to: On the above display, display the second screen including a visual object for executing the TTS; In response to a third input indicating selection of the visual object, a second function is executed for transmitting, through the call connection, a third voice signal representing the second text message input based on the second input; Based on the execution of the second function, causing the second text message to be displayed within the first area of ​​the second screen. Electronic devices.

12. In claims 1 to 11, the instructions, when executed by the at least one processor, cause the electronic device to: Executing a first function for transmitting the second voice signal based on the first voice characteristic of the designated speaker through the call connection; and In response to said third input, causing said second function to be executed for transmitting said third voice signal based on a second voice characteristic of a user of said electronic device through said call connection. Electronic devices.

13. In claims 1 to 12, the instructions, when executed by the at least one processor, cause the electronic device to: Based on the execution of the first function, displaying the third text message having the first designated color corresponding to the first function within the first area; and Based on the execution of the second function, causing the second text message having the second designated color to be displayed within the first area. Electronic devices.

14. In claims 1 to 13, the instructions, when executed by the at least one processor, cause the electronic device to: Displaying the first screen including a first visual object for displaying the second screen and a second visual object for establishing a voice call for the call connection using a microphone and a speaker of the electronic device; In response to said first input indicating selection of said first visual object, causing said communication circuit to be controlled to establish said call connection; Electronic devices.

15. A method of an electronic device including a display, a communication circuit and a processor, An operation (810) of receiving a first input for establishing the call connection based on TTS, based on a first screen for call connection; In response to said first input, controlling said communication circuit to establish said call connection, and displaying a second screen including a first area capable of displaying at least one first text message corresponding to at least a portion of a first voice signal to be provided by said call connection on said display (820); An operation (830) of determining whether to transmit a second voice signal prior to completion of the second input, based on a duration of the second input, while receiving a second input representing a second text message to be converted by the TTS based on a second area of ​​the second screen; An operation (840) for executing a function for transmitting the second voice signal, which notifies the reception of the second input, through the call connection based on determining the transmission of the second voice signal; and Based on the execution of the above function, comprising an action of displaying a third text message corresponding to the second voice signal within the first area of ​​the second screen; method.

Images