Method for determining end point detection time and electronic device for performing the same

Abstract

A method of determining an end point detection (EPD) time and an electronic device for performing the same are disclosed. An electronic device according to an embodiment includes: a processor; and a memory electrically connected to the processor and storing instructions executed by the processor, where when the instructions are executed, the processor identifies an utterance, determines an end point detection (EPD) time for terminating the identification of the utterance based on utterance data stored in the memory, and transmits the determined end point detection (EPD) time to the processor. The utterance data may include sentence data including a pattern related to a sentence type of the utterance and word data including a word of the utterance, and terminate recognition of the utterance when no additional user utterance is input within an end point detection time, where the utterance data may include sentence data including a pattern related to a sentence type of the utterance and word data including a word of the utterance.

Description

Technical Field

[0001] The present disclosure relates to a method for determining an end point detection time based on an individual speech history of a user and an electronic device for executing the method. Background Art

[0002] Various electronic devices are distributed in which a speech recognition function is equipped to provide services based on the user's speech. In speech recognition, determining whether the user's speech is complete may have a significant impact on the speech recognition performance.

[0003] Voice activity detection (VAD) using speech signals can be used to determine whether a user's utterance is complete.

[0004] The above information is presented as background information only to assist with an understanding of the present disclosure. No determination has been made, and no assertion is made, as to whether any of the above might be applicable as prior art with respect to the present disclosure. Summary of the invention

[0005] Various aspects of the present disclosure are intended to at least solve the above problems and / or disadvantages and to provide at least the advantages described below. Therefore, one aspect of the present disclosure is to provide a method for determining an end point detection time based on a user's individual speech history and an electronic device for executing the method.

[0006] Additional aspects will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the presented embodiments.

[0007] According to one aspect of the present disclosure, an electronic device includes a processor and a memory, the memory being electrically connected to the processor and configured to store commands executed by the processor. The processor may be configured to recognize speech when executing commands. The processor may be configured to determine an end point detection (EPD) time for terminating recognition of speech based on speech data stored in the memory. The processor may be configured to terminate recognition of speech when no additional speech is input within the EPD time. The speech data may include sentence data and word data, the sentence data including a pattern related to a sentence type of speech, and the word data including words of speech.

[0008] According to another aspect of the present disclosure, an electronic device is provided. The electronic device includes a processor and a memory, the memory being electrically connected to the processor and configured to store a command executed by the processor. When the command is executed, the processor may be configured to determine the category of the speech. The processor may be configured to compare the pattern of the speech with the pattern stored in the speech data based on the speech data corresponding to the category. When the pattern of the speech is different from the pattern stored in the speech data, the processor may be configured to determine whether the speech is a complete sentence, an incomplete sentence, or an ambiguous sentence. The processor may be configured to determine the EPD time for terminating the recognition of the speech based on the result of comparing the pattern of the speech with the pattern stored in the speech data and whether the speech is a complete sentence, an incomplete sentence, or an ambiguous sentence. The processor may be configured to terminate the recognition of the speech when no additional speech is input within the EPD time. The speech data may include sentence data and word data, the sentence data including a pattern related to the sentence type of the speech, and the word data including the words of the speech.

[0009] According to another aspect of the present disclosure, a method for determining an EPD time is provided. The method includes recognizing an utterance, determining an EPD time for terminating recognition of the utterance based on utterance data 112 stored in a memory 130, and terminating recognition of the utterance when no further utterance is input within the EPD time. The utterance data may include sentence data 113 and word data, the sentence data 113 including a pattern related to a sentence type of the utterance 195, and the word data including words of the utterance 195.

[0010] Other aspects, advantages, and salient features of the present disclosure will become apparent to those skilled in the art from the following detailed description which, in conjunction with the accompanying drawings, discloses various embodiments of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a block diagram illustrating an electronic device in a network environment according to an embodiment of the present disclosure;

[0012] Figure 2 is a block diagram illustrating an integrated intelligent system according to an embodiment of the present disclosure;

[0013] Figure 3 is a diagram illustrating an operation of an electronic device determining an end point detection (EPD) time according to an embodiment of the present disclosure;

[0014] Figure 4 is a flowchart illustrating an operation method of an electronic device determining an EPD time according to an embodiment of the present disclosure;

[0015] Figure 5is a flowchart illustrating an operation method of an electronic device determining an EPD time according to an embodiment of the present disclosure;

[0016] Figure 6 is a diagram showing utterance data according to an embodiment of the present disclosure;

[0017] Figure 7 is a diagram showing a user interface for registering an utterance in utterance data according to an embodiment of the present disclosure;

[0018] Figure 8 is a diagram showing categories of utterances according to an embodiment of the present disclosure;

[0019] Fig. 9 is a diagram showing a category ratio of an utterance according to an embodiment of the present disclosure;

[0020] Fig.10 is a diagram showing sentence data according to an embodiment of the present disclosure;

[0021] Fig.11 is a diagram illustrating word data according to an embodiment of the present disclosure;

[0022] FIG. 12A to FIG. 12G is a diagram showing an EPD time determined by an electronic device for each utterance according to an embodiment of the present disclosure; and

[0023] FIG. 13A to FIG. 13C is a diagram illustrating an operation of an electronic device determining an EPD time according to an EPD command according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] Throughout the drawings, it should be noted that like reference numbers are used to depict the same or similar elements, features, and structures.

[0025] The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of the various embodiments of the present disclosure as defined by the claims and their equivalents. It includes various specific details to assist in understanding, but these details are to be regarded as exemplary only. Therefore, it will be appreciated by those of ordinary skill in the art that various changes and modifications may be made to the various embodiments described herein without departing from the scope and spirit of the present disclosure. In addition, descriptions of well-known functions and structures may be omitted for clarity and conciseness.

[0026] The terms and words used in the following description and claims are not limited to the bibliographical meanings, but are merely used by the inventor to enable a clear and consistent understanding of the present disclosure. Therefore, it will be apparent to those skilled in the art that the following description of various embodiments of the present disclosure is provided for illustrative purposes only and not for the purpose of limiting the present disclosure as defined by the appended claims and their equivalents.

[0027] It should be understood that the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to "a component surface" includes reference to one or more of such surfaces. Figure 1 is a block diagram showing an electronic device in a network environment according to an embodiment of the present disclosure. Figure 1 , the electronic device 101 in the network environment 100 may communicate with the electronic device 102 via the first network 198 (e.g., a short-range wireless communication network), or communicate with at least one of the electronic device 104 or the server 108 via the second network 199 (e.g., a long-range wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 via the server 108. According to another embodiment, the electronic device 101 may include a processor 120, a memory 130, an input module 150, a sound output module 155, a display module 160, an audio module 170, and a sensor module 176, an interface 177, a connection terminal 178, a haptic module 179, a camera module 180, a power management module 188, a battery 189, a communication module 190, a subscriber identification module (SIM) 196, or an antenna module 197. In some embodiments, at least one of the above components (e.g., the connection terminal 178) may be omitted from the electronic device 101, or one or more other components may be added to the electronic device 101. In some embodiments, some of the components (eg, sensor module 176, camera module 180, or antenna module 197) may be integrated into a single component (eg, display module 160).

[0028] The processor 120 may execute, for example, software (e.g., program 140) to control at least one other component (e.g., hardware or software component) of the electronic device 101 connected to the processor 120, and may perform various data processing or calculations. According to another embodiment, as at least part of the data processing or calculation, the processor 120 may store a command or data received from another component (e.g., sensor module 176 or communication module 190) in the volatile memory 132, process the command or data stored in the volatile memory 132, and store the resulting data in the non-volatile memory 134. According to another embodiment, the processor 120 may include a main processor 121 (e.g., a central processing unit (CPU) or an application processor (AP)) or an auxiliary processor 123 (e.g., a graphics processing unit (GPU), a neural processing unit (NPU), an image signal processor (ISP), a sensor hub processor, or a communication processor (CP)) that may operate independently of the main processor 121 or in conjunction with the main processor 121. For example, when the electronic device 101 includes the main processor 121 and the auxiliary processor 123, the auxiliary processor 123 is adapted to consume less power or be specific to a designated function than the main processor 121. The auxiliary processor 123 may be implemented separately from the main processor 121 or as part of the main processor 121.

[0029] The auxiliary processor 123 may control at least some of the functions or states related to at least one of the components of the electronic device 101 (e.g., display module 160, sensor module 176, or communication module 190) instead of the main processor 121 when the main processor 121 is in an inactive (e.g., sleep) state, or control at least some of the functions or states related to at least one of the components of the electronic device 101 (e.g., display module 160, sensor module 176, or communication module 190) together with the main processor 121 when the main processor 121 is in an active state (e.g., executing an application). According to another embodiment, the auxiliary processor 123 (e.g., ISP or CP) may be implemented as a part of another component (e.g., camera module 180 or communication module 190) related to the function of the auxiliary processor 123. According to another embodiment, the auxiliary processor 123 (e.g., NPU) may include a hardware structure designated for artificial intelligence (AI) model processing. The AI ​​model may be generated by machine learning. Such learning may be performed, for example, by the electronic device 101 in which artificial intelligence is executed, or via a separate server (e.g., server 108). The learning algorithm may include, but is not limited to, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. The AI ​​model may include multiple artificial neural network layers. The artificial neural network may include, for example, a deep neural network (DNN), a convolutional neural network (CNN), a recursive neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recursive deep neural network (BRDNN), a deep Q network, or a combination of two or more thereof, but is not limited thereto. The AI ​​model may additionally or optionally include a software structure in addition to a hardware structure.

[0030] The memory 130 may store various data used by at least one component of the electronic device 101 (e.g., the processor 120 or the sensor module 176). The various pieces of data may include, for example, input data or output data of software (e.g., the program 140) and commands related thereto. The memory 130 may include a volatile memory 132 or a non-volatile memory 134.

[0031] The program 140 may be stored as software in the memory 130 , and may include, for example, an operating system (OS) 142 , middleware 144 , or applications 146 .

[0032] The input module 150 may receive a command or data to be used by another component (e.g., the processor 120) of the electronic device 101 from outside (e.g., a user) of the electronic device 101. The input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus).

[0033] The sound output module 155 can output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as playing multimedia or playing recordings. The receiver may be used to receive incoming calls. According to another embodiment, the receiver may be implemented separately from the speaker or as part of the speaker.

[0034] The display module 160 can visually provide information to the outside of the electronic device 101 (e.g., a user). The display module 160 may include, for example, a display, a hologram device, or a projector, and a control circuit for controlling a corresponding one of the display, the hologram device, and the projector. According to yet another embodiment, the display module 160 may include a touch sensor adapted to sense a touch, or a pressure sensor adapted to measure the strength of a force caused by a touch.

[0035] The audio module 170 can convert sound into an electrical signal, and vice versa. According to another embodiment, the audio module 170 can obtain sound via the input module 150, or output sound via the sound output module 155 or an external electronic device (e.g., an electronic device 102 such as a speaker or earphone) directly or wirelessly connected to the electronic device 101.

[0036] The sensor module 176 may detect an operating state (e.g., power or temperature) of the electronic device 101 or an environmental state (e.g., a user's state) outside the electronic device 101, and generate an electrical signal or data value corresponding to the detected state. According to yet another embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyroscope sensor, an atmospheric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illumination sensor.

[0037] The interface 177 may support one or more designated protocols for direct (e.g., by wire) or wireless coupling of the electronic device 101 to an external electronic device (e.g., the electronic device 102). According to another embodiment, the interface 177 may include, for example, a high-definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

[0038] The connection terminal 178 may include a connector, and the electronic device 101 may be physically connected to an external electronic device (e.g., the electronic device 102) via the connector. According to another embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0039] The haptic module 179 may convert electrical signals into mechanical stimulation (eg, vibration or movement) or electrical stimulation, which may be recognized by the user via his or her tactile sense or kinesthetic sense. According to yet another embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0040] The camera module 180 may capture still images and moving images. According to yet another embodiment, the camera module 180 may include one or more lenses, image sensors, ISPs, or flashes.

[0041] The power management module 188 may manage power supplied to the electronic device 101. According to yet another embodiment, the power management module 188 may be implemented as, for example, at least a portion of a power management integrated circuit (PMIC).

[0042] The battery 189 may supply power to at least one component of the electronic device 101. According to yet another embodiment, the battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0043] The communication module 190 can support establishing a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (e.g., electronic device 102, electronic device 104, or server 108), and perform communication via the established communication channel. The communication module 190 may include one or more CPs that can operate independently of the processor 120 (e.g., AP) and support direct (e.g., wired) communication or wireless communication. According to another embodiment, the communication module 190 may include a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (e.g., a local area network (LAN) communication module or a power line communication (PLC) module). The corresponding one of these communication modules can communicate via a first network 198 (e.g., a short-range communication network such as Bluetooth TM The wireless communication module 192 may communicate with the external electronic device 104 using a wireless communication module (e.g., wireless fidelity (Wi-Fi) direct, or infrared data association (IrDA)) or a second network 199 (e.g., a remote communication network such as a traditional cellular network, a fifth generation (5G) network, a next generation communication network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules may be implemented as a single component (e.g., a single chip), or may be implemented as multiple components (e.g., multiple chips) separated from each other. The wireless communication module 192 may use subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the SIM 196 to identify and authenticate the electronic device 101 in a communication network (e.g., a first network 198 or a second network 199).

[0044] The wireless communication module 192 can support 5G networks after the fourth generation (4G) network and next generation communication technologies, such as new radio (NR) access technology. NR access technology can support enhanced mobile broadband (eMBB), massive machine type communication (mMTC) or ultra-reliable and low latency communication (URLLC). The wireless communication module 192 can support high frequency bands (e.g., mmWave bands) to achieve, for example, high data transmission rates. The wireless communication module 192 can support various technologies for ensuring performance on high frequency bands, such as, for example, beamforming, massive multiple input and multiple output (MIMO), full dimensional MIMO (FD-MIMO), array antennas, analog beamforming or massive antennas. The wireless communication module 192 can support various requirements specified in the electronic device 101, an external electronic device (e.g., an electronic device 104) or a network system (e.g., a second network 199). According to another embodiment, the wireless communication module 192 can support a peak data rate for implementing eMBB (e.g., 20 Gbps or more), loss coverage for implementing mMTC (e.g., 164 dB or less), or U-plane delay for implementing URLLC (e.g., 0.5 ms or less for each of the downlink (DL) and uplink (UL), or 1 ms or less round trip).

[0045] The antenna module 197 may send a signal or power to the outside of the electronic device 101 (e.g., an external electronic device) or receive a signal or power from the outside of the electronic device 101 (e.g., an external electronic device). According to yet another embodiment, the antenna module 197 may include an antenna including a radiating element including a conductive material or a conductive pattern formed in or on a substrate (e.g., a printed circuit board (PCB)). According to yet another embodiment, the antenna module 197 may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication scheme used in a communication network such as the first network 198 or the second network 199 may be selected from the plurality of antennas by, for example, the communication module 190. A signal or power may be sent or received between the communication module 190 and the external electronic device via the selected at least one antenna. According to yet another embodiment, another component (e.g., a radio frequency integrated circuit (RFIC)) other than the radiating element may be additionally formed as a part of the antenna module 197.

[0046] According to another embodiment, the antenna module 197 may form a millimeter wave antenna module. According to another embodiment, the millimeter wave antenna module may include a PCB, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the PCB and capable of supporting a specified high frequency band (e.g., millimeter wave frequency band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the PCB and capable of transmitting or receiving signals in the specified high frequency band.

[0047] At least some of the above components can be coupled to each other and transmit signals (e.g., commands or data) therebetween via an inter-peripheral communication scheme (e.g., a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)).

[0048] According to another embodiment, commands or data may be sent or received between the electronic device 101 and the external electronic device 104 via a server 108 coupled to the second network 199. Each of the external electronic devices 102 and 104 may be a device of the same type or a different type as the electronic device 101. According to another embodiment, all or some of the operations to be performed by the electronic device 101 may be performed at one or more external electronic devices (e.g., external electronic devices 102 and 104 and server 108). For example, if the electronic device 101 needs to automatically perform a function or service, or in response to a request from a user or another device, the electronic device 101 may request one or more external electronic devices to perform at least a portion of the function or service instead of or in addition to performing the function or service. The one or more external electronic devices receiving the request may perform at least a portion of the requested function or service, or an additional function or additional service related to the request, and may transmit the result of the execution to the electronic device 101. The electronic device 101 may provide the result as at least a part of the reply to the request with or without further processing the result. To this end, for example, cloud computing, distributed computing, mobile edge computing (MEC) or client-server computing technology can be used. The electronic device 101 can use, for example, distributed computing or MEC to provide ultra-low latency services. In another embodiment, the external electronic device 104 may include an Internet of Things (IoT) device. The server 108 may be an intelligent server using machine learning and / or neural networks. According to another embodiment, the external electronic device 104 or the server 108 may be included in the second network 199. The electronic device 101 can be applied to intelligent services (e.g., smart homes, smart cities, smart cars, or health care) based on 5G communication technology or IoT-related technologies.

[0049] Figure 2 is a block diagram illustrating an integrated intelligent system according to an embodiment of the present disclosure.

[0050] refer to Figure 2 , the integrated intelligent system 10 may include an electronic device 101, an intelligent server 200, and a service server 300. The service server 300 may include a CP service A 301 and a CP service B 302.

[0051] The electronic device 101 may be a terminal device (or electronic device) that can be connected to the Internet, and may be, for example, a mobile phone, a smart phone, a personal digital assistant (PDA), a notebook computer, a television (TV), a white appliance, a wearable device, a head-mounted display (HMD), or a smart speaker.

[0052] According to the illustrated embodiment, the electronic device 101 may include an interface 177 (e.g., Figure 1 interface 177), input module 150 (eg, Figure 1 Input module 150), speaker 155 (e.g., Figure 1 The sound output module 155), the display module 160 (for example, Figure 1 display module 160), memory 130 (eg, Figure 1 Memory 130) or processor 120 (e.g., Figure 1 The components listed above may be operatively or electronically connected to each other.

[0053] The interface 177 can be connected to an external device and is configured to send data to and receive data from the external device. The input module 150 can receive sound (e.g., user speech) and convert the sound into an electrical signal. The sound output module 155 can output the electrical signal as sound (e.g., speech).

[0054] The display module 160 may be configured to display an image or video. The display module 160 may also display a graphical user interface (GUI) of an app (or application) being executed. The display module 160 may receive touch input via a touch sensor. For example, the display module 160 receives text input in an on-screen keyboard area displayed in the display module 160 via a touch sensor.

[0055] The memory 130 may store a client module 131, a software development kit (SDK) 133, and a plurality of apps 135. The client module 131 and the SDK 133 may configure a framework (or solution program) for performing general functions. In addition, the client module 131 or the SDK 133 may configure a framework for processing user input (e.g., voice input, text input, or touch input).

[0056] The plurality of apps 135 stored in the memory 130 may be programs for performing designated functions. The plurality of apps 135 may include a first app 135_1, a second app 135_2, and the like. Each of the plurality of apps 135 may include a plurality of actions for performing designated functions. For example, the application may include an alarm clock app, a messaging app, and / or a scheduling app. The plurality of apps 135 may be executed by the processor 120 to sequentially perform at least a portion of the plurality of actions.

[0057] The processor 120 may control the overall operation of the electronic device 101. For example, the processor 120 is electrically connected to the interface 177, the input module 150, the sound output module 155, and the display module 160 to perform a designated operation.

[0058] The processor 120 may also perform a designated function by executing a program stored in the memory 130. For example, the processor 120 executes at least one of the client module 131 or the SDK 133 to perform the following operation for processing a user input. The processor 120 may control the operation of the plurality of apps 135 through, for example, the SDK 133. The following operation as an operation of the client module 131 or the SDK 133 may be performed by the processor 120.

[0059] The client module 131 may receive user input. For example, the client module 131 receives a voice signal corresponding to a user's speech sensed by the input module 150. As another example, the client module 131 receives a touch input sensed by the display module 160. As yet another example, the client module 131 receives a text input sensed by a keyboard or an on-screen keyboard. In addition, the client module 131 may receive various types of user input sensed by an input module included in the electronic device 101 or an input module connected to the electronic device 101. The client module 131 may send the received user input to the intelligent server 200. The client module 131 may send the state information of the electronic device 101 together with the received user input to the intelligent server 200. The state information may be, for example, execution state information of an app.

[0060] The client module 131 may receive a result corresponding to the received user input. For example, when the intelligent server 200 is capable of calculating a result corresponding to the received user input, the client module 131 may receive a result corresponding to the received user input. The client module 131 may display the received result on the display module 160. In addition, the client module 131 may output the received result in an audio form through the sound output module 155.

[0061] The client module 131 may receive a plan corresponding to the received user input. The client module 131 may display the results of executing multiple actions of the app according to the plan on the display module 160. For example, the client module 131 sequentially displays the results of executing multiple actions on the display module 160, and outputs the results in audio form through the sound output module 155. For example, the electronic device 101 displays only a portion of the results of executing multiple actions (e.g., the result of the last action) on the display module 160, and outputs the portion of the results in audio form through the sound output module 155.

[0062] According to an embodiment, the client module 131 may receive a request for obtaining information required to calculate a result corresponding to the user input from the smart server 200. According to another embodiment, the client module 131 may send the required information to the smart server 200 in response to the request.

[0063] The client module 131 may send information about the results of performing the plurality of actions according to the plan to the smart server 200. The smart server 200 may use the information about the results to confirm that the received user input has been processed correctly.

[0064] The client module 131 may include a speech recognition module. According to another embodiment, the client module 131 may recognize a voice input for executing a limited function through the speech recognition module. For example, the client module 131 executes an intelligent application for processing voice input to perform an organic operation through a specified input (e.g., wake up!).

[0065] The intelligent server 200 may receive information related to the user voice input from the electronic device 101 through the communication network. According to another embodiment, the intelligent server 200 may change the data related to the received voice input into text data. According to another embodiment, the intelligent server 200 may generate a plan for executing a task corresponding to the user voice input based on the text data.

[0066] According to another embodiment, the plan can be generated by an artificial intelligence (AI) system. The artificial intelligence system can be a rule-based system or a neural network-based system (e.g., a feedforward neural network (FNN) or a recurrent neural network (RNN)). Alternatively, the artificial intelligence system can be a combination of them or other artificial intelligence systems. According to another embodiment, the plan can be selected from a set of predefined plans, or can be generated in real time in response to a user request. For example, the AI ​​system selects at least one plan from the predefined plans.

[0067] The intelligent server 200 may send the result according to the generated plan to the electronic device 101, or send the generated plan to the electronic device 101. According to another embodiment, the electronic device 101 may display the result according to the plan on the display module 160. According to another embodiment, the electronic device 101 may display the result of executing the action according to the plan on the display module 160.

[0068] The intelligent server 200 may include a front end 210 , a natural language platform 220 , a capsule database (DB) 230 , an execution engine 240 , a terminal user interface 250 , a management platform 260 , a big data platform 270 , or an analysis platform 280 .

[0069] The front end 210 may receive the received user input from the electronic device 101. The front end 210 may send a response corresponding to the user input.

[0070] According to yet another embodiment, the natural language platform 220 may include an automatic speech recognition (ASR) module 221 , a natural language understanding (NLU) module 223 , a planner module 225 , a natural language generator (NLG) module 227 , or a text to speech (TTS) module 229 .

[0071] The ASR module 221 may convert data related to the voice input received from the electronic device 101 into text data. The NLU module 223 may use the text data of the voice input to discern the user's intention. For example, the NLU module 223 may discern the user's intention by performing syntactic analysis or semantic analysis on the user input in the form of text data. The NLU module 223 may use linguistic features (e.g., grammatical elements) of morphemes or phrases to discern the meaning of words extracted from the user input, and determine the user's intention by matching the meaning of the discerned words with the intention. The NLU module 223 may obtain intention information corresponding to the user's utterance. The intention information may refer to information indicating the user's intention determined by analyzing the text data. The intention information may include information indicating the operation or function that the user intends to perform by using the device.

[0072] The planner module 225 may generate a plan using the parameters and intent determined by the NLU module 223. According to another embodiment, the planner module 225 may determine multiple domains required to perform the task based on the determined intent. The planner module 225 may determine multiple actions included in each of the multiple domains determined based on the intent. According to another embodiment, the planner module 225 may determine the parameters required to perform the multiple actions determined or the result values ​​output by performing the multiple actions. The parameters and the result values ​​may be defined as concepts of a specified form (or class). Therefore, the plan may include multiple actions and multiple concepts determined by the user's intent. The planner module 225 may determine the relationship between the multiple actions and the multiple concepts step by step (or hierarchically). For example, the planner module 225 determines the execution order of the multiple actions determined based on the user's intent based on the multiple concepts. In other words, the planner module 225 may determine the execution order of the multiple actions based on the parameters required for the execution of the multiple actions and the results output by the execution of the multiple actions. Therefore, the planner module 225 may generate a plan including connection information (e.g., ontology) between the multiple actions and the multiple concepts. The planner module 225 may generate a plan using information stored in the capsule DB 230 , which stores a set of relationships between concepts and actions.

[0073] The NLG module 227 may change the designated information into a text form. The information changed into a text form may be in the form of a natural language speech. The TTS module 229 may change the information in the text form into information in the speech form.

[0074] According to yet another embodiment, some or all of the functions of the natural language platform 220 may also be implemented in the electronic device 101 .

[0075] The capsule DB 230 may store information about the relationship between a plurality of concepts and actions corresponding to a plurality of domains. A capsule according to another embodiment may include a plurality of action objects (or action information) and a concept object (or concept information) included in a plan. According to another embodiment, the capsule DB 230 may store a plurality of capsules in the form of a concept action network (CAN). According to another embodiment, a plurality of capsules may be stored in a function registry included in the capsule DB 230.

[0076] The capsule DB 230 may include a policy registry storing policy information required for determining a plan corresponding to the voice input. The policy information may include reference information for determining a plan in the case where there are multiple plans corresponding to the user input. According to another embodiment, the capsule DB 230 may include a subsequent registry storing information about subsequent actions for suggesting subsequent actions to the user in a specified situation. The subsequent action may include, for example, a subsequent utterance. According to another embodiment, the capsule DB 230 may include a layout registry storing layout information of information output by the electronic device 101. According to another embodiment, the capsule DB 230 may include a vocabulary registry storing vocabulary information included in the capsule information. According to another embodiment, the capsule DB 230 may include a dialogue registry storing information about a dialogue (or interaction) with the user. The capsule DB 230 may update the stored objects through a developer tool. The developer tool may include, for example, a function editor for updating an action object or a concept object. The developer tool may include a vocabulary editor for updating a vocabulary. The developer tool may include a policy editor for generating and registering a policy for determining a plan. The developer tool may include a dialogue editor for generating a dialogue with the user. The developer tool may include a subsequent editor for activating a subsequent target and editing a subsequent utterance providing a prompt. The subsequent target may be determined based on the current set target, the user's preference, or environmental conditions. In another embodiment, the capsule DB 230 may also be implemented in the electronic device 101.

[0077] The execution engine 240 can use the generated plan to calculate the result. The terminal user interface 250 can send the calculated result to the electronic device 101. Therefore, the electronic device 101 can receive the result and provide the received result to the user. The management platform 260 can manage the information used by the intelligent server 200. The big data platform 270 can collect the user's data. The analysis platform 280 can manage the quality of service (QoS) of the intelligent server 200. For example, the analysis platform 280 manages the components and processing rate (or efficiency) of the intelligent server 200.

[0078] The service server 300 may provide a specified service (e.g., food order or hotel reservation) to the electronic device 101. According to yet another embodiment, the service server 300 may be a server operated by a third party. The service server 300 may provide information to be used to generate a plan corresponding to the received voice input to the intelligent server 200. The provided information may be stored in the capsule DB 230. In addition, the service server 300 may provide the intelligent server 200 with result information according to the plan.

[0079] In the integrated intelligent system 10, the electronic device 101 can provide various intelligent services to the user in response to user input. The user input can include, for example, input through a physical button, touch input, or voice input.

[0080] In another embodiment, the electronic device 101 can provide a speech recognition service through a smart app (or speech recognition app) stored therein. In this case, for example, the electronic device 101 recognizes the user's speech or voice input received through the microphone, and provides the user with a service corresponding to the recognized voice input.

[0081] In another embodiment, the electronic device 101 can perform a specified action based on the received voice input alone or together with the intelligent server 200 and / or the service server. For example, the electronic device 101 executes an app corresponding to the received voice input and performs the specified action through the executed app.

[0082] In another embodiment, when the electronic device 101 provides services together with the intelligent server 200 and / or the service server, the electronic device 101 can detect the user's speech using the input module 150 and generate a signal (or voice data) corresponding to the detected user's speech. The electronic device 101 can use the interface 177 to send the speech data to the intelligent server 200.

[0083] The intelligent server 200 may generate a plan for performing a task corresponding to a voice input or a result of performing an action according to the plan as a response to a voice input received from the electronic device 101. The plan may include, for example, a plurality of actions for performing a task corresponding to a user's voice input, and a plurality of concepts related to the plurality of actions. A concept may define a parameter input to the execution of a plurality of actions or a result value output by the execution of a plurality of actions. The plan may include connection information between a plurality of actions and a plurality of concepts.

[0084] The electronic device 101 may receive the response using the interface 177. The electronic device 101 may output a voice signal generated inside the electronic device 101 to the outside using the sound output module 155, or may output an image generated inside the electronic device 101 to the outside using the display module 160.

[0085] Figure 3 1 is a diagram showing an electronic device (eg, Figure 2 FIG. 1 is a diagram of an operation of an electronic device 101) determining an end point detection (EPD) time.

[0086] refer to Figure 3The electronic device 101 may include any one of the ASR module 221, the EPD determination module 110, the sentence type determination module 111, the speech data 112, and the speech interval detection module 116, such as Figure 3 shown.

[0087] For example, the speech data 112 includes at least one of the sentence data 113, the word data 114, and the EPD command 115. For example, the electronic device 101 generates the speech data 112 according to the sentences and words generated based on the input speech 195. For example, the electronic device 101 generates the speech data 112 according to the generated sentences or words based on the user's input. The electronic device 101 can store the generated speech data 112 in a memory (e.g., Figure 2 in the memory 130).

[0088] For example, the sentence data 113 includes a sentence generated according to the utterance 195. For example, the word data 114 includes words generated according to the utterance 195. For example, the EPD command 115 includes at least one of a word or a sentence generated according to the user's input.

[0089] For example, the utterance data 112 is generated according to at least one of the utterance 195 or the user's input and may be generated according to the user's utterance history.

[0090] For example, the ASR module 221 may generate text using the utterance 195. The electronic device 101 may generate the utterance data 112 using the text generated by the ASR module 221.

[0091] For example, the electronic device 101 generates the utterance data 112 using the final text generated by the ASR module 221. For example, the final text indicates a text generated after recognizing that the utterance 195 is complete.

[0092] According to an embodiment, the voice interval detection module 116 may detect a voice interval (eg, a voice pattern) and a non-voice interval (eg, a non-voice pattern) using the input speech 195. For example, the speech 195 refers to an audio signal input to the electronic device 101.

[0093] For example, the speech interval detection module 116 includes at least one of a speech interval determination module 117 and a voice-based model 118. For example, the speech interval detection module 116 inputs the input utterance 195 to the speech interval determination module 117.

[0094] For example, the speech interval determination module 117 inputs the input utterance 195 (eg, an audio signal) to the speech-based model 118. The speech-based model 118 may use the input utterance 195 to distinguish a speech interval from a non-speech interval.

[0095] For example, the speech-based model 118 sends a voice activity detection (VAD) to the speech interval determination module 117 based on whether speech is detected in the input utterance 195 .

[0096] For example, the speech-based model 118 sends a VAD of 1 to the speech interval determination module 117 during a time when speech is detected. For example, the speech-based model 118 sends a VAD of 0 to the speech interval determination module 117 during a time when speech is not detected.

[0097] For example, the speech interval determination module 117 determines a speech interval or a non-speech interval based on the VAD. For example, regarding an interval where the VAD is 1, the speech interval determination module 117 determines the interval as a speech interval. For example, regarding an interval where the VAD is 0, the speech interval determination module 117 determines the interval as a non-speech interval.

[0098] For example, the speech interval determination module 117 sends the EPD to the EPD determination module 110 according to the result of detecting the speech. For example, during the time when the speech is detected, the speech interval determination module 117 sends the EPD of 1 to the EPD determination module 110. For example, during the time when the speech is not detected, the speech interval determination module 117 may send the EPD of 0 to the EPD determination module 110.

[0099] As an example, the EPD determination module 110 determines an EPD time for terminating the recognition of the utterance 195. For example, with respect to the EPD, the EPD time refers to a hangover time for determining whether the utterance 195 is completed. For example, when the non-speech interval lasts longer than the EPD time or the hangover time, the electronic device 101 determines that the utterance 195 is completed. For example, when the utterance 195 is completed, the electronic device 101 terminates the recognition of the utterance 195.

[0100] For example, the EPD determination module 110 determines the EPD time based on the utterance data 112 or the sentence type. For example, the EPD determination module 110 determines the EPD time using the utterance data 112. For example, the EPD determination module 110 determines the EPD time using the sentence type.

[0101] In another embodiment, the EPD determination module 110 receives text from the ASR module 221. For example, the EPD determination module 110 receives partial text from the ASR module 221. For example, the partial text is some text corresponding to the utterance 195 generated by the ASR module 221. For example, the EPD determination module 110 uses the utterance data 112 and the text or the partial text to determine whether a pattern of the utterance 195 is substantially the same as a pattern stored in the utterance data 112. For example, the EPD determination module 110 uses the utterance data 112 and the text or the partial text to determine whether a pattern of the utterance 195 is similar to a pattern stored in the utterance data 112.

[0102] For example, when the pattern of the utterance 195 is substantially the same as the pattern stored in the utterance data 112, the EPD determination module 110 determines the EPD time according to the pattern. For example, the initially set EPD time may be approximately 600 ms. For example, when the pattern of the utterance 195 is substantially the same as the pattern stored in the utterance data 112, the EPD determination module 110 determines the EPD time to be an EPD time of a length such as approximately 100 ms or approximately 2,000 ms according to the pattern.

[0103] For example, when the user's utterance intention is an utterance end intention, the EPD determination module 110 determines the EPD time to be approximately 100 ms. For example, when the user's utterance intention is a subsequent utterance intention, the EPD determination module 110 determines the EPD time to be approximately 2,000 ms. For example, when the pattern of utterance 195 is a pattern of terminating the recognition of utterance 195 early, the EPD determination module 110 determines the EPD time to be an EPD time (e.g., approximately 100 ms) faster than the initially set EPD time (e.g., approximately 600 ms).

[0104] For example, when the pattern of utterance 195 is a pattern in which a subsequent utterance is input after utterance 195, EPD determination module 110 determines the EPD time to be an EPD time (eg, approximately 2,000 ms) slower than the initially set EPD time (eg, approximately 600 ms).

[0105] In the above example, the initial set EPD time of about 600 ms and the EPD times of about 100 ms and 2,000 ms determined by the EPD determination module 110 are examples and are not limited thereto. For example, the initial set EPD time may be about 900 ms. For example, the EPD determination module 110 determines the EPD time to be about 50 ms, about 150 ms, about 200 ms, about 1,500 ms, about 2,500 ms, etc. according to the user's speech intention. For example, the EPD determination module 110 determines the EPD time to be faster or slower than the initially set EPD time according to the user's speech intention.

[0106] As described above, when the speech 195 substantially matches a specific pattern, the electronic device 101 determines that the EPD time is faster than the initially set EPD time in consideration of the user's intention to terminate the speech 195 for speech recognition. The electronic device 101 shortens the waiting time for speech recognition and improves user convenience through the fast EPD time.

[0107] When the utterance 195 is not newly input within the fast EPD time, the electronic device 101 may terminate the recognition of the utterance 195 .

[0108] For example, terminating the recognition of the utterance 195 means that the ASR module 221 terminates the recognition of the utterance 195 to be additionally input. For example, when the recognition of the utterance 195 is completed, the electronic device 101 uses the text generated by the ASR module 221 to perform an operation according to the user's intention.

[0109] For example, when the utterance 195 is newly recognized after a long waiting time following a pattern, the EPD determination module 110 determines the EPD time to be an EPD time (eg, 1,000 ms or approximately 2,000 ms) slower than the initially set EPD time (eg, approximately 600 ms).

[0110] In the case of a mode in which speech 195 is additionally input after a long waiting time, electronic device 101 determines the EPD time to be slower than the initially set EPD time in consideration of the user's intention. Electronic device 101 receives the user's subsequent speech in consideration of the user's intention not to end speech 195 when a specific mode is input by a slow EPD time.

[0111] For example, when the pattern of the utterance 195 is similar to the pattern stored in the utterance data 112, the EPD determination module 110 determines the EPD time according to the pattern. For example, the EPD time initially set is about 600ms. For example, when the pattern of the utterance 195 is similar to the pattern stored in the utterance data 112, the EPD determination module 110 determines the EPD time according to the pattern. For example, when the user's utterance intention is to end the utterance 195, the EPD determination module 110 determines the EPD time to be about 100ms. For example, when the user's utterance intention of the pattern is a subsequent utterance intention, the EPD determination module 110 determines the EPD time to be about 2,000ms.

[0112] When the pattern of the utterance 195 is similar to the pattern stored in the utterance data 112 , the EPD determination module 110 may determine the EPD time of the pattern of the utterance 195 to be the same as the EPD time of the pattern stored in the utterance data 112 .

[0113] For example, the EPD determination module 110 determines whether the pattern of the utterance 195 is substantially the same as or similar to the stored pattern based on at least one of the sentence data 113 and the word data 114 stored in the utterance data 112. For example, the EPD determination module 110 compares the text received from the ASR module 221 with the sentences stored in the sentence data 113. For example, the EPD determination module 110 compares the pattern of the utterance 195 with the stored pattern by comparing the sentence data 113 with the sentence type, composition, and words of the text. For example, the EPD determination module 110 compares the word data 114 with the words in the text to compare the pattern of the utterance 195 with the stored pattern.

[0114] As an example, the EPD determination module 110 transmits the text to the sentence type determination module 111. For example, the EPD determination module 110 transmits the partial text to the sentence type determination module 111. For example, the EPD determination module 110 receives a sentence type corresponding to the partial text from the sentence type determination module 111 in response to the partial text.

[0115] For example, the sentence type includes an incomplete sentence type, a complete sentence type, and an ambiguous sentence type. For example, the incomplete sentence type refers to a sentence that is not completed. For example, the incomplete sentence type refers to an incomplete sentence, such as "I ate", "tomorrow's weather", "songs of singer A", "tomorrow's schedule".

[0116] For example, the complete sentence type refers to a sentence that is completely finished. For example, the complete sentence type refers to a complete sentence, such as "tell me tomorrow's weather", "tell me tomorrow's schedule", "play singer A's song", etc.

[0117] For example, an ambiguous sentence type refers to a sentence type that belongs neither to an incomplete sentence type nor to a complete sentence type.

[0118] For example, the electronic device 101 classifies or determines the sentence type of the utterance 195. For example, the sentence type determination module 111 determines the sentence type of the received text.

[0119] For example, the sentence type determination module 111 determines the sentence type in consideration of language grammar and / or a special purpose (eg, availability of an incomplete sentence) or constraints on the received text.

[0120] For example, the sentence type determination module 111 considers the purpose of the text and determines the sentence type of the text. For example, texts such as "Today's weather", "10 o'clock alarm", and "Mom calls" correspond to incomplete sentence types according to the language grammar sentence is not completely ended. Texts such as "Today's weather", "10 o'clock alarm", and "Mom calls" can correspond to incomplete sentence types according to the language grammar, but the sentence type determination module 111 can consider the purpose of the text and classify texts with incomplete sentence types (such as "Today's weather", "10 o'clock alarm", and "Mom calls") with availability as complete sentence types. For example, texts with incomplete sentence types with availability correspond to incomplete sentences according to the language grammar, but can refer to texts based on which the user's intention and the operation according to the command can be determined.

[0121] For example, the sentence type determination module 111 may determine an inverted sentence as a complete sentence. For example, the sentence "message to my sister" is classified as an incomplete sentence in terms of language grammar, but the sentence type determination module 111 determines that the sentence is a complete sentence.

[0122] For example, the sentence type determination module 111 considers the constraint situation (e.g., ambiguity of language grammar) to determine the sentence type of the text. For example, in the case of a text that can be classified as a complete sentence type or an incomplete sentence type according to the interpretation (such as "tomorrow's weather"), the sentence type determination module 111 considers the constraint situation and thus determines that the text is an ambiguous sentence type.

[0123] For example, the sentence type determination module 111 determines the text as an ambiguous sentence type when the grammatical type of the text cannot be determined, such as a text with a typographical error or misprinted text or a single letter mark. For example, when the ASR module 221 receives a text including a typographical error or a grammatical error, when a short sentence makes it difficult to determine whether the sentence is a complete sentence or an incomplete sentence, or when the sentence is an ambiguous sentence, the sentence type determination module 111 determines such a sentence as an ambiguous sentence type. For example, the sentence type determination module 111 determines the sentence type of the utterance 195 using the text and the utterance data 112 received from the EPD determination module 110. For example, the sentence type determination module 111 determines the sentence type of the utterance 195 using at least one of a sentence pattern and a word stored in the utterance data 112.

[0124] For example, the EPD determination module 110 determines the EPD time according to the sentence type. For example, when the sentence type is a complete sentence, the EPD determination module 110 determines the EPD time to be faster than the initially set EPD time (eg, approximately 600 ms) (eg, approximately 300 ms).

[0125] For example, when the sentence type is an incomplete sentence, the EPD determination module 110 determines the EPD time to be an EPD time (eg, approximately 1,000 ms) slower than the initially set EPD time (eg, approximately 600 ms).

[0126] For example, when the sentence type is an ambiguous sentence type, the EPD determination module 110 determines the EPD time as an initially set EPD time (eg, about 600 ms).

[0127] The electronic device 101 determines the EPD time according to the sentence type of the utterance 195 , taking into consideration the user's intention of additionally inputting the utterance 195 .

[0128] For example, the EPD determination module 110 causes the ASR module 221 to terminate the recognition of the utterance 195 according to the EPD time. For example, when no speech interval is detected within the determined EPD time, the EPD determination module 110 causes the ASR module 221 to terminate the recognition of the utterance 195. For example, when the recognition of the utterance 195 is terminated, the ASR module 221 transmits the text of the recognized utterance 195 to the NLU module (e.g., Figure 2 NLU module 223).

[0129] For example, when no speech interval is detected within the determined EPD time, the ASR module 221 terminates the recognition of the utterance 195. For example, the ASR module 221 receives a result of detecting a speech interval or a non-speech interval from the speech interval detection module 116.

[0130] Figure 3 2 is a diagram illustrating an operation of the electronic device 101 determining an EPD time using the ASR module 221, the EPD determination module 110, the sentence type determination module 111, the utterance data 112, and the voice interval detection module 116 according to an embodiment.

[0131] According to an embodiment, the processor 120 of the electronic device 101 may perform at least one operation of the ASR module 221, the EPD determination module 110, the sentence type determination module 111, and the speech interval detection module 116, such as Figure 3 For example, Figure 3 The operation of each component shown in (eg, the ASR module 221, the EPD determination module 110, the sentence type determination module 111, and the speech interval detection module 116) is performed by the processor of the electronic device 101 (eg, Figure 2 Executed by processor 120).

[0132] exist Figure 3In the example, the speech data 112 may refer to the individual speech history of the user. The electronic device 101 may determine the EPD time suitable for the individual user by determining the EPD time according to the individual speech history of the user. The electronic device 101 may determine the EPD time according to the speech history of the user to terminate the recognition of the speech 195 in advance or delay the termination of the recognition of the speech 195, and may recognize the user's voice according to the user's intention.

[0133] For example, the speech data 112 is initially set by using standardized data including speech histories of multiple users. For example, the speech histories of multiple users are classified according to information of each user, such as the residential area, gender, language, and age of the multiple users. For example, the speech data 112 is set by using user information of the electronic device 101. For example, the electronic device 101 initially sets the speech data 112 by using speech histories corresponding to the information of the user in the standardized data.

[0134] For example, the electronic device 101 updates the utterance data 112 using the utterance history of the user in the utterance data 112 initially set based on the standardized data, or may replace the utterance data 112 according to the utterance history of the user.

[0135] Figure 4 1 is a diagram showing an electronic device (eg, Figure 2 Flow chart of an operating method for determining EPD time of an electronic device 101).

[0136] In the following examples, the operations are performed sequentially, but not necessarily sequentially. For example, the order of the operations is changed and at least two of the operations are performed in parallel.

[0137] According to an embodiment, operations 310 to 330 may be performed by an electronic device (eg, Figure 2 A processor (eg, Figure 2 Executed by processor 120).

[0138] refer to Figure 4 In operation 310, the electronic device 101 according to another embodiment may identify whether a speech (eg, Figure 3 For example, a processor of the electronic device 101 (e.g., Figure 2 The processor 120) uses an ASR module (e.g., Figure 2 and Figure 3 The ASR module 221) determines whether speech 195 is input.

[0139] In operation 320, the electronic device 101 according to another embodiment may generate a signal based on the speech data (eg, Figure 3The EPD time is determined by using the speech data 112 of the detection unit. In the example, the EPD time refers to the time of the detection end point.

[0140] For example, in operation 320 , the electronic device 101 determines the EPD time based on at least one of whether the pattern of the utterance 195 is substantially the same as a pattern stored in the utterance data 112 and whether the pattern of the utterance 195 is similar to a pattern stored in the utterance data 112 .

[0141] For example, in operation 320, the electronic device 101 determines the EPD time according to the sentence type of the utterance 195. The sentence type may include a complete sentence type, an incomplete sentence type, or an ambiguous sentence type.

[0142] In operation 330, the electronic device 101 according to another embodiment may determine whether the speech 195 is input within the EPD time. For example, when the speech 195 is newly input within the determined EPD time, the new EPD time is determined according to operation 320. For example, when the speech 195 is not input within the determined EPD time, the electronic device 101 terminates the recognition of the speech 195. For example, when the recognition of the speech 195 is terminated, the electronic device 101 uses the text corresponding to the input speech 195 to perform an operation according to the intention included in the speech 195.

[0143] Figure 5 2 is an electronic device (eg, Figure 2 Flow chart of an operating method of an electronic device 101).

[0144] In the following examples, the operations may be performed sequentially, but not necessarily sequentially. For example, the order of the operations may be changed, and at least two of the operations may be performed in parallel.

[0145] According to an embodiment, operations 405 to 460 may be performed by an electronic device (eg, Figure 2 A processor (eg, Figure 2 Executed by processor 120).

[0146] refer to Figure 5 , in operation 405 , the electronic device 101 according to another embodiment may identify whether the speech 195 is input. Figure 4 The description of operation 310 may be applied to operation 405 in substantially the same manner.

[0147] In operation 410, the electronic device 101 may determine the category of the utterance 195. For example, a processor (e.g., Figure 2The processor 120 of the electronic device 101 determines the category of the speech 195 by using the K-means algorithm. The K-means algorithm can group a given data piece into K clusters and can operate in a manner that minimizes the variance of the difference in distance to each cluster. For example, the processor 120 of the electronic device 101 determines from a plurality of cluster categories a category that corresponds to the speech (e.g., Figure 3 195) corresponding to the category of discourse.

[0148] In order to determine the category of speech 195, electronic device 101 can use known algorithms different from the above examples (for example, mean shift, Gaussian mixture model, density-based spatial clustering of applications with noise (DBSCAN), agglomerative clustering, etc.) to cluster multiple categories, and can determine the category corresponding to speech 195.

[0149] In operation 415, the electronic device 101 may determine whether the pattern of the speech 195 matches the pattern stored in the speech data (eg, Figure 3 The pattern in the speech data 112) is basically the same.

[0150] In operation 420 , the electronic device 101 may determine whether a pattern of the utterance 195 is similar to a pattern stored in the utterance data 112 .

[0151] For example, the electronic device 101 associates the sentence corresponding to the utterance 195 and the words and sentence data included in the utterance 195 (eg, Figure 3 sentence data 113) and / or word data (e.g., Figure 3 The electronic device 101 compares the word data 114 to determine the similarity. For example, the electronic device 101 determines the similarity based on a similarity model. For example, the similarity model refers to a pre-trained and tuned model. For example, the similarity model uses an input sentence to determine whether the input sentence is similar to a stored sentence or word.

[0152] For example, in operations 415 and 420, the electronic device 101 compares the text corresponding to the utterance 195 with the sentences stored in the sentence data 113. For example, the electronic device 101 determines whether the sentence pattern of the text is substantially the same as the sentence pattern stored in the sentence data 113. For example, the electronic device 101 determines whether the sentence of the text is similar to the sentence stored in the sentence data 113.

[0153] For example, in operations 415 and 420, the electronic device 101 uses various well-known algorithms to determine whether a pattern of the utterance 195 is substantially the same as or similar to a pattern stored in the utterance data 112 or to determine the similarity of sentences.

[0154] According to yet another embodiment, in operation 425, the electronic device 101 determines whether the sentence type of the utterance 195 is a complete sentence type.

[0155] According to yet another embodiment, in operation 430, the electronic device 101 determines whether the sentence type of the utterance 195 is an incomplete sentence type.

[0156] For example, in operations 425 and 430, the electronic device 101 determines the sentence type of the utterance 195 according to the type of the last word in the utterance 195. For example, according to the utterance history of the user, each word included in the word data 114 is determined as one of a clear complete sentence type, a complete sentence type, a clear incomplete sentence type, an incomplete sentence type, and an ambiguous sentence type.

[0157] For example, when the utterance 195 without another voice input after word A is repeatedly input, the electronic device 101 determines word A as a complete sentence type. For example, when the utterance 195 without another voice input after word A is input as much as a specified threshold or more, the electronic device 101 may determine word A as a clear complete sentence type.

[0158] For example, when the utterance 195 in which the voice is additionally input after the word B is repeatedly input, the electronic device 101 determines the word B as an incomplete sentence type. For example, when the utterance 195 in which the voice is additionally input after the word B is input as much as a specified threshold or more, the electronic device 101 sets the word B as a clear incomplete sentence type.

[0159] For example, when the utterance 195 in which a voice is additionally input after the word C and the utterance 195 in which a voice is not additionally input after the word C are input, the electronic device 101 sets the word C as an ambiguous sentence type.

[0160] In the above example, the electronic device 101 sets the sentence type of each word by comparing the threshold with the number of additional inputs of the utterance 195 after each word. The sentence type set for each word can be set according to the user's utterance pattern. For example, since no additional utterance is input after inputting word A according to the user's utterance pattern in the above example, the electronic device 101 sets the type of word A to a clear and complete sentence type.

[0161] For example, the electronic device 101 determines the sentence type of the utterance 195 according to the sentence type corresponding to the last text of the utterance 195 .

[0162] For example, in operations 425 and 430, when the sentence type of the utterance 195 does not correspond to a complete sentence or an incomplete sentence, the electronic device 101 determines the sentence type of the utterance 195 as an ambiguous sentence.

[0163] The method by which the electronic device 101 determines the sentence type of the utterance 195 is not limited to the examples in the above operations 425 and 430. For example, the electronic device 101 determines the user's intention to input a voice or the user's intention not to input a voice based on the utterance 195 corresponding to a complete sentence or the utterance 195 corresponding to an incomplete sentence by various known methods.

[0164] In the above operations 415, 420, 425, and 430, the electronic device 101 may use the utterance data 112 corresponding to the category of the utterance 195. For example, in operation 415, when the category of the utterance 195 is A, the electronic device 101 may use the utterance data 112 corresponding to the category A in the utterance data 112 to determine whether the pattern of the utterance 195 is the same as the stored pattern.

[0165] For example, in operation 415, the electronic device 101 uses the sentence data 113 and / or the word data 114 corresponding to the category A of the utterance 195 to determine whether the pattern of the utterance 195 is substantially the same as the stored pattern.

[0166] Substantially the same as the above-described example, in at least one of operations 420 , 425 , and 430 , the electronic device 101 may perform an operation by using the utterance data 112 of substantially the same category as the utterance 195 .

[0167] For example, in the following operations 435, 440, 445, 450, and 455, the length of the very fast EPD time corresponds to the first length, the length of the fast EPD time may correspond to the second length, the length of the initially set EPD time corresponds to the third length, the length of the slow EPD time corresponds to the fourth length, and the length of the very slow EPD time corresponds to the fifth length. For example, the length of each of the EPD times is determined as (first length) < (second length) < (third length) < (fourth length) < (fifth length).

[0168] In operation 435, the electronic device 101 determines the EPD time according to the matching mode. For example, in operation 435, the electronic device 101 determines the EPD time according to the matching mode.

[0169] For example, the electronic device 101 determines the EPD time in consideration of the user's speech intention. For example, when the pattern of the speech 195 substantially matches the pattern stored in the speech data 112 and the user's speech intention of the pattern is the speech end intention, the electronic device 101 determines an EPD time (e.g., an EPD time of the first length) that is faster than the initially set EPD time.

[0170] For example, in the case where the pattern of the utterance 195 is substantially consistent with the pattern stored in the utterance data 112 and the utterance intention of the user of the pattern is the utterance end intention, it refers to the completion of the utterance 195. When the pattern of the utterance 195 substantially matches the pattern stored in the utterance data 112 and the utterance intention of the user of the pattern is the utterance end intention, the electronic device 101 can determine that the utterance 195 is completed and determine the EPD time as the first length of the EPD time, which is the fastest among the set EPD times.

[0171] For example, when the pattern of speech 195 matches the pattern stored in speech data 112 and the user's speech intention of the pattern is a subsequent speech intention, the electronic device 101 determines the EPD time as an EPD time slower than the initially set EPD time (e.g., an EPD time of the fifth length). For example, when the user's speech intention of the pattern is a subsequent speech intention, the user delays the end of speech 195 so that speech 195 may not end for a long time. When the pattern of speech 195 matches a specific pattern stored in speech data 112 and the user's speech intention of the pattern is a subsequent speech intention, the electronic device 101 can determine that speech 195 is not completed, and then determine the EPD time of the fifth length, which is the slowest of multiple set EPD times. For example, the specific pattern corresponding to the fifth length of the EPD time is determined by the user's input. For example, the specific pattern corresponding to the extremely slow EPD time is determined based on the user's speech history.

[0172] According to yet another embodiment, in operation 440, the electronic device 101 may determine the EPD time according to the similar pattern. For example, in operation 440, the electronic device 101 may determine the EPD time according to the similar pattern.

[0173] For example, in operation 440, according to the similarity pattern and / or the utterance intention of the user of the similarity pattern, the electronic device 101 determines the EPD time to be an EPD time slower than the initially set EPD time (e.g., an EPD of the fourth length or the fifth length) or an EPD time faster than the initially set EPD time (e.g., an EPD of the first length or the second length). The description of operation 435 can be applied to operation 440 in substantially the same manner.

[0174] For example, when the pattern of speech 195 is similar to the pattern stored in speech data 112 and the speech intention of the user of the similar pattern is the speech end intention, the electronic device 101 determines that speech 195 is completed and determines the EPD time as an EPD time of a second length that is faster than the initially set EPD time.

[0175] For example, when the pattern of speech 195 is similar to the pattern stored in speech data 112 and the speech intention of the user of the similar pattern is a subsequent speech intention, the electronic device 101 determines that speech 195 is not completed and therefore determines the EPD time as an EPD time of a third length that is slower than the initially set EPD time.

[0176] According to another embodiment, the electronic device 101 can set the EPD time corresponding to each sentence pattern. For example, the electronic device 101 sets the EPD time of the first length and the second length for sentence pattern 1, and can set the EPD time of the fifth length and the fourth length for sentence pattern 2.

[0177] For example, when the pattern of the utterance 195 is substantially the same as the sentence pattern 1 in operation 415, the electronic device 101 determines the EPD time as the first length of the EPD time in operation 435. For example, when the pattern of the utterance 195 is similar to the sentence pattern 1 in operation 420, the electronic device 101 determines the EPD time as the second length of the EPD time in operation 440.

[0178] For example, when the pattern of the utterance 195 is substantially the same as the sentence pattern 2 in operation 415, the electronic device 101 determines the EPD time as the fifth length of the EPD time in operation 435. For example, when the pattern of the utterance 195 is similar to the sentence pattern 2 in operation 420, the electronic device 101 determines the EPD time as the fourth length of the EPD time in operation 440.

[0179] In the above example, the EPD time of the first length set for sentence pattern 1 may correspond to the EPD time set for a user utterance having a pattern substantially the same as sentence pattern 1. For example, the EPD time of the second length set for sentence pattern 1 corresponds to the EPD time set for a user utterance having a pattern similar to sentence pattern 1. The description given above may also be applied to the EPD times of the fifth length and the fourth length set in sentence pattern 2.

[0180] According to another embodiment, the electronic device 101 may determine the EPD time according to the complete mode in operation 445. For example, when the sentence type of the utterance 195 is a complete sentence type or a clear complete sentence type, the electronic device 101 determines the EPD time as an EPD time (e.g., a first length or a second length) that is faster than the initially set EPD time.

[0181] According to another embodiment, the electronic device 101 may determine the EPD time according to the incomplete mode in operation 450. For example, when the sentence type of the utterance 195 is an incomplete sentence type or a clear incomplete sentence type, the electronic device 101 determines the EPD time as an EPD time (e.g., the fourth length or the fifth length) slower than the initially set EPD time.

[0182] According to another embodiment, in operation 455, the electronic device 101 may determine the EPD time according to a preset EPD time. For example, when the sentence type of the utterance 195 is an ambiguous sentence type, the electronic device 101 determines the EPD time as the initially set EPD time.

[0183] The classification of the first length to the fifth length is illustrative and not limited thereto. For example, the length of each EPD time may be classified into the first length to the third length, and the lengths are set as (first length)>(second length)>(third length).

[0184] exist Figure 4 and Figure 5 In the embodiment of the present invention, the electronic device 101 can determine the EPD time using the speech data 112 generated based on the speech history of the user. Considering the user's speech ending intention or the user's subsequent speech intention, the electronic device 101 can determine the EPD time based on the pattern of the speech 195 frequently input by the user, and can detect the end point of the speech 195.

[0185] For example, when the user finishes the utterance 195 , the electronic device 101 determines the EPD time to be faster or slower than the initially set time, and terminates the recognition of the utterance 195 in advance to perform an operation according to the input utterance 195 .

[0186] Figure 4 and Figure 5 may be an example of operations performed by the electronic device 101 to determine the EPD time. However, the order of operations is not limited thereto, and some operations may be omitted.

[0187] Figure 6 is a diagram showing speech data (eg, Figure 3 FIG. 112 ) of the speech data.

[0188] In an example, an electronic device (e.g., Figure 2 The electronic device 101) can generate the following by using the user's speech history previously input Figure 6 For example, the electronic device 101 receives the speech data 112 according to the previously input speech (e.g., Figure 3The utterance data 112 is generated by inputting a count 510 of an utterance 195), an input date 520, a category 530 of the utterance, and a count 540 of the category. For example, the count 510 of the utterance input refers to how many times the applicable utterance 195 is input, and the count 540 of the category input may refer to how many times the applicable category is input. Figure 6 , the music category 530 - 1 is input 33 times, the weather category 530 - 2 may be input 20 times, and the call category 530 - 3 may be input 10 times.

[0189] For example, the input date 520 indicates a date on which the applicable utterance 195 was input or the last date on which the applicable utterance 195 was input.

[0190] Figure 7 is a diagram showing a method for using speech data (eg, Figure 3 The speech data 112) registers the speech (for example, Figure 3 Diagram of the user interface of the speech 195).

[0191] In an example, an electronic device (e.g., Figure 2 The electronic device 101) may provide a user interface for registering speech 195, such as Figure 7 For example, the electronic device 101 includes a display module (eg, Figure 2 The display module 160 is provided by using the display module 160. Figure 7 User interface 700 is shown.

[0192] For example, the electronic device 101 generates speech data 112 based on the user's input. Figure 7 , sentences such as "today's weather" 710-1, "play BTS songs" 710-2, "tomorrow's weather" 710-3, "play IU songs" 710-4, and "latest news" 710-5 can be input by using the user speech registration interface 710. The electronic device 101 can be input by entering Figure 7 The sentences 710-1 to 710-5 shown as the registered items of the utterance 195 are registered as the utterance 195 to generate the utterance data 112. For example, the electronic device 101 generates the following utterance data using the registered utterance 195: Figure 6 Speech data 112 is shown.

[0193] In the example of the display module 160, the electronic device 101 may update the speech data 112 according to a preset update standard. For example, the update standard includes a newly input standard such as a specific time period and a count of the speech 195. For example, the electronic device 101 may store the input speech 195 in a storage space (e.g., Figure 2When the count of newly stored utterances is greater than the set count or every predetermined period, the electronic device 101 may process the stored utterances 195 and update the utterance data 112.

[0194] For example, the electronic device 101 may display a display object 720 for setting a user input (such as “immediate execution” or “reserved execution”) on the display module 160, such as Figure 7 As shown. The electronic device 101 updates the speech data 112 based on the user's input to the settings (such as "execute now" and "keep for execution"). For example, when the "execute now" input is received from the user, the electronic device 101 processes the speech 195 stored in the storage space and updates the speech data 112. For example, when the "keep for execution" input is received from the user, the electronic device 101 updates the speech data 112 by processing the speech 195 stored in the storage space at the set time.

[0195] Figure 8 is a diagram illustrating categories of utterances according to an embodiment of the present disclosure.

[0196] For example, electronic devices (e.g. Figure 2 The electronic device 101) can be used to input previously input speech (e.g., Figure 3 195) to perform clustering into multiple categories. For example, refer to Figure 8 , the electronic device 101 can cluster the categories of the pre-input utterance 195 into multiple categories, such as music, weather, alarm clock, and call.

[0197] For example, the electronic device 101 determines the category of the input user speech. For example, the electronic device 101 determines the category of the speech 195 as the newly input "play song A" as music in the category, such as Figure 8 shown.

[0198] For example, according to an embodiment, Figure 8 The categories of the utterances shown in represent the categories of the utterances clustered by the electronic device 101 based on the K-means algorithm. For example, the electronic device 101 randomly generates certain K centroids and includes each point in the cluster of the nearest centroid. The electronic device 101 can calculate a new centroid using the average value of the points included in each cluster.

[0199] For example, the electronic device 101 performs sentence embedding clustering using a K-means algorithm. For example, the electronic device 101 may include a sentence embedding clustering module.

[0200] For example, initially K centroids are generated according to a certain number (e.g., K) of utterance categories. Figure 2When more than a set number of utterances are stored in the utterance data 112 of the electronic device 101, the electronic device 101 may use the sentence embedding clustering module to update the utterance category. For example, the electronic device 101 includes the points according to the stored utterance data 112 in the cluster of the nearest centroid, and may update the centroid of each cluster. For example, the electronic device 101 uses a pre-trained sentence embedding vector table to express the sentence corresponding to the utterance data 112 as a vector. The electronic device 101 may use various methods other than the above-mentioned K-means algorithm to sort the utterance categories of the utterance data 112, and the utterance categories are not limited to Figure 8 Those discourse categories shown in .

[0201] Fig. 9 is a diagram illustrating category ratios of utterances according to an embodiment of the present disclosure.

[0202] For example, electronic devices (e.g. Figure 2 The electronic device 101 may cluster previously input utterances (e.g., Figure 3 195), such as Figure 8 as shown, and can be Fig. 9 The shown represents the proportion of each category. Fig. 9 , music categories (e.g. Fig. 9 Music) accounts for about 49%, and the call category ( Fig. 9 The call in Fig. 9 ) accounted for about 29%, and the weather category ( Fig. 9 Weather in the Fig. 9 The proportion of alarms in the system is about 7%.

[0203] For example, the electronic device 101 determines the category of the user's speech based on the ratio of the categories. Figure 8 When the distance to the centroid of the music category is the same as the distance to the centroid of the alarm category, the electronic device 101 performs clustering according to the following example: Fig. 9 The proportion of each category shown applies a weight to determine the category of the utterance 195. For example, the electronic device 101 determines the category of the utterance 195 by assigning a greater weight to a category having a greater proportion.

[0204] For example, in Fig. 9 In the category proportions shown, as a result of clustering the utterance 195, when the distance to the centroid of the music category is the same as the distance to the centroid of the alarm category, the electronic device 101 determines the category of the utterance 195 as the music category because the proportion of the music category (approximately 49%) is greater than the proportion of the alarm category (approximately 7%).

[0205] Fig.10is a diagram showing sentence data (for example, Figure 3 Figure 113 of sentence data.

[0206] refer to Fig.10 , according to the electronic device of the embodiment (for example, Figure 2 The electronic device 101 may be based on speech data (e.g., Figure 3 For example, the electronic device 101 uses Figure 6 The speech data 112 shown generates Fig.10 The sentence data 113 shown is shown. For example, the segments of the sentence data 113 are classified according to the utterance, the count of the utterance, and the category.

[0207] Fig.11 is a diagram showing word data (eg, Figure 3 A diagram of word data 114).

[0208] refer to Fig.11 , according to the electronic device of the embodiment (for example, Figure 2 The electronic device 101 may be based on speech data (e.g., Figure 3 The speech data 112) generates word data 114. For example, the electronic device 101 uses Figure 6 The speech data 112 shown is used to generate Fig.11 The word data 114 is shown. For example, segments of the word data 114 are categorized according to utterances, counts of utterances, categories, and types.

[0209] For example, the type corresponding to a word is set to complete, clear complete, incomplete, clear incomplete, or ambiguous.

[0210] For example, the electronic device 101 uses the last piece of text corresponding to the utterance 195 and the word data 114 to determine the utterance (e.g., Figure 3 For example, the ASR module 221 generates a text corresponding to the speech 195 according to the speech 195.

[0211] For example, when the type of word corresponding to the last text is complete, the electronic device 101 determines the sentence type of the utterance 195 as a complete sentence. For example, when the type of word corresponding to the last text is clear and complete, the electronic device 101 determines the sentence type of the utterance 195 as a clear and complete sentence.

[0212] For example, when the type of the word corresponding to the last text is incomplete, the electronic device 101 determines the sentence type of the utterance 195 as an incomplete sentence. For example, when the type of the word corresponding to the last text is clearly incomplete, the electronic device 101 determines the sentence type of the utterance 195 as a clearly incomplete sentence.

[0213] For example, when the type of the word corresponding to the last text is ambiguous, the electronic device 101 determines the sentence type of the utterance 195 as an ambiguous sentence. For example, the electronic device 101 determines the EPD time according to the sentence type of the utterance 195. For example, when the sentence type of the utterance 195 is a clear complete sentence, a complete sentence, an incomplete sentence, or a clear incomplete sentence, the electronic device 101 determines the EPD time as an extremely fast EPD time (e.g., about 100ms), a fast EPD time (e.g., about 300ms), a slow EPD time (e.g., about 800ms), or an extremely slow EPD time (e.g., about 1,000ms).

[0214] For example, the electronic device 101 determines the type of a word using the utterance data 112. For example, when an utterance 195 different from the previous utterance is always input after word A, the electronic device 101 determines the type of word A as clear and incomplete. For example, when a different utterance 195 is not always input after word B, the electronic device 101 determines the type of word B as clear and complete.

[0215] For example, the electronic device 101 determines the type of word C based on the frequency of inputting different utterances 195 after word C. For example, the electronic device 101 determines the type of word C by comparing the ratio of inputting different utterances 195 after word C with the ratio of inputting non-different utterances 195 after word C.

[0216] For example, by using a plurality of preset ratios and by using a ratio of different utterances 195 input after word C and a ratio of no different utterances 195 input after word C, the type of word C is determined as at least one of clear and complete, complete, incomplete, clear and incomplete, and ambiguous.

[0217] For example, refer to Figures 6 to 11 , the electronic device 101 determines whether the pattern of the utterance 195 is substantially the same as the pattern stored in the utterance data 112. When the pattern of the utterance 195 is substantially the same as the pattern stored in the utterance data 112, the electronic device 101 may determine the EPD time according to the pattern.

[0218] For example, when the speech 195 "play the song of singer A" is input, the electronic device 101 determines the category of the speech 195 as "music", such as Figure 8 shown.

[0219] For example, the electronic device 101 uses the words 195 and 196 of "playing the song of singer A". Fig.10 The electronic device 101 compares the pattern of the speech 195 with the pattern stored in the speech data 112 by using the sentence data 113 of the "music" category. For example, the electronic device 101 determines that the pattern of the speech 195 of "playing the song of singer A" is the same as Fig.10 The pattern of “play XX's song” in the sentence data 113 is basically the same.

[0220] For example, when "playing songs of singer A" is substantially the same as the pattern of "playing songs of XX" stored in the speech data 112, the electronic device 101 determines the EPD time according to the matching pattern. For example, the electronic device 101 determines the EPD time to be extremely fast (e.g., about 100 ms) according to the matching pattern.

[0221] For example, refer to Figures 6 to 11 , the electronic device 101 determines whether the pattern of the utterance 195 is similar to the pattern stored in the utterance data 112. When the pattern of the utterance 195 is similar to the pattern stored in the utterance data 112, the electronic device 101 can determine the EPD time according to the pattern.

[0222] For example, when the speech 195 of "the weather in city A" is input, the electronic device 101 determines the category of the speech 195 as "weather", such as Figure 8 shown.

[0223] For example, the electronic device 101 uses the utterance 195 and Fig.10 The electronic device 101 compares the pattern of the utterance 195 with the pattern stored in the utterance data 112 by using the sentence data 113 of the "weather" category of the utterance 195. For example, the electronic device 101 determines that the pattern of the utterance 195 of "the weather in city A" is Fig.10 The pattern of "weather XX" of sentence data 113 is similar.

[0224] For example, the electronic device 101 uses the matching probability between the pattern of the utterance 195 and the pattern stored in the utterance data 112 to determine whether the pattern of the utterance 195 is substantially the same as, similar to, or dissimilar to the pattern stored in the utterance data 112. For example, the electronic device 101 calculates the matching probability between the pattern of the utterance 195 and the pattern stored in the utterance data 112. For example, the matching probability represents the similarity between the pattern of the utterance 195 and the pattern stored in the utterance data 112.

[0225] For example, when the matching probability is greater than a preset first threshold, the electronic device 101 may determine that the pattern of the speech 195 is substantially the same as the pattern stored in the speech data 112 .

[0226] For example, when the matching probability is equal to or less than a preset first threshold and greater than a preset second threshold, the electronic device 101 determines that the pattern of the speech 195 is similar to the pattern stored in the speech data 112. The magnitude of the second threshold may be set to be smaller than the magnitude of the first threshold.

[0227] For example, when the matching probability is equal to or less than a preset second threshold, the electronic device 101 may determine that the pattern of the speech 195 is not similar to the pattern stored in the speech data 112 .

[0228] For example, when the pattern of the speech 195 of "weather in city A" is similar to the pattern of "weather XX" stored in the speech data 112, the electronic device 101 determines the EPD time according to the matching pattern. For example, the electronic device 101 determines the EPD time as a fast EPD time (e.g., about 200ms) according to the similar pattern.

[0229] For example, when the utterance 195 of "singer A's song" is input, the electronic device 101 determines that the pattern of the utterance 195 of "singer A's song" is similar to the pattern of "play XX's song" or "XX's song" stored in the sentence data 113. When the utterance 195 of "singer A's song" is input, the electronic device 101 determines the fast EPD time according to the similar pattern.

[0230] When the utterance 195 of "song by singer A" is input, even when the sentence type of the utterance 195 is incomplete, the electronic device 101 can determine the user's intention based on the pattern of the utterance 195. Even when an incomplete sentence is input, the electronic device 101 can determine the utterance end intention or the subsequent utterance intention based on the pattern of the utterance 195 and determine the EPD time.

[0231] FIG. 12A to FIG. 12G FIG. 2 is a diagram showing an electronic device (eg, Figure 2 The electronic device 101) for each utterance (eg, Figure 3 195) to determine the EPD time diagram.

[0232] FIG. 12A to FIG. 12G The EPD time shown in can be determined by the electronic device 101 based on the speech 195 and the speech data (e.g., Figure 3 The speech data 112) is used to determine.

[0233] refer to Fig. 12A , when the speech 195 of “call mom” is input, the electronic device 101 can use the speech data 112 to determine the EPD time.

[0234] For example, when the pattern and words of the utterance 195 of "Call Mom" ​​are substantially the same as the pattern and words stored in the utterance data 112, the electronic device 101 determines the EPD time as an extremely fast EPD time (eg, approximately 100 ms).

[0235] For example, in Fig. 12A , the electronic device 101 uses the utterance 195 and the utterance data 112 to determine that the utterance 195 is completed according to the user's intention, and determines that the EPD time is extremely fast.

[0236] refer to Fig. 12B and Fig. 12C , when an utterance 195 such as "what is the" or "today's lunch" is input, the electronic device 101 determines the EPD time using the utterance data 112.

[0237] For example, by comparing the pattern and words of the utterance 195 of “what is the” or “today's lunch” with the pattern and words stored in the utterance data 112, the electronic device 101 determines the EPD time as a slow EPD time (eg, approximately 1,000 ms).

[0238] For example, the electronic device 101 predicts the occurrence of the subsequently input utterance 195 based on a pattern stored in the utterance data 112 (which is similar to the pattern of "what is the" or "today's lunch"), and determines that the EPD time is slow.

[0239] For example, the electronic device 101 determines the sentence type of the utterance 195 by using the last word "the" from "what is the" or "lunch" from "today's lunch". For example, when the types of the words "the" and "lunch" stored in the word data 114 are incomplete or clearly incomplete, the electronic device 101 determines that the utterance 195 is incomplete.

[0240] For example, when an utterance 195 (such as “abc” or “what”) is additionally input within a determined EPD time (for example, approximately 1,000 ms), the electronic device 101 recognizes the additionally input utterance 195 .

[0241] For example, when an additional utterance 195 is input, the electronic device 101 re-determines the EPD time using the newly input utterance 195. Fig. 12B, the electronic device 101 uses the utterance 195 of "what is the" as the previous input and the utterance 195 of "abc" as the additional input to compare the pattern of the utterance 195 with the stored pattern. For example, when the pattern of the utterance 195 is substantially the same as the stored pattern, the electronic device 101 determines the EPD time as a fast EPD time (e.g., about 200 ms), such as Fig. 12B shown.

[0242] For example, the electronic device 101 uses the newly input utterance 195 of "abc" to determine the EPD time according to the type of the last word "abc".

[0243] For example, in Fig. 12C In the example, the electronic device 101 compares the pattern of the utterance 195 with the stored pattern by using the previously input utterance 195 of "today's lunch" and the additionally input utterance 195 of "what". For example, when the pattern of the utterance 195 is similar to the stored pattern, the electronic device 101 determines the EPD time as a fast EPD time (e.g., about 200 ms), such as Fig. 12B shown.

[0244] For example, the electronic device 101 receives the speech 195 as a whole by using the previously input speech 195 and the additionally input speech 195. For example, the electronic device 101 recognizes the speech 195, such as Fig. 12B and Fig. 12C "what is theabc" and "what is the today's lunch" in.

[0245] exist Fig. 12B and Fig. 12C , the electronic device 101 determines whether the patterns of the user's utterances 195 "what is the abc?" and "what is the today's lunch?" are substantially the same as, similar to, or dissimilar to the stored patterns by using the patterns of the utterances 195. The electronic device 101 can determine the EPD time based on whether the pattern of the utterance 195 is the same as, similar to, or dissimilar to the stored patterns and based on the user's utterance intention for the stored patterns.

[0246] exist Fig. 12C In , utterance 195 may correspond to an incomplete sentence, and the pattern of utterance 195 is similar to the stored pattern. Fig. 12CIn the example, the user's speech intention of the stored pattern similar to the speech 195 of "what is the lunch today?" may be a speech end intention. The electronic device 101 may determine the user's intention (e.g., speech end intention or subsequent speech intention) according to the stored pattern, and may determine the EPD time as a fast EPD time (e.g., about 200 ms).

[0247] For example, in Fig. 12C , when a user utterance intention of a storage mode similar to the utterance 195 of “what is the today's lunch?” is a subsequent utterance intention, the electronic device 101 may determine the EPD time as a slow EPD time (eg, approximately 1,000 ms) according to the subsequent utterance intention.

[0248] refer to Fig.12D and Fig.12E , the electronic device 101 can determine that the EPD time is an extremely slow EPD time by comparing the pattern of the utterance 195 with the stored pattern. For example, the electronic device 101 determines that the EPD time is an extremely slow EPD time (e.g., about 2,000 ms) by comparing the pattern of the user's utterances 195 "Baby" and "Hey" with the pattern of the utterance 195 stored in the utterance data 112 and then input after "Baby" and "Hey".

[0249] For example, the electronic device 101 determines the EPD time by using the frequency or count of the pattern of the utterance 195 input subsequently after “Baby” and “Hey” stored in the utterance data 112. For example, the greater the frequency of the pattern of the utterance 195 input subsequently after “baby” and “hey” stored in the utterance data 112, the electronic device 101 determines the EPD time to be a slower EPD time.

[0250] refer to Fig.12D and Fig.12E , when the utterance 195 is a clear incomplete sentence, the electronic device 101 may determine the EPD time as an extremely slow EPD time. Fig.12D and Fig.12E In the example, when the types of the words "baby" and "hey" stored in the word data 114 are clear and incomplete, the electronic device 101 may determine the EPD time as an extremely slow EPD time (eg, approximately 2,000 ms).

[0251] For example, when the pattern of the utterance 195 is an incomplete sentence with a high utterance frequency or a clear incomplete sentence, the electronic device 101 may determine the EPD time as an extremely slow EPD time (e.g., about 2,000 ms). For example, based on the pattern of the utterance 195, the electronic device 101 may determine the user's utterance intention as a subsequent utterance intention in the case of an incomplete word or a word with a high utterance frequency. In the case of an incomplete word or a word with a high utterance frequency, the electronic device 101 may determine the EPD time as a slow EPD time (e.g., about 2,000 ms).

[0252] exist Fig.12D and Fig.12E In the example, when the utterance 195 of "call mom" is additionally input within the EPD time (about 2,000 ms), the electronic device 101 can determine a new EPD time (for example, about 100 ms). Considering the additional input utterance 195, the determination of the new EPD time can be compared with the reference Fig. 12B and 12C The descriptions provided are essentially the same.

[0253] refer to Fig.12F and Figure 12G , the pattern of the utterance 195 of "Suwon" or "hmm" may not be substantially the same as or similar to the stored pattern. The sentence type of the utterance 195 of "Suwon" or "hmm" may correspond to an ambiguous sentence. When the sentence type of the utterance 195 is an ambiguous sentence, the electronic device 101 may determine the EPD time as a preset EPD time (e.g., about 600ms).

[0254] exist Fig.12F and Figure 12G , when the utterance 195 of “weather” or “YouTube” is additionally input after the utterance 195 of “Suwon” or “hmm” within the EPD time (e.g., about 600ms), the electronic device 101 may determine a new EPD time (e.g., about 100ms).

[0255] exist Fig.12F and Figure 12G According to an embodiment, the electronic device 101 may determine the EPD time based on the user's speech intention. Fig.12F and Figure 12G All utterances 195 of "Suwon weather" and "hmm YouTube" in may correspond to incomplete sentences that are clear in terms of language grammar.

[0256] For example, even when the utterance 195 is a clear incomplete sentence in terms of language grammar, the electronic device 101 can determine the user's utterance intention based on a pattern that is substantially the same as or similar to the utterance 195. Based on the user's utterance intention, even when the utterance 195 is a clear incomplete sentence, the electronic device 101 can determine the EPD time as a fast EPD time or an extremely fast EPD time. Fig.12F and Figure 12G As shown, even when the utterance 195 is a clear incomplete sentence, the electronic device 101 can determine the EPD time as an extremely fast EPD time (e.g., about 100 ms) in consideration of the user's utterance intention.

[0257] exist Fig.12F and Figure 12G In the example, the user utterance intention of the utterance 195 may be the utterance ending intention. For example, the electronic device 101 may determine the user utterance intention of the utterance 195 based on the utterance data 112. Fig.12F In the example, when the user's utterance intention having a pattern substantially the same as or similar to the utterance 195 of “Suwon Weather” is an utterance end intention, the electronic device 101 may determine the user's utterance intention as an utterance end intention.

[0258] For example, the electronic device 101 may use the speech pattern of the user of the speech data 112 to determine the user's speech intention of the pattern. For example, when pattern A is repeatedly input without a subsequent speech input, the electronic device 101 may determine that the user's speech intention corresponding to pattern A is a speech end intention. For example, when pattern B is repeatedly input together with a subsequent speech input, the electronic device 101 may determine that the user's speech intention corresponding to pattern B is a subsequent speech intention.

[0259] For example, the electronic device 101 may compare a preset threshold with the input count of the pattern stored in the speech data 112, and the electronic device 101 may determine the user's speech intention for the pattern. For example, when pattern A is input more than a set threshold (e.g., 10 times) without subsequent speech being input, the electronic device 101 may determine that the user's speech intention corresponding to pattern A is a speech end intention.

[0260] For example, when pattern A is input multiple times without inputting a subsequent utterance and is input B times together with a subsequent utterance, the electronic device 101 can determine the utterance intention of the user corresponding to pattern A by using A times of inputting pattern A without a subsequent utterance, B times of inputting pattern A with a subsequent utterance, and a set threshold. For example, when (AB) is greater than or equal to the set threshold, the electronic device 101 can determine that the utterance intention of the user corresponding to pattern A is an utterance end intention. For example, when (BA) is greater than or equal to the set threshold, the electronic device 101 can determine that the utterance intention of the user corresponding to pattern A is a subsequent utterance intention.

[0261] Figures 13A to 13C 1 is a diagram showing an electronic device (eg, Figure 2 The electronic device 101) responds to the EPD command (for example, Figure 3 The EPD command 115) determines the operation of the EPD time.

[0262] In an example, the EPD command 115 may include at least one of a force EPD command 1310 , a long pause command 1320 , and a reset command 1330 .

[0263] For example, the forced EPD command 1310 (e.g., the speech “end”, “terminate”, “finish”, etc.) and the long pause command 1320 (e.g., “for a moment”, “slow down”, “wait”, “hold” speech, etc.) may refer to the termination of the recognition speech according to the preset length of EPD time when the forced EPD command 1310 or the long pause command 1320 is input to the electronic device 101 (e.g., Figure 3 195) of the command.

[0264] For example, the length of the EPD time corresponding to the forced EPD command 1310 may be set to be extremely fast (eg, approximately 100 ms), and the length of the EPD time corresponding to the long pause command 1320 may be set to be extremely slow (eg, approximately 2,000 ms).

[0265] refer to Fig.13A When the forced EPD command 1310 is input to the electronic device 101 together with the speech 1, the electronic device 101 may determine the EPD time to be a preset length (eg, approximately 100 ms). Fig.13A In the example, the electronic device 101 can determine the user's speech intention as the speech end intention based on the forced EPD command 1310. The electronic device 101 can terminate the recognition of the speech 195 earlier than the initially set EPD time (e.g., about 600ms) according to the speech end intention, and can quickly perform an operation according to the input speech 195.

[0266] refer to Fig. 13B When the long pause command 1320 is input to the electronic device 101 together with the speech 1, the electronic device 101 may determine the EPD time to be a preset length (eg, approximately 2,000 ms). Fig. 13B , the electronic device 101 may determine the user's speech intention as a subsequent speech intention based on the long pause command 1320. The electronic device 101 may pause the termination of the recognition of the speech 195 according to the intention of the next speech exceeding the initially set EPD time (e.g., about 600ms). The electronic device 101 may recognize the speech 2 input within the preset EPD time (e.g., about 2,000ms) and the previously input speech 1 as speech 195 according to the intention of the next speech.

[0267] For example, the reset command 1330 may refer to a command for ignoring the speech 195 that is input before the reset command 1330 is input to the electronic device 101 .

[0268] For example, refer to Fig. 13C , when utterance 1, a reset command 1330, and then utterance 2 are input, the electronic device 101 may determine the EPD time by using only utterance 2 input before the reset command 1330 (excluding utterance 1).

[0269] For example, in Fig. 13C In the example, the electronic device 101 may exclude the utterance 1 input before the reset command 1330 from the subsequent voice recognition operation. Fig. 13C In the ASR module (for example, Figure 2 The ASR module 221 of utterance 2 may send only the text corresponding to utterance 2 to the NLU module (e.g., Figure 2 NLU module 223) to perform subsequent speech recognition operations.

[0270] For example, the ASR module 221 may send the utterance 195 to the NLU module 223 based on the long pause command 1320 or the reset command 1330 .

[0271] For example, in Fig. 13B , the ASR module 221 may transmit utterance 1 and utterance 2 to the NLU module 223 .

[0272] For example, in Fig. 13C In the example, the ASR module 221 may not transmit the utterance 1 input before the reset command 1330 to the NLU module 223. For example, the ASR module 221 may transmit the utterance 2 input after the reset command 1330 to the NLU module 223.

[0273] According to the electronic device (for example, Figure 2The electronic device 101) may be electrically connected to a processor (eg, Figure 2 120), and may include a memory configured to store commands executed by the processor 120 (e.g., Figure 2 When executing the command, the processor 120 may be configured to recognize the speech (e.g., Figure 3 The processor 120 may be configured to process the speech data based on the speech data stored in the memory 130 (e.g., Figure 3 The processor 120 may be configured to terminate the recognition of the utterance 195 when the utterance 195 is not input within the EPD time. The utterance data 112 may include sentence data containing a pattern related to the sentence type of the utterance 195 (e.g., Figure 3 sentence data 113) and word data containing words of utterance 195 (e.g., Figure 3 word data 114).

[0274] The processor 120 may be configured to determine a category of the utterance 195. The processor 120 may be configured to determine an EPD time using the sentence data 113 and the word data 114 corresponding to the category.

[0275] Processor 120 may be configured to determine whether utterance 195 is substantially identical to a pattern stored in utterance data 112 by using utterance 195 and utterance data 112. Processor 120 may be configured to determine an EPD time based on whether utterance 195 is substantially identical to a pattern.

[0276] The processor 120 may be configured to determine whether the utterance 195 is similar to a pattern stored in the utterance data 112 by using the utterance 195 and the utterance data 112. The processor 120 may be configured to determine an EPD time based on whether the utterance 195 is similar to the pattern.

[0277] Processor 120 may be configured to determine the EPD time based on the frequency of the pattern.

[0278] Processor 120 may be configured to determine whether utterance 195 is a complete sentence using utterance 195 and utterance data 112. Processor 120 may be configured to determine an EPD time based on whether utterance 195 is a complete sentence.

[0279] Processor 120 may be configured to determine whether utterance 195 is an incomplete sentence using utterance 195 and utterance data 112. Processor 120 may be configured to determine an EPD time based on whether utterance 195 is an incomplete sentence.

[0280] Processor 120 may be configured to recognize whether utterance 195 includes a preset EPD command (e.g., Figure 3 The processor 120 may be configured to determine the EPD time according to the EPD command 115.

[0281] The processor 120 may be configured to update the utterance data 112 according to a preset update criterion.

[0282] According to the electronic device (for example, Figure 2 The electronic device 101) may be electrically connected to a processor (eg, Figure 2 120), and may include a memory configured to store commands executed by the processor 120 (e.g., Figure 2 The processor 120 may be configured to determine the category of the speech when executing the command. Based on the speech data corresponding to the category (e.g., Figure 3 112), the processor 120 may be configured to convert the speech (e.g., Figure 3 The processor 120 may be configured to determine an EPD time for terminating recognition of the utterance 195 based on at least one of the following: a result of comparing the pattern of the utterance 195 with the pattern stored in the utterance data 112, and whether the utterance 195 is a complete sentence, an incomplete sentence, or an ambiguous sentence; and when no additional utterance 195 is input within the EPD time, the processor 120 may be configured to terminate recognition of the utterance 195. The utterance data 112 may include sentence data (e.g., Figure 3 sentence data 113) and word data (e.g., Figure 3 The word data 114 includes the patterns related to the sentence type of the utterance 195, and the word data includes the words of the utterance 195.

[0283] Processor 120 may be configured to determine whether utterance 195 is substantially identical to a pattern stored in utterance data 112 by using utterance 195 and utterance data 112. Processor 120 may be configured to determine an EPD time based on whether utterance 195 is substantially identical to a pattern.

[0284] The processor 120 may be configured to determine whether the utterance 195 is similar to a pattern stored in the utterance data 112 by using the utterance 195 and the utterance data 112. The processor 120 may be configured to determine an EPD time based on whether the utterance 195 is similar to the pattern.

[0285] Processor 120 may be configured to determine the EPD time based on the frequency of the pattern.

[0286] A method for determining an EPD time according to an embodiment may include recognizing an utterance (e.g., Figure 3 195), based on the words stored in the memory (e.g., Figure 2 130) in the speech data (e.g., Figure 3 The utterance data 112) determines an EPD time for terminating recognition of the utterance 195, and when no further utterance 195 is input within the EPD time, terminates recognition of the utterance 195. The utterance data 112 may include sentence data containing a pattern related to the sentence type of the utterance 195 (e.g., Figure 3 sentence data 113) and word data containing words of utterance 195 (e.g., Figure 3 word data 114).

[0287] The determination of the EPD time may further include determining the category of the utterance. The determination of the EPD time may determine the EPD time using the sentence data 113 and the word data 114 corresponding to the category.

[0288] The determination of the EPD time may include using the utterance 195 and the utterance data 112 to determine whether the utterance 195 is identical to a pattern stored in the utterance data 112. The determination of the EPD time may include determining the EPD time based on whether the utterance 195 is substantially identical to the pattern.

[0289] The determination of the EPD time may include determining whether the utterance 195 is similar to a pattern stored in the utterance data 112 by using the utterance 195 and the utterance data 112. The determination of the EPD time may include determining the EPD time based on whether the utterance 195 is similar to a pattern.

[0290] The determination of the EPD time may include determining whether the utterance 195 is a complete sentence using the utterance 195 and the utterance data 112. The determination of the EPD time may include determining the EPD time based on whether the utterance 195 is a complete sentence.

[0291] The determination of the EPD time may include determining whether the utterance 195 is an incomplete sentence using the utterance 195 and the utterance data 112. The determination of the EPD time may include determining the EPD time based on whether the utterance 195 is an incomplete sentence.

[0292] The electronic device according to the embodiment may be one of various types of electronic devices. The electronic device may include, for example, a portable communication device (e.g., a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a household appliance. According to an embodiment of the present disclosure, the electronic device is not limited to those described above.

[0293] It should be understood that the embodiments of the present disclosure and the terms used therein are not intended to limit the technical features set forth herein to specific embodiments, and include various changes, equivalents, or replacements of the corresponding embodiments. As used herein, each of "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "at least one of A, B or C" may include any one of the items listed together in the corresponding one of the phrases, or all possible combinations thereof. Terms such as "first", "second", or "first" or "second" may be used simply to distinguish a component from other components being discussed, and do not limit the component in other respects (e.g., importance or order). It should be understood that if an element (e.g., a first element) is referred to as being "coupled," "coupled to," "connected to," or "connected to" another element (e.g., a second element) with or without the term "operably" or "communicatively," it means that the element can be coupled to the other element directly (e.g., by wire), wirelessly, or via a third element.

[0294] As used in conjunction with the embodiments of the present disclosure, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with other terms (e.g., "logic," "logic block," "component," or "circuit"). A module may be a single integrated component or its smallest unit or portion suitable for performing one or more functions. For example, according to an embodiment, a module may be implemented in the form of an application specific integrated circuit (ASIC).

[0295] The embodiments as described herein may be implemented as software (e.g., program 140) including one or more instructions stored in a storage medium (e.g., internal memory 136 or external memory 138) readable by a machine (e.g., electronic device 101). For example, a processor (e.g., processor 120) of a machine (e.g., electronic device 101) may call at least one of the one or more instructions stored in the storage medium and execute it. This allows the machine to be operated to perform at least one function according to at least one of the called instructions. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory" simply means that the storage medium is a tangible device and does not include signals (e.g., electromagnetic waves), but the term does not distinguish between a location where data is semi-permanently stored in the storage medium and a location where data is temporarily stored in the storage medium.

[0296] According to an embodiment, the method according to an embodiment of the present disclosure may be included and provided in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)), or via an application store (e.g., PlayStore). TM ) online distribution (e.g., download or upload), or directly between two user devices (e.g., smartphones). If distributed online, at least a portion of the computer program product may be temporarily generated or at least temporarily stored in a machine-readable storage medium, such as a memory of a manufacturer's server, an application store's server, or a relay server.

[0297] According to an embodiment, each component (e.g., a module or a program) in the above-mentioned components may include a single entity or multiple entities, and some of the multiple entities may be separately arranged in different components. According to an embodiment, one or more of the above-mentioned components may be omitted, or one or more other components may be added. Alternatively or additionally, multiple components (e.g., a module or a program) may be integrated into a single component. In this case, according to an embodiment, the integrated component may still perform one or more functions of each component in the multiple components in the same or similar manner as performed by a corresponding one of the multiple components before integration. According to an embodiment, the operations performed by a module, a program or another component may be performed sequentially, in parallel, repeatedly or heuristically, or one or more of the operations may be performed or omitted in a different order, or one or more other operations may be added.

[0298] While the present disclosure has been shown and described with reference to various embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents.

Claims

1. An electronic device (101), comprising: at least one processor (120); and a memory (130) electrically connected to the at least one processor (120) and configured to store commands executable by the processor (120), wherein the at least one processor (120) recognizes an utterance (195) when executing the command, determines an end point detection (EPD) time for terminating recognition of the utterance (195) based on the utterance data (112) stored in the memory (130), and terminates recognition of the utterance (195) when no further input of the utterance (195) occurs within the EPD time, and The speech data (112) includes: sentence data (113) comprising patterns associated with sentence types of the utterance (195), and Word data (114) comprising words of the utterance (195).

2. The electronic device (101) according to claim 1, wherein: The at least one processor (120) determines a category of the utterance, and determines the EPD time using the sentence data (113) and the word data (114) corresponding to the category.

3. The electronic device (101) according to any one of claims 1 and 2, wherein: The at least one processor (120) determines whether the utterance (195) is substantially the same as a pattern stored in the utterance data (112) by using the utterance (195) and the utterance data (112), and determines the EPD time based on whether the utterance (195) is substantially the same as the pattern.

4. The electronic device (101) according to any one of claims 1 to 3, wherein: The at least one processor (120) determines whether the utterance is similar to a pattern stored in the utterance data (112) by using the utterance (195) and the utterance data (112), and determines the EPD time based on whether the utterance (195) is similar to the pattern type.

5. The electronic device (101) according to any one of claims 1 to 4, wherein: The at least one processor (120) determines the EPD time based on a frequency of the pattern.

6. The electronic device (101) according to any one of claims 1 to 5, wherein: The at least one processor (120) uses the utterance (195) and the utterance data (112) to determine whether the utterance (195) is a complete sentence, and determines the EPD time based on whether the utterance (195) is a complete sentence.

7. The electronic device (101) according to any one of claims 1 to 6, wherein: The at least one processor (120) uses the utterance (195) and the utterance data (112) to determine whether the utterance (195) is an incomplete sentence, and determines the EPD time based on whether the utterance (195) is an incomplete sentence.

8. The electronic device (101) according to any one of claims 1 to 7, wherein: The at least one processor (120) identifies whether the speech (195) includes a preset EPD command (115); and determines the EPD time according to the EPD command (115).

9. The electronic device according to any one of claims 1 to 8, wherein: The at least one processor (120) updates the speech data (112) according to a preset update standard.

10. A method for determining an end point detection (EPD) time, the method comprising: identifying discourse (195); determining an EPD time for terminating recognition of the utterance (195) based on the utterance data (112) stored in the memory (130); as well as When no further input of the utterance (195) occurs within the EPD time, the recognition of the utterance (195) is terminated, The speech data (112) includes: sentence data (113) comprising patterns associated with sentence types of the utterance (195), and Word data (114) comprising words of the utterance (195).

11. The method according to claim 10, further comprising: Determine the category of the utterance, The determining of the EPD time includes determining the EPD time using the sentence data (113) and the word data (114) corresponding to the category.

12. The method according to any one of claims 10 and 11, wherein determining the EPD time comprises: determining whether the utterance (195) is substantially the same as a pattern stored in the utterance data (112) by using the utterance (195) and the utterance data (112); as well as The EPD time is determined based on whether the utterance (195) is substantially identical to the pattern.

13. The method according to any one of claims 10 to 12, wherein determining the EPD time comprises: determining whether the utterance (195) is similar to a pattern stored in the utterance data (112) by using the utterance (195) and the utterance data (112); as well as The EPD time is determined based on whether the utterance (195) is similar to the pattern.

14. The method according to any one of claims 10 to 13, wherein determining the EPD time comprises: using the utterance (195) and the utterance data (112) to determine whether the utterance (195) is a complete sentence; as well as The EPD time is determined according to whether the utterance (195) is the complete sentence.

15. The method according to any one of claims 10 to 14, wherein determining the EPD time comprises: using the utterance (195) and the utterance data (112) to determine whether the utterance (195) is an incomplete sentence; as well as The EPD time is determined according to whether the utterance (195) is an incomplete sentence.

Images