Adaptive speech endpoint detection method and apparatus

Abstract

An adaptive speech endpoint detection method according to a first aspect of the present invention comprises: a step of determining the context of a user’s speech through speech recognition of the user’s speech; a step of determining an end point detection (EPD) reference time for the speech based on the speech context; and a step of detecting an end point for the speech based on the end point detection reference time.

Description

Technology Field

[0001] The present invention relates to an adaptive utterance endpoint detection method and apparatus. Background Technology

[0002] A call bot refers to an artificial intelligence system that automatically processes voice conversations through a voice-based interface. Call bots can be utilized in call centers, customer service, or reservation systems, and can be designed to converse in real-time in a manner similar to humans.

[0003] Endpoint detection (EPD) technology can be utilized in real-time conversational scenarios involving callbots. Since the callbot's response process can be triggered when an endpoint is detected for a user's utterance and it is determined that the user has finished speaking, the criteria for determining the endpoint can be critical. For example, if the threshold for the endpoint detection time is set to a long duration, the silence observation time becomes longer, delaying the determination of the endpoint after the user's utterance. Consequently, the callbot's response utterance is delayed, leading the user to perceive the callbot system as slow. Conversely, if the threshold for the endpoint detection time is set to a short duration, the silence observation time becomes shorter, causing the endpoint to be determined earlier. Consequently, the callbot's response utterance is faster, leading the user to perceive the system as fast; however, this can result in the callbot's response utterance being executed even when the user's utterance has not actually ended.

[0004] Generally, a fixed silence time is used for endpoint detection regarding the aforementioned utterance; however, considering actual human conversation, the silence time is not fixed, which can lead to unnatural, inefficient, or degraded performance of the callbot due to incorrect endpoint judgment. The problem to be solved

[0005] The problem that the present invention aims to solve includes providing a technology in which the endpoint of an utterance is determined flexibly by considering the speaker's characteristics or the conversational situation.

[0006] However, the problems that the present invention aims to solve are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art to which the present invention pertains from the description below. means of solving the problem

[0007] An adaptive speech endpoint detection method according to a first aspect of the present invention comprises: a step of determining the context of a user’s speech through speech recognition of the user’s speech; a step of determining an end point detection (EPD) reference time for the speech based on the speech context; and a step of detecting an end point for the speech based on the end point detection reference time.

[0008] The above utterance context may include a first utterance context in which the user intends to continue the utterance, a second utterance context in which the user intends to end the utterance, or a third utterance context in which it is unclear whether the user will end the utterance.

[0009] The above endpoint detection reference time may include an initial silence maximum duration, a minimum trailing silence, or a maximum recognition time.

[0010] In the step of determining the endpoint detection reference time, if the utterance context is determined to be the first utterance context, the initial maximum silence duration, the minimum silence duration, or the maximum recognition time can be updated in a direction that increases.

[0011] In the step of detecting the above endpoint, if the utterance context is determined to be the second utterance context, the endpoint can be detected between the time when the utterance context is determined to be the second utterance context and the time when new speech recognition is performed after the time when the utterance context is determined to be the second utterance context.

[0012] In the step of determining the endpoint detection reference time, if the utterance context is determined to be the third utterance context, the initial maximum silence duration or the minimum silence duration can be updated in a direction that increases.

[0013] The above minimum silence duration may be applied in a situation where the endpoint is not detected for an extended period after the occurrence of a third utterance context.

[0014] The above maximum duration of the initial silence can be applied in a situation where endpoint detection occurs after the occurrence of a third utterance context and a new voice is input.

[0015] The above method may further include a step of determining the speech rate of the user. In this case, the degree to which the maximum recognition time is increased may be determined to be inversely proportional to the speech rate.

[0016] The above method may further include a step of determining the speech rate of the user. In this case, the degree to which the initial maximum silence duration or the minimum silence duration is increased may be determined to be inversely proportional to the speech rate.

[0017] The above method may further include a step of invalidating the detected endpoint if, in the step of detecting the endpoint, the utterance context is not determined to be the second utterance context and the endpoint is detected.

[0018] The above utterance may be a response to an utterance in the form of voice or text by an artificial intelligence model that performs a conversation with the user. In this case, the utterance context may include a fourth utterance context containing a response corresponding to a question from the artificial intelligence model in the utterance, a fifth utterance context containing all the information requested by the artificial intelligence model in the utterance, or a sixth utterance context containing some of the information requested by the artificial intelligence model in the utterance. Accordingly, in the step of detecting the endpoint, if the utterance context is determined to be the fourth utterance context or the fifth utterance context, the endpoint may be detected between the time when the utterance context is determined to be the fourth utterance context or the fifth utterance context and the time when new speech recognition is performed after the time when the utterance context is determined to be the fourth utterance context or the fifth utterance context.

[0019] The above method may further include a step of invalidating the detected endpoint if, in the step of detecting the endpoint, the utterance context is not determined to be the fourth utterance context or the fifth utterance context and the endpoint is detected.

[0020] An adaptive utterance endpoint detection method according to another embodiment of the first aspect of the present invention comprises, for a user performing a conversation with an artificial intelligence model, the step of acquiring a response utterance of the user to the artificial intelligence model and determining the response speed of the user; the step of determining an end point detection (EPD) reference time for the response utterance based on the response speed; and the step of detecting an end point for the response utterance based on the end point detection reference time.

[0021] The above method may further include a step of determining the user's utterance context through speech recognition of the user's response utterance. In this case, the endpoint detection reference time may include an initial silence maximum duration, a minimum trailing silence duration, or a maximum recognition time. Additionally, in the step of determining the endpoint detection reference time, if the utterance context is determined to be a first utterance context in which the user intends to continue the response utterance, or a third utterance context in which it is unclear whether the user will end the response utterance, the maximum recognition time, the initial silence maximum duration, or the minimum silence duration may be updated by comparing the response speed with a preset value.

[0022] An adaptive speech endpoint detection device according to a second aspect of the present invention comprises a memory capable of storing computer-executable instructions and a processor, wherein the computer-executable instructions are executed by the processor, thereby determining the context of a user's speech through speech recognition of the user's speech, determining an end point detection (EPD) reference time for the speech based on the speech context, and detecting an end point for the speech based on the end point detection reference time.

[0023] A computer-readable recording medium storing computer-executable instructions according to a third aspect of the present invention, wherein the computer-executable instructions, when executed by a processor, enable the processor to perform a method comprising: determining the context of a user’s utterance through speech recognition of the user’s utterance; determining an end point detection (EPD) reference time for the utterance based on the utterance context; and detecting an end point for the utterance based on the end point detection reference time.

[0024] A computer program stored in a computer-readable recording medium according to a fourth aspect of the present invention, the computer program comprises instructions for the processor to perform a method comprising: determining a context of a user’s utterance through speech recognition of the user’s utterance when executed by a processor; determining an end point detection (EPD) reference time for the utterance based on the utterance context; and detecting an end point for the utterance based on the end point detection reference time. Effects of the invention

[0025] According to one embodiment, the endpoint of an utterance can be determined dynamically by considering the speaker's characteristics or the conversation situation, and accordingly, the endpoint of the utterance can be determined efficiently and the accuracy of the endpoint determination can be improved, thereby improving the performance of the callbot.

[0026] The effects obtainable from the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure belongs from the description below. Brief explanation of the drawing

[0027] FIG. 1 is a block diagram exemplarily illustrating an adaptive utterance endpoint detection device according to one embodiment. Figure 2 is a block diagram exemplifying the function of an adaptive utterance endpoint detection program. FIG. 3 is a flowchart exemplarily illustrating an adaptive utterance endpoint detection method according to one embodiment. FIG. 4 is a flowchart showing an example of adaptively setting the endpoint detection reference time according to the utterance context. FIG. 5 is a flowchart showing an example of adaptively setting an endpoint detection reference time by further considering the utterance speed in the utterance context. Specific details for implementing the invention

[0028] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but can be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims.

[0029] In describing the embodiments of the present invention, specific descriptions of known functions or configurations will be omitted if it is determined that such detailed descriptions could unnecessarily obscure the essence of the invention. Furthermore, the terms described below are defined in consideration of their functions in the embodiments of the present invention, and these definitions may vary depending on the intentions or practices of the user or operator. Therefore, such definitions should be based on the content throughout this specification.

[0030] The terms used in this specification will be briefly explained, and the invention will be described in detail.

[0031] The terms used in this specification have been selected to be as widely used as possible, taking into account the functions of the present invention; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this invention should be defined not merely by their names, but based on their meanings and the overall content of the invention.

[0032] When a part of a specification is described as 'comprising' a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0033] Additionally, the term "part" as used in the specification refers to software or hardware components, such as FPGAs or ASICs, and the "part" performs certain roles. However, the meaning of "part" is not limited to software or hardware. The "part" may be configured to reside in an addressable storage medium or configured to run one or more processors. Thus, by example, the "part" includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and "parts" may be combined into a smaller number of components and "parts" or further separated into additional components and "parts."

[0034] Below, embodiments of the present invention are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present invention.

[0035] FIG. 1 is a block diagram exemplarily illustrating an adaptive utterance endpoint detection device according to one embodiment.

[0036] As shown in FIG. 1, the adaptive utterance endpoint detection device (100) may include an input unit (110), an output unit (120), a processor (130), a memory (140), or a communication unit (160).

[0037] For convenience of explanation, the following description describes an example in which an adaptive speech endpoint detection device (100) includes an input unit (110), an output unit (120), a processor (130), a memory (140), or a communication unit (160), but is not limited thereto. That is, each unit configuration may be provided outside the adaptive speech endpoint detection device (100) and operate in a manner that interacts with the adaptive speech endpoint detection device (100).

[0038] The input unit (110) may include a user interface for receiving commands, information, etc., used to control the adaptive speech endpoint detection device (100). Additionally, the input unit (110) may be a hardware device (e.g., keyboard, mouse, microphone, touchpad, etc.) capable of directly receiving commands, information, etc., used to control the adaptive speech endpoint detection device (100).

[0039] In one embodiment, the input unit (110) can receive information necessary for the adaptive speech endpoint detection method from the user. Specifically, the user can input information including voice data through speech through the input unit (110).

[0040] The output unit (120) can provide information including voice data through speech, voice recognition results, endpoint detection reference time, endpoint, and response to the user's speech (e.g., text data or voice data) as visual information to the user through an interface.

[0041] The processor (130) can control the overall operation of the adaptive utterance endpoint detection device (100) to carry out the present invention.

[0042] The processor (130) can execute the adaptive utterance endpoint detection program (150) by loading the adaptive utterance endpoint detection program (150) and the information required for the execution of the adaptive utterance endpoint detection program (150) from memory (140).

[0043] The processor (130) can control the adaptive speech endpoint detection device (100) to store data received from an external device through the communication unit (160) in the memory (140). Additionally, the processor (130) can control the adaptive speech endpoint detection device (100) to transmit and receive information including voice data through speech, voice recognition results, endpoint detection reference time, endpoint, and a response to the user's speech (e.g., text data or voice data) with an external device through the communication unit (160).

[0044] The processor (130) may refer to a processing device such as a microprocessor, a central processing unit (CPU), a graphic processing unit (GPU), a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or a micro controller unit (MCU), but is not limited to the embodiments described above.

[0045] The memory (140) can store information necessary for the execution of the adaptive utterance endpoint detection program (150) and the adaptive utterance endpoint detection program (150). Additionally, the memory (140) can store processing results by the processor (130).

[0046] The adaptive utterance endpoint detection program (150) may mean software comprising instructions programmed to perform the method according to the present invention.

[0047] The memory (140) can store information including voice data through speech, voice recognition results, endpoint detection reference time, endpoint, and response to the user's speech (e.g., text data or voice data). Additionally, the memory (140) can store information received from an external device through the communication unit (160).

[0048] Memory (140) may refer to a computer-readable recording medium such as a magnetic medium such as a hard disk, floppy disk, and magnetic tape, an optical recording medium such as a CD-ROM and DVD, a magneto-optical medium such as a floptical disk, a random access memory such as a DRAM (dynamic random access memory), a SRAM (static random access memory), a hardware device specifically configured to store and execute program instructions such as a flash memory, but is not limited to the embodiments described above.

[0049] The communication unit (160) may be a wireless communication module capable of performing wireless communication by adopting a communication method such as CDMA, GSM, W-CDMA, TD-SCDMA, WiBro, LTE, EPC, 5G, wireless LAN, Wi-Fi, Bluetooth, Zigbee, WFD (Wi-Fi Direct), UWB (Ultra Wide Band), infrared communication (IrDA; infrared data association), BLE (Bluetooth Low Energy) or NFC (Near Field Communication), but is not limited to the above-described embodiment.

[0050] In addition, information input and output through the input unit (110) and output unit (120), information stored in the memory (140), and information transmitted and received through the communication unit (160) include all information related to the present invention and are not limited to the embodiments described above.

[0051] The function or operation of the adaptive speech endpoint detection program (150) will be examined in detail through FIG. 2.

[0052] Figure 2 is a block diagram exemplifying the function of an adaptive utterance endpoint detection program.

[0053] As shown in FIG. 2, the adaptive speech endpoint detection program (150) may include a speech context determination unit (210), a speech rate determination unit (220), an endpoint detection reference time determination unit (230), and an endpoint detection unit (240). The speech context determination unit (210), the speech rate determination unit (220), the endpoint detection reference time determination unit (230), and the endpoint detection unit (240) are exemplary divisions of the functions of the adaptive speech endpoint detection program (150) and are not limited thereto.

[0054] According to the embodiment, the functions of the utterance context determination unit (210), the utterance speed determination unit (220), the endpoint detection reference time determination unit (230), and the endpoint detection unit (240) can be merged / separated and can be implemented as a series of instructions included in at least one program.

[0055] The utterance context determination unit (210), utterance rate determination unit (220), endpoint detection reference time determination unit (230), and endpoint detection unit (240) may be implemented by a processor (130) and may refer to a data processing device embedded in hardware having a physically structured circuit to perform a function expressed by a code or command included in an adaptive utterance endpoint detection program (150) stored in memory (140).

[0056] The speech context determination unit (210) can determine the speech context of the user through speech recognition of the user's speech.

[0057] A user may refer to a person attempting to communicate with a specific call bot. A call bot may refer to an artificial intelligence system that automatically processes voice conversations through a voice-based interface. Call bots can be utilized in call centers, customer service, or reservation systems, and can be designed to converse in real-time in a manner similar to humans. Call bots may utilize voice data or text data as input or output data.

[0058] Speech recognition can refer to STT (speech to text) technology. Among technologies that process input natural language, STT technology is a speech recognition technology that recognizes speech and converts it into text. Conversely, TTS (text to speech) technology is a speech synthesis technology that converts text into speech. These STT and TTS technologies can be applied to callbots.

[0059] The utterance context determination unit (210) can determine the utterance context or the user's intention by using text data obtained through speech recognition of the user's utterance. For example, the utterance context determination unit (210) can determine the utterance context or the user's intention by performing syntactic analysis or semantic analysis on user input in the form of text data.

[0060] In one embodiment, the utterance context determination unit (210) can determine the meaning of a word extracted from user input by using linguistic features (e.g., grammatical elements) of a morpheme or phrase, and determine the utterance context or the user's intention by matching the determined meaning of the word to the intention.

[0061] In one embodiment, the utterance context determination unit (210) may include an utterance context analysis model that understands and interprets the context of a conversation by utilizing natural language processing (NLP) technology. For example, the utterance context analysis model may include rule-based models, statistical models, machine learning-based models, deep neural networks (DNN), or deep learning models. Specifically, the utterance context analysis model may be implemented through artificial intelligence models such as BERT (bidirectional encoder representations from transformers), GPT (generative pre-trained transformer), T5 (text-to-text transfer transformer), or XLNet.

[0062] A speech context analysis model analyzes the context of an utterance to make decisions regarding whether the speaker intends to continue the utterance, end the utterance, or consider whether to end the utterance. For example, if the speaker intends to continue the utterance, the sentence may be connected without a clear end, or conjunctions such as "and," "also," "for example," or "so" may be used. As another example, if the speaker intends to end the utterance, the sentence may end with a clear terminal ending or end without a conjunction. As yet another example, if the speaker considers whether to end the utterance, the speech may be cut off, hesitant expressions such as "um," "geuge," or "eo," the same word or sentence may be repeatedly modified, uncertain expressions may be used, or the sentence structure may remain in an incomplete state, such as "geureoja," "jamkanman," or "it boja."

[0063] The utterance context may include a first utterance context in which the user intends to continue the utterance, a second utterance context in which the user intends to end the utterance, or a third utterance context in which it is unclear whether the user will end the utterance.

[0064] The firing speed determining unit (220) can determine the firing speed of the user.

[0065] In one embodiment, the utterance rate may be determined as 'number of utterance characters / utterance time'. In this case, the judgment interval for determining the utterance rate may be the utterance immediately preceding the time of the utterance rate, or the last N utterances in the past.

[0066] The end point detection reference time determination unit (230) can determine an end point detection (EPD) reference time for an utterance based on the utterance context. Here, the end point detection reference time may include an initial silence maximum duration, a minimum trailing silence duration, or a maximum recognition time.

[0067] The initial silence maximum duration may refer to the maximum waiting time in the absence of utterance. In other words, if the pre-set initial silence maximum duration is exceeded without utterance, that point in time may be determined as the endpoint.

[0068] The minimum trailing silence refers to the minimum time that silence persists after an utterance. In other words, if silence persists beyond the preset minimum trailing silence after an utterance, that point in time can be determined as the endpoint.

[0069] The maximum recognition time may refer to the time at which an endpoint is determined regardless of whether utterance occurs. In other words, if the preset maximum recognition time is exceeded, regardless of whether utterance occurs, that point in time may be determined as the endpoint.

[0070] The endpoint detection reference time determination unit (230) can be updated in a direction that increases the maximum recognition time when the utterance context is determined to be a first utterance context in which the user intends to continue the utterance. Additionally, when the utterance context is determined to be a first utterance context in which the user intends to continue the utterance, the endpoint detection reference time determination unit (230) can be updated in a direction that increases the initial maximum silence duration or minimum silence duration in order to detect the endpoint for a newly input utterance after the endpoint detection for the corresponding utterance is completed. That is, after the first utterance context, speech recognition can be continued without detecting the endpoint even if the silence is longer than usual.

[0071] The endpoint detection reference time determination unit (230) can detect an endpoint between the time when the utterance context is determined to be the second utterance context and the time when new speech recognition is performed after the time when the utterance context is determined to be the second utterance context.

[0072] The endpoint detection standard time determination unit (230) can update the initial maximum silence duration or minimum silence duration in a direction that increases when the utterance context is determined to be a third utterance context in which it is unclear whether the user will end the utterance.

[0073] When the endpoint detection reference time determination unit (230) determines that the utterance context is the first utterance context or the third utterance context, it can compare the utterance speed determined by the utterance speed determination unit (220) with a preset value and update the maximum recognition time, the initial maximum silence duration, or the minimum silence duration in a direction that increases.

[0074] When the endpoint detection standard time determination unit (230) updates the maximum recognition time, the initial maximum silence duration, or the minimum silence duration in a direction that increases or decreases, a preset increase value or decrease value may be used, but an increase value or decrease value determined through the following mathematical formula may also be used.

[0075]

[0076] Here, t may represent the endpoint detection reference time being updated, t0 the endpoint detection reference time before updating, R the variable tuning variable, S the firing rate, and C the correction constant.

[0077] The end point detection unit (240) can detect the end point of a speech based on the end point detection reference time. That is, the end point detection unit (240) can perform an end point detection (EPD) function. End point detection is a function that detects the end point of a voice speech. When a user stops speaking while talking, it is detected that the user's speech has stopped and the voice recognition result is confirmed. At this time, the function of detecting the time when the user has completed voice input can be referred to as end point detection.

[0078] In one embodiment, the endpoint detection unit (240) may invalidate the detected endpoint when the endpoint is detected without the utterance context being determined as the second utterance context. For example, if the endpoint is detected while the utterance context for a utterance input in real time is determined as the first utterance context or the third utterance context, it may be determined that it is unclear whether the user intends to continue the utterance or end the utterance, so the endpoint detected in this way may be invalidated.

[0079] In one embodiment, the utterance may be for an artificial intelligence model that performs a conversation with a user. In this case, the utterance context may include a fourth utterance context containing a response corresponding to a question from the artificial intelligence model in the utterance, a fifth utterance context containing all the information requested by the artificial intelligence model in the utterance, or a sixth utterance context containing some of the information requested by the artificial intelligence model in the utterance. Accordingly, when the utterance context is determined to be the fourth utterance context or the fifth utterance context, the endpoint detection reference time determination unit (230) can detect an endpoint between the time when the utterance context is determined to be the fourth utterance context or the fifth utterance context and the time when new speech recognition is performed after the time when the utterance context is determined to be the fourth utterance context or the fifth utterance context.

[0080] In one embodiment, the endpoint detection unit (240) may invalidate the detected endpoint if the endpoint is detected when the utterance context is not determined to be the fourth utterance context or the fifth utterance context. For example, if the endpoint is detected while the utterance context for a utterance input in real time is determined to be the sixth utterance context, it may be determined that the utterance does not contain a response corresponding to the question of the artificial intelligence model, or that the utterance does not contain all the information requested by the artificial intelligence model, and thus the endpoint detected in this way may be invalidated. In one embodiment, the utterance speed determination unit (220) may determine the user's response speed by obtaining the user's response utterance to the artificial intelligence model for a user performing a conversation with the artificial intelligence model. Here, the response speed may be determined based on the user's response time to the artificial intelligence model's utterance (e.g., TTS utterance). For example, if the user's response speed to the artificial intelligence model's utterance is slow, the maximum recognition time, the initial maximum silence duration, or the minimum silence duration may be updated in a direction that increases, taking into account this response speed. As another example, if the user's reaction speed to the AI ​​model's utterance is fast, the maximum recognition time, initial maximum silence duration, or minimum silence duration may be updated to decrease in consideration of this reaction speed.

[0081] In one embodiment, the user's response speed can be determined as follows:

[0082]

[0083] For example, if the AI ​​model's utterance end time is 1000ms and the user's response utterance start time is 1200ms, the response speed can be determined to be 200ms. As another example, if the AI ​​model's utterance end time is 1000ms and the user's response utterance start time is 800ms, which is earlier than the AI ​​model's utterance end time, the response speed can be determined to be -200ms. In other words, the response speed can be negative.

[0084] The endpoint detection standard time determination unit (230) can update the maximum recognition time, initial silence maximum duration, or minimum silence duration by comparing the response speed with a preset value when the utterance context is determined to be a first utterance context in which the user intends to continue the response utterance, or a third utterance context in which it is unclear whether the user will end the utterance.

[0085] FIG. 3 is a flowchart illustrating an adaptive utterance endpoint detection method according to one embodiment. Here, the method illustrated in FIG. 3 can be executed by the adaptive utterance endpoint detection device (100) illustrated in FIG. 1. Furthermore, since the flowchart illustrated in FIG. 3 is merely illustrative, depending on the embodiment, each step may be executed in a different order than that described in the flowchart, or additional steps not described in the flowchart may be executed, or one or more of the steps described in the flowchart may not be executed.

[0086] As shown in FIG. 3, an adaptive speech endpoint detection method according to one embodiment comprises the steps of determining the context of a user’s speech through speech recognition of the user’s speech (S310), determining a reference time for end point detection (EPD) for the speech based on the speech context (S320), and detecting an endpoint for the speech based on the reference time for end point detection (S330).

[0087] FIG. 4 is a flowchart showing an example of adaptively setting the endpoint detection reference time according to the utterance context.

[0088] The speech context determination unit (210) can determine the speech context by analyzing the user's speech (S410).

[0089] At this time, the utterance context determined by analyzing the user's utterance may include a first utterance context in which the user intends to continue the utterance, a second utterance context in which the user intends to end the utterance, or a third utterance context in which it is unclear whether the user will end the utterance.

[0090] When the utterance context is determined to be the first utterance context in which the user intends to continue the utterance, the maximum recognition time is updated in a direction that increases, and the endpoint can be detected (S420).

[0091] When the utterance context is determined to be a second utterance context in which the user intends to end the utterance, the endpoint can be detected immediately regardless of the endpoint detection reference time (S430).

[0092] If the utterance context is determined to be a third utterance context where it is unclear whether the user will end the utterance, the initial maximum silence duration or minimum silence duration is updated in a direction that increases so that the endpoint can be detected (S440).

[0093] FIG. 5 is a flowchart showing an example of adaptively setting an endpoint detection reference time by further considering the utterance speed in the utterance context.

[0094] The speech context determination unit (210) can determine the speech context by analyzing the user's speech (S410).

[0095] When the utterance context is determined to be a first utterance context in which the user intends to continue the utterance, the utterance speed and a preset value are compared and updated in a direction that increases the maximum recognition time so that the endpoint can be detected (S520). At this time, the degree to which the maximum recognition time increases can be determined according to Equation 1.

[0096] When the utterance context is determined to be a second utterance context in which the user intends to end the utterance, the endpoint can be detected immediately regardless of the endpoint detection reference time (S430).

[0097] When the utterance context is determined to be a third utterance context where it is unclear whether the user will end the utterance, the utterance speed and a preset value are compared, and the initial maximum silence duration or minimum silence duration is updated in a direction that increases, so that the endpoint can be detected (S540). At this time, the degree to which the initial maximum silence duration or minimum silence duration increases can be determined according to Equation 1.

[0098] As described above, according to one embodiment, the endpoint of an utterance can be determined dynamically by considering the speaker's characteristics or conversational situation, and accordingly, the endpoint of the utterance can be determined efficiently and the accuracy of the endpoint determination can be improved, thereby improving the performance of the callbot.

[0099] The embodiments of the present invention described above may be implemented through various means. For example, the embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof.

[0100] Combinations of each block of the block diagram attached to the present invention and each step of the flowchart may be performed by computer program instructions. Since these computer program instructions may be loaded into an encoding processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, the instructions performed through the encoding processor of the computer or other programmable data processing equipment create means for performing the functions described in each block of the block diagram or each step of the flowchart. Since these computer program instructions may also be stored in computer-available or computer-readable memory that can be directed toward the computer or other programmable data processing equipment to implement the function in a specific way, the instructions stored in computer-available or computer-readable memory may also produce a manufactured item containing instruction means for performing the function described in each block of the block diagram or each step of the flowchart. Since computer program instructions can be loaded onto a computer or other programmable data processing equipment, instructions that execute a computer or other programmable data processing equipment by performing a series of operation steps on the computer or other programmable data processing equipment to create a process executed by the computer can also provide steps for executing the functions described in each block of the block diagram and each step of the flowchart.

[0101] Additionally, each block or each step may represent a module, segment, or part of code containing one or more executable instructions for executing a specified logical function(s). In some embodiments, the functions mentioned in the blocks or steps may occur out of order. For example, two blocks or steps depicted consecutively may actually be performed substantially simultaneously, or the blocks or steps may be performed in reverse order according to the corresponding function.

[0102] The above description is merely an illustrative explanation of the technical concept of the present invention, and those skilled in the art to which the present invention pertains will be able to make various modifications and variations within the scope of the essential quality of the present invention. Accordingly, the embodiments disclosed in the present invention are intended to explain, not limit, the technical concept of the present invention, and the scope of the technical concept of the present invention is not limited by such embodiments. The scope of protection of the present invention shall be interpreted by the claims below, and all technical concepts within the equivalent scope shall be interpreted as being included within the scope of rights of the present invention. Explanation of the symbols

[0103] 100: Adaptive utterance endpoint detection device 150: Adaptive Utterance Endpoint Detection Program

Claims

Claim 1 An adaptive speech endpoint detection method performed by an adaptive speech endpoint detection device, comprising: a step of determining the context of a user’s speech through speech recognition of the user’s speech; a step of determining an end point detection (EPD) reference time for the speech based on the speech context; and a step of detecting an endpoint for the speech based on the end point detection reference time. Claim 2 An adaptive speech endpoint detection method according to claim 1, wherein the speech context comprises a first speech context in which the user intends to continue the speech, a second speech context in which the user intends to end the speech, or a third speech context in which it is unclear whether the user will end the speech. Claim 3 An adaptive speech endpoint detection method according to paragraph 2, wherein the endpoint detection reference time includes an initial silence maximum duration, a minimum trailing silence duration, or a maximum recognition time. Claim 4 In paragraph 3, an adaptive speech endpoint detection method that, in the step of determining the endpoint detection reference time, if the speech context is determined to be the first speech context, updates the initial maximum silence duration, the minimum silence duration, or the maximum recognition time in a direction that increases. Claim 5 An adaptive speech endpoint detection method according to claim 3, wherein, in the step of detecting the endpoint, if the speech context is determined to be the second speech context, the endpoint is detected between the time when the speech context is determined to be the second speech context and the time when new speech recognition is performed after the time when the speech context is determined to be the second speech context. Claim 6 In the third paragraph, an adaptive speech endpoint detection method that, in the step of determining the endpoint detection reference time, updates the initial maximum silence duration or the minimum silence duration in a direction that increases when the speech context is determined to be the third speech context. Claim 7 An adaptive speech endpoint detection method according to claim 4, further comprising a step of determining the speech rate of the user, wherein the degree to which the maximum recognition time is increased is determined to be inversely proportional to the speech rate. Claim 8 An adaptive speech endpoint detection method according to claim 6, further comprising the step of determining the speech rate of the user, wherein the degree to which the initial maximum silence duration or the minimum silence duration is increased is determined to be inversely proportional to the speech rate. Claim 9 An adaptive speech endpoint detection method according to claim 4, wherein the method further comprises the step of invalidating the detected endpoint when the speech context is not determined to be the second speech context and the endpoint is detected in the step of detecting the endpoint. Claim 10 An adaptive speech endpoint detection method according to claim 1, wherein the speech is a response to a speech in the form of voice or text of an artificial intelligence model that performs a conversation with the user, and the speech context includes a fourth speech context containing a response corresponding to a question of the artificial intelligence model in the speech, a fifth speech context containing all of the information requested by the artificial intelligence model in the speech, or a sixth speech context containing some of the information requested by the artificial intelligence model in the speech. Claim 11 An adaptive speech endpoint detection method according to claim 10, wherein, in the step of detecting the endpoint, if the speech context is determined to be the fourth speech context or the fifth speech context, the endpoint is detected between the time when the speech context is determined to be the fourth speech context or the fifth speech context and the time when new speech recognition is performed after the time when the speech context is determined to be the fourth speech context or the fifth speech context. Claim 12 An adaptive speech endpoint detection method according to claim 10, wherein the method further comprises the step of invalidating the detected endpoint when the endpoint is detected and the speech context is not determined to be the fourth speech context or the fifth speech context. Claim 13 An adaptive utterance endpoint detection method performed by an adaptive utterance endpoint detection device, comprising: a step of determining the response speed of a user who performs a conversation with an artificial intelligence model by acquiring a response utterance of the user to the artificial intelligence model; a step of determining an end point detection (EPD) reference time for the response utterance based on the response speed; and a step of detecting an end point for the response utterance based on the end point detection reference time. Claim 14 In claim 13, the method further comprises the step of determining the utterance context of the user through speech recognition of the user's response utterance, wherein the endpoint detection reference time includes an initial silence maximum duration, a minimum trailing silence duration, or a maximum recognition time, and in the step of determining the endpoint detection reference time, if the utterance context is determined to be a first utterance context in which the user intends to continue the response utterance or a third utterance context in which it is unclear whether the user will end the response utterance, the maximum recognition time, the initial silence maximum duration, or the minimum silence duration is updated by comparing the response speed with a preset value. Claim 15 An adaptive speech endpoint detection device comprising: a memory capable of storing computer-executable instructions; and a processor, wherein the computer-executable instructions are executed by the processor, thereby determining the context of the user’s speech through speech recognition of the user’s speech, determining an end point detection (EPD) reference time for the speech based on the speech context, and detecting an endpoint for the speech based on the end point detection reference time. Claim 16 A computer-readable recording medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, enable the processor to perform a method comprising: determining the context of a user’s utterance through speech recognition of the user’s utterance; determining an end point detection (EPD) reference time for the utterance based on the utterance context; and detecting an end point for the utterance based on the end point detection reference time. Claim 17 A computer program stored on a computer-readable recording medium, the computer program comprising instructions for the processor to perform a method comprising: determining the context of a user’s utterance through speech recognition of the user’s utterance when executed by a processor; determining an end point detection (EPD) reference time for the utterance based on the utterance context; and detecting an end point for the utterance based on the end point detection reference time.

Images