Interpretation apparatus and real-time continuous interpretation method
The AI-based interpretation device segments and translates voice input into sentence units in real-time, addressing the challenges of continuous speech and long sentences, ensuring accurate and natural translation.
Patent Information
- Application Number
- PCT/KR2026/096810
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-09
- Filing Date
- 2026-05-11
- Publication Date
- 2026-11-12
AI Technical Summary
Conventional interpretation devices face challenges in providing real-time continuous interpretation due to difficulties in accurately detecting sentence endings, especially in cases of continuous speech or long sentences, leading to translation errors and degraded quality.
An AI-based interpretation device and method that includes an STT module, control module, alignment module, and translation module to segment and analyze voice input into designated units, connect and separate text into sentence units, and translate in real-time, using silent intervals and user inputs to manage sentence termination.
Enables real-time sentence-level interpretation services by accurately dividing and translating continuous voice input, maintaining interpretation quality even in challenging conditions such as continuous speech or fast delivery, thereby improving translation accuracy and natural flow.
Smart Images

Figure KR2026096810_12112026_PF_FP_ABST
Abstract
Description
Interpretation device and real-time continuous interpretation method
[0001] The various embodiments disclosed in this document relate to AI-based interpretation technology.
[0002] Recently, various interpretation devices utilizing machine learning (ML) or artificial intelligence (AI) have been emerging. Conventional interpretation devices can be applied in a wide range of ways, from one-on-one interpretation to simultaneous interpretation for multiple people.
[0003] An interpretation device must translate voice input on a sentence-by-sentence basis to improve the accuracy of the translation results. Therefore, the device must properly detect the end of sentences in the voice input to ensure a certain level of interpretation quality. To this end, conventional interpretation devices detected sentence endings by methods such as detecting silent (pause) periods in the voice input or detecting termination signals based on user input. In other words, it was a method of interpreting sequentially only after the sentences had concluded.
[0004] For example, sentence termination detection based on silent intervals in voice input automatically detected sentence terminations by detecting these intervals. However, this method may incorrectly detect the middle of the voice input as a sentence termination when the speaker speaks slowly. Furthermore, the aforementioned silent interval-based sentence termination detection may fail to detect silent intervals in cases where speaker voice input occurs continuously, such as in TV news. Alternatively, if the silent interval value (threshold) is set too long, real-time continuous interpretation becomes impossible as the interpretation device continues to accumulate the input conversation if the detected silent interval is not sufficiently long. As another example, sentence termination detection based on a user-operated end button made it difficult for the conversation to flow naturally due to the inconvenience of the user having to press the voice input end button every time and the resulting time delay. As yet another example, sentence termination detection was performed in a STT (speech to text) module by combining voice input pitch variation, silent interval recognition, and speaker switching.
[0005] However, conventional sentence-ending detection methods face significant difficulties in real-time continuous interpretation when the speaker's voice input is continuous, such as in TV news, or when long sentences are input, such as in seminar presentations. Furthermore, for real-time continuous interpretation, the text continuously generated by the STT module must be appropriately divided and translated; however, dividing the text in the middle of a sentence can lead to translation errors. Additionally, in cases like news or seminars where there is a single speaker and content must be delivered within a set time, speech is very fast, making it difficult to detect silent segments. Consequently, text division based on silent segments is impossible, rendering it impossible to provide real-time continuous interpretation services. Moreover, methods that divide voice input into fixed time intervals result in dividing the text between a single phrase or word, which can significantly degrade translation quality. Therefore, a technology is required that separates the text continuously output by the STT module in response to continuous voice input into units (e.g., sentences) that are advantageous for translation.
[0006] Various embodiments disclosed in this document can provide an interpretation device and a real-time interpretation method capable of providing sentence-level interpretation services in real time based on voice input.
[0007] An interpretation device according to an embodiment disclosed in this document may include: an input device for detecting voice input; an STT module for outputting a string of characters converted from the voice input into text; a control module for generating a segmented text by dividing the string of characters from the STT module into designated units; an output device; a sorting module for acquiring the segmented text, and if there is previously stored unaligned sentence information (not a NULL value), generating a connected text by concatenating the acquired segmented text and the unaligned sentence information, then separating and analyzing the generated connected text into sentence units, outputting a sorted text determined to be a completed sentence, and storing the remaining part excluding the outputted sorted text as unaligned sentence information; and a translation module for generating a translation result by translating the outputted sorted text and outputting it through the output device.
[0008] Meanwhile, if the input split text is a termination signal, the sorting module outputs unsorted sentence information as sorted text and can set the unsorted sentence information to a NULL value or to include a NULL value.
[0009] In addition, a real-time interpretation method according to an embodiment disclosed in this document may include: converting real-time voice input into text through an STT module; sequentially generating split text by dividing the converted text into designated units; if the split text is obtained and there is previously stored unaligned sentence information (not a NULL value), connecting the obtained split text and the unaligned sentence information to generate connected text; separating and analyzing the generated connected text into sentence units and outputting an aligned text determined to be a completed sentence; storing the remaining part excluding the output aligned text as unaligned sentence information; and translating the output aligned text to generate a translation result and outputting the translation result.
[0010] Meanwhile, if the input split text is a termination signal, the sorting module outputs unaligned sentence information as sorted text and can set the unaligned sentence information to a NULL value or to include a NULL value.
[0011] According to the various embodiments disclosed in this document, a sentence-level interpretation service can be provided in real time based on voice input. In addition, various effects that can be identified directly or indirectly through this document may be provided.
[0012] FIG. 1 shows a flowchart of a real-time interpretation method according to one embodiment.
[0013] FIG. 2 shows a configuration diagram of an interpretation device according to one embodiment.
[0014] FIG. 3 shows a flowchart of a method for generating aligned text according to one embodiment.
[0015] Figure 4 shows an example of separating a string into sentence units through an AI module according to one embodiment.
[0016] FIGS. 5 and 6 are flowcharts illustrating a real-time interpretation process according to one embodiment, with the operating agent distinguished.
[0017] In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components.
[0018] Depending on whether there is unaligned sentence information, at least one split text or unaligned sentence information can be combined to generate connected text.
[0019] In operation 130, the interpreter device (200) can generate a connected text by concatenating the split text and the unaligned sentence information if there is previously stored unaligned sentence information (or if it is not NULL).
[0020] In operation 140, if there is no previously stored unaligned sentence information (or if it is NULL), the interpreter device (200) can generate a connected text by concatenating the split text and the split text immediately following it.
[0021] In operation 150, the interpretation device (200) can separate and analyze the connected text into sentence units, output the completed sentence as aligned text, and store the remaining parts. For example, the interpretation device (200) separates and analyzes the connected text into sentence units, and if there is a part within the connected text that is determined to be a completed sentence, it can output the part determined to be a completed sentence as aligned text and store the remaining parts excluding the output aligned text as unaligned sentence information. On the other hand, if there is no part determined to be a completed sentence, the interpretation device (200) can store the entire connected text as unaligned sentence information.
[0022] In operation 160, the interpreter device (200) generates a translation result as it translates the output alignment text and can output the translation result.
[0023] The operations of Fig. 1 may continue until the interpretation service is terminated by detecting voice input or other input signals by the interpretation device (200).
[0024] In this way, the interpretation device (200) according to one embodiment can provide a real-time interpretation service by dividing the voice input into a specified unit (e.g., a specified time or a specified number of words) to generate a plurality of segmented texts, connecting the segmented texts to separate and analyze the connected text into sentences, and selectively outputting and translating the completed sentences into aligned texts.
[0025] In addition, the interpretation device (200) according to one embodiment can improve interpretation accuracy by interpreting by connecting a part of a previous sentence (unaligned sentence information) to a subsequent sentence to maintain the connectivity of the divided texts.
[0026] FIG. 2 shows a configuration diagram of an interpretation device according to one embodiment.
[0027] Referring to FIG. 2, an interpretation device (200) according to one embodiment may include an input device (210), an output device (220), a memory (230), and a processor (240). In one embodiment, some components of the interpretation device (200) may be omitted, or additional components may be included. Additionally, some of the components of the interpretation device (200) may be combined to form a single entity, which can perform the same functions as the components prior to combination. The interpretation device (200) may be a user terminal and may be composed of a plurality of devices including a user terminal and a server device.
[0028] The input device (210) can receive or detect voice input from a user using the interpretation device (200). The input device (210) can further detect or receive button or touch input (e.g., a termination signal in response to a user's translation request). The input device (210) may include at least one input detection circuit among, for example, a button, a touchscreen, and a microphone.
[0029] The output device (220) can visually or audibly output at least one data of symbols, numbers, or characters under the control of the processor (240). The output device (220) may include, for example, at least one output device among a liquid crystal display, an OLED, a touchscreen display, and a speaker.
[0030] The memory (230) may include various forms of volatile or non-volatile memory. For example, the memory (230) may include ROM (read only memory) and RAM (random access memory). In one embodiment, the memory (230) may be located inside or outside the processor (240), and the memory (230) may be connected to the processor (240) through various known means. The memory (230) may store various data used by at least one component of the interpretation device (200) (e.g., processor (240)). The data may include, for example, input data or output data for software and related commands. For example, the memory (230) may store at least one instruction and data for providing a real-time interpretation service. The memory (230) may store instructions and data for driving the STT module (241), control module (243), alignment module (245), and translation module (247). For example, memory (230) can store information related to specified units, threshold counts, and threshold times.
[0031] The processor (240) can control at least one other component (e.g., a hardware or software component) of the interpretation device (200) and can perform various data processing or operations. The processor (240) may include, for example, at least one of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, an application processor, an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA), and may have multiple cores. According to one embodiment, the processor (240) may include a speech-to-text (STT) module (241), a control module (243), an alignment module (245), and a translation module (247). The STT module (241), the control module (243), the alignment module (245), and the translation module (247) may be software modules or hardware modules that are included in the processor (240) or executed by the processor (240).
[0032] According to one embodiment, when the STT module (241) receives voice input from the input device (210), it can convert the received voice input into a text string and sequentially output the converted string.
[0033] The string output from the STT module (241) does not include markers indicating the beginning and end of a sentence. For example, English sentences begin with a capital letter and end with a period, but the string from the STT module (241) does not include markers that distinguish the beginning and end of the sentence. Of course, some STT modules (241) may distinguish words determined as names or the first-person pronoun 'I' by capitalizing them, but this differs from sentence separation markers. Alternatively, a high-performance STT module (241) can recognize the end of a sentence when a change in voice (speaker change) occurs. In this case, the STT module (241) can distinguish the end of the sentence recognition result by, for example, a period or a speaker identifier. Meanwhile, since there is no voice input during the pause (silent) interval in the voice input, the STT module (241) does not convert the voice input into text and does not output the converted text.
[0034] According to one embodiment, the control module (243) may obtain a string output from the STT module (241) and divide the obtained string into specified units to generate split text. The specified units may be a specified time (e.g., every 3 seconds) or a specified number of words (e.g., 10 words).
[0035] According to one embodiment, when the control module (243) receives an interpretation request through the input device (210), it may open an interpretation session by inputting (or transmitting) first and second instruction prompts requesting sentence alignment and real-time interpretation, respectively, to the alignment module (245) and the translation module (247). The first instruction prompt may include first language information of the input segmented text, a sentence-unit alignment request, and an output format of the aligned text. The second instruction prompt may include first and second language information of the aligned text, and an output format of the translation result.
[0036] According to one embodiment, the control module (243) can sequentially input split text 1, split text 2, …, split text N, which are strings from the STT module (241) divided into specified units through an open interpretation session, into the sorting module (245).
[0037] In one embodiment, the STT module (241) may modify (correct) previously generated text during the process of converting voice input into text. In preparation for this, the control module (243) may wait until at least a threshold number of strings output from the STT module (241) are accumulated in the buffer (memory (230)) and then generate split text based on the strings. For example, if the specified unit is 20 words, the control module (243) may wait without generating split text until 25 words, which is the sum of the threshold number (e.g., 5) and the specified number of words (e.g., 20), are accumulated in the buffer. When 25 words are accumulated in the buffer, the control module (243) may obtain the first 20 of them and generate split text.
[0038] In one embodiment, the control module (243) can detect a silent interval if text is not output from the STT module (241) for a threshold time (e.g., 2 seconds) or longer. When the control module (243) detects a silent interval, it can output a silent interval marker as split text. When the control module (243) obtains a user's termination signal through the input device (210), it can output a termination signal marker as split text. The silent interval marker and the termination signal marker are markers that are distinct from each other and can be set to a newline character, a special character, or a designated string. When the control module (243) opens an interpretation session, it can guide the alignment module (245) to process the silent interval marker and the termination signal marker via an instruction prompt.
[0039] According to one embodiment, when the alignment module (245) receives a split text from the control module (243), it can generate a connected text by connecting at least one split text or unaligned text information depending on whether there is previously stored unaligned text information.
[0040] For example, when the alignment module (245) receives split text, it can check whether there is previously stored unaligned sentence information. If the alignment module (245) has previously stored unaligned sentence information (if the unaligned sentence information is not a NULL value), it can generate a connected text by connecting the currently received split texts with previously received split texts (or unaligned sentence information). On the other hand, if there is no previously stored unaligned sentence information (if the unaligned sentence information is a NULL value), the alignment module (245) can generate a connected text by connecting the currently entered split text with subsequently entered split texts.
[0041] According to one embodiment, the alignment module (245) can separate the connected text into sentence units and analyze whether it is a complete sentence. When the alignment module (245) detects one or more complete sentences from the connected text, it can output the detected sentences as aligned text and store the remaining text, excluding the output aligned text, as unaligned sentence information. The output aligned text can be transmitted to the translation module (247).
[0042] When the alignment module (245) receives a silent interval marker from the control module (243), it may wait until it receives the next split text.
[0043] According to one embodiment, the alignment module (245) can store unaligned sentence information until it receives a termination signal marker from the control module (243) and then repeat connecting to the split text.
[0044] When the alignment module (245) receives a termination signal, it can transmit the text stored as unaligned sentence information to the translation module (247). If the input (acquired) split text is a termination signal (or a termination signal indicator), the alignment module (245) can output the unaligned sentence information as the aligned text and set the unaligned sentence information to a NULL value or set it to include a NULL value.
[0045] According to one embodiment, the translation module (247) may open an interpretation session for translation from a first language to a second language in accordance with the instruction prompt of the control module (243). When the translation module (247) receives alignment text (first language) from the alignment module (245) in the interpretation session, it may translate from the first language to the second language and transmit the translation result to the control module (243) through the interpretation session. The translation module (247) may output the translation result in a format instructed by the control module (243).
[0046] The control module (243) can obtain a translation result including aligned text of the first language and a translation of the second language from the translation module (247). The control module (243) can output the obtained translation result through the output device (220). For example, the control module (243) can display the translation result through the output device (220) (display). Alternatively, the control module (243) may output the translation result as sound through the output device (220).
[0047] According to various embodiments, the alignment module (245) can process a silent interval marker similarly to a termination signal marker. For example, when the alignment module (245) receives a silent interval marker, it can initially transmit the text stored as unaligned translation information to the translation module (247). Subsequently, when the alignment module (245) receives a new split text, it can connect the transmitted unaligned translation information with the new split text and perform the sentence-unit separation process again.
[0048] According to various embodiments, when the control module (243) obtains information related to speaker switching from the STT module (241), it can transmit the segmented text related to the speaker switching information to the alignment module (245). When the alignment module (245) receives the segmented text related to the speaker switching information, it can transmit the previously stored unaligned sentence information to the translation module (247). Subsequently, the alignment module (245) can generate aligned text by concatenating the segmented text related to the new speaker and separating it into sentence units.
[0049] According to various embodiments, if there is a silent interval marker immediately after the last sentence of the text connected by the alignment module (245), there is a high probability that the last sentence is terminated. Therefore, if there is a silent interval marker immediately after the last sentence of the text connected by the alignment module (245), the alignment module (245) may transmit the text connected as an aligned text to the translation module (247). In this case, the alignment module (245) may store a NULL value as unaligned sentence information.
[0050] According to various embodiments, the control module (243) may provide the real-time interpretation service described above until a request to end translation is received through the input device (210). When a request to end translation is received, the control module (243) may instruct the alignment module (245) and the translation module (247) to end the interpretation session. In this case, the alignment module (245) and the translation module (247) may end the interpretation service.
[0051] FIG. 3 shows a flowchart of a method for generating aligned text according to one embodiment.
[0052] Referring to FIG. 3, in operation 310, when the alignment module (245) receives split text from the control module (243), it can check in operation 320 whether the input split text is a silent section indicator. If the alignment module (245) does not receive split text in operation 310, it can wait until split text is received in operation 310.
[0053] In operation 320, if the alignment module (245) confirms that the input split text is not a silent interval marker, in operation 330, it can check whether it is a termination marker.
[0054] In operation 330, if the alignment module (245) determines that the input split text is not a termination mark, in operation 340, the currently input split text can be connected to the previously input split text (or unaligned sentence information).
[0055] In operation 350, the alignment module (245) can analyze the connected text by separating it into sentences. In operation 360, it can check whether the connected sentences contain at least one complete sentence.
[0056] In operation 360, if it is confirmed that there is at least one completed sentence, the alignment module (245) can store the remainder of the connected text, excluding the completed sentence, as unaligned sentence information in operation 370.
[0057] In operation 380, the alignment module (245) can provide the completed sentence from the connected text to the translation module (247) as the alignment text.
[0058] Meanwhile, if the alignment module (245) confirms in operation 320 that the split text is a silent interval marker, or in operation 360 that the connected text is not separated into sentence units, it can return to operation 310 and wait until the next split text is received.
[0059] Additionally, in operation 330, if the alignment module (245) confirms that the split text is a termination signal marker, in operation 390, it can transmit unaligned sentence information to the translation module (247).
[0060] In the above-described embodiment, the alignment module (245) and the translation module (247) may be implemented as AI modules. The AI module may be connected to cloud AI using the API of conversational AI, or it may be an on-device AI module included in the interpretation device (200). Alternatively, the AI module may be implemented using a large language model (LM) API. When implemented using an LLM API, software for managing prompt messages is required, but the processing speed may be faster. The control module (243) can perform alignment and translation functions using the AI module by inputting an instruction prompt to the AI module and receiving the response from the AI module. Below, an instruction prompt when the alignment module (245) and the translation module (247) are implemented as conversational AI will be described.
[0061] When the control module (243) receives an interpretation request through the input device (210), it may input an instruction prompt to the AI module, such as the one shown in Table 1 below. In the instruction prompt below, the original text message is a split text, the correction message is an aligned text, NS is a buffer that stores unaligned sentence information, eee is a marker for a termination signal, and gap is a marker for a silent interval.
[0062] As a professional translator, you must translate content transmitted in real-time. The languages used are Korean (L1) and English (L2). Please translate Korean into English and English into Korean. The source message consists of sentences, but may include incomplete sentences. 1. Generate a correction message composed of the remaining parts of the source message, excluding the last sentence. 2. If the sentences in the correction message are incomplete, repeat step 1. If they are complete or you have repeated step 2, proceed to step 3. 3. Translate the correction message and output it in the format [Translation: ....]. Save the removed parts to NS and output them in the format [NS:....]. 4. Receive a new source message. 5. If this new source message is 'eee', proceed to step 12. 6. If this new source message is 'gap', proceed to step 13. 7. Create a concatenation message by appending the NS saved in step 3 to the very beginning of the new source message. 8. Remove the last sentence from the connection message and generate a correction message composed of the remaining parts. 9. If the sentences in the correction message are not perfect, repeat step 4. If they are perfect or you have repeated step 4, go to 7. 10. Translate the correction message and output it in the format [Translation: ....]. Save the removed parts to NS and output them in the format [NS:....]. 11. Go to 4. 12. Output the translated content of NS in the format [Translation:....] and exit. 13. Output the translated content of NS in the format [Translation:....] and go to 1. The interpretation session starts with the 'sss' prompt and ends with the 'eee' prompt. If you understand the rules, please answer 'OK'.
[0063] According to one embodiment, the real-time conversation may be a series of utterances as shown in Table 2 below.
[0064] you promised me that we would stay together forever you still say that you love me but now you're telling me you have to leave and I'm completely shocked <cr>what happened please tell me if I've done something wrong let me know you haven't done anything wrong at all even when everyone else spoke ill of me you stood by my side i want to be with you forever too but but the thought that others blame and insult you because you're trying to protect me is so painful I can't stand it
[0065] FIG. 4 illustrates an example of separating a string into sentence units using an AI module according to one embodiment. According to one embodiment, the AI module can divide a string of continuous conversation into sentence units. Referring to FIG. 4, the control module (243) inputs an instruction prompt (410) instructing the AI module to divide the input string into sentence units, and can obtain a response message (420) containing sentences with serial numbers from the AI module. FIG. 4 is an example of a case where the entire string is input at once and separated into sentence units; however, in order to interpret the voice input of a conversation process in real time, a process of translating the text converted from the voice input one sentence at a time is required. Hereinafter, a real-time interpretation process based on an AI module according to one embodiment will be described with reference to FIG. 5 and FIG. 6. FIG. 5 and FIG. 6 are flowcharts showing the real-time interpretation process according to one embodiment, distinguished by the operating agents.
[0066] The STT module (241) can continuously output the string of the utterance 1 based on real-time conversation. The control module (243) can divide the string into word units every 3 seconds to sequentially generate split texts.
[0067] Referring to FIG. 5, in operation 511, the control module (243) can generate split text #1 "you promised me that we would stay" and transmit it to the alignment module (245).
[0068] In operation 521, since there is no previously received split text, the alignment module (245) can verify split text #1 as connected text 1 "you promised me that we would stay".
[0069] In operation 522, the alignment module (245) separates the connected text 1 into sentences, but since it is not complete, it can be stored as unaligned sentence information "1. you promised me that we would stay".
[0070] In operation 512, the control module (243) can generate split text #2 "together forever you still say that you" and send it to the alignment module (245).
[0071] In operation 523, the alignment module (245) can generate the connected text 2 "you promised me that we would stay together forever you still say that you" by concatenating the previously stored unaligned sentence information "you promised me that we would stay" with the split text #2.
[0072] In operation 524, the alignment module (245) can separate the connected text 2 into sentence units to generate two sentence units, “1. you promised me that we would stay together forever” and “2. you still say that you”. The alignment module (245) can transmit the first completed sentence to the translation module (247) and store the second incomplete sentence as unaligned sentence information.
[0073] In operation 531, the translation module (247) can receive the first alignment text (original text below) of the first language, translate it into the second language, and provide the translation result 1 to the control module (243).
[0074] In operation 513, the control module (243) can generate split text #3 "love me but now you're telling me" from the string and send it to the alignment module (245).
[0075] In operation 525, the previously saved unaligned sentence information "you still say that you" and split text #3 can be combined to generate connected text 3 "you still say that you love me but now you're telling me".
[0076] In operation 526, the alignment module (245) can separate the connected text 3 into sentence units to generate two sentence-unit texts, "1. you still say that you love me" and "2. but now you're telling me". The alignment module (245) can transmit the first completed sentence to the translation module (247) as the alignment text and store the second incomplete sentence as unaligned sentence information.
[0077] In operation 532, the translation module (247) can translate the alignment text received from the alignment module (245) and transmit the translation result 2, which includes the original text and the translation, to the control module (243).
[0078] Referring to FIG. 6, in operation 514, the control module (243) can generate the split text #4 "you have to leave and I'm completely shocked" and send it to the alignment module (245).
[0079] In operation 527, the alignment module (245) can generate the connected text 4 "but now you're telling me you have to leave and I'm completely".
[0080] In operation 528, the alignment module (245) can separate the connected text 4 into sentence units to generate two sentence-unit texts: "1. but now you're telling me you have to leave" and "2. and I'm completely shocked". The alignment module (245) can transmit the preceding completed sentence to the translation module (247) and store the subsequent incomplete sentence as unaligned sentence information.
[0081] In operation 515, the control module (243) can transmit a termination signal label to the alignment module (245) with the split text #5.
[0082] In operation 529, the alignment module (245) indicates a termination signal. <cr>When received, the unaligned sentence information "and I'm completely shocked" can be transmitted to the translation module (247).
[0083] In operation 534, the translation module (247) can translate the alignment text received from the alignment module (245) and transmit the translation result 4, which includes the original text and the translation, to the control module (243).
[0084] Afterwards, the control module (243), alignment module (245), and translation module (247) can each perform division, alignment, and translation functions for the remainder of utterance 1.
[0085] In this way, the interpretation device (200) according to one embodiment performs interpretation whenever a sentence is detected, so that it can provide real-time interpretation services on a sentence-by-sentence basis even when the user's speech continues continuously.
[0086] In addition, the interpretation device (200) according to one embodiment does not simply interpret a silent section as the end of a sentence as in the conventional method, but determines whether it is the end of a sentence or the middle of a sentence according to the sentence structure and performs translation (interpretation) accordingly. Therefore, even when the speaker speaks slowly, it can provide sentence-unit interpretation as needed, thereby ensuring the quality of interpretation to some extent.
[0087] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B or C,” “at least one of A, B and C,” and “at least one of A, B, or C” may each include any one of the items listed together in the corresponding phrase, or any possible combination thereof. Terms such as “first,” “second,” or “first” or “second” may be used simply to distinguish a component from another corresponding component and do not limit the components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as "coupled" or "connected" to another (e.g., 2nd) component, with or without the terms "functionally" or "communicationly," it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.
[0088] As used herein, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).
[0089] Various embodiments of this document may be implemented as software (e.g., a program) comprising one or more instructions stored in a storage medium (e.g., memory (230)) (e.g., internal memory or external memory) that can be read by a machine (e.g., an interpreter device). For example, a processor (e.g., processor (240)) of a device (e.g., interpreter device (200)) may call at least one of one or more instructions stored from a storage medium and execute it. This enables the device to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. A storage medium readable by the device may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily in the storage medium.
[0090] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TM It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.
[0091] Components according to various embodiments of this document may be implemented in software or in hardware form, such as a digital signal processor (DSP), a field programmable gate array (FPGA), or an application-specific integrated circuit (ASIC), and may perform specific roles. The term "components" is not limited to software or hardware, and each component may be configured to reside in an addressable storage medium or configured to run one or more processors. As an example, components may include components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables.
[0092] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the components of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to the integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.< / cr> < / cr>
Claims
1. Regarding interpretation devices, Input device capable of detecting voice input; An STT module that outputs a string converted from the above voice input into text; A control module that generates split text by dividing the string from the above STT module into specified units; an output device; A sorting module that acquires the above-mentioned split text, and if there is previously stored unaligned sentence information, generates a connected text by concatenating the acquired split text and the unaligned sentence information, separates and analyzes the generated connected text into sentence units, outputs a sorted text determined to be a complete sentence, and stores the remaining part excluding the outputted sorted text as unaligned sentence information; An interpretation device comprising a translation module that generates a translation result by translating the output alignment text and outputs it through the output device.
2. In Paragraph 1, The above-mentioned designated unit includes a designated time unit or a designated word count unit, and The above control module is an interpretation device that divides the converted text into the specified time units or into the specified number of words units.
3. In paragraph 1, the control module An interpretation device that divides the output string into the specified units when the string of the unit obtained by adding a threshold to the specified unit is output by the STT module.
4. In paragraph 1, the control module is, An interpretation device that, upon receiving an interpretation request through the above input device, guides the alignment module to the language information of the divided text through an instruction prompt and instructs sentence alignment.
5. In paragraph 1, the control module causes the alignment module, An interpretation device that, when there is no previously stored unaligned sentence information, connects the split text and subsequently input split text to generate the connected text.
6. In paragraph 1, the control module causes the alignment module, An interpretation device that stores the entire linked text as unaligned sentence information when the above linked text does not contain the above completed sentence.
7. In Paragraph 1, An interpretation device wherein the control module adds a silent interval marker when the silent interval in the voice input is longer than or equal to a threshold time, and causes the alignment module to transmit the unaligned sentence information to the translation module when the silent interval marker is confirmed after the last sentence of the connected text.
8. In paragraph 1, the control module is, An interpretation device that, when the input split text is a termination signal indicator, outputs the unaligned sentence information as the aligned text and sets the unaligned sentence information to a NULL value.
9. In paragraph 8, the control module is, An interpretation device that, upon confirming the above termination signal marker, causes the alignment module to transmit the above unaligned sentence information to the translation module.
10. In paragraph 1, the control module is, An interpretation device that, upon receiving an interpretation request through the above-mentioned input device, provides language information of the original text and translation and the output format of the above-mentioned alignment text, and transmits an instruction prompt to the above-mentioned translation module to perform a translation according to the above-mentioned language information and generate the above-mentioned output format.
11. Regarding real-time interpretation methods, The operation of converting real-time voice input into text through an STT module; The operation of dividing the above converted text into specified units to sequentially generate split text; If there is previously stored unaligned sentence information obtained by acquiring the above-mentioned split text, the operation of generating a connected text by connecting the acquired split text and the unaligned sentence information; The operation of separating and analyzing the generated linked text into sentence units and outputting aligned text determined to be a complete sentence; An operation to store the remaining parts excluding the above-mentioned aligned text as unaligned sentence information; and A real-time interpretation method comprising the operation of translating the above-mentioned output alignment text to generate and output a translation result.
12. In Paragraph 11, the above-mentioned designated unit is, A real-time interpretation method that includes a specified time unit or a specified number of words.
13. In Clause 11, the operation of generating the above-mentioned link text is, A real-time interpretation method comprising, in the case where there is no previously stored unaligned sentence information, an operation to generate the connected text by connecting the split text and the subsequently input split text.
14. In paragraph 11, the operation of generating the above-mentioned link text is, A real-time interpretation method comprising the operation of storing the entire linked text as unaligned sentence information when the above linked text does not contain the above completed sentence.
15. In Paragraph 11, If the input split text is a termination signal marker, the operation further includes outputting the previously stored unaligned sentence information as the aligned text and setting the unaligned sentence information to a NULL value. A real-time interpretation method in which the above-mentioned generation and output operation includes the operation of translating the above-mentioned unaligned sentence information when the above-mentioned termination signal marker is confirmed.
16. In Paragraph 11, The operation of generating the above-mentioned split text includes the operation of adding a silent interval marker when the silent interval in the voice input is longer than or equal to a threshold time, and A real-time interpretation method in which the above-mentioned generating and outputting operation includes the operation of translating the above-mentioned unaligned sentence information when the above-mentioned silent section marker is confirmed after the last sentence of the above-mentioned connected text.
