Voice transcription system and voice translation system

Abstract

To obtain a dictation text that does not contain duplication in a dictation of spoken dialogue text between terminals.SOLUTION: A speech dictation system 1 includes: a determination unit 12 that determines a terminal associated with a duplicate text recognized in duplicate among multiple terminals as a duplicate input terminal; a similarity calculation unit 13 that regards a speech text associated with the duplicate input terminal as the target text to be determined, and calculates a similarity between the target text and a neighboring text, which is a speech text located before and after the target text in a dictation text in time series; a duplicate determination unit 14 that determines that the target text is the duplicate text when the similarity is greater than a given threshold value; a deletion unit 15 that deletes the target text determined as the duplicate text; and an output unit 16 that outputs the dictation text from which the duplicate text has been deleted.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] This invention relates to a speech transcription system and a speech translation system. [Background technology]

[0002] Conferencing tools are known that allow interaction between computer-based terminals. In conversations using such conferencing tools, for example, if audio input from one terminal is output from the speaker of another terminal, and the microphone of the other terminal receives the audio output from the speaker, phenomena such as echo and howling occur. To prevent these phenomena, techniques for echo cancellation by processing the audio signal are known. For example, Patent Document 1 discloses a technique for so-called echo cancellation by removing the audio component of the content output from the speaker from the audio component collected by the microphone. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2012-134923 [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] Incidentally, speech transcription systems that record voice conversations between multiple terminals as text are used for creating meeting minutes using conferencing tools. In the terminals used for the conversation, when the voice output from the speaker is input to the microphone, duplication sometimes occurred in the transcribed text of the conversation. Depending on the configuration of the speech transcription system and conferencing tool, it is not always possible to perform appropriate echo cancellation at the voice signal stage, and there was a need to prevent duplication in the transcribed text.

[0005] Therefore, the present invention has been made in view of the above problems, and aims to obtain a transcription text that does not contain repetition when transcribing voice conversations between terminals. [Means for solving the problem]

[0006] To solve the above problems, a speech transcription system relating to one aspect of the present disclosure is a speech transcription system that records a transcript text obtained by speech recognition of speech dialogue between multiple terminals, and which includes speech text associated with each terminal that acquired the speech, comprising: a determination unit that determines, by a predetermined method, which terminals among the multiple terminals may be associated with duplicate text as duplicate input terminals, wherein duplicate text is speech text that is recognized twice in the transcript text based on a single speech sound in the dialogue; a similarity calculation unit that calculates the similarity between the speech text associated with the duplicate input terminal as the target text, neighboring texts which are speech texts that are located before and after the target text in the transcript text in chronological order, and the target text; a duplicate determination unit that determines that the target text is duplicate text when the similarity is equal to or greater than a given threshold; a deletion unit that deletes the target text that has been determined to be duplicate text; and an output unit that outputs the transcript text from which the duplicate text has been deleted.

[0007] Based on the above aspects, the target text can be extracted from the transcribed text by determining the duplicate input terminals that may be associated with the duplicate text. Duplicate text is identical or similar to neighboring text located before or after it in chronological order. By determining that a target text is duplicate text if its similarity to neighboring text is above a predetermined threshold, it is possible to recognize utterances that have been recognized multiple times based on a single utterance. Therefore, by deleting the target text that has been determined to be duplicate text, it is possible to obtain a transcribed text that does not contain duplicates. [Effects of the Invention]

[0008] In the text transcription of voice-based dialogue between terminals, it is possible to obtain a non-duplicate transcription text.

Brief Description of the Drawings

[0009] [Figure 1] It is a diagram showing an example of the device configuration of the voice transcription system of this embodiment. [Figure 2] It is a block diagram showing the functional configuration of the voice transcription device of this embodiment. [Figure 3] It is a hardware block diagram of the voice transcription device. [Figure 4] It is a diagram showing an example of obtaining the transcription text. [Figure 5] It is a diagram showing an example of calculating the similarity between the target text and the neighboring text. [Figure 6] It is a diagram showing an example of calculating the similarity between the target text and the neighboring text and the difference in the utterance timing. [Figure 7] It is a diagram showing an example of the transcription text with duplicate text deleted. [Figure 8] It is a flowchart showing the processing content of the voice transcription method in the voice transcription system. [Figure 9] It is a diagram showing the configuration of the voice transcription program. [Figure 10] It is a block diagram showing the functional configuration of another example of the voice transcription device of this embodiment. [Figure 11] It is a diagram showing an example of the learning data used for machine learning of the duplicate text prediction model. [Figure 12] It is a diagram showing an example of duplicate determination by the duplicate text prediction model. [Figure 13] It is a functional block diagram showing an example of the configuration of the voice translation system including the voice transcription system.

Modes for Carrying Out the Invention

[0010] Embodiments of the speech transcription system and speech translation system according to the present invention will be described with reference to the drawings. Where possible, the same parts will be denoted by the same reference numerals, and redundant descriptions will be omitted.

[0011] Figure 1 shows an example of the device configuration of the speech transcription system according to this embodiment. The speech transcription system 1 is a system that records transcribed text, which includes spoken text obtained by speech recognition of speech dialogue between multiple terminals. Each spoken text included in the transcribed text is associated with the terminal T from which the audio was acquired.

[0012] The speech transcription system 1 illustrated in Figure 1 includes at least two terminals T configured to send and receive information over a network N. In this embodiment, each function of the speech transcription system 1 is implemented, for example, as a speech transcription device 10 configured on terminal 10. Alternatively, each terminal T may have each function of the speech transcription system 1, or the speech transcription system 1 may be configured on a separate server configured to communicate with terminals T.

[0013] Figure 2 shows the functional configuration of the speech transcription device 10 according to this embodiment. As shown in Figure 2, the speech transcription device 10 functionally comprises a text acquisition unit 11, a determination unit 12, a similarity calculation unit 13, a duplicate determination unit 14, a deletion unit 15, and an output unit 16. Each of these functional units 11 to 16 may be configured in a single device or distributed across multiple devices. Each function of the speech transcription device 10 may be implemented by a processor 1001.

[0014] The block diagram shown in Figure 2 represents functional units. These functional blocks (components) are realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one device that is physically or logically coupled, or it may be realized using two or more physically or logically separated devices that are directly or indirectly connected (for example, using wired or wireless connections). A functional block may also be realized by combining the above one device or the above multiple devices with software.

[0015] Functions include, but are not limited to, judgment, decision, judgment, calculation, calculation, processing, derivation, investigation, exploration, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, assumption, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating (mapping), and assigning. For example, a functional block (configuration part) that enables transmission is called a transmitting unit or transmitter. As mentioned above, the method of implementation is not particularly limited.

[0016] For example, the speech transcription device 10 in one embodiment of the present invention may function as a computer. Figure 3 shows an example of the hardware configuration of the speech transcription device 10 according to this embodiment. Physically, the speech transcription device 10 may be configured as a computer device including a processor 1001, memory 1002, storage 1003, communication device 1004, input device 1005, output device 1006, bus 1007, etc.

[0017] In the following explanation, the term "device" can be replaced with "circuit," "device," "unit," etc. The hardware configuration of the speech transcription device 10 may include one or more of the devices shown in the figure, or it may be configured to omit some of the devices.

[0018] Each function in the speech transcription device 10 is realized by loading predetermined software (programs) onto hardware such as the processor 1001 and memory 1002, causing the processor 1001 to perform calculations and control communication by the communication device 1004, as well as the reading and / or writing of data to the memory 1002 and storage 1003.

[0019] The processor 1001 controls the entire computer, for example, by running an operating system. The processor 1001 may consist of a central processing unit (CPU) that includes interfaces with peripheral devices, control units, arithmetic units, registers, etc. For example, the various functional units 11 to 16 shown in Figure 2 may be implemented by the processor 1001.

[0020] Furthermore, the processor 1001 reads programs (program code), software modules, and data from the storage 1003 and / or communication device 1004 into the memory 1002 and executes various processes accordingly. The program used is one that causes the computer to execute at least a part of the operations described in the above embodiment. For example, each functional unit 11 to 16 of the speech transcription device 10 may be stored in the memory 1002 and implemented by a control program that runs on the processor 1001. Although the above processes have been described as being executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented on one or more chips. The program may also be transmitted from a network via a telecommunications line.

[0021] Memory 1002 is a computer-readable recording medium and may consist of at least one of the following: ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. Memory 1002 may also be called a register, cache, main memory, etc. Memory 1002 can store executable programs (program code), software modules, etc., for carrying out the speech transcription method according to one embodiment of the present invention.

[0022] The storage 1003 is a computer-readable recording medium and may consist of at least one of the following: an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disc, a digital multipurpose disc, a Blu-ray® disc), a smart card, flash memory (e.g., a card, a stick, a key drive), a floppy® disk, a magnetic strip, etc. The storage 1003 may also be called an auxiliary storage device. The above-mentioned storage medium may be, for example, a database, server, or other suitable medium including memory 1002 and / or storage 1003.

[0023] The communication device 1004 is hardware (transceiver / receiver device) for communicating between computers via a wired and / or wireless network, and is also referred to as a network device, network controller, network card, communication module, etc.

[0024] The input device 1005 is an input device that accepts input from an external source (e.g., a keyboard, mouse, microphone, switch, button, sensor, etc.). The output device 1006 is an output device that outputs to an external source (e.g., a display, speaker, LED lamp, etc.). The input device 1005 and the output device 1006 may be configured as an integrated unit (e.g., a touch panel).

[0025] Furthermore, each device, such as the processor 1001 and the memory 1002, is connected by a bus 1007 for communicating information. The bus 1007 may consist of a single bus or different buses may be used for communication between devices.

[0026] Furthermore, the speech transcription device 10 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array), and some or all of each functional block may be realized by such hardware. For example, the processor 1001 may be implemented using at least one of these pieces of hardware.

[0027] Next, the various functions of the speech transcription device 10 will be described. The text acquisition unit 11 acquires the transcribed text. In the speech transcription system 1 of this embodiment, for example, the transcribed text is acquired based on the audio information of a conferencing tool that enables voice dialogue between terminals.

[0028] Figure 4 shows an example of obtaining transcribed text. In the example shown in Figure 4, the text acquisition unit 11 acquires the data from user A's terminal T. A The input speech voice sp1 and user B's terminal T are recorded in the above location. BThe system obtains a transcript text ST obtained by speech recognition of the input speech voice sp2. The speech recognition technology used here is not limited; for example, existing speech recognition technologies can be applied.

[0029] The speech texts st11 and st12 included in the transcribed text ST are recorded by the terminal T that acquired the audio on which the speech text is based. A As information to identify the user, the user identification information "User A" is associated with it. In addition, the utterance texts st21 and st22 are associated with the terminal T that acquired the audio on which the utterance texts are based. B As information to identify it, the user identification information "User B" is associated with it.

[0030] The spoken audio sp2 is from user B's terminal T. B This is the sound collected and input at terminal T. However, the spoken voice sp2 is not the spoken voice uttered by user B, but terminal T B The voice of user A output by the speaker is fed back to terminal T via fb. B This is audio input from the microphone. Therefore, the transcribed text ST includes duplicated speech text based on user A's speech.

[0031] The determination unit 12 determines, using a predetermined method, which of the multiple terminals T may be associated with duplicate text as a duplicate input terminal. Duplicate text is spoken text that is recognized twice in the transcribed text ST based on a single spoken audio in the dialogue.

[0032] Specifically, the determination unit 12 may determine that a terminal T whose settings regarding the output mode of the dialogue audio are set to a specific setting that causes duplicate text to occur is a duplicate input terminal.

[0033] Specifically, the determination unit 12 acquires information on the settings regarding the output mode of the voice at each terminal T, for example, via a network. Then, the determination unit 12 determines that the terminal T in which the setting regarding the output mode is a specific setting is a duplicate input terminal.

[0034] The specific setting may be a setting for outputting the received voice of the conversation through a speaker. That is, in the terminal T of user B, when the voice of the conversation received from the terminal T of user A is output from the speaker, the output voice passes through the air and is input from the microphone of the terminal T. Then, the voice input from the microphone is transmitted to the terminal T. Therefore, the text acquisition unit 11 configured in the terminal T acquires the speech text based on the speech voice of user A acquired by the microphone of the terminal T as the transcription text ST, and also acquires the speech text based on the speech voice of user A that is output from the speaker of the terminal T and acquired by the microphone of the terminal T as the transcription text ST. B in the terminal T of user B, A when the voice of the conversation received from the terminal T of user A is output from the speaker, the output voice passes through the air and is input from the microphone of the terminal T. B Then, the voice input from the microphone is transmitted to the terminal T. A Therefore, the terminal T A configured with the text acquisition unit 11 acquires the speech text based on the speech voice of user A acquired by the microphone of the terminal T as the transcription text ST, and also acquires the speech text based on the speech voice of user A that is output from the speaker of the terminal T and acquired by the microphone of the terminal T as the transcription text ST. A configured with the text acquisition unit 11 acquires the speech text based on the speech voice of user A acquired by the microphone of the terminal T as the transcription text ST, and also acquires the speech text based on the speech voice of user A that is output from the speaker of the terminal T and acquired by the microphone of the terminal T as the transcription text ST. B output from the speaker of the terminal T, B and acquired by the microphone of the terminal T as the transcription text ST.

[0035] In this way, by referring to the settings regarding the output mode of the voice at each terminal T, the duplicate input terminal can be accurately determined from the terminals where the voice conversation is being conducted. Also, by determining the terminal having the setting to output the voice through the speaker as the duplicate input terminal, the duplicate input terminal can be surely determined from the terminals where the voice conversation is being conducted.

[0036] Further, when the voice transcription system 1 is configured in one of the plurality of terminals T, the determination unit 12 may determine the other terminals other than the one terminal as duplicate input terminals.

[0037] Specifically, as illustrated in FIG. 4, the terminal T of user A A and the terminal T of user B BA voice conversation takes place between them, and the voice transcription system 1 is sent to terminal T A When configured as such, the determination unit 12 determines that the terminal T of user B B It may be determined that this is a duplicate input terminal. This makes it easy to identify terminals that are highly likely to be duplicate input terminals from among the terminals where voice interaction is taking place.

[0038] The similarity calculation unit 13 extracts the utterance text associated with the duplicate input terminal as the target text, which is the text to be judged. The similarity calculation unit 13 then calculates the similarity between the target text and neighboring texts, which are utterance texts that are located before or after the target text in chronological order in the transcribed text.

[0039] Figure 5 shows an example of calculating the similarity between the target text and neighboring text. As shown in Figure 5, the similarity calculation unit 13 calculates the similarity between user B's terminal T B If it is determined that terminal T is a duplicate input terminal, terminal T will be selected from the transcribed text ST. B Extract the speech text st21 "User B: So, let's start the meeting," which is one of the speech texts associated with the target, as the target text tt1.

[0040] Next, the similarity calculation unit 13 extracts one of the utterances located before or after the target text tt1 in the transcribed text ST, namely st11 "User A: Let's begin the meeting," as the neighboring text nt1.

[0041] The similarity calculation unit 13 then calculates the similarity sm1 between the target text tt1 and the neighboring text nt1. In the example shown in Figure 5, 11 out of 12 characters match between the target text tt1 and the neighboring text nt1, so the calculated similarity sm1 is 91.7%.

[0042] Furthermore, neighboring text may be utterances that are chronologically positioned before the target text in the transcribed text ST. In the transcribed text ST, utterances that are recognized redundantly based on a single utterance are likely to be located relatively later in the chronological order. Therefore, by using utterances that are positioned before the target text as neighboring text and determining whether the target text is redundant, redundant text can be extracted accurately with minimal processing load.

[0043] The similarity calculation unit 13 may calculate the similarity between the target text tt1 and the neighboring text nt1 at the so-called surface level of the text. In other words, the similarity calculation unit 13 does not need to determine the similarity between texts at a semantic level.

[0044] The duplicate detection unit 14 determines that the target text is a duplicate text if the similarity between the target text and neighboring text is equal to or greater than a given threshold.

[0045] In the example shown in Figure 5, if the given threshold for determining similarity is, for example, 80%, the duplicate determination unit 14 determines that the target text tt1 is a duplicate text because the similarity sm1 between the target text tt1 and the neighboring text nt1 is 91.7%.

[0046] Furthermore, the duplicate detection unit 14 may also consider the utterance timing of each utterance text when determining duplicate text. That is, the duplicate detection unit 14 may determine that the target text is duplicate text if the similarity between the neighboring text and the target text is greater than or equal to a given threshold, and the difference between the utterance timing of the neighboring text and the utterance timing of the target text in the dialogue audio is within a given threshold.

[0047] Figure 6 shows an example of calculating the similarity and utterance timing difference between the target text and neighboring text. In the example shown in Figure 6, the utterance timing (time) of the target text tt1 is (12:00:01-12:00:04), and the utterance timing (time) of the neighboring text nt1 is (12:00:00-12:00:03). Therefore, the duplicate detection unit 14 calculates the utterance timing difference sm2 of "1 second". If the given threshold for utterance timing is 3 seconds, the duplicate detection unit 14 determines that the target text tt1 is a duplicate text because the similarity sm1 between the target text tt1 and neighboring text nt1 is 91.7% (≧80%), and the utterance timing difference sm2 is 1 second within 3 seconds.

[0048] Thus, in addition to the similarity with neighboring text being above a given threshold, the condition for determining whether a target text is a duplicate text is that the difference in utterance timing in the audio is within a given threshold. As a result, the accuracy of duplicate text detection is improved, and even if the duplicate texts recognized based on a single utterance do not perfectly match, the similarity threshold can be lowered, allowing for accurate extraction of duplicate text.

[0049] The deletion unit 15 deletes the text that has been determined to be duplicate text from the transcript text. Figure 7 shows an example of deleting duplicate text from the transcript text. As shown in Figure 7, when the duplication determination unit 14 determines that the spoken texts st21 and st22 included in the transcript text ST are duplicate text, the deletion unit 15 generates the transcript text ST2 from which the spoken texts st21 and st22 have been deleted.

[0050] The output unit 16 outputs the transcript text from which duplicate text has been removed. In this way, by removing the text that has been determined to be duplicate text, it is possible to obtain a transcript text that does not contain duplicates.

[0051] Figure 8 is a flowchart showing the processing steps of the speech transcription method in speech transcription system 1.

[0052] In step S1, the text acquisition unit 11 acquires the transcribed text.

[0053] In step S2, the determination unit 12 determines which of the multiple terminals T may be duplicate input terminals associated with the duplicate text.

[0054] In step S3, the similarity calculation unit 13 extracts the utterance text associated with the duplicate input terminal as the target text.

[0055] In step S4, the similarity calculation unit 13 calculates the similarity between the target text and neighboring texts located before and after the target text in chronological order in the transcribed text.

[0056] In step S5, the duplicate determination unit 14 determines that the target text is a duplicate text if the similarity between the target text and the neighboring text is greater than or equal to a given threshold. The duplicate determination unit 14 may also consider the utterance timing of each utterance text. That is, the duplicate determination unit 14 may determine that the target text is a duplicate text if the similarity between the neighboring text and the target text is greater than or equal to a given threshold, and the difference between the utterance timing of the neighboring text and the utterance timing of the target text in the dialogue audio is within a given threshold.

[0057] In step S6, the deletion unit 15 deletes the text that has been determined to be duplicate text from the transcribed text.

[0058] In step S7, the similarity calculation unit 13 determines whether all target texts have been determined to be duplicates. If it is determined that all target texts have been determined to be duplicates, the process proceeds to step S8. On the other hand, if it is not determined that all target texts have been determined to be duplicates, the process returns to step S3, and steps S3 to S6 are repeated.

[0059] In step S8, the output unit 16 outputs the transcript text from which duplicate text has been removed.

[0060] Furthermore, the speech transcription system 1 of this embodiment may also be applied to a system that performs real-time sequential transcription. In this case, the text acquisition unit 11 sequentially acquires the spoken text that constitutes the transcription text while the dialogue is ongoing.

[0061] The duplicate detection unit 14 sequentially determines whether a target text, which is an utterance text associated with a duplicate input terminal, is a duplicate text when the amount of acquired utterance text exceeds a given amount, based on the similarity between the target text and neighboring texts, etc. For example, the duplicate detection unit 14 may perform the determination when the acquired utterance text exceeds a predetermined number of characters (e.g., 10 characters).

[0062] The output unit 16 then sequentially outputs the transcript text with duplicate text removed. By outputting the transcript text in this way, it is possible to obtain the transcript text with duplicate text removed even while the dialogue is ongoing. The duplicate detection unit 14 may, after the end of the dialogue, determine again whether each utterance text is duplicate text, and the output unit 16 may output the transcript text with duplicate text removed again after the end of the dialogue.

[0063] Next, with reference to Figure 9, a speech transcription program for causing a computer to function as the speech transcription system 1 of this embodiment will be described. Figure 9 is a diagram showing the configuration of the speech transcription program. The speech transcription program P1 is composed of a main module m10 that comprehensively controls the speech transcription process in the speech transcription system 1 (speech transcription device 10), a text acquisition module m11, a judgment module m12, a similarity calculation module m13, a duplicate judgment module m14, a deletion module m15, and an output module m16. Each module m11 to m16 realizes the respective functions for each functional unit 11 to 16.

[0064] The audio transcription program P1 may be transmitted via a transmission medium such as a communication line, or it may be stored on a recording medium M1, as shown in Figure 9.

[0065] Figure 10 is a block diagram showing the functional configuration of another example of the speech transcription system of this embodiment. As shown in Figure 10, the speech transcription device 10A constituting the speech transcription system 1A functionally comprises a text acquisition unit 11A, a duplicate detection unit 14A, a deletion unit 15A, and an output unit 16A. The text acquisition unit 11A, the duplicate detection unit 14A, the deletion unit 15A, and the output unit 16A may be implemented by a processor 1001.

[0066] The text acquisition unit 11A acquires transcribed text obtained through speech recognition of voice dialogues between multiple terminals. The transcribed text includes the utterance text associated with each terminal that acquired the audio.

[0067] The duplicate detection unit 14A includes a duplicate text prediction model md, inputs a predetermined number of consecutive utterances in a time series contained in the transcribed text to the duplicate text prediction model md, and obtains a label output from the duplicate text prediction model md indicating whether or not each utterance is duplicate text.

[0068] The duplicate text prediction model MD is a binary classification model constructed using machine learning with training data that includes a predetermined number of consecutive spoken utterances in a time series of transcribed text, and a binary label indicating whether each utterance is a duplicate or not. The duplicate text prediction model MD may also be a model that includes a neural network.

[0069] Figure 11 shows an example of training data used for machine learning of the duplicate text prediction model md. As shown in Figure 11, the training data used for machine learning of the duplicate text prediction model md consists of pairs of input information (explanatory variables) that include a predetermined number of consecutive utterances in the transcribed text and information that identifies the terminal associated with each utterance, and output information (target variable) that includes a label indicating whether each utterance is a duplicate or not (1: duplicate text, 0: non-duplicate text).

[0070] In the machine learning of the duplicate text prediction model (md), input information from the training data is fed into the input layer of the md duplicate text prediction model, and the weights, parameters, etc. that make up the model are adjusted and updated so that the error between the information output from the output layer and the output information from the training data is minimized.

[0071] The trained duplicate text prediction model md outputs a label indicating whether each utterance is a duplicate text, in response to the input of a predetermined number of consecutive utterances in time series.

[0072] Figure 12 shows an example of duplicate text detection of spoken text using the duplicate text prediction model md. The duplicate detection unit 14A inputs a predetermined number (7 in the example in Figure 12) of consecutive spoken text IDs in the time series contained in the transcribed text to the duplicate text prediction model md. The duplicate detection unit 14A then obtains a label od indicating whether each spoken text is a duplicate text or not.

[0073] The duplicate text prediction model MD, which includes a pre-trained neural network, can be viewed as a program that is loaded or referenced by a computer, causing the computer to perform predetermined processes and realize predetermined functions.

[0074] In other words, the trained duplicate text prediction model md of this embodiment is used in a computer equipped with a CPU and memory. Specifically, the computer's CPU operates in accordance with instructions from the trained duplicate text prediction model md stored in memory, performing calculations on the input data input to the input layer of the neural network based on the trained weight coefficients (parameters) and response functions corresponding to each layer, and outputting the result (probability) from the output layer.

[0075] The deletion unit 15A removes the spoken text that is identified as duplicate text by a label from the transcript. The output unit 16A outputs the transcript from which the duplicate text has been removed.

[0076] According to the speech transcription system 1A, the trained duplicate text prediction model md provides information indicating whether or not the spoken text included in the transcription is duplicate text, based on the input of the spoken text. This allows for efficient extraction of duplicate text from the transcription. By deleting the extracted duplicate text, it becomes possible to obtain a transcription that does not contain duplicates.

[0077] Figure 13 is a functional block diagram showing an example of the configuration of a speech translation system including speech transcription systems 1 and 1A. Speech translation system 2 is a system that outputs translated text, which is the content of a speech conversation between multiple terminals translated into a predetermined language.

[0078] As shown in Figure 13, the speech translation system 2 includes a speech transcription system 1,1A and comprises a translation unit 21 and a translation output unit 22.

[0079] The translation unit 21 translates the transcript text, from which duplicate text has been removed and output by the output units 16, 16A of the speech transcription system 1, 1A, into a predetermined language to generate translated text. Alternatively, the translation unit 21 may translate the transcript text into text in the predetermined language using existing translation processing technology.

[0080] The translation output unit 22 outputs the translated text generated by the translation unit 21. In this way, since the translated text is generated based on the transcribed text from which duplicate text has been removed by the speech transcription system 1,1A, it is possible to obtain translated text that does not include translations of speech text that have been recognized multiple times based on a single speech audio. In addition, since the transcribed text from which duplicate text has been removed is translated, the processing load related to the translation process can be reduced.

[0081] According to the speech transcription system 1, speech transcription device 10, speech transcription method, and speech transcription program P1 of this embodiment described above, the target text can be extracted from the transcribed text by determining the duplicate input terminal that may be associated with the duplicate text. Duplicate text is identical or similar to neighboring text located before or after it in chronological order. By determining that a target text whose similarity to neighboring text is above a predetermined threshold is a duplicate text, it is possible to recognize speech text that has been recognized multiple times based on a single speech utterance. Therefore, by deleting the target text that has been determined to be a duplicate text, it is possible to obtain a transcribed text that does not contain duplicates.

[0082] The invention described herein can be understood, for example, as follows:

[0083] A first aspect of the present disclosure is a speech transcription system that records a transcript obtained by speech recognition of a speech dialogue between multiple terminals, and which includes speech text associated with each terminal that acquired the speech, comprising: a determination unit that determines, by a predetermined method, which terminals among the multiple terminals may be associated with duplicate text as duplicate input terminals, wherein duplicate text is speech text that is recognized twice in the transcript text based on a single utterance in the dialogue; a similarity calculation unit that calculates the similarity between the speech text associated with the duplicate input terminal as the target text, neighboring texts which are speech texts that are located before and after the target text in the transcript text in chronological order, and the target text; a duplicate determination unit that determines that the target text is duplicate text if the similarity is greater than or equal to a given threshold; a deletion unit that deletes the target text that has been determined to be duplicate text; and an output unit that outputs the transcript text from which the duplicate text has been deleted.

[0084] Based on the above aspects, the target text can be extracted from the transcribed text by determining the duplicate input terminals that may be associated with the duplicate text. Duplicate text is identical or similar to neighboring text located before or after it in chronological order. By determining that a target text is duplicate text if its similarity to neighboring text is above a predetermined threshold, it is possible to recognize utterances that have been recognized multiple times based on a single utterance. Therefore, by deleting the target text that has been determined to be duplicate text, it is possible to obtain a transcribed text that does not contain duplicates.

[0085] In the speech transcription system relating to the second aspect, the determination unit in the speech transcription system relating to the first aspect may determine that a terminal whose settings regarding the output manner of the dialogue audio are set to a specific setting that causes duplicate text to occur is a duplicate input terminal.

[0086] Based on the above aspects, it is possible to accurately identify duplicate input terminals from among terminals where voice interaction is taking place.

[0087] In the speech transcription system relating to the third aspect, the specific setting in the speech transcription system relating to the second aspect may be a setting to output the dialogue audio through a speaker.

[0088] Based on the above aspects, it is possible to reliably identify duplicate input terminals from among terminals where voice interaction is taking place.

[0089] In the speech transcription system relating to the fourth aspect, in the speech transcription system relating to the first aspect, the determination unit may determine that, when the speech transcription system is configured on one of multiple terminals, the terminals other than the one terminal in question are duplicate input terminals.

[0090] Based on the above aspects, it is possible to identify terminals that are highly likely to be duplicate input terminals from among terminals where voice interaction is taking place.

[0091] In the speech transcription system relating to the fifth aspect, in the speech transcription system relating to any one of the first to fourth aspects, the duplicate determination unit may determine that the target text is duplicate text if the similarity with neighboring text is greater than or equal to a given threshold, and the difference between the utterance timing of neighboring text in the speech dialogue and the utterance timing of the target text is within a given threshold.

[0092] Based on the above aspects, the condition for determining whether a target text is a duplicate text is that, in addition to the similarity with neighboring text being above a given threshold, the difference in utterance timing in the audio is within a given threshold. Therefore, even if the duplicate texts recognized based on a single utterance do not perfectly match, the similarity threshold can be lowered to perform the determination, allowing for accurate extraction of duplicate text.

[0093] In the speech transcription system relating to the sixth aspect, in the speech transcription system relating to any one of the first to fifth aspects, the neighboring text may be the spoken text that is chronologically positioned before the target text in the transcribed text.

[0094] Based on the above aspects, in the transcribed text, utterances recognized redundantly based on a single utterance are likely to be located later in the timeline. Therefore, by using utterances located before the target text as neighboring texts and determining whether the target text is a duplicate, duplicate text can be extracted accurately with minimal processing load.

[0095] In the speech transcription system relating to the seventh aspect, the system further comprises a text acquisition unit that acquires transcription text, wherein the speech transcription system relating to any one of the first to sixth aspects acquires utterance texts that constitute the transcription text sequentially during and after the conversation, a duplicate determination unit sequentially determines whether the target text is duplicate text when the amount of acquired utterance text exceeds a given amount, and an output unit sequentially outputs the transcription text from which the duplicate text has been removed.

[0096] Based on the above aspects, spoken text is acquired not only after the dialogue has ended but also while the dialogue is ongoing. As a result, the extraction of target text and the determination of whether the target text is duplicate text are performed, and a transcript with duplicate text removed can be obtained even while the dialogue is ongoing.

[0097] The speech recognition system relating to the eighth aspect is a speech transcription system that records a transcript text obtained by speech recognition of a speech dialogue between multiple terminals, and which includes the utterance text associated with each terminal that acquired the speech, and includes a duplicate determination unit that inputs a predetermined number of consecutive utterance texts in a time series included in the transcript text into a duplicate text prediction model and obtains a label from the duplicate text prediction model indicating whether or not each utterance text is a duplicate text, wherein a duplicate text is an utterance text that has been recognized twice in the transcript text based on a single utterance in the dialogue, a deletion unit that deletes the utterance text that has been indicated as a duplicate text by the label from the transcript text, and an output unit that outputs the transcript text from which the duplicate text has been deleted, wherein the duplicate text prediction model is a binary classification model constructed by machine learning using training data that includes a predetermined number of consecutive utterance texts in the time series of the transcript text and a label that indicates binary whether or not each utterance text is a duplicate text, and outputs a label indicating whether or not each utterance text is a duplicate text in response to the input of a predetermined number of consecutive utterance texts in a time series.

[0098] Based on the aspects described above, a trained duplicate text prediction model can provide information indicating whether or not a spoken text is duplicate text, depending on the input of the transcribed text. This allows for efficient extraction of duplicate text from the transcribed text. By removing the extracted duplicate text, it becomes possible to obtain a transcribed text that does not contain duplicates.

[0099] The first aspect of the speech translation system includes a speech transcription system relating to any one of the first to eight aspects, and is a speech translation system that outputs translated text, which is text obtained by translating the content of a speech dialogue between multiple terminals into a predetermined language, and comprises a translation unit that generates translated text by translating the transcript text, from which duplicate text has been removed and output by an output unit, into a predetermined language, and a translation output unit that outputs the translated text.

[0100] Based on the above aspects, since the speech transcription system generates translated text based on the transcribed text from which duplicate text has been removed, it is possible to obtain translated text that does not include translations of duplicated speech text recognized based on a single speech audio. In addition, since the transcribed text from which duplicate text has been removed is translated, the processing load related to the translation process can be reduced.

[0101] Although the present disclosure has been described in detail above, it will be clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the intent and scope of the present disclosure as defined by the claims. Therefore, the descriptions in the present disclosure are illustrative and not intended to be restrictive in any way.

[0102] Information notification is not limited to the embodiments described herein and may be carried out by other means. For example, information notification may be carried out by physical layer signaling (e.g., DCI (Downlink Control Information), UCI (Uplink Control Information)), upper layer signaling (e.g., RRC (Radio Resource Control) signaling, MAC (Medium Access Control) signaling, broadcast information (MIB (Master Information Block), SIB (System Information Block))), other signals, or combinations thereof. RRC signaling may also be called RRC messages, and may be, for example, RRC Connection Setup messages, RRC Connection Reconfiguration messages, etc.

[0103] Each aspect / embodiment described herein may be applied to systems utilizing LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G, 5G, FRA (Future Radio Access), W-CDMA®, GSM®, CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20, UWB (Ultra-WideBand), Bluetooth®, and other appropriate systems, and / or next-generation systems extended based thereon. Furthermore, multiple systems may be applied in combination (for example, a combination of at least one of LTE and LTE-A with 5G).

[0104] The processing procedures, sequences, flowcharts, etc., of each aspect / embodiment described herein may be reordered, provided they are consistent with each other. For example, the methods described herein present the elements of various steps in an exemplary order and are not limited to that specific order.

[0105] The specific operations described in this disclosure as being performed by a base station may, in some cases, be performed by its upper node. In a network consisting of one or more network nodes having a base station, it is clear that various operations performed for communication with a terminal can be performed by the base station and at least one other network node (for example, an MME or S-GW, but not limited to these). Although the above example illustrates a case where there is one other network node besides the base station, it may also be a combination of multiple other network nodes (for example, an MME and an S-GW).

[0106] Information can be output from a higher layer (or lower layer) to a lower layer (or higher layer). Input and output may also occur via multiple network nodes.

[0107] Input and output information may be stored in a specific location (e.g., memory) or managed in a management table. Input and output information may be overwritten, updated, or appended to. Output information may be deleted. Input information may be sent to other devices.

[0108] The determination may be made by a value represented by 1 bit (0 or 1), by a boolean value (true or false), or by a numerical comparison (for example, a comparison with a predetermined value).

[0109] Each aspect / embodiment described herein may be used individually, in combination, or switched between as needed during implementation. Furthermore, notification of specific information (e.g., notification that "X is") is not limited to explicit notification, but may also be implicit (e.g., by not providing such notification).

[0110] Software should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, and so on, whether they are called software, firmware, middleware, microcode, hardware description languages, or by any other name.

[0111] Furthermore, software, instructions, etc., may be transmitted and received via a transmission medium. For example, if software is transmitted from a website, server, or other remote source using wired technologies such as coaxial cable, fiber optic cable, twisted pair, and digital subscriber lines (DSL) and / or wireless technologies such as infrared, radio, and microwave, these wired and / or wireless technologies are included in the definition of a transmission medium.

[0112] The information, signals, etc. described in this disclosure may be represented using any of the various different techniques. For example, the data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0113] In addition, terms described in this disclosure and / or terms necessary for understanding this specification may be replaced with terms having the same or similar meaning.

[0114] The terms “system” and “network” as used herein are interchangeable.

[0115] Furthermore, the information, parameters, etc., described herein may be expressed as absolute values, relative values ​​from a given value, or by corresponding other information. For example, wireless resources may be indicated by an index.

[0116] The names used for the parameters described above are not restrictive in any way. Furthermore, the formulas and other expressions using these parameters may differ from those expressly disclosed in this disclosure. Various channels (e.g., PUCCH, PDCCH, etc.) and information elements can be identified by any suitable name, and therefore, the various names assigned to these various channels and information elements are not restrictive in any way.

[0117] As used in this disclosure, the terms “determining” and “determining” may encompass a wide variety of actions. “Determining” may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, inquiry (e.g., searching in a table, database, or other data structure), and ascertaining. “Determining” may also include, for example, receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, and accessing (e.g., accessing data in memory). Furthermore, "judgment" and "decision" can include considering something as having been "judged" or "decided" after resolving, selecting, choosing, establishing, comparing, etc. In other words, "judgment" and "decision" can include considering something as having been "judged" or "decided" after some action. Also, "judgment (decision)" can be reinterpreted as "assuming," "expecting," or "considering."

[0118] As used in this disclosure, the phrase "based on" does not mean "based solely on" unless otherwise specified. In other words, the phrase "based on" means both "based solely on" and "based at least on."

[0119] Where the designations “first,” “second,” etc., are used herein, no reference to those elements shall generally limit the quantity or order of those elements. These designations may be used herein as a convenient way to distinguish between two or more elements. Thus, references to the first and second elements shall not imply that only two elements may be employed therein, or that the first element must precede the second element in any way.

[0120] To the extent that “include,” “including,” and their variations are used herein or in the claims, these terms are intended to be inclusive, as is the term “comprising.” Furthermore, the term “or” as used herein or in the claims is not intended to be exclusive OR.

[0121] In this disclosure, if articles are added through translation, such as a, an, and the in English, this disclosure may include the fact that the noun following these articles is plural.

[0122] In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combine" may be interpreted similarly to "different." [Explanation of Symbols]

[0123] 1,1A…Speech transcription system, 10,10A…Speech transcription device, 11,11A…Text acquisition unit, 12…Determination unit, 13…Similarity calculation unit, 14,14A…Duplicate detection unit, 15,15A…Deletion unit, 16,16A…Output unit, 21…Translation unit, 22…Translation output unit, M1…Recording medium, m10…Main module, m11…Text acquisition module, m12…Determination module, m13…Similarity calculation module, m14…Duplicate detection module, m15…Deletion module, m16…Output module, md…Duplicate text prediction model, N…Network, P1…Speech transcription program, T,T A ,T B ...specified.

Claims

1. A speech transcription system that records transcribed text obtained through speech recognition of voice dialogue between multiple terminals, including the utterance text associated with each terminal that acquired the voice, A determination unit that determines, by a predetermined method, which of the plurality of terminals may be associated with duplicate text, as a duplicate input terminal, wherein the duplicate text is a spoken text that is recognized twice in the transcribed text based on a single spoken audio in the dialogue, The utterance text associated with the duplicate input terminal is designated as the target text, and the similarity calculation unit calculates the similarity between the neighboring utterance texts located before and after the target text in the transcribed text in chronological order and the target text. A duplicate determination unit determines that the target text is a duplicate text if the similarity is equal to or greater than a given threshold, A deletion unit that deletes the target text that has been determined to be duplicate text, An output unit that outputs the transcribed text from which the duplicate text has been removed, A speech-to-text transcription system equipped with [features / equipment].

2. The determination unit determines that a terminal whose settings regarding the output mode of the dialogue audio are set to a specific setting that causes the duplicate text to occur is the duplicate input terminal. The speech transcription system according to claim 1.

3. The aforementioned specific setting is a setting to output the audio of the dialogue through a speaker. The speech transcription system according to claim 2.

4. The determination unit determines that, when the voice transcription system is configured on one of the multiple terminals, the other terminals other than the one specified as the duplicate input terminal. The speech transcription system according to claim 1.

5. The duplicate determination unit determines that the target text is the duplicate text if the similarity with the neighboring text is greater than or equal to a given threshold, and the difference between the utterance timing of the neighboring text and the utterance timing of the target text in the audio of the dialogue is within a given threshold. The speech transcription system according to any one of claims 1 to 4.

6. The aforementioned neighboring text is the utterance text that is located chronologically before the target text in the transcribed text. The speech transcription system according to claim 1.

7. The system further comprises a text acquisition unit that acquires the aforementioned transcribed text, The text acquisition unit sequentially acquires the spoken text that constitutes the transcribed text during and after the conversation. The duplicate determination unit sequentially determines whether the target text is the duplicate text when the amount of acquired speech text exceeds a given amount. The output unit sequentially outputs the transcribed text from which the duplicate text has been removed. The speech transcription system according to claim 1.

8. A speech transcription system that records transcribed text obtained through speech recognition of voice dialogue between multiple terminals, including the utterance text associated with each terminal that acquired the voice, A duplicate determination unit inputs a predetermined number of consecutive utterances in a time series contained in the transcribed text into a duplicate text prediction model and obtains a label from the duplicate text prediction model indicating whether each utterance is a duplicate text, wherein the duplicate text is an utterance that is recognized twice in the transcribed text based on a single utterance in the dialogue. A deletion unit that deletes the spoken text that is identified as duplicate text by the label from the transcribed text, The system includes an output unit that outputs the transcribed text from which the duplicate text has been removed, The duplicate text prediction model is a binary classification model constructed by machine learning using training data that includes a predetermined number of consecutive utterances in the time series of the transcribed text and a binary label indicating whether each utterance is a duplicate text, and outputs a label indicating whether each utterance is a duplicate text in response to the input of the predetermined number of consecutive utterances in the time series. A voice transcription system.

9. A speech translation system comprising the speech transcription system described in claim 1, which outputs translated text that is text obtained by translating the content of the speech dialogue between the plurality of terminals into a predetermined language, A translation unit that translates the transcribed text output by the output unit, from which the duplicate text has been removed, into the predetermined language to generate the translated text, A translation output unit that outputs the aforementioned translated text, A voice translation system equipped with [features / equipment].

Images