Voice recognition system and voice recognition method

Abstract

To provide means of preventing shortage of a memory in conventional voice recognition.SOLUTION: A voice recognition system executes detection processing for detecting delimitation that is separated by predetermined criteria in text data that is a result of voice recognition processing. Then, the voice recognition system retains the text data prior to the delimitation detected by the detection processing, executes discarding processing in which voice data corresponding to the text data and data temporarily generated in a storage part in executing the voice recognition processing, and executes the voice recognition processing on the voice data excluding sections discarded by the discarding processing.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a voice recognition system and a voice recognition method. [Background technology]

[0002] Various conventional speech recognition methods are known, and one of them is the following method. That is, when recognizing an input speech, a lattice is generated, which is a graph of output candidates in which one or more characters or words that may constitute a spoken sentence are expressed as a directed graph, and the most likely candidate is selected from the candidates in the lattice (hereinafter, this method is referred to as the "conventional method"). However, the longer the input speech time, the larger the lattice becomes, and a large amount of memory is required. For this reason, in speech recognition using the conventional method, for example, voice activity detection (VAD) is used to extract speech segments and silent segments are removed, thereby limiting data input to speech recognition at one time to only the speech segments, thereby suppressing memory consumption.

[0003] However, in a speech recognition system that restricts data using VAD and performs speech recognition using conventional methods, if the VAD recognizes noise as speech, and if the speech section cannot be properly extracted from the sound source, the noise will also become the speech section, so data restriction is ineffective and the problem of memory shortages cannot be solved.

[0004] Patent Document 1 discloses a speech-to-text device that divides voice data at a predetermined unit time to generate divided data and deletes silent parts from the divided data in order to provide accurate text data without placing a burden on an engine that converts voice data into text data in a speech-to-text process. Cited Document 2 discloses a device that determines whether an input voice is voiced or unvoiced for each analysis interval and sets only the voiced interval as the target interval for network input. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2019-090917 A [Patent Document 2] Japanese Patent Application Publication No. 05-316197 Summary of the Invention [Problem to be solved by the invention]

[0006] As described above, in speech recognition using conventional methods, when noise is recognized as speech and a speech section cannot be appropriately extracted from the sound source, there is a problem of running out of memory.

[0007] An object of the present invention is to provide a means for preventing memory shortage in conventional speech recognition. [Means for solving the problem]

[0008] A first aspect of the present invention is a voice recognition system that performs a voice recognition process that stores voice data in a storage unit, recognizes voice from the voice data stored in the storage unit using the storage unit as a working area, and converts the voice data into text data, a detection process that detects a boundary that is separated according to a predetermined criterion in the text data that is the result of the voice recognition process, and a discarding process that retains the text data stored in the storage unit that precedes the boundary detected by the detection process, and discards the voice data corresponding to the text data and data temporarily generated in the storage unit when performing the voice recognition process, and performs the voice recognition process on the voice data excluding the section discarded by the discarding process.

[0009] In the present invention, the speech recognition system executes a detection process to detect a segment that is separated according to a predetermined criterion in text data that is the result of the speech recognition process. The speech recognition system then holds the text data before the segment detected by the detection process, executes a discard process to discard the speech data corresponding to the text data and data that is temporarily generated in the storage unit when executing the speech recognition process, and executes the speech recognition process on the speech data excluding the section discarded by the discard process. In this way, if a certain period of filler, a sequence of specific words, or a period is set as the segment of the predetermined criterion, the sentence will not be separated in the middle of the sentence, and it is possible to prevent erroneous recognition due to a segment being separated in the middle of the sentence.

[0010] Furthermore, as described above, the voice data stored in the storage unit and corresponding to the text data preceding the delimiter detected by the detection process and the data temporarily generated in the storage unit when executing the voice recognition process are discarded, so that the storage capacity (memory) of the storage unit provided in the voice recognition system can be freed up and the storage capacity (memory) of the storage unit can be prevented from being depleted due to long-term voice recognition.

[0011] The voice recognition system of the second invention is the voice recognition system of the first invention, characterized in that in the detection process, divisions determined according to predetermined criteria are defined as sentence divisions, and sentence divisions are detected in the text data.

[0012] In the present invention, the speech recognition system detects the position of the voice at the end of a sentence by taking a segment that is separated according to a predetermined criterion in the detection process, thereby making it possible to detect an appropriate segment from text data.

[0013] The voice recognition system of the third invention is characterized in that, in the voice recognition system of the first invention, in the detection process, a segment separated by a predetermined criterion is defined as a segment where a filler or a specific word continues for a certain period of time, and that a filler or a specific word continues for a certain period of time in the text data is detected.

[0014] In the present invention, the voice recognition system detects a certain period of time of a filler or a specific word (e.g., "hello," "yes," etc.) as a segment that is segmented according to a predetermined criterion in the detection process, and detects that the filler or the specific word continues for a certain period of time. This makes it possible to detect an appropriate segment from text data.

[0015] Here, a "filler" is an utterance or transition word that fills the gap while a speaker is choosing the next word when speaking (such as in a conversation). For example, "well," "uh," "eh," "um," etc.

[0016] A fourth aspect of the present invention is the voice recognition system of the first aspect of the present invention, characterized in that in the detection process, a break defined by a predetermined criterion is regarded as a period, and a period is detected in the text data.

[0017] In the present invention, the voice recognition system detects the positions of periods, which are determined based on a predetermined criterion, in the detection process, thereby making it possible to detect appropriate periods from text data.

[0018] The speech recognition system of the fifth invention is the speech recognition system of the first invention, characterized in that in the speech recognition process, a lattice, which is a graph of output candidates expressed as a directed graph, is generated from the speech data, and the most likely candidate is selected from the candidates of the generated lattice, thereby recognizing the speech.

[0019] The voice recognition system of the sixth invention is the voice recognition system of the first invention, characterized in that a voice activity detection process is performed to detect a voice activity in the voice data before the voice recognition process is performed, and the voice recognition process is performed on the voice data after the voice activity detection process.

[0020] A voice recognition system according to a seventh aspect of the present invention is the voice recognition system according to the first aspect of the present invention, characterized in that in the voice recognition process, voice is recognized from the voice data at predetermined intervals.

[0021] The voice recognition system of an eighth aspect of the present invention is the voice recognition system of the seventh aspect of the present invention, characterized in that the predetermined intervals are constant intervals or intervals that are different from each other.

[0022] A ninth aspect of the invention is the voice recognition system of the first aspect of the invention, characterized in that, in the discarding process, the text data stored in the memory unit and following the boundary detected by the detection process is discarded.

[0023] A voice recognition method of the tenth invention includes the steps of: storing voice data in a storage unit; recognizing voice from the voice data stored in the storage unit using the storage unit as a working area; and converting the voice data into text data; detecting a boundary that is determined based on a predetermined criterion in the text data that is the result of the voice recognition process; and discarding a voice data corresponding to the text data and data temporarily generated in the storage unit when performing the voice recognition process, the voice recognition method comprising the steps of: storing voice data in a storage unit;

[0024] The speech recognition method of an eleventh invention is the speech recognition method of the tenth invention, characterized in that in the detection process, a segment separated according to a predetermined criterion is defined as a segment continued by a filler or a specific word for a certain period of time, and that a segment continued by a filler or a specific word for a certain period of time is detected in the text data.

[0025] A speech recognition method of a twelfth aspect of the present invention is the speech recognition method of the tenth aspect of the present invention, characterized in that in the detection process, a break defined by a predetermined criterion is regarded as a period, and a period is detected in the text data. Effect of the Invention

[0026] According to the present invention, it is possible to provide a means for preventing memory shortage in speech recognition. [Brief description of the drawings]

[0027] [Figure 1] 4 is a flowchart showing a processing operation of the voice recognition system according to the present embodiment. [Diagram 2] 11A and 11B are diagrams for explaining a process for detecting that a filler has continued for a certain period of time. [Diagram 3] 11A and 11B are diagrams for explaining a process of detecting the position of a period; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] Hereinafter, an embodiment of the present invention will be described. A voice recognition system according to an embodiment of the present invention executes a voice recognition process that recognizes voice from voice data and converts it into text data. For example, the voice recognition system is configured with a personal computer (hereinafter referred to as "PC"). A CPU (Central Processing Unit) that functions as a control unit of a PC that operates as a voice recognition system executes voice recognition process and the like in accordance with a program that causes the PC to function as a voice recognition system. The CPU executes the voice recognition process using a RAM (Random Access Memory) (storage unit) that functions as a work area.

[0029] The voice recognition system may be configured with a cloud server or the like. Even in this case, the voice recognition process and the like may be executed by a CPU or the like of the cloud server. Hereinafter, although the voice recognition process and the like are executed by a CPU or the like, the expression "the CPU executes the process" will not be used, but rather "the voice recognition system executes the process".

[0030] A voice recognition system performs a voice activity detection (VAD) process to detect a voice section in voice data, and then performs a voice recognition process on the voice data that has undergone the voice activity detection process. In a conventional voice recognition system, the voice activity detection process limits the voice data to only the voice section, and a lattice is generated to perform the voice recognition process. In a sound source in which the voice section cannot be properly extracted, noise is recognized as a voice in the VAD, and noise is also considered to be a voice section, so that the data restriction is ineffective, and there is a problem of insufficient storage capacity (memory) in the storage unit.

[0031] In order to solve the above problems, the voice recognition system according to this embodiment performs a detection process for detecting a break that is determined based on a predetermined criterion in text data that is the result of the voice recognition process, and a discard process for storing in a storage unit only the text data preceding the break that is detected by the detection process, and discarding the voice data corresponding to the text data and the data that is temporarily generated in the storage unit when executing the voice recognition process. In the detection process, the voice recognition system considers a break that is determined based on a predetermined criterion as a break in a sentence, and detects a break in the text data as a sentence. The processing operations of the voice recognition system that perform the detection process, discard process, etc. will be described below.

[0032] 1 is a flowchart showing the processing operation of the voice recognition system according to this embodiment. The voice recognition system executes a voice activity detection process to detect a voice activity (S1). In this embodiment, in order to reduce the amount of voice data to be subjected to the voice recognition process, the voice activity detection process is performed to limit the voice data to a voice activity, but the amount of voice data may be reduced by other processes such as diarization.

[0033] Next, the voice recognition system executes a voice recognition process to recognize voice from the voice data and convert it into text data (S2). The voice recognition system stores the voice data in a storage unit, recognizes voice from the voice data stored in the storage unit using the storage unit as a working area, and converts it into text data. The voice recognition system executes the voice recognition process from the voice data at a predetermined interval. For example, the voice recognition system recognizes voice from the voice data every 10 seconds. The voice recognition system may recognize voice from the voice data at a fixed interval (e.g., 10 seconds), or may recognize voice from the voice data at different intervals (e.g., 15 seconds, 10 seconds).

[0034] In speech recognition processing, a speech recognition system generates a lattice, which is a graph of output candidates expressed as a directed graph, from speech data, and recognizes speech by selecting the most likely candidate from the candidates in the generated lattice. The lattice is made up of nodes (points) and edges (lines), and each node and edge has a cost that determines the likelihood, and the lower the cost, the higher the likelihood.

[0035] For example, first, a word or phrase is treated as a node, a phoneme is identified from an acoustic model that outputs the probability that a feature extracted from speech is close to which phoneme, and combinations of phonemes are converted into words using a pronunciation dictionary that records the correspondence between phoneme strings and words. Next, a language model is used to model the probability of the context of word sequences from a large amount of text, increasing the probability of frequently occurring word combinations and decreasing the probability of rarely occurring word combinations. This makes it possible to output the most likely (lowest cost) candidate from the generated lattice candidates as the speech recognition result.

[0036] Next, the voice recognition system executes a detection process to detect a segment that is separated according to a predetermined criterion in the text data that is the result of the voice recognition process (S3). Specifically, as described above, the voice recognition system regards a segment that is separated according to a predetermined criterion as a segment of a sentence, and detects a segment of a sentence in the text data. Further details will be described later.

[0037] Next, the speech recognition system holds the text data before the detected delimiter stored in the storage unit, and executes a discarding process to discard the speech data corresponding to the text data and the data generated in executing the speech recognition process (S4). The data generated in executing the speech recognition process includes a lattice, which is a graph of output candidates expressed as an effective graph, and text data, which is the result of the speech recognition process. Next, the speech recognition system executes the speech recognition process on the speech data excluding the section discarded by the discarding process (S5).

[0038] In the detection process in S3, the voice recognition system detects that a certain period of time of fillers or specific words continues in the text data as a section that is separated by a predetermined criterion, for example. Here, a filler is an utterance or a connecting word that fills the gap while a speaker is choosing the next word when speaking (such as in a conversation). For example, it is "well", "uh", "eh", "um", etc. Also, a specific word is a word that is repeatedly uttered, for example, "yes" or "hello". In the case of "yes", it may be repeatedly uttered, such as "yes, yes". For example, when the words "hello well uh um um um how are you feeling" are uttered, a filler is a word that serves as a section of a sentence between "well uh um um" and "how are you feeling".

[0039] For example, if the voice data is "It's been a while. Well, uh, um, today I would like to continue our discussion from last time," the voice recognition system performs voice recognition processing on the first 10 seconds of voice data "Hello, well, uh, um, nice to meet you today," as shown in FIG. 2(a). The voice recognition system performs voice recognition processing and stores text data "Hello, well, uh, um, nice to meet you today" in a storage unit. The voice recognition system performs detection processing on this text data. Here, since the text data contains a series of fillers "well, uh, um, um," the system detects a sentence break between "Hello, well, uh, um" and "Nice to meet you today," as shown in FIG. 2(b). As shown in FIG. 2(c), the voice recognition system holds the text data of "Hello, well, uh, uh, um," before the detected segment, and discards the voice data corresponding to the text data before the detected segment, data generated when performing voice recognition processing such as a lattice, which is a graph of output candidates expressed as a directed graph, and the text data of "Thank you for your help today," after the detected segment. Furthermore, as shown in FIG. 2(d), the voice recognition system performs voice recognition processing up to the next 10-second segment, "Thank you for your help today." Then, it performs detection processing and discarding processing. As a result, the voice recognition system can output "Hello, well, uh, uh, um, Thank you for your help today" as the voice recognition result.

[0040] The output here includes displaying the voice recognition result on a display, transmitting it to an external device, printing it out on a printer, storing it in a memory unit, passing the processing result to another processing device or another program, or converting the voice recognition result into voice data and outputting it from a speaker, etc. For example, the contents of a voice conversation over the telephone or the like may be stored in the memory unit as text data of the voice recognition result. Note that the output is not limited to the described examples.

[0041] In addition, in the detection process in S3, the voice recognition system detects periods in the text data, assuming that a break defined by a predetermined criterion is a period.

[0042] For example, in the case where the speech data is "Hello, it's been a while. Today, we would like to continue our discussion from last time. Thank you for your cooperation.", as shown in FIG. 3(a), the speech recognition system executes speech recognition processing on the first 10 seconds of speech data "Hello, it's been a while. Today, we will continue our discussion from last time." Since there is a period, the speech recognition system detects the period between "Hello, it's been a while." and "Today, we will continue our discussion from last time," as shown in FIG. 3(b). As shown in FIG. 3(c), the speech recognition system holds the text data up to "Hello, it's been a while." before the detected period, and discards the speech data corresponding to the text data before the detected period, the data generated when executing speech recognition processing such as a lattice, which is a graph of output candidates expressed as a directed graph, and the text data of "Today, we will continue our discussion from last time" after the detected period.

[0043] Next, as shown in FIG. 3(d), the speech recognition system executes speech recognition processing up to the next 10-second segment, "Today, we would like to continue our discussion from last time. Thank you very much." The speech recognition system detects the period between "Today, we would like to continue our discussion from last time" and "Thank you very much," which is the end position of the segment, in the same way as the detection processing of FIG. 3(b). The speech recognition system holds the text data up to "Today, we would like to continue our discussion from last time," which is before the detected segment, in the same way as FIG. 3(c), and discards the voice data corresponding to the text data before the detected segment, the data generated when executing speech recognition processing such as a lattice, which is a graph of output candidates expressed as a directed graph, and the text data of "Thank you very much" after the detected segment. Finally, the speech recognition system executes speech recognition processing up to "Thank you very much." As a result, it is possible to output "It's been a long time. Today, we would like to continue our discussion from last time. Thank you very much." as the recognition result.

[0044] As described above, in this embodiment, the voice recognition system executes a detection process to detect a segment that is separated according to a predetermined criterion in text data that is the result of the voice recognition process. The voice recognition system then holds the text data preceding the segment detected by the detection process, executes a discard process to discard the voice data corresponding to the text data and data that was temporarily generated in the storage unit when executing the voice recognition process, and executes the voice recognition process on the voice data excluding the segment discarded by the discard process.

[0045] As a result, if a certain amount of filler, a sequence of specific words, or a period is set as a predetermined criterion for dividing a sentence, the sentence will not be divided in the middle of the sentence, thereby preventing misrecognition due to division in the middle of the sentence.

[0046] Furthermore, as described above, the voice data stored in the storage unit and corresponding to the text data preceding the delimiter detected by the detection process and the data temporarily generated in the storage unit when executing the voice recognition process are discarded, so that the storage capacity (memory) of the storage unit provided in the voice recognition system can be freed up and the storage capacity (memory) of the storage unit can be prevented from being depleted due to long-term voice recognition.

[0047] In the present embodiment, the voice recognition system detects a certain period of time of a filler or a specific word (e.g., "hello," "yes," etc.) as a segment that is segmented according to a predetermined criterion in the detection process, and detects that a certain period of time of a filler or a specific word has continued. This makes it possible to detect an appropriate segment from text data.

[0048] In the present embodiment, the voice recognition system detects the positions of the speech of a period, which is a segment that is separated according to a predetermined criterion, in the detection process, thereby making it possible to detect an appropriate segment from text data.

[0049] Although an embodiment of the present invention has been described above, the forms to which the present invention can be applied are not limited to the above-described embodiment, and as exemplified below, appropriate modifications can be made without departing from the spirit of the present invention.

[0050] In the above embodiment, the description has been given of the case where the segment is a certain length of filler or a specific word, or a period. However, the segment may be a comma. [Industrial Applicability]

[0051] The present invention can be suitably adopted in a voice recognition system and a voice recognition method.

Claims

1. a voice recognition process for storing voice data in a storage unit, recognizing voice from the voice data stored in the storage unit using the storage unit as a working area, and converting the voice data into text data; a detection process for detecting a segment that is segmented according to a predetermined criterion in the text data that is a result of the speech recognition process; a discarding process for retaining the text data stored in the storage unit and preceding the delimiter detected by the detection process, and discarding the voice data corresponding to the text data and the data temporarily generated in the storage unit when the voice recognition process is executed; Run a voice recognition system which performs the voice recognition process on the voice data excluding the section discarded by the discard process;

2. 2. The speech recognition system according to claim 1, wherein in the detection process, a segment defined according to a predetermined criterion is regarded as a segment as a sentence, and a segment as a sentence is detected in the text data.

3. The speech recognition system according to claim 1, characterized in that in the detection process, a segment that is segmented according to a predetermined criterion is defined as a segment that is a sequence of a filler or a specific word for a certain period of time, and that a sequence of a filler or a specific word for a certain period of time is detected in the text data.

4. 2. The speech recognition system according to claim 1, wherein in the detection process, a break defined according to a predetermined criterion is regarded as a period, and a period is detected in the text data.

5. The speech recognition system according to claim 1, characterized in that in the speech recognition process, a lattice, which is a graph of output candidates expressed as a directed graph, is generated from the speech data, and the most likely candidate is selected from the candidates in the generated lattice, thereby recognizing the speech.

6. performing a voice activity detection process for detecting a voice activity in the voice data before performing the voice recognition process; The speech recognition system according to claim 1 , wherein the speech recognition process is executed on the speech data that has been subjected to the speech activity detection process.

7. 2. The voice recognition system according to claim 1, wherein in said voice recognition processing, voice is recognized from said voice data at predetermined intervals.

8. 8. The speech recognition system according to claim 7, wherein the predetermined intervals are constant intervals or intervals that are different from each other.

9. 2. The speech recognition system according to claim 1, wherein the discarding process discards the text data stored in the storage unit and subsequent to the delimiter detected by the detection process.

10. a voice recognition process for storing voice data in a storage unit, recognizing voice from the voice data stored in the storage unit using the storage unit as a working area, and converting the voice data into text data; a detection process for detecting a segment that is segmented according to a predetermined criterion in the text data that is a result of the speech recognition process; a discarding process for retaining the text data stored in the storage unit and preceding the delimiter detected by the detection process, and discarding the voice data corresponding to the text data and the data temporarily generated in the storage unit when the voice recognition process is executed; Run a speech recognition method for recognizing a speech signal by performing the speech recognition process on the speech signal from which the discarded portion has been removed;

11. 11. The speech recognition method according to claim 10, wherein in the detection process, a segment defined according to a predetermined criterion is regarded as a segment as a sentence, and a segment as a sentence is detected in the text data.

12. The speech recognition method according to claim 10, characterized in that in the detection process, a segment that is segmented according to a predetermined criterion is defined as a segment that is a sequence of a filler or a specific word for a certain period of time, and that a sequence of a filler or a specific word for a certain period of time is detected in the text data.

13. 11. The speech recognition method according to claim 10, wherein in the detection process, a break defined according to a predetermined criterion is regarded as a period, and a period is detected in the text data.

Images