Information processing systems, information processing methods, and programs

Abstract

This invention provides an information processing system, information processing method, and program that can improve the accuracy of speech recognition. [Solution] In an information processing system in which a management server is connected to a user terminal via a communication network, the management server comprises: an audio data acquisition unit that acquires audio data to be recognized; a phoneme division unit that divides the acquired audio data into phoneme units; a match determination unit that determines whether the divided phoneme sequence matches a word registered in a dictionary, using the similarity between phoneme sequences, in which each phoneme is represented in a multidimensional space based on at least two or more phonological features; and an error correction unit that replaces the phoneme sequence with a word according to the similarity.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] In speech recognition, a technique has been proposed for dividing speech data into phoneme units and detecting phoneme boundaries using auditory attention features (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In conventional speech recognition technology, in phoneme segmentation and recognition, high-precision error correction based on similarity considering phonetic features has not been sufficiently performed.

[0005] The present invention has been made in view of such a background, and an object thereof is to provide a technology capable of improving the accuracy of speech recognition.

Means for Solving the Problems

[0006] The main invention of the present invention for solving the above problems is an information processing system, comprising: an audio data acquisition unit that acquires audio data to be speech recognized; a phoneme segmentation unit that divides the acquired audio data into phoneme units; a match determination unit that determines a match between the segmented phoneme sequence and a word registered in a dictionary using a similarity between phoneme sequences represented in a multidimensional space based on at least two or more phonetic features for each phoneme; and an error correction unit that replaces the phoneme sequence with the word according to the similarity. <​​Further issues and solutions disclosed in this application will be made clear in the section on embodiments of the invention and in the drawings. [Effects of the Invention]

[0008] According to the present invention, the accuracy of speech recognition can be improved. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example of the overall configuration of an information processing system. [Figure 2] This figure shows an example of the hardware configuration of management server 2. [Figure 3] This figure shows an example of the software configuration for management server 2. [Figure 4] This diagram illustrates the speech recognition processing flow in an information processing system. [Modes for carrying out the invention]

[0010] <System Overview> The following describes an information processing system according to one embodiment of the present invention. The information processing system of this embodiment acquires speech data to be recognized and divides the acquired speech data into phonemes. Then, for each divided phoneme sequence, the system determines the match with a word registered in a dictionary using the similarity between phoneme sequences, which are represented in a multidimensional space based on at least two or more phonological features for each phoneme, and performs error correction by replacing the phoneme sequences with words according to the similarity. As a result, it is possible to achieve highly accurate speech recognition that takes phonological features into consideration, compared to conventional speech recognition technology.

[0011] Figure 1 shows an example of the overall configuration of an information processing system. The information processing system in this embodiment includes a management server 2. The management server 2 is connected to the user terminal 1 via a communication network. The communication network is, for example, the internet and is constructed using public telephone networks, mobile phone networks, wireless communication channels, Ethernet (registered trademark), etc.

[0012] User terminal 1 is a computer operated by the user. User terminal 1 can be, for example, a smartphone, a tablet computer, or a personal computer. User terminal 1 is equipped with a microphone and acquires the user's voice as audio data and transmits it to the management server 2.

[0013] The management server 2 may be a general-purpose computer such as a workstation or personal computer, or it may be logically implemented through cloud computing. The management server 2 performs speech recognition processing on the voice data received from the user terminal 1 and sends the recognition result to the user terminal 1.

[0014] <Management Server 2> Figure 2 shows an example of the hardware configuration of the management server 2. Note that the illustrated configuration is just one example, and other configurations are also possible. The management server 2 includes a CPU 201, memory 202, storage device 203, communication interface 204, input device 205, and output device 206. The storage device 203 stores various data and programs, such as a hard disk drive, solid-state drive, or flash memory. The communication interface 204 is an interface for connecting to a communication network, such as an adapter for connecting to Ethernet®, a modem for connecting to a public telephone network, a wireless communication device for wireless communication, or a USB (Universal Serial Bus) connector or RS232C connector for serial communication. The input device 205 is for inputting data, such as a keyboard, mouse, touch panel, button, or microphone. The output device 206 is for outputting data, such as a display, printer, or speaker. Furthermore, each functional unit of the management server 2, as described later, is realized by the CPU 201 reading programs stored in the storage device 203 into memory 202 and executing them, and each storage unit of the management server 2 is realized as part of the storage area provided by memory 202 and storage device 203.

[0015] Figure 3 is a diagram showing an example of the software configuration of the management server 2. The management server 2 includes an audio data acquisition unit 211, a phoneme segmentation unit 212, a matching determination unit 213, an error correction unit 214, a transmission unit 215, a dictionary storage unit 231, and a user feature storage unit 232. <Management Server 2>

[0016] Hereinafter, the functional units of the management server 2 will be described.

[0017] The dictionary storage unit 231 stores dictionary data used for speech recognition. The dictionary storage unit 231 stores by associating a word with a phoneme sequence corresponding to the word. For example, for the word "konnichiwa", the phoneme sequence " / k / o / N / n / i / t / i / w / a / " is stored in association. Phonemes are represented by international phonetic symbols. The dictionary storage unit 231 stores phonetic feature quantities for each phoneme. The phonetic feature quantities include voicing, articulation point, articulation method, nasality, lateral sound property, frication, and labialization.

[0018] The user feature storage unit 232 stores the voice features for each user. The user feature storage unit 232 stores by associating the identification information of the user with the voice features of that user. The voice features include the pronunciation tendency of specific phonemes, the confusion pattern between phoneme sequences, the speaking speed, and the like.

[0019] The audio data acquisition unit 211 acquires audio data to be speech recognized. The audio data acquisition unit 211 receives audio data from the user terminal 1 via the communication interface 204. The audio data acquisition unit 211 temporarily stores the received audio data in the memory 202.

[0020] The phoneme segmentation unit 212 segments the acquired audio data into phoneme units. The phoneme segmentation unit 212 performs acoustic analysis on the audio data and detects phoneme boundaries. The phoneme segmentation unit 212 segments the audio data into phoneme units based on the detected phoneme boundaries, and generates a phoneme sequence in which each phoneme is represented by international phonetic symbols. For example, a phoneme sequence " / k / o / N / n / i / t / i / w / a / " is generated from the audio data "konnichiwa".

[0021] The consistency determination unit 213 determines the consistency between the segmented phoneme sequence and the words registered in the dictionary storage unit 231 by using the similarity between phoneme sequences expressed in a multi-dimensional space based on at least two or more phonetic features for each phoneme. The consistency determination unit 213 calculates the similarity between phoneme sequences by using a cost function. The edit distance (Levenshtein distance) is used as the cost function. Other cost functions that can be adopted in addition to the edit distance include the Damerau-Levenshtein distance, the Hamming distance, and the Jaro-Winkler distance.

[0022] The consistency determination unit 213 calculates the similarity between phoneme sequences by using sonority, place of articulation, and manner of articulation as phonetic features. For example, the phoneme / p / and the phoneme / b / have a high similarity because they have the same place of articulation and manner of articulation and only differ in sonority. The consistency determination unit 213 sets the similarity according to different criteria for each vowel and consonant. For vowels, the similarity is calculated mainly based on the position of the tongue and the shape of the lips, and for consonants, the similarity is calculated based on the place of articulation and the manner of articulation.

[0023] The consistency determination unit 213 sets the similarity according to different criteria for each addition, deletion, or replacement of a phoneme in the combination of consecutive phonemes. For the replacement of a phoneme, the cost is set based on the similarity of phonetic features, and for the addition or deletion of a phoneme, a fixed cost is set. The consistency determination unit 213 sets the similarity to different values for each user based on the voice features stored in the user feature storage unit 232.

[0024] The error correction unit 214 replaces the phoneme sequence with a word according to the similarity. The error correction unit 214 extracts, as candidates, words whose similarity calculated by the consistency determination unit 213 exceeds a predetermined threshold value. When multiple words satisfy the predetermined similarity, the error correction unit 214 performs error correction on the word with the highest similarity. The error correction unit 214 generates the corrected word sequence as the speech recognition result.

[0025] The transmitting unit 215 transmits the speech recognition result generated by the error correction unit 214 to the user terminal 1. The transmitting unit 215 transmits the speech recognition result via the communication interface 204.

[0026] Figure 4 illustrates the speech recognition processing flow in an information processing system.

[0027] The voice data acquisition unit 211 acquires voice data to be recognized from the user terminal 1 (S101). The phoneme splitting unit 212 splits the acquired voice data into phonemes and generates a phoneme sequence (S102). The match determination unit 213 reads dictionary data from the dictionary storage unit 231 (S103) and determines the match between the split phoneme sequence and the words registered in the dictionary using the similarity between phoneme sequences based on phonological features (S104). The match determination unit 213 reads the user's voice features from the user feature storage unit 232 and adjusts the similarity for each user (S105). The error correction unit 214 replaces the phoneme sequence with words according to the calculated similarity and generates the voice recognition result (S106). The transmission unit 215 transmits the generated voice recognition result to the user terminal 1 (S107).

[0028] As described above, the information processing system of this embodiment can achieve highly accurate speech recognition compared to conventional speech recognition technologies by correcting errors in speech recognition using the similarity between phoneme sequences based on phonological features. Furthermore, by adjusting the similarity to take into account the speech characteristics of each user, speech recognition that accommodates individual differences can be performed. In addition, by setting the similarity using different criteria for vowels and consonants, appropriate error correction can be performed according to the phonological features.

[0029] Although these embodiments have been described above, they are intended to facilitate understanding of the present invention and are not intended to limit its interpretation. The present invention can be modified and improved without departing from its spirit, and equivalents thereof are also included.

[0030] For example, the processing performed by each functional unit of the management server 2 described above may be executed by any of the functional units. Furthermore, different functional units may be added to perform some of the processing performed by each of the functional units described above. Also, the functional units of the management server 2 may be distributed across multiple computers.

[0031] Furthermore, the information stored in each memory unit of the management server 2 may be stored in any of the memory units. That is, the information stored in the multiple memory units mentioned above may be stored in a single memory unit, or a portion of the information stored in one memory unit may be stored in another memory unit.

[0032] <Example 1> In the embodiment described above, an example was shown in which the phoneme splitting unit 212 splits the speech data into phonemes. However, the phoneme splitting unit 212 may also split the data into syllables or moras. In this case, the matching determination unit 213 calculates the similarity using the phonological features of the syllables or moras. For example, in Japanese, the moras "ka" and "ga" differ only in the voicing of the consonant portion, so they are calculated to have a high similarity. The error correction unit 214 corrects errors in speech recognition based on such syllable or mora-level similarity.

[0033] <Modification 2> In the embodiment described above, an example was shown in which edit distance is used as the cost function, but the matching unit 213 may calculate the similarity using a neural network trained by machine learning. In this case, the matching unit 213 outputs the similarity between phoneme sequences using a neural network that takes phonological features as input. The neural network is pre-trained using a large amount of speech data and correct labels. The error correction unit 214 replaces the phoneme sequences with words based on the similarity calculated by the neural network.

[0034] <Variation 3> In the embodiment described above, an example was shown in which the management server 2 receives and processes voice data from the user terminal 1. However, the user terminal 1 may perform the voice recognition processing. In this case, the user terminal 1 includes a voice data acquisition unit 211, a phoneme segmentation unit 212, a matching determination unit 213, an error correction unit 214, a dictionary storage unit 231, and a user feature storage unit 232. The user terminal 1 directly processes the voice data acquired from the microphone and displays the voice recognition results on the screen. This reduces communication delay and enables real-time voice recognition.

[0035] <Modification 4> In the embodiment described above, an example using a single dictionary storage unit 231 was shown, but multiple dictionaries for different specialized fields may also be used. In this case, the dictionary storage unit 231 stores different dictionary data for each specialized field, such as the medical field, the legal field, and the technical field. The matching determination unit 213 selects a dictionary corresponding to a field specified by the user or a field estimated from the audio content, and uses the selected dictionary to determine whether the phoneme sequence matches a word. The error correction unit 214 corrects errors based on the selected specialized field dictionary, thereby achieving more accurate speech recognition for audio containing specialized terminology.

[0036] <Modification 5> In the embodiments described above, examples were shown in which phonological features such as voicing, point of articulation, and manner of articulation were used. However, the matching unit 213 may also calculate similarity using acoustic features. In this case, the matching unit 213 calculates the similarity between phoneme sequences using acoustic features such as fundamental frequency, formant frequency, and spectral envelope in addition to phonological features. The matching unit 213 integrates the similarity based on phonological features and the similarity based on acoustic features using a weighted average to calculate the final similarity. The error correction unit 214 performs more precise error correction based on the integrated similarity.

[0037] <Disclosure Items> Furthermore, this disclosure also includes the following configurations. [Item 1] A voice data acquisition unit that acquires voice data to be used for speech recognition, A phoneme splitting unit that divides the acquired audio data into phonemes, A matching unit determines, for the divided phoneme sequence, whether it matches a word registered in a dictionary, using the similarity between phoneme sequences, in which each phoneme is represented in a multidimensional space based on at least two or more phonological features. An error correction unit that replaces the phoneme sequence with the word according to the similarity, An information processing system equipped with the following features. [Item 2] The information processing system according to item 1, wherein the phonological features include at least one of voicing, point of articulation, and manner of articulation. [Item 3] The aforementioned similarity is determined by different criteria for each vowel and consonant, according to the information processing system described in item 1. [Item 4] The error correction unit, when multiple words satisfy a predetermined similarity level, corrects the error to the word with the highest similarity level, as described in item 1 of the information processing system. [Item 5] The information processing system described in item 1, wherein the similarity is set for each combination of consecutive phonemes, according to different criteria for each addition, deletion, or replacement of phonemes. [Item 6] It further includes a user feature memory unit that stores the voice characteristics of each user, The information processing system described in item 1, wherein the similarity is set to a different value for each user based on the voice characteristics stored in the user feature storage unit. [Item 7] Steps include acquiring audio data to be used for speech recognition, The steps include dividing the acquired audio data into phonemes, The steps include determining whether the divided phoneme sequence matches a word registered in a dictionary using the similarity between phoneme sequences, in which each phoneme is represented in a multidimensional space based on at least two or more phonological features, A step of replacing the phoneme sequence with the word according to the similarity, A method of information processing performed by a computer. [Item 8] Steps include acquiring audio data to be used for speech recognition, The steps include dividing the acquired audio data into phonemes, The steps include determining whether the divided phoneme sequence matches a word registered in a dictionary using the similarity between phoneme sequences, in which each phoneme is represented in a multidimensional space based on at least two or more phonological features, A step of replacing the phoneme sequence with the word according to the similarity, A program that causes a computer to execute something. [Explanation of symbols]

[0038] 1 User terminal 2 Management Server

Claims

1. A voice data acquisition unit that acquires voice data to be used for speech recognition, A phoneme splitting unit that divides the acquired audio data into phonemes, A matching unit determines, for the divided phoneme sequence, whether it matches a word registered in a dictionary, using the similarity between phoneme sequences, which are represented in a multidimensional space based on at least two or more phonological features including voicing. An error correction unit that replaces the phoneme sequence with the word according to the similarity, An information processing system equipped with the following features.

2. The information processing system according to claim 1, wherein the phonological feature quantity includes at least one of voicing, point of articulation, and manner of articulation.

3. The information processing system according to claim 1, wherein the similarity is set by criteria based on different phonological features for vowels and consonants, respectively.

4. The information processing system according to claim 1, wherein the error correction unit, when multiple words satisfy a predetermined similarity, corrects the error to the word with the highest similarity.

5. The information processing system according to claim 1, wherein the similarity is defined as the edit distance between the phoneme sequence and the phoneme sequence corresponding to a word registered in the dictionary, and a different cost is set for each type of editing operation, such as adding, deleting, or replacing a phoneme, and the similarity is set based on the edit distance calculated based on the cost.

6. It further includes a user feature memory unit that stores the voice characteristics of each user, The information processing system according to claim 1, wherein the similarity is set to a different value for each user based on the voice characteristics stored in the user feature storage unit.

7. Steps include acquiring audio data to be used for speech recognition, The steps include dividing the acquired audio data into phonemes, The steps include determining whether the divided phoneme sequence matches a word registered in a dictionary, using the similarity between phoneme sequences, which are represented in a multidimensional space based on at least two or more phonological features including voicing, for each phoneme sequence; A step of replacing the phoneme sequence with the word according to the similarity, A method of information processing performed by a computer.

8. Steps include acquiring audio data to be used for speech recognition, The steps include dividing the acquired audio data into phonemes, The steps include determining whether the divided phoneme sequence matches a word registered in a dictionary, using the similarity between phoneme sequences, which are represented in a multidimensional space based on at least two or more phonological features including voicing, for each phoneme sequence; A step of replacing the phoneme sequence with the word according to the similarity, A program that causes a computer to execute something.

Images