Information processing device, information processing method, voice recognition device, voice recognition method, and computer-readable recording medium

Abstract

This information processing device comprises an extraction unit that extracts a corrected portion of a first correction result on the basis of a difference between a first voice recognition result output from a voice recognition model for converting voice into text and a first correction result obtained by correcting the first voice recognition result, a generation unit that generates a correction pattern by associating the corrected portion and the portion before correction corresponding to the corrected portion of the first voice recognition result, a correction unit that corrects a second voice recognition result stored in the past on the basis of the correction pattern, and a learning unit that trains the voice recognition model by using a second correction result obtained by correcting the second voice recognition result as training data.

Description

Information processing device, information processing method, voice recognition device, voice recognition method, and computer-readable recording medium

[0001] The present disclosure relates to an information processing device, an information processing method, a speech recognition device, a speech recognition method, and a computer-readable recording medium used in speech recognition.

[0002] In a speech recognition model, the speech recognition results output from the speech recognition model are corrected, and the corrected results are used as training data to periodically train the speech recognition model. However, in order to improve the speech recognition accuracy of a speech recognition model, a huge amount of training data correction is required. Therefore, it is desirable to reduce the amount of training data correction and generate a highly accurate speech recognition model.

[0003] As a related technique, Patent Document 1 discloses an error correction model training device that simply and sequentially trains an error correction model. The error correction model training device in Patent Document 1 recognizes input speech and uses an error correction model for correcting word error trends in speech recognition to correct errors in the selection of speech recognition results obtained from the input speech and output the corrected results. The error correction model training device in Patent Document 1 also learns word error trends from the difference between linguistic features included in the speech recognition results and linguistic features included in the correction results, and updates the error correction model according to the learned word error trends.

[0004] JP 2014-224860 A

[0005] In Patent Document 1, an error correction model is updated by learning word error trends. However, Patent Document 1 does not correct multiple previously stored speech recognition results and use the multiple corrected speech recognition results as training data for learning.

[0006] An example of an objective of the present disclosure is to reduce corrections of speech recognition results and generate a highly accurate speech recognition model.

[0007] In order to achieve the above object, an information processing device according to one aspect of the present disclosure is characterized by having: an extraction unit that extracts a correction portion of a first correction result based on a difference between a first speech recognition result output from a speech recognition model that converts speech to text and a first correction result obtained by correcting the first speech recognition result; a generation unit that generates a correction pattern by associating the correction portion with a pre-correction portion of the first speech recognition result that corresponds to the correction portion; a correction unit that corrects a second speech recognition result that was previously stored based on the correction pattern; and a learning unit that trains the speech recognition model using the second correction result obtained by correcting the second speech recognition result as learning data.

[0008] Furthermore, to achieve the above object, an information processing method according to one aspect of the present disclosure is characterized in that an information processing device extracts a corrected portion of a first correction result based on a difference between a first speech recognition result output from a speech recognition model that converts speech to text and a first correction result obtained by correcting the first speech recognition result, generates a correction pattern by associating the corrected portion with a pre-correction portion of the first speech recognition result that corresponds to the corrected portion, corrects a second speech recognition result that was previously stored based on the correction pattern, and trains the speech recognition model using the second correction result obtained by correcting the second speech recognition result as training data.

[0009] Furthermore, in order to achieve the above object, a computer-readable recording medium having a program recorded thereon according to one aspect of the present disclosure causes a computer to execute the following processes: extracting a correction portion of a first correction result based on a difference between a first speech recognition result output from a speech recognition model that converts speech to text and a first correction result obtained by correcting the first speech recognition result; generating a correction pattern by associating the correction portion with a pre-correction portion of the first speech recognition result that corresponds to the correction portion; correcting a second speech recognition result that was previously stored based on the correction pattern; and training the speech recognition model using the second correction result obtained by correcting the second speech recognition result as training data.

[0010] In order to achieve the above object, according to one aspect of the present disclosure, a speech recognition device that converts speech to text using a speech recognition model includes: an extraction unit that extracts a correction portion of a first correction result based on a difference between a first speech recognition result output from a speech recognition model and a first correction result obtained by correcting the first speech recognition result; a generation unit that generates a correction pattern by associating the correction portion with a pre-correction portion of the first speech recognition result that corresponds to the correction portion; a correction unit that corrects a second speech recognition result that was previously stored based on the correction pattern; and a learning unit that trains the speech recognition model using, as learning data, a second correction result obtained by correcting the second speech recognition result.

[0011] Furthermore, in order to achieve the above object, according to one aspect of the present disclosure, there is provided a speech recognition method for converting speech into text using a speech recognition model, wherein the speech recognition device extracts a correction portion of a first correction result based on a difference between a first speech recognition result output from a speech recognition model and a first correction result obtained by correcting the first speech recognition result, generates a correction pattern by associating the correction portion with a pre-correction portion of the first speech recognition result corresponding to the correction portion, corrects a second speech recognition result that was previously stored based on the correction pattern, and trains the speech recognition model using the second correction result obtained by correcting the second speech recognition result as training data.

[0012] Furthermore, in order to achieve the above object, in one aspect of the present disclosure, a computer-readable recording medium having recorded thereon a program for converting speech into text using a speech recognition model causes the computer to execute the following processes: extracting a correction portion of a first correction result based on a difference between a first speech recognition result output from a speech recognition model and a first correction result obtained by correcting the first speech recognition result; generating a correction pattern by associating the correction portion with a pre-correction portion of the first speech recognition result that corresponds to the correction portion; correcting a second speech recognition result that was previously stored based on the correction pattern; and training the speech recognition model using the second correction result obtained by correcting the second speech recognition result as training data.

[0013] As described above, according to the present disclosure, it is possible to reduce the number of corrections to speech recognition results and generate a highly accurate speech recognition model.

[0014] FIG. 1 is a diagram illustrating an example of an information processing device. FIG. 2 is a diagram illustrating an example of a system having an information processing device. FIG. 3 is a diagram illustrating the operation of an extraction unit and a generation unit in the case of Japanese. FIG. 4 is a diagram illustrating the operation of an extraction unit and a generation unit in the case of English. FIG. 5 is a diagram illustrating the operation of a correction unit in the case of Japanese. FIG. 6 is a diagram illustrating the operation of a correction unit in the case of English. FIG. 7 is a diagram illustrating the operation of an information processing device. FIG. 8 is a diagram illustrating the operation of a speech recognition device. FIG. 9 is a diagram illustrating an example of a computer that realizes an information processing device and a speech recognition device in the embodiment and modified examples.

[0015] (Embodiment) The configuration of an information processing device in an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram for explaining an example of an information processing device.

[0016] 1 is a device (a learning support device that supports learning of a speech recognition model) that generates a highly accurate speech recognition model by correcting previously stored speech recognition results based on corrected portions of the speech recognition results of a speech recognition model and training the speech recognition model using the corrected speech recognition results as training data. Also, as shown in FIG. 1, the information processing device 10 has an extraction unit (extraction means) 11, a generation unit (generation means) 12, a correction unit (correction means) 13, and a learning unit (learning means) 14.

[0017] The extraction unit 11 extracts the corrected portion of the first correction result based on the difference between a first speech recognition result output from a speech recognition model that converts speech into text and a first correction result obtained by correcting the first speech recognition result.

[0018] The generation unit 12 generates a correction pattern by associating the correction portion with the uncorrected portion corresponding to the correction portion in the first speech recognition result.

[0019] The correction unit 13 corrects the second speech recognition result stored in the past based on the correction pattern.

[0020] The learning unit 14 learns the voice recognition model using the second correction result obtained by correcting the second voice recognition result as learning data.

[0021] In this manner, in the embodiment, the previously stored speech recognition result is corrected based on the corrected portion of the speech recognition result of the speech recognition model, so that the correction of the previously stored speech recognition result can be reduced. Furthermore, by training the speech recognition model using the corrected speech recognition result as training data, the speech recognition accuracy of the speech recognition model can be further improved.

[0022] [System Configuration] Next, the configuration of the information processing device 10 in the embodiment will be described in more detail with reference to Fig. 2. Fig. 2 is a diagram showing an example of a system including an information processing device.

[0023] 2 , the system 100 according to the embodiment includes a speech recognition device 10A, a speech input device 20, a storage device 30, an input device 40, and an output device 50. The speech recognition device 10A includes a speech recognition unit 15 and an information processing device 10. The information processing device 10 includes an editing unit 16, an extraction unit 11, a generation unit 12, a correction unit 13, and a learning unit 14.

[0024] The speech recognition device 10A is, for example, an information processing device such as a CPU (Central Processing Unit), a programmable device such as an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), or a circuit equipped with one or more of these, a server computer, a personal computer, or a mobile terminal.

[0025] The voice input device 20 is, for example, an electronic component such as a microphone that converts sound into an electric signal, or an acoustic device (microphone) that has such an electronic component built in and is intended for collecting sound. The voice input device 20 may be provided in the voice recognition device 10A.

[0026] The storage device 30 is a database, a server computer, a circuit having a memory, or the like. The storage device 30 stores, for example, at least information such as a speech recognition model, speech recognition results, and correction results. In the example of Fig. 2, the storage device 30 is provided outside the speech recognition device 10A, but it may also be provided inside the speech recognition device 10A. Furthermore, the storage device 30 may be constructed using multiple storage devices.

[0027] The input device 40 is, for example, a touch panel, a mouse, a keyboard, etc. The input device 40 may be provided in the voice recognition device 10A.

[0028] The output device 50 acquires output information (described later) converted into an outputtable format, and outputs generated images, sounds, etc. based on the output information. The output device 50 is, for example, an image display device using a liquid crystal, an organic electroluminescence (EL) display, or a cathode ray tube (CRT). The image display device may also include an audio output device such as a speaker. The output device 50 may also be a printing device such as a printer. The output device 50 may also be provided in the speech recognition device 10A.

[0029] The voice recognition device 10A, the voice input device 20, the storage device 30, the input device 40, and the output device 50 are communicably connected via a network. The network is a general network constructed using communication lines such as the Internet, a local area network (LAN), a dedicated line, a telephone line, an in-house network, a mobile communication network, Bluetooth (registered trademark), or Wi-Fi (Wireless Fidelity) (registered trademark).

[0030] The speech recognition device will now be described in detail. The speech recognition unit 15 first acquires speech waveform information corresponding to speech uttered by a speaker from the speech input device 20. Next, the speech recognition unit 15 inputs the acquired speech waveform information into a speech recognition model, converts the speech output from the speech recognition model into text, and acquires a first speech recognition result. The acquired first speech recognition result is stored in the storage device 30.

[0031] In the operation phase, the speech recognition model inputs speech waveform information (speech speech data) corresponding to the speech uttered by the speaker into the trained speech recognition model, converts the speech speech data into text (transcribes it), and outputs the speech recognition results (text data).

[0032] The voice recognition model may be, for example, Whisper or DeepSpeech.

[0033] The editing unit 16 presents the first speech recognition result to a user (editor) through a user interface, allowing the user to correct (edit) the first speech recognition result. Specifically, the editing unit 16 displays the user interface on the output device 60. The editing unit 16 also stores the first correction result corrected by the user in the storage device 30.

[0034] The user interface displays at least the first speech recognition result (text data), the first correction result (text data), the second speech recognition result (text data), and the second correction result (text data). However, the user interface may display data other than the text data described above. Furthermore, the user interface may highlight (highlight) the corrected portions of the first correction result and the corrected portions of the second speech recognition result, for example.

[0035] The highlighting may be performed by adding highlighting to the text of the corrected portion, changing the font type, color, bold, italics, etc.

[0036] The extraction unit 11 first obtains a first speech recognition result output from the speech recognition model from the storage device 30. Next, the extraction unit 11 obtains a first correction result obtained by correcting the first speech recognition result from the storage device 30. Next, the extraction unit 11 performs a morphological analysis process on the first speech recognition result and the first correction result. Next, the extraction unit 11 compares the morphemes of the first speech recognition result with the morphemes of the first correction result by dynamic programming, and extracts differences (different parts) on a morpheme-by-morpheme basis. In other words, the extraction unit 11 extracts the corrected part of the first correction result and the part before correction that corresponds to the corrected part.

[0037] Morphological analysis is a process that breaks down sentences expressed in natural language into the smallest units of meaning in the language (morphemes) and determines the part of speech and inflections of each one.

[0038] Dynamic programming is an algorithm that associates each morpheme between two different sentences segmented into morphemes. When making the association, the similarity between morphemes is considered, and the optimal association that maximizes the similarity is searched for. The similarity can be defined, for example, as "1" if two morphemes match and "0" if they do not match.

[0039] The generation unit 12 first generates a correction pattern by associating the correction part with the part before correction corresponding to the correction part of the first speech recognition result. The association is performed when the correction part is a noun in morpheme units. For parts other than nouns, such as particles, no association is made. Next, the generation unit 12 stores the correction pattern in the storage device 30.

[0040] FIG. 3 is a diagram for explaining the operations of the extraction unit and the generation unit in the case of Japanese. In the example of FIG. 3, first, the first speech recognition result 31 (recognition result 1) is displayed on the user interface. Next, the user operates the user interface using the input device 40 to make corrections. As a result, the first speech recognition result 31 is corrected as the first correction result 32 (correction result 1) in FIG. 3.

[0041] Next, the extraction unit 11 executes morphological analysis processing on the first speech recognition result and the first correction result respectively, and obtains morphological analysis results 33 and 34. The morphological analysis results 33 and 34 may be displayed on the user interface. Next, the extraction unit 11 extracts, by dynamic programming processing, the correction parts "above the sky", "cold air", "wo" in FIG. 3 and the parts before correction corresponding to the correction parts of the first speech recognition result, namely, "upper cool", "dry period", "oh".

[0042] Next, the generation unit 12 generates a correction pattern by associating each of the nouns "above the sky" and "cold air" in the correction part with the parts before correction corresponding to the correction parts of the first speech recognition result, namely, "upper cool" and "dry period". In the example of FIG. 3, the correction patterns are the pattern in which the part before correction "upper cool" is associated with the correction part "above the sky" and the pattern in which the part before correction "dry period" is associated with the correction part "cold air".

[0043] 4 is a diagram illustrating the operation of the extraction unit and the generation unit for English. In the example of FIG. 4, a first speech recognition result 41 (recognition result 1) is first displayed on the user interface. Next, the user operates the user interface using the input device 40 to make corrections. As a result, the first speech recognition result 41 is corrected to the first correction result 42 (correction result 1) in FIG. 4.

[0044] Next, the extraction unit 11 performs a morphological analysis process on each of the first speech recognition result and the first correction result, and obtains morphological analysis results 43 and 44. The morphological analysis results 43 and 44 may be displayed on a user interface. Next, the extraction unit 11 extracts, by dynamic planning, the corrected parts in FIG. 4 , "technique" and "allows," and the uncorrected parts corresponding to the corrected parts in the first speech recognition result, "tech nick" and "a lounge."

[0045] Next, the generation unit 12 associates the pre-correction parts "technique" and "a lounge" corresponding to the corrected parts in the first speech recognition result with the nouns "technique" and "allows," respectively, to generate correction patterns. In the example of Fig. 4, the correction patterns are a pattern in which the pre-correction part "technique" is associated with the corrected part "technique," and a pattern in which the pre-correction part "a lounge" is associated with the corrected part "allows."

[0046] The correction unit 13 first uses the uncorrected portion of the correction pattern as a query to search for a second speech recognition result including the uncorrected portion from second speech recognition results previously stored in the storage device 30. Next, the correction unit 13 corrects the uncorrected portion of the searched second speech recognition result based on the corrected portion of the correction pattern. Thereafter, the correction unit 13 stores in the storage device 30 a second correction result obtained by correcting the second speech recognition result.

[0047] Specifically, in correcting the second speech recognition result, the correction unit 13 first calculates a first similarity that represents the sum of the similarity between a pre-correction portion included in the second speech recognition result and its preceding sentence and the similarity between the pre-correction portion and its succeeding sentence. The correction unit 13 also calculates a second similarity that represents the sum of the similarity between the corrected portion included in the second correction result and its preceding sentence and the similarity between the corrected portion and its succeeding sentence.

[0048] Thereafter, if the second similarity is higher than the first similarity, the correction unit 13 corrects the second speech recognition result. Note that the similarity is calculated using, for example, natural language processing (e.g., BERT, etc.).

[0049] The calculation method for the first and second similarities will now be described. For example, assume that sentence S0 is "November to February is the dry season" and sentence S-1 preceding sentence S0 is "Thailand has a tropical monsoon climate." First, sentence S0 to be corrected is corrected to sentence S'0, "November to February is cold." In this case, the first similarity d1 can be expressed as d1 = Compare(S-1, S0), and the second similarity d2 can be expressed as d2 = Compare(S-1, S'0). Here, Compare() is a function that calculates the similarity between two sentences. If the second similarity d2 is greater than the first similarity d1, "dry season" is corrected to "cold." Otherwise, no correction is made.

[0050] This calculation method can also be applied to English. For example, the case of determining whether to correct from "tech nick" to "technique" will be described. The sentence S_0 is "It is the expected tech nick.", and the previous sentence S-1 of the sentence S0 is "Automatic speech recognition transforms speech into text." will be described. First, the sentence S0 to be corrected will be corrected to the sentence S′0 "It is the expected technique" after correction. In that case, as described above, the first similarity d1 is expressed as d1 = Compare(S-1, S0), and the second similarity d2 is expressed as d2 = Compare(S-1, S′0). And when the second similarity d2 is greater than the first similarity d1, correct from "tech nick" to "technique", and if not, do not correct.

[0051] Figure 5 is a diagram for explaining the operation of the correction unit in the case of Japanese. In the example of Figure 5, the correction unit 13 uses the correction pattern 51 to search for the second speech recognition result including the parts "upper cool" and "dry period" before correction from the second speech recognition result stored in the storage device 30 in the past, and the second speech recognition result 52 (inside the broken line) including the parts "upper cool" and "dry period" before correction is searched.

[0052] Next, the correction unit 13 selects a second speech recognition result that can be corrected from the searched second speech recognition results based on the similarity. Next, the correction unit 13 corrects the part before correction of the second speech recognition result that can be corrected based on the corrected part of the correction pattern. In the example of Figure 5, "The airplane is flying at an altitude of 10,000 meters" and "It will get cold throughout the country due to the influence of the Siberian cold air mass" are selected as the second speech recognition results that can be corrected (○). After that, the correction unit 13 corrects the part before correction based on the corrected part of the correction pattern.

[0053] 6 is a diagram illustrating the operation of the correction unit for English. In the example of Fig. 6, the correction unit 13 uses a correction pattern 61 to search for a second speech recognition result including the pre-correction parts "tech nick" and "technique" from second speech recognition results previously stored in the storage device 30, and a second speech recognition result 62 (enclosed by a dashed line) including the pre-correction parts "a lounge" and "allows" is found.

[0054] Next, the correction unit 13 selects a correctable second speech recognition result from the searched second speech recognition results based on the similarity. Next, the correction unit 13 corrects the pre-correction portion of the correctable second speech recognition result based on the correction portion of the correction pattern. In the example of FIG. 6 , "Meditation is a tech nick that helps to relieve stress and improve focus." and "The pass a lounge you to visit all the museums in the city for free." are selected as correctable (◯) second speech recognition results. Then, the correction unit 13 corrects the pre-correction portion based on the correction portion of the correction pattern.

[0055] The user interface may display a message indicating that the retrieved second speech recognition result is correctable. In the examples of Figures 5 and 6, whether or not the result is correctable is displayed using "o" or "x", but other methods of display may also be used. Other methods include, for example, checking or color coding.

[0056] The learning unit 14 first obtains a second correction result obtained by correcting the second speech recognition result. Next, the learning unit 14 inputs the second correction result as training data into the speech recognition model and performs training. As a result, the speech recognition accuracy of the speech recognition model is improved.

[0057] (Variation) Furthermore, the learning unit 14 may train the speech recognition model using the second speech recognition result and the second correction result as training data. As a result, the speech recognition accuracy of the speech recognition model is further improved. The reason is that even in the second speech recognition result containing a speech recognition error, if the frequency of correct recognition is higher than the frequency of errors, machine learning is more likely to learn expressions with high frequency, resulting in improved accuracy. On the other hand, expressions with high error frequency are not learned correctly, and therefore require manual correction.

[0058] [Device Operation] Next, the operation of the information processing device 10 and the speech recognition device 10A in the embodiment will be described with reference to Figures 7 and 8. Figure 7 is a diagram for explaining the operation of the information processing device. Figure 8 is a diagram for explaining the operation of the speech recognition device. In the following description, the figures will be referenced as appropriate. Furthermore, in the embodiment, an information processing method and a speech recognition method are implemented by operating the information processing device and the speech recognition device. Therefore, the description of the information processing method and the speech recognition method in the embodiment will be replaced with the description of the operation of the information processing device and the speech recognition device below.

[0059] The information processing device shown in Fig. 1 will now be described. As shown in Fig. 7, first, the extraction unit 11 extracts a corrected portion of the first correction result based on the difference between a first speech recognition result output from a speech recognition model that converts speech into text and a first correction result obtained by correcting the first speech recognition result (step A1).

[0060] Next, the generation unit 12 generates a correction pattern by associating the correction portion with the uncorrected portion corresponding to the correction portion in the first speech recognition result (step A2).

[0061] Next, the correction unit 13 corrects the previously stored second speech recognition result based on the correction pattern (step A3).

[0062] Next, the learning unit 14 learns the speech recognition model using the second correction result obtained by correcting the second speech recognition result as learning data (step A4).

[0063] The speech recognition device shown in Fig. 2 will now be described. As shown in Fig. 8, in the speech recognition phase, the speech recognition unit 15 first acquires speech waveform information corresponding to speech uttered by a speaker from the speech input device 20 (step B1). Next, the speech recognition unit 15 inputs the acquired speech waveform information into a speech recognition model and acquires a first speech recognition result in which the speech output from the speech recognition model is converted into text (step B2). Thereafter, the speech recognition unit 15 stores the acquired first speech recognition result in the storage device 30.

[0064] Next, in the speech recognition phase, the editing unit 16 displays the first speech recognition result on the user interface (step B3). Next, if there is a part to be corrected in the first speech recognition result (step B4: Yes), the first speech recognition result is corrected (edited) by the user using the user interface, and the corrected first correction result is stored in the storage device 30.

[0065] If there is no part to be corrected in the first speech recognition result (step B4 : No), the editing unit 16 stores the first speech recognition result in the storage device 30 .

[0066] Next, the extraction unit 11 acquires from the storage device 30 the first speech recognition result output from the speech recognition model during the learning phase and the first correction result obtained by correcting the first speech recognition result (step B5). Next, the extraction unit 11 performs morphological analysis processing on each of the first speech recognition result and the first correction result (step B6). Next, the extraction unit 11 compares the morphemes of the first speech recognition result with the morphemes of the first correction result through dynamic programming processing, and extracts differences (different parts) on a morpheme-by-morpheme basis (step B7). That is, the extraction unit 11 extracts the corrected part of the first correction result and the part before correction corresponding to the corrected part.

[0067] Next, in the learning phase, the generation unit 12 generates a correction pattern by associating the correction portion with the uncorrected portion corresponding to the correction portion in the first speech recognition result (step B8). Note that the association in step A12 is performed when the correction portion is a noun on a morpheme basis, but is not performed when it is not a noun, for example, a particle. Next, the generation unit 12 stores the correction pattern in the storage device 30.

[0068] Next, in the learning phase, the correction unit 13 uses the uncorrected portion of the correction pattern as a query to search for a second speech recognition result including the uncorrected portion from second speech recognition results previously stored in the storage device 30 (step B9). Next, the correction unit 13 corrects the uncorrected portion of the searched second speech recognition result based on the corrected portion of the correction pattern (step B10). Thereafter, the correction unit 13 stores in the storage device 30 a second correction result obtained by correcting the second speech recognition result.

[0069] Specifically, in step A10, the correction unit 13 first calculates a first similarity representing the sum of the similarity between the pre-correction portion included in the second speech recognition result and its preceding sentence and the similarity between the pre-correction portion and its succeeding sentence when correcting the second speech recognition result. Next, in step A15, the correction unit 13 calculates a second similarity representing the sum of the similarity between the corrected portion included in the second correction result and its preceding sentence and the similarity between the corrected portion and its succeeding sentence. Thereafter, in step A15, the correction unit 13 corrects the second speech recognition result if the second similarity is higher than the first similarity. Thereafter, the correction unit 13 stores the second correction result in the storage device 30.

[0070] Next, in the learning phase, the learning unit 14 obtains a second correction result obtained by correcting the second speech recognition result (step B11). Next, the learning unit 14 inputs the second correction result as training data into the speech recognition model and performs training (step B12). As a result, the speech recognition accuracy of the speech recognition model is improved.

[0071] In steps B11 and B12, the learning unit 14 may learn the speech recognition model using the second speech recognition result and the second correction result as learning data, thereby further improving the speech recognition accuracy of the speech recognition model.

[0072] [Advantages of the embodiment] As described above, according to the embodiment, a previously stored speech recognition result is corrected based on a corrected portion of the speech recognition result of a speech recognition model, so that the correction of the previously stored speech recognition result can be reduced. Furthermore, by training the speech recognition model using the corrected speech recognition result as training data, the speech recognition accuracy of the speech recognition model can be further improved.

[0073] The information processing device 10 and the speech recognition device 10A described above are useful for, for example, automatic creation of minutes, interpretation and translation between multiple languages, voice input, device activation and operation, conversation (chat) with a computer, etc. For example, they can be used in call centers and chats of various organizations such as medical institutions, police, and fire departments.

[0074] [Program] The program in the embodiment may be a program that causes a computer to execute steps A1 to A4 and B1 to B12 shown in Figures 7 and 8. By installing and executing this program on a computer, the information processing device, speech recognition device, and methods thereof in the embodiment can be realized. In this case, the processor of the computer functions as a speech recognition unit 15, an editing unit 16, an extraction unit 11, a generation unit 12, a correction unit 13, and a learning unit 14 and performs processing.

[0075] The program in the embodiment may be executed by a computer system constructed by a plurality of computers, in which case, for example, each computer may function as one of the speech recognition unit 15, editing unit 16, extraction unit 11, generation unit 12, correction unit 13, and learning unit 14.

[0076] [Physical Configuration] A computer that realizes an information processing device and a speech recognition device by executing a program in the embodiment and the modified example will now be described with reference to Fig. 9. Fig. 9 is a diagram for explaining an example of a computer that realizes an information processing device and a speech recognition device in the embodiment and the modified example.

[0077] 9, the computer 110 includes a CPU (Central Processing Unit) 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader / writer 116, and a communication interface 117. These components are connected to each other via a bus 121 so as to be able to communicate data with each other. Note that the computer 110 may include a GPU or an FPGA in addition to or instead of the CPU 111.

[0078] The CPU 111 loads a program in the embodiment, which is composed of a group of codes and stored in the storage device 113, into the main memory 112 and executes each code in a predetermined order to perform various calculations. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory).

[0079] The program in the embodiment is provided in a state stored in a computer-readable recording medium 120. The program in the embodiment may be distributed over the Internet connected via the communication interface 117.

[0080] Specific examples of the storage device 113 include a hard disk drive and a semiconductor storage device such as a flash memory. The input interface 114 mediates data transmission between the CPU 111 and input devices 118 such as a keyboard and a mouse. The display controller 115 is connected to a display device 119 and controls the display on the display device 119.

[0081] The data reader / writer 116 mediates data transmission between the CPU 111 and the recording medium 120, reads programs from the recording medium 120, and writes processing results from the computer 110 to the recording medium 120. The communication interface 117 mediates data transmission between the CPU 111 and other computers.

[0082] Specific examples of the recording medium 120 include general-purpose semiconductor storage devices such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), magnetic recording media such as flexible disks, or optical recording media such as CD-ROMs (Compact Disk Read Only Memory).

[0083] Note that the information processing device and the speech recognition device in the embodiment and the modified example can be realized not by a computer on which a program is installed but by hardware corresponding to each part, for example, an electronic circuit. Furthermore, the information processing device and the speech recognition device may be realized in part by a program and in the remaining part by hardware. In the embodiment and the modified example, the computer is not limited to the computer shown in FIG. 9.

[0084] [Supplementary Note] The following supplementary note is further disclosed regarding the above-described embodiment. Some or all of the above-described embodiment can be expressed by (Supplementary Note 1) to (Supplementary Note 30) described below, but is not limited to the following description.

[0085] (Supplementary Note 1) An information processing device comprising: an extraction unit that extracts a correction portion of a first correction result based on a difference between a first speech recognition result output from a speech recognition model that converts speech to text and a first correction result obtained by correcting the first speech recognition result; a generation unit that generates a correction pattern by associating the correction portion with a pre-correction portion of the first speech recognition result that corresponds to the correction portion; a correction unit that corrects a second speech recognition result that was previously stored based on the correction pattern; and a learning unit that trains the speech recognition model using a second correction result obtained by correcting the second speech recognition result as learning data.

[0086] (Supplementary Note 2) The information processing device according to Supplementary Note 1, wherein the learning unit further learns the speech recognition model using the second speech recognition result and the second correction result as the learning data.

[0087] (Supplementary Note 3) The information processing device according to Supplementary Note 1, further comprising a user interface that presents the first speech recognition result to a user and allows the user to correct the first speech recognition result.

[0088] (Supplementary Note 4) The information processing device according to Supplementary Note 3, wherein the correction unit extracts a portion to be corrected from the second speech recognition result based on the correction pattern, and the user interface displays the portion to be corrected from the second speech recognition result in an emphasized manner.

[0089] (Supplementary Note 5) The information processing device according to Supplementary Note 1, wherein, in correcting the second speech recognition result, the correction unit calculates a first similarity by summing a similarity between a pre-correction portion included in the second speech recognition result and a preceding sentence of the pre-correction portion and a similarity between the pre-correction portion and a succeeding sentence of the pre-correction portion; and calculates a second similarity by summing a similarity between the pre-correction portion included in the second correction result and a preceding sentence of the pre-correction portion and a similarity between the pre-correction portion and a succeeding sentence of the pre-correction portion; and corrects the second speech recognition result if the second similarity is higher than the first similarity.

[0090] (Supplementary Note 6) An information processing method, in which an information processing device extracts a corrected portion of a first correction result based on a difference between a first speech recognition result output from a speech recognition model that converts speech to text and a first correction result obtained by correcting the first speech recognition result, generates a correction pattern by associating the corrected portion with a pre-correction portion of the first speech recognition result that corresponds to the corrected portion, corrects a second speech recognition result that was previously stored based on the correction pattern, and trains the speech recognition model using the second correction result obtained by correcting the second speech recognition result as training data.

[0091] (Supplementary Note 7) The information processing method according to Supplementary Note 6, wherein the information processing device further trains the speech recognition model using the second speech recognition result and the second correction result as the training data.

[0092] (Supplementary Note 8) The information processing method according to Supplementary Note 6, wherein the information processing device presents the first speech recognition result to a user and displays a user interface that allows the user to correct the first speech recognition result.

[0093] (Supplementary Note 9) The information processing method according to Supplementary Note 8, wherein the information processing device extracts a portion to be corrected from the second speech recognition result based on the correction pattern, and the user interface displays the portion to be corrected from the second speech recognition result in an emphasized manner.

[0094] (Supplementary Note 10) The information processing method according to Supplementary Note 6, wherein the information processing device, in correcting the second speech recognition result, calculates a first similarity by summing a similarity between a pre-correction portion included in the second speech recognition result and a preceding sentence of the pre-correction portion and a similarity between the pre-correction portion and a succeeding sentence of the pre-correction portion, calculates a second similarity by summing a similarity between the pre-correction portion included in the second correction result and a preceding sentence of the pre-correction portion and a similarity between the pre-correction portion and a succeeding sentence of the pre-correction portion, and corrects the second speech recognition result if the second similarity is higher than the first similarity.

[0095] (Supplementary Note 11) A computer-readable recording medium having recorded thereon a program that causes a computer to execute the following processes: extracting a corrected portion of a first correction result based on a difference between a first speech recognition result output from a speech recognition model that converts speech to text and a first correction result obtained by correcting the first speech recognition result; generating a correction pattern by associating the corrected portion with a pre-correction portion of the first speech recognition result that corresponds to the corrected portion; correcting a second speech recognition result that was previously stored based on the correction pattern; and training the speech recognition model using the second correction result obtained by correcting the second speech recognition result as training data.

[0096] (Supplementary Note 12) A computer-readable recording medium having recorded thereon the program according to Supplementary Note 11, which causes the computer to further execute a process of training the speech recognition model using the second speech recognition result and the second correction result as the training data.

[0097] (Supplementary Note 13) A computer-readable recording medium having recorded thereon the program according to Supplementary Note 11, which causes the computer to execute a process of presenting the first speech recognition result to a user and displaying on an output device a user interface that allows the user to correct the first speech recognition result.

[0098] (Supplementary Note 14) A computer-readable recording medium having recorded thereon the program according to Supplementary Note 13, which causes the computer to execute the following process: extracting a portion to be corrected from the second speech recognition result based on the correction pattern; and displaying the portion to be corrected from the second speech recognition result in an emphasized manner in the user interface.

[0099] (Supplementary Note 15) A computer-readable recording medium having recorded thereon a program according to Supplementary Note 11 that causes the computer to execute the following processes: in correcting the second speech recognition result, calculate a first similarity by summing the similarity between the pre-correction portion included in the second speech recognition result and a first sentence of the pre-correction portion and the similarity between the pre-correction portion and a second sentence of the pre-correction portion; and calculate a second similarity by summing the similarity between the pre-correction portion included in the second correction result and a first sentence of the pre-correction portion and the similarity between the pre-correction portion and a second sentence of the pre-correction portion; and correcting the second speech recognition result if the second similarity is higher than the first similarity.

[0100] (Supplementary Note 16) A speech recognition device that converts speech into text using a speech recognition model, the speech recognition device having: an extraction unit that extracts a correction portion of a first correction result based on a difference between a first speech recognition result output from a speech recognition model and a first correction result obtained by correcting the first speech recognition result; a generation unit that generates a correction pattern by associating the correction portion with a pre-correction portion of the first speech recognition result that corresponds to the correction portion; a correction unit that corrects a second speech recognition result that was previously stored based on the correction pattern; and a learning unit that trains the speech recognition model using a second correction result obtained by correcting the second speech recognition result as learning data.

[0101] (Supplementary Note 17) The speech recognition device according to Supplementary Note 16, wherein the learning unit further learns the speech recognition model using the second speech recognition result and the second correction result as the learning data.

[0102] (Supplementary Note 18) The speech recognition device according to Supplementary Note 16, further comprising a user interface that presents the first speech recognition result to a user and allows the user to correct the first speech recognition result.

[0103] (Supplementary Note 19) The speech recognition device according to Supplementary Note 18, wherein the correction unit extracts a portion to be corrected from the second speech recognition result based on the correction pattern, and the user interface displays the portion to be corrected from the second speech recognition result in an emphasized manner.

[0104] (Supplementary Note 20) The speech recognition device according to Supplementary Note 16, wherein, in correcting the second speech recognition result, the correction unit calculates a first similarity by summing a similarity between a pre-correction portion included in the second speech recognition result and a preceding sentence of the pre-correction portion and a similarity between the pre-correction portion and a succeeding sentence of the pre-correction portion; and calculates a second similarity by summing a similarity between the pre-correction portion included in the second correction result and a preceding sentence of the pre-correction portion and a similarity between the pre-correction portion and a succeeding sentence of the pre-correction portion; and corrects the second speech recognition result if the second similarity is higher than the first similarity.

[0105] (Supplementary Note 21) A speech recognition method comprising: a speech recognition device that converts speech into text using a speech recognition model, the speech recognition device extracting a correction portion of a first correction result based on a difference between a first speech recognition result output from a speech recognition model and a first correction result obtained by correcting the first speech recognition result; generating a correction pattern by associating the correction portion with a pre-correction portion of the first speech recognition result that corresponds to the correction portion; correcting a second speech recognition result that was previously stored based on the correction pattern; and training the speech recognition model using the second correction result obtained by correcting the second speech recognition result as training data.

[0106] (Supplementary Note 22) The speech recognition method according to Supplementary Note 21, wherein the speech recognition device further trains the speech recognition model using the second speech recognition result and the second correction result as the training data.

[0107] (Supplementary Note 23) The speech recognition method according to Supplementary Note 21, wherein the speech recognition device presents the first speech recognition result to a user and displays a user interface that allows the user to correct the first speech recognition result.

[0108] (Supplementary Note 24) The speech recognition method according to Supplementary Note 23, wherein the speech recognition device extracts a portion to be corrected from the second speech recognition result based on the correction pattern, and the user interface displays the portion to be corrected from the second speech recognition result in an emphasized manner.

[0109] (Supplementary Note 25) The speech recognition method according to Supplementary Note 21, wherein the speech recognition device, in correcting the second speech recognition result, calculates a first similarity by summing a similarity between a pre-correction part included in the second speech recognition result and a preceding sentence of the pre-correction part and a similarity between the pre-correction part and a succeeding sentence of the pre-correction part, calculates a second similarity by summing a similarity between the pre-correction part included in the second correction result and a preceding sentence of the pre-correction part and a similarity between the pre-correction part and a succeeding sentence of the pre-correction part, and corrects the second speech recognition result if the second similarity is higher than the first similarity.

[0110] (Supplementary Note 26) A computer that converts speech to text using a speech recognition model, comprising: a computer-readable recording medium having recorded thereon a program that causes the computer to execute the following processes: extracting a correction portion of a first correction result based on a difference between a first speech recognition result output from the speech recognition model and a first correction result obtained by correcting the first speech recognition result; generating a correction pattern by associating the correction portion with a pre-correction portion of the first speech recognition result that corresponds to the correction portion; correcting a second speech recognition result that was previously stored based on the correction pattern; and training the speech recognition model using the second correction result obtained by correcting the second speech recognition result as training data.

[0111] (Supplementary Note 27) A computer-readable recording medium having recorded thereon the program according to Supplementary Note 26, which causes the computer to further execute a process of training the speech recognition model using the second speech recognition result and the second correction result as the training data.

[0112] (Supplementary Note 28) A computer-readable recording medium having recorded thereon the program according to Supplementary Note 26, which causes the computer to execute a process of presenting the first speech recognition result to a user and displaying on an output device a user interface that allows the user to correct the first speech recognition result.

[0113] (Supplementary Note 29) A computer-readable recording medium having recorded thereon the program according to Supplementary Note 28, which causes the computer to execute the following process: extracting a correction portion of the second speech recognition result based on the correction pattern; and displaying the correction portion of the second speech recognition result in an emphasized manner in the user interface.

[0114] (Supplementary Note 30) A computer-readable recording medium having recorded thereon the program according to Supplementary Note 26, which causes the computer to execute the following processes: in correcting the second speech recognition result, calculate a first similarity by summing the similarity between a pre-correction portion included in the second speech recognition result and a first sentence of the pre-correction portion and the similarity between the pre-correction portion and a second sentence of the pre-correction portion; and calculate a second similarity by summing the similarity between the pre-correction portion included in the second correction result and a first sentence of the pre-correction portion and the similarity between the pre-correction portion and a second sentence of the pre-correction portion; and correcting the second speech recognition result if the second similarity is higher than the first similarity.

[0115] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0116] This application claims priority based on Japanese Patent Application No. 2024-114424, filed on July 18, 2024, the disclosure of which is incorporated herein in its entirety by reference.

[0117] According to the above description, it is possible to reduce the correction of speech recognition results and generate a highly accurate speech recognition model, which is useful in fields where speech recognition is required.

[0118] 10 Information processing device 10A Speech recognition device 11 Extraction unit 12 Generation unit 13 Correction unit 14 Learning unit 15 Speech recognition unit 16 Editing unit 20 Speech input device 30 Storage device 40 Input device 50 Output device 100 System 110 Computer 111 CPU 112 Main memory 113 Storage device 114 Input interface 115 Display controller 116 Data reader / writer 117 Communication interface 118 Input device 119 Display device 120 Recording medium 121 Bus

Claims

1. An information processing device having: extraction means for extracting a corrected portion of a first correction result based on a difference between a first speech recognition result output from a speech recognition model that converts speech to text and a first correction result obtained by correcting the first speech recognition result; generation means for generating a correction pattern by associating the corrected portion with a pre-correction portion of the first speech recognition result that corresponds to the corrected portion; correction means for correcting a second speech recognition result that was previously stored based on the correction pattern; and learning means for training the speech recognition model using a second correction result obtained by correcting the second speech recognition result as learning data.

2. The information processing device according to claim 1, wherein the learning means further learns the speech recognition model using the second speech recognition result and the second correction result as the learning data.

3. The information processing device according to claim 1, further comprising a user interface that presents the first speech recognition result to a user and allows the user to correct the first speech recognition result.

4. The information processing device according to claim 3, wherein the correction means extracts the portion of the second speech recognition result to be corrected based on the correction pattern, and the user interface displays the portion of the second speech recognition result to be corrected in an emphasized manner.

5. The information processing device according to claim 1, wherein the correction means, in correcting the second speech recognition result, calculates a first similarity by summing the similarity between the pre-correction part included in the second speech recognition result and the preceding sentence of the pre-correction part and the similarity between the pre-correction part and the following sentence of the pre-correction part, calculates a second similarity by summing the similarity between the pre-correction part included in the second correction result and the preceding sentence of the pre-correction part and the similarity between the pre-correction part and the following sentence of the pre-correction part, and corrects the second speech recognition result if the second similarity is higher than the first similarity.

6. An information processing method, comprising: an information processing device extracting a corrected portion of a first correction result based on a difference between a first speech recognition result output from a speech recognition model that converts speech to text and a first correction result obtained by correcting the first speech recognition result; generating a correction pattern by associating the corrected portion with a pre-correction portion of the first speech recognition result that corresponds to the corrected portion; correcting a second speech recognition result that was previously stored based on the correction pattern; and training the speech recognition model using the second correction result obtained by correcting the second speech recognition result as training data.

7. A computer-readable recording medium having recorded thereon a program that causes a computer to execute the following processes: extracting a corrected portion of a first correction result based on a difference between a first speech recognition result output from a speech recognition model that converts speech to text and a first correction result obtained by correcting the first speech recognition result; generating a correction pattern by associating the corrected portion with a pre-correction portion of the first speech recognition result that corresponds to the corrected portion; correcting a second speech recognition result that was previously stored based on the correction pattern; and training the speech recognition model using the second correction result obtained by correcting the second speech recognition result as training data.

8. A speech recognition device that converts speech into text using a speech recognition model, the speech recognition device comprising: extraction means that extracts a corrected portion of a first correction result based on a difference between a first speech recognition result output from a speech recognition model and a first correction result obtained by correcting the first speech recognition result; generation means that generates a correction pattern by associating the corrected portion with a pre-correction portion of the first speech recognition result that corresponds to the corrected portion; correction means that corrects a second speech recognition result that was previously stored based on the correction pattern; and learning means that trains the speech recognition model using the second correction result obtained by correcting the second speech recognition result as learning data.

9. A speech recognition method for converting speech into text using a speech recognition model, wherein the speech recognition device: extracts a corrected portion of a first correction result based on a difference between a first speech recognition result output from the speech recognition model and a first correction result obtained by correcting the first speech recognition result; generates a correction pattern by associating the corrected portion with a pre-correction portion of the first speech recognition result that corresponds to the corrected portion; corrects a second speech recognition result that was previously stored based on the correction pattern; and trains the speech recognition model using the second correction result obtained by correcting the second speech recognition result as training data.

10. A computer that converts speech to text using a speech recognition model, comprising a computer-readable recording medium having recorded thereon a program that causes the computer to execute the following processes: extracting a corrected portion of a first correction result based on a difference between a first speech recognition result output from the speech recognition model and a first correction result obtained by correcting the first speech recognition result; generating a correction pattern by associating the corrected portion with a pre-correction portion of the first speech recognition result that corresponds to the corrected portion; correcting a second speech recognition result that was previously stored based on the correction pattern; and training the speech recognition model using the second correction result obtained by correcting the second speech recognition result as training data.

Images