Speech recognition processing device, speech recognition processing method, and recording medium

Abstract

This speech recognition processing device comprises: a speech recognition means that performs speech recognition on input speech and outputs text data corresponding to the input speech; an inference means that detects an erroneously recognized term erroneously recognized in the speech recognition from the text data corresponding to the input speech, and infers the correct term corresponding to the erroneously recognized term; a generation means that generates a plurality of sentences including the correct term; a speech synthesis means that synthesizes speech for the plurality of sentences, and outputs speech data corresponding to the plurality of sentences; and a training means that trains a speech recognition model by using the speech data corresponding to the plurality of sentences.

Description

Speech recognition processing device, speech recognition processing method, and recording medium

[0001] The present disclosure relates to the technical fields of a speech recognition processing device, a speech recognition processing method, and a recording medium.

[0002] There are known techniques related to speech recognition processing that recognizes speech data and converts it into text data. For example, Patent Literature 1 discloses performing speech recognition processing using a speech recognition model, converting text data into speech data by speech synthesis processing, and creating a new model using a dataset including speech data.

[0003] JP 2019-211689 A

[0004] An object of this disclosure is to provide a speech recognition processing device, a speech recognition processing method, and a recording medium that aim to improve upon the techniques disclosed in prior art documents.

[0005] One aspect of the speech recognition processing device disclosed herein includes speech recognition means for performing speech recognition on input speech and outputting text data corresponding to the input speech; estimation means for detecting erroneously recognized terms that have been erroneously recognized in the speech recognition from the text data corresponding to the input speech and estimating correct terms corresponding to the erroneously recognized terms; generation means for generating a plurality of sentences including the correct terms; speech synthesis means for performing speech synthesis on the plurality of sentences and outputting speech data corresponding to the plurality of sentences; and training means for training a speech recognition model using the speech data corresponding to the plurality of sentences.

[0006] One aspect of the speech recognition processing method of the present disclosure is one in which at least one computer performs speech recognition on input speech, outputs text data corresponding to the input speech, detects incorrectly recognized terms that have been incorrectly recognized in the speech recognition from the text data corresponding to the input speech, estimates correct terms corresponding to the incorrectly recognized terms, generates a plurality of sentences including the correct terms, performs speech synthesis on the plurality of sentences, outputs speech data corresponding to the plurality of sentences, and trains a speech recognition model using the speech data corresponding to the plurality of sentences.

[0007] One aspect of the recording medium of this disclosure is a recording medium having recorded thereon a computer program that causes at least one computer to execute a speech recognition processing method, which includes performing speech recognition on input speech, outputting text data corresponding to the input speech, detecting, from the text data corresponding to the input speech, incorrectly recognized terms that have been incorrectly recognized in the speech recognition, estimating correct terms corresponding to the incorrectly recognized terms, generating a plurality of sentences including the correct terms, performing speech synthesis on the plurality of sentences, outputting speech data corresponding to the plurality of sentences, and training a speech recognition model using the speech data corresponding to the plurality of sentences.

[0008] 1 is a block diagram showing the hardware configuration of a first speech recognition processing device. FIG. 2 is a block diagram showing the functional configuration of the first speech recognition processing device. FIG. 3 is a flowchart showing the operation flow of the first speech recognition processing device. FIG. 4 is a block diagram showing the configuration of a correct term estimation unit and a sentence generation unit in a second speech recognition processing device. FIG. 5 is a block diagram showing the configuration of a correct term estimation unit and a sentence generation unit in a modified example of the second speech recognition processing device. FIG. 6 is a block diagram showing a specific operation example of the second speech recognition processing device. FIG. 7 is a block diagram showing the functional configuration of a third speech recognition processing device. FIG. 8 is a flowchart showing the operation flow of the third speech recognition processing device. FIG. 9 is a block diagram showing an operation example using auxiliary information in the third speech recognition processing device.

[0009] Hereinafter, embodiments of a speech recognition processing device, a speech recognition processing method, and a recording medium will be described with reference to the drawings.

[0010] First Embodiment A first speech recognition processing device will be described with reference to FIGS. 1 to 3. FIG.

[0011] (Hardware Configuration) First, the hardware configuration of the first speech recognition processing device will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the hardware configuration of the first speech recognition processing device.

[0012] 1, the first speech recognition processing device 1 includes a processor 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, a storage device 14, an input device 15, and an output device 16. The processor 11, RAM 12, ROM 13, storage device 14, input device 15, and output device 16 are connected to each other via a data bus 17. The data bus 17 may be an interface other than a data bus (for example, a LAN, a USB, etc.).

[0013] The processor 11 loads a computer program. For example, the processor 11 is configured to load a computer program stored in at least one of the RAM 12, the ROM 13, and the storage device 14. Alternatively, the processor 11 may load a computer program stored in a computer-readable storage medium using a storage medium reading device (not shown). The processor 11 may acquire (i.e., load) a computer program from a device (not shown) located outside the first speech recognition processing device 1 via a network interface. The processor 11 executes the loaded computer program to perform various processes. When the processor 11 executes the loaded computer program, functional blocks related to the processes executed by the first speech recognition processing device 1 are realized within the processor 11. In other words, the processor 11 may function as a controller that executes each control in the first speech recognition processing device 1.

[0014] The processor 11 may be configured as, for example, a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or a quantum processor. The processor 11 may be configured as one of these, or may be configured to use multiple processors in parallel.

[0015] The RAM 12 temporarily stores computer programs executed by the processor 11. The RAM 12 temporarily stores data that the processor 11 temporarily uses while it is executing the computer programs. The RAM 12 may be, for example, a dynamic random access memory (D-RAM) or a static random access memory (SRAM). Alternatively, other types of volatile memory may be used instead of the RAM 12.

[0016] The ROM 13 stores computer programs executed by the processor 11. The ROM 13 may also store fixed data. The ROM 13 may be, for example, a programmable read-only memory (PROM) or an erasable read-only memory (EPROM). Alternatively, other types of non-volatile memory may be used instead of the ROM 13.

[0017] The storage device 14 stores data that is to be saved long-term by the first speech recognition processing device 1. The storage device 14 may operate as a temporary storage device for the processor 11. The storage device 14 may store computer programs executed by the processor 11. The storage device 14 may include, for example, at least one of a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device.

[0018] The input device 15 is a device that receives input instructions from a user of the first speech recognition processing device 1. The input device 15 may include, for example, at least one of a keyboard, a mouse, and a touch panel. The input device 15 may also be, for example, a device that includes a microphone and is capable of voice input.

[0019] The output device 16 is a device that outputs information related to the first speech recognition processing device 1 to the outside. For example, the output device 16 may be a display device (e.g., a display or a monitor) that can display information related to the first speech recognition processing device 1. The output device 16 may also be a speaker or the like that can output information related to the speech recognition processing device 1 as audio.

[0020] The first speech recognition processing device may be configured to include some of the components described in Fig. 1. For example, the first speech recognition processing device 1 may be configured to include only the processor 11, RAM 12, and ROM 13 among the above-mentioned components. In this case, the storage device 14, input device 15, and output device 16 may each be provided as an external device to the first speech recognition processing device 1. Furthermore, some of the calculation functions of the first speech recognition processing device 1 may be realized by an external server, cloud, etc.

[0021] (Functional Configuration) Next, the functional configuration of the first speech recognition processing device 1 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the functional configuration of the first speech recognition processing device.

[0022] 2, the first speech recognition processing device 1 is configured to be able to train a speech recognition model using training data generated using input speech. The first speech recognition processing device 1 is configured to include, as components for realizing its functions, a speech recognition unit 110, a correct term estimation unit 120, a sentence generation unit 130, a speech synthesis unit 140, and a training unit 150. Note that each of the speech recognition unit 110, the correct term estimation unit 120, the sentence generation unit 130, the speech synthesis unit 140, and the training unit 150 may be a processing block realized by the above-mentioned processor 11 (see FIG. 1).

[0023] The speech recognition unit 110 is configured to be capable of performing speech recognition on input speech. The input speech is input as speech data, and may be, for example, conversation data obtained by recording a conversation between people or conference data obtained by recording a conference. The speech recognition unit 110 is configured to perform speech recognition on such input speech and output text data corresponding to the input speech. That is, the speech recognition unit 110 is configured to be capable of converting speech data into text data (so-called transcription). Note that the specific method of speech recognition performed by the speech recognition unit 110 is not particularly limited. The speech recognition unit 110 may perform speech recognition using, for example, a speech recognition model prepared in advance. The speech recognition model may be, for example, a model including a neural network trained by deep learning. The text data corresponding to the input speech converted by the speech recognition unit 110 is configured to be output to the correct term estimation unit 120.

[0024] The correct term estimation unit 120 is configured to detect incorrectly recognized terms from the text data output from the speech recognition unit 110. The incorrectly recognized terms are terms incorrectly recognized during speech recognition by the speech recognition unit 110. That is, the incorrectly recognized terms are terms that are recognized with a meaning different from the term intended by the speaker. The correct term estimation unit 120 is configured to estimate correct terms corresponding to the detected incorrectly recognized terms. The correct terms are terms correctly recognized during speech recognition by the speech recognition unit 110. That is, the correct terms are terms obtained by correcting the incorrectly recognized terms to the term intended by the speaker. Note that the specific methods by which the correct term estimation unit 120 detects incorrectly recognized terms and the specific methods by which the correct term estimation unit 120 estimates correct terms are not particularly limited. The correct term estimation unit 120 may use, for example, a large language model (LLM). The configuration in which the correct term estimation unit 120 uses a large language model will be described in detail in another embodiment described later. Information about the correct terms estimated by the correct term estimation unit 120 is configured to be output to the sentence generation unit 130.

[0025] The sentence generation unit 130 is configured to generate a plurality of sentences including the correct terms estimated by the correct term estimation unit 120. That is, the sentence generation unit 130 is configured to generate a plurality of text data including the correct terms. The number of sentences and the number of characters generated by the sentence generation unit 130 may be set in advance. Alternatively, the number of sentences and the number of characters generated by the sentence generation unit 130 may be set appropriately depending on the correct terms. In this case, the correct term estimation unit 120 may be configured to output information regarding the number of sentences and the number of characters to the sentence generation unit 130 in addition to information regarding the correct terms. Note that the specific method for generating sentences by the sentence generation unit 130 is not particularly limited. The sentence generation unit 130 may use, for example, a large-scale language model. The configuration in which the sentence generation unit 130 uses a large-scale language model will be described in detail in another embodiment described later. Information regarding the plurality of sentences generated by the sentence generation unit 130 is configured to be output to the speech synthesis unit 140.

[0026] The speech synthesis unit 140 is configured to be able to perform speech synthesis on each of the multiple sentences generated by the sentence generation unit 130. That is, the speech synthesis unit 140 is configured to be able to convert the multiple pieces of text data generated by the sentence generation unit 130 into speech data by speech synthesis. Note that the specific method of speech synthesis performed by the speech synthesis unit 140 is not particularly limited. The speech synthesis unit 140 may perform speech synthesis using, for example, a speech synthesis model prepared in advance. The speech synthesis model may be, for example, a model including a neural network trained by deep learning. The speech data corresponding to the multiple sentences converted by the speech synthesis unit 140 is configured to be output to the learning unit 150.

[0027] The learning unit 150 is configured to be able to train a speech recognition model using the speech data converted by the speech synthesis unit 140 (i.e., speech data corresponding to multiple sentences generated by the sentence generation unit 130). That is, the learning unit 150 is configured to be able to train a speech recognition model by using the speech data output from the speech synthesis unit 140 as training data. The learning method used by the learning unit 150 is not particularly limited. For example, the learning unit 150 may train the speech recognition model using a backpropagation algorithm. The speech data used as training data may be linked to pre-conversion text data (i.e., text data generated by the sentence posture unit 130) as correct answer data. In this case, the speech synthesis unit 140 may output a pair of speech data and text data to the learning unit 150. The speech recognition model trained by the learning unit 150 is typically different from the speech recognition model used by the speech recognition unit 110. However, the learning unit 150 may also be configured to train the speech recognition model used by the speech recognition unit 110.

[0028] (Flow of Operation) Next, the flow of operation of the first speech recognition processing device 1 will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of operation of the first speech recognition processing device.

[0029] 3, when the operation of the first speech recognition processing device 1 starts, the speech recognition unit 110 first acquires input speech (step S101). The speech recognition unit 110 may acquire, for example, accumulated speech data from a storage device or may acquire real-time spoken speech data directly from a microphone. The speech recognition unit 110 then performs speech recognition on the input speech and outputs text data corresponding to the input speech (step S102).

[0030] Next, the correct term estimation unit 120 detects incorrectly recognized terms from the text data output from the speech recognition unit (step S103), and estimates correct terms corresponding to the detected incorrectly recognized terms (step S104).

[0031] Next, the sentence generation unit 130 generates a plurality of sentences including the correct terms estimated by the correct term estimation unit 120 (step S105). After that, the speech synthesis unit 140 performs speech synthesis on the plurality of sentences generated by the sentence generation unit 130 and outputs speech data corresponding to the plurality of sentences (step S106).

[0032] Next, the learning unit 150 learns a speech recognition model using the speech data output from the speech synthesis unit 140 (step S107). After the learning is completed, the learning unit 150 may execute a process of outputting and saving the trained speech recognition model.

[0033] (Technical Effects) Next, technical effects obtained by the first speech recognition processing device 1 will be described.

[0034] 1 to 3, in the first speech recognition processing device 1, incorrectly recognized terms are detected from text data corresponding to input speech, and correct terms corresponding to the incorrectly recognized terms are estimated. Then, a speech recognition model is trained using speech data corresponding to a plurality of sentences including the correct terms. In this way, it is possible to properly train the speech recognition model for terms that are likely to be incorrectly recognized during speech recognition.

[0035] For example, in everyday conversations, simple words are often used, making it difficult for a speech recognition model to make a misrecognition. On the other hand, in conversations in special business environments, technical terms and other difficult terms are often used, making it more likely for a speech recognition model to make a misrecognition. To address this issue, as already described, the first speech recognition processing device 1 detects incorrectly recognized terms (i.e., difficult terms that have been incorrectly recognized) from text data corresponding to input speech, and estimates correct terms (i.e., terms that would be correctly recognized) corresponding to the incorrectly recognized terms. Then, sentences containing the correct terms are converted into speech data and used as training data. Therefore, the first speech recognition processing device 1 makes it possible to generate a speech recognition model that can appropriately recognize difficult terms that are difficult to recognize.

[0036] In addition, when learning difficult terms, it is usually necessary to manually create training data. However, as described above, the first speech recognition processing device 1 automatically generates training data from input speech. Therefore, as long as speech data containing difficult terms (e.g., conversation data from a business site) is available, training data can be appropriately generated from the input speech without relying on manual labor. Therefore, the first speech recognition processing device 1 can reduce the labor costs and time costs involved in training.

[0037] The speech recognition model trained by the first speech recognition processing device 1 can be applied to various cases where difficult terms such as technical terms are used. For example, the speech recognition model may be used for automatic subtitling or automatic minutes creation. Alternatively, the speech recognition model may be used for machine operation using speech recognition at construction sites, etc.

[0038] <Second embodiment> A second speech recognition processing device 1 will be described with reference to Figures 4 to 6. The second speech recognition processing device 1 differs in part in configuration and operation from the above-described first speech recognition processing device 1, but other parts may be similar to the first speech recognition processing device 1. Therefore, hereinafter, parts that differ from the first embodiment will be described in detail, and descriptions of other overlapping parts will be omitted as appropriate.

[0039] (Configuration of the correct term estimation unit and the sentence generation unit) First, the configuration of the correct term estimation unit 120 and the sentence generation unit 130 in the second speech recognition processing device 1 will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the configuration of the correct term estimation unit and the sentence generation unit in the second speech recognition processing device. Note that Fig. 4 illustrates only some of the components described in Fig. 2, and omits the illustration of the other components.

[0040] 4 , the correct term estimation unit 120 in the second speech recognition processing device 1 is configured to operate using a first large-scale language model 210. Specifically, the correct term estimation unit 120 detects misrecognized terms using the first large-scale language model 210. The correct term estimation unit 120 also estimates correct terms using the first large-scale language model 210. The first large-scale language model may be a large-scale language model prepared in advance for the correct term estimation unit 120.

[0041] The sentence generation unit 130 in the second speech recognition processing device 1 is configured to operate using the second large-scale language model 220. Specifically, the sentence generation unit 130 generates a plurality of sentences including correct terms using the second large-scale language model 220. The second large-scale language model may be a large-scale language model prepared in advance for the sentence generation unit 130.

[0042] In the above example, a configuration has been described in which both the correct term estimation unit 120 and the sentence generation unit 130 use large-scale language models, but only one of the correct term estimation unit 120 and the sentence generation unit 130 may be configured to use a large-scale language model. For example, the correct term estimation unit 120 may estimate correct terms using the first large-scale language model 210, while the sentence generation unit 130 may generate sentences including correct terms without using the second large-scale language model 220. Alternatively, the correct term estimation unit 120 may estimate correct terms without using the first large-scale language model 210, while the sentence generation unit 130 may generate sentences including correct terms using the second large-scale language model 220.

[0043] (Modification) Next, the configuration of the correct term estimation unit 120 and the sentence generation unit 130 in a modification of the second speech recognition processing device 1 will be described with reference to Fig. 5. Fig. 5 is a block diagram showing the configuration of the correct term estimation unit and the sentence generation unit in a modification of the second speech recognition processing device. Note that in Fig. 5, the same reference numerals are used to designate the same elements as those shown in Fig. 4.

[0044] 5 , the correct term estimation unit 120 and the sentence generation unit 130 in the modified example of the second speech recognition processing device 1 are configured to operate using a common large-scale language model 250. That is, the correct term estimation unit 120 in the modified example detects misrecognized terms using the common large-scale language model 250. The correct term estimation unit 120 in the modified example estimates correct terms using the common large-scale language model 250. The sentence generation unit 130 in the modified example generates a plurality of sentences including correct terms using the common large-scale language model 250. In this way, the correct term estimation unit 120 and the sentence generation unit 130 may be configured to operate using the same large-scale language model.

[0045] (Operation Example) Next, an operation example of the second speech recognition processing device 1 (specifically, an operation example of the correct term estimation unit 120 and the sentence generation unit 130) will be described with reference to Fig. 6. Fig. 6 is a block diagram showing a specific operation example of the second speech recognition processing device.

[0046] As shown in Figure 6, assume that text data "When military collapse syndrome occurs, ..." (i.e., text data converted by speech recognition by the speech recognition unit 110) is input to the correct term estimation unit 120 in the second speech recognition processing device 1.

[0047] In this case, the correct term estimation unit 120 uses a large-scale language model to detect misrecognized terms from the input text data. In this example, the term "military collapse syndrome" is detected as a misrecognized term.

[0048] The correct term estimation unit 120 then estimates a correct term corresponding to the detected incorrectly recognized term using a large-scale language model. Here, the correct term estimation unit 120 estimates a correct term based on the pronunciation pattern (in other words, the reading) of the incorrectly recognized term. The correct term 120 may be, for example, a term that is read exactly the same as the incorrectly recognized term or a term that is read differently by only one character.

[0049] The reading of "national military collapse syndrome" detected as a misrecognized term is "hogunhokaishokougun." Therefore, the correct term estimation unit 120 searches for a term with a reading close to "hogunhokaishokougun" as the correct term. As a result, the term "bee colony collapse syndrome" is estimated as the correct term here. By estimating the correct term based on the pronunciation pattern in this way, it is possible to appropriately estimate the correct term corresponding to the misrecognized term.

[0050] Next, the sentence generation unit 130 generates a plurality of sentences including the correct term using the large-scale language model. Here, the sentence generation unit 130 generates the sentence "Colony collapse disorder is a phenomenon in which honeybees rush away in large numbers for an unknown reason" and the sentence "In the United States, colony collapse disorder has been reported, in which worker bees disappear, leaving only the queen bee and larvae behind." The plurality of sentences generated in this way are converted into speech data by speech synthesis, as already described, and are used to train the speech recognition model.

[0051] (Technical Effects) Next, technical effects obtained by the second speech recognition processing device 1 will be described.

[0052] As described with reference to FIGS. 4 to 6 , in the second speech recognition processing device 1, the correct term estimation unit 120 operates using a large-scale language model. This allows for appropriate detection of misrecognized terms from text data corresponding to input speech. Furthermore, it allows for appropriate estimation of correct terms corresponding to the detected misrecognized terms. Furthermore, in the second speech recognition processing device 1, the sentence generation unit 130 operates using a large-scale language model. This makes it possible to appropriately generate multiple sentences including correct terms. Note that, in the second speech recognition processing device 1, as described with reference to FIG. 5 , the correct term estimation unit 120 and the sentence generation unit 130 may use the same large-scale language model. This allows for appropriate execution of the successive operations (i.e., operations of detecting misrecognized terms, estimating correct terms, and estimating multiple sentences) performed by the correct term estimation unit 120 and the sentence generation unit 130 based on a common large-scale language model.

[0053] <Third embodiment> A third speech recognition processing device 1 will be described with reference to Figures 7 to 9. The third speech recognition processing device 1 differs in part in configuration and operation from the first and second speech recognition processing devices 1 described above, but other parts may be similar to the first and second speech recognition processing devices 1. Therefore, hereinafter, only parts that differ from the embodiments already described will be described in detail, and descriptions of other overlapping parts will be omitted as appropriate.

[0054] (Functional Configuration) First, the functional configuration of the third speech recognition processing device 1 will be described with reference to Fig. 7. Fig. 7 is a block diagram showing the functional configuration of the third speech recognition processing device. Note that in Fig. 7, the same elements as those shown in Fig. 2 are denoted by the same reference numerals.

[0055] 7, the third speech recognition processing device 1 is configured to include, as components for realizing its functions, a speech recognition unit 110, a correct term estimation unit 120, a sentence generation unit 130, a speech synthesis unit 140, a learning unit 150, and an auxiliary information analysis unit 160. That is, the third speech recognition processing device 1 further includes the auxiliary information analysis unit 160 in addition to the configuration described in the first embodiment (see FIG. 2). Note that the auxiliary information analysis unit 160 may be a processing block realized by the above-mentioned processor 11 (see FIG. 1).

[0056] The auxiliary information analysis unit 160 is configured to analyze the input additional information and output the auxiliary information to the correct term estimation unit 120 and the sentence generation unit 130. The additional information is information related to the input speech, and may be, for example, conversation information including the conversation before and after the input speech, or an outline of the input speech (e.g., genre, etc.). Alternatively, the additional information may be conference materials (e.g., presentation materials) used in the conference where the input speech is acquired. The auxiliary information is information obtained by analyzing the additional information and is information for assisting the operations of the correct term estimation unit 120 and the sentence generation unit 130. The auxiliary information may be information that assists the correct term estimation unit 120 in detecting misrecognized terms. The auxiliary information may be information that assists the correct term estimation unit 120 in estimating correct terms. The auxiliary information may be information that assists the sentence generation unit 130 in generating sentences.

[0057] In the above example, the auxiliary information analysis unit 160 outputs auxiliary information to both the correct term estimation unit 120 and the sentence generation unit 130. However, the auxiliary information analysis unit 160 may be configured to output auxiliary information to only one of the correct term estimation unit 120 and the sentence generation unit 130. That is, the correct term estimation unit 120 may be configured to operate using auxiliary information, while the sentence generation unit 130 may be configured to operate without using auxiliary information. Alternatively, the correct term estimation unit 120 may be configured to operate without using auxiliary information, while the sentence generation unit 130 may be configured to operate using auxiliary information.

[0058] (Operation Flow) Next, the operation flow of the third speech recognition processing device 1 will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the operation flow of the third speech recognition processing device. Note that in Fig. 8, the same processes as those shown in Fig. 3 are denoted by the same reference numerals.

[0059] 8, when the operation of the third speech recognition processing device 1 starts, the speech recognition unit 110 first acquires input speech (step S101). Then, the speech recognition unit 110 performs speech recognition on the input speech and outputs text data corresponding to the input speech (step S102).

[0060] Meanwhile, the auxiliary information analysis unit 160 acquires additional information (step S201), analyzes the acquired additional information, and outputs the auxiliary information to the correct term estimation unit 120 and the sentence generation unit 130 (step S202).

[0061] Although the example described above shows that the processes of steps S201 and S202 are executed in parallel with the processes of steps S101 and S102, the processes of steps S201 and S202 may be executed at different times. For example, the processes of steps S201 and S202 may be executed before the processes of steps S101 and S102 are executed. Alternatively, the processes of steps S201 and S202 may be executed after the processes of steps S101 and S102 are executed.

[0062] Next, the correct term estimation unit 120 detects misrecognized terms from the text data output from the speech recognition unit using the auxiliary information input from the auxiliary information analysis unit 160 (step S203).The correct term estimation unit 120 then estimates correct terms corresponding to the misrecognized terms using the auxiliary information input from the auxiliary information analysis unit 160 (step S204).

[0063] Next, the sentence generation unit 130 uses the auxiliary information input from the auxiliary information analysis unit 160 to generate a plurality of sentences including the correct terms estimated by the correct term estimation unit 120 (step S205). After that, the speech synthesis unit 140 performs speech synthesis on the plurality of sentences generated by the sentence generation unit 130 and outputs speech data corresponding to the plurality of sentences (step S106).

[0064] Next, the learning unit 150 learns a speech recognition model using the speech data output from the speech synthesis unit 140 (step S107). After the learning is completed, the learning unit 150 may execute a process of outputting and saving the trained speech recognition model.

[0065] (Example of operation using auxiliary information) Next, an example of operation using auxiliary information in the third speech recognition processing device 1 will be specifically described with reference to Fig. 9. Fig. 9 is a block diagram showing an example of operation using auxiliary information in the third speech recognition processing device.

[0066] 9 , in the third speech recognition processing device 1, additional information, such as conversation information and meeting materials, is input to the auxiliary information analysis unit 160. The auxiliary information analysis unit 160 analyzes the additional information and outputs the auxiliary information to the correct term estimation unit 120 and the sentence generation unit 130.

[0067] The auxiliary information analysis unit 160 analyzes the conversation information (specifically, information including the conversation before and after the input speech) input as additional information, and outputs information about utterances before and after the input speech as auxiliary information to the correct information estimation unit 120. In this way, by using the auxiliary information about speech uttered before and after the input speech, the flow of context can be accurately grasped. As a result, the accuracy of detecting misrecognized terms and estimating correct terms can be improved.

[0068] For example, as in the example shown in Figure 6, if "Colony Collapse Disorder" is mistakenly recognized as "Army Collapse Disorder," if there is a comment about bees before or after the input speech, it is possible to detect with higher accuracy that "Army Collapse Disorder" is the misrecognized term, and to estimate with higher accuracy that "Colony Collapse Disorder" is the correct term.

[0069] The auxiliary information analysis unit 160 also analyzes the conversation information and meeting information input as additional information and outputs information related to the term of interest to the correct information estimation unit 120 as auxiliary information. The auxiliary information analysis unit 160 also analyzes the conversation information and meeting information input as additional information and outputs information related to the style of the sentence and the topic of the sentence to the correct information estimation unit 120 as auxiliary information. The style of the sentence may indicate characteristics or tendencies in written expression (e.g., vocabulary, grammar, etc.). The style of the sentence may also include a style of writing. Examples of styles include literary style (written language), colloquial style (spoken language), Japanese style, Chinese style, epistolary style, and essay style. In this way, the correct term estimation unit 120 can appropriately perform its detection operation and its correct term estimation operation based on the term of interest. The sentence generation unit 120 can more appropriately perform its sentence generation operation based on the style of the sentence and the topic.

[0070] For example, as in the example shown in Figure 6, if "colony collapse disorder" is predicted as the correct term, by inputting auxiliary information indicating that the topic is about bees, a natural sentence that is in line with the topic can be generated.

[0071] (Technical Effects) Next, technical effects obtained by the third speech recognition processing device 1 will be described.

[0072] 7 to 9, in the third speech recognition processing device 1, auxiliary information obtained by analyzing the additional information is output to the correct term estimation unit 120 and the sentence generation unit 130. In this way, the correct term estimation unit 120 can more appropriately perform the operation of detecting misrecognized terms, the operation of estimating correct terms, and the operation of generating sentences in the sentence generation unit 130.

[0073] The scope of each embodiment also includes a processing method in which a program that operates the configuration of each embodiment to realize the functions of the above-described embodiments is recorded on a recording medium, the program recorded on the recording medium is read as code, and the program is executed on a computer. In other words, a computer-readable recording medium is also included in the scope of each embodiment. Furthermore, each embodiment includes not only a recording medium on which the above-described program is recorded, but also the program itself.

[0074] Examples of recording media that can be used include floppy disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, magnetic tapes, non-volatile memory cards, and ROMs. Furthermore, the scope of each embodiment is not limited to programs that execute processes by themselves, but also includes programs that execute processes by operating on an OS in conjunction with other software or expansion board functions. Furthermore, the program itself may be stored on a server, and part or all of the program may be downloadable from the server to a user terminal. The program may be provided to the user in, for example, a SaaS (Software as a Service) format.

[0075] <Supplementary Notes> The above-described embodiment may be further described as in the following supplementary notes, but is not limited to the following.

[0076] (Supplementary Note 1) The speech recognition processing device described in Supplementary Note 1 is a speech recognition processing device including: speech recognition means that performs speech recognition on input speech and outputs text data corresponding to the input speech; estimation means that detects incorrectly recognized terms that have been erroneously recognized in the speech recognition from the text data corresponding to the input speech and estimates correct terms that correspond to the incorrectly recognized terms; generation means that generates a plurality of sentences including the correct terms; speech synthesis means that performs speech synthesis on the plurality of sentences and outputs speech data corresponding to the plurality of sentences; and learning means that trains a speech recognition model using the speech data corresponding to the plurality of sentences.

[0077] (Supplementary Note 2) The speech recognition processing device described in Supplementary Note 2 is the speech recognition processing device described in Supplementary Note 1, wherein the estimation means uses a large-scale language model to perform at least one of detecting the misrecognized terms and estimating the correct terms.

[0078] (Supplementary Note 3) The speech recognition processing device according to Supplementary Note 3 is the speech recognition processing device according to Supplementary Note 2, wherein the estimation means estimates the correct term based on a pronunciation pattern of the incorrectly recognized term.

[0079] (Supplementary Note 4) The speech recognition processing device according to Supplementary Note 4 is the speech recognition processing device according to any one of Supplementary Notes 1 to 3, wherein the generating means generates the plurality of sentences using a large-scale language model.

[0080] (Supplementary Note 5) The speech recognition processing device according to Supplementary Note 5 is the speech recognition processing device according to any one of Supplementary Notes 1 to 4, wherein the estimation means and the generation means use the same large-scale language model.

[0081] (Appendix 6) The speech recognition processing device described in Appendix 6 is the speech recognition processing device described in any one of Appendices 1 to 5, further comprising auxiliary information analysis means for analyzing input additional information and outputting auxiliary information that assists the operation of at least one of the estimation means and the generation means.

[0082] (Supplementary Note 7) The speech recognition processing device according to Supplementary Note 7 is the speech recognition processing device according to Supplementary Note 6, wherein the auxiliary information analysis means outputs information about speech uttered before and after the input speech as the auxiliary information.

[0083] (Appendix 8) A speech recognition processing device as described in Appendix 6 or 7, wherein the auxiliary information analysis means outputs information regarding at least one of terms, topics, and sentence style estimated based on the additional information as the auxiliary information.

[0084] (Supplementary Note 9) The speech recognition processing method described in Supplementary Note 9 is a speech recognition processing method in which at least one computer performs speech recognition on input speech, outputs text data corresponding to the input speech, detects incorrectly recognized terms that have been incorrectly recognized in the speech recognition from the text data corresponding to the input speech, estimates correct terms corresponding to the incorrectly recognized terms, generates a plurality of sentences including the correct terms, performs speech synthesis on the plurality of sentences, outputs speech data corresponding to the plurality of sentences, and trains a speech recognition model using the speech data corresponding to the plurality of sentences.

[0085] (Supplementary Note 10) The recording medium described in Supplementary Note 10 is a recording medium having recorded thereon a computer program for causing at least one computer to execute a speech recognition processing method, which includes performing speech recognition on input speech, outputting text data corresponding to the input speech, detecting, from the text data corresponding to the input speech, incorrectly recognized terms that have been incorrectly recognized in the speech recognition, estimating correct terms corresponding to the incorrectly recognized terms, generating a plurality of sentences including the correct terms, performing speech synthesis on the plurality of sentences, outputting speech data corresponding to the plurality of sentences, and training a speech recognition model using the speech data corresponding to the plurality of sentences.

[0086] (Supplementary Note 11) The computer program described in Supplementary Note 11 is a computer program that causes at least one computer to execute a speech recognition processing method, which performs speech recognition on input speech, outputs text data corresponding to the input speech, detects incorrectly recognized terms that have been incorrectly recognized in the speech recognition from the text data corresponding to the input speech, estimates correct terms corresponding to the incorrectly recognized terms, generates a plurality of sentences including the correct terms, performs speech synthesis on the plurality of sentences, outputs speech data corresponding to the plurality of sentences, and trains a speech recognition model using the speech data corresponding to the plurality of sentences.

[0087] This disclosure may be modified as appropriate within the scope that does not contradict the gist or idea of ​​the invention that can be read from the claims and the entire specification, and a speech recognition processing device, a speech recognition processing method, and a recording medium that incorporate such modifications are also included in the technical idea of ​​this disclosure.

[0088] REFERENCE SIGNS LIST 1 Speech recognition processing device 11 Processor 12 RAM 13 ROM 14 Storage device 15 Input device 16 Output device 17 Data bus 110 Speech recognition unit 120 Correct term estimation unit 130 Sentence generation unit 140 Speech synthesis unit 150 Learning unit 160 Auxiliary information analysis unit 210 First large-scale language model 220 Second large-scale language model 250 Common large-scale language model

Claims

1. A speech recognition processing device comprising: speech recognition means for performing speech recognition on input speech and outputting text data corresponding to the input speech; estimation means for detecting erroneously recognized terms from the text data corresponding to the input speech and estimating correct terms corresponding to the erroneously recognized terms; generation means for generating a plurality of sentences including the correct terms; speech synthesis means for performing speech synthesis on the plurality of sentences and outputting speech data corresponding to the plurality of sentences; and training means for training a speech recognition model using the speech data corresponding to the plurality of sentences.

2. The speech recognition processing device according to claim 1, wherein the estimation means uses a large-scale language model to perform at least one of detecting the misrecognized terms and estimating the correct terms.

3. The speech recognition processing device according to claim 2, wherein the estimation means estimates the correct term based on a pronunciation pattern of the incorrectly recognized term.

4. The speech recognition processing device according to claim 1, wherein the generation means generates the plurality of sentences using a large-scale language model.

5. The speech recognition processing device according to claim 1, wherein the estimation means and the generation means use the same large-scale language model.

6. A speech recognition processing device according to any one of claims 1 to 5, further comprising auxiliary information analysis means for analyzing input additional information and outputting auxiliary information that assists the operation of at least one of said estimation means and said generation means.

7. The speech recognition processing device according to claim 6, wherein the auxiliary information analysis means outputs information relating to speech uttered before and after the input speech as the auxiliary information.

8. The speech recognition processing device according to claim 6, wherein the auxiliary information analysis means outputs, as the auxiliary information, information relating to at least one of a term, a topic, and a sentence style estimated based on the additional information.

9. A speech recognition processing method in which at least one computer performs speech recognition on input speech, outputs text data corresponding to the input speech, detects incorrectly recognized terms from the text data corresponding to the input speech, and estimates correct terms corresponding to the incorrectly recognized terms, generates a plurality of sentences including the correct terms, performs speech synthesis on the plurality of sentences, and outputs speech data corresponding to the plurality of sentences, and trains a speech recognition model using the speech data corresponding to the plurality of sentences.

10. A recording medium having recorded thereon a computer program for causing at least one computer to execute a speech recognition processing method, which comprises: performing speech recognition on input speech, outputting text data corresponding to the input speech, detecting misrecognized terms that have been incorrectly recognized in the speech recognition from the text data corresponding to the input speech, and estimating correct terms corresponding to the misrecognized terms, generating a plurality of sentences including the correct terms, performing speech synthesis on the plurality of sentences, outputting speech data corresponding to the plurality of sentences, and training a speech recognition model using the speech data corresponding to the plurality of sentences.

Images