Training data generation device, speech recognition model generation device, training data generation method, and program

Abstract

A training data generation device (10) comprises: a voice recognition unit (140); and a generation unit (160). The voice recognition unit (140) generates text information by inputting voice data into a first voice recognition model which has already been trained. The generation unit (160) generates training data that includes the voice data and the text information. The first voice recognition model is a model that has been trained, using a synthetic sound which was generated by using input information relating to a predetermined item and previously prepared formatted text information.

Description

Training data generation device, speech recognition model generation device, training data generation method, speech recognition model generation method, and recording medium

[0001] The present invention relates to a training data generation device, a speech recognition model generation device, a training data generation method, a speech recognition model generation method, and a recording medium.

[0002] In order to obtain a trained model for speech recognition, a large amount of training data must be prepared.

[0003] Patent Literature 1 describes generating synthetic speech of optimal sentence examples for added words as speech data to be used for training a speech recognition system. Patent Literature 1 also describes generating optimal sentence examples using sentence example templates.

[0004] Patent Document 2 describes a method of recognizing a user's speech using a recognition engine trained with training data for each user, and generating training data including the speech and the recognition results.

[0005] International Publication No. 2021 / 215352 International Publication No. 2021 / 059968

[0006] In the above-mentioned Patent Document 1, the synthesized speech generated by the program is used as speech data for training a system that recognizes human voices. Therefore, there is a problem that there is a limit to the improvement of recognition accuracy by training using that speech data. Furthermore, the above-mentioned Patent Document 2 generates training data for each user, which makes it difficult to improve the recognition accuracy of speech recognition regardless of the user.

[0007] In view of the above-mentioned problems, an example of an object of the present invention is to provide a training data generation device, a speech recognition model generation device, a training data generation method, a speech recognition model generation method, and a recording medium that improve the recognition accuracy of a speech recognition model.

[0008] According to one aspect of the present invention, there is provided a training data generation device comprising: a speech recognition means for generating text information by inputting speech data into a trained first speech recognition model; and a generation means for generating training data including the speech data and the text information, wherein the first speech recognition model is a model trained using synthetic speech generated using input information related to predetermined items and predetermined standard text information.

[0009] According to one aspect of the present invention, there is provided a training data generation device comprising: a speech recognition means for inputting speech data into a trained first speech recognition model and a trained second speech recognition model, and generating output results of the first speech recognition model and the second speech recognition model, respectively; a determination means for determining whether or not there is a difference between the output result of the first speech recognition model and the output result of the second speech recognition model; and a generation means for generating training data including the speech data when the determination means determines that there is a difference between the output result of the first speech recognition model and the output result of the second speech recognition model.

[0010] According to one aspect of the present invention, there is provided a speech recognition model generation device that uses the training data generated by the training data generation device described above to train the second speech recognition model.

[0011] According to one aspect of the present invention, there is provided a training data generation method, in which one or more computers generate text information by inputting speech data into a trained first speech recognition model, and generate training data including the speech data and the text information, wherein the first speech recognition model is a model trained using synthetic speech generated using input information related to a predetermined item and predetermined standard text information.

[0012] According to one aspect of the present invention, there is provided a training data generation method in which one or more computers input speech data into a trained first speech recognition model and a trained second speech recognition model, respectively, to generate output results for the first speech recognition model and the second speech recognition model, determine whether there is a difference between the output result of the first speech recognition model and the output result of the second speech recognition model, and generate training data including the speech data if it is determined that there is a difference between the output result of the first speech recognition model and the output result of the second speech recognition model.

[0013] According to one aspect of the present invention, there is provided a speech recognition model generation method, in which one or more computers use the training data generated by the above-mentioned training data generation method to train the second speech recognition model.

[0014] According to one aspect of the present invention, there is provided a computer-readable recording medium having a program recorded thereon, the program causing a computer to function as a training data generation device, the training data generation device comprising: a speech recognition means for generating text information by inputting speech data into a trained first speech recognition model; and a generation means for generating training data including the speech data and the text information, wherein the first speech recognition model is a model trained using synthetic speech generated using input information relating to predetermined items and predetermined standard text information.

[0015] According to one aspect of the present invention, there is provided a computer-readable recording medium having a program recorded thereon, the program causing a computer to function as a training data generation device, the training data generation device comprising: speech recognition means for inputting speech data into a trained first speech recognition model and a trained second speech recognition model, and generating output results of the first speech recognition model and the second speech recognition model, respectively; determination means for determining whether or not there is a difference between the output result of the first speech recognition model and the output result of the second speech recognition model; and generation means for generating training data including the speech data when the determination means determines that there is a difference between the output result of the first speech recognition model and the output result of the second speech recognition model.

[0016] According to one aspect of the present invention, there is provided a computer-readable recording medium having a program recorded thereon, the program causing a computer to function as a speech recognition model generation device, the speech recognition model generation device being realized by the program recorded on the recording medium and using the training data generated by a training data generation device to train the second speech recognition model.

[0017] According to one aspect of the present invention, a training data generation device, a speech recognition model generation device, a training data generation method, a speech recognition model generation method, and a recording medium are provided that improve the recognition accuracy of a speech recognition model.

[0018] FIG. 1 is a diagram illustrating an overview of a training data generation device according to a first embodiment. FIG. 2 is a diagram illustrating an overview of a first speech recognition model. FIG. 3 is a diagram illustrating an example of an overview of a method for generating a first speech recognition model. FIG. 4 is a diagram illustrating an example of a functional configuration of a training data generation device according to a first embodiment. FIG. 5 is a diagram illustrating an example of an overview of a method by which a first model generation unit generates a first speech recognition model. FIG. 6 is a diagram illustrating an example of a functional configuration of a speech recognition model generation device according to a first embodiment. FIG. 7 is a diagram illustrating an example of a computer for realizing a training data generation device. FIG. 8 is a diagram illustrating an overview of a training data generation method according to the first embodiment. A flowchart illustrating an example of the flow of the training data generation method according to the first embodiment. FIG. 9 is a diagram illustrating an overview of a training data generation device according to a second embodiment. FIG. 10 is a diagram illustrating an example of the functional configuration of a training data generation device according to the second embodiment. FIG. 11 is a diagram illustrating an overview of a training data generation method according to the second embodiment. A flowchart illustrating an example of the flow of a training data generation method according to the second embodiment.

[0019] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, like components are designated by like reference numerals, and the description thereof will be omitted as appropriate.

[0020] First Embodiment FIG. 1 is a diagram illustrating an overview of a training data generation device 10 according to a first embodiment. FIG. 2 is a diagram illustrating an overview of a first speech recognition model 51. The training data generation device 10 includes a speech recognition unit 140 and a generation unit 160. The speech recognition unit 140 generates text information by inputting speech data into a trained first speech recognition model 51. The generation unit 160 generates training data including speech data and text information. The first speech recognition model 51 is a model trained using synthetic speech generated using input information related to predetermined items and pre-prepared standard text information.

[0021] According to this training data generation device 10, it is possible to improve the recognition accuracy of the voice recognition model.

[0022] A detailed example of the training data generating device 10 according to this embodiment will be described below.

[0023] In this embodiment, the voice data is data obtained by recording a person's speech. In other words, the voice data is not so-called synthetic sound data, which is artificially generated by a machine or the like. Furthermore, the voice data is data indicating a voice waveform or data indicating features of a voice waveform. The voice data is, for example, data obtained by recording a call voice in a voice call or a video call. As a specific example, the voice data is data obtained by recording a call voice for requesting the dispatch of an emergency vehicle (e.g., police, fire engine, ambulance). As another example, the voice data may be data obtained by recording the call voice of various call centers. One voice data may be generated for a series of calls, or multiple voice data may be generated by dividing a series of calls into multiple parts. However, the voice data is not limited to data obtained by recording a call.

[0024] 3 is a diagram illustrating an example of an outline of a method for generating the first speech recognition model 51. Both the first speech recognition model 51 and the second speech recognition model 52 are speech recognition models obtained by machine learning. As shown in FIG. 2, the first speech recognition model 51 is a trained model that can convert speech data into text information indicating content corresponding to the speech data. In other words, the input data of the first speech recognition model 51 includes speech data, and the output data of the first speech recognition model 51 includes text information. The first speech recognition model 51 is a model generated by training the second speech recognition model 52 using synthetic speech.

[0025] Like the first speech recognition model 51, the second speech recognition model 52 is a model capable of converting speech data into text information indicating the content corresponding to the speech data. The second speech recognition model 52 is also a model trained using a plurality of training data including speech data and text information indicating the content corresponding to the speech data. It is preferable that the second speech recognition model 52 is a model that has not been trained using synthetic speech. However, the second speech recognition model 52 may also be a model that has been trained using synthetic speech as part of its training.

[0026] That is, the first speech recognition model 51 may be a model trained using both one or more pieces of training data including speech data other than synthetic speech and one or more pieces of training data including synthetic speech throughout the entire training history. The first speech recognition model 51 may be a model trained using at least one piece of training data including synthetic speech. Of the multiple pieces of training data used to train the first speech recognition model 51, only one piece of training data may include synthetic speech. Synthetic speech will be described in detail later.

[0027] By training using synthetic speech, the first speech recognition model 51 is expected to have higher speech recognition accuracy than the second speech recognition model 52. The speech recognition unit 140 of the training data generation device 10 inputs speech data to the first speech recognition model 51, causing it to output text information. The generation unit 160 then generates training data, which associates the speech data input to the first speech recognition model 51 with the text information output from the first speech recognition model 51. As described above, the speech data included in this training data is not synthetic speech, but recorded data of actual speech by a person. While synthetic speech differs from actual speech in terms of phrase features, prosodic features, frequency characteristics, etc., the training data generation device 10 according to this embodiment generates training data including recorded data of actual speech by a person. Therefore, the training data generated by the training data generation device 10 according to this embodiment enables training that reflects these features, thereby achieving a speech recognition model with higher recognition accuracy.

[0028] 4 is a diagram illustrating the functional configuration of the training data generation device 10 according to this embodiment. In the example shown in this figure, the training data generation device 10 further includes an acquisition unit 110, a first model generation unit 120, a template text storage unit 130, a model storage unit 150, and a training data storage unit 170. In addition, in the example shown in this figure, the first model generation unit 120 includes a synthetic speech text generation unit 121, a synthetic speech generation unit 122, and a first training unit 123. Note that one or more of the template text storage unit 130, the model storage unit 150, and the training data storage unit 170 may be storage devices provided external to the training data generation device 10. Each functional component of the training data generation device 10 will be described in detail below.

[0029] The acquisition unit 110 acquires input information and voice data that are associated with each other. The input information corresponds to the content of the utterance in the voice data associated with the input information. For example, when the voice data is obtained by recording a telephone call, the input information is generated as follows. For example, the call recipient (inputter) inputs the content of the call into a terminal while the call is being made. For example, the terminal presents the inputter with multiple items to be input, and the input work is performed by filling in the input field for each item. Then, input information indicating the inputted content of the call is generated. As another example, after the call ends, the inputter inputs the content of the call into the terminal, thereby generating input information.

[0030] For example, an identification ID is assigned to the voice data of each call, and the input information is associated with the voice data by associating the identification ID of the call with the input information. The input information may also include the identification ID of the voice data. As another example, the input information may include the identification ID of the receiving terminal of the call and the date and time of receiving the call, thereby associating the input information with the voice data. In this case, the voice data is assigned information indicating the identification ID of the receiving terminal and the recording date and time (i.e., the date and time of receiving the call).

[0031] The items included in the input information correspond to the items to be entered into the terminal described above. The multiple items included in the input information may include, for example, one or more of the following: "name of the callee," "address," "telephone number," and "items related to the subject of the call." For example, if the call is a request for the dispatch of an emergency vehicle (e.g., police, fire engine, or ambulance), the "items related to the subject of the call" may include, for example, the "type of incident" (e.g., whether it is an incident, accident, fire, or sudden illness) and the "location of the request" (the location of the accident, etc.). Here, the "type of incident" may be indicated by, for example, a predetermined number or symbol for each of the incident, accident, fire, and sudden illness. For example, if the call is a request for the dispatch of an ambulance, the "items related to the subject of the call" may further include one or more of the following: "body part," "condition of injury, etc.," and "symptoms." The items included in the input information may vary depending on the "type of incident." For example, if the "type of incident" is an incident or an accident, the "items related to the subject of the call" may include the "scene situation" (e.g., a vehicle rollover). For example, if the call is for placing a telephone shopping order, the "items related to the business" may include "information indicating the purchased product," "quantity purchased," "delivery address," etc. The content entered for each item may be text.

[0032] Such input work is usually performed within the scope of call reception work and does not need to be performed specially in order to have the training data generation device 10 generate training data. Therefore, by using the training data generation device 10, training data can be generated without requiring special effort, and the accuracy of the speech recognition model can be improved. However, the input information is not limited to the above example, and may be any information that can generate synthesis text by applying the content of each item to standard text information, as described below. The input information does not necessarily have to be related to the speech data acquired by the acquisition unit 110. In this case, multiple pieces of input information may be used to generate multiple synthesis texts and multiple synthetic sounds. The first speech recognition model 51 may be a model trained using multiple synthetic sounds.

[0033] The method by which the acquisition unit 110 acquires the input information and voice data is not particularly limited, but for example, the acquisition unit 110 can acquire the input information and voice data by reading them from a storage device that stores the input information and voice data. As another example, the acquisition unit 110 may acquire the input information directly from a terminal to which the call content is input.

[0034] The acquisition unit 110 can acquire multiple pieces of speech data. The acquisition unit 110 may acquire each piece of speech data each time it is generated, or may acquire multiple pieces of speech data at once. It is preferable that the first model generation unit 120 generates a first speech recognition model 51 for each piece of speech data acquired by the acquisition unit 110.

[0035] FIG. 5 is a diagram illustrating an example of an outline of a method by which the first model generation unit 120 generates the first speech recognition model 51. The synthetic speech text generation unit 121 of the first model generation unit 120 acquires input information corresponding to, for example, certain speech data. The synthetic speech text generation unit 121 also acquires formula text information from the formula text storage unit 130. The formula text information is prepared in advance and stored in the formula text storage unit 130. The formula text information is information indicating a formula text, such as, for example, "This is xx. An accident occurred at yy." The synthetic speech text generation unit 121 generates the synthetic speech text by applying the input information to the formula text information. Specifically, for example, the "xx" part of "This is xx. An accident occurred at yy." is replaced with the name indicated in the input information, and the "yy" part is replaced with the requested location indicated in the input information. In this way, the synthetic speech text generation unit 121 can generate the synthetic speech text using the formula text information and the input information.

[0036] Here, the synthetic speech text generation unit 121 may select the fixed text information to be used from multiple fixed text information stored in the fixed text storage unit 130. For example, the fixed text storage unit 130 stores fixed text information for each type of incident. Each piece of associated text information is associated with one of the types of incident. The synthetic speech text generation unit 121 then selects the fixed text information corresponding to the type of incident indicated in the input information as the fixed text information to be used. For example, if the type of incident is an accident, the fixed text information "This is xx. An accident has occurred at yy" is selected; if the type of incident is a fire, the fixed text information "This is xx. A fire has occurred at yy" is selected; and if the type of incident is a sudden illness, the fixed text information "This is xx. There is a person who is suddenly ill at yy" is selected. The synthetic speech text generation unit 121 then generates the text for synthetic speech using the selected fixed text information in the same manner as described above.

[0037] The synthetic speech generation unit 122 acquires the synthetic speech text generated by the synthetic speech text generation unit 121 and converts it into synthetic speech. The synthetic speech corresponds to the content of the synthetic speech text and is equivalent to the speech of the synthetic speech text being read aloud. Existing technology can be used as a method for converting the synthetic speech text into synthetic speech. The synthetic speech text generation unit 121 can convert the synthetic speech text into synthetic speech using, for example, a trained model that takes text as input and outputs synthetic speech.

[0038] The first learning unit 123 generates training data for generating the first speech recognition model 51 by associating the synthetic speech text generated by the synthetic speech text generation unit 121 with the synthetic speech generated by the synthetic speech generation unit 122. The first learning unit 123 then uses the generated training data to train the second speech recognition model 52, thereby generating the first speech recognition model 51. The second speech recognition model 52 is stored in the model storage unit 150, and the first learning unit 123 can read the second speech recognition model 52 from the model storage unit 150 and use it to generate the first speech recognition model 51. The first speech recognition model 51 trained using training data including the synthetic speech text and the synthetic speech is output to the speech recognition unit 140. In this way, the first model generation unit 120 can generate the first speech recognition model 51 with improved recognition accuracy by using input information without requiring much effort.

[0039] 4 , the speech recognition unit 140 acquires the first speech recognition model 51 generated by the first model generation unit 120. Then, the speech data acquired by the acquisition unit 110 is input to the acquired first speech recognition model 51. Then, text information corresponding to the speech data is generated as an output of the first speech recognition model 51.

[0040] The generation unit 160 generates training data by associating the speech data acquired by the acquisition unit 110 with the text information generated by the speech recognition unit 140. The generation unit 160 stores the generated training data in the training data storage unit 170, for example. However, the generation unit 160 may instead output the generated training data to an external device.

[0041] The training data generation device 10 may not include the acquisition unit 110 and the first model generation unit 120. In this case, the first speech recognition model 51, which has been trained in advance using synthetic speech, is stored in a storage device accessible by the speech recognition unit 140, and the speech recognition unit 140 can read and use the first speech recognition model 51.

[0042] The effect of generating the first speech recognition model 51 for each piece of speech data, as performed by the first model generation unit 120, is described below. The first speech recognition model 51 generated by the first model generation unit 120 as described above is considered to have particularly high recognition accuracy for the speech data acquired by the acquisition unit 110. In other words, the first speech recognition model 51 trained with synthetic speech based on input information associated with certain speech data k can be considered a model particularly suited to recognizing that speech data k. Such a first speech recognition model 51 is likely to be able to correctly recognize that speech data k. In other words, text information obtained by inputting speech data k into such a first speech recognition model 51 is likely to correctly represent the spoken content of the speech data k. Therefore, the text information can be suitably used as correct answer data for training data. Note that the first speech recognition model 51 may be deleted after generating correct answer data for that speech data k. A new first speech recognition model 51 may be generated for another piece of speech data k+1.

[0043] The training data generated by the training data generation device 10 is preferably used for training the second speech recognition model 52, but may also be used for training a speech recognition model other than the second speech recognition model 52.

[0044] FIG. 6 is a diagram illustrating the functional configuration of a speech recognition model generation device 20 according to this embodiment. The speech recognition model generation device 20 uses training data generated by the training data generation device 10 to train the second speech recognition model 52. As described above, the first speech recognition model 51 is expected to have higher recognition accuracy than the second speech recognition model 52. Therefore, by using training data in which text information generated by the first speech recognition model 51 is used as correct answer data, the speech recognition accuracy of the second speech recognition model 52 can be improved. Furthermore, since the second speech recognition model 52 can be trained using speech data that is not synthetic speech, it is possible to further improve recognition accuracy for actual speech.

[0045] In the example shown in the figure, the speech recognition model generation device 20 includes a second learning unit 220. The second learning unit 220 acquires the second speech recognition model 52 from the model storage unit 150 and acquires the training data generated by the training data generation device 10 from the training data storage unit 170. The second learning unit 220 then trains the second speech recognition model 52 using the acquired training data, thereby generating a second speech recognition model 52 with improved recognition accuracy. The speech recognition model generation device 20 may acquire the training data every time it is generated by the training data generation device 10, or may acquire the training data collectively after multiple pieces of training data are generated by the training data generation device 10 and stored in the training data storage unit 170.

[0046] The speech recognition model generation device 20 may update the second speech recognition model 52 stored in the model storage unit 150 with the trained second speech recognition model 52. The updated second speech recognition model 52 can be used again in the training data generation device 10 to generate the first speech recognition model 51.

[0047] The speech recognition model generation device 20 may be integrated with the training data generation device 10 or may be a device separate from the training data generation device 10 .

[0048] According to the speech recognition model generation device 20 of this embodiment, a speech recognition model generation method is executed in which one or more computers train the second speech recognition model 52 using the training data generated by the training data generation device 10.

[0049] The hardware configuration of the training data generation device 10 is described below. Each functional component of the training data generation device 10 may be realized by hardware that realizes the functional component (e.g., a hardwired electronic circuit, etc.), or by a combination of hardware and software (e.g., a combination of an electronic circuit and a program that controls it). Below, a case where each functional component of the training data generation device 10 is realized by a combination of hardware and software will be further described.

[0050] FIG. 7 is a diagram illustrating a computer 1000 for implementing the training data generation device 10. The computer 1000 may be any computer. For example, the computer 1000 may be a system on chip (SoC), a personal computer (PC), a server machine, a tablet terminal, or a smartphone. The computer 1000 may be a dedicated computer designed to implement the training data generation device 10, or may be a general-purpose computer. The training data generation device 10 may be implemented by a single computer 1000 or a combination of multiple computers 1000.

[0051] The computer 1000 includes a bus 1020, a processor 1040, a memory 1060, a storage device 1080, an input / output interface 1100, and a network interface 1120. The bus 1020 is a data transmission path through which the processor 1040, the memory 1060, the storage device 1080, the input / output interface 1100, and the network interface 1120 transmit and receive data to and from each other. However, the method of interconnecting the processor 1040 and other components is not limited to bus connection. The processor 1040 may be any of various processors, such as a central processing unit (CPU), a graphics processing unit (GPU), or a field-programmable gate array (FPGA). The memory 1060 is a main storage device implemented using a random access memory (RAM) or the like. The storage device 1080 is an auxiliary storage device implemented using a hard disk, a solid state drive (SSD), a memory card, a read-only memory (ROM), or the like.

[0052] The input / output interface 1100 is an interface for connecting the computer 1000 to an input / output device. For example, an input device such as a keyboard and an output device such as a display are connected to the input / output interface 1100. The input / output interface 1100 may be connected to the input device or output device via a wireless connection or a wired connection.

[0053] The network interface 1120 is an interface for connecting the computer 1000 to a network. This communication network is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). The network interface 1120 may be connected to the network via a wireless connection or a wired connection.

[0054] The storage device 1080 stores program modules that realize each functional component of the training data generation device 10. The processor 1040 reads each of these program modules into the memory 1060 and executes them to realize the function corresponding to each program module. Furthermore, when the fixed text storage unit 130, the model storage unit 150, and the training data storage unit 170 are each provided inside the training data generation device 10, the fixed text storage unit 130, the model storage unit 150, and the training data storage unit 170 are realized by the storage device 1080.

[0055] The hardware configuration of a computer that realizes the speech recognition model generation device 20 according to this embodiment is shown in Fig. 7, for example, similar to the training data generation device 10. However, a storage device 1080 of a computer 1000 that realizes the speech recognition model generation device 20 stores program modules that realize the functions of the speech recognition model generation device 20.

[0056] FIG. 8 is a diagram showing an overview of the training data generation method according to this embodiment. The training data generation method according to this embodiment is executed by one or more computers. The training data generation method according to this embodiment includes a speech recognition step S10 and a generation step S11. In the speech recognition step S10, text information is generated by inputting speech data into a trained first speech recognition model 51. In the generation step S11, training data including speech data and text information is generated. The first speech recognition model 51 is a model trained using synthetic speech generated using input information related to a predetermined item and standard text information prepared in advance.

[0057] FIG. 9 is a flowchart illustrating the flow of the training data generation method according to this embodiment. In the training data generation method according to this embodiment, the acquisition unit 110 acquires input information and speech data that are associated with each other (S100). Next, the synthetic speech text generation unit 121 generates a synthetic speech text using the input information and standard text information, and the synthetic speech generation unit 122 generates a synthetic speech using the synthetic speech text (S110). Next, the first training unit 123 trains the second speech recognition model 52 using the synthetic speech text and the synthetic speech to generate a first speech recognition model 51 (S120). Next, the speech recognition unit 140 inputs speech data into the first speech recognition model 51 to generate text information (S130). Then, the generation unit 160 generates training data that includes speech data and text information in an associated state (S140). The processes from S100 to S140 are performed, for example, for each piece of speech data.

[0058] As described above, according to this embodiment, the speech recognition unit 140 generates text information by inputting speech data into the trained first speech recognition model 51. The generation unit 160 generates training data including speech data and text information. The first speech recognition model 51 is a model trained using synthetic speech generated using input information and pre-prepared standard text information. Therefore, training data including speech data can be easily generated using the first speech recognition model 51, the accuracy of which has been improved using synthetic speech. Consequently, a speech recognition model with high recognition accuracy is realized.

[0059] Second Embodiment FIG. 10 is a diagram illustrating an overview of a training data generation device 10 according to a second embodiment. The training data generation device 10 according to this embodiment includes a speech recognition unit 140, a determination unit 180, and a generation unit 160. The speech recognition unit 140 inputs speech data to a trained first speech recognition model and a trained second speech recognition model, respectively, to generate output results for the first speech recognition model and the second speech recognition model. The determination unit 180 determines whether there is a difference between the output result of the first speech recognition model and the output result of the second speech recognition model. The generation unit 160 generates training data including speech data when the determination unit 180 determines that there is a difference between the output result of the first speech recognition model and the output result of the second speech recognition model.

[0060] According to the training data generation device 10 of this embodiment, the recognition accuracy of the voice recognition model can be improved.

[0061] A detailed example of the training data generation device 10 according to this embodiment will be described below. However, the training data generation device 10 according to this embodiment is not limited to the following example.

[0062] 11 is a diagram illustrating the functional configuration of a training data generation device 10 according to this embodiment. The training data generation device 10 according to this embodiment is the same as the training data generation device 10 according to the first embodiment, except for the points described below.

[0063] In this embodiment, when the first model generation unit 120 generates the first speech recognition model 51, the speech recognition unit 140 inputs the speech data acquired by the acquisition unit 110 to the generated first speech recognition model 51, as in the first embodiment. The speech recognition unit 140 also inputs the same speech data as that input to the first speech recognition model 51 to the second speech recognition model 52 read from the model storage unit 150. Then, text information, which is the output result, is obtained from each of the first speech recognition model 51 and the second speech recognition model 52.

[0064] However, the training data generation device 10 does not have to include the acquisition unit 110 and the first model generation unit 120. In this case, the speech recognition unit 140 reads and acquires the first speech recognition model 51 and the second speech recognition model 52 stored in advance in a storage device accessible from the speech recognition unit 140. However, the first speech recognition model 51 is a model with higher speech recognition accuracy than the second speech recognition model 52.

[0065] The determination unit 180 compares the text information that is the output result of the first speech recognition model 51 and the text information that is the output result of the second speech recognition model 52, both of which are generated by the speech recognition unit 140. For example, if the output result of the first speech recognition model 51 and the output result of the second speech recognition model 52 match, the determination unit 180 outputs determination result information to the generation unit 160, indicating that there is no need to generate training data. If the output result of the first speech recognition model 51 and the output result of the second speech recognition model 52 do not match, the determination unit 180 outputs determination result information to the generation unit 160, indicating that training data should be generated.

[0066] The generation unit 160 acquires determination result information from the determination unit 180. If the generation unit 160 acquires determination result information indicating that it is not necessary to generate training data, the generation unit 160 does not generate training data, and the training data generation device 10 ends processing of the speech data. If the generation unit 160 acquires determination result information indicating that training data should be generated, the generation unit 160 generates training data. Specifically, the generation unit 160 generates training data that associates the speech data acquired by the acquisition unit 110 with text information that is the output result of the first speech recognition model 51 generated by the speech recognition unit 140. The generation unit 160 stores the generated training data in the training data storage unit 170, for example. However, the generation unit 160 may instead output the generated training data to an external device.

[0067] According to the training data generation device 10 of this embodiment, training data including speech data is generated only when it is determined that there is a difference between the output result of the first speech recognition model 51 and the output result of the second speech recognition model 52. When the same speech data is input, if the output result of the first speech recognition model 51 and the output result of the second speech recognition model 52 are the same, it is highly likely that both speech recognition models produced correct output. In this case, further training using that speech data in the second speech recognition model 52 is not very effective.

[0068] On the other hand, as described above in the first embodiment, the first speech recognition model 51 is expected to have higher recognition accuracy than the second speech recognition model 52. Therefore, when the same speech data is input, if the output result of the first speech recognition model 51 and the output result of the second speech recognition model 52 differ, the output result of the first speech recognition model 51 is likely to be more accurate than the output result of the second speech recognition model 52. Therefore, it is preferable to generate training data using the output result of the first speech recognition model 51. By training using the generated training data, the recognition accuracy of the second speech recognition model 52 can be improved.

[0069] In this way, the determination unit 180 determines whether there is a difference between the output results of the first speech recognition model and the output results of the second speech recognition model, thereby generating training data that enables efficient learning.

[0070] The speech recognition model generation device 20 according to this embodiment is the same as the speech recognition model generation device 20 according to the first embodiment, except that training is performed on the second speech recognition model 52 using training data generated by the training data generation device 10 according to the second embodiment.

[0071] The hardware configuration of a computer that realizes the training data generation device 10 according to this embodiment is shown, for example, in Fig. 7, similar to the training data generation device 10 according to the first embodiment. The hardware configuration of a computer that realizes the speech recognition model generation device 20 according to this embodiment is shown, for example, in Fig. 7, similar to the speech recognition model generation device 20 according to the first embodiment. However, the storage device 1080 of the computer 1000 that realizes the training data generation device 10 according to this embodiment further stores a program module that realizes the determination unit 180 of the training data generation device 10 according to this embodiment.

[0072] FIG. 12 is a diagram illustrating an overview of a training data generation method according to this embodiment. The training data generation method according to this embodiment is executed by one or more computers. The training data generation method according to this embodiment includes a speech recognition step S20, a determination step S21, and a generation step S22. In the speech recognition step S20, speech data is input to each of a trained first speech recognition model and a trained second speech recognition model, thereby generating output results of the first speech recognition model and the second speech recognition model. In the determination step S21, it is determined whether there is a difference between the output result of the first speech recognition model and the output result of the second speech recognition model. In the generation step S22, if it is determined that there is a difference between the output result of the first speech recognition model and the output result of the second speech recognition model, training data including speech data is generated.

[0073] FIG. 13 is a flowchart illustrating the flow of the training data generation method according to this embodiment. The processes from S200 to S220 are the same as the processes from S100 to S120 in the first embodiment. In the training data generation method according to this embodiment, after S220, the speech recognition unit 140 inputs speech data to the first speech recognition model 51 and the second speech recognition model 52, respectively, to generate text information as the output results of each speech recognition model (S230). The determination unit 180 then determines whether there is a difference between the output results of the first speech recognition model 51 and the second speech recognition model 52 (S240). If it is determined in S240 that there is a difference (Yes in S240), the generation unit 160 generates training data including the speech data (S250). Then, processing related to the speech data ends. On the other hand, if it is determined in S240 that there is no difference (No in S240), no training data is generated and processing related to the speech data ends. The processes from S200 to S250 are performed for each piece of audio data, for example.

[0074] A modified example of the method in which the determining unit 180 determines whether there is a difference between the output result of the first speech recognition model 51 and the output result of the second speech recognition model 52 will be described below.

[0075] Instead of determining whether the output result of the first speech recognition model 51 and the output result of the second speech recognition model 52 match, the determination unit 180 may determine whether there is a difference between the output result of the first speech recognition model 51 and the output result of the second speech recognition model 52 based on whether the target word is included in each of the output results of the first speech recognition model 51 and the second speech recognition model 52. The target word is, for example, one or more of the contents of multiple items included in the input information. Preferably, the target word is all of the contents of multiple items included in the input information. The determination unit 180 can identify the target word using predetermined information indicating the items to be the target word and the input information acquired by the acquisition unit 110.

[0076] In this modification, if there is a difference in the recognition results of the target words, the determination unit 180 determines that there is a difference between the output result of the first speech recognition model 51 and the output result of the second speech recognition model 52. Specifically, the determination unit 180 detects one or more target words included in the text information that is the output result of the first speech recognition model 51. The determination unit 180 also detects one or more target words included in the text information that is the output result of the second speech recognition model 52. Then, if the one or more target words detected in the output result of the first speech recognition model 51 and the one or more target words detected in the output result of the second speech recognition model 52 all match, the determination unit 180 determines that there is no difference between the output result of the first speech recognition model 51 and the output result of the second speech recognition model 52. Then, if one or more target words detected in the output result of the first speech recognition model 51 do not match one or more target words detected in the output result of the second speech recognition model 52, it is determined that there is a difference between the output result of the first speech recognition model 51 and the output result of the second speech recognition model 52.

[0077] For example, suppose the target words are word A, word B, and word C. Here, if word A, word B, and word C are detected in the output result of first speech recognition model 51, and only word A and word B are detected in the output result of second speech recognition model 52, determination unit 180 determines that there is a difference between these output results.

[0078] As another example, when there are multiple target words, the determination unit 180 may determine whether there is a difference between the two output results by comparing the number of detected target words. That is, when the number of target words detected in the output result of the first speech recognition model 51 matches the number of target words detected in the output result of the second speech recognition model 52, the determination unit 180 determines that there is no difference between the output result of the first speech recognition model 51 and the output result of the second speech recognition model 52. On the other hand, when the number of target words detected in the output result of the first speech recognition model 51 does not match the number of target words detected in the output result of the second speech recognition model 52, the determination unit 180 determines that there is a difference between the output result of the first speech recognition model 51 and the output result of the second speech recognition model 52.

[0079] If at least one of the following (1) to (3) is true, the determination unit 180 may output determination result information indicating that it is not necessary to generate training data to the generation unit 160: (1) At least one target word is detected only in the output result of the second speech recognition model 52; (2) The number of target words detected in the output result of the second speech recognition model 52 is greater than the number of target words detected in the output result of the first speech recognition model 51; or (3) No target words are detected in both the output result of the first speech recognition model 51 and the output result of the second speech recognition model 52.

[0080] Next, the operation and effect of this embodiment will be described. In this embodiment, the same operation and effect as in the first embodiment can be obtained. In addition, the determination unit 180 determines whether there is a difference between the output result of the first speech recognition model and the output result of the second speech recognition model, thereby generating training data that enables efficient training.

[0081] Although the embodiments of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various other configurations can also be adopted.

[0082] In addition, although the flowcharts used in the above description show multiple steps (processes) in a sequential order, the order of steps executed in each embodiment is not limited to the order shown. In each embodiment, the order of the steps shown in the drawings can be changed as long as it does not cause any problems in terms of content. Furthermore, the above-described embodiments can be combined as long as the content is not contradictory.

[0083] Some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes. 1-1. A training data generation device comprising: speech recognition means for generating text information by inputting speech data into a trained first speech recognition model; and generation means for generating training data including the speech data and the text information, wherein the first speech recognition model is a model trained using synthetic speech generated using input information related to predetermined items and pre-prepared standard text information. 1-2. The training data generation device described in 1-1., further comprising acquisition means for acquiring the input information and the speech data that are associated with each other. 1-3. The training data generation device described in 1-2., wherein the acquisition means acquires a plurality of the speech data, and further comprising first model generation means for generating the first speech recognition model for each of the speech data. 1-4. 1-3. 2-1. The training data generation device described in 2-1., wherein the first model generation means generates the first speech recognition model by training the second speech recognition model using the synthetic speech. 2-1. The training data generation device comprises: speech recognition means that generates output results of the first speech recognition model and the second speech recognition model by inputting speech data into each of the trained first speech recognition model and the second speech recognition model; determination means that determines whether there is a difference between the output result of the first speech recognition model and the output result of the second speech recognition model; and generation means that generates training data including the speech data when the determination means determines that there is a difference between the output result of the first speech recognition model and the output result of the second speech recognition model. 2-2. The training data generation device described in 2-1., further comprising first model generation means that generates the first speech recognition model by training the second speech recognition model using synthetic speech. 2-3. 2-2. 2. The training data generation device according to claim 1, wherein the synthetic sound is a sound generated using input information relating to a predetermined item and standard text information prepared in advance.2-4. The training data generation device according to 2-2. or 2-3., further comprising an acquisition means for acquiring the input information and the speech data associated with each other. 2-5. The training data generation device according to 2-4., wherein the acquisition means acquires a plurality of pieces of speech data, and the first model generation means generates the first speech recognition model for each piece of speech data. 2-6. The training data generation device according to any one of 2-2. to 2-5., wherein the generation means generates training data including the output result of the first speech recognition model and the speech data when the determination means determines that there is a difference between the output result of the first speech recognition model and the output result of the second speech recognition model. 3-1. A speech recognition model generation device that trains the second speech recognition model using the training data generated by the training data generation device according to any one of 1-4. and 2-1. to 2-6. 4-1. A training data generation method in which one or more computers generate text information by inputting speech data into a trained first speech recognition model, and generate training data including the speech data and the text information, wherein the first speech recognition model is a model trained using synthetic speech generated using input information related to a predetermined item and pre-prepared standard text information. 4-2. The training data generation method described in 4-1, wherein the one or more computers further acquire the input information and the speech data that are associated with each other. 4-3. The training data generation method described in 4-2, wherein the one or more computers acquire a plurality of pieces of the speech data, and further generate the first speech recognition model for each piece of the speech data. 4-4. The training data generation method described in 4-3, wherein the one or more computers generate the first speech recognition model by training a second speech recognition model using the synthetic speech.5-1. A training data generation method in which one or more computers input speech data into a trained first speech recognition model and a trained second speech recognition model, respectively, to generate output results for the first speech recognition model and the second speech recognition model, determine whether there is a difference between the output result of the first speech recognition model and the output result of the second speech recognition model, and generate training data including the speech data if it is determined that there is a difference between the output result of the first speech recognition model and the output result of the second speech recognition model. 5-2. The training data generation method described in 5-1., wherein the one or more computers further generate the first speech recognition model by training the second speech recognition model using synthetic sound. 5-3. The training data generation method described in 5-2., wherein the synthetic sound is sound generated using input information related to a predetermined item and pre-prepared standard text information. 5-4. 5-2. or 5-3. 5-5. The training data generation method described in 5-4., wherein the one or more computers further acquire the input information and the speech data that are associated with each other. 5-5. The training data generation method described in 5-4., wherein the one or more computers acquire a plurality of pieces of speech data and generate the first speech recognition model for each piece of speech data. 5-6. The training data generation method described in any one of 5-2. to 5-5., wherein the one or more computers generate training data including the output result of the first speech recognition model and the speech data when it is determined that there is a difference between the output result of the first speech recognition model and the output result of the second speech recognition model. 6-1. A speech recognition model generation method in which one or more computers train the second speech recognition model using the training data generated by the training data generation method described in any one of 4-4. and 5-1. to 5-6.7-1. A program causing a computer to function as a training data generation device, wherein the training data generation device comprises: speech recognition means for generating text information by inputting speech data into a trained first speech recognition model; and generation means for generating training data including the speech data and the text information, wherein the first speech recognition model is a model trained using synthetic sound generated using input information related to a predetermined item and pre-prepared standard text information. 7-2. A program in accordance with 7-1., wherein the training data generation device further comprises acquisition means for acquiring the input information and the speech data associated with each other. 7-3. A program in accordance with 7-2., wherein the acquisition means acquires a plurality of pieces of the speech data, and the training data generation device further comprises first model generation means for generating the first speech recognition model for each piece of the speech data. 7-4. A program in accordance with 7-3., wherein the first model generation means generates the first speech recognition model by training a second speech recognition model using the synthetic sound. 8-1. 8-2. A program that causes a computer to function as a training data generation device, wherein the training data generation device comprises: speech recognition means that generates output results of a trained first speech recognition model and a trained second speech recognition model by inputting speech data into each of the first speech recognition model and the second speech recognition model; determination means that determines whether there is a difference between the output result of the first speech recognition model and the output result of the second speech recognition model; and generation means that generates training data including the speech data when the determination means determines that there is a difference between the output result of the first speech recognition model and the output result of the second speech recognition model. 8-2. A program in accordance with 8-1., wherein the training data generation device further comprises first model generation means that generates the first speech recognition model by training the second speech recognition model using synthetic speech.8-3. A program according to 8-2., wherein the synthetic sound is sound generated using input information related to a predetermined item and pre-prepared standard text information. 8-4. A program according to 8-2. or 8-3., wherein the training data generation device further comprises an acquisition means for acquiring the input information and the speech data associated with each other. 8-5. A program according to 8-4., wherein the acquisition means acquires a plurality of pieces of speech data, and the first model generation means generates the first speech recognition model for each piece of speech data. 8-6. A program according to any one of 8-2. to 8-5., wherein the generation means generates training data including the output result of the first speech recognition model and the speech data when the determination means determines that there is a difference between the output result of the first speech recognition model and the output result of the second speech recognition model. 9-1. A program causing a computer to function as a speech recognition model generation device, wherein the speech recognition model generation device is a program according to 7-4. and 8-1. 10-1. A computer-readable recording medium having a program recorded thereon, the program causing a computer to function as a training data generation device, the training data generation device comprising: a speech recognition means for generating text information by inputting speech data into a trained first speech recognition model; and a generation means for generating training data including the speech data and the text information, the first speech recognition model being a model trained using synthetic speech generated using input information related to a predetermined item and predetermined standard text information. 10-2. A recording medium according to 10-1, wherein the training data generation device further comprises an acquisition means for acquiring the input information and the speech data associated with each other.10-3. The recording medium described in 10-2., wherein the acquisition means acquires a plurality of pieces of the voice data, and the training data generation device further includes first model generation means for generating the first voice recognition model for each piece of the voice data. 10-4. The recording medium described in 10-3., wherein the first model generation means generates the first voice recognition model by training a second voice recognition model using the synthetic sound. 11-1. 11-2. A computer-readable recording medium having a program recorded thereon, the program causing a computer to function as a training data generation device, the training data generation device comprising: speech recognition means for inputting speech data into a trained first speech recognition model and a trained second speech recognition model, and generating output results of the first speech recognition model and the second speech recognition model, respectively; determination means for determining whether there is a difference between the output result of the first speech recognition model and the output result of the second speech recognition model; and generation means for generating training data including the speech data when the determination means determines that there is a difference between the output result of the first speech recognition model and the output result of the second speech recognition model. 11-2. A recording medium described in 11-1, wherein the training data generation device further comprises first model generation means for generating the first speech recognition model by training the second speech recognition model using synthetic speech. 11-3. A recording medium described in 11-2, wherein the synthetic speech is sound generated using input information related to a predetermined item and predetermined standard text information. 11-4. The recording medium described in 11-2. or 11-3., wherein the training data generation device further comprises an acquisition means for acquiring the input information and the speech data associated with each other. 11-5. The recording medium described in 11-4., wherein the acquisition means acquires a plurality of pieces of the speech data, and the first model generation means generates the first speech recognition model for each piece of the speech data.11-6. A recording medium according to any one of 11-2. to 11-5., wherein the generation means generates training data including the output result of the first speech recognition model and the speech data when the determination means determines that there is a difference between the output result of the first speech recognition model and the output result of the second speech recognition model. 12-1. A computer-readable recording medium having a program recorded thereon, the program causing a computer to function as a speech recognition model generation device, the speech recognition model generation device training the second speech recognition model using the training data generated by a training data generation device, which is realized by a program recorded on a recording medium according to any one of 10-4. and 11-1. to 11-6.

[0084] This application claims priority based on Japanese Patent Application No. 2022-107582, filed on July 4, 2022, the disclosure of which is incorporated herein by reference in its entirety.

[0085] 10 Training data generation device 20 Speech recognition model generation device 51 First speech recognition model 52 Second speech recognition model 110 Acquisition unit 120 First model generation unit 121 Synthetic speech text generation unit 122 Synthetic speech generation unit 123 First training unit 130 Fixed text storage unit 140 Speech recognition unit 150 Model storage unit 160 Generation unit 170 Training data storage unit 180 Determination unit 220 Second training unit 1000 Computer 1020 Bus 1040 Processor 1060 Memory 1080 Storage device 1100 Input / output interface 1120 Network interface

Claims

1. a speech recognition means for generating text information by inputting speech data to a trained first speech recognition model; generating means for generating training data including the voice data and the text information; The first speech recognition model is a model that is trained using synthetic speech generated using input information related to a predetermined item and standard text information prepared in advance. Training data generation device.

2. 2. The training data generating device according to claim 1, The apparatus further includes an acquisition unit for acquiring the input information and the voice data associated with each other. Training data generation device.

3. 3. The training data generating device according to claim 2, The acquiring means acquires a plurality of the voice data, The speech recognition system further includes a first model generating means for generating the first speech recognition model for each of the speech data. Training data generation device.

4. 4. The training data generating device according to claim 3, The first model generating means generates the first speech recognition model by training a second speech recognition model using the synthetic sound. Training data generation device.

5. The second speech recognition model is trained using the training data generated by the training data generation device according to claim 4. A voice recognition model generator.

6. One or more computers generating text information by inputting speech data into a trained first speech recognition model; generating training data including the voice data and the text information; The first speech recognition model is a model that is trained using synthetic speech generated using input information related to a predetermined item and standard text information prepared in advance. Training data generation method.

7. Making a computer function as a training data generating device, The learning data generation device includes: a speech recognition means for generating text information by inputting speech data to a trained first speech recognition model; generating means for generating training data including the voice data and the text information; The first speech recognition model is a model that is trained using synthetic speech generated using input information related to a predetermined item and standard text information prepared in advance. program.