Processing device, processing method, and recording medium

The processing device enhances speech recognition by determining important data pairs and generating additional learning data through specific audio processing techniques, addressing the inadequacy in reflecting important content in existing technologies and reducing user workload.

WO2024232429A1PCT designated stage expired Publication Date: 2024-11-14NEC CORP
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2024/017384
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-11
Filing Date
2024-05-10
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

Existing speech recognition technologies do not effectively differentiate and process important learning data from less important data, leading to inadequate reflection of content in learning processes.

Method used

A processing device and method that determines the importance of audio and text data pairs, generating additional learning data through processing techniques like speech rate conversion, voice quality conversion, and noise superimposition for important pairs, while maintaining the utterance content, thereby increasing the learning data for important pairs.

Benefits of technology

Enhances the learning process by increasing the volume of learning data for important pairs, improving the reflection of content in speech recognition engines and reducing user workload in data generation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2024017384_14112024_PF_FP_ABST
    Figure JP2024017384_14112024_PF_FP_ABST
Patent Text Reader

Abstract

The present invention provides a processing device comprising: a speech recognition unit that executes speech recognition processing on input speech data and outputs text data representing the spoken content of the input speech data; a determination unit that determines the importance, as training data, of each pair of the input speech data and the text data; and a generation unit that processes the input speech data about an important pair in which the importance satisfies a prescribed condition and that generates at least one piece of the processed speech data and generates additional training data in which the processed speech data and the text data are combined.
Need to check novelty before this filing date? Find Prior Art

Description

Processing device, processing method, and recording medium

[0001] The present invention relates to a processing device, a processing method, and a program.

[0002] A technology related to the present invention is disclosed in Patent Document 1. Patent Document 1 discloses a technology for generating training data for a voice recognition engine. Specifically, Patent Document 1 discloses a technology that uses training data (voice data) collected in the past and converted data (voice data) converted using the collected training data.

[0003] Japanese Patent Application Laid-Open No. 2022-108803

[0004] The importance of collected training data may vary depending on the training data. Therefore, it is desirable to perform training that better reflects the content of important training data. However, if all training data is uniformly treated in the same way, such training cannot be achieved. Patent Document 1 does not disclose this problem or a means for solving it.

[0005] In view of the above-mentioned problems, one example of the object of the present invention is to provide a processing device, a processing method, and a program that enable training of a speech recognition engine to better reflect the content of important training data among collected training data.

[0006] According to one aspect of the present invention, there is provided a processing device having: a speech recognition means for performing speech recognition processing on input speech data and outputting text data indicating the spoken content of the input speech data; a determination means for determining the importance as learning data for each pair of the input speech data and the text data; and a generation means for processing the input speech data of an important pair, which is a pair whose importance satisfies a predetermined condition, to generate at least one processed speech data, and for generating additional learning data by combining the processed speech data and the text data.

[0007] According to one aspect of the present invention, a processing method is provided in which one or more computers perform speech recognition processing on input speech data, output text data indicating the spoken content of the input speech data, determine the importance as learning data for each pair of the input speech data and the text data, process the input speech data of an important pair, which is a pair whose importance satisfies a predetermined condition, to generate at least one processed speech data, and generate additional learning data by combining the processed speech data and the text data.

[0008] According to one aspect of the present invention, there is provided a program that causes a computer to function as: a speech recognition means that performs speech recognition processing on input speech data and outputs text data indicating the spoken content of the input speech data; a determination means that determines the importance as learning data for each pair of the input speech data and the text data; and a generation means that processes the input speech data of an important pair, which is a pair whose importance satisfies a predetermined condition, to generate at least one processed speech data, and generates additional learning data by combining the processed speech data and the text data.

[0009] According to one aspect of the present invention, a processing device, processing method, and program are realized that enable training of a speech recognition engine to more accurately reflect the content of important training data among collected training data.

[0010] The above and other objects, features and advantages will become more apparent from the following preferred embodiments and the accompanying drawings.

[0011] FIG. 1 is a diagram showing an example of a functional block diagram of a processing device. FIG. 2 is a diagram for explaining an example of processing executed by the processing device. FIG. 3 is a diagram for explaining another example of processing executed by the processing device. FIG. 4 is a diagram showing an example of a hardware configuration of a processing device. FIG. 5 is a diagram showing an example of a screen output by the processing device. FIG. 6 is a flowchart showing an example of a processing flow of the processing device. FIG. 7 is a diagram showing an example of a functional block diagram of a processing device.

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, like components are designated by like reference numerals, and the description thereof will be omitted as appropriate.

[0013] 1 is a functional block diagram showing an overview of a processing device 10 according to a first embodiment. The processing device 10 includes a speech recognition unit 11, a determination unit 12, and a generation unit 13.

[0014] The voice recognition unit 11 executes a voice recognition process on the input voice data and outputs text data indicating the spoken content of the input voice data.

[0015] The determination unit 12 determines the importance as learning data (hereinafter, sometimes simply referred to as "importance") for each pair of the input voice data and the text data (hereinafter, sometimes simply referred to as "pair").

[0016] The generation unit 13 processes input speech data of an important pair, which is a pair whose importance level satisfies a predetermined condition, to generate at least one piece of processed speech data, and generates additional training data by combining the processed speech data with the text data.

[0017] In this way, the processing device 10 according to the first embodiment generates processed speech data from input speech data for important pairs that are important as training data, and generates additional training data that includes each of the generated processed speech data. As a result, the amount of training data for important pairs increases by the amount of additional training data. In this way, the processing device 10 according to the first embodiment increases the amount of training data for important pairs that are important as training data, thereby enabling training that better reflects the content of the important training data.

[0018] Second Embodiment "Overview"

[0019] The processing device 10 of the second embodiment is a specific embodiment of the processing device 10 of the first embodiment.

[0020] That is, the processing device 10 performs a speech recognition process on the input speech data and outputs text data indicating the spoken content of the input speech data. Next, the processing device 10 determines the importance of each pair of input speech data and text data as learning data.

[0021] The processing device 10 then performs a learning data generation process for each pair according to the importance. For important pairs whose importance satisfies a predetermined condition (a condition indicating importance), the processing device 10 performs the process shown in Fig. 2. On the other hand, for unimportant pairs whose importance does not satisfy the predetermined condition (a condition indicating importance), the processing device 10 performs the process shown in Fig. 3.

[0022] An important pair is shown in Fig. 2. The important pair includes input speech data and text data generated by executing speech recognition processing on the input speech data.

[0023] The processing device 10 processes the input speech data of the important pair to generate at least one piece of processed speech data. The processing data is generated by at least one of speech rate conversion processing, voice quality conversion processing, noise superposition processing, and sound quality conversion processing. Therefore, the speech content of the processed speech data is the same as the speech content of the input speech data.

[0024] As shown in the figure, the processing device 10 generates training data by combining the input speech data and text data of the important pairs, and also generates additional training data by combining the text data of the important pairs with at least one piece of processed speech data.

[0025] An unimportant pair is shown in Fig. 3. The unimportant pair includes input speech data and text data generated by executing speech recognition processing on the input speech data.

[0026] The processing device 10 generates training data by combining input speech data and text data for unimportant pairs. The processing device 10 does not generate processed speech data or additional training data including processed speech data for unimportant pairs.

[0027] In this way, for important pairs, the processing device 10 generates "learning data combining the input speech data and text data of the important pair" and "additional training data combining the text data of the important pair with at least one piece of processed speech data." For unimportant pairs, the processing device 10 generates "learning data combining the input speech data and text data of the unimportant pair."

[0028] According to the processing device 10, the training data for important pairs can be made larger than the training data for unimportant pairs by the amount of additional training data. By training the speech recognition engine with such training data, training that better reflects the content of the training data for important pairs can be achieved.

[0029] The configuration of the processing device 10 will be described in detail below.

[0030] "Hardware Configuration" An example of the hardware configuration of the processing device 10 will be described. Each functional unit of the processing device 10 is realized by any combination of hardware and software. Those skilled in the art will understand that there are various variations in the realization method and device. Software includes programs that are pre-loaded in the device before shipping, and programs downloaded from recording media such as CDs (Compact Discs) or servers on the Internet.

[0031] FIG. 4 is a block diagram illustrating an example of the hardware configuration of a processing device 10. As shown in FIG. 4, the processing device 10 has a processor 1A, a memory 2A, an input / output interface 3A, a peripheral circuit 4A, and a bus 5A. The peripheral circuit 4A includes various modules. The processing device 10 does not necessarily have to have the peripheral circuit 4A. Note that the processing device 10 may be composed of multiple devices that are physically and / or logically separated. In this case, each of the multiple devices may have the above hardware configuration.

[0032] The bus 5A is a data transmission path for the processor 1A, memory 2A, peripheral circuit 4A, and input / output interface 3A to mutually transmit and receive data. The processor 1A is, for example, a processing unit such as a CPU or a graphics processing unit (GPU). The memory 2A is, for example, a random access memory (RAM) or a read-only memory (ROM). The input / output interface 3A includes interfaces for acquiring information from input devices, external devices, external servers, external sensors, cameras, etc., and interfaces for outputting information to output devices, external devices, external servers, etc. The input / output interface 3A also includes an interface for connecting to a communication network such as the Internet. Examples of input devices include a keyboard, mouse, microphone, physical buttons, touch panel, etc. Examples of output devices include a display, speaker, printer, mailer, etc. The processor 1A can issue commands to each module and perform calculations based on the results of those calculations.

[0033] "Functional Configuration" Next, the functional configuration of the processing device 10 of the second embodiment will be described in detail. Fig. 1 shows an example of a functional block diagram of the processing device 10 of the second embodiment. As shown in the figure, the processing device 10 of the second embodiment has a speech recognition unit 11, a determination unit 12, and a generation unit 13.

[0034] The voice recognition unit 11 executes a voice recognition process on the input voice data, and outputs utterance content text data based on the result of the voice recognition process.

[0035] "Input voice data" is voice data that indicates the content of a person's speech. The voice recognition unit 11 acquires multiple pieces of input voice data. The length of one piece of input voice data is, for example, about several seconds, but is not limited to this. The lengths of the multiple pieces of input voice data may all be the same or may be different from each other.

[0036] The speech recognition unit 11 can acquire input speech data by any means. "Acquisition" includes at least one of the following: the device itself retrieving data or information stored in another device or storage medium (active acquisition); and the device itself inputting data or information output from another device (passive acquisition). Examples of active acquisition include making a request to another device and receiving a response, and accessing and reading from another device or storage medium. Examples of passive acquisition include receiving information that is distributed (or transmitted, push notification, etc.). Furthermore, "acquisition" may also mean selecting and acquiring data or information from received data or information, or selecting and receiving distributed data or information.

[0037] The speech recognition unit 11 performs speech recognition processing on input speech data using a pre-prepared speech recognition engine. The speech recognition engine is generated by learning based on training data that combines speech data with text data that indicates the spoken content of the speech data. When speech data is input to the speech recognition engine, text data that indicates the spoken content of the speech data is output. There are no particular restrictions on the configuration of the speech recognition engine, and any technology can be used. The training data generated by the processing device 10 can be used to retrain the speech recognition engine.

[0038] The "utterance content text data" is text data indicating the utterance content of the input voice data. The voice recognition unit 11 may output the recognition result (text data) output from the voice recognition engine as the utterance content text data as is.

[0039] Additionally, the speech recognition unit 11 may execute a correction process on the recognition result (text data) output from the speech recognition engine, and may output the text data after correcting the recognition result (text data) output from the speech recognition engine as speech content text data.

[0040] An example of the correction process will now be described. The speech recognition unit 11 outputs the recognition result (text data) output from the speech recognition engine to the user. This output is realized via an output device such as a display or a projection device. The speech recognition unit 11 then accepts user input to correct the output recognition result (text data). The user specifies a portion of the output text data that needs to be corrected and inputs the correct content for the specified portion. Receiving feedback from the user regarding such a recognition result can be realized using any widely known technology.

[0041] The determination unit 12 determines the importance of each pair of input speech data and utterance content text data as learning data. One pair is composed of one piece of input speech data and utterance content text data indicating the utterance content of the input speech data.

[0042] "Importance" is indicated by any index. For example, importance may be indicated by two values, "important" and "unimportant." Alternatively, importance may be indicated by four values ​​from 0 to 3. For example, the larger the value, the higher the importance. Note that the examples here are merely examples, and the index indicating importance is not limited to these.

[0043] Next, the process of determining the importance will be described. The determination unit 12 receives a user input specifying the importance of each pair as learning data. Then, the determination unit 12 determines the importance specified by the user input as the importance of each pair.

[0044] Here, an example of a UI (user interface) screen that accepts a user input specifying the importance level will be described. Note that this UI screen is merely an example and is not limited to this.

[0045] The determination unit 12 outputs a UI screen, such as that shown in Fig. 5, to the user. The output is realized via an output device such as a display or a projection device. The determination unit 12 can then accept user input specifying the importance of each pair via the UI screen.

[0046] The UI screen shown in FIG. 5 includes columns for before editing, after editing, and importance.

[0047] The "After Editing" column displays the spoken content text data output from the speech recognition unit 11. That is, if the recognition result (text data) output from the speech recognition engine is corrected based on user input, the corrected text data is displayed. On the other hand, if the recognition result (text data) output from the speech recognition engine is not corrected based on user input, the recognition result output from the speech recognition engine is displayed. Each piece of text data is linked to the date and time it was spoken. The date and time it was spoken is identified, for example, based on the time code or timestamp of the input speech data. In the figure, one pair of spoken content text data is displayed per row.

[0048] The "Before Editing" column displays the recognition results (text data) output from the voice recognition engine. Each piece of text data is linked to the date and time it was spoken. The date and time it was spoken is determined, for example, based on the time code or timestamp of the input voice data.

[0049] The "Importance" column displays the importance level specified by the user. The number of blackened stars indicates the importance level. In this example, the importance level is indicated by four values ​​from 0 to 3. The higher the value, the higher the importance level. When the number of blackened stars is 3, the importance level is 3. When the number of blackened stars is 0, the importance level is 0. Also, when no stars are displayed, the importance level is 0.

[0050] The determination unit 12 displays three stars in the importance column of the illustrated UI screen only for pairs that have been corrected based on user input. The determination unit 12 then accepts user input to fill in or white out these stars. Note that the determination unit 12 does not display three stars in the importance column of the illustrated UI screen for text data that has not been corrected based on user input. The determination unit 12 does not accept user input specifying the importance of such text data. The determination unit 12 determines the importance of such text data to be 0.

[0051] The UI screen shown in the figure displays a list of data for multiple pairs and allows the importance of multiple pairs to be specified collectively. As another example, the UI screen may be configured to display data for each pair and accept input to specify the importance.

[0052] In this way, the determination unit 12 can display the results of the speech recognition process and the correction content based on the user input, and can accept a user input specifying the importance via a UI screen that accepts the user input specifying the importance. Note that other processes for determining the importance will be described in the following embodiments.

[0053] Returning to FIG. 1 , for an important pair, which is a pair whose importance level satisfies a predetermined condition, the generation unit 13 generates learning data for the important pair using a first method. The first method is the method described above using FIG. 2 . Then, for an unimportant pair, which is a pair whose importance level does not satisfy the predetermined condition, the generation unit 13 generates learning data for the unimportant pair using a second method. The second method is the method described above using FIG. 3 .

[0054] The "predetermined condition" is a condition for determining whether each pair is important, and is defined in advance using the importance level. For example, if the importance level is represented by two values, "important" and "unimportant," the predetermined condition is "importance is 'important'." Alternatively, if the importance level is represented by four values ​​from 0 to 3 (the higher the value, the more important), the predetermined condition is "importance is M or higher," where M is one of 1, 2, and 3.

[0055] Next, a first method and a second method for generating training data will be described.

[0056] 2, the generation unit 13 processes the input speech data of the important pair to generate at least one piece of processed speech data. Then, the generation unit 13 generates training data by combining the input speech data of the important pair with utterance content text data (denoted as "text data" in the figure). The generation unit 13 also generates additional training data by combining the utterance content text data of the important pair with at least one piece of processed speech data.

[0057] The processed voice data is generated by at least one of speech rate conversion processing, voice quality conversion processing, noise superimposition processing, and sound quality conversion processing.

[0058] The "speech speed conversion process" is a process for speeding up or slowing down the speech speed.

[0059] "Voice conversion processing" is a process for converting the voice quality of a speaker. For example, the voice quality of input voice data can be converted to a voice quality corresponding to a different gender, age, etc.

[0060] "Noise superimposition processing" is processing in which any noise is superimposed on input voice data.

[0061] "Sound quality conversion processing" is processing for converting the sound quality of audio. For example, the sound quality of input audio data can be converted to that of telephone audio, microphone audio, etc.

[0062] The speech content of the processed voice data generated by such processing is the same as the speech content of the input voice data.

[0063] - Second Method - The generation unit 13 generates training data by combining input speech data of unimportant pairs and utterance content text data (denoted as "text data" in the figure), as shown in Fig. 3. Note that the generation unit 13 does not generate processed speech data or additional training data including processed speech data for unimportant pairs.

[0064] The generation unit 13 can transmit the generated training data to a training server. The training server is a server that trains the above-mentioned speech recognition engine. The generation unit 13 may transmit training data generated by real-time processing to the training server. Alternatively, the generation unit 13 may transmit training data generated by batch processing to the training server.

[0065] Alternatively, the generation unit 13 may store the generated learning data in a predetermined storage device. The storage device may be provided within the processing device 10 or may be provided in an external device configured to be able to communicate with the processing device 10.

[0066] Next, an example of the processing flow of the processing device 10 will be described with reference to the flowcharts of Figures 6 and 7. The processing shown in Figure 6 is executed by the speech recognition unit 11. The processing shown in Figure 7 is executed by the determination unit 12 and the generation unit 13.

[0067] First, the flow of processing shown in FIG. 6 will be described.

[0068] The processing device 10 is in a state of waiting for acquisition of input voice data (S10). When the processing device 10 acquires the input voice data (Yes in S10), it executes a voice recognition process on the input voice data (S11) and outputs the recognition result to the user (S12). The user inputs an input to correct the output recognition result or to confirm it without correction.

[0069] If an input for correcting the output recognition result is received (Yes in S13), the processing device 10 corrects the recognition result based on the input (S14).Then, the processing device 10 outputs the corrected recognition result as utterance content text data indicating the utterance content of the input voice data (S15).

[0070] On the other hand, if no input to correct the output recognition result is received (No in S13), the processing device 10 outputs the output recognition result as speech content text data indicating the speech content of the input voice data (S15).

[0071] Next, the flow of the process shown in FIG. 7 will be described.

[0072] The processing device 10 is in a state of waiting for acquisition of a pair of input speech data and utterance content text data (S20). When the processing device 10 acquires the pair of input speech data and utterance content text data (Yes in S20), it determines the importance of the pair as learning data (S21).

[0073] For example, the processing device 10 may receive a user input specifying the importance of the pair. Then, the processing device 10 may determine the importance of the pair to be the importance specified by the user input. The input specifying the importance of each pair may be executed in advance, and the result may be stored in the processing device 10. Then, in S21, the processing device 10 may read out the importance of the pair to be processed from the importance of each pair stored in advance. Alternatively, in S21, the processing device 10 may receive an input specifying the importance of the pair to be processed from the user.

[0074] If the importance level satisfies the predetermined condition (Yes in S22), the processing device 10 processes the input voice data to generate at least one piece of processed voice data (S23).Then, the processing device 10 transmits training data, which is a combination of the input voice data and the at least one piece of processed voice data, and the utterance content text data, to the training server (S24).

[0075] On the other hand, if the importance does not satisfy the predetermined condition (No in S22), the processing device 10 transmits learning data that combines the input voice data and the utterance content text data to the learning server (S25).

[0076] In S24 and S25, the processing device 10 may store the learning data in a predetermined storage device instead of transmitting the learning data to the learning server.

[0077] "Effects" The processing device 10 of the second embodiment determines the importance as training data for each pair of input speech data and utterance content text data. Then, for important pairs that are important as training data, the processing device 10 generates processed speech data from the input speech data and performs a process of generating additional training data including each of the generated processed speech data. On the other hand, for unimportant pairs that are not important as training data, the processing device 10 does not generate processed speech data or additional training data including the generated processed speech data. As a result, the amount of training data for important pairs is greater than the amount of training data for unimportant pairs by the amount of the additional training data.

[0078] In this way, the processing device 10 according to the second embodiment increases the number of training data items related to important pairs that are important as training data, thereby enabling training that better reflects the content of the important training data. Furthermore, the processing device 10 according to the second embodiment increases the number of training data items related to important pairs by processing the speech data, thereby reducing the workload on the user for increasing the training data.

[0079] Furthermore, the processing device 10 of the second embodiment can generate processed voice data using at least one of speech rate conversion processing, voice quality conversion processing, noise superposition processing, and sound quality conversion processing. In this case, the speech content of the processed voice data is the same as the speech content of the input voice data. Therefore, the speech content text data indicating the speech content of the input voice data can be used as is as text data to be combined with each processed voice data in generating training data. In this way, according to the processing device 10 of the second embodiment, multiple additional training data can be generated using the speech content text data as is, thereby efficiently increasing the training data.

[0080] <Third Embodiment> The processing device 10 of the third embodiment determines the importance as learning data for each pair of input speech data and utterance content text data by a method not described in the first and second embodiments. Specifically, the processing device 10 of the third embodiment determines the importance based on the content of corrections made to the recognition result output from the speech recognition engine. This will be described in detail below.

[0081] The determination unit 12 determines the importance based on the content of corrections made to the recognition result output from the speech recognition engine. "Contents of corrections made to the recognition result output from the speech recognition engine" refers to whether or not a correction based on a user input has been made, and if so, the specific content of the correction. As described in the second embodiment, the speech recognition unit 11 can make corrections based on a user input to the recognition result (text data) output from the speech recognition engine.

[0082] The determining unit 12 determines the importance of each pair based on the value of at least one of the following items 1 to 9.

[0083] (Item 1) Whether or not a user input to correct the recognition result output from the voice recognition engine has been received.

[0084] In this case, the item value is, for example, P corresponding to "a user input to correct the recognition result output from the voice recognition engine has been received." 1 and P corresponding to "user input to correct the recognition result output from the speech recognition engine is not accepted" 2 It is. 1 and P 2 is, for example, a predetermined numerical value.

[0085] The determination unit 12 sets the importance of the pair that has received the user input higher than the importance of the pair that has not received the user input. That is, the importance of the pair that has received the user input to correct the recognition result output from the speech recognition engine is higher than the importance of the pair that has not received the user input.

[0086] (Item 2) The number of characters corrected in the user input.

[0087] In this case, the item value is, for example, the number of characters. The determining unit 12 assigns a higher importance to an item whose number of characters has been corrected by user input.

[0088] (Item 3) Whether or not the word corrected by user input is included in the training data that has already been trained.

[0089] In this case, the item value is, for example, Q corresponding to "The word corrected by the user input is included in the learned training data." 1 and the Q corresponding to "words corrected by user input are not included in the training data" 2 Other item values ​​include, for example, Q corresponding to "All words corrected by user input are included in the training data" 1 and Q corresponding to "at least one of the words corrected by the user input is not included in the training data" 2 It may be. 1 and Q 2 is, for example, a predetermined numerical value.

[0090] If the word corrected by the user input is not included in the learned learning data, the determination unit 12 increases the importance of the word.

[0091] When this item is adopted, the determination unit 12 performs morphological analysis on the utterance content text data to extract words included in the utterance content text data. Then, the determination unit 12 determines, on a word-by-word basis, whether or not the utterance content text data has been corrected by user input, and extracts the corrected words.

[0092] Thereafter, the determination unit 12 determines whether the extracted corrected word is included in the learned training data. For example, the learned training data is stored in a storage device of the processing device 10 or an external device configured to be able to communicate with the processing device 10. The determination unit 12 determines whether the extracted corrected word is included in the utterance content text data of the learned training data stored in the storage device.

[0093] As another example, a database of words included in the utterance content text data of the training data that has already been trained may be generated in advance, and the determination unit 12 may then determine whether or not the extracted corrected word is included in this database.

[0094] (Item 4) The number of trained training data sets that contain words modified by user input.

[0095] In this case, the item value is, for example, the number of pieces of training data that have been trained and contain words that have been corrected by user input.

[0096] The determination unit 12 assigns a higher importance to a word that has been corrected by user input when the number of pieces of learned training data that contain the word is smaller.

[0097] The determination unit 12 can make this determination using a method similar to the method described in item 3. For example, a database of words included in the utterance content text data of the learned training data is generated in advance. In this database, the number of learned training data items including each word is registered in association with each word. The determination unit 12 determines whether the extracted corrected word is included in this database. If the extracted corrected word is included in the database, the determination unit 12 reads out the "number of learned training data items" registered in association with the word.

[0098] (Item 5) The number of words corrected by user input that are not included in the training data.

[0099] In this case, the item value is, for example, the number of words that have been corrected by user input and that are not included in the learned training data.

[0100] When correcting the recognition result output from the speech recognition engine, multiple words may be corrected. Therefore, the determination unit 12 determines the importance based on the number of words corrected by the user input that are not included in the learned training data.

[0101] The determining unit 12 assigns a higher importance to words that are corrected by user input and that are not included in the learned learning data, as the number of such words increases.

[0102] The determination unit 12 can determine, for each word corrected by user input, whether or not the word is included in the learned training data, using the same method as described in Item 3. Then, based on the determination result, the determination unit 12 can count the number of words corrected by user input that are not included in the learned training data.

[0103] (Item 6) Whether the word corrected by user input is included in the additional training data that has already been trained.

[0104] In this case, the item value is, for example, R corresponding to "The word corrected by the user input is included in the additional training data that has already been trained." 1and R corresponding to "words corrected by user input are not included in the additional training data already learned." 2 In addition, the item value is, for example, R corresponding to "All words corrected by user input are included in the additional training data that has already been trained." 1 and R corresponding to "at least one of the words corrected by the user input is not included in the additional training data already trained." 2 It may be R 1 and R 2 is, for example, a predetermined numerical value.

[0105] The content of the above-mentioned item 3 is whether or not the word corrected by the user input is included in the "trained training data." In contrast, the content of item 6 is whether or not the word corrected by the user input is included in the "trained additional training data." The "trained training data" is a concept that partly includes the "trained additional training data."

[0106] The additional training data is training data generated in association with the important pairs generated by the generating unit 13, as described in the second embodiment.

[0107] For example, the trained additional training data is stored in a storage device of the processing device 10 or an external device configured to be able to communicate with the processing device 10. The determination unit 12 determines whether the utterance content text data of the trained additional training data stored in this storage device includes a word corrected by user input.

[0108] As another example, a database of words included in the utterance content text data of the additional training data that has already been trained may be generated in advance, and the determination unit 12 may then determine whether or not the database includes the word corrected by the user input.

[0109] (Item 7) The number of additional training data sets that have been trained and contain words that have been modified by user input.

[0110] In this case, the item value is, for example, the number of pieces of additional training data that have been trained and contain words that have been corrected by user input.

[0111] The determination unit 12 assigns a higher importance to a word corrected by user input when the number of pieces of learned additional training data that contain the word is smaller.

[0112] The determination unit 12 can make this determination using a method similar to the method described in item 6. For example, a database of words included in the utterance content text data of the trained additional training data is generated in advance. In this database, the number of trained additional training data that include each word is registered in association with each word. The determination unit 12 determines whether or not this database includes a word that has been corrected by user input. If the database includes a word that has been corrected by user input, the determination unit 12 reads out the "number of trained additional training data" registered in association with the word.

[0113] (Item 8) The number of words corrected by user input that are not included in the additional training data that has already been trained.

[0114] In this case, the item value is, for example, the number of words that have been corrected by user input and that are not included in the additional training data that has already been trained.

[0115] When correcting the recognition result output from the speech recognition engine, multiple words may be corrected. Therefore, the determination unit 12 determines the importance level based on the number of words corrected by the user input that are not included in the additional training data that has already been learned.

[0116] The determination unit 12 assigns a higher importance to words that are corrected by user input and that are not included in the additional training data that has already been learned.

[0117] The determination unit 12 can determine, for each word corrected by user input, whether or not the word is included in the already-learned additional training data, using a method similar to that described in Item 6. Then, based on the determination result, the determination unit 12 can count the number of words corrected by user input that are not included in the already-learned additional training data.

[0118] (Item 9) Whether the word corrected by the user input is a predetermined noun.

[0119] In this case, the item value is, for example, S corresponding to "The word corrected by the user input is a predetermined noun." 1 and S corresponding to "the word corrected by the user input is not a predetermined noun" 2 In addition, the item value is, for example, S corresponding to "at least one of the words corrected by the user input is a predetermined noun." 1 and S corresponding to "all of the words corrected by the user input are not the specified nouns" 2 It may be: S 1 and S 2 is, for example, a predetermined numerical value.

[0120] The "predetermined noun" is an important noun designated by the user. The user can set proper nouns such as place names, personal names, or other arbitrary nouns such as technical terms as the predetermined noun.

[0121] If the word corrected by the user input is a predetermined noun, the determination unit 12 assigns a higher importance to it than if it is not a predetermined noun.

[0122] The determination unit 12 can determine whether the word corrected by the user's input is a proper noun by using a well-known technique such as morphological analysis. Alternatively, dictionary data in which predetermined nouns are registered may be generated in advance and stored in the processing device 10. Then, the determination unit 12 may determine whether the word corrected by the user's input is registered in the dictionary data.

[0123] The determination unit 12 can determine the importance of each pair based on the item value of at least one of the above items 1 to 9. An importance calculation model that inputs at least one item value of the above items 1 to 9 and outputs an importance is generated in advance and stored in the processing device 10. The determination unit 12 determines the importance using the importance calculation model. The importance calculation model is configured using a function, a table, or the like. One possible example is a model that calculates the sum of the products of the item values ​​of each of multiple items and the weighting coefficients for each item as a score, and determines which score ranges correspond to each of multiple importance levels this score falls within. Note that this example is merely an example and is not limiting.

[0124] The user may be able to set a weight for each of the above items 1 to 9. The importance calculation model may then be modified based on the weight of each item set by the user. In the case of the model exemplified above, the weight coefficient of each item is modified based on the weight of each item set by the user.

[0125] Other configurations of the processing apparatus 10 of the third embodiment are similar to those of the processing apparatus 10 of the first and second embodiments.

[0126] The processing device 10 of the third embodiment achieves the same effects as the processing devices 10 of the first and second embodiments. Furthermore, the processing device 10 of the third embodiment can automatically determine the importance of a pair of input voice data and utterance content text data as learning data. As a result, the user does not need to specify the importance for each pair, reducing the user's workload.

[0127] Furthermore, according to the processing device 10 of the third embodiment, it is possible to determine the importance based on the content of corrections made to the recognition result output from the speech recognition engine. Specifically, it is possible to determine the importance based on at least one of the above items 1 to 9. As a result, it is possible to accurately determine the importance of the training data.

[0128] <Fourth embodiment> The processing device 10 of the fourth embodiment determines the importance as learning data for each pair of input voice data and utterance content text data by a method not described in the first to third embodiments. Specifically, the processing device 10 of the fourth embodiment determines the importance based on at least one of whether learning has been performed in the past and the number of times learning has been performed in the past. This will be described in detail below.

[0129] The determining unit 12 determines the importance level based on at least one of whether learning has been performed in the past and the number of times learning has been performed in the past.

[0130] The determining unit 12 determines the importance of each pair based on the value of at least one of the following items 10 and 11.

[0131] (Item 10) Whether or not it has been studied in the past.

[0132] In this case, the item value is, for example, T corresponding to "has been learned in the past." 1 and T corresponding to "not previously learned" 2 It is. T 1 and T 2 is a predetermined numerical value.

[0133] The determination unit 12 can determine whether a pair has been previously trained by comparing the utterance content text data of the pair to be determined with the utterance content text data of the trained training data. There are various conditions for determining whether a pair has been trained, and any of the following may be used, for example.

[0134] - There is trained training data that has utterance content text data that completely matches the utterance content text data of the pair to be judged. - There is trained training data that has utterance content text data that includes a predetermined percentage or more of the words included in the utterance content text data of the pair to be judged. - There is trained training data that has utterance content text data that includes words that have been corrected by the user in the utterance content text data of the pair to be judged. - There is trained additional training data that has utterance content text data that completely matches the utterance content text data of the pair to be judged. - There is trained additional training data that has utterance content text data that includes a predetermined percentage or more of the words included in the utterance content text data of the pair to be judged. - There is trained additional training data that has utterance content text data that includes words that have been corrected by the user in the utterance content text data of the pair to be judged.

[0135] (Item 11) Number of times of previous learning.

[0136] In this case, the item value is, for example, the number of times of past learning. For example, the determination unit 12 compares the utterance content text data of the pair to be determined with the utterance content text data of the learned learning data, and extracts the learned learning data that satisfies a predetermined condition with the utterance content text data of the pair to be determined. Then, the determination unit 12 can set the number of extracted learned learning data as the item value.

[0137] The determination unit 12 can set, for example, any of the following numbers as the item value:

[0138] - The number of trained training data that has utterance content text data that completely matches the utterance content text data of the pair to be determined. - The number of trained training data that has utterance content text data that includes a predetermined percentage or more of the words included in the utterance content text data of the pair to be determined. - The number of trained training data that has utterance content text data that includes words corrected by the user in the utterance content text data of the pair to be determined. - The number of trained additional training data that has utterance content text data that completely matches the utterance content text data of the pair to be determined. - The number of trained additional training data that has utterance content text data that includes a predetermined percentage or more of the words included in the utterance content text data of the pair to be determined. - The number of trained additional training data that has utterance content text data that includes words corrected by the user in the utterance content text data of the pair to be determined.

[0139] The determination unit 12 can determine the importance of each pair based on the item value of at least one of the items 10 and 11. An importance calculation model that inputs the item value of at least one of the items 10 and 11 and outputs the importance is generated in advance and stored in the processing device 10. The determination unit 12 determines the importance using the importance calculation model. The importance calculation model is configured using a function, a table, or the like. One possible example is a model that calculates the sum of the products of the item values ​​of each of a plurality of items and the weighting coefficients for each item as a score, and determines which score ranges correspond to each of a plurality of importance levels the score falls within. Note that this example is merely an example and is not limiting.

[0140] The user may be able to set the weights for each of the items 10 and 11. The importance calculation model may then be modified based on the weights for each item set by the user. In the case of the model exemplified above, the weight coefficients for each item are modified based on the weights for each item set by the user.

[0141] Furthermore, the determination unit 12 may determine the importance of each pair using the same method as described above, based on the value of at least one of the items 1 to 11. Items 1 to 9 are the items described in the third embodiment.

[0142] Other configurations of the processing apparatus 10 of the fourth embodiment are similar to those of the processing apparatus 10 of the first to third embodiments.

[0143] The processing device 10 of the fourth embodiment achieves the same effects as the processing devices 10 of the first to third embodiments. Furthermore, the processing device 10 of the fourth embodiment can automatically determine the importance of a pair of input voice data and utterance content text data as learning data. As a result, the user does not need to specify the importance for each pair, reducing the user's workload.

[0144] Furthermore, according to the processing device 10 of the fourth embodiment, the importance can be determined based on whether or not learning has been performed in the past and the number of times. Specifically, the importance can be determined based on at least one of the above items 10 and 11. As a result, the importance of the learning data can be determined with high accuracy.

[0145] Fifth Embodiment A processing device 10 of a fifth embodiment determines the importance of each pair of input speech data and utterance content text data as learning data by using a method not described in the first to fourth embodiments. Specifically, the processing device 10 of the fifth embodiment determines the importance based on at least one of whether the utterance content text data includes a predetermined noun and the number of predetermined nouns included. This will be described in detail below.

[0146] The determination unit 12 determines the importance level based on at least one of whether the utterance content text data includes a predetermined noun and the number of predetermined nouns included in the utterance content text data.

[0147] The determining unit 12 determines the importance of each pair based on the value of at least one of the following items 12 and 13.

[0148] (Item 12) Whether or not the utterance content text data includes a predetermined noun.

[0149] In this case, the item value is, for example, U corresponding to "the utterance content text data includes a predetermined noun." 1 and U corresponding to "the utterance content text data does not contain a predetermined noun" 2 It is. 1 and U 2 is a predetermined numerical value.

[0150] The "predetermined noun" is an important noun designated by the user. The user can set proper nouns such as place names, personal names, or other arbitrary nouns such as technical terms as the predetermined noun.

[0151] The determination unit 12 extracts nouns from the utterance content text data using a well-known technique such as morphological analysis, and determines whether the extracted nouns are proper nouns. Alternatively, dictionary data in which predetermined nouns are registered may be generated in advance and stored in the processing device 10. The determination unit 12 may then determine whether the noun extracted from the utterance content text data is registered in the dictionary data.

[0152] (Item 13) The number of predetermined nouns included in the utterance content text data.

[0153] In this case, the item value is, for example, the number of predetermined nouns included in the utterance content text data. The determination unit 12 can count the number of predetermined nouns included in the utterance content text data using the method described in Item 12.

[0154] The determination unit 12 can determine the importance for each pair based on the item value of at least one of the items 12 and 13. An importance calculation model that inputs the item value of at least one of the items 12 and 13 and outputs the importance is generated in advance and stored in the processing device 10. The determination unit 12 determines the importance using the importance calculation model. The importance calculation model is configured using a function, a table, or the like. One example is a model that calculates the sum of the products of the item values ​​of each of a plurality of items and the weighting coefficients for each item as a score, and determines which score ranges correspond to each of a plurality of importance levels the score falls within. Note that this example is merely an example and is not limiting.

[0155] The user may be able to set the weights for each of the items 12 and 13. The importance calculation model may then be modified based on the weights for each item set by the user. In the case of the model exemplified above, the weight coefficients for each item are modified based on the weights for each item set by the user.

[0156] The determination unit 12 may also determine the importance of each pair using a method similar to the method described above, based on the value of at least one of the items 1 to 13. Items 1 to 9 are the items described in the third embodiment. Items 10 and 11 are the items described in the fourth embodiment.

[0157] Other configurations of the processing apparatus 10 of the fifth embodiment are similar to those of the processing apparatus 10 of the first to fourth embodiments.

[0158] The processing device 10 of the fifth embodiment achieves the same effects as the processing devices 10 of the first to fourth embodiments. Furthermore, the processing device 10 of the fifth embodiment can automatically determine the importance of a pair of input voice data and utterance content text data as learning data. As a result, the user does not need to specify the importance for each pair, thereby reducing the user's workload.

[0159] Furthermore, according to the processing device 10 of the fifth embodiment, the importance of the utterance content text data can be determined based on at least one of whether the utterance content text data contains a predetermined noun and the number of predetermined nouns it contains. Specifically, the importance can be determined based on at least one of the above items 12 and 13. As a result, the importance of the utterance content text data can be determined with high accuracy.

[0160] Sixth Embodiment The processing device 10 of the sixth embodiment differs from the first to fifth embodiments in that it includes a means for training a speech recognition engine based on training data generated by the generation unit 13. In the first to fifth embodiments, an external device (a training server) different from the processing device 10 includes a means for training a speech recognition engine based on training data generated by the generation unit 13.

[0161] 8 shows an example of a functional block diagram of the processing device 10 according to the sixth embodiment. As shown in the figure, the processing device 10 according to the sixth embodiment includes a speech recognition unit 11, a determination unit 12, a generation unit 13, and a learning unit 14.

[0162] The learning unit 14 learns the speech recognition engine based on the learning data generated by the generation unit 13. The speech recognition engine to be learned is, for example, the speech recognition engine used in the speech recognition process by the speech recognition unit 11. The learning unit 14 can learn the speech recognition engine based on any widely known technology.

[0163] Other configurations of the processing apparatus 10 of the sixth embodiment are similar to those of the processing apparatus 10 of the first to fifth embodiments.

[0164] According to the processing apparatus 10 of the sixth embodiment, the same effects as those of the processing apparatuses 10 of the first to fifth embodiments are achieved.

[0165] <Modifications> Modifications applicable to the processing apparatus 10 of the first to sixth embodiments will be described.

[0166] "Variation 1" The generation unit 13 can generate additional training data in a number corresponding to the importance. The higher the importance, the more additional training data the generation unit 13 generates. The number of additional training data to be generated for each importance is determined in advance. Then, the generation unit 13 generates the number of additional training data corresponding to the importance in accordance with the rule.

[0167] According to this modification, the same effects as those of the processing device 10 of the first to sixth embodiments are achieved. Furthermore, according to this modification, it is possible to generate more learning data of important pairs with higher importance. As a result, it becomes possible to perform learning that better reflects the contents of important pairs with higher importance.

[0168] "Variation 2" In the first to sixth embodiments, the generation unit 13 generates training data for unimportant pairs by combining input speech data and utterance content text data of the unimportant pairs using the method described with reference to FIG. 3.

[0169] As a variation of this process, the generation unit 13 may not execute the process of generating training data for unimportant pairs, i.e., the generation unit 13 may not generate training data that combines input speech data and utterance content text data for unimportant pairs.

[0170] According to this modification, the same effects as those of the processing apparatus 10 of the first to sixth embodiments are realized.

[0171] <Usage Scenarios> The processing devices 10 according to the first to sixth embodiments can be used, for example, in call centers, police departments, and the like that accept telephone reports. For example, audio data recording the contents of a telephone call is input to the processing device 10 as input audio data. An operator accepting the call checks the results of the speech recognition processing output from the processing device 10 and makes corrections, etc. as necessary. The operator also saves the text data of the speech content after corrections, etc., as a record of the contents of the telephone call.

[0172] The processing device 10 can generate training data for training a voice recognition engine used in such situations, for example.

[0173] Although the embodiments of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various other configurations may be adopted. The configurations of the above-described embodiments may be combined with each other, or some of the configurations may be replaced with other configurations. Furthermore, various modifications may be made to the configurations of the above-described embodiments without departing from the spirit of the invention. Furthermore, the configurations and processes disclosed in the above-described embodiments and modified examples may be combined with each other.

[0174] In addition, in the flowcharts used in the above description, multiple steps (processes) are described in order. However, the order of the steps performed in each embodiment is not limited to the order described. In each embodiment, the order of the steps shown in the drawings can be changed to the extent that the content is not affected. Furthermore, the above-described embodiments can be combined to the extent that the content is not contradictory.

[0175] Some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes: 1. A processing device comprising: speech recognition means that executes speech recognition processing on input speech data and outputs text data indicating the spoken content of the input speech data; determination means that determine the importance as learning data for each pair of the input speech data and the text data; and generation means that processes the input speech data of an important pair, where the importance satisfies a predetermined condition, to generate at least one piece of processed speech data, and generates additional learning data that combines the processed speech data and the text data. 2. The processing device according to 1, wherein the generation means does not generate the processed speech data or the additional learning data including the processed speech data for an unimportant pair, where the importance does not satisfy the predetermined condition. 3. 3. The processing device according to 1 or 2, wherein the speech recognition means accepts a user input to correct a result of the speech recognition processing, and the determination means determines the importance based on at least one of: whether the user input has been accepted, the number of characters corrected by the user input, whether the word corrected by the user input is included in learned training data, the number of learned training data including the word corrected by the user input, the number of words corrected by the user input that are not included in the learned training data, whether the word corrected by the user input is included in the learned additional training data, the number of learned additional training data including the word corrected by the user input, the number of words corrected by the user input that are not included in the learned additional training data, and whether the word corrected by the user input is a predetermined noun. 4. The processing device according to any of 1 to 3, wherein the determination means determines the importance based on at least one of whether the word has been previously learned and the number of times it has been learned. 5. The processing device according to any one of 1 to 4, wherein the determination means determines the importance based on at least one of whether the text data includes a predetermined noun and the number of the predetermined nouns included in the text data.6. The processing device according to any one of 1 to 5, wherein the determination means receives a user input specifying the importance as training data for each pair. 7. The processing device according to 6, wherein the speech recognition means receives a user input to correct a result of the speech recognition processing, and the determination means receives a user input specifying the importance via a UI screen that displays the result of the speech recognition processing and the correction content and receives a user input specifying the importance. 8. The processing device according to any one of 1 to 7, wherein the generation means performs at least one of a speech rate conversion process, a voice quality conversion process, a noise superimposition process, and a sound quality conversion process on the input speech data to generate the processed speech data. 9. A processing method in which one or more computers perform a speech recognition process on the input speech data and output text data indicating the speech content of the input speech data, determine the importance as training data for each pair of the input speech data and the text data, process the input speech data of an important pair, which is the pair whose importance satisfies a predetermined condition, to generate at least one processed speech data, and generate additional training data by combining the processed speech data and the text data. 10. The processing method according to 9, wherein the one or more computers do not generate the processed speech data or the additional training data including the processed speech data for a non-important pair, which is a pair whose importance does not satisfy the predetermined condition.11. The processing method described in 9 or 10, wherein the one or more computers accept a user input that corrects the result of the speech recognition processing, and determine the importance based on at least one of the following: whether the user input has been accepted, the number of characters corrected by the user input, whether the word corrected by the user input is included in learned training data, the number of learned training data including the word corrected by the user input, the number of words corrected by the user input that are not included in the learned training data, whether the word corrected by the user input is included in the learned additional training data, the number of learned additional training data including the word corrected by the user input, the number of words corrected by the user input that are not included in the learned additional training data, and whether the word corrected by the user input is a predetermined noun. 12. The processing method described in any of 9 to 11, wherein the one or more computers determine the importance based on at least one of whether the word has been previously learned and the number of times it has been learned. 13. The processing method according to any one of 9 to 12, wherein the one or more computers determine the importance based on at least one of whether the text data includes a predetermined noun and the number of the predetermined nouns included in the text data. 14. The processing method according to any one of 9 to 13, wherein the one or more computers accept a user input specifying the importance as training data for each pair. 15. The processing method according to 14, wherein the one or more computers accept a user input to correct a result of the speech recognition processing, and accept the user input specifying the importance via a UI screen that displays the result of the speech recognition processing and the correction content and accepts the user input specifying the importance. 16. The processing method according to any one of 9 to 15, wherein the one or more computers perform at least one of a speech rate conversion process, a voice quality conversion process, a noise superimposition process, and a sound quality conversion process on the input speech data to generate the processed speech data.17. A program causing a computer to function as: speech recognition means that executes speech recognition processing on input speech data and outputs text data indicating the spoken content of the input speech data; determination means that determines the importance as learning data for each pair of the input speech data and the text data; and generation means that processes the input speech data of an important pair, which is the pair whose importance satisfies a predetermined condition, to generate at least one piece of processed speech data, and generates additional learning data that combines the processed speech data and the text data. 18. The program according to 17, wherein the generation means does not generate the processed speech data or the additional learning data including the processed speech data for an unimportant pair, which is the pair whose importance does not satisfy the predetermined condition. 19. 20. The program according to 17 or 18, wherein the speech recognition means accepts a user input to correct a result of the speech recognition processing, and the determination means determines the importance based on at least one of: whether the user input has been accepted, the number of characters corrected by the user input, whether the word corrected by the user input is included in learned training data, the number of learned training data including the word corrected by the user input, the number of words corrected by the user input that are not included in the learned training data, whether the word corrected by the user input is included in the learned additional training data, the number of learned additional training data including the word corrected by the user input, the number of words corrected by the user input that are not included in the learned additional training data, and whether the word corrected by the user input is a predetermined noun. 21. The program according to any of 17 to 19, wherein the determination means determines the importance based on at least one of whether the word has been previously learned and the number of times it has been learned. 21. The program according to any one of 17 to 20, wherein the determination means determines the importance based on at least one of whether the text data includes a predetermined noun and the number of the predetermined nouns included in the text data.22. The program according to any one of 17 to 21, wherein the determination means receives a user input specifying the importance as learning data for each pair. 23. The program according to 22, wherein the speech recognition means receives a user input to correct a result of the speech recognition processing, and the determination means receives a user input specifying the importance via a UI screen that displays the result of the speech recognition processing and the correction content and receives a user input specifying the importance. 24. The program according to any one of 17 to 23, wherein the generation means performs at least one of speech rate conversion processing, voice quality conversion processing, noise superimposition processing, and sound quality conversion processing on the input speech data to generate the processed speech data.

[0176] This application claims priority based on Japanese Patent Application No. 2023-078497, filed May 11, 2023, the disclosure of which is incorporated herein in its entirety by reference.

[0177] 10 Processing device 11 Speech recognition unit 12 Determination unit 13 Generation unit 14 Learning unit 1A Processor 2A Memory 3A Input / output I / F 4A Peripheral circuit 5A Bus

Claims

1. A processing device having: a voice recognition means for executing a voice recognition process on input voice data and outputting text data indicating the spoken content of the input voice data; a determination means for determining the importance as learning data for each pair of the input voice data and the text data; and a generation means for processing the input voice data of an important pair, which is a pair whose importance satisfies a predetermined condition, to generate at least one processed voice data, and to generate additional learning data by combining the processed voice data and the text data.

2. The processing device according to claim 1, wherein the generation means does not generate the processed voice data or the additional learning data including the processed voice data for non-important pairs, which are pairs whose importance does not satisfy the specified condition.

3. The processing device of claim 1 or 2, wherein the voice recognition means accepts user input to modify the results of the voice recognition processing, and the determination means determines the importance based on at least one of the following: whether or not the user input has been accepted; the number of characters corrected by the user input; whether or not the word corrected by the user input is included in the learned learning data; the number of learned learning data including the word corrected by the user input; the number of words corrected by the user input that are not included in the learned learning data; whether or not the word corrected by the user input is included in the learned additional learning data; the number of learned additional learning data including the word corrected by the user input; the number of words corrected by the user input that are not included in the learned additional learning data; and whether or not the word corrected by the user input is a specified noun.

4. A processing device according to any one of claims 1 to 3, wherein the determining means determines the importance based on at least one of whether or not learning has been performed in the past and the number of times learning has been performed in the past.

5. A processing device according to any one of claims 1 to 4, wherein the determination means determines the importance based on at least one of whether or not the text data includes a specified noun and the number of the specified nouns included in the text data.

6. The processing device according to claim 1, wherein the determination means receives a user input specifying the importance as learning data for each of the pairs.

7. The processing device according to claim 6, wherein the voice recognition means accepts user input to modify the results of the voice recognition processing, and the determination means accepts user input specifying the importance via a UI (user interface) screen that displays the results of the voice recognition processing and the contents of the modifications and accepts user input specifying the importance.

8. A processing device according to any one of claims 1 to 7, wherein the generation means performs at least one of speech speed conversion processing, voice quality conversion processing, noise superposition processing, and sound quality conversion processing on the input voice data to generate the processed voice data.

9. A processing method in which one or more computers perform a speech recognition process on input speech data, output text data indicating the spoken content of the input speech data, determine the importance as learning data for each pair of the input speech data and the text data, process the input speech data of an important pair, which is a pair whose importance satisfies a predetermined condition, to generate at least one processed speech data, and generate additional learning data by combining the processed speech data and the text data.

10. The processing method described in claim 9, wherein the one or more computers do not generate the processed voice data or the additional learning data including the processed voice data for non-important pairs, which are pairs whose importance does not satisfy the specified condition.

11. The processing method of claim 9 or 10, wherein the one or more computers receive user input to modify the results of the speech recognition processing, and determine the importance based on at least one of the following: whether or not the user input has been accepted; the number of characters modified by the user input; whether or not the word modified by the user input is included in the learned learning data; the number of learned learning data including the word modified by the user input; the number of words modified by the user input that are not included in the learned learning data; whether or not the word modified by the user input is included in the learned additional learning data; the number of learned additional learning data including the word modified by the user input; the number of words modified by the user input that are not included in the learned additional learning data; and whether or not the word modified by the user input is a specified noun.

12. A processing method according to any one of claims 9 to 11, wherein the one or more computers determine the importance based on at least one of whether or not the information has been learned in the past and the number of times the information has been learned in the past.

13. A processing method described in any one of claims 9 to 12, wherein the one or more computers determine the importance based on at least one of whether or not the text data contains a specified noun and the number of the specified nouns contained in the text data.

14. The processing method according to any one of claims 9 to 13, wherein the one or more computers accept a user input that specifies the importance as learning data for each of the pairs.

15. A recording medium having recorded thereon a program that causes a computer to function as: a voice recognition means that executes voice recognition processing on input voice data and outputs text data indicating the spoken content of the input voice data; a determination means that determines the importance as learning data for each pair of the input voice data and the text data; and a generation means that processes the input voice data of an important pair, which is a pair whose importance satisfies a predetermined condition, to generate at least one processed voice data, and generates additional learning data by combining the processed voice data and the text data.

16. The recording medium according to claim 15, wherein the generation means does not generate the processed voice data or the additional learning data including the processed voice data for non-important pairs, which are pairs whose importance does not satisfy the specified condition.

17. The recording medium of claim 15 or 16, wherein the voice recognition means accepts user input to modify the result of the voice recognition processing, and the determination means determines the importance based on at least one of the following: whether or not the user input has been accepted; the number of characters corrected by the user input; whether or not the word corrected by the user input is included in the learned learning data; the number of learned learning data including the word corrected by the user input; the number of words corrected by the user input that are not included in the learned learning data; whether or not the word corrected by the user input is included in the learned additional learning data; the number of learned additional learning data including the word corrected by the user input; the number of words corrected by the user input that are not included in the learned additional learning data; and whether or not the word corrected by the user input is a specified noun.

18. A recording medium according to any one of claims 15 to 17, wherein the determining means determines the importance based on at least one of whether or not learning has been performed in the past and the number of times learning has been performed in the past.

19. A recording medium according to any one of claims 15 to 18, wherein the determination means determines the importance based on at least one of whether or not the text data includes a specified noun and the number of the specified nouns included in the text data.

20. A recording medium according to any one of claims 15 to 19, wherein the determination means receives a user input specifying the importance as learning data for each of the pairs.

Citation Information

Patent Citations

  • Program, apparatus and method for estimating evaluation level with respect to learning item on the basis of person's remark

    JP2016085284A

  • Semantic generation method, semantic generation device, and program

    JP2017228272A

  • Systems and methods evaluating user audio profiles for continuous speech recognition

    US10395640B1

  • Methods and apparatus for enhancing speech analytics

    US20090292541A1

  • Voice recognition system and voice recognition method

    US20220319521A1