Voice recognition system, voice recognition method and program

Abstract

To provide a voice recognition system, a voice recognition method and a program which improve voice recognition accuracy, by generating teacher data suitable for voice recognition of spoken words.SOLUTION: In a voice recognition system, a cloud server 3 consisting of a voice recognition server 10 and a data management server 30, a data acquisition device 50, a controller 60, a voice output device 70 and a communication terminal 90 communicate with each other through a communication network 5. The voice recognition server comprises: a voice recognition engine 12 which performs voice recognition process with respect to a user's utterance; an area extraction unit 15 which extracts a specific area included in first text data which is transcription of the user's utterance; and a text conversion unit 16 which converts the first text data into second text data, on the basis of the extracted specific area. The voice recognition engine 12 uses the converted second text data for machine learning.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present disclosure relates to a voice recognition system, a voice recognition method, and a program. [Background technology]

[0002] Smart speakers and multilingual translation systems using voice recognition are attracting attention. A smart speaker is a device that allows you to operate indoor appliances and provide information such as weather forecasts by using voice commands. A multilingual translation system is a translation device that uses a smartphone, a dedicated terminal, or the like. In a multilingual translation system, for example, a person's voice is input into a microphone, the input voice is converted into text by voice recognition, and then translated into a desired language by a translation process and output from a speaker. Furthermore, systems that generate records of conversations with customers at call centers and systems that automatically generate meeting records are being put into practical use, and these systems also utilize voice recognition technology.

[0003] In addition, in such a system, a method of performing machine learning for speech recognition using speech data synthesized from text is known as a method for improving the accuracy of speech recognition. For example, Patent Document 1 discloses the contents of an acoustic model adaptation process using training speech data with a synchronization signal added and recorded speech data. Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional methods use synthetic speech made from natural language (literary style) that is suitable for speech recognition, and there was a problem in that no improvement was seen when performing speech recognition on synthetic speech made from text that is transcribed from so-called spoken language (colloquial style). [Means for solving the problem]

[0005] In order to solve the above-mentioned problems, the invention according to claim 1 is a voice recognition system that performs voice recognition processing on a user's speech using a voice recognition engine, comprising: a region extraction means that extracts a specific region included in first text data that is a transcription of the speech; and a text conversion means that converts the first text data into second text data based on the extracted specific region. a synthetic voice generating means for generating synthetic voice using the second text data; The converted second text data and the synthesized voice. This is a voice recognition system that uses the above for machine learning of the voice recognition engine. Effect of the Invention

[0006] According to the present invention, by generating training data suitable for speech recognition of spoken language, it is possible to improve the accuracy of speech recognition. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram illustrating an example of an overall configuration of a voice recognition system. [Diagram 2] FIG. 1 is a diagram for explaining an example of a usage scene of a voice recognition system. [Diagram 3] FIG. 2 is a diagram illustrating an example of an external view of a data acquisition device. [Figure 4] FIG. 2 is a diagram illustrating an example of a hardware configuration of a voice recognition server and a data management server. [Diagram 5] FIG. 1A is a diagram showing an example of the hardware configuration of a data acquisition device, and FIG. 1B is a diagram showing an example of the hardware configuration of a sound output device. [Figure 6] FIG. 2 is a diagram illustrating an example of a hardware configuration of a control device and a communication terminal. [Figure 7] 2 is a diagram illustrating an example of a functional configuration of a voice recognition server, a data management server, and a communication terminal. [Figure 8] 4 is a diagram for explaining an example of the operation of a voice recognition engine, a lip reading processor, and an integrator. FIG. [Figure 9]4 is a diagram for explaining an example of the operation of a voice recognition engine, a lip reading processor, and an integrator. FIG. [Figure 10] FIG. 1 is a diagram for explaining an example of image features used in machine lip reading. [Figure 11] FIG. 2 is a diagram illustrating an example of a functional configuration of a data acquisition device, a control device, and an audio output device. [Figure 12] FIG. 11 is a sequence diagram showing an example of a process for generating teacher data used in machine learning of a voice recognition engine. [Figure 13] FIG. 2 is a diagram showing an example of text data that is a transcription of a user's speech. [Figure 14] FIG. 11 is a diagram showing an example of converted text data. [Figure 15] 10 is a sequence diagram showing an example of a process for generating training data used in machine learning of a speech recognition engine using acoustic characteristic conversion; FIG. [Figure 16] FIG. 11 is a sequence diagram showing an example of a process for generating training data used in machine learning of a voice recognition engine using playback recording. [Figure 17] 1A and 1B are diagrams illustrating an example of a covered audio output device. [Figure 18] FIG. 13 is a diagram for explaining an example of playback and recording using a covered audio output device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the description of the drawings, the same elements are given the same reference numerals, and duplicated description will be omitted.

[0009] ●Embodiment● Overview of the voice recognition system First, an outline of the configuration of a voice recognition system according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the overall configuration of a voice recognition system. The voice recognition system 1 shown in Fig. 1 is a system for achieving high recognition accuracy in a new meeting place where, for example, users wear masks or the like to prevent droplet infection and where partitions such as acrylic panels are installed in meeting rooms.

[0010] As shown in FIG. 1, the voice recognition system 1 includes a cloud server 3 including a voice recognition server 10 and a data management server 30, a data acquisition device 50, a control device 60, a voice output device 70, and a communication terminal 90. The cloud server 3, the control device 60, and the communication terminal 90 constituting the voice recognition system 1 can communicate with each other via a communication network 5. The communication network 5 is constructed by the Internet, a mobile communication network, a LAN (Local Area Network), and the like. The communication network 5 may include not only wired communication but also wireless communication networks such as 3G (3rd Generation), 4G (4th Generation), 5G (5th Generation), Wi-Fi (Wireless Fidelity) (registered trademark), WiMAX (Worldwide Interoperability for Microwave Access), or LTE (Long Term Evolution).

[0011] The voice recognition server 10 is a server computer for performing voice recognition processing using a voice recognition engine on the voice data acquired by the data acquisition device 50. The data management server 30 is a server computer for managing teacher data used for machine learning of the voice recognition engine. When playing back and recording voice data, the data management server 30 transmits recording data to the control device 60 to control the playing and recording. The voice recognition server 10 and the data management server 30 may be configured by a single server computer.

[0012] The communication terminal 90 is a computer such as a notebook PC used by an annotator that performs annotation of unused voice data for machine learning of a voice recognition engine. The annotator performs transcription and time stamp processing for performing supervised learning as machine learning of the voice recognition engine.

[0013] The data acquisition device 50, the control device 60, and the audio output device 70 are used when playing back and recording audio data of a conference or the like. The data acquisition device 50 and the audio output device 70 can communicate with the cloud server 3 via the control device 60. The data acquisition device 50 acquires audio data of the audio uttered by the user, and image data of the user who uttered the audio or the user's surroundings. The audio output device 70 plays back the audio data and outputs the audio when playing back and recording. The control device 60 controls the processing or operation of the data acquisition device 50 and the audio output device 70.

[0014] The data acquisition device 50 is used not only for playing back and recording recorded data, but also for real-time voice recognition processing in, for example, a conference. The control device 60 may have a configuration in which some or all of its functions are provided in the data acquisition device 50. That is, the data acquisition device 50 may be configured to be able to directly communicate with the cloud server 3 via the communication network 5.

[0015] Fig. 2 is a diagram for explaining an example of a usage scene of the voice recognition system 1. Fig. 2 shows an example of a conference attended by multiple users as an example of a usage scene of the voice recognition system 1. Fig. 2 shows a table 810 installed in a conference room 800, multiple (conference attendees 81 to 86) around the table 810, a data acquisition device 50, and a cloud server 3.

[0016] The voice recognition system 1 is configured to improve the voice recognition accuracy by acquiring the voices of one or more conference attendees 81-86 with a microphone and using voice data, which is data indicating the contents of the acquired voices, for machine learning. The voice recognition system 1 is also configured to improve the voice recognition accuracy by capturing images of one or more conference attendees 81-86 with a camera and using imaging data, which is data indicating the contents of the captured images, for machine learning. Note that the voice recognition system 1 may be configured to collect only voice data to improve the voice recognition accuracy. However, the voice recognition system 1 can further improve the voice recognition accuracy by collecting imaging data in addition to the voice data.

[0017] The voice recognition system 1 includes a data acquisition device 50 installed in the center of a table 810, a whiteboard 830 installed between the wall of a conference room 800 and the table 810, and a cloud server 3. Voice data acquired by the data acquisition device 50 is transmitted to the cloud server 3 via the whiteboard 830, and voice recognition processing is performed by a voice recognition engine or the like implemented in the cloud server 3. Text data obtained as a result of the voice recognition is transmitted to the whiteboard 830, and subtitles are displayed on the whiteboard 830. In addition, the text data is used to summarize the contents of the speech as minutes. Note that a technology for automatically converting statements made at a meeting, lecture, interview, or the like into text by voice recognition processing, or a technology for creating minutes, is well known, and therefore a detailed description thereof will be omitted.

[0018] The data acquisition device 50 is a device that acquires the voices of the multiple conference attendees 81-86 present around the table 810. Note that the data acquisition device 50 is configured to acquire images of the multiple conference attendees 81-86 or the surroundings of the conference attendees 81-86 in addition to the voices.

[0019] There is a lot of training data on near-field speech where the user (speaker) is close to the microphone (for example, within 1 m), and a method of using speech synthesized from text for machine learning of speech recognition is already known. However, the text that is a transcript of near-field speech is often natural text (literary style) suitable for speech recognition. On the other hand, although the demand for speech recognition of speech in meetings and the like is increasing, speech in meetings and the like often uses complex sentences or colloquial spoken language, and for example, when a conventional transcript of near-field speech is used for speech recognition of spoken language, the speech recognition accuracy is reduced. In addition, even if speech data of spoken language can be obtained, the burden of subsequent annotation is imposed. As such, since the conventional speech recognition technology is mainly for near-field speech recognition (near-field speech recognition), there is a problem that it is difficult to apply it to machine learning of far-field speech recognition (far-field speech recognition) where the user (speaker) is at a distance of a certain distance or more (for example, 1 m or more) from the microphone.

[0020] Therefore, the speech recognition system 1 generates training data having characteristics of spoken language and distant speech, thereby improving the accuracy of speech recognition by machine learning using training data that is close to a conference scene, even though it is synthetic speech. The speech recognition system 1 performs text conversion processing on areas of the transcription of the user's speech that include overlapping speech and the like that are difficult to use for machine learning, thereby decomposing and reconstructing the transcription into single-utterance text that can be applied to machine learning. The speech recognition system 1 decomposes the transcription of the user's speech into a combination of single utterances, but retains the characteristics of spoken language, making it suitable for learning speech recognition of spoken language.

[0021] Furthermore, the voice recognition system 1 creates synthetic voice using the text of the single utterance, converts the created synthetic voice into a voice that sounds like a distant speech, and uses the converted text data and voice data for machine learning in the voice recognition engine 12. In this way, the voice recognition system 1 can improve the voice recognition accuracy in usage scenarios such as conferences by generating training data suitable for speech recognition of spoken language and distant speech. Furthermore, the voice recognition system 1 generates training data by synthetic voice using a transcription of the utterance, thereby reducing the burden required for annotation.

[0022] As shown in FIG. 2, the meeting attendees 81 to 86 are wearing coverings such as masks and face shields. Furthermore, partitions are installed in the spaces between the meeting attendees. This is an example of a meeting where measures to prevent infection with the novel coronavirus that spread in 2020 have been taken, and it is expected that such meetings with measures to prevent infection will become commonplace in the future. The voice recognition system 1 can generate training data adapted to new usage scenarios such as wearing a mask in order to accommodate usage scenarios where wearing a mask, which greatly affects the performance of voice recognition, has become commonplace.

[0023] Fig. 3 is a diagram showing an example of an external view of a data acquisition device. Fig. 3 shows the external appearance of data acquisition device 50, as well as the scenery of conference room 800 captured by data acquisition device 50. Data acquisition device 50 includes housing 40, microphone 400 which is an audio detection unit, and camera 410 which is an imaging unit. A multi-microphone capable of capturing audio of multiple channels is used as microphone 400. A multi-camera capable of capturing images of multiple channels is used as camera 410. The multi-camera is, for example, a combination of multiple imaging units each with an angle of view of 90° or more.

[0024] The housing 40 includes a disk-shaped base 510a that is placed on the table 810, and a columnar extension 510b that extends vertically from the base 510a and places a plurality of microphones 400 and the like at a position away from the table 810. The housing 40 also includes a disk-shaped unit installation section 510c that is provided on the upper portion of the extension 510b and in which a plurality of multi-microphones and a multi-camera are arranged. The shape of the housing 40 is not limited to the illustrated example as long as it is a structure that can accommodate at least one camera 410 and one microphone 400.

[0025] Among the multiple microphones 400, one microphone 400 is provided on the upper part of the unit installation section 510c. The remaining microphones 400 are provided at a location other than the upper part of the unit installation section 510c, for example, on the side part of the unit installation section 510c. The side part is, for example, a part including a virtual plane parallel to a horizontal plane perpendicular to the vertical direction, among the entire outer periphery of the unit installation section 510c. On the side part of the unit installation section 510c, the multiple microphones 400 are installed so as to be spaced apart from each other in the circumferential direction. In this way, by installing the multiple microphones 400, the data acquisition device 50 is arranged so that each microphone 400 faces each conference attendee even when multiple conference attendees 81 to 86 are present around the table 810. Therefore, the data acquisition device 50 can shorten the distance from the microphone 400 to each conference attendee, and can input voice with a high S / N ratio.

[0026] ●Hardware configuration Next, the hardware configuration of the device or terminal constituting the speech recognition system according to the embodiment will be described with reference to Figures 4 to 6. Note that components may be added or deleted from the hardware configurations shown in Figures 4 to 6 as necessary.

[0027] ○Hardware configuration of voice recognition server○ First, the hardware configuration of the voice recognition server 10 will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of the hardware configuration of the voice recognition server. Each piece of hardware configuration of the voice recognition server 10 is indicated by a reference number in the 100 range. The voice recognition server 10 is constructed by a computer, and includes a processor 101, a memory 102, a network I / F (Interface) 103, and a bus line 109, as shown in Fig. 4.

[0028] The processor 101 is a computing means configured with a microcomputer, a general purpose graphics processing unit (GPU), a system large scale integration (LSI), etc. The memory 102 is a storage means configured with a random access memory (RAM), a read only memory (ROM), etc. The network I / F 103 is an interface for data communication using the communication network 5. The bus line 109 is an address bus, a data bus, etc. for electrically connecting each component such as the processor 101 shown in FIG. 4.

[0029] In the speech recognition server 10, for example, the processor 101 installs virtual machine software (virtualization application) stored in the memory 102, thereby running a virtual machine. The virtual machine software installs an individual OS by emulating individual hardware on a host OS (Operating System). This makes it possible to run multiple virtual machines independently of each other on a single system. In the cloud environment, software (data collection software) that collects data from the data acquisition device 50, software (analysis software) that analyzes the data, and the like are constructed. By using this virtualization technology, it is possible to realize efficient use of resources, suppression of initial investment costs for hardware, power saving, space saving, and the like.

[0030] ○Data management server hardware configuration○ Fig. 4 is a diagram showing an example of the hardware configuration of the data management server. Each piece of hardware configuration of the data management server 30 is indicated by a reference number in parentheses in the 300 range. The data management server 30 is constructed by a computer, and as shown in Fig. 4, has the same configuration as the voice recognition server 10, so a description of each piece of hardware configuration will be omitted.

[0031] ○Hardware configuration of data acquisition device○ 5A is a diagram showing an example of the hardware configuration of a data acquisition device 50. The data acquisition device 50 includes a CPU 501, a ROM 502, a RAM 503, a communication I / F 504, microphones 400-1 to 400-n (n is an integer equal to or greater than 1), cameras 410-1 to 410-n (n is an integer equal to or greater than 1), and a bus line 509.

[0032] The CPU 501 controls the operation of the entire data acquisition device 50. The CPU 501 is a calculation device that realizes each function of the data acquisition device 50 by reading various data such as programs stored in the ROM 502 onto the RAM 503 and executing processing. The ROM 502 is a non-volatile memory that stores various data such as programs executed by the CPU 501. The RAM 603 is a volatile memory used as a work area (working area) of the CPU 501. The communication I / F 504 is an interface for connecting to, for example, a whiteboard 830 or the control device 60, which is an example of an external device. The bus line 509 is an address bus or a data bus for electrically connecting each component such as the CPU 501 shown in FIG. 5(A).

[0033] ○Hardware configuration of audio output device○ 5B is a diagram showing an example of the hardware configuration of the audio output device 70. The audio output device 70 includes a CPU 701, a ROM 702, a RAM 703, a communication I / F 704, a speaker 705, and a bus line 709.

[0034] The CPU 701 controls the overall operation of the audio output device 70. The CPU 701 is a calculation device that realizes each function of the audio output device 70 by reading various data such as programs stored in the ROM 702 onto the RAM 703 and executing processing. The ROM 702 is a non-volatile memory that stores various data such as programs executed by the CPU 701. The RAM 703 is a volatile memory used as a work area (working area) of the CPU 701. The communication I / F 704 is, for example, an interface for connecting to the control device 60, which is an example of an external device. The speaker 705 is a type of output unit that outputs a sound signal according to the control of the CPU 701. The bus line 709 is an address bus or a data bus for electrically connecting each component such as the CPU 701 shown in FIG. 5(B).

[0035] ○Control device hardware configuration○ Fig. 6 is a diagram showing an example of the hardware configuration of the control device. Each hardware component of the control device 60 is indicated by a reference number in the 600 series. The control device 60 is constructed by a computer, and includes a CPU 601, a ROM 602, a RAM 603, a HD (Hard Disk) 604, a HDD (Hard Disk Drive) controller 605, a display 606, an external device connection I / F 608, a network I / F 609, a bus line 610, a keyboard 611, a pointing device 612, a DVD-RW (Digital Versatile Disk Rewritable) drive 614, and a media I / F 616, as shown in Fig. 6.

[0036] The CPU 601 controls the overall operation of the control device 60. The CPU 601 is a calculation device that realizes each function of the control device 60 by reading various data such as programs stored in the ROM 602 or the HD 604 onto the RAM 603 and executing processing. The ROM 602 is a non-volatile memory that stores various data such as programs executed by the CPU 601. The RAM 603 is a volatile memory used as a work area (working area) of the CPU 601. The HD 604 stores various data such as programs. The HDD controller 605 controls the reading or writing of various data from the HD 604 according to the control of the CPU 601. The display 606 displays various information such as a cursor, a menu, a window, characters, or an image. The display 606 may be a touch panel display equipped with an input means. The external device connection I / F 608 is an interface for connecting various external devices. In this case, the external device is, for example, the data acquisition device 50 or the audio output device 70. The network I / F 609 is an interface for data communication using the communication network 5. The bus line 610 is an address bus or a data bus for electrically connecting the components such as the CPU 601 shown in FIG.

[0037] The keyboard 611 is a type of input means having a plurality of keys for inputting characters, numbers, various instructions, etc. The pointing device 612 is a type of input means for selecting or executing various instructions, selecting a processing target, moving a cursor, etc. The input means may be not only the keyboard 611 and the pointing device 612, but also a touch panel, etc. The DVD-RW drive 614 controls reading or writing of various data from a DVD-RW 613 as an example of a removable recording medium. The removable recording medium is not limited to a DVD-RW, and may be a DVD-R or a Blu-ray (registered trademark) Disc, etc. The media I / F 616 controls reading or writing (storing) of data from a recording medium 115 such as a flash memory.

[0038] ○Hardware configuration of communication terminal○ Fig. 6 is a diagram showing an example of the hardware configuration of a communication terminal. Each piece of hardware configuration of the communication terminal 90 is indicated by a reference number in parentheses in the 900 range. The communication terminal 90 is constructed by a computer, and as shown in Fig. 6, has the same configuration as the control device 60, so a description of each piece of hardware configuration will be omitted.

[0039] Each of the above programs may be recorded in a computer-readable recording medium as an installable or executable file and distributed. Examples of the recording medium include a CD-R (Compact Disc Recordable), a DVD (Digital Versatile Disk), a Blu-ray Disc, an SD card, and a USB memory. The recording medium may be provided domestically or internationally as a program product. For example, the cloud server 3 realizes the speech recognition method according to the present invention by executing the program according to the present invention.

[0040] ●Function configuration Next, the functional configuration of the voice recognition system according to the embodiment will be described with reference to Fig. 7 to Fig. 11. Fig. 7 is a diagram showing an example of the functional configuration of a voice recognition server, a data management server, and a communication terminal. Fig. 11 is a diagram showing an example of the functional configuration of a data acquisition device, a control device, and a voice output device. Fig. 7 to Fig. 11 show devices or terminals shown in Fig. 1 that are related to the processing or operation described below.

[0041] ○Functional configuration of the voice recognition server○ First, the functional configuration of the voice recognition server 10 will be described with reference to Fig. 7. The voice recognition server 10 has a transmission / reception unit 11, a voice recognition engine 12, a lip reading processing unit 13, an integration unit 14, a region extraction unit 15, a text conversion unit 16, a synthetic voice creation unit 17, an acoustic characteristic conversion unit 18, and a data input / output unit 19. Each of these units is a function or means realized by any of the components shown in Fig. 4 operating in response to an instruction from the processor 101 in accordance with a program for the voice recognition server stored in the memory 102.

[0042] The transmission / reception unit 11 is realized mainly by the processing of the processor 101 for the network I / F 103, and transmits and receives various data or information to and from other devices or terminals via the communication network 5.

[0043] The voice recognition engine 12 includes a voice feature extraction unit 12a and a voice recognition unit 12b. The lip reading processor 13 includes an image feature extraction unit 13a and a machine lip reading unit 13b.

[0044] Here, the operations of the voice recognition engine 12, the lip reading processor 13, and the integrator 14 will be described with reference to Fig. 8 and Fig. 9. Fig. 8 and Fig. 9 are diagrams for explaining an example of the operations of the voice recognition engine, the lip reading processor, and the integrator. The voice feature extractor 12a extracts voice features, which are features as input values ​​for machine learning, from the voice data from the data acquisition device 50. The voice feature extractor 12a, for example, inputs a plurality of voice data acquired by the data acquisition device 50, cuts out each voice data for each unit time (for each frame), calculates spectrum features of the voice signal for each frame, such as MFCC (Mel-Frequency Cepstrum Coefficients) and Mel-cepstrum features, and normalizes them.

[0045] The speech recognition unit 12b performs machine learning using the features extracted by the speech feature extraction unit 12a and recognizes speech. The speech recognition unit 12b is a classifier that identifies features of speech. The speech recognition unit 12b can be exemplified by a deep neural network (DNN). The DNN has an input layer, an intermediate layer called a hidden layer, and an output layer. The DNN is configured to have a multilayer structure by increasing the number of intermediate layers. In order to recognize speech using a DNN, it is most effective to perform supervised learning using information called teacher labels or training data for the DNN. Since a high computing power is required to realize the DNN, it is desirable to realize the DNN in the speech recognition server 10, but if the data acquisition device 50 is equipped with a GPU or the like having a high computing power, the DNN may be realized in the data acquisition device 50. In addition to the DNN, the speech recognition unit 12b may use methods such as a support vector machine (SVM) and a scale-invariant feature transform (SIFT).

[0046] There are various DNNs used for speech recognition, and one that has recently gained prominence is the End-to-End model. The End-to-End model is a model that directly converts input speech into text through a single neural network, without dividing it into multiple functions such as an acoustic model, a language model, and a dictionary, and is also called a one-stop model. The End-to-End model has the advantages of being easy to implement and having a fast response speed due to its simple structure, but it requires a large amount of training data. In addition, a front-end that performs acoustic processing such as noise cancellation is often implemented before the speech recognition, but it is possible to achieve robust recognition performance by training speech recognition using noisy speech as it is, without performing noise cancellation at the front-end. Similarly, it is more effective to train speech recognition using muffled speech caused by wearing a mask as it is, rather than converting speech uttered while wearing a mask clearly at the front-end and then performing speech recognition.

[0047] The image feature extraction unit 13a extracts image features, which are features serving as input values ​​for machine learning, from the captured image data from the data acquisition device 50. An example of image features used for machine lip reading is shown in FIG.

[0048] FIG. 10 is a diagram for explaining an example of image features used in machine lip reading. First, the image feature extraction unit 13a recognizes, for example, the face of a conference attendee from the entire image captured by the camera 410. A general algorithm may be used for face recognition. Next, the image feature extraction unit 13a extracts lips from the recognized face. Then, the image feature extraction unit 13a extracts the time-series movements of each of the multiple points plotted as shown in FIG. 10 from the extracted image of the lips as features. The extracted features are the features of the mouth (lips) of the conference attendee captured by the camera 410 to perform machine lip reading. The machine lip reading unit 13b performs machine learning using the extracted features. For example, in the case of a noisy conference, the machine lip reading unit 13b performs machine learning using the features of the mouths of each of the multiple conference attendees. Note that the method of extracting features is publicly known, so a detailed description will be omitted.

[0049] The integration unit 14 fuses the results of machine lip reading by the machine lip reading unit 13b with the results of speech recognition by the speech recognition unit 12b. A technique that uses not only the results of speech recognition by the speech recognition unit 12b but also a moving image of the lips at the time of speaking is called multimodal speech recognition. In multimodal speech recognition, the input moving image is converted into a time-series image feature, and this image feature and speech feature are fused to generate an acoustic image feature. Then, speech recognition is performed by using this acoustic image feature. Multimodal speech recognition is a useful means for improving the accuracy of speech recognition in a conference.

[0050] Returning to FIG. 7, the region extraction unit 15 is realized by the processing of the processor 101, and extracts a specific region included in the text data which is a transcription of the user's speech. Specifically, the region extraction unit 15 acquires the text data which is a transcription of the user's speech stored in the data management server 30, and extracts a specific region included in the acquired text data. Here, the text data which is a transcription of the user's speech is data which includes the speeches of multiple users due to remote speech such as a conference, and there is an area which includes overlapping speeches in which multiple users' speeches overlap, which are difficult to use for machine learning, or speeches of a predetermined length or more. Therefore, the region extraction unit 15 extracts a specific region which is difficult to use for machine learning, which is included in the text data which is a transcription of the user's speech. For example, the region extraction unit 15 extracts an area where multiple users' speeches overlap, which is included in the text data which is a transcription of the user's speech. In addition, the region extraction unit 15 extracts an area where the speeches included in the text data which is a transcription of the user's speech are of a predetermined length or more. The text data that is a transcription of the user's speech and is used in the area extraction unit 15 is an example of the first text data.

[0051] The text conversion unit 16 is realized by the processing of the processor 101, and converts text data, which is a transcription of a user's speech, into text based on the specific region extracted by the region extraction unit 15. Specifically, the text conversion unit 16 decomposes the text data, which is a transcription of a user's speech, based on the extracted specific region, and reconstructs the text data into text data of a single utterance applicable to machine learning. For example, the text conversion unit 16 converts the text data, which is a transcription of a user's speech, into text data from which the specific region extracted from the text data is excluded, by text processing using pattern matching. Here, the converted text data converted by the text conversion unit 16 is an example of second text data.

[0052] Synthetic speech creation unit 17 is mainly realized by the processing of processor 101, and creates synthetic speech using the converted text data converted by text conversion unit 16. Synthetic speech creation unit 17 creates synthetic speech by, for example, TTS (Test to Speech) using the converted text data.

[0053] The acoustic characteristic conversion unit 18 is realized by processing of the processor 101, and performs pseudo-conversion of the acoustic characteristics of the synthetic voice created by the synthetic voice creation unit 17. The acoustic characteristic conversion unit 18, for example, pseudo-converts the acoustic characteristics of the synthetic voice created by the synthetic voice creation unit 17 into acoustic characteristics equivalent to remote speech of multiple users. The acoustic characteristics equivalent to remote speech are, for example, acoustic characteristics of a voice acquired when the user (speaker) and the microphone are apart by a predetermined distance or more (for example, 1 m or more).

[0054] The data input / output unit 19 is realized mainly by the processing of the processor 101 for the network I / F 103, and inputs / outputs (transmits / receives) data with the data management server 30.

[0055] ○Functional configuration of data management server○ Next, the functional configuration of the data management server 30 will be described with reference to Fig. 7. The data management server 30 has a transmission / reception unit 31, a data management unit 32, and a data input / output unit 33. Each of these units is a function or means realized by any of the components shown in Fig. 4 operating according to an instruction from the processor 301 in accordance with the data management server program stored in the memory 302. The data management server 30 also has a storage unit 3000 constructed by the memory 302 shown in Fig. 4. The storage unit 3000 stores teacher data used for machine learning of the voice recognition engine 12.

[0056] The transmission / reception unit 31 is realized mainly by the processing of the processor 301 for the network I / F 303, and transmits and receives various data or information to and from other devices or terminals via the communication network 5.

[0057] The data management unit 32 is mainly realized by the processing of the processor 301, and manages the teacher data used for machine learning of the voice recognition engine 12. Here, the teacher data includes the learning data processed by the voice recognition server 10, and the processed data (e.g., teacher flag) annotated by the communication terminal 90. The learning data also includes voice data (recorded data) and image data (captured image data) before and after processing by the voice recognition server 10, voice data converted by the acoustic characteristic conversion unit 18, and text data indicating text (character strings) transcribed from the voice data.

[0058] The data input / output unit 33 is realized mainly by the processing of the processor 301 for the network I / F 303, and inputs / outputs (transmits / receives) various data to / from the voice recognition server 10.

[0059] ○Functional configuration of communication terminal○ Next, the functional configuration of the communication terminal 90 will be described with reference to Fig. 7. The communication terminal 90 has a transmission / reception unit 91, a reception unit 92, a display control unit 93, and an annotation processing unit 94. Each of these units is a function or means realized by any of the components shown in Fig. 6 operating in response to an instruction from the CPU 901 in accordance with the program for the communication terminal loaded from the HD 904 onto the RAM 903.

[0060] The transmission / reception unit 91 is mainly realized by the processing of the CPU 901 for the network I / F 909, and transmits and receives various data or information to and from other devices or terminals via the communication network 5.

[0061] The reception unit 92 is mainly realized by the processing of the CPU 901 on the keyboard 911 or the pointing device 912, and receives various selections or inputs from a user (annotator). The display control unit 93 is mainly realized by the processing of the CPU 901, and displays various screens on a display unit such as a display 906.

[0062] The annotation processing unit 94 is mainly realized by the processing of the CPU 901, and performs annotation processing on voice data that is not used for machine learning in the voice recognition engine 12.

[0063] ○Functional configuration of data acquisition device○ Next, the functional configuration of the data acquisition device 50 will be described with reference to Fig. 11. The data acquisition device 50 has a start / end control unit 51, a synchronization control unit 52, a recording control unit 53, and a communication control unit 54. Each of these units is a function or means realized by any of the components shown in Fig. 5(A) operating according to an instruction from the CPU 501 in accordance with the program for the data acquisition device loaded from the ROM 502 onto the RAM 503. The data acquisition device 50 also has a storage unit 5000 constructed by the ROM 502 shown in Fig. 5(A).

[0064] Start / end control unit 51 is mainly realized by processing of CPU 901, and for example controls the start and end of recording by multiple microphones 400-1 to 400-n, and also controls the start and end of imaging by multiple cameras 410-1 to 410-n. Synchronization control unit 52 is mainly realized by processing of CPU 901, and controls the synchronization of multiple pieces of audio data acquired by one or more microphones 400, and also controls the synchronization of one or more pieces of imaging data captured by one or more cameras 410.

[0065] Recording control unit 53 is mainly realized by processing of CPU 501, and controls recording of audio data and imaging data acquired by microphone 400 and camera 410 in storage unit 5000. Communication control unit 54 is mainly realized by processing of CPU 501 for communication I / F 504, and controls communication with external devices such as whiteboard 830 or control device 60. The communication control is, for example, control to transmit a plurality of pieces of audio data and imaging data controlled by synchronization control unit 52 to cloud server 3 via whiteboard 830 or control device 60.

[0066] ○Control device functional configuration○ Next, the functional configuration of the control device 60 will be described with reference to Fig. 11. The control device 60 has a transmission / reception unit 61, a recording control unit 62, and a playback control unit 63. Each of these units is a function or means realized by any of the components shown in Fig. 6 operating in response to an instruction from the CPU 601 in accordance with the control device program loaded from the HD 604 onto the RAM 603.

[0067] The transmission / reception unit 61 is realized mainly by the processing of the CPU 601 for the network I / F 609, and transmits and receives various data or information to and from other devices or terminals via the communication network 5.

[0068] The recording control unit 62 is mainly realized by the processing of the CPU 601 for the external device connection I / F 608, and controls data acquisition processing such as sound and video recording in the data acquisition device 50. The playback control unit 63 is mainly realized by the processing of the CPU 601 for the external device connection I / F 608, and controls playback processing of recorded data in the audio output device 70.

[0069] ○Functional configuration of audio output device○ Next, the functional configuration of the audio output device 70 will be described with reference to Fig. 11. The audio output device 70 has a playback data acquisition unit 71 and a playback processing unit 72. These units are functions or means realized by operating any of the components shown in Fig. 5(B) by instructions from the CPU 701 in accordance with the audio output device program loaded from the ROM 702 onto the RAM 703.

[0070] The playback data acquisition unit 71 is mainly realized by the processing of the CPU 701 for the communication I / F 704, and acquires playback data (recorded data) to be played back and recorded. The playback processing unit 72 is mainly realized by the processing of the CPU 701, and performs playback processing of the audio related to the playback data (recorded data) acquired by the playback data acquisition unit 71.

[0071] Processing or operation of the embodiment ○Generating training data○ Next, the process or operation of the voice recognition system according to the embodiment will be described with reference to Fig. 12 to Fig. 18. First, the process of creating text data having characteristics of spoken language and distant speech, and synthetic voice based on the text data will be described with reference to Fig. 12 to Fig. 14. Fig. 12 is a sequence diagram showing an example of a process of generating teacher data used for machine learning of the voice recognition engine.

[0072] First, the data input / output unit 33 of the data management server 30 transmits text data, which is a transcription of the user's speech, to the voice recognition server 10 (step S11). This text data is stored in the storage unit 3000 as teacher data used for machine learning in the voice recognition engine 12. As a result, the data input / output unit 19 of the voice recognition server 10 receives the text data transmitted from the data management server 30.

[0073] Next, the area extraction unit 15 of the voice recognition server 10 extracts a specific area included in the text data received in step S11 (step S12). Here, the text data used in the area extraction process in step S12 will be described with reference to FIG. 13. FIG. 13 is a diagram showing an example of text data that is a transcription of a user's speech. The text data shown in FIG. 13 is data in which the speeches of multiple users are transcribed through remote speech such as a conference. In the text data shown in FIG. 13, the order of the speech (speech number) is associated with the start and end times when the speech was made, and the content of the speech.

[0074] As shown in FIG. 13, the text data, which is a transcription of the user's speech, includes an area where the speeches of multiple users overlap (for example, a portion marked with "F" in the speech content text), an area where the user's speech includes laughter (for example, speech No. "2"), and an area where the user's speech is longer than a predetermined length (for example, speech No. "3"). If these areas are included in the text data as teacher data, the recognition accuracy of the user's speech decreases, making it difficult to apply machine learning. Therefore, the area extraction unit 15 extracts, from the text data, which is a transcription of the user's speech, an area where the speeches of multiple users overlap and an area where the user's speech is longer than a predetermined length, etc., as specific areas to which machine learning is difficult to apply.

[0075] Next, the text conversion unit 16 performs text conversion of the text data, which is a transcription of the user's speech, based on the specific region extracted in step S12 (step S13). Specifically, the text conversion unit 16 decomposes the text data based on the specific region extracted by the region extraction unit 15, and converts it into text data of a single speech applicable to machine learning. For example, the text conversion unit 16 detects a tag indicating the specific region extracted by the region extraction unit 15 by pattern matching, and decomposes the part corresponding to the tag into another speech. In addition, for example, when there is an overlap of tags indicating a specific region, the text conversion unit 16 decomposes only the overlapping part using a stack. Then, the text conversion unit 16 combines all the speeches excluding the decomposed parts, and then re-decomposes the character string into a length that allows optimal speech synthesis, thereby reconstructing the text and acquiring the converted text data.

[0076] FIG. 14 is a diagram showing an example of text data after conversion. As shown in FIG. 14, the converted text data is a single-utterance text (character string) from which an area where a plurality of user utterances overlap (for example, a portion marked with "F" in the utterance content text) and an area where the user utterance includes laughter are excluded from the text data, which is a transcription of the user utterance shown in FIG. 13. The converted text data is a text (character string) from which an area where the user utterance is longer than a predetermined length (for example, utterance No. "3" shown in FIG. 13) is excluded. In this way, the text conversion unit 16 performs a text conversion process on an area including overlapping utterances that are difficult to use for machine learning among the text data, which is a transcription of the user utterance, to decompose and reconstruct the single-utterance text that is applicable to machine learning, and convert it into single-utterance text data that is applicable to machine learning.

[0077] Next, synthetic voice creation unit 17 creates synthetic voice data, which is a synthetic voice of the converted text data by TTS (Test to Speech) of the converted text data converted by text conversion unit 16 (step S14).

[0078] As a result, the voice recognition server 10 converts the text data into text data suitable for speech recognition of spoken language and creates synthetic voice using the converted text data, thereby improving the voice recognition accuracy using the converted text data and synthetic voice data as training data. Note that, in steps S12 and S13, the process of extracting a specific area not suitable for machine learning and converting the data into text data from which the extracted specific area is excluded has been described, but the voice recognition server 10 may be configured to extract a specific area from an area that can be used for machine learning and convert the data into text data including only the extracted specific area (from which the area not extracted is excluded).

[0079] Acoustic characteristic conversion processing Next, an example of converting acoustic characteristics of a synthetic voice created by the voice recognition server 10 will be described with reference to Fig. 15 as a process of generating teacher data used for machine learning in the voice recognition engine 12. Fig. 15 is a sequence diagram showing an example of a process of generating teacher data used for machine learning in the voice recognition engine by converting acoustic characteristics.

[0080] First, the acoustic characteristic conversion unit 18 of the voice recognition server 10 executes an acoustic characteristic conversion process of the synthetic voice created in step S14 (step S21). In order to generate teacher data corresponding to the remote speech, the acoustic characteristic conversion unit 18 converts the acoustic characteristics of the synthetic voice created by the synthetic voice creation unit 17 into acoustic characteristics corresponding to the remote speech in a pseudo manner. For example, the acoustic characteristic conversion unit 18 performs pseudo conversion of the acoustic characteristics by signal processing of the synthetic voice using an impulse response or the like. The acoustic characteristic conversion unit 18 may use a method using filter processing such as a low-pass filter, or a method of constructing a DNN and converting the acoustic characteristics into those corresponding to the remote speech.

[0081] Next, the data input / output unit 19 of the voice recognition server 10 transmits to the data management server 30 learning data including the converted text data converted in step S13 and the converted voice data whose acoustic characteristics have been converted in step S21 (step S22). As a result, the data input / output unit 33 of the data management server 30 receives the learning data transmitted from the voice recognition server 10.

[0082] Next, the transmitting / receiving unit 31 of the data management server 30 transmits the learning data received in step S22 to the communication terminal 90 (step S23). As a result, the transmitting / receiving unit 91 of the communication terminal 90 receives the learning data transmitted from the data management server 30.

[0083] Next, the annotation processing unit 94 of the communication terminal 90 executes annotation processing using the learning data received in step S23 (step S24). Specifically, the annotation processing unit 94 transcribes the received learning data into text (character strings), performs time stamp processing, generates teacher labels, etc., in response to input from an annotator using the communication terminal 90. Since the voice recognition system 1 has already created synthetic voice from text data, which is a transcription of the user's speech, and text data converted by the text conversion unit 16, it is possible to reduce the burden required for annotation processing.

[0084] The teacher labels can be obtained by manually transcribing and time stamping the voice data and using the transcribed data for learning, or by extracting a highly reliable output result from the text (voice recognition output) output from the existing voice recognition engine 12 as the teacher label. The former method of manually creating all teacher labels and performing machine learning is called supervised learning, while the latter method of using a highly reliable output result as the teacher label without human intervention is called semi-supervised learning. In the voice recognition system 1, when semi-supervised learning is performed, if the recognition results based on the voice data acquired by the multiple microphones 400 all have the same content, it is considered that the reliability is high and that the recognition results are used as the teacher label. Previously used learning data has been annotated and can be used as teacher data.

[0085] Next, the transmitting / receiving unit 91 of the communication terminal 90 transmits the processed data annotated in step S24 to the data management server 30 (step S25). As a result, the transmitting / receiving unit 31 of the data management server 30 receives the processed data transmitted from the communication terminal 90. Then, the data management unit 32 of the data management server 30 stores the processed data received in step S25 and the teacher data including the learning data received in step S22 in the storage unit 3000 (step S26).

[0086] In this way, the voice recognition system 1 uses the text data converted by the text conversion unit 16 and the converted voice data converted by the acoustic characteristic conversion unit 18 as teacher data for machine learning in the voice recognition engine 12, thereby improving the voice recognition accuracy in usage scenarios such as meetings that have characteristics of spoken language and remote speech.

[0087] Playback and recording processing 16 and 18, an example of performing replay recording using synthetic voice created by the voice recognition server 10 will be described as a process of generating teacher data used for machine learning in the voice recognition engine 12. Fig. 16 is a sequence diagram showing an example of a process of generating teacher data used for machine learning in the voice recognition engine using replay recording.

[0088] First, the transmitting / receiving unit 11 of the voice recognition server 10 transmits a playback / recording request to the control device 60 to request playback / recording of the synthetic voice created in step S14 (step S31). This playback / recording request includes the synthetic voice data created in step S14. As a result, the transmitting / receiving unit 61 of the control device 60 receives the playback / recording request transmitted from the voice recognition server 10.

[0089] Next, the playback control unit 63 of the control device 60 transmits a playback start request to the audio output device 70, indicating a request to start playing the synthetic voice data (step S32). This playback start request includes the synthetic voice data received in step S31. As a result, the playback data acquisition unit 71 of the audio output device 70 receives the playback start request transmitted from the control device 60. In addition, the recording control unit 62 of the control device 60 transmits a recording start request to the data acquisition device 50, indicating a request to start recording (step S33). As a result, the communication control unit 54 of the data acquisition device 50 receives the recording start request transmitted from the control device 60.

[0090] The playback processing unit 72 of the voice output device 70 executes playback processing of the voice related to the synthetic voice data received in step S32 (step S34). Furthermore, the start / end control unit 51 of the data acquisition device 50 starts recording by the microphone 400 in response to the recording start request received in step S33 (step S35). Here, the data acquisition device 50 and the voice output device 70 are installed at positions that are apart by a predetermined distance or more (e.g., 1 m or more). This allows the voice recognition system 1 to play back and record voice that corresponds to a remote speech similar to that in an actual meeting or the like.

[0091] Next, the communication control unit 54 of the data acquisition device 50 transmits the recorded data acquired in step S35 to the control device 60 (step S36). As a result, the recording control unit 62 of the control device 60 receives the recorded data transmitted from the data acquisition device 50. Then, the transmission / reception unit 61 of the control device 60 transmits the recorded data received in step S36 to the voice recognition server 10 (step S37). As a result, the transmission / reception unit 11 of the voice recognition server 10 receives the recorded data transmitted from the control device 60.

[0092] Next, the voice recognition server 10 executes voice recognition processing of the recorded data received in step S37 (step S38). This voice recognition processing is the same as the processing described in FIG. 7 to FIG. 9. Since the processing in step S38 is a replay recording, processing by the lip reading processing unit 13 is not required. Then, the data input / output unit 19 transmits learning data including the converted text data converted in step S13 and the voice processing data subjected to the voice recognition processing in step S38 to the data management server 30 (step S39). As a result, the data input / output unit 33 of the data management server 30 receives the learning data transmitted from the voice recognition server 10. The subsequent processing in steps S40 to S43 is the same as the processing in steps S23 to S26 shown in FIG. 15, and therefore the description will be omitted.

[0093] In this way, the voice recognition system 1 can convert the synthetic voice data into voice data suitable for far-field voice recognition by performing playback and recording using the data acquisition device 50 and the voice output device 70 installed at a predetermined distance or more. As a result, even when playback and recording are used, the voice recognition system 1 can improve the voice recognition accuracy in usage scenarios such as meetings that have the characteristics of spoken language and far-field voice recognition.

[0094] Here, an example of a case where playback and recording are performed using an audio output device 70 in which a covering such as a mask is attached to the audio output section will be described with reference to Figures 17 and 18. Figure 17 is a diagram showing an example of an audio output device with a covering.

[0095] FIG. 17(A) shows an audio output device 70 in which a mask 750 is attached to a dummy head 790 and a speaker 705 serving as an audio output unit is provided inside the mask 750. In the audio output device 70 shown in FIG. 17(A), the speaker 705 is provided so as to be covered by the mask 750, and audio when a mask is worn can be recorded by playing back recorded data from the speaker 705 covered by the mask 750. On the other hand, FIG. 17(B) shows an audio output device 70 in which a face shield 760 (also called a face guard or a mask guard) is attached to a dummy head 790 and a speaker 705 serving as an audio output unit is provided inside the face shield 760. In the audio output device 70 shown in FIG. 17(B), the speaker 705 is provided so as to be covered by the face shield 760, and audio when a face shield is worn can be recorded by playing back recorded data from the speaker 705 covered by the face shield 760.

[0096] 18, by installing a shield such as a partition 850 that partially shields the space between the installation position of the data acquisition device 50 and the installation position of the audio output device 70, it is possible to more faithfully reproduce, for example, a conference venue where measures to prevent infection with the new coronavirus have been taken. The partition 850 is, for example, a transparent acrylic panel or vinyl curtain installed to prevent splashes.

[0097] In this way, by performing playback recording in a recording environment in which a voice output device 70 as shown in FIG. 17 and a partition 850 as shown in FIG. 18 are installed, the voice recognition system 1 can perform playback recording that is more suited to actual usage scenarios such as meetings where wearing masks is commonplace.

[0098] Effect of the embodiment As described above, the voice recognition system 1 generates training data suitable for spoken language, thereby improving the accuracy of voice recognition by machine learning using training data close to an actual meeting scene. In addition, the voice recognition system 1 generates training data by synthetic voice using a transcription of an utterance, thereby reducing the burden required for annotation. Furthermore, the voice recognition system 1 uses voice that has been converted into acoustic characteristics equivalent to a remote speech of a synthetic voice or a playback recording that reproduces a remote speech as training data, thereby improving the accuracy of voice recognition in a usage scene such as a meeting that has a feature of a remote speech.

[0099] ●Summary● As described above, the voice recognition system according to one embodiment of the present invention is a voice recognition system 1 that performs voice recognition processing on a user's speech using a voice recognition engine 12, and includes a region extraction unit 15 (an example of a region extraction means) that extracts a specific region included in first text data that is a transcription of the user's speech, and a text conversion unit 16 (an example of a text conversion means) that converts the first text data into second text data based on the extracted specific region, and uses the converted second text data for machine learning in the voice recognition engine 12. In this way, the voice recognition system 1 can improve the voice recognition accuracy by generating teacher data suitable for speech recognition of spoken language.

[0100] In addition, in the voice recognition system according to an embodiment of the present invention, the region extraction unit 15 (an example of a region extraction means) extracts a region where multiple user utterances overlap, which is included in the first text data, which is a transcription of the user utterance. The region extraction unit 15 also extracts a region where the user utterances included in the first text data are longer than a predetermined length. Then, the text conversion unit 16 (an example of a text conversion means) converts the first text data into second text data in which a specific region is excluded from the first text data, by text processing using pattern matching. In this way, the voice recognition system 1 can improve the voice recognition accuracy by converting the converted text data into text data suitable for speech recognition of spoken language, using the converted text data as training data.

[0101] Furthermore, the voice recognition system according to an embodiment of the present invention includes a synthetic voice creation unit 17 (an example of a synthetic voice creation means) that creates synthetic voice using second text data converted by a text conversion unit 16 (an example of a text conversion means), and an acoustic characteristic conversion unit 18 (an example of an acoustic characteristic conversion means) that pseudo-converts the acoustic characteristics of the created synthetic voice, and uses the second text data and the converted voice data converted by the acoustic characteristic conversion unit 18 for machine learning of the voice recognition engine 12. In addition, the acoustic characteristic conversion unit 18 pseudo-converts the acoustic characteristics of the synthetic voice into acoustic characteristics corresponding to the remote speech of the multiple users. As a result, the voice recognition system 1 uses the second text data, which is the converted text data, and the converted voice data as teacher data for machine learning of the voice recognition engine 12, thereby improving the voice recognition accuracy in a usage scene such as a conference having characteristics of spoken language and remote speech.

[0102] Moreover, the voice recognition system according to an embodiment of the present invention includes a synthetic voice creation unit 17 (an example of synthetic voice creation means) that creates synthetic voice using the second text data converted by the text conversion unit 16 (an example of text conversion means), and performs playback recording using the created synthetic voice. Moreover, the voice recognition system 1 includes a voice output device 70 that outputs the created synthetic voice, and a data acquisition device 50 that acquires the voice data output by the voice output device 70, and uses the second text data and the voice data acquired by the data acquisition device 50 for machine learning of the voice recognition engine 12. Furthermore, the voice output device 70 and the data acquisition device 50 are installed at positions that are separated by a predetermined distance or more. As a result, the voice recognition system 1 performs playback recording using the data acquisition device 50 and the voice output device 70 that are installed at a predetermined distance or more, and can convert the synthetic voice data into voice data suitable for voice recognition of remote speech (Far-Field voice recognition). As a result, the voice recognition system 1 can improve the voice recognition accuracy in usage scenes such as meetings that have characteristics of spoken language and remote speech, even when playback recording is used.

[0103] Furthermore, in the voice recognition system according to an embodiment of the present invention, the voice output device 70 includes a covering (e.g., a mask 750 or a face shield 760) that covers the speaker 705 (an example of an output unit) that outputs the synthetic voice. This allows the voice recognition system 1 to perform playback and recording that more closely resembles actual usage scenarios, such as meetings, where wearing a mask is commonplace.

[0104] ●Additional Information● Each function of the above-described embodiment can be realized by one or more processing circuits. Here, the "processing circuit" in the present embodiment includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), a SOC (System on a chip), a GPU (Graphics Processing Unit), and a conventional circuit module designed to execute each function described above.

[0105] In addition, the various tables in the above-described embodiment may be generated by the learning effect of machine learning, and the data of each associated item may be classified by machine learning, making it unnecessary to use tables. Here, machine learning is a technology for making a computer acquire human-like learning capabilities, and refers to a technology in which a computer autonomously generates an algorithm required for judgment such as data identification from learning data previously acquired, and applies this to new data to make predictions. The learning method for machine learning may be any of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning, or may be a combination of these learning methods, and any learning method for machine learning may be used.

[0106] Although the speech recognition system, speech recognition method, and program according to one embodiment of the present invention have been described above, the present invention is not limited to the above-described embodiment, and other modifications, changes, deletions, and other variations can be made within the scope of what a person skilled in the art can conceive. In any aspect, as long as the functions and effects of the present invention are achieved, it is within the scope of the present invention. [Explanation of symbols]

[0107] 1. Voice Recognition System 3. Cloud Server 5. Communication Network 10 Speech Recognition Server 11 Transmitter / receiver 12 Voice Recognition Engine 15 Area extraction unit (an example of area extraction means) 16 Text conversion unit (an example of a text conversion means) 17 Synthetic voice creation unit (an example of a synthetic voice creation means) 18 Acoustic characteristic conversion unit (an example of an acoustic characteristic conversion means) 19 Data input / output section 30 Data Management Server 31 Transmitter / receiver 32 Data Management Department 33 Data input / output section 50 Data Acquisition Device 60 Control device 70 Audio output device 81~85 Meeting attendees (example of users) 705 Speaker (an example of an output section) 750 Mask (an example of a covering) 760 Face shield (an example of a covering) 850 Partition (an example of a shield) [Prior art documents] [Patent documents]

[0108] [Patent Document 1] JP 2008-129028 A

Claims

1. A voice recognition system that performs voice recognition processing using a voice recognition engine on a user's speech, A region extraction means for extracting a specific region included in the first text data which is a transcription of the utterance; a text conversion means for converting the first text data into second text data based on the extracted specific region; a synthetic voice generating means for generating synthetic voice using the second text data, A speech recognition system that uses the converted second text data and the created synthetic speech for machine learning of the speech recognition engine.

2. 2. The speech recognition system according to claim 1, wherein the region extraction means extracts a region in which speeches of a plurality of users included in the first text data overlap.

3. 3. The speech recognition system according to claim 1, wherein the region extraction means extracts a region in which the utterance contained in the first text data has a predetermined length or more.

4. 4. The speech recognition system according to claim 1, wherein the text conversion means converts the first text data into the second text data obtained by excluding the specific region from the first text data by text processing using pattern matching.

5. 5. The speech recognition system according to claim 1, further comprising: an acoustic characteristic conversion means for converting acoustic characteristics of the generated synthetic voice; A voice recognition system that uses the second text data and the converted voice data converted by the acoustic characteristic conversion means for machine learning of the voice recognition engine.

6. 6. The speech recognition system according to claim 5, wherein said acoustic characteristic conversion means converts acoustic characteristics of said synthetic speech into pseudo acoustic characteristics corresponding to remote speech of a plurality of said users.

7. A speech recognition system according to any one of claims 1 to 4, A voice recognition system that reproduces and records the synthesized voice.

8. 8. The speech recognition system according to claim 7, further comprising: a voice output device for outputting the synthesized voice; a data acquisition device for acquiring the voice data output by the voice output device, A voice recognition system that uses the second text data and the voice data acquired by the data acquisition device for machine learning of the voice recognition engine.

9. The voice recognition system according to claim 8 , wherein the voice output device and the data acquisition device are placed at positions spaced apart by a predetermined distance or more.

10. The voice recognition system according to claim 8 or 9, wherein the voice output device includes a cover for covering an output section that outputs the synthetic voice.

11. A speech recognition method executed by a speech recognition system that performs speech recognition processing using a speech recognition engine on a user's speech, comprising: a region extraction step of extracting a specific region included in the first text data which is a transcription of the utterance; a text conversion step of converting the first text data into second text data based on the extracted specific region; a synthetic speech generating step of generating synthetic speech using the second text data; A speech recognition method in which the converted second text data and the created synthetic speech are used for machine learning of the speech recognition engine.

12. A program for causing a computer to execute the speech recognition method according to claim 11.

Images