Speech recognition text display system, speech recognition text display device, speech recognition text display method and program

Abstract

To provide a speech recognition text display system, a speech recognition text display device, a speech recognition text display method, and a program that enable participants to easily understand text information converted from speech signals.SOLUTION: A speech recognition text display system for converting speech signals into text information to display it, comprises: an acquisition unit that acquires speech signals; a speech recognition unit that speech-recognizes speech signals acquired by the acquisition unit and outputs text information; and a display unit that displays the text information. The speech recognition unit calculates recognition likelihood for each homonym for words whose pronunciation is determined by speech recognition to have multiple homonyms in common, and switches whether to output the determined words as text information converted into Chinese characters or text information using kana characters according to the magnitude of each calculated value.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a speech recognition text display system, a speech recognition text display device, a speech recognition text display method, and a program. [Background technology]

[0002] Conventionally, when multiple participants are in a conference, there is known a device that converts the contents (audio signals) spoken by each participant into text and displays it (see, for example, Patent Document 1). Such a device is sometimes used to support hearing-impaired people in participating in the conference. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-179480 Summary of the Invention [Problem to be solved by the invention]

[0004] For example, in the device described in Patent Document 1, when a word spoken by a conference participant contains a homonym, the word may be converted into a kanji character that the speaker did not intend and displayed. In this case, it may be difficult for other participants to understand what the speaker said. This problem is particularly pronounced when hearing-impaired people participate in a conference, because hearing-impaired people understand the contents of the conference by reading text.

[0005] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a speech recognition text display system, a speech recognition text display device, a speech recognition text display method, and a program that can make it easier for participants to understand text information converted from speech signals. [Means for solving the problem]

[0006] (1) In order to achieve the above object, a speech recognition text display system (1) according to one embodiment of the present invention is a speech recognition text display system that converts a speech signal into text information and displays it, and includes an acquisition unit (222) that acquires the speech signal, a speech recognition unit (speech recognition unit 223, text conversion unit 224, dependency analysis unit 225, calculation unit 226, conversion switching unit 227) that performs speech recognition on the speech signal acquired by the acquisition unit and outputs the text information, and a display unit (display unit 203, display unit 303, minutes creation unit 228, image output unit 241) that displays the text information, wherein the speech recognition unit calculates a recognition likelihood for each of a plurality of homonyms that share a common pronunciation for a word that is determined to have such homonyms in the speech recognition, and switches between outputting the determined word as the text information converted into kanji or as the text information in kana characters depending on the magnitude of the calculated recognition likelihoods.

[0007] (2) In addition, in a speech recognition text display system according to one aspect of the present invention, when the maximum value of the calculated plurality of recognition likelihoods is lower than a predetermined value, the speech recognition unit may output the determined word as text information in kana characters without converting it into kanji.

[0008] (3) In addition, in a speech recognition text display system according to one aspect of the present invention, when displaying the text information, the display unit may display the determined word that is output in kana characters by the speech recognition unit in the text information in a format different from that of other words.

[0009] (4) In order to achieve the above object, a speech recognition text display device according to one embodiment of the present invention is a speech recognition text display device that converts a speech signal into text information and displays it, and includes an acquisition unit that acquires the speech signal, a speech recognition unit that performs speech recognition on the speech signal acquired by the acquisition unit and outputs the text information, and a display unit that displays the text information.The speech recognition unit calculates a recognition likelihood for each of a word that is determined to have multiple homonyms with a common pronunciation in the speech recognition, and switches between outputting the determined word as the text information converted into kanji or as the text information in kana characters depending on the magnitude of the calculated recognition likelihoods.

[0010] (5) In order to achieve the above object, a speech recognition text display method according to one embodiment of the present invention is a speech recognition text display method in a speech recognition text display system that converts a speech signal into text information and displays it, and includes an acquisition step in which an acquisition unit acquires a speech signal, a speech recognition step in which a speech recognition unit performs speech recognition on the speech signal acquired by the acquisition unit and outputs the text information, and a display step in which a display unit displays the text information, wherein in the speech recognition step, for a word determined in the speech recognition to have multiple homonyms with a common pronunciation, the speech recognition unit calculates a recognition likelihood for each of the multiple homonyms, and switches between outputting the determined word as the text information converted into kanji or as the text information in kana characters depending on the magnitude of the calculated multiple recognition likelihoods.

[0011] (6) In order to achieve the above object, a program according to one aspect of the present invention causes a speech recognition text display system that converts a speech signal into text information and displays it to execute the following steps: an acquisition step of acquiring a speech signal; a speech recognition step of performing speech recognition on the speech signal acquired by the acquisition unit and outputting the text information; and a display step of displaying the text information. In the speech recognition step, for a word that is determined in the speech recognition to have multiple homonyms with a common pronunciation, a recognition likelihood for each of the multiple homonyms is calculated, and depending on the magnitude of the calculated multiple recognition likelihoods, the determined word is switched between being output as the text information converted into kanji or as the text information in kana characters. [Effects of the Invention]

[0012] According to the above (1), (4), (5), or (6), it is possible to provide an effect that the participants can easily understand the text information converted from the voice signal.

[0013] According to the above-mentioned (2), it is possible to reduce the possibility that text information converted into kanji characters that the speaker does not intend will be displayed. According to (3) above, participants were able to distinguish words displayed in text information using kana characters because the recognition likelihood of homonyms was low. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a block diagram showing an example of the configuration of a voice recognition text display device (voice recognition text display system) according to the present embodiment. [Figure 2] 10A and 10B are diagrams illustrating an example of an image displayed on a display unit of the parent device according to the embodiment. [Figure 3] 10A and 10B are diagrams illustrating an example of an image displayed on a display unit of the slave device according to the embodiment. [Figure 4] 10 is a flowchart showing an example of a processing procedure performed by a voice recognition text display device (voice recognition text display system) according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0016] First, an example of a situation in which the voice recognition text display device (voice recognition text display system) of this embodiment is used will be described. The speech recognition text display device (speech recognition text display system) of this embodiment is used in a conference attended by two or more participants. Some of the participants may have difficulty speaking. Participants who can speak wear a microphone or use a terminal equipped with a microphone (smartphone, tablet terminal, personal computer, etc.). Participants who are hearing impaired use a terminal into which text can be input. The speech recognition text display device recognizes speech signals uttered by participants, converts them into text, and displays the text on each participant's terminal. The speech recognition text display device also displays text information input by hearing impaired participants as text on each participant's terminal.

[0017] FIG. 1 is a block diagram showing an example of the configuration of a voice recognition text display device (voice recognition text display system) 1 according to this embodiment. As shown in Fig. 1, a voice recognition text display device (voice recognition text display system) 1 includes a parent device 2 and child devices 3a, 3b, etc. When one of the child devices 3a, 3b, etc. is not specified, it will be simply referred to as child device 3. The master unit 2 and the slave unit 3 are connected via a wired or wireless network 4 .

[0018] The parent device 2 includes a sound pickup unit 201, an operation unit 202, a display unit 203, a communication unit 204, an authentication unit 211, an acoustic model and dictionary storage unit 221, an acquisition unit 222, a voice recognition unit 223, a text conversion unit 224, a dependency analysis unit 225, a minutes creation unit 228, a minutes storage unit 229, a text acquisition unit 231, and an image output unit 241. The dependency analysis unit 225 includes a calculation unit 226 and a conversion switching unit 227. However, the calculation unit 226 or the conversion switching unit 227 may be provided separately from the dependency analysis unit 225.

[0019] The slave unit 3 includes a sound pickup unit 301, an operation unit 302, a display unit 303, a communication unit 304, and a processing unit 305. The sound pickup unit 301, the operation unit 302, the display unit 303, the communication unit 304, and the processing unit 305 are connected via a bus 306.

[0020] <Handset 3> First, the slave unit 3 will be described. The slave device 3 is, for example, a smartphone, a tablet terminal, a personal computer, etc. The slave device 3 may include an audio output unit, a motion sensor, a GPS (Global Positioning System), etc.

[0021] The sound collection unit 301 is a microphone. The sound collection unit 301 collects a voice signal of a user, converts the collected voice signal from an analog signal to a digital signal, and outputs the converted digital voice signal to the processing unit 305.

[0022] The operation unit 302 detects user operations and outputs the detection results to the processing unit 305. The operation unit 302 is, for example, a touch panel sensor provided on the display unit 303, or a wired or wirelessly connected keyboard.

[0023] The processing unit 305 generates setting information based on the operation result of the operation unit 302, and outputs the generated setting information to the communication unit 304. Here, the setting information includes identification information of the participants. The setting information may include information indicating whether the sound collection unit is being used and information indicating whether the operation unit is being used. The processing unit 305 generates a login instruction based on the operation result of the operation unit 302, and outputs the generated login instruction to the communication unit 304. Here, the login instruction includes identification information of the participants and identification information of the remote unit 3. The processing unit 305 adds identification information to text information based on the operation result of the operation unit 302, and outputs the text information to the communication unit 304. The processing unit 305 adds identification information to the audio signal output by the sound collection unit 301, and outputs the text information to the communication unit 304. The processing unit 305 acquires image data output by the communication unit 304, and outputs the acquired image data to the display unit 303. The processing unit 305 establishes communication with the parent unit 2 based on the login permission information output by the communication unit 304. When the processing unit 305 receives a speech restriction instruction (input restriction instruction) from the parent unit 2, it may impose a restriction on text input. Furthermore, when the processing unit 305 receives a speech restriction instruction from the parent unit 2, it may also impose a restriction on voice input.

[0024] The display unit 303 displays the image data output by the processing unit 305. The display unit 303 is, for example, a liquid crystal display device, an organic EL (electroluminescence) display device, an electronic ink display device, etc. The image displayed on the display unit 303 will be described later.

[0025] The communication unit 304 transmits the setting information output by the processing unit 305 to the parent unit 2 via the network 4. The communication unit 304 transmits the login instruction output by the processing unit 305 to the parent unit 2 via the network 4. The communication unit 304 transmits the text information or audio signal output by the processing unit 305 to the parent unit 2 via the network 4. The transmitted text information or audio signal includes the user's identification information and the identification information of the child unit 3. The communication unit 304 receives image data transmitted by the parent unit 2 and outputs the received image data to the processing unit 305. When the communication unit 304 receives information permitting login transmitted by the parent unit 2, it outputs the received information permitting login to the processing unit 305.

[0026] <Base unit 2> Next, the base unit 2 will be described. The base unit 2 is, for example, a notebook computer.

[0027] The sound collection unit 201 is a microphone. The sound collection unit 201 collects a voice signal of a user, converts the collected voice signal from an analog signal to a digital signal, and outputs the converted digital voice signal to the acquisition unit 222.

[0028] The operation unit 202 detects a user's operation and outputs the detection result to the text acquisition unit 231. The operation unit 202 is, for example, a touch panel sensor or a keyboard provided on the display unit 203. During login processing, the operation unit 202 outputs the operation detection result to the authentication unit 211.

[0029] The display unit 203 is, for example, a liquid crystal display device, an organic EL display device, an electronic ink display device, etc. The display unit 203 displays image data output by the image output unit 241. The images displayed on the display unit 203 will be described later.

[0030] The communication unit 204 receives an audio signal transmitted by the handset 3 and outputs the received audio signal to the acquisition unit 222. The communication unit 204 receives text information transmitted by the handset 3 and outputs the received text information to the text acquisition unit 231. The communication unit 204 receives a login instruction transmitted by the handset 3 and outputs the received login instruction to the authentication unit 211. The communication unit 204 transmits image data output by the image output unit 241 to the handset 3 via the network 4. The communication unit 204 transmits information permitting login output by the authentication unit 211 to the handset 3 via the network 4.

[0031] The authentication unit 211 determines whether to permit login based on the identification information of the participant and the identification information of the slave unit 3 included in the login instruction output by the communication unit 204. If the login is permitted, the authentication unit 211 outputs information permitting login to the communication unit 204. The authentication unit 211 determines whether to permit login of the user of the master unit 2 based on the result of operation of the operation unit 202. If the login is permitted, the authentication unit 211 outputs information permitting login to each functional unit and permits operation of each functional unit. The functional units are the communication unit 204, the authentication unit 211, the acoustic model and dictionary storage unit 221, the acquisition unit 222, the speech recognition unit 223, the text conversion unit 224, the dependency analysis unit 225, the calculation unit 226, the conversion switching unit 227, the minutes creation unit 228, the minutes storage unit 229, the text acquisition unit 231, and the image output unit 241.

[0032] The acoustic model and dictionary storage unit 221 stores, for example, an acoustic model, a language model, a word dictionary, etc. An acoustic model is a model based on sound features, and a language model is a model of information about words and their arrangement. A word dictionary is a dictionary with a large vocabulary, such as a large vocabulary word dictionary. The base unit 2 may store and update words not stored in the acoustic model and dictionary storage unit 221. The acoustic model and dictionary storage unit 221 may have a database (DB) for each conference, for example. For example, a first DB may be for general conferences, a second DB for presentations, and a third DB for international conferences. Using a DB tailored to the conference in this way makes it easier to appropriately convert homonyms, etc.

[0033] The acquisition unit 222 acquires the audio signal output by the sound collection unit 201 or the audio signal output by the communication unit 204 , and outputs the acquired audio signal to the voice recognition unit 223 .

[0034] The speech recognition unit 223 acquires the speech signal output by the acquisition unit 222. The speech recognition unit 223 detects a speech signal for a speech section from the speech signal. The speech section detection is performed, for example, by detecting a speech signal equal to or greater than a predetermined threshold as a speech section. The speech recognition unit 223 may also detect a speech section using other well-known methods. The speech recognition unit 223 performs speech recognition on the detected speech signal for the speech section by referring to the acoustic model and dictionary storage unit 221 using a well-known method. The speech recognition unit 223 performs speech recognition using, for example, the method disclosed in Japanese Patent Application Laid-Open No. 2015-64554. The speech recognition unit 223 outputs the recognized recognition result and the speech signal to the text conversion unit 224. The speech recognition unit 223 outputs the recognition result and the speech signal in association with each other, for example, for each sentence, for each speech section, or for each utterance. When audio signals are input simultaneously, the audio recognition unit 223 performs audio recognition for each audio pickup unit (201 or 301) by, for example, time-division processing. When the microphone is a microphone array, the audio recognition unit 223 also performs well-known audio recognition processing such as sound source separation processing, sound source localization processing, and sound source identification processing.

[0035] The text conversion unit 224 converts the recognition result output by the speech recognition unit 223 into text by referring to the acoustic model and dictionary storage unit 221. The text information includes information of at least one character. The text conversion unit 224 outputs the converted text information and the acquired voice signal to the dependency analysis unit 225. The text conversion unit 224 may convert the recognition result of the utterance information into text by deleting interjections such as "ah," "um," "eh," and "well," etc.

[0036] The dependency analysis unit 225 performs morphological analysis and dependency analysis on the text information output by the text conversion unit 224 or the text information output by the communication unit 204, by referring to the acoustic model and dictionary storage unit 221. For the dependency analysis, for example, a shift-reduce method, a spanning tree method, or an SVM (Support Vector Machines) in a stepwise application method for chunk identification is used. The dependency analysis unit 225 outputs the text information on which the morphological analysis and dependency analysis have been performed, and the analysis results, to the minutes creation unit 228. When the dependency analysis unit 225 acquires the audio signal output by the text conversion unit 224, it also outputs the acquired audio signal to the minutes creation unit 228. Here, the dependency analysis unit 225 outputs to the minutes creation unit 228 text information that has been processed by the calculation unit 226 and the conversion switching unit 227, which will be described below, in addition to (or instead of) the information described above.

[0037] For a word determined to have multiple homonyms with the same pronunciation through speech recognition (morphological analysis and dependency analysis), the calculation unit 226 calculates a recognition likelihood for each of the multiple homonyms. For example, the word "kousei" (construction) has multiple homonyms with the same pronunciation, such as "kousei" (construction), "kosei" (steel), "kosei" (welfare), "kosei" (fairness), etc. The calculation unit 226 calculates the recognition likelihood for each of "kousei" (construction), "kosei" (steel), "kosei" (fairness), etc., by referring to the acoustic model and dictionary storage unit 221 or the context before and after "kousei" in the text information and other information. The calculation unit 226 outputs the multiple recognition likelihoods obtained by the calculation to the conversion switching unit 227.

[0038] The conversion switching unit 227 switches between outputting text information converted into kanji by the dependency analysis unit 225 or outputting text information in kana characters for words determined to have multiple homonyms with a common pronunciation, depending on the magnitudes of the multiple recognition likelihoods output by the calculation unit 226. Note that "kana characters" includes hiragana and katakana. For example, the conversion switching unit 227 switches between outputting the word "kousei" as "kousei" or "kousei", or outputting it as "kousei" or "kousei", etc., depending on the recognition likelihood of each of "kousei", "kousei", "kousei", "kousei", etc. output by the calculation unit 226. As an example, when the maximum value of the multiple recognition likelihoods output by the calculation unit 226 is lower than a predetermined value, the conversion switching unit 227 may output text information in kana characters without having the dependency analysis unit 225 convert a word (``kousei'') that is determined to have multiple homonyms with a common pronunciation into kanji.

[0039] The minutes creation unit 228 creates minutes for each speaker based on the text information output by the dependency analysis unit 225 or the text acquisition unit 231. The minutes creation unit 228 stores the created minutes and an audio signal corresponding to them in the minutes storage unit 229. The minutes creation unit 228 also outputs the created minutes to the image output unit 241. Note that the minutes creation unit 228 may create the minutes by deleting interjections such as "ah," "er," "uh," and "well," etc.

[0040] The minutes storage unit 229 stores the minutes and the audio signals in association with each other.

[0041] The text acquisition unit 231 acquires the operation result output by the operation unit 202 or the operation result of the operation unit 302 output by the communication unit 204, and generates text information based on the acquired result. The text acquisition unit 231 outputs the generated text information to the minutes creation unit 228.

[0042] The image output unit 241 acquires the minutes information output by the minutes creating unit 228. The image output unit 241 generates image data based on the minutes information and outputs the generated image data to the display unit 203 and the communication unit 204.

[0043] <Display image of parent unit 2> Next, an example of an image displayed on the display unit 203 of the base unit 2 will be described. FIG. 2 is a diagram showing an example of an image displayed on the display unit 203 of the parent device 2 according to this embodiment. An image g10 is the image displayed on the display unit 203 of the parent device 2.

[0044] Area g100 is an area where participant information is edited. Area g101 is an area for participant information. Reference symbol g102 is the name of the participant. Reference symbol g103 is an icon indicating that the participant will input text using the operation unit 202 of the parent unit 2 or the operation unit 302 of the child unit 3. Reference symbol g104 is an icon indicating that the participant will speak using the sound pickup unit 201 of the parent unit 2 or the sound pickup unit 301 of the child unit 3. Reference symbol g105 is the number (or identification information) of the microphone used by the participant.

[0045] Area g200 is an area where the minutes are displayed. Note that FIG. 2 shows the state after login. Symbol g201 is a login / logout button image. Symbol g202 is a start / end button image of the voice recognition text display device (voice recognition text display system) 1. Symbol g203 is a display that lights up while the voice recognition text display device (voice recognition text display system) 1 is in use. Symbol g204 is a button image that displays the minutes stored in the minutes storage unit 229 and plays back audio signals. Symbol g205 is a button image that allows the user of the base unit 2 to select whether or not to use the sound pickup unit 201.

[0046] Reference symbol g211 is text information input by the first participant by operating the operation unit (202 or 302). Reference symbol g212 is a pictogram input by the first participant by operating the operation unit (202 or 302). Reference symbol g213 is information indicating the date and time when the first participant input the text information and pictogram. Reference symbol g214 is the name of the first participant.

[0047] Reference symbol g221 is text information input by the second participant by operating the operation unit (202 or 302). Reference symbol g222 is text information obtained by voice recognition of what was spoken by the second participant. Reference symbol g223 is an icon indicating that text was input by operating the operation unit (202 or 302). Reference symbol g224 is an icon indicating that speech was input using the sound collection unit (201 or 301). Reference symbol g231 is text information obtained by voice recognition of what was spoken by the third participant.

[0048] Reference symbol g241 is an example of displaying text information in kana characters for the word "kousei" uttered by the second participant, according to the recognition likelihood of each of the homonyms "kousei" (construction), "kosei" (steel), and the like. As shown in the illustrated example, the display unit 203 may display a word ("kousei") that has been determined to have multiple homonyms with the same pronunciation through speech recognition and that has been output as text information in kana characters by the dependency analysis unit 225 in a format different from that of other words. Examples of ways to change the format include italics, bold, underlining, highlighting, character color, character size, font, and other aspects, as well as combinations of these. In the illustrated example, "kousei" is displayed in bold and italics, so that "kousei" is displayed in a format different from that of the other text information.

[0049] In order for the display unit 203 to display the above, the image output unit 241 generates image data in which the word is displayed in a format different from that of other words.

[0050] 2 is an example, and is not limited to this example, and the image displayed on the display unit 203 is not limited to this example. For example, the display unit 203 may not display a word ("kousei") that has been determined to have multiple homonyms with the same pronunciation through speech recognition and that has been output as text information in kana characters by the dependency analysis unit 225 in a format different from that of other words.

[0051] <Display screen of handset 3> Next, an example of an image displayed on the display unit 303 of the slave device 3 will be described. FIG. 3 is a diagram showing an example of an image displayed on the display unit 303 of the slave unit 3 according to this embodiment. An image g30 is an image displayed on the display unit 303 of the child device 3.

[0052] Area g300 is an area where the minutes are displayed. Reference symbol g311 is text information input by the first participant by operating the operation unit (202 or 302). Reference symbol g321 is text information input by the second participant by operating the operation unit (202 or 302). Reference symbol g322 is text information obtained by voice recognition of what was spoken by the second participant. Reference symbol g331 is text information obtained by voice recognition of what was spoken by the third participant. Area g301 is an area for text input. Note that the operation unit 302 may be a software keyboard displayed on the display unit 303, and may be connected to the handset 3 by wire or wirelessly.

[0053] Symbol g341 is an example of displaying the word "kousei" uttered by the second participant as text information in kana characters, similar to symbol g241 in FIG. 2. As shown in the illustrated example, similar to the display unit 203, the display unit 303 may display a word ("kousei") that has been determined to have multiple homonyms with a common pronunciation through speech recognition and that has been output as text information in kana characters by the dependency analysis unit 225, in a format different from that of other words. In order for the display unit 303 to perform the above display, the image output unit 241 generates image data in which the word is displayed in a format different from that of other words.

[0054] 3 is an example, and is not limited to this example, and the image displayed on the display unit 303 may not be limited to this example. For example, the display unit 303 may not display a word ("kousei") that has been determined to have multiple homonyms with the same pronunciation through speech recognition and that has been output as text information in kana characters by the dependency analysis unit 225 in a format different from that of other words.

[0055] <Processing performed by the speech recognition text display device (speech recognition text display system) 1> Next, a description will be given of an example of a processing procedure performed by the speech recognition text display device (speech recognition text display system) 1. Fig. 4 is a flowchart showing an example of a processing procedure performed by the speech recognition text display device (speech recognition text display system) 1 according to this embodiment.

[0056] (Step S1) The authentication unit 211 performs login processing based on the operation content of the operation unit (202 or 302). For example, when each user operates the operation unit (202 or 302) to input identification information (user ID) that identifies the user and a password, the authentication unit 211 performs login processing based on the input identification information and password.

[0057] (Step S2) When the user inputs using the sound collection unit (201 or 301), the acquisition unit 222 acquires the voice signal output by the sound collection unit 201 or the communication unit 204, and outputs the acquired voice signal to the voice recognition unit 223.

[0058] (Step S3) The voice recognition unit 223 acquires the voice signal output by the acquisition unit 222, and performs voice recognition processing on the acquired voice signal.

[0059] (Step S4) The text conversion unit 224 performs text conversion processing on the speech recognition result.

[0060] (Step S5) The dependency analysis unit 225 performs dependency analysis and morphological analysis on the converted text information for each speaker.

[0061] (Step S6) The dependency analysis unit 225 determines whether or not there are multiple homonyms with the same pronunciation for each word included in the analyzed text information, with reference to the acoustic model and dictionary storage unit 221. If the dependency analysis unit 225 determines that there are multiple homonyms (step S6; YES), the process proceeds to step S7, and if it determines that there are not multiple homonyms (step S6; NO), the process proceeds to step S11.

[0062] (Step S7) The calculation unit 226 calculates the recognition likelihood for each homonym for a word determined to have multiple homonyms with the same pronunciation, and outputs the calculation result to the conversion switching unit 227.

[0063] (Step S8) The conversion switching unit 227 determines whether the maximum value of the calculated values ​​of the plurality of recognition likelihoods is equal to or less than a predetermined value. If the conversion switching unit 227 determines that the maximum value is equal to or less than the predetermined value (Step S8; YES), the process proceeds to Step S9, and if the conversion switching unit 227 determines that the maximum value exceeds the predetermined value (Step S8; NO), the process proceeds to Step S10.

[0064] (Step S9) The dependency analysis unit 225 outputs the words determined to have multiple homonyms with the same pronunciation to the minutes creation unit 228 as text information in kana characters.

[0065] (Step S10) The dependency analysis unit 225 converts words that are determined to have multiple homonyms with the same pronunciation into the homonym (kanji) with the highest recognition likelihood, and outputs the converted text information to the minutes creation unit 228.

[0066] (Step S11) The dependency analysis unit 225 outputs words that are determined not to have multiple homonyms with the same pronunciation as text information in kanji to the minutes creation unit 228. Note that words that cannot be converted into kanji are output to the minutes creation unit 228 as text information in kana characters.

[0067] (Step S12) When the user inputs data using the operation unit (202 or 302), the text acquisition unit 231 acquires the operation results output by the operation unit 202 or the communication unit 204, generates text information based on the acquired results, and outputs the information to the minutes creation unit 228.

[0068] (Step S13 ) The minutes creating unit 228 creates minutes based on the text information output by the dependency analyzing unit 225 or the text acquiring unit 231 , and outputs the minutes to the image output unit 241 .

[0069] (Step S14) The image output unit 241 generates an image to be displayed on the display unit (203 or 303) based on the minutes output by the minutes creating unit 228, and outputs it to the display unit 203 or the communication unit 204.

[0070] (Step S15) The display unit (203 or 303) displays the image output by the image output unit 241.

[0071] The voice recognition text display device (voice recognition text display system) 1 thereafter repeats the processes of steps S2 to S15. The process in FIG. 4 is an example, and the present invention is not limited to this.

[0072] As described above, in this embodiment, a word having multiple homonyms with the same pronunciation is displayed in kana characters according to the recognition likelihood of each homonym. This embodiment reduces the possibility that text information converted into kanji characters that the speaker does not intend is displayed, making it easier for participants to understand the text information converted from the voice signal.

[0073] In the above example, the voice recognition text display device (voice recognition text display system) 1 allows both text input via the operation unit (202 or 302) and text input using voice recognition via the sound collection unit (201 or 301), but this is not limited to this. For example, the voice recognition text display device (voice recognition text display system) 1 may only allow text input using voice recognition via the sound collection unit (201 or 301).

[0074] In the above example, the voice recognition text display device 1 is provided with a base unit 2 and multiple handset units 3, but this is not limiting. For example, the voice recognition text display device 1 may be provided with only one handset unit 3, or the voice recognition text display device 1 may not be provided with any handset unit 3.

[0075] Furthermore, each of the authentication unit 211, acoustic model and dictionary storage unit 221, acquisition unit 222, speech recognition unit 223, text conversion unit 224, dependency analysis unit 225, calculation unit 226, conversion switching unit 227, minutes creation unit 228, minutes storage unit 229, text acquisition unit 231, and image output unit 241 may be included in the slave unit 3. Similarly, the processing unit 305 may be included in the master unit 2.

[0076] Furthermore, each functional unit of the speech recognition text display system 1 may be provided in a device other than the parent unit 2 and the child unit 3. Alternatively, each functional unit of the speech recognition text display system 1 does not have to be provided in the parent unit 2, the child unit 3, or other physical device, but may be provided on one or more servers or clouds. The functional units are the communication unit 204, the authentication unit 211, the acoustic model and dictionary storage unit 221, the acquisition unit 222, the speech recognition unit 223, the text conversion unit 224, the dependency analysis unit 225, the calculation unit 226, the conversion switching unit 227, the minutes creation unit 228, the minutes storage unit 229, the text acquisition unit 231, the image output unit 241, the communication unit 304, and the processing unit 305.

[0077] A program for implementing all or part of the functions of the speech recognition text display device (speech recognition text display system) 1 of the present invention may be recorded on a computer-readable recording medium, and the program may be loaded into a computer system and executed to perform all or part of the processing performed by the speech recognition text display device (speech recognition text display system) 1. Note that the term "computer system" as used herein includes hardware such as an OS and peripheral devices. The term "computer system" also includes a WWW system equipped with a homepage provision environment (or display environment). The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems. The term "computer-readable recording medium" also includes devices that retain a program for a certain period of time, such as volatile memory (RAM) within a computer system that acts as a server or client when the program is transmitted via a network such as the Internet or a communication line such as a telephone line.

[0078] The program may also be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may also be a program that realizes part of the above-mentioned functions. Furthermore, the program may be a so-called differential file (differential program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

[0079] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0080] 1... Speech recognition text display device (speech recognition text display system) 203... Display unit 222... Acquisition unit 224... Text conversion unit 225... Dependency analysis unit 226... Calculation unit 227... Conversion switching unit 241... Image output unit 303... Display unit

Claims

1. A speech recognition and text display system for converting a speech signal into text information and displaying the text information, comprising: an acquisition unit for acquiring the speech signal; a speech recognition unit that performs speech recognition on the speech signal acquired by the acquisition unit and outputs the text information; a text conversion unit that converts the recognition result output by the speech recognition unit into text information by referring to an acoustic model and dictionary storage unit, and outputs the converted text information; a dependency analysis unit that performs morphological analysis and dependency analysis on the text information output by the text conversion unit by referring to the acoustic model and dictionary storage unit, and outputs the analyzed text information; an operation unit that detects a user's operation and outputs the detection result; a text acquisition unit that acquires the detection result output by the operation unit, generates text information based on the acquired detection result, and outputs the generated text information; a minutes creation unit that creates minutes based on the text information output by the dependency analysis unit or the text information output by the text acquisition unit; a display unit that displays the text information based on the minutes created by the minutes creation unit; Equipped with the speech recognition unit calculates a recognition likelihood for each of the plurality of homonyms for which the speech recognition has determined that there are a plurality of homonyms with a common pronunciation, based on the context before and after the homonym in the text information, and switches between outputting the determined word as the text information converted into kanji or as the text information in kana characters, depending on the magnitudes of the plurality of recognition likelihoods calculated; The text acquisition unit outputs the text information directly to the minutes creation unit without going through the dependency analysis unit, so that the display unit displays the text information converted into kanji in an area where the text information based on the input from the operation unit is displayed, even if there are words with multiple homonyms. Speech recognition text display system.

2. When the maximum value of the calculated plurality of recognition likelihoods is lower than a predetermined value, the speech recognition unit outputs the determined word as the text information in kana characters without converting the determined word into kanji.

2. The speech recognition and text display system of claim 1.

3. When displaying the text information, the display unit displays the determined word and the word output in kana characters by the speech recognition unit in the text information in a format different from that of other words.

3. A speech recognition and text display system according to claim 1.

4. A voice recognition text display device that converts a voice signal into text information and displays it, an acquisition unit that acquires the audio signal; a speech recognition unit that performs speech recognition on the speech signal acquired by the acquisition unit and outputs the text information; a text conversion unit that converts the recognition result output by the speech recognition unit into text information by referring to an acoustic model and dictionary storage unit, and outputs the converted text information; a dependency analysis unit that performs morphological analysis and dependency analysis on the text information output by the text conversion unit by referring to the acoustic model and dictionary storage unit, and outputs the analyzed text information; an operation unit that detects a user's operation and outputs the detection result; a text acquisition unit that acquires the detection result output by the operation unit, generates text information based on the acquired detection result, and outputs the generated text information; a minutes creation unit that creates minutes based on the text information output by the dependency analysis unit or the text information output by the text acquisition unit; a display unit that displays the text information based on the minutes created by the minutes creation unit, the speech recognition unit calculates a recognition likelihood for each of the plurality of homonyms for which the speech recognition has determined that there are a plurality of homonyms with a common pronunciation, based on the context before and after the homonym in the text information, and switches between outputting the determined word as the text information converted into kanji or as the text information in kana characters, depending on the magnitudes of the plurality of recognition likelihoods calculated; The text acquisition unit outputs the text information directly to the minutes creation unit without going through the dependency analysis unit, so that the display unit displays the text information converted into kanji in an area where the text information based on the input from the operation unit is displayed, even if there are words with multiple homonyms. Speech recognition text display device.

5. A speech recognition text display method in a speech recognition text display system that converts a speech signal into text information and displays the text information, comprising: an acquisition step in which an acquisition unit acquires a voice signal; a speech recognition step in which a speech recognition unit performs speech recognition on the speech signal acquired by the acquisition unit and outputs the text information; a text conversion step in which a text conversion unit converts the recognition result output by the speech recognition unit into text information by referring to an acoustic model and dictionary storage unit, and outputs the converted text information; a dependency analysis step in which a dependency analysis unit performs morphological analysis and dependency analysis on the text information output by the text conversion unit by referring to the acoustic model and dictionary storage unit, and outputs the analyzed text information; an operation step in which the operation unit detects an operation by a user and outputs a detection result; a text acquisition step in which a text acquisition unit acquires the detection result output by the operation unit, generates text information based on the acquired detection result, and outputs the generated text information; a minutes creation step in which a minutes creation unit creates minutes based on the text information output by the dependency analysis unit or the text information output by the text acquisition unit; a display step in which a display unit displays the text information based on the minutes created by the minutes creating unit, In the speech recognition step, for a word determined to have a plurality of homonyms with a common pronunciation in the speech recognition, the speech recognition unit calculates a recognition likelihood for each of the plurality of homonyms based on the context before and after the homonym in the text information, and switches between outputting the determined word as the text information converted into kanji or as the text information in kana characters depending on the magnitudes of the plurality of recognition likelihoods calculated; In the text acquisition step, the text acquisition unit outputs the text information directly to the minutes creation unit without going through the dependency analysis unit, so that in the display step, the display unit displays the text information converted into kanji in an area where the text information based on the input from the operation unit is displayed, even if there is a word with multiple homonyms. Speech recognition text display method.

6. A speech recognition and text display system that converts speech signals into text information and displays it. an acquisition step of acquiring an audio signal; a speech recognition step of performing speech recognition on the speech signal acquired in the acquisition step and outputting the text information; a text conversion step of converting the recognition result outputted in the speech recognition step into text information by referring to an acoustic model and dictionary storage unit, and outputting the converted text information; a dependency analysis step of performing morphological analysis and dependency analysis on the text information output in the text conversion step by referring to the acoustic model and dictionary storage unit, and outputting the analyzed text information; an operation step of detecting a user's operation on the operation unit and outputting the detection result; a text acquisition step of acquiring the detection result output in the operation step, generating text information based on the acquired detection result, and outputting the generated text information; a minutes creation step of creating minutes based on the text information output in the dependency analysis step or the text information output in the text acquisition step; a display step of displaying the text information based on the minutes created in the minutes creation step, In the speech recognition step, for a word determined to have a plurality of homonyms with the same pronunciation in the speech recognition, a recognition likelihood is calculated for each of the plurality of homonyms based on the context before and after the homonym in the text information, and the determined word is output either as the text information converted into kanji or as the text information in kana characters depending on the magnitude of the plurality of recognition likelihoods calculated; The text information output in the text acquisition step is directly used in the minutes creation step without going through the dependency analysis step, so that in the display step, even if a word has multiple homonyms, the text information converted into kanji is displayed in the area where the text information based on the input from the operation unit is displayed. program.

Images