Generate AI Utilization Technology

Abstract

Providing user-friendly voice processing equipment utilizing AI-generated speech. [Solution] In the speech processing device 10, the speech recognition unit 20 recognizes the speaker's voice and inputs the speaker's voice as text data to the generation AI 30. The generation AI 30 generates a response sentence to the voice recognized by the speech recognition unit 20 as text data. The speech synthesis unit 40 outputs the response sentence generated by the generation AI 30 as voice. The speech recognition unit 20 accepts voice input even while the speech synthesis unit 40 is outputting voice. The speech recognition unit 20 inputs the character data, which excludes the character data corresponding to the voice output by the speech synthesis unit 40, to the generation AI 30.

Description

Technical Field

[0001] The present invention relates to a technology utilizing generative AI.

Background Art

[0002] The spread of generative AI (Generative Artificial Intelligence), which automatically generates new data and content based on learned data, is rapidly progressing.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By combining existing speech recognition and speech synthesis technologies with generative AI, it is possible to create a voice processing device that generates a response according to a speaker's voice using generative AI and outputs it as voice, and the realization of human-to-human-like communication is expected. In such a voice processing device, when using speech synthesis and speech recognition, they are not used simultaneously, and it is considered possible to suppress malfunction by stopping speech recognition during speech synthesis. However, in a configuration where speech synthesis and speech recognition are performed alternately, the voice processing device cannot utilize speech recognition during speech synthesis, so it cannot stop a conversation midway like a human and listen to the other person's speech, resulting in poor usability.

[0005] Furthermore, by combining existing speech recognition technology with generative AI, it is possible to generate text (such as meeting summaries or minutes) corresponding to the statements made by meeting participants using the generative AI. On the other hand, depending on the meeting environment, the voices of people who are not meeting participants (hereinafter referred to as "external parties") may be included. When the voices of external parties are input to the generative AI, the quality of the text generated by the generative AI deteriorates. Also, for example, if the importance of each participant's statement differs, it is not always best to input the statements of all participants into the generative AI. Against this backdrop, the inventors recognized that in a speech processing device that combines speech recognition and generative AI, the only option available is to input all incoming audio into the generative AI, which is inconvenient to use.

[0006] The present invention aims to provide a user-friendly voice processing device that utilizes generation AI. [Means for solving the problem]

[0007] One aspect of the present invention is Voice recognition unit, A generation AI that generates a response sentence to the speech recognized by the speech recognition unit, The system comprises a speech synthesis unit that outputs the aforementioned response as speech, The aforementioned speech recognition unit, The aforementioned speech synthesis unit accepts speech input even while it is outputting speech. The text data, excluding the text data corresponding to the speech output by the speech synthesis unit, is input to the generation AI. This is a voice processing device that utilizes AI for voice generation.

[0008] Another aspect of the present invention is, Voice recognition unit, The system includes a generation AI that generates text based on the speech recognized by the speech recognition unit, The aforementioned speech recognition unit has a speaker recognition function and inputs character data, excluding character data corresponding to the speech of some of the multiple speakers, into the generating AI. This is a voice processing device that utilizes AI for voice generation. [Effects of the Invention]

[0009] According to the present invention, it becomes possible to provide a user-friendly voice processing device that utilizes generation AI. [Brief explanation of the drawing]

[0010] [Figure 1] Functional block diagram of the audio processing device 10 according to Embodiment 1. [Figure 2] A conceptual diagram illustrating the processing flow when speech recognition and speech synthesis are performed alternately. [Figure 3] A conceptual diagram illustrating the processing flow when performing speech recognition and speech synthesis simultaneously. [Figure 4] Functional block diagram of the voice processing device 10A according to Embodiment 2. [Figure 5] Functional block diagram of the voice processing device 10B according to Embodiment 3. [Modes for carrying out the invention]

[0011] (Embodiment 1) Figure 1 is a functional block diagram of the voice processing device 10 according to Embodiment 1. The functions of each block shown in Figure 1 are realized by a combination of hardware such as a microphone, speaker, and computer, and software. The voice processing device 10 includes a voice recognition unit 20, a generation AI 30, and a voice synthesis unit 40.

[0012] The speech recognition unit 20 recognizes the speaker's voice and inputs the speaker's voice as text data to the generation AI 30. Specifically, for example, the speech recognition unit 20 converts the speaker's voice into an electrical signal using a microphone, converts that electrical signal into text data, and inputs it to the generation AI 30.

[0013] The generating AI 30 generates text data of the response sentence to the speech recognized by the speech recognition unit 20.

[0014] The speech synthesis unit 40 outputs the response sentence generated by the generation AI 30 as speech. Specifically, for example, the speech synthesis unit 40 converts the response sentence, which is text data, into an electrical signal that serves as the basis for speech, and inputs the electrical signal into a speaker to output it as speech.

[0015] The speech recognition unit 20 accepts voice input even while the speech synthesis unit 40 is outputting speech. That is, the speech processing device 10 causes the speech recognition unit 20 and the speech synthesis unit 40 to function simultaneously. In this case, if the speech recognition unit 20 recognizes the speech output by the speech synthesis unit 40 (hereinafter referred to as "AI response speech") and inputs it to the generation AI 30, the output sentence of the generation AI 30 will return to the generation AI 30 (the result of the speech processing device 10 erroneously processing its own speech as an input), which may cause malfunction.

[0016] Therefore, the speech recognition unit 20 inputs character data excluding the character data corresponding to the AI response speech to the generation AI 30. For example, the speech recognition unit 20 has a speaker recognition function and identifies the AI response speech from the input speech. Alternatively, the speech recognition unit 20 compares the recognized speech with the response sentence generated by the generation AI 30 immediately before the recognition to identify the AI response speech. In this case, for example, if the coincidence rate between the two is equal to or higher than a threshold value, it is determined as the AI response speech, and if not, it is determined not to be the AI response speech. As a method for identifying the AI response speech, the above two methods may be used in combination. If the AI response speech is identified, the character data corresponding to the AI response speech can be excluded, and the character data corresponding to the speech of a speaker other than the speech processing device 10 can be extracted.

[0017] When using the speaker recognition function to exclude the character data corresponding to the AI response speech, by registering the AI response speech as an exclusion target in advance, for example, by measuring and storing the feature amount of the AI response speech, the speech recognition unit 20 can smoothly identify the AI response speech.

[0018] Figure 2 is a conceptual diagram showing the processing flow when speech recognition and speech synthesis are performed alternately, in the case of a comparative example. When speech recognition and speech synthesis are performed alternately, the function of the speech recognition unit 20 is stopped while the speech synthesis unit 40 is outputting speech, so the speech recognition unit 20 does not recognize the AI ​​response voice. However, even if the user speaks while the speech synthesis unit 40 is outputting speech, the speech recognition unit 20 cannot recognize the user's voice, which is inconvenient.

[0019] Figure 3 is a conceptual diagram showing the processing flow when speech recognition and speech synthesis are performed simultaneously in an embodiment. When speech recognition and speech synthesis are performed simultaneously, the speech recognition unit 20 recognizes not only the user's voice but also the AI ​​response voice. However, it identifies the AI ​​response voice and inputs the character data that excludes the character data corresponding to that voice to the generation AI 30. This suppresses a malfunction loop in which the generation AI 30 outputs a sentence according to the output sentence of the generation AI 30.

[0020] This embodiment provides the following effects and advantages.

[0021] (1) The speech recognition unit 20 accepts speech input even while the speech synthesis unit 40 is outputting speech. Therefore, even while the speech synthesis unit 40 is outputting speech, if the user speaks, the speech processing unit 10 can stop outputting speech from the speech synthesis unit 40 and listen to what the user is saying. Thus, conversation between the user and the speech processing unit 10 becomes smoother and easier to use.

[0022] (2) The speech recognition unit 20 inputs character data excluding the character data corresponding to the AI ​​response voice to the generating AI 30. This suppresses malfunctions caused by the output sentence of the generating AI 30 being returned to the generating AI 30. As a result, the conversation between the user and the speech processing device 10 becomes more accurate and smoother, and is easier to use.

[0023] (3) Advanced speech recognition systems typically include a standard feature for recognizing the speaker from voice characteristics, and the additional load associated with excluding text data corresponding to AI response speech using the speaker recognition function is limited.

[0024] (Embodiment 2) Figure 4 is a functional block diagram of the voice processing device 10A according to Embodiment 2. The following explanation will focus on the differences from Embodiment 1. The voice processing device 10A is the same as the voice processing device 10 in Figure 1, but without the voice synthesis unit 40.

[0025] Various types of speech are input to the speech recognition unit 20. Among the input speech, there may be speech that does not need to be input to the generation AI 30, or that is preferable not to input (hereinafter referred to as "unnecessary speech"). Unnecessary speech is, for example, the speech of an outsider in a meeting. Alternatively, in cases where the importance of what each participant says differs, the speech of a participant whose speech is of low importance is also an example of unnecessary speech. Note that meeting participants and outsiders are just examples of speakers.

[0026] The speech recognition unit 20 inputs the generated text data, with the text data corresponding to unwanted speech excluded, into the generating AI 30. By registering the voices of participants that should be targeted for speech recognition, i.e., voices other than unwanted speech (hereinafter referred to as "necessary speech"), as extraction targets, for example by measuring and storing the feature quantities of the necessary speech, the speech recognition unit 20 can smoothly identify unwanted speech. Once the unwanted speech is identified, the text data corresponding to the unwanted speech can be excluded, and the text data corresponding to the necessary speech can be extracted.

[0027] The generating AI 30 generates text (such as meeting summaries or minutes) based on the speech recognized by the speech recognition unit 20, i.e., the character data corresponding to the required speech, and outputs it as text data.

[0028] According to this embodiment, the speech recognition unit 20 can input character data excluding character data corresponding to unnecessary speech to the generation AI 30. This makes it easier to use compared to the case where the only option is to input all input speech to the generation AI 30. In addition, it suppresses the deterioration of the quality of the sentences generated by the generation AI 30 due to the input of external voices. Furthermore, it allows the generation AI 30 to generate sentences based only on the voices of participants whose speeches are of high importance, thereby improving the quality of the sentences generated by the generation AI 30. Moreover, reducing the amount of character data input to the generation AI 30 reduces the load on the generation AI 30.

[0029] (Embodiment 3) Figure 5 is a functional block diagram of the voice processing device 10B according to Embodiment 3. The following explanation will focus on the differences from Embodiment 2. The voice processing device 10B is the voice processing device 10A of Figure 4 with the addition of a voice synthesis unit 40. The voice synthesis unit 40 has the same functions as in Embodiment 1.

[0030] In this embodiment, by making the speech output by the speech synthesis unit 40 unnecessary speech, malfunctions caused by the output sentence of the generation AI 30 being returned to the generation AI 30 (the speech processing device 10B mistakenly processing its own utterance as input) are suppressed.

[0031] In addition to the effects and advantages of Embodiment 2, this embodiment provides the following effects and advantages.

[0032] (1) The speech synthesis unit 40 outputs the sentence generated by the generation AI 30 based on the required speech as speech. Therefore, it is possible to respond to statements made by participants who speak the required speech (hereinafter referred to as "important participants") during the meeting, thereby improving convenience.

[0033] (2) Since the text data corresponding to unwanted speech is not input to the generating AI 30, the speech synthesis unit 40 does not respond to unwanted speech, and the disruption of the meeting due to responses to unwanted speech is suppressed.

[0034] (3) The speech recognition unit 20 accepts speech input even while the speech synthesis unit 40 is outputting speech. Therefore, even while the speech synthesis unit 40 is outputting speech, if an important participant speaks, the speech processing device 10 can stop the speech output from the speech synthesis unit 40 and listen to what the important participant is saying. Thus, there is no need to wait for the speech synthesis unit 40 to finish outputting speech, making it easy to use.

[0035] Although the present invention has been described above using embodiments as examples, the present invention is not limited to these embodiments. Various modifications are possible to each of the matters specifically described in the embodiments within the scope of the claims. [Explanation of symbols]

[0036] 10, 10A, 10B...Speech processing unit, 20...Speech recognition unit, 30...Generating AI, 40...Speech synthesis unit.

Claims

1. Voice recognition unit, A generation AI that generates a response sentence to the speech recognized by the speech recognition unit, The system comprises a speech synthesis unit that outputs the aforementioned response as speech, The aforementioned speech recognition unit, The aforementioned speech synthesis unit accepts speech input even while it is outputting speech. The text data, excluding the text data corresponding to the speech output by the speech synthesis unit, is input to the generating AI. A voice processing device utilizing generative AI.

2. The speech recognition unit has a speaker recognition function and identifies the speech output by the speech synthesis unit from the input speech, as described in claim 1, a speech processing device utilizing generation AI.

3. The speech recognition unit compares the recognized speech with the response sentence generated by the generation AI immediately before recognition, and identifies the speech output by the speech synthesis unit, as described in claim 1, for the speech processing device utilizing the generation AI.

4. Voice recognition unit, The system includes a generation AI that generates sentences based on the speech recognized by the speech recognition unit, The aforementioned speech recognition unit has a speaker recognition function and inputs character data, excluding character data corresponding to the speech of some of the multiple speakers, into the generating AI. A voice processing device utilizing generative AI.

5. The speech processing device utilizing generating AI according to claim 4, comprising a speech synthesis unit that outputs the aforementioned sentence as speech.

Images