Streaming audio generation system

Abstract

Providing a user-friendly streaming audio generation system. [Solution] The streaming speech generation system 10 includes a speech recognition unit 20 that inputs the speaker's voice as text data to a generation AI 30, a generation AI 30 that generates a response sentence to the voice recognized by the speech recognition unit 20 as text data, a splitting unit 35 that receives the response sentence (text data) output by the generation AI 30 as streaming data, and sequentially divides it into short sentences and outputs them in real time, and a speech synthesis unit 40 that outputs the short sentences output by the splitting unit 35 as speech.

Description

Technical Field

[0001] The present invention relates to a stream voice generation system utilizing generative AI.

Background Art

[0002] The spread of generative AI (Generative Artificial Intelligence), which automatically generates new data and content based on learned data, is progressing rapidly.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By combining existing speech recognition and speech synthesis technologies with generative AI, it is possible to generate a response according to the speaker's voice using generative AI and output it as voice, and the realization of human-to-human-like communication is expected. On the other hand, since generative AI performs processing based on a vast amount of data, it may take time to generate a response sentence. If the output of the response voice to the user's utterance is delayed, the communication will not be smooth and the usability will be poor.

[0005] An object of the present invention is to provide a user-friendly stream voice generation system utilizing generative AI.

Means for Solving the Problems

[0006] One aspect of the present invention is a speech recognition unit, a generative AI that generates a response sentence to the voice recognized by the speech recognition unit, a splitting unit that receives the response sentence output by the generative AI as streaming data, splits it into short sentences, and sequentially outputs them The system includes a speech synthesis unit that outputs the short sentence output by the division unit as speech. It is a streaming audio generation system. [Effects of the Invention]

[0007] According to the present invention, it becomes possible to provide a user-friendly stream audio generation system that utilizes generation AI. [Brief explanation of the drawing]

[0008] [Figure 1] Functional block diagram of the stream audio generation system 10 according to the embodiment. [Figure 2] A flowchart showing an example of the division process flow in the division section 35 of Figure 1. [Figure 3] A conceptual diagram showing an example of the text splitting process by the splitting unit 35. [Modes for carrying out the invention]

[0009] Figure 1 is a functional block diagram of a stream speech generation system 10 (speech processing device) according to an embodiment. The function of each block shown in Figure 1 is realized by a combination of hardware such as a microphone, speaker, and computer, and software. The stream speech generation system 10 includes a speech recognition unit 20, a generation AI 30, a splitting unit 35, and a speech synthesis unit 40.

[0010] The speech recognition unit 20 recognizes the speaker's voice and inputs the speaker's voice as text data to the generation AI 30. Specifically, for example, the speech recognition unit 20 converts the speaker's voice into an electrical signal using a microphone, converts that electrical signal into text data, and inputs it to the generation AI 30.

[0011] The generating AI 30 generates text data of the response sentence to the speech recognized by the speech recognition unit 20.

[0012] The splitting unit 35 is, for example, a splitting AI, which receives the response sentence (text data) output by the generating AI 30 as streaming data, and sequentially splits it into short sentences and outputs them in real time. The splitting unit 35, for example, uses AI to perform morphological analysis on the received sentence to segment it, and estimates the points to be split based on the types of characters, punctuation marks, and phrases that frequently appear at the points to be split. After that, it cuts the sentence at commas, periods when the minimum number of characters is exceeded, or other points to be split, creating natural short sentences.

[0013] The speech synthesis unit 40 outputs the short sentences output by the splitting unit 35 as speech. Specifically, for example, the speech synthesis unit 40 converts the short sentences, which are text data, into electrical signals that form the basis of speech, and inputs these electrical signals to a speaker to output them as speech.

[0014] The speech recognition unit 20 accepts speech input even while the speech synthesis unit 40 is outputting speech. In other words, the stream speech generation system 10 operates the speech recognition unit 20 and the speech synthesis unit 40 simultaneously. In this case, when the speech recognition unit 20 recognizes the speech output by the speech synthesis unit 40 (hereinafter referred to as "AI response speech") and inputs it to the generation AI 30, the output sentence of the generation AI 30 is returned to the generation AI 30 (resulting in the stream speech generation system 10 mistakenly processing its own utterance as input), which may cause malfunctions.

[0015] The speech recognition unit 20 then inputs the character data, excluding the character data corresponding to the AI ​​response voice, into the generating AI 30. For example, the speech recognition unit 20 has a speaker recognition function and identifies the AI ​​response voice from the input voice. Alternatively, the speech recognition unit 20 identifies the AI ​​response voice by comparing the recognized voice with the response sentence generated by the generating AI 30 immediately before recognition. The above two methods may be used in combination as a method for identifying the AI ​​response voice. Once the AI ​​response voice is identified, the character data corresponding to the AI ​​response voice can be excluded, and character data corresponding to the voice of a speaker other than the stream voice generation system 10 can be extracted.

[0016] When excluding character data corresponding to the AI response voice using the speaker recognition function, it is necessary to register the AI response voice as an exclusion target in advance. For example, by measuring and storing the feature amounts of the AI response voice, the voice recognition unit 20 can smoothly identify the AI response voice.

[0017] FIG. 2 is a flowchart showing an example of the flow of the splitting process in the splitting unit 35.

[0018] The splitting unit 35 receives the streaming data of the response sentence output by the generation AI 30 (S1). The splitting unit 35 inputs the received streaming data (hereinafter referred to as "received data") into its own AI model (S3), and estimates the splitting points by the AI (S5). The splitting unit 35 estimates appropriate splitting points based on, for example, the character types, punctuation marks, or phrases frequently appearing at the locations to be split within the sentence of the received data.

[0019] If the number of characters in the short sentence when the received data is split at the splitting points estimated by the AI (hereinafter referred to as "AI estimated points") exceeds the minimum number of characters, the splitting unit 35 splits the received data at the AI estimated points ( "Yes" in S7, S9). If the number of characters in the short sentence does not exceed the minimum number of characters, the splitting unit 35 splits the received data at the first punctuation mark exceeding the minimum number of characters ( "No" in S7, S11). The natural short sentences thus extracted are sent to the speech synthesis unit 40 in real time (S13) and immediately output as speech.

[0020] FIG. 3 is a conceptual diagram showing an example of the text splitting process by the splitting unit 35.

[0021] The splitting unit 35 uses, for example, feature - based scoring. Specifically, for example, the following feature amounts are considered to judge the splitting points. · The character itself: Whether the current character and the previous character correspond to a specific pattern (e.g., a specific phrase or character type). · Character type: The type of characters such as Chinese characters, hiragana, katakana, alphabet, numbers, etc. • Split status: Previous split status (whether it was split or continued).

[0022] The splitting unit 35 applies pre-learned weights to these features and calculates a score (probability). If the score (probability) is positive, it is determined to be a splitting point, and the word is split. This method is computationally intensive and therefore suitable for real-time processing.

[0023] This embodiment provides the following effects and advantages.

[0024] (1) The splitting unit 35 receives the response sentence (text data) output by the generating AI 30 as streaming data, and sequentially splits it into short sentences and sends them to the speech synthesis unit 40 in real time. As a result, the delay in the response voice to the user's utterance is suppressed, communication becomes smoother, and it is easier to use. Specifically, the user can proceed with the conversation smoothly without waiting for the generating AI 30 to finish generating the response sentence, enabling quick and natural communication and improving the user experience.

[0025] (2) The splitting unit 35 appropriately splits the response sentence output by the generating AI 30 into natural short sentences, thereby suppressing the interruption of the output voice of the speech synthesis unit 40 at unnatural positions, making it easy to use.

[0026] (3) The speech recognition unit 20 accepts speech input even while the speech synthesis unit 40 is outputting speech. Therefore, even while the speech synthesis unit 40 is outputting speech, if the user speaks, the stream speech generation system 10 can stop outputting speech from the speech synthesis unit 40 and listen to what the user is saying. Thus, the interaction between the user and the stream speech generation system 10 becomes smoother and easier to use.

[0027] (4) The speech recognition unit 20 inputs character data excluding the character data corresponding to the AI ​​response voice to the generation AI 30. This suppresses malfunctions caused by the output sentence of the generation AI 30 being returned to the generation AI 30. As a result, the interaction between the user and the stream speech generation system 10 becomes more accurate and smoother, and easier to use.

[0028] Although the present invention has been described above using embodiments as examples, the present invention is not limited to these embodiments. Various modifications are possible to each of the matters specifically described in the embodiments within the scope of the claims. [Explanation of Symbols]

[0029] 10...Stream voice generation system (voice processing device), 20...Speech recognition unit, 30...Generating AI, 35...Splitting unit, 40...Speech synthesis unit.

Claims

1. Voice recognition unit, A generation AI that generates a response sentence to the speech recognized by the speech recognition unit, A splitting unit receives the response sentence output by the generating AI as streaming data, divides it into short sentences, and outputs them sequentially. The system includes a speech synthesis unit that outputs the short sentence output by the division unit as speech. Streaming audio generation system.

2. The aforementioned speech recognition unit, The aforementioned speech synthesis unit accepts speech input even while it is outputting speech. The text data, excluding the text data corresponding to the speech output by the speech synthesis unit, is input to the generating AI. The stream audio generation system according to claim 1.

Images