Speaker prediction method, speaker prediction apparatus, and communication system

By acquiring speech and conversation history, combining speech features and facial recognition, and using neural network algorithms to predict the next speaker, the problem of insufficient prediction accuracy in existing technologies is solved, and high-precision speaker prediction is achieved.

CN114400013BActive Publication Date: 2026-02-27YAMAHA CORP
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202111170526.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2020-10-09
Filing Date
2021-10-08
Publication Date
2026-02-27
Estimated Expiration
2041-10-08

AI Technical Summary

Technical Problem

Existing technologies struggle to accurately predict the next speaker while the current speaker is speaking.

Method used

By acquiring the speech and conversation history of multiple speakers, and utilizing speech features, facial recognition, and conversation history records, combined with neural network algorithms, the system predicts the next speaker to speak, and improves prediction accuracy through beamforming processing and image control.

Benefits of technology

It achieves high-precision prediction of the next speaker while the current speaker is speaking, improving the accuracy and real-time performance of the prediction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114400013B_ABST
    Figure CN114400013B_ABST
Patent Text Reader

Abstract

The present application provides a speaker prediction method, a speaker prediction device, and a communication system. The next speaker to speak can be predicted with high accuracy during the current speaker is speaking. The speaker prediction method is a speaker prediction method for acquiring voices of a plurality of speakers and recording conversation histories of the plurality of speakers, the speaker prediction method acquiring voices, determining speakers of the acquired voices, predicting a next speaker to speak from the plurality of speakers based on the determined speakers and the conversation histories.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] One embodiment of the present application relates to a speaker prediction method, a speaker prediction device, and a communication system for predicting a speaker. BACKGROUND

[0002] In Patent Literature 1, a prediction device that predicts a speaker who will speak next from among a plurality of persons is disclosed.

[0003] PRIOR ART DOCUMENTS

[0004] PATENT LITERATURE

[0005] Patent Literature 1: Japanese Patent Application Publication No. 2012-29209 SUMMARY

[0006] PROBLEMS TO BE SOLVED BY THE INVENTION

[0007] It is desirable to predict a speaker who will speak next with higher accuracy during a period in which a current speaker is speaking.

[0008] Therefore, an object of one embodiment of the present application is to provide a speaker prediction method, a speaker prediction device, and a communication system that can predict a speaker who will speak next with high accuracy during a period in which a current speaker is speaking.

[0009] MEANS FOR SOLVING PROBLEMS

[0010] The speaker prediction method is a speaker prediction method that acquires voices of a plurality of speakers and records a conversation history of the plurality of speakers, acquires a voice, determines a speaker of the acquired voice, and predicts a speaker who will speak next from among the plurality of speakers on the basis of the determined speaker and the conversation history.

[0011] EFFECTS OF THE INVENTION

[0012] According to one embodiment of the present application, a speaker who will speak next can be predicted with high accuracy during a period in which a current speaker is speaking. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 is a block diagram that shows the structure of the sound signal processing device 1.

[0014] Figure 2 is a functional block diagram of the sound signal processing device 1.

[0015] Figure 3 is a functional block diagram of the sound signal processing device 1.

[0016] Figure 4 is a functional block diagram of the sound signal processing device 1.

[0017] Figure 5 is a flowchart showing the operation of the sound signal processing method.

[0018] Figure 6 is a flowchart showing the operation of the sound signal processing method.

[0019] Figure 7 is a flowchart showing the operation of the sound signal processing method.

[0020] Figure 8 is a diagram showing an example of an image captured by the camera 11.

[0021] Figure 9 is a diagram showing an example of a conversation history.

[0022] Figure 10 is a block diagram showing the functional structure of the sound signal processing apparatus 1 in a case where sound signal processing is performed. DETAILED DESCRIPTION

[0023] Figure 1 is a block diagram showing the structure of the sound signal processing apparatus 1. The sound signal processing apparatus 1 includes a camera 11, a CPU 12, a DSP 13, a flash memory 14, a RAM 15, a user interface (I / F) 16, a speaker 17, six microphones 18A to 18F, and a communication section 19. Note that, in the present embodiment, a signal refers to a digital signal.

[0024] The camera 11, the speaker 17, and the microphones 18A to 18F are arranged, for example, above or below a display (not shown). The camera 11 acquires an image of a user located in front of the display (not shown). The microphones 18A to 18F acquire a voice of the user located in front of the display (not shown). The speaker 17 outputs a voice to the user located in front of the display (not shown). Note that the number of microphones is not limited to six. The microphone can be one microphone. The number of microphones in the present embodiment is six, and constitutes an array microphone. The DSP 13 performs a beamforming process on a sound signal acquired by the microphones 18A to 18F.

[0025] The CPU 12 functions as a control section that uniformly controls the operation of the sound signal processing apparatus 1 by reading out a program for operation from the flash memory 14 to the RAM 15. Note that the program does not need to be stored in the flash memory 14 of the apparatus in advance. The CPU 12 can also be read out to the RAM 15 by downloading from a server or the like each time, for example.

[0026] The DSP 13 is a signal processing section that processes the video signal and the sound signal respectively under the control of the CPU 12. The DSP 13 functions as an image processing section that performs, for example, framing processing for cutting out an image of a speaker from the video signal. In addition, the DSP 13 functions as a sound signal processing section for performing, for example, beamforming.

[0027] The communication section 19 transmits the video signal and the sound signal on the near-end side processed by the DSP 13 to another device. The other device refers to a sound signal processing device on the far-end side connected via the Internet or the like. In addition, the communication section 19 receives a video signal and a sound signal from the other device. The communication section 19 outputs the received video signal to a display (not shown). The communication section 19 outputs the received sound signal to the speaker 17. The display displays an image acquired by a camera of the other device. The speaker 17 outputs a voice of a speaker acquired by a microphone of the other device. Thus, the sound signal processing device 1 functions as a communication system for performing a voice conversation at a remote place.

[0028] Figure 2 Figure 3 and Figure 4 is a functional block diagram of the sound signal processing device 1. Figure 5 Figure 6 and Figure 7 is a flowchart showing the operation of the sound signal processing method. Figure 2 Figure 3 and Figure 4 The functional structure shown in FIGS. 1 to 4 is realized by the CPU 12 and the DSP 13.

[0029] The sound signal processing device 1 has, in terms of functions, a voice acquisition section 100, an image acquisition section 101, a near-end speaker determination section 102, a conversation history recording section 103, a far-end speaker determination section 104, a camera image control section 105, and a prediction section 51.

[0030] The near-end speaker determination section 102 and the far-end speaker determination section 104 correspond to the speaker determination section of the present application. Figure 2 shows a structure in which speaker determination is performed on the basis of a video signal and a sound signal on the near-end side and recorded as a conversation history, Figure 5 shows the operation in this case.

[0031] ​​​The voice acquisition unit 100 inputs sound signals from microphones 18A to 18F as proximal-side sound signals (S11). Furthermore, the image acquisition unit 101 acquires image signals from camera 11 as a proximal-side speaker image (S12). The proximal-side speaker determination unit 102 determines the proximal-side speaker based on the proximal-side sound signals and the proximal-side speaker image (S13). The conversation history recording unit 103 records the identifier of the speaker determined by the proximal-side speaker determination unit 102, along with their vocalization order, as conversation history (S14).

[0032] The near-end speaker determination unit 102 determines the speaker based on, for example, speech features. Speech features are specific information that varies from speaker to speaker. Speech features include, for example, average volume (power), highest pitch frequency, lowest pitch frequency, or speech rate. The near-end speaker determination unit 102 determines the speaker by extracting speech features from the sound signal, for example, using a prescribed algorithm such as a neural network. Alternatively, the near-end speaker determination unit 102 can also determine the speaker based on the direction of arrival of the speech. For example, the near-end speaker determination unit 102 can determine the timing of the speaker's speech arrival at the microphones by calculating the correlation between sound signals acquired by multiple microphones. The near-end speaker determination unit 102 can determine the direction of arrival of the speaker's speech based on the positional relationship of each microphone and the timing of the speech arrival.

[0033] Alternatively, the near-end speaker determination unit 102 can also determine the speaker by performing speaker face recognition processing. Speaker face recognition processing is a process that identifies the position of the speaker's face based on the near-end speaker image by employing a prescribed algorithm, such as using a neural network.

[0034] Figure 8 This is an example diagram showing an image captured by camera 11. Figure 8 In the example, camera 11 captures facial images of multiple users located along the length (depth) of table T.

[0035] Viewed from above, table T is rectangular. Camera 11 captures images of four users located on the left and right sides of table T along its shorter side, as well as users located further away from table T.

[0036] The near-speaker identification unit 102 identifies the speaker's face based on such an image captured by the camera 11. Figure 8 In the example, user A1, located in the lower left corner of the image, is speaking. The near-speaker determination unit 102, based on multiple frames of images, identifies facial images, such as those showing changes in the mouth image, as the speaker's face. Additionally, in Figure 8In the example of FIG. 9, the other users A2 to A5 are being subjected to face recognition, but are not the speaker. Therefore, the near-speaker determining section 102 recognizes the face of the user Al as the face of the speaker.

[0037] In addition, the near-speaker determining section 102 can also set a bounding box (Bounding Box) as shown by the quadrangle in the figure at the position of the recognized face of the speaker. The near-speaker determining section 102 can determine the distance to the speaker based on the size of the bounding box. A table or function or the like that indicates the relationship between the size of the bounding box and the distance is stored in advance in the flash memory 14. The near-speaker determining section 102 can also compare the size of the set bounding box with the table stored in the flash memory 14 to find the distance to the speaker.

[0038] The near-speaker determining section 102 finds the two-dimensional coordinates (X, Y coordinates) of the set bounding box and the distance to the speaker as the position information of the speaker. The conversation history recording section 103 also records the position information of the speaker.

[0039] Figure 3 a configuration in which speaker determination is performed based on an image signal and a sound signal on the far side and the speaker is recorded as a conversation history, Figure 6 an action in this case. The voice acquiring section 100 receives the far-side sound signal via the communication section 19 (S21). In addition, the image acquiring section 101 receives the far-side speaker image via the communication section 19 (S22). The far-speaker determining section 104 determines the speaker on the far side based on the far-side sound signal and the far-side speaker image (S23). The conversation history recording section 103 records the identifier of the speaker determined by the far-speaker determining section 104 and the order thereof as a conversation history (S24).

[0040] The method of speaker determination by the far-speaker determining section 104 is the same as the method of speaker determination by the near-speaker determining section 102. The far-speaker determining section 104 determines the speaker based on, for example, voice feature amounts. Alternatively, the far-speaker determining section 104 can also determine the speaker by performing face recognition processing of the speaker from the received far-side speaker image.

[0041] Furthermore, in the case where the device on the far side is a device that can determine the speaker as well as the present device, the far-speaker determining section 104 can also receive information for determining the speaker from the device on the far side. In this case, the far-speaker determining section 104 can determine the speaker on the far side based on the received information.

[0042] Further, the near-end speaker determiner 102 and the far-end speaker determiner 104 do not necessarily determine a plurality of utterances (utterances) of a certain speaker to be a single speaker (utterances of the same speaker). For example, even if the utterances are similar in voice feature amount, the near-end speaker determiner 102 and the far-end speaker determiner 104 can determine them to be different speakers. The near-end speaker determiner 102 and the far-end speaker determiner 104 can also determine them to be different speakers in a case where the voice feature amounts and the like are slightly different. Even if it is assumed that the near-end speaker determiner 102 and the far-end speaker determiner 104 determine the same speaker to be different speakers and record the conversation history, the prediction unit 51 is able to predict the next speaker. That is, the near-end speaker determiner 102 and the far-end speaker determiner 104 can also determine the speaker based on an element that excessively subdivides to a degree at which the next speaker can be predicted or a degree at which one speaker is determined to be different speakers. Further, the near-end speaker determiner 102 and the far-end speaker determiner 104 can also record the voice feature amount of the speaker itself, the arrival direction of the voice of the speaker itself, the face image of the speaker itself, or the position information of the speaker itself in the conversation history as a result of speaker determination.

[0043] Further, in the present embodiment, an example in which the device on the far-end side is one and communication of 1 to 1 is performed is shown, but the sound signal processing device 1 also performs communication with devices on the far-end side of multiple sites and functions as a multi-site communication system. In a case where communication with devices on the far-end side of multiple sites is performed, the far-end speaker determiner 104 performs processing for determining a speaker for each device on the far-end side and determines all of the far-end speakers.

[0044] Figure 9 is a diagram showing an example of a conversation history. The conversation history recording unit 103 records the conversation history in chronological order. For example, in the example of Figure 9 from time t1 to time t2, the near-end speaker Al utters; from time t2 to time t3, the far-end speaker Bl utters. From time t3 to time t4, the near-end speaker A2 utters; from time t4 to time t5, the far-end speaker B2 utters. From time t5 to time t6, the near-end speaker Al utters; from time t6 to time t7, the far-end speaker Bl utters. From time t7 to time t8, the near-end speaker A2 utters; from time t8 to time t9, the far-end speaker Bl utters. Thereafter, from time t9 onward, the near-end speaker A4 utters. Further, recording of the time information is not necessary. The conversation history recording unit 103 can also record only an identifier showing who the speaker is and the order (number) of the speaker.

[0045] The sound signal processing apparatus 1 predicts the speaker who will utter next based on the conversation history. The sound signal processing apparatus 1 functions as a speaker prediction apparatus. Figure 4 The functional configuration of the sound signal processing apparatus 1 in the case of making a prediction, Figure 7 The action in this case is shown. Figure 7 The example of the case where the speaker who is currently on the remote side is uttering shows the action of the prediction (the action of the speaker prediction method) in this case.

[0046] The speech acquisition section 100 receives the remote-side sound signal via the communication section 19 (S31). In addition, the image acquisition section 101 receives the remote-side speaker image via the communication section 19 (S32). The remote speaker determination section 104 determines the speaker on the remote side based on the remote-side sound signal and the remote-side speaker image (S33). In addition, in the case where the apparatus on the remote side is an apparatus that can determine the speaker as well as the present apparatus, the remote speaker determination section 104 can also receive information for determining the speaker from the apparatus on the remote side. In this case, the remote speaker determination section 104 can determine the speaker on the remote side based on the received information.

[0047] Thereafter, the prediction section 51 refers to the conversation history recorded by the conversation history recording section 103 (S34), and predicts the speaker who will utter next (S35). Specifically, the prediction section 51 detects the position where the speaker on the remote side determined by the remote speaker determination section 104 utters, from the conversation history, and predicts the speaker who will utter next in accordance with the utterance probability of the speaker at the detected position. For example, in the case where the remote speaker determination section 104 determines the speaker Bl on the remote side, the prediction section 51 detects t2-t3, t6-t7, and t8-t9 in the conversation history shown in FIG. 8. Then, the prediction section 51 calculates the utterance probability of each speaker based on the history of the speaker who will utter next at these times. In the example of FIG. 8, the speaker A2 utters twice and the speaker A4 utters once after the speaker Bl utters. Therefore, the utterance probability of the speaker A2 is 66.7%, and the utterance probability of the speaker A4 is 33.3%. The prediction section 51 predicts that the speaker who will utter next is A2 in accordance with the utterance probabilities. Figure 9 Figure 9

[0048] ​​Further, in a case where the proximal speaker determination section 102 or the distal speaker determination section 104 records the speaker's voice feature amount itself, the speaker's voice arrival direction itself, the speaker's face image itself, or the speaker's position information itself, or the like as the speaker determination result in the conversation history, the prediction section 51 retrieves the speaker determination results close to the current speaker determination result from the conversation history recording section 103. Then, the prediction section 51 can predict the next speaker to utter based on the plurality of speaker determination results recorded next to the time when the current speaker determination result is recorded, from the speaker determination results recorded in the conversation history recording section 103. In this case, for example, the prediction section 51 can set the average of the plurality of speaker determination results recorded next to the time when the current speaker determination result is retrieved as the result of the speaker prediction. Alternatively, the prediction section 51 can set the speaker determination result in which the sum of the difference amounts from other speaker prediction results is the minimum as the representative value as the result of the speaker prediction.

[0049] Further, the prediction section 51 can also predict the next speaker to utter based on the image received by the communication section 19 or the image captured by the camera 11. For example, it is predicted that the person who raises his or her hand is the next speaker to utter. Alternatively, it is predicted that the person who opens his or her eyes is the next speaker to utter. Further, the prediction section 51 can also predict the next speaker to utter based on the sound signal received by the communication section 19 or the sound signal acquired by the microphones 18A to 18F. For example, the prediction section 51 predicts that the corresponding speaker utters next in a case where a specific utterance such as "aa" or "ee" is recognized. However, the prediction section 51 gives priority to the prediction based on the conversation history. Preferably, when there are a plurality of speakers with the same or close (for example, within 10%) utterance probabilities, the prediction section 51 also predicts the next speaker to utter based on the received image.

[0050] Thereafter, the camera image control section 105 performs control of the image captured by the camera 11 in accordance with the prediction result of the prediction section 51 (S36). The control of the image is, for example, framing processing. The camera image control section 105 performs framing processing that cuts out the face image of the speaker A4 from the image of the camera 11 acquired by the image acquisition section 101. The positions of the respective speakers are recorded in the conversation history recording section 103. The camera image control section 105 cuts out the image of the position corresponding to the predicted speaker. The camera image control section 105 outputs the image after the framing processing to the communication section 19. The communication section 19 transmits the image after the framing processing to the distal side.

[0051] Thus, the face image of the speaker A2 is displayed on the display (not shown) on the far-end side. Therefore, the speaker Bl on the far-end side can make a speech while looking at the speaker who is likely to speak next.

[0052] Further, in the above, the framing processing performed by the camera image control section 105 to cut out the image of the predicted speaker from the image captured by the camera 11 is shown, but the camera image control section 105 can also perform a blur processing on a position other than the predicted speaker, for example. Further, the camera image control section 105 can be controlled to perform zoom-in capturing of the face of the predicted speaker by controlling the capturing direction and zoom of the camera 11.

[0053] Further, in a case where the near-end speaker determination section 102 determines the speaker on the near-end side, the camera image control section 105 performs the framing processing to cut out the determined speaker on the near-end side. Therefore, the image of the speaker who is currently speaking is transmitted to the device on the far-end side.

[0054] However, even in a case where the near-end speaker determination section 102 determines the speaker on the near-end side, the prediction section 51 can predict the speaker who is next to speak. The camera image control section 105 can cut out the image of the predicted speaker who is next to speak and transmit it to the device on the far-end side via the communication section 19. In a case where the prediction section 51 predicts that the speaker who is next to speak is the speaker on the near-end side, the camera image control section 105 performs the framing processing to cut out the face image of the speaker on the near-end side from the image of the camera 11. The camera image control section 105 transmits the cut-out image of the speaker on the near-end side to the device on the far-end side.

[0055] In a case where the prediction section 51 predicts that the speaker who is next to speak is the speaker on the far-end side, the camera image control section 105 performs the framing processing to cut out the face image of the speaker on the far-end side from the image received via the communication section 19. The camera image control section 105 displays the cut-out image of the speaker on the far-end side on the display (not shown).

[0056] However, in a case where the camera image control section 105 transmits the cut-out image of the speaker on the near-end side to the device on the far-end side, it is preferable to periodically switch the image of the speaker on the near-end side who is currently speaking and the image of the speaker on the near-end side who is predicted to be next to speak and transmit it to the device on the far-end side. For example, the camera image control section 105 transmits the image of the speaker on the near-end side who is predicted to be next to speak to the extent of once every 10 seconds. Alternatively, the camera image control section 105 can also synthesize the image of the speaker on the near-end side who is predicted to be next to speak in a small size in the image of the speaker on the near-end side who is currently speaking and transmit it to the device on the far-end side.

[0057] In addition, the sound signal processing apparatus 1 can perform sound signal processing on the sound signal acquired by the microphones based on the prediction result of the prediction section 51. Figure 10 is a block diagram showing the functional structure of the sound signal processing apparatus 1 in the case where sound signal processing is performed based on the prediction result. The sound signal processing apparatus 1 functionally includes a filter generation section 52 and a sound signal processing section 53. In addition, the sound signal processing section 53 functionally includes a beamforming processing section 501, a gain correction section 502, and an equalizer 503. These functional structures are realized by the CPU 12 and the DSP 13.

[0058] The filter generation section 52 generates a correction filter based on the prediction result of the prediction section 51. The correction filter includes at least the filter processing in the beamforming processing section 501.

[0059] The beamforming processing section 501 performs beamforming by implementing filter processing on the sound signals acquired by the microphones 18A to 18F respectively and synthesizing them. The signal processing involved in the beamforming can be any method such as a Delay Sum method, a Griffiths Jim type, a Sidelobe Canceller type, or a Frost type Adaptive Beamformer.

[0060] The filter generation section 52 generates filter coefficients for forming directivity toward the position of the speaker predicted to be the next to speak and sets them to the beamforming processing section 501. Thus, the sound signal processing apparatus 1 can acquire the speech of the speaker predicted to be the next to speak with a high SN ratio from the beginning of the speech.

[0061] In addition, the filter generation section 52 sets the gain of the gain correction section 502 based on the prediction result of the prediction section 51. The farther the distance between the microphones, the more the speech of the speaker attenuates. Therefore, the filter generation section 52 generates a gain correction filter that increases the level of the sound signal the greater the distance between the speaker predicted to be the next to speak and sets it to the gain correction section 502. Thus, the sound signal processing apparatus 1 can acquire the speech of the speaker predicted to be the next to speak with a stable level regardless of the distance from the speaker.

[0062] Further, the filter generating section 52 can set the frequency characteristic of the equalizer 503 such that the level of the high frequency is increased more as the distance between the predicted next speaker to utter is larger. The voice of a speaker has a tendency that the high frequency attenuates more than the low frequency as the distance between the microphone is farther. Therefore, the sound signal processing apparatus 1 can acquire the voice of the predicted next speaker to utter with stable sound quality regardless of the distance from the speaker.

[0063] The description of this embodiment should be considered in all respects as illustrative only, and not restrictive. The scope of the present application is not represented by the above-described embodiment, but by the claims. Further, the scope of the present application includes the scope equivalent to the claims.

[0064] Label Explanation

[0065] 1 … sound signal processing apparatus

[0066] 11 … camera

[0067] 12 … CPU

[0068] 13 … DSP

[0069] 14 … flash memory

[0070] 15 … RAM

[0071] 16 … user I / F

[0072] 17 … speaker

[0073] 18A to 18F … microphones

[0074] 19 … communication section

[0075] 51 … prediction section

[0076] 52 … filter generating section

[0077] 53 … sound signal processing section

[0078] 100 … voice acquiring section

[0079] 101 … image acquiring section

[0080] 102 … near speaker determining section

[0081] 103 … conversation history recording section

[0082] 104 … far speaker determining section

[0083] 105 … camera image control section

[0084] 501 … beamforming processing section

[0085] 502 … gain correction section

[0086] 503 … equalizer.

Claims

1. A speaker prediction method of acquiring voices of a plurality of speakers and recording conversation histories of the plurality of speakers, wherein a voice is acquired, a speaker of the acquired voice is determined, a position where the determined speaker has made a sound is detected from the conversation histories, and a speaker who is to make a sound next is predicted from the plurality of speakers based on a probability of making a sound by a speaker who is to make a sound next from the detected position, based on the determined speaker and the conversation histories.

2. The speaker prediction method according to claim 1, wherein control of an image captured by a camera is performed based on a result of the prediction.

3. The speaker prediction method according to claim 1 or 2, wherein sound signal processing of a sound signal acquired by a microphone is performed based on a result of the prediction.

4. The speaker prediction method according to claim 2, wherein the control of the image includes framing processing.

5. The speaker prediction method according to claim 3, wherein the sound signal processing includes beamforming processing.

6. The speaker prediction method according to claim 1 or 2, wherein the speaker of the acquired voice is determined based on a voice feature amount of the acquired voice.

7. The speaker prediction method according to claim 1 or 2, wherein a direction of arrival of a voice is estimated, and the speaker of the acquired voice is determined based on the direction of arrival of the voice.

8. The speaker prediction method according to claim 1 or 2, wherein an image of a camera is acquired, and the speaker of the acquired voice is determined based on the image of the camera.

9. The speaker prediction method according to claim 1 or 2, wherein the conversation histories include conversation histories of speakers on a far-end side and on a near-end side, and at least a voice of a speaker on the far-end side is determined to predict a speaker on the near-end side who is to make a sound next. a voice acquisition section that acquires voices of a plurality of speakers; and a conversation history recording section that records conversation histories of the plurality of speakers, wherein the voice acquisition section acquires a voice, the speaker prediction device further includes: a speaker determination section that determines a speaker of the acquired voice; and a prediction section that detects a position where the determined speaker has made a sound from the conversation histories, and predicts a speaker who is to make a sound next from the plurality of speakers based on a probability of making a sound by a speaker who is to make a sound next from the detected position, based on the determined speaker and the conversation histories.

11. The speaker prediction device according to claim 10, wherein the speaker prediction device includes a camera image control section that performs control of an image captured by a camera based on a result of the prediction.

12. The speaker prediction device according to claim 10 or 11, wherein the speaker prediction device includes a sound signal processing section that performs sound signal processing of a sound signal acquired by a microphone based on a result of the prediction.

13. The speaker prediction device according to claim 11, wherein ​ ​ ​ ​ ​ ​ 10. A speaker prediction apparatus comprising: ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ The control of the image includes framing processing.

14. The speaker prediction apparatus according to claim 12, wherein The sound signal processing includes beamforming processing.

15. The speaker prediction apparatus according to claim 10 or 11, wherein The speaker determination section determines a speaker of the acquired voice based on a voice feature amount of the acquired voice.

16. The speaker prediction apparatus according to claim 10 or 11, wherein The speaker determination section estimates a direction of arrival of a voice, and determines a speaker of the acquired voice based on the direction of arrival of the voice.

17. The speaker prediction apparatus according to claim 10 or 11, wherein The speaker prediction apparatus includes an image acquisition section that acquires an image of a camera, The speaker determination section determines a speaker of the acquired voice based on the image of the camera.

18. The speaker prediction apparatus according to claim 10 or 11, wherein The conversation history includes a conversation history of a speaker on a far-end side and a near-end side, The prediction section predicts a speaker who is to utter next, based on at least a voice of a speaker on the far-end side.

19. A communication system comprising: a communication section that receives a voice of a speaker from a far-end side; a microphone that acquires a voice of a speaker on a near-end side; and a conversation history recording section that records a conversation history of a plurality of speakers including the voice received by the communication section and the voice acquired by the microphone, wherein The communication system further includes: a speaker determination section that determines a speaker of the voice on the far-end side received by the communication section or the voice on the near-end side acquired by the microphone; and a prediction section that detects a position where the determined speaker utters, based on the conversation history, and predicts a speaker who is to utter next from a plurality of the speakers, based on a probability of utterance of a speaker who is to utter next at the detected position, of the determined speaker and the conversation history.

Citation Information

Patent Citations

  • Audio processing system

    JP2012029209A

  • Generating real-time director's cuts of live-streamed events using roles

    US20200267427A1