Speech control program, speech control method, and speech control device

Abstract

To solve the problem that although a technique to deterring an overlap of voices among attendants in a voice conference is required and, for example, a method for determining utterance priority levels of speakers based upon relativity to a conference subject is available as one of the methods for deterring the overlap of voices, the method is to determine the priority levels in advance based upon static information before the conference start, and cannot be adapted to cases in which speaker of high utterance priority levels change as proceedings advance and change.SOLUTION: An utterance control device is caused to execute processing to: calculate, for a plurality of attendants who spoke within a predetermined time, priority levels of utterance based upon feature words of conference-related mail sentences and keywords acquired from conference voices one after another; and mute, based upon the utterance priority levels, utterance by attendances other than a specific person.SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] The present invention relates to a speech control program, a speech control method, and a speech control device. [Background technology]

[0002] With the advancement of remote work, many people are now holding voice conferences over remote networks. In voice conferences, the terminals of the participants are far from the server hosting the conference, so there may be a delay before the voice is transmitted to others.

[0003] As a result, even if a participant speaks when the speaker has finished speaking, another participant may begin speaking, or the speaker may continue speaking, causing the participants' voices to overlap and disrupting the progress of the conference. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 8-9063 [Patent Document 2] JP 2006-60364 A [Patent Document 3] Special Publication No. 2002-522998 [Patent Document 4] Japanese Patent Application Publication No. 6-1890052 Summary of the Invention [Problem to be solved by the invention]

[0005] However, unlike face-to-face conferences, voice conferences make it difficult for participants to observe each other's postures and facial expressions, making it difficult to time their speech. Therefore, there is a demand for technology to prevent voices from overlapping in voice conferences.

[0006] One method for preventing voice overlap is to determine the priority of each speaker's speech based on the relevance of the speech to the conference agenda. However, this method determines the priority before the conference starts based on static information, and cannot handle cases in which speakers with high priority change as the proceedings progress and change, as occurs in actual conferences.

[0007] In one aspect, an object of the present invention is to provide a speech control program, a speech control method, and a speech control device that can prevent overlapping of voices between participants in an audio conference. [Means for solving the problem]

[0008] In the first proposal, the speech control device calculates speech priority for multiple participants who speak within a specified period of time based on characteristic words in conference-related email sentences and keywords acquired sequentially from the conference audio, and executes a process to mute speech from all participants other than specific ones based on the speech priority. Effect of the Invention

[0009] In one aspect, voice overlap between participants in an audio conference can be prevented. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of an overall configuration of an information processing system according to a first embodiment. [Diagram 2] FIG. 2 is a functional block diagram of the speech control device 100 according to the first embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an overall flow of the speech control process according to the first embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of word extraction of past conference voice according to the first embodiment. [Diagram 5] FIG. 5 is a diagram illustrating an example of how the subject relevance degree is determined according to the first embodiment. [Figure 6]FIG. 6 is a diagram illustrating an example of extracted characteristic words from an email text according to the first embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of the determination of position points according to the first embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of calculation of a reference comment priority level according to the first embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of keyword extraction of the conference voice according to the first embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of points added to a message priority level according to the first embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of speech control according to the first embodiment. [Figure 12] FIG. 12 is a diagram illustrating an example of an output of a speech control display according to the first embodiment. [Figure 13] FIG. 13 is a flowchart of the comment priority calculation process according to the first embodiment. [Figure 14] FIG. 14 is a flowchart of the speech control process according to the first embodiment. [Figure 15] FIG. 15 is a diagram illustrating an example of a hardware configuration. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiments of the speech control program, speech control method, and speech control device disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to these embodiments. Furthermore, each embodiment can be appropriately combined within a range that does not cause inconsistency. EXAMPLES

[0012] [Overall configuration example] FIG. 1 is a diagram showing an example of the overall configuration of an information processing system according to a first embodiment. As shown in FIG. 1, the information processing system according to the first embodiment is a system in which a speech control device 100 and participant terminals 200-1 to 200-n (n is any integer, hereinafter collectively referred to as "participant terminals 200") are connected to each other via a network N so as to be able to communicate with each other. Note that the network N can be any of various communication networks, such as an intranet, whether wired or wireless. Furthermore, the network N is not a single network, and may be configured, for example, by connecting an intranet and the Internet via a network device such as a gateway or other device (not shown).

[0013] The remark control device 100 is an entity that executes this embodiment including Example 1, and is, for example, an information processing device such as a server computer that is managed by a company to which the participants of the audio conference belong. Alternatively, the remark control device 100 may be a cloud server device that is managed by a cloud service provider that provides cloud computing services. Furthermore, the remark control device 100 may be a distributed computing system made up of multiple computers.

[0014] The remark control device 100 provides an audio conference service to the participant terminals 200, receives the voice of each participant from each participant terminal 200, and distributes the voice of each participant to participant terminals 200 used by other participants. The remark control device 100 may also receive and distribute, in addition to the voice, video captured by the camera function of the participant terminal 200 or a Web camera device connected to the participant terminal 200.

[0015] In addition, the speech control device 100 calculates speech priority based on characteristic words in email sentences related to the conference and keywords acquired successively from the conference audio, and when the voices of participants overlap, mutes the speech of participants other than the one with the highest speech priority.

[0016] The participant terminal 200 is an information processing terminal used by a participant of the audio conference. The participant terminal 200 may be a desktop PC or a notebook PC installed in the participant's company or home, or may be a mobile terminal such as a smartphone or a tablet PC carried by the participant.

[0017] The participant terminal 200 is equipped with a microphone and speaker function for participants to participate in an audio conference, or is connected to an external device that provides these functions. An application for participating in an audio conference is pre-installed in the participant terminal 200. Alternatively, if the participant terminal 200 uses a Web application on the remark control device 100 that has the same function as such an application, the application does not need to be installed in the participant terminal 200.

[0018] Participants can conduct an audio conference by transmitting and receiving voices between each other via the participant terminal 200.

[0019] [Functional configuration of the speech control device 100] Next, a description will be given of the functional configuration of the remark control device 100 shown in Fig. 1. Fig. 2 is a functional block diagram showing the functional configuration of the remark control device 100 according to the first embodiment. As shown in Fig. 2, the remark control device 100 has a communication unit 110, a storage unit 120, and a control unit 130.

[0020] The communication unit 110 is a processing unit that controls communication with other devices such as the participant terminal 200, and is, for example, a communication interface such as a network interface card.

[0021] The storage unit 120 is an example of a storage device, such as a memory or a hard disk, that stores various data and programs executed by the control unit 130. The storage unit 120 stores mail information 121, schedule information 122, organization information 123, and meeting information 124.

[0022] The mail information 121 stores information such as reply mails for each participant to the invitation mail for the audio conference and characteristic words for each participant extracted from the body of the reply mail. The mail information 121 may be obtained from an information processing device such as a mail server other than the remark control device 100.

[0023] The schedule information 122 stores schedule information of the conference of the participants of the audio conference, etc. The schedule information 122 may be acquired from an information processing device such as a schedule management server other than the speech control device 100.

[0024] The organization information 123 stores information such as the participants' job titles and job title points set for each job title. The organization information 123 may be acquired from an information processing device other than the remark control device 100.

[0025] The conference information 124 stores information such as the conference voice received from the participant terminal 200, words extracted from the conference voice, and the number of occurrences of the words.

[0026] It should be noted that various information other than the above information can be stored in the storage unit 120. Furthermore, the data configuration of each piece of information is not limited to the above.

[0027] The control unit 130 is a processing unit that controls the entire remark control device 100, and is, for example, a processor. The control unit 130 includes an extraction unit 131, a determination unit 132, a calculation unit 133, a muting unit 134, and an output unit 135. Each processing unit is an example of an electronic circuit included in a processor or an example of a process executed by a processor.

[0028] The extraction unit 131 performs a morphological analysis on the email text for each participant to extract characteristic words. The extraction unit 131 can use a statistical method such as the tf-idf method to extract characteristic words.

[0029] The extraction unit 131 also converts the conference voice into text, performs morphological analysis, and extracts keywords. More specifically, the extraction unit 131 converts the conference voice of a certain number of past conferences, for example, 20 conferences, into text, performs morphological analysis, and preliminarily extracts words and the number of occurrences of the words for each conference. The extraction unit 131 also converts the voice of the current conference received from the participant terminal 200 into text in real time, performs morphological analysis, and extracts words and the number of occurrences of the words. Then, the extraction unit 131 extracts important words, i.e., keywords, in the current conference based on the words and the number of occurrences of the past conferences and the words and the number of occurrences of the current conference. The extraction unit 131 can also use statistical methods such as the tf-idf method to extract keywords.

[0030] The determination unit 132 determines the agenda relevance, which is the degree of relevance to the conference, for each participant based on the number of characters in the email text. For example, the determination unit 132 may determine the agenda relevance in proportion to the number of characters in the email text.

[0031] Furthermore, when the determination unit 132 receives the voices of at least two or more of the participants who have started speaking within a predetermined time, it determines the participant with the highest speech priority calculated by the calculation unit 133 as the specific person. Any speech other than that of the specific person determined here will be muted, but the reason why a certain amount of time is provided for the reception of the voice within the predetermined time is to take into account the time lag between the participant terminals 200 for the reception of the voice.

[0032] The calculation unit 133 calculates the comment priority for multiple participants who have spoken within a predetermined time based on the characteristic words of the email text related to the conference and the keywords successively acquired from the conference voice. More specifically, for example, a participant who has a larger number of matches between the characteristic words extracted from the email text for each participant and the keywords extracted from the conference voice is determined to be more relevant to the current topic, and the comment priority is calculated to be higher. Note that since the keywords extracted from the conference voice change successively in response to changes in the conference topic, the comment priority for each participant also changes dynamically during the conference.

[0033] Furthermore, the calculation unit 133 can calculate the comment priority based on the subject relevance and job title of the participant determined in advance by the determination unit 132 from the number of characters in the email text. The comment priority determined here is a comment priority that can be determined before the conference and serves as a reference for the comment priority that changes dynamically during the conference as described above. In this way, the comment control device 100 calculates a reference comment priority for each participant before the conference, and dynamically changes the comment priority in response to changes in the subject and the like during the conference. This allows the comment control device 100 to handle cases in which participants with high comment priority change as the proceedings progress, as occurs in actual conferences.

[0034] The muting unit 134 mutes comments from participants other than the specific participant with the highest speech priority determined by the determination unit 132 based on the speech priority calculated by the calculation unit 133. Furthermore, when any participant is speaking, the muting unit 134 prioritizes the speech of the speaker who is already speaking, and therefore mutes comments from participants other than the speaker regardless of the speech priority of each participant.

[0035] When the speech of a participant is muted, the output unit 135 causes the participant terminal 200 to output a display indicating that the speech is muted and a display indicating that the speech has priority.

[0036] [Function details] Next, the utterance control process according to the first embodiment will be described in detail. Fig. 3 is a diagram showing an overall flow of the utterance control process according to the first embodiment. In the example of Fig. 3, the utterance control process is executed by the utterance control device 100. In the example of Fig. 3, steps S1 to S4 of the utterance control process are processing steps executed before the conference, and steps S5 to S9 are processing steps executed during the conference.

[0037] First, the speech control device 100 extracts words from past conference audio (step S1). This is a process of extracting reference words from the conference audio of a certain number of past conferences, for example 20 conferences, in order to extract characteristic keywords from the conference audio of the conference in which the speech of participants is to be controlled.

[0038] Next, the speech control device 100 determines the topic relevance for each participant from the number of characters in the conference-related email (step S2). Here, the topic relevance is determined so that, for example, the topic relevance is higher for participants who write more characters in their emails, based on the idea that participants who write more characters in their conference-related emails will be more relevant to the conference.

[0039] Next, the speech control device 100 extracts characteristic words from the conference-related email text for each participant (step S3). The characteristic words extracted here for each participant are compared with keywords extracted from the conference voice of the conference for which the participant's speech is controlled, and the speech priority of the participant is controlled so that, for example, the closer the characteristic words match the keywords, the higher the speech priority of that participant is.

[0040] Note that steps S2 and S3 may be executed by searching schedule information 122 every 10 minutes, and if there is a meeting that is to start within 10 minutes, searching for a reply email to the invitation email for that meeting from email information 121. Note that the search interval for schedule information 122 is not limited to 10 minutes.

[0041] Next, the speech control device 100 determines position points for each participant (step S4). Here, the position points are determined so that the higher the position of the participant, the higher the position. This is based on the idea that, in the speech control process according to the first embodiment, the higher the position points, the higher the speech priority, and therefore, particularly in Japan, priority is given to the speech of participants with higher positions. Therefore, when there is no need to control speech priority based on position, it is not necessary to determine position points.

[0042] In the example of FIG. 3, steps up to step S4 are shown to be executed before the conference, but the execution timing of each processing step can be changed, for example, step S4 can be executed during the conference.

[0043] Next, the speech control device 100 determines a standard speech priority for each participant (step S5). This is a process that is executed, for example, at the start of the conference, to determine a speech priority for each participant based on the agenda relevance determined in step S2 and the position points determined in step S4.

[0044] Next, the speech control device 100 extracts keywords from the audio of the ongoing conference (step S6). This is a process in which words are sequentially extracted in real time from the audio of the ongoing conference, and characteristic keywords are extracted by comparing them with the reference words from the audio of past conferences extracted in step S1.

[0045] Next, the comment control device 100 increments the comment priority determined in step S5 (step S7). This is a process of dynamically incrementing the comment priority so that the more the characteristic words for each participant extracted in step S3 match the keywords extracted in step S6, the higher the comment priority of that participant becomes.

[0046] Next, the speech control device 100 controls the speech of the participants when the voices of the conference participants overlap (step S8). For example, when the voices of multiple participants start to speak and overlap, this is a process of muting the voices of participants other than the participant with the highest speech priority. Or, when a specific participant is already speaking, the voice of a participant who starts speaking later is muted.

[0047] Next, when the speech control device 100 mutes speeches other than those of the specific participant, it outputs a speech control display (step S9). This is a process of making the participant terminal 200 output, for example, a display indicating that the speech has been muted or a display indicating that the speech has been prioritized, when a speech has been muted in step S8.

[0048] The overall flow of the speech control process according to the first embodiment has been described above with reference to Fig. 3. Next, each of the processing steps S1 to S9 of the speech control process described with reference to Fig. 3 will be described in more detail with reference to Figs.

[0049] 4 is a diagram illustrating an example of word extraction from past conference voice according to Example 1. The extraction process described with reference to FIG. 4 corresponds to step S1 in FIG.

[0050] As shown in Fig. 4, the remark control device 100 converts conference voices of past conferences A to C into text, performs morphological analysis, and extracts words and their frequency of occurrence for each conference. Note that, although the example in Fig. 4 shows three past conferences, A to C, the number of conferences to be extracted is not limited to three. Furthermore, a speech-to-text conversion tool such as Azure Speech API (Application Programming Interface) may be used to convert conference voices into text.

[0051] 5 is a diagram illustrating an example of how to determine the subject relevance according to the embodiment 1. The determination process described with reference to FIG. 5 corresponds to step S2 in FIG.

[0052] 5, the remark control device 100 determines the agenda relevance for each participant from the number of characters in the reply email to the conference-related invitation email for which the participant's remarks are to be controlled. The agenda relevance may be determined as 0, 1, 2, etc. for each of the ranges of the number of characters in the reply email, 0-99, 100-199, 200-299, etc. In addition, when one participant has multiple emails, the number of characters in the reply email may be the total number of characters in the multiple emails, but quoted portions in the reply emails are not counted.

[0053] 6 is a diagram showing an example of extraction of characteristic words from an email text according to Example 1. The extraction process described with reference to FIG. 6 corresponds to step S3 in FIG.

[0054] As shown in Fig. 6, the comment control device 100 first extracts, for each participant, words and their occurrence counts from the conference-related email text that controls the participant's comments. If one participant has multiple emails, the occurrence count of a word may be the total occurrence count of the multiple emails, but quoted parts in reply emails are excluded from the count. The comment control device 100 then extracts words with high tf-idf values, for example, the top 10%, as characteristic words of the participant that characterize the participant's interests.

[0055] 7 is a diagram illustrating an example of the determination of position points according to the embodiment 1. The determination process described with reference to FIG. 7 corresponds to step S4 in FIG.

[0056] As shown in Fig. 7, the remark control device 100 determines position points for each participant by, for example, awarding higher points to participants with higher positions, which are preset higher according to their position. Note that the position point values ​​in Fig. 7 are merely examples and are not limited to these. Furthermore, the positions may be set more finely or coarsely than in the example shown in Fig. 7.

[0057] 8 is a diagram illustrating an example of calculation of a reference comment priority level according to the embodiment 1. The calculation process described with reference to FIG. 8 corresponds to step S5 in FIG.

[0058] 8, the speech control device 100 determines a standard speech priority for each participant by, for example, adding up the agenda relevance determined in step S2 and the position points determined in step S4. Note that the speech priority determined here is merely a standard, and the speech priority can change dynamically based on keywords successively extracted from the conference voice.

[0059] 9 is a diagram illustrating an example of keyword extraction from the conference voice according to the embodiment 1. The extraction process described with reference to FIG. 9 corresponds to step S6 in FIG.

[0060] 9, the speech control device 100 first converts the audio of an ongoing conference into text in predetermined time units, such as one minute, and performs morphological analysis to extract words and their occurrence counts. A speech-to-text conversion tool such as Azure Speech API may be used to convert the audio of an ongoing conference into text.

[0061] Next, the speech control device 100 compares the words extracted from the audio of the ongoing conference and their occurrence counts with the words extracted from the audio of past conferences extracted in step S1 of Fig. 3. The speech control device 100 then extracts characteristic keywords that appear in the audio of the ongoing conference from the comparison results. Keywords are extracted using a statistical method such as the tf-idf method, and words with high tf-idf values, for example, the top 10%, are extracted as keywords.

[0062] The reason why words and their occurrence counts from the audio of an ongoing meeting are extracted in specified time units, such as one minute, is to accommodate cases where important words change as the proceedings progress, etc.

[0063] 10 is a diagram illustrating an example of adding points to a message priority level according to the embodiment 1. The adding point process described with reference to FIG. 10 corresponds to step S7 in FIG.

[0064] As shown in FIG. 10, the speech control device 100 adds, for example, the number of matches between the characteristic words for each participant extracted in step S3 of FIG. 3 and the keywords for the ongoing conference extracted in step S6 to the speech priority determined in step S5.

[0065] As described above, the keywords of the ongoing conference extracted in step S6 are extracted sequentially in predetermined time units, such as one minute, and therefore the increment of the comment priority in step S7 is also performed every time a keyword is extracted. At that time, the previous increment of the comment priority is reset, and the number of matches between the newly extracted keyword and the characteristic words of each participant is incremented to the comment priority.

[0066] 11 is a diagram illustrating an example of speech control according to the embodiment 1. The control process described with reference to FIG. 11 corresponds to step S8 in FIG.

[0067] As shown in Fig. 11, the remark control device 100 controls the remarks of conference participants when they speak at the same time and their voices overlap. In the example of Fig. 11, the remark control device 100 receives the voices of A and B starting to speak at the same time, so the remark control device 100 compares the remark priorities of A and B, and gives priority to the voice of A, who has a higher remark priority, and mutes the voice of B. Note that the voices received at the same time here refer to voices received within, for example, 0.5 seconds or 1 second, taking into account the time lag between the participant terminals 200 relative to the reception of the voices.

[0068] 11, when participants speak at different times and the voices do not overlap, the remarks are not muted and are delivered to the participant terminal 200 by the remark control device 100. Also, when another participant, C, is already speaking, as in the case of D's remark in the example of Fig. 11, the remark control device 100 mutes the remarks of D, who started speaking later, regardless of the remark priority.

[0069] As shown in FIG. 11, the speech control device 100 converts the voice that is not muted and is distributed to the participant terminal 200 into text in predetermined time units, such as one minute units, and extracts keywords.

[0070] 12 is a diagram showing an example of an output of the speech control display according to the embodiment 1. The output process described with reference to FIG. 12 corresponds to step S9 in FIG.

[0071] As shown in Fig. 12, when the speech control device 100 mutes speech other than that of a specific participant, it causes each participant terminal 200 to output a speech control display indicating that speech is being controlled. One example of a speech control display is a muted display 250 indicating that speech is being muted, which is output to the participant terminal 200-1 of the participant whose speech is being muted, as shown in Fig. 12. On the other hand, a priority speech display 260 indicating that speech is being distributed with priority is output to the participant terminal 200-2 of the participant making a priority speech to the muted participant.

[0072] Note that the speech control display shown in FIG. 12 is merely an example, and for example, a display indicating that the participant is muted or making a priority speech may be output near the icon or name of each participant in the participant list displayed on the participant terminal 200.

[0073] [Processing flow] The overall flow of the speech control process and each processing step have been described in detail above using Fig. 3. The speech control process can be divided into a speech priority calculation process that calculates speech priorities for controlling participants' speech when their voices overlap, and a speech control process that controls speech based on speech priorities when their voices overlap. Each process will be described using the flowcharts in Figs. 13 and 14.

[0074] First, a flow of the comment priority calculation process according to the first embodiment will be described, which is executed by the comment control device 100. Fig. 13 is a flowchart showing the flow of the comment priority calculation process according to the first embodiment. The comment priority calculation process shown in Fig. 13 is a process for calculating a comment priority for controlling comments of participants when their voices overlap.

[0075] The speech control device 100 searches the schedule information 122, for example, every 10 minutes, and if there is a meeting that is to start within 10 minutes, obtains a reply mail to the invitation mail for that meeting from the mail information 121 (step S101).

[0076] Next, the statement control device 100 determines, for each participant, an agenda relevance level, which is the degree of relevance to the conference, in proportion to the number of characters in the email text acquired in step S101 (step S102).

[0077] Next, the speech control device 100 determines a standard speech priority for each participant based on the agenda relevance determined in step S102 (step S103). Note that at this time, the speech control device 100 may obtain, for example, position points from the organization information 123, which are set higher for participants with higher positions, and may determine the standard speech priority further based on the position points.

[0078] Next, the statement control device 100 performs a morphological analysis on the email text acquired in step S101 for each participant, and extracts the top 10% of characteristic words using the tf-idf method (step S104).

[0079] Next, if the speech control device 100 has not received the speech of the participant from the participant terminal 200 (step S105: No), it waits for the speech to be received. On the other hand, if the speech has been received (step S105: Yes), the speech control device 100 converts the received speech into text, performs morphological analysis, and extracts keywords (step S106).

[0080] In addition, regarding the keyword extraction in step S106, the speech control device 100 extracts words and their occurrence counts from each conference voice received in one minute in one-minute increments, for example. Then, the speech control device 100 compares the words extracted from each conference voice and their occurrence counts with the words and their occurrence counts of voices of past conferences extracted in advance, and extracts the top 10% of keywords using the tf-idf method.

[0081] Next, the statement control device 100 compares, for example, for each participant, the keywords extracted in step S106 with the characteristic words extracted in step S104, and calculates the number of matches (step S107).

[0082] Next, the statement control device 100, for example, uses the number of matches calculated in step S107 for each participant to add points to the standard statement priority determined in step S103 (step S108). Note that for participants with zero matches, the added points are also zero, so essentially no points are added.

[0083] Next, if a predetermined time such as one minute has not elapsed from a predetermined interval such as one minute (step S109: No), the process returns to step S105, and the remark control device 100 waits to receive further voice until the end of the conference. On the other hand, if a predetermined time such as one minute has elapsed (step S109: Yes), the remark control device 100 resets the remark priority points added in step S108 and returns to the standard remark priority (step S110). After executing step S110, the remark control device 100 returns to step S105, and waits to receive further voice until the end of the conference.

[0084] Next, a flow of the utterance control process according to the first embodiment, which is executed by the utterance control device 100, will be described. Fig. 14 is a flowchart showing the flow of the utterance control process according to the first embodiment. The utterance control process shown in Fig. 14 is a process for controlling utterances based on utterance priority when voices of participants overlap.

[0085] First, if the speech control device 100 has not received the participant's voice from the participant terminal 200 (step S201: No), it waits for the reception of the voice. On the other hand, if the speech control device 100 has received the voice of another participant (step S201: Yes), it determines whether the other participant is speaking or not (step S202).

[0086] If another participant is already speaking (step S202: Yes), the speech control device 100 mutes the voice received in step S201 (step S205).

[0087] On the other hand, if no participant is speaking (step S202: No), the speech control device 100 determines whether or not there is conference voice received simultaneously in addition to the voice received in step S201 (step S203). Note that simultaneous reception here may mean reception within, for example, 0.5 seconds or 1 second, taking into account the time lag between the participant terminals 200 for receiving the voice.

[0088] If there is no other conference voice that has been received simultaneously (step S203: No), the speech control device 100 distributes the voice received in step S201 to each of the participant terminals 200 without muting it, and the speech control process shown in FIG. 14 ends.

[0089] On the other hand, if there is another conference voice that has been received simultaneously (step S203: Yes), the speech control device 100 determines the speaker based on the speech priority calculated in the speech priority calculation process shown in FIG. 13 (step S204).

[0090] Next, the statement control device 100 distributes the voice of the speaker determined in step S204 to each of the participant terminals 200 without muting it, and mutes the voices of all speakers other than the speaker (step S205).

[0091] Next, the remark control device 100 causes the participant terminal 200 to output a remark control display (step S206). The remark control display is, for example, a display indicating that a remark has been muted or a display indicating that a remark has been prioritized, and can be output to the participant terminal 200 of the participant whose remark has been muted or prioritized. Alternatively, the remark control device 100 may output a remark control display indicating that a remark is being muted or that a priority remark is being made near the icon or name of each participant in the participant list displayed on the participant terminal 200. After step S206 is executed, the remark control process shown in FIG. 14 ends.

[0092] [effect] As described above, the speech control program causes the speech control device 100 to calculate speech priority for multiple participants who speak within a specified period of time based on characteristic words in conference-related email sentences and keywords acquired successively from the conference audio, and to execute a process of muting speech from all participants other than specific ones based on the speech priority.

[0093] As a result, the speech control device 100 mutes the speech of participants other than the one with the highest priority among the participants whose speeches overlap in the audio conference, thereby making it possible to prevent speech overlap between participants in the audio conference.

[0094] In addition, the speech control program further causes the speech control device 100 to execute a process of determining the degree of relevance to the conference for each participant based on the number of characters in the email text, and the speech control device 100 calculates the speech priority based on the degree of relevance.

[0095] As a result, the speech control device 100 can use speech priorities that are more suitable for each audio conference, and prevent overlapping of voices between participants in the audio conference.

[0096] Furthermore, the speech control device 100 calculates speech priority based on the role of the participant.

[0097] As a result, the speech control device 100 can use speech priorities more suited to the participants of the audio conference, thereby preventing overlapping of voices between participants in the audio conference.

[0098] The speech control program also causes the speech control device 100 to perform morphological analysis on the email text for each participant, extract characteristic words, convert the conference audio into text, perform morphological analysis, and further execute the process of extracting keywords, and the speech control device 100 calculates the speech priority based on the degree of relevance and the number of matches between the characteristic words and the keywords.

[0099] As a result, the speech control device 100 can dynamically change the speech priority in response to changes in the agenda during the audio conference, and use the speech priority that is more suitable for the conference content, thereby preventing overlapping of voices between participants in the audio conference.

[0100] In addition, the keyword extraction process that the utterance control program causes the utterance control device 100 to execute includes processes of converting the conference audio into text, performing morphological analysis, extracting a first word and a first number of occurrences of the first word, converting a second conference audio other than the conference audio into text, performing morphological analysis, extracting a second word and a second number of occurrences of the second word for each conference, and extracting keywords based on the first word, the first number of occurrences, the second word, and the second number of occurrences.

[0101] As a result, the speech control device 100 can dynamically change the speech priority in response to changes in the agenda during the audio conference, and use the speech priority that is more suitable for the conference content, thereby preventing overlapping of voices between participants in the audio conference.

[0102] Furthermore, the comment control device 100 uses the tf-idf method to extract feature words and keywords.

[0103] This allows the speech control device 100 to extract characteristic words for each participant and keywords for the conference voice that are more suited to the participants in the audio conference and the contents of the conference.

[0104] In addition, the speech control program further causes the speech control device 100 to execute a process of determining the participant with the highest speech priority as the specific person when the speech start voices of at least two or more of the participants are received within a second specified time.

[0105] This allows the speech control device 100 to prevent overlapping of voices between participants in an audio conference.

[0106] Moreover, the speech control program further causes the speech control device 100 to execute a process of muting comments made by anyone other than the participant who is speaking when that participant is speaking.

[0107] This allows the speech control device 100 to prevent overlapping of voices between participants in an audio conference.

[0108] [system] The information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. In addition, the specific examples, distributions, values, etc. described in the embodiments are merely examples and can be changed as desired.

[0109] Furthermore, the components of each device shown in the figure are conceptual and functional, and do not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure. In other words, all or part of them can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc. For example, the extraction unit 131 and decision unit 132 of the speech control device 100 can be integrated.

[0110] Furthermore, each processing function performed by each device may be realized, in whole or in part, by a CPU and a program analyzed and executed by the CPU, or may be realized as hardware using wired logic.

[0111] [Hardware] The hardware configuration of the above-mentioned remark control device 100 will be described. Fig. 15 is a diagram showing an example of the hardware configuration. As shown in Fig. 15, the remark control device 100 has a communication unit 100a, an HDD (Hard Disk Drive) 100b, a memory 100c, and a processor 100d. The units shown in Fig. 15 are connected to each other via a bus or the like.

[0112] The communication unit 100a is a network interface card or the like, and communicates with other servers. The HDD 100b stores programs and DBs that operate the functions shown in FIG.

[0113] The processor 100d reads out a program that executes the same processes as the processing units shown in Fig. 2 from the HDD 100b or the like, and loads it into the memory 100c, thereby operating a process that executes the functions described in Fig. 2. For example, this process executes the same functions as the processing units of the remark control device 100. Specifically, for example, the processor 100d reads out a program having the same functions as the determination unit 132, the calculation unit 133, or the like from the HDD 100b or the like. Then, the processor 100d executes a process that executes the same processes as the determination unit 132, the calculation unit 133, or the like.

[0114] In this way, the utterance control device 100 operates as an information processing device that executes each process by reading and executing a program. The utterance control device 100 can also realize functions similar to those of the above-mentioned embodiments by reading the program from a recording medium using a media reading device and executing the read program. Note that the program in these other embodiments is not limited to being executed by the utterance control device 100. For example, the present invention can be similarly applied to cases where another computer or server executes a program, or where these cooperate to execute a program.

[0115] This program can be distributed via a network such as the Internet. This program can also be recorded on a computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, a magneto-optical disk (MO), or a digital versatile disk (DVD), and can be executed by being read out from the recording medium by a computer.

[0116] In addition, the following supplementary notes are disclosed regarding the embodiments including the above-mentioned Example 1.

[0117] (Appendix 1) The speech control device includes: Calculate comment priorities for multiple participants who comment within a specified time period based on characteristic words in emails related to the meeting and keywords sequentially acquired from the meeting audio; Muting speeches from other than those of specific participants based on the speech priority. A speech control program for executing a process.

[0118] (Appendix 2) The speech control device includes: For each of the participants, a degree of relevance to the conference is determined based on the number of characters in the email text. Further processing is performed, The comment control program according to claim 1, wherein the comment control device calculates the comment priority based on the relevance.

[0119] (Supplementary Note 3) The speech control program according to Supplementary Note 2, wherein the speech control device calculates the speech priority based on the role of the participant.

[0120] (Appendix 4) The speech control device includes: performing a morphological analysis on the email text for each of the participants to extract the characteristic words; The conference voice is converted into text, morphological analysis is performed, and the keywords are extracted. Further processing is performed, The comment control program according to claim 2 or 3, wherein the comment control device calculates the comment priority based on the relevance and the number of matches between the feature word and the keyword.

[0121] (Note 5) The process of extracting the keywords executed by the speech control device is converting the conference voice into text, performing morphological analysis, and extracting a first word and a first number of occurrences of the first word; A second conference voice other than the conference voice is converted into text, a morphological analysis is performed, and a second word and a second number of occurrences of the second word are extracted for each conference; Extracting the keywords based on the first word, the first number of occurrences, the second word, and the second number of occurrences. 5. The speech control program according to claim 4, further comprising a process for:

[0122] (Appendix 6) The comment control program according to any one of appendices 1 to 5, wherein the comment control device uses the tf-idf method to extract the characteristic words and the keywords.

[0123] (Appendix 7) The speech control device includes: When speech start voices of at least two or more of the participants are received within a second predetermined time, the participant having the highest speech priority is determined to be the specific person. 7. The speech control program according to claim 1, further comprising the step of executing a process.

[0124] (Appendix 8) The speech control device includes: If any of the participants is speaking, mute all other participants 8. The speech control program according to claim 1, further comprising: a processor for executing a process.

[0125] (Appendix 9) A speech control device, Calculate comment priorities for multiple participants who comment within a specified time period based on characteristic words in emails related to the meeting and keywords sequentially acquired from the meeting audio; Muting speeches from other than those of specific participants based on the speech priority. A speech control method comprising the steps of:

[0126] (Appendix 10) For multiple participants who make comments within a specified time, calculate comment priorities based on characteristic words in emails related to the meeting and keywords sequentially obtained from the meeting audio, Muting speeches from other than those of specific participants based on the speech priority. A speech control device comprising a control unit for executing processing.

[0127] (Appendix 11) The control unit: For each of the participants, a degree of relevance to the conference is determined based on the number of characters in the email text. Further processing is carried out, 11. The speech control device according to claim 10, wherein the control unit calculates the speech priority based on the relevance.

[0128] (Supplementary Note 12) The speech control device according to Supplementary Note 11, wherein the control unit calculates the speech priority based on the role of the participant.

[0129] (Appendix 13) The control unit: performing a morphological analysis on the email text for each of the participants to extract the characteristic words; The conference voice is converted into text, morphological analysis is performed, and the keywords are extracted. Further processing is carried out, 13. The comment control device according to claim 11 or 12, wherein the control unit calculates the comment priority based on the relevance and the number of matches between the feature word and the keyword.

[0130] (Appendix 14) The process of extracting the keywords executed by the control unit is converting the conference voice into text, performing morphological analysis, and extracting a first word and a first number of occurrences of the first word; A second conference voice other than the conference voice is converted into text, a morphological analysis is performed, and a second word and a second number of occurrences of the second word are extracted for each conference; Extracting the keywords based on the first word, the first number of occurrences, the second word, and the second number of occurrences. 14. The speech control device according to claim 13, further comprising a process for:

[0131] (Supplementary Note 15) The speech control device according to any one of Supplementary Notes 10 to 14, wherein the control unit uses a TF-IDF method to extract the feature words and the keywords.

[0132] (Appendix 16) The control unit: When speech start voices of at least two or more of the participants are received within a second predetermined time, the participant having the highest speech priority is determined to be the specific person. 16. The speech control device according to any one of claims 10 to 15, further comprising a process.

[0133] (Appendix 17) The control unit: If any of the participants is speaking, mute all other participants 17. The speech control device according to any one of claims 10 to 16, further comprising a process.

[0134] (Appendix 18) A processor; a memory operatively connected to said processor; A speech control device comprising: Calculate comment priorities for multiple participants who comment within a specified time period based on characteristic words in emails related to the meeting and keywords sequentially acquired from the meeting audio; Muting speeches from other than those of specific participants based on the speech priority. A speech control device that executes processing.

[0135] Although the embodiments of the present invention have been described above, the present invention may be embodied in various different forms other than the above-described embodiments. [Explanation of symbols]

[0136] 100 Speech control device 100a Communications Department 100b HDD 100c Memory 100d Processor 110 Communications Department 120 Storage section 121 Email Information 122 Schedule Information 123 Organization information 124 Meeting Information 130 Control section 131 Extraction part 132 Decision Section 133 Calculation section 134 Mute section 135 Output section 200 participant terminals 250 Muted display 260 Priority speech indication

Claims

1. A system having a speech control device and a plurality of participant terminals capable of communicating with each other via a network, in which a plurality of participants using the plurality of participant terminals hold a conference by voice, The speech control device includes: calculating, for each participant, a speech priority level that is higher for a participant having a greater number of matches between the characteristic words and the keywords based on the characteristic words of the email texts related to the conference and the keywords sequentially acquired from the conference voice; Based on the speech priority, muting speeches of the participants other than those of a specific person having the highest speech priority. A speech control program for executing a process.

2. The speech control device includes: For each of the participants, a degree of relevance to the conference is determined based on the number of characters in the email text. Further processing is performed, 2. The program according to claim 1, wherein the speech control device calculates the speech priority based on the relevance.

3. 3. The speech control program according to claim 2, wherein the speech control device calculates the speech priority based on the title of the participant.

4. The speech control device includes: performing a morphological analysis on the email text for each of the participants to extract the characteristic words; The conference voice is converted into text, morphological analysis is performed, and the keywords are extracted. Further processing is performed, 4. The comment control program according to claim 2, wherein the comment control device calculates the comment priority based on the relevance and the number of matches between the feature words and the keywords.

5. The process of extracting the keyword that is executed by the speech control device includes: Converting the conference voice of the current conference into text, performing morphological analysis, and extracting a first word and a first number of occurrences of the first word; Converting a second conference speech of a past conference into text, performing a morphological analysis, and extracting a second word and a second number of occurrences of the second word for each conference; Extracting the keywords based on the first word, the first number of occurrences, the second word, and the second number of occurrences.

5. The speech control program according to claim 4, further comprising a process for:

6. 6. The comment control program according to claim 1, wherein the comment control device uses a tf-idf method to extract the feature words and the keywords.

7. The speech control device includes: When speech start voices of at least two of the participants are received within a second predetermined time, the participant having the highest speech priority is determined to be the specific person.

7. The speech control program according to claim 1, further comprising: a processor for executing a process.

8. The speech control device includes: If any of the participants is speaking, mute all other participants 8. The speech control program according to claim 1, further comprising: a processor for executing a process.

9. A system having a speech control device and a plurality of participant terminals capable of communicating with each other via a network, in which a plurality of participants using the plurality of participant terminals hold a conference by voice, The speech control device, calculating, for each participant, a speech priority level that is higher for a participant having a greater number of matches between the characteristic words and the keywords based on the characteristic words of the email texts related to the conference and the keywords sequentially acquired from the conference voice; Based on the speech priority, muting speeches of the participants other than those of a specific person having the highest speech priority. A speech control method comprising the steps of:

10. A speech control device used in a system having a speech control device and a plurality of participant terminals capable of communicating with each other via a network, the speech control device being used in a system in which a plurality of participants using the plurality of participant terminals hold a conference by voice, calculating, for each participant, a speech priority level that is higher for a participant having a greater number of matches between the characteristic words and the keywords based on the characteristic words of the email texts related to the conference and the keywords sequentially acquired from the conference voice; Based on the speech priority, muting speeches of the participants other than those of a specific person having the highest speech priority. A speech control device comprising a control unit for executing processing.

Images