Meeting support device and meeting support method

Abstract

To provide a conference support device and a support method that control the conversation of a specific participant not to be transmitted to the entire conference during a remote conference.SOLUTION: In a conference support system, a conference support device 10 includes a main conference management unit that sets a main conference and manages the connections of terminal devices participating in the main conference, and an individual conference management unit that associates multiple individual conferences with the main conference and manages connections of the terminal devices participating in each of the multiple individual conferences. When the main conference management unit receives first utterance data indicating the content of utterances for the main conference from the terminal device, the main conference management unit transmits the first utterance data to the individual conference management unit. When the individual conference management unit receives the first utterance data from the main conference management unit, the individual conference management unit transmits the first utterance data to the terminal devices participating in each of the plurality of individual conferences. When the individual conference management unit receives second comment data indicating the utterance content for one individual conference from one terminal device participating in one individual conference, the individual conference management unit transmits the second utterance data to the terminal devices participating in one individual conference.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a conference support device and a conference support method. [Background technology]

[0002] In recent years, remote conference systems using the Internet have become common. For example, Patent Document 1 discloses a technology that allows specific participants to communicate privately during a conference. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 7-202887 Summary of the Invention [Problem to be solved by the invention]

[0004] When holding a conference involving multiple companies, there are times when conversations between participants from the same company are required, rather than between the entire group. In such cases, it is necessary to control the conversations of participants from the same company so that they are not heard by the entire group.

[0005] To provide a conference support device and a conference support method that can control the contents of conversations exchanged between specific participants during a remote conference so that the contents of conversations are not transmitted to the entire group in an unintended manner by a specific participant. [Means for solving the problem]

[0006] The conference support device of the present invention manages a main conference and multiple individual conferences to which terminal devices participating in the main conference belong, and includes: a main conference management unit that sets the main conference and manages the connections of the terminal devices participating in the main conference; and an individual conference management unit that associates the multiple individual conferences with the main conference and manages the connections of the terminal devices participating in each of the multiple individual conferences.When the main conference management unit receives first utterance data indicating the content of a speech made to the main conference from the terminal device, it transmits the first utterance data to the individual conference management unit.When the individual conference management unit receives the first utterance data from the main conference management unit, it transmits the first utterance data to the terminal devices participating in each of the multiple individual conferences.Furthermore, when the individual conference management unit receives second utterance data indicating the content of a speech made to an individual conference from a terminal device participating in an individual conference, it transmits the second utterance data to the terminal device participating in the individual conference.

[0007] A conference support method according to the present invention is a conference support method for managing a main conference and multiple individual conferences to which terminal devices participating in the main conference belong, the method comprising the steps of: setting the main conference and managing connections of the terminal devices participating in the main conference; associating the multiple individual conferences with the main conference and managing connections of the terminal devices participating in each of the multiple individual conferences; when first utterance data indicating the content of a speech made to the main conference is received from the terminal device, transmitting the first utterance data to the individual conference; when the first utterance data is received in the individual conference, transmitting the first utterance data to the terminal devices participating in each of the multiple individual conferences; and when second utterance data indicating the content of a speech made to one individual conference is received from one terminal device participating in the individual conference in the individual conference, transmitting the second utterance data to the terminal device participating in the one individual conference. [Effects of the Invention]

[0008] According to the present invention, it is possible to control the conversation of a specific participant during a remote conference so that it is not heard by everyone. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a conference system according to the first embodiment. [Figure 2] FIG. 2 is a block diagram showing an example of the configuration of the conference support device according to the first embodiment. [Figure 3] FIG. 3 is a block diagram showing an example of the configuration of a terminal device according to the first embodiment. [Figure 4] FIG. 4 is a diagram for explaining the remote conference method according to the first embodiment. [Figure 5] FIG. 5 is a flowchart showing the flow of the preparation process for the remote conference according to the first embodiment. [Figure 6] FIG. 6 is a flowchart showing the flow of the process of starting a remote conference according to the first embodiment. [Figure 7] FIG. 7 is a flowchart showing the flow of processing in the terminal device according to the first embodiment. [Figure 8] FIG. 8 is a diagram showing an example of a conference screen according to the first embodiment. [Figure 9] FIG. 9 is a diagram showing an example of a conference screen in the speaking state according to the first embodiment. [Figure 10] FIG. 10 is a diagram showing an example of a conference screen in the default state according to the first embodiment. [Figure 11] FIG. 11 is a diagram showing an example of a display mode of the conference screen according to the first embodiment. [Figure 12] FIG. 12 is a diagram showing an example of a display mode of the conference screen according to the first embodiment. [Figure 13] FIG. 13 is a diagram showing an example of a display mode of the conference screen according to the first embodiment. [Figure 14] FIG. 14 is a flowchart showing the flow of processing performed by the conference support device according to the first embodiment. [Figure 15] FIG. 15 is a flowchart showing the flow of processing performed by the conference support device according to the second embodiment. [Figure 16]FIG. 16 is a flowchart showing the flow of processing performed by the conference support device according to the third embodiment. [Figure 17] FIG. 17 is a flowchart showing the flow of the playback process according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to this embodiment, and in the following embodiments, the same components are designated by the same reference numerals, and redundant explanations will be omitted.

[0011] [First embodiment] (Conference system) An example of the configuration of a conference system according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of a conference system according to the first embodiment.

[0012] 1, the conference support system 1 includes a conference support device 10 and terminal devices 12-1 to 12-n (n is an integer equal to or greater than 2). When there is no need to distinguish between the terminal devices 12-1 to 12-n, they are collectively referred to as terminal devices 12. The conference support device 10 and the terminal devices 12 are communicatively connected via a network N. The conference support system 1 is a system for conducting a remote conference (e.g., a web conference) among multiple participants.

[0013] The conference support device 10 is a device that supports a remote conference. The conference support device 10 is realized by a server device or the like. The terminal device 12 is a terminal device used by the participants of the remote conference. In this embodiment, the terminal device 12 transmits the content of statements made by the participants of the remote conference to the conference support device 10. The conference support device 10 supports the remote conference by executing processing according to the content of the statements.

[0014] [Conference support equipment] An example of the configuration of the conference support device according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the configuration of the conference support device according to the first embodiment.

[0015] The conference support device 10 sets up and manages a main conference and sub-conferences in a remote conference attended by multiple participants. A main conference is a conference attended by all participants attending the remote conference. A sub-conference is a conference attended by specific participants attending the remote conference. A main conference is also called a master conference. A sub-conference is also called an individual conference. The conference support device 10 sets up one main conference in a remote conference. The conference support device 10 sets up multiple sub-conferences in a remote conference. The conference support device 10 supports the remote conference by managing comments made in the main conference and comments made in the sub-conferences. Here, setting up a main conference means performing initial settings to start a conference program for the main conference and then starting the program. Also, setting up a sub-conference means performing initial settings to start a conference program for the sub-conference and then starting the program.

[0016] 2, the conference supporting device 10 includes a communication unit 20, a storage unit 22, and a control unit 24. The conference supporting device 10 is configured, for example, by a general-purpose server device.

[0017] The communication unit 20 executes communication between the conference supporting apparatus 10 and an external device via the network N. The communication unit 20 executes communication between the conference supporting apparatus 10 and the terminal device 12, for example.

[0018] The storage unit 22 stores various types of information. The storage unit 22 stores information such as the contents of calculations performed by the control unit 24 and programs. As will be described later, the storage unit 22 stores audio data related to speeches made by participants in a remote conference. The storage unit 22 includes, for example, at least one of a RAM (Random Access Memory), a main storage device such as a ROM (Read Only Memory), and an external storage device such as an HDD (Hard Disk Drive).

[0019] The control unit 24 controls each unit of the conference supporting device 10. The control unit 24 includes, for example, an information processing device such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), and a storage device such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The control unit 24 executes a program that controls the operation of the conference supporting device 10 according to the present invention. The control unit 24 may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 24 may be realized by a combination of hardware and software.

[0020] The control unit 24 includes a conference information acquisition unit 30, a participant information acquisition unit 32, a type information acquisition unit 34, a main conference management unit 36, and an individual conference management unit 38.

[0021] The conference information acquisition unit 30 acquires conference information related to a remote conference from the terminal device 12 of the participant who is holding the remote conference. The conference information includes the duration of the remote conference, a main conference ID, and a sub-conference ID. The main conference ID includes, for example, access information, identification information of the main conference, and information on the type of the main conference. The sub-conference ID includes, for each sub-conference that has been set, access information, identification information of the sub-conference, and information on the type of the sub-conference. The conference information may include information other than the above information.

[0022] The participant information acquisition unit 32 acquires participant information about all participants participating in the remote conference from the terminal device 12 of the participant hosting the remote conference. The participant information includes participant identification information for identifying the participant, affiliation information, contact information, etc. The participant identification information is, for example, information about the participant's name and nickname. The affiliation information is, for example, information about the organization, such as the company, to which the participant belongs. The contact information is, for example, information about the participant's email address.

[0023] The type information acquisition unit 34 acquires, from the terminal device 12 of the participant who made the comment, type information of the comment indicating whether the comment made by the participant in the remote conference is a comment directed to the main conference or a subconference.

[0024] The main conference management unit 36 ​​manages connections between multiple terminal devices used by multiple participants based on the conference information acquired by the conference information acquisition unit 30 and the participant information acquired by the participant information acquisition unit 32. The main conference management unit 36 ​​sets up a main conference in which all participants can participate. Here, the timing of setting up the main conference may be as follows: when the main conference is not running and a registration request is received from a terminal device that can first participate in the main conference, the terminal device that made the registration request is authenticated. If the authentication is permitted, the main conference is set up and started, and a registration process is performed to permit the authenticated terminal device to connect to the conference and register the terminal device. Another timing for setting up the main conference may be, for example, to set up and start the main conference when a predetermined time before the main conference start time arrives and wait for a registration request from the terminal device. Another timing for setting up the main conference may be, for example, to set up and start the main conference when a connection request for association with the main conference is received from a subconference and the main conference has not been set up. The main conference manager 36 receives first utterance data indicating the content of utterances made by participants in the main conference from multiple terminal devices 12 via the communication unit 20. The main conference manager 36 transmits the first utterance data to the individual conference manager 38. Here, the utterance data may be transmitted together with additional information including at least terminal identification information and time information such as a timestamp, for identifying which terminal device 12 transmitted the audio data and when. This additional information may be added as header information to the utterance data. This additional information may be added to the beginning or end of the utterance data as header information. Alternatively, if the utterance data is transmitted in segments of a predetermined size, the additional information may be included as header information for each segment. The first utterance data from the multiple terminal devices 12 may be mixed by the main conference manager 36 and transmitted to the individual conference manager 38. When the first utterance data from the multiple terminal devices 12 is mixed by the main conference manager 36, the additional information may be re-attached to the data so as to include the terminal identification information of the utterance data to be mixed, and then transmitted.When the first utterance data from the multiple terminal devices 12 is not mixed by the main conference manager 36, the audio of the utterance data received at the receiving terminal may be mixed and played back based on time information such as a timestamp included in the first utterance data. Furthermore, the main conference manager 36 does not directly transmit the first utterance data to the multiple terminal devices 12.

[0025] The individual conference management unit 38 manages connections between the multiple terminal devices 12 used by each of the multiple participants based on the conference information acquired by the conference information acquisition unit 30 and the participant information acquired by the participant information acquisition unit 32. The individual conference management unit 38 sets up a subconference in which a specific participant from the multiple participants will participate. Here, the timing of setting up the subconference may be as follows: when the subconference to be attended by the terminal device 12 is not active and a registration request for connection to the conference and registration is received from the terminal device 12 that is first eligible to participate in the subconference, the terminal device 12 that made the registration request is authenticated; if authentication is permitted, the individual conference is set up and started, and a registration process is performed to permit the authenticated terminal device 12 to connect to the conference and register the terminal device 12. Another timing for setting up the subconference may be, for example, to set up and start the subconference when a predetermined time before the subconference start time arrives, and wait for a registration request from the terminal device 12. Here, when a new subconference is set up and started, the main conference manager 36 may be requested to send a connection request to the main conference to associate the subconference with the main conference. The individual conference manager 38 sets up multiple subconferences based on the subconference ID included in the conference information. The individual conference manager 38 receives first utterance data from the main conference manager 36. The individual conference manager 38 receives second utterance data indicating the content of utterances made to the subconference from the terminal devices 12 of participants participating in the subconference out of multiple terminal devices 12 via the communication unit 20. The individual conference manager 38 transmits the second utterance data to the terminal devices 12 of the participants participating in the subconference via the communication unit 20. Here, the utterance data may be transmitted together with additional information including at least terminal identification information and time information such as a timestamp, for identifying which terminal device 12 transmitted the voice data and when. This additional information may be added as header information to the utterance data. This additional information may be added to the beginning or end of the utterance data as header information, or if the utterance data is divided into pieces of a predetermined size and transmitted, the additional information may be included as header information for each piece of data.The second utterance data from the multiple terminal devices 12 may be mixed by the individual conference management unit 38 and transmitted to the terminal devices 12 of the participants participating in the subconference. If the second utterance data from the multiple terminal devices 12 is not mixed by the individual conference management unit 38, the audio of the utterance data received at the receiving terminal may be mixed and played back based on time information such as a timestamp included in the second utterance data. If the second utterance data from the multiple terminal devices 12 is mixed by the individual conference management unit 38, the additional information may be re-added to include the terminal identification information of the utterance data that was the subject of mixing before transmission. The individual conference management unit 38 transmits the first utterance data via the communication unit 20 to the terminal devices 12 of the participants participating in the subconference. Here, when the individual conference management unit 38 transmits the first utterance data to the terminal devices 12 of the participants participating in the subconference, the first utterance data and the second utterance data may be mixed and transmitted. When first utterance data and second utterance data from multiple terminal devices 12 are mixed by the individual conference management unit 38, it is preferable to re-attach additional information to include the terminal identification information of the utterance data to be mixed and transmit the additional information. Furthermore, when the first utterance data and second utterance data are mixed by the individual conference management unit 38, it is preferable to include identification flag information indicating that the first utterance data and second utterance data have been mixed in the additional information. When the individual conference management unit 38 does not mix the first utterance data and second utterance data, the audio of the utterance data received at the receiving terminal may be mixed and played back based on time information such as timestamps included in the first utterance data and second utterance data. Furthermore, the individual conference management unit 38 does not transmit the second utterance data to the main conference management unit 36. Note that by managing the content of utterances in a subconference managed by the individual conference management unit 38 so that it is not transmitted to the main conference managed by the main conference management unit 36, utterances to the subconference are reliably blocked from the main conference.

[0026] [Terminal Device] An example of the configuration of a terminal device according to the first embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing an example of the configuration of a terminal device according to the first embodiment.

[0027] 3, the terminal device 12 includes an input unit 50, a display unit 52, an audio output unit 54, a storage unit 56, a communication unit 58, and a control unit 60. The terminal device 12 is, for example, an information terminal device such as a personal computer, a tablet terminal, a smartphone, or a business wireless communication device. In this embodiment, the terminal device 12 is not particularly limited as long as it is an information terminal device with a communication function.

[0028] The input unit 50 accepts various input operations for the terminal device 12. The input unit 50 outputs an input signal corresponding to the accepted input operation to the control unit 60. The input unit 50 includes, for example, a keyboard, a mouse, a touch panel, buttons, switches, a microphone, etc. When a touch panel is used as the input unit 50, the input unit 50 is disposed on the display unit 52.

[0029] The display unit 52 displays various types of video. For example, the display unit 52 displays a remote conference setting screen and information related to the remote conference display screen. The display unit 52 is a type of notification unit. The display unit 52 is a display including, for example, a liquid crystal display (LCD), an organic electroluminescence (EL) display, etc.

[0030] The audio output unit 54 is a speaker that outputs audio. The audio output unit 54 outputs the audio uttered by the remote conference participants. Here, when multiple pieces of utterance data are received and it is necessary to play back the multiple pieces of utterance data at the same time, it is preferable to mix the utterance data based on time information such as timestamps and output the audio uttered by the remote conference participants.

[0031] The storage unit 56 stores information such as programs and the contents of calculations performed by the control unit 60. The storage unit 56 includes at least one of a RAM, a main storage device such as a ROM, and an external storage device such as an HDD.

[0032] The communication unit 58 executes communication between the terminal device 12 and an external device. The communication unit 58 executes communication between the terminal device 12 and the conference supporting device 10, for example.

[0033] The control unit 60 controls each unit of the terminal device 12. The control unit 60 has, for example, an information processing device such as a CPU or an MPU, and a storage device such as a RAM or a ROM. The control unit 60 executes a program that controls the operation of the terminal device 12 according to the present invention. The control unit 60 may be realized by, for example, an integrated circuit such as an ASIC or an FPGA. The control unit 60 may be realized by a combination of hardware and software.

[0034] [Management method] The remote conference management method according to the first embodiment will be described with reference to Fig. 4. Fig. 4 is a diagram for explaining the remote conference method according to the first embodiment.

[0035] In the example shown in Fig. 4, eight participants are assumed to be participating in the remote conference: User U1, User U2, User U3, User U4, User U5, User U6, User U7, and User U8. Users U1 to U3 belong to a first organization. User U4 belongs to a second organization different from the first organization. Users U5 to U8 belong to a third organization different from the first and second organizations.

[0036] The main conference management unit 36 ​​sets up a main conference 100 in which all users U1 to U8 participate. The individual conference management unit 38 sets up a subconference 110 in which users U1 to U3 participate. The individual conference management unit 38 sets up a subconference 120 in which user U4 participates. The individual conference management unit 38 sets up a subconference 130 in which users U5 to U8 participate. In other words, in this embodiment, participants in a remote conference participate in both the main conference and the subconference. The conference support device 10 manages a remote conference by setting up a hierarchical structure of a main conference and subconferences.

[0037] In this embodiment, by default, utterances of participants in a remote conference are shared only among participants in the same subconference. For example, the individual conference management unit 38, by default, transmits and receives second utterance data 141, which indicates the content of a utterance made by any one of users U1 to U3 participating in the subconference 110, between the terminal devices 12 of users U1 to U3 via the subconference 110. For example, the individual conference management unit 38, by default, transmits and receives second utterance data 142, which indicates the content of a utterance made by user U4 participating in the subconference 120, between the terminal device 12 of user U4 via the subconference 120. For example, the individual conference management unit 38, by default, transmits and receives third utterance data 143, which indicates the content of a utterance made by users U5 to U8 participating in the subconference 130, between the terminal devices 12 of users U5 to U8 via the subconference 130.

[0038] Users U1 to U8 can make statements in the main conference using the terminal device 12. For example, users U1 to U8 can make statements in the main conference by switching the state of the terminal device 12 from the default state to the speaking state. For example, when user U3 switches the state of the terminal device 12 to the speaking state and makes a statement in the main conference 100, the terminal device 12 of user U3 transmits the content of the statement to the main conference manager 36 as first utterance data 151.

[0039] The first utterance data 151 is transmitted from the main conference 100 to the subconferences 110, 120, and 130. That is, the main conference manager 36 transmits the first utterance data 151 to the individual conference manager 38.

[0040] In the subconference 110, the individual conference manager 38 transmits the first utterance data 151 from the user U1 to the terminal device 12 of the user U3.

[0041] In the subconference 120, the individual conference manager 38 transmits the first utterance data 151 to the terminal device 12 of the user U4.

[0042] In the subconference 130, the individual conference manager 38 transmits the first utterance data 151 from the user U5 to the terminal device 12 of the user U8.

[0043] That is, in this embodiment, the second utterance data 141, the second utterance data 142, and the third utterance data 143, which indicate the content of utterances made in the subconferences 110, 120, and 130, are not transmitted to the main conference 100. This prevents the content of utterances made in a subconference from being transmitted to another subconference.

[0044] Furthermore, in this embodiment, a utterance to the main conference 100 is transmitted from user U1 to the terminal device 12 of user U8 via the subconferences 110, 120, and 130. That is, in this embodiment, a subconference transmits second utterance data indicating the content of the utterance to the same subconference and first utterance data to the main conference to each participant in the same subconference.

[0045] [Meeting preparation process] The preparation process for a remote conference according to the first embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the flow of the preparation process for a remote conference according to the first embodiment. The process shown in Fig. 5 is executed by the terminal device 12 of the organizer who will be holding the remote conference.

[0046] The control unit 60 acquires participant information of the participants of the remote conference (step S10). Specifically, the control unit 60 acquires the participant information input by the host of the remote conference to the input unit 50. Then, the process proceeds to step S12.

[0047] The control unit 60 acquires conference information of the remote conference (step S12). Specifically, the control unit 60 acquires the conference information that the host of the remote conference inputted to the input unit 50 on the system screen. Then, the process proceeds to step S14.

[0048] The control unit 60 transmits the teleconference holding information to the participants of the teleconference (step S14). Specifically, the control unit 60 transmits the holding information including the main conference ID and the sub conference ID to the terminal devices 12 of the participants of the teleconference via the communication unit 58. Then, the processing of FIG. 5 ends.

[0049] [Meeting start process] The process of starting a remote conference according to the first embodiment will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the flow of the process of starting a remote conference according to the first embodiment. The process shown in Fig. 6 is executed by the conference supporting device 10 when starting a remote conference.

[0050] The participant information acquiring unit 32 acquires participant information from the terminal device 12 of the participant participating in the remote conference (step S20). Specifically, the participant information acquiring unit 32 acquires the participant information when the participant starts communication with the conference supporting device 10 using the terminal device 12 to participate in the remote conference. Then, the process proceeds to step S22.

[0051] The individual conference management unit 38 sets up a subconference (step S22). Specifically, the individual conference management unit 38 reserves resources such as a network session for calls between participants in the subconference. The individual conference management unit 38 reserves resources for transmitting second utterance data, which is a digitalized version of the utterance content related to the utterance to the subconference, to the same subconference. The second utterance data includes various information such as speaker information, audio data, and image data. The individual conference management unit 38 registers the subconference in the main conference managed by the main conference management unit 36. Then, the process proceeds to step S24.

[0052] The main conference manager 36 sets up the main conference (step S24). Specifically, the main conference manager 36 accepts registrations of all participants in the remote conference. The main conference manager 36 reserves resources for transmitting first utterance data, which is a digitalized version of utterance content related to utterances made to the main conference, to the subconference. The first utterance data includes various types of information such as speaker information, audio data, and image data. Then, the processing of FIG. 6 ends.

[0053] [Terminal device processing] The processing content of the terminal device according to the first embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing the flow of processing of the terminal device according to the first embodiment. Fig. 7 shows processing executed by the terminal device 12 of each participant when a remote conference is being held.

[0054] The control unit 60 starts the remote conference (step S30). Specifically, the control unit 60 starts the remote conference by establishing communication with the conference supporting device 10 and registering for the main conference and the sub-conference. Then, the process proceeds to step S32.

[0055] The control unit 60 causes the display unit 52 to display a conference screen (step S32). Fig. 8 is a diagram showing an example of the conference screen according to the first embodiment. In the example shown in Fig. 8, the conference screen 200 includes a first group 210, a second group 220, a third group 230, a fourth group 240, and a speak button 250.

[0056] The first group 210 indicates the group to which the participant using the terminal device 12 belongs. Three users, user U1, user U2, and user U3, belong to the first group 210. The columns for user U1, user U2, and user U3 display names, nicknames, and the like. The participants belonging to the first group 210 are registered in the first sub-conference.

[0057] The second group 220 indicates a group different from the first group 210. Three users, user U4, user U5, and user U6, belong to the second group 220. The columns for user U4, user U5, and user U6 display names, nicknames, and the like. The participants belonging to the second group 220 are registered in a second sub-conference that is different from the first sub-conference.

[0058] The third group 230 indicates a group different from the first group 210 and the second group. Two users, user U7 and user U8, belong to the third group 230. The columns for user U7 and user U8 display names, nicknames, etc. The participants belonging to the third group 230 are registered in a third sub-conference that is different from the first sub-conference and the second sub-conference.

[0059] The fourth group 240 indicates a group different from the first group 210, the second group 220, and the third group 230. One user, user U9, belongs to the fourth group 240. The column for user U9 displays the user's name, nickname, etc. The participants belonging to the fourth group 240 are registered in a fourth sub-conference that is different from the first sub-conference, the second sub-conference, and the third sub-conference.

[0060] The speak button 250 is a button that is pressed when making a statement in the main conference. A user of the terminal device 12 can switch the state of the terminal device 12 from the default state to the speaking state by pressing the speak button 250 using the input unit 50. For example, in this embodiment, the speaking state is maintained only while the speak button 250 is pressed. In this case, the type information acquisition unit 34 acquires type information indicating that a statement is made to the main conference while the speak button 250 is pressed. When the speak button 250 is not pressed, the type information acquisition unit 34 acquires type information indicating whether a statement is made to the subconference. Pressing the speak button 250 is one type of predetermined operation that is determined to be a statement made in the main conference.

[0061] Returning to FIG. 7, the control unit 60 determines whether or not the user of the terminal device 12 has spoken (step S34). Specifically, the control unit 60 determines that a speech has been made when there is voice input to the input unit 50. If it is determined that a speech has been made (step S34; Yes), the process proceeds to step S36. If it is not determined that a speech has been made (step S34; No), the process proceeds to step S42.

[0062] If the determination in step S34 is Yes, the control unit 60 determines whether or not the state is in the speaking state (step S36). Specifically, the control unit 60 determines that the state is in the speaking state while the speak button 250 is pressed, and determines that the state is in the default state when the speak button 250 is not pressed. If the state is in the speaking state (step S36; Yes), the process proceeds to step S38. If the state is not in the speaking state (step S36; No), the process proceeds to step S40.

[0063] If the determination in step S36 is Yes, the control unit 60 transmits first utterance data regarding the utterance content for the main conference to the conference supporting device 10 via the communication unit 58 (step S38). FIG. 9 is a diagram showing an example of a conference screen in the speaking state according to the first embodiment. As shown in FIG. 9, the control unit 60 highlights the entire conference screen 200A. For example, the control unit 60 highlights the entire conference screen 200A by displaying it in red. The control unit 60 highlights the column of user U1 who is speaking. For example, the control unit 60 highlights the column of user U1 by displaying it in green. Then, the process proceeds to step S42.

[0064] If the determination in step S36 is No, the control unit 60 transmits second utterance data regarding the utterance content for the subconference to the conference supporting device 10 via the communication unit 58 (step S40). FIG. 10 is a diagram showing an example of a conference screen in a default state according to the first embodiment. As shown in FIG. 10, the control unit 60 highlights the column of user U1 who is speaking, as shown in conference screen 200B. For example, the control unit 60 highlights the column of user U1 by displaying it in green. Then, the process proceeds to step S42.

[0065] If the determination in step S34 is No, after step S38 and after step S40, the control unit 60 determines whether or not utterance data has been received from the conference supporting device 10 (step S42). If it is determined that utterance data has been received (step S42; Yes), the control unit 60 proceeds to step S44. If it is determined that utterance data has not been received (step S42; No), the control unit 60 proceeds to step S46.

[0066] If the determination in step S42 is Yes, the control unit 60 changes the display mode of the conference screen (step S44). Figures 11, 12, and 13 are diagrams showing examples of the display mode of the conference screen according to the first embodiment.

[0067] 11, when the control unit 60 receives first utterance data from user U4 of the second group 220, for example, the control unit 60 highlights the frame between the conference screen 200C and the second group 220. For example, the control unit 60 highlights the frame between the conference screen 200C and the second group 220 in yellow. The control unit 60 highlights the field for user U4. For example, the control unit 60 highlights the field for user U4 by displaying it in green.

[0068] 12, when the control unit 60 receives first utterance data from user U4 of the second group 220 and user U9 of the fourth group 240, for example, the control unit 60 highlights the frames of the conference screen 200D, the second group 220, and the fourth group 240. For example, the control unit 60 highlights the frames of the conference screen 200C, the second group 220, and the fourth group 240 in yellow. The control unit 60 highlights the columns of user U4 and user U9. For example, the control unit 60 highlights the columns of user U4 and user U9 by displaying them in green.

[0069] 13, when the control unit 60 receives first utterance data from user U2 who is in the same first group 210 as user U1 who is the user of the terminal device 12, the control unit 60 highlights the conference screen 200E and the frame of the first group 210. For example, the control unit 60 highlights the conference screen 200E and the frame of the first group 210 in yellow. The control unit 60 highlights the column for user U2. For example, the control unit 60 highlights the column for user U2 by displaying it in green.

[0070] 13, the control unit 60 may change the display mode of the conference screen 200E depending on whether the first utterance data is received from user U2 or the second utterance data is received. For example, the control unit 60 may highlight the entire conference screen 200E when the first utterance data is received from user U2, and may highlight only the column for the first group 210 when the second utterance data is received. This makes it easier for user U1 to know whether a utterance from user U2 is directed to the main conference or the sub-conference.

[0071] Returning to FIG. 7, if the determination in step S42 is No, or after step S44, the control unit 60 determines whether or not to end the process (step S46). Specifically, the control unit 60 determines to end the process when the remote conference has ended or when a process to end the remote conference has been accepted. If it is determined to end the process (step S46; Yes), the process in FIG. 7 ends. If it is not determined to end the process (step S46; No), the process proceeds to step S34.

[0072] [Terminal device processing] The processing content of the conference supporting device according to the first embodiment will be described with reference to Fig. 14. Fig. 14 is a flowchart showing the flow of processing of the conference supporting device according to the first embodiment. Fig. 14 shows processing executed by the conference supporting device 10 of each participant when a remote conference is being held.

[0073] The control unit 24 starts the remote conference (step S50). Specifically, the control unit 24 starts the remote conference by having the main conference manager 36 set up a main conference and the individual conference manager 38 set up a sub-conference and register it with the main conference. Then, the process proceeds to step S52.

[0074] The main conference manager 36 determines whether or not first utterance data for the main conference has been received (step S52). If it is determined that the first utterance data has been received (step S52; Yes), the process proceeds to step S54. If it is determined that the first utterance data has not been received (step S52; No), the process proceeds to step S58.

[0075] If the determination in step S52 is Yes, the main conference manager 36 transmits the first utterance data to all subconferences set by the individual conference manager 38 (step S54). Specifically, the main conference manager 36 transmits the first utterance data to the individual conference manager 38. Then, the process proceeds to step S56.

[0076] The individual conference manager 38 transmits the first utterance data to each terminal device 12 participating in each sub conference room (step S56), and then proceeds to step S58.

[0077] If the determination in step S52 is No, or after step S56, the individual conference manager 38 determines whether or not second utterance data for the subconference has been received (step S58). If it is determined that second utterance data has been received (step S58; Yes), the process proceeds to step S60. If it is not determined that second utterance data has been received (step S58; No), the process proceeds to step S62.

[0078] If the determination in step S58 is Yes, the individual conference manager 38 transmits the second utterance data to the terminal devices 12 that belong to the same subconference as the subconference to which the terminal device 12 that transmitted the second utterance data belongs (step S60), and then proceeds to step S62.

[0079] If the determination in step S58 is No, or after step S60, the control unit 24 determines whether or not to end the process (step S62). Specifically, the control unit 24 determines to end the process when the remote conference has ended. If it is determined to end the process (step S62; Yes), the control unit 24 ends the process of Fig. 14. If it is not determined to end the process (step S62; No), the control unit 24 proceeds to step S52.

[0080] As described above, in the first embodiment, a main conference and sub-conferences are set up in a remote conference, and control is performed so that comments from the main conference are transmitted to the sub-conferences and comments from the sub-conferences are not transmitted to the main conference. This makes it possible to control so that comments from a specific sub-conference are not transmitted to other sub-conferences.

[0081] [Second embodiment] [Terminal device processing] The processing contents of the conference supporting device according to the second embodiment will be described with reference to Fig. 15. Fig. 15 is a flowchart showing the flow of processing of the conference supporting device according to the second embodiment. Fig. 15 shows processing executed by the conference supporting device 10 of each participant when a remote conference is being held.

[0082] In the second embodiment, the terminal device 12 that transmitted the first utterance data is identified, and communication of the identified terminal device 12 is restricted. In the second embodiment, the terminal device 12 transmits the first utterance data and the second utterance data to the conference supporting device 10 together with terminal identification information for uniquely identifying the terminal device 12.

[0083] The processing from step S70 to step S74 is the same as the processing from step S50 to step S54 shown in FIG. 14, and therefore a description thereof will be omitted.

[0084] After step S74, the individual conference management unit 38 identifies the terminal device 12 that transmitted the first utterance data (step S76). Specifically, the individual conference management unit 38 identifies the terminal device 12 based on the terminal identification information included in the first utterance data. For example, the terminal identification information includes the ID of the terminal device 12, user information about the user who uses the terminal device 12, information about the subconference to which the terminal device 12 belongs, etc. The terminal identification information may also include other information. Then, the process proceeds to step S78.

[0085] The individual conference management unit 38 restricts communication with the identified terminal device 12 (step S78). Specifically, the individual conference management unit 38 prohibits transmission and reception of information between the identified terminal device 12 and the individual conference management unit 38. Then, the process proceeds to step S80.

[0086] The individual conference manager 38 transmits the first utterance data to terminal devices 12 other than the identified terminal device 12 (step S80). As a result, the terminal device 12 of the user who is speaking in the main conference will no longer output audio related to the first utterance data and the second utterance data while the user is speaking, improving usability. Then, the process proceeds to step S82.

[0087] The individual conference management unit 38 transmits the withdrawal information of the identified terminal device 12 to the terminal devices 12 participating in the same subconference as the identified terminal device 12 (step S82). This allows the users of the terminal devices 12 participating in the subconference to easily understand that a user of the same subconference is speaking in the main conference. Then, the process proceeds to step S84.

[0088] The processing from step S84 to step S88 is the same as the processing from step S58 to step S62 shown in Fig. 14, and therefore description thereof will be omitted. Note that, in step S84, if second utterance data is received from a terminal device 12 that belongs to the same subconference as the terminal device 12 that transmitted the first utterance data, the second utterance data is not transmitted to the terminal device 12 that transmitted the first utterance data in step S86.

[0089] As described above, in the second embodiment, while a user is speaking in the main conference, transmission of the first utterance data and the second utterance data to the user's terminal device is prohibited. This prevents the terminal device 12 from outputting audio while the user is speaking, improving usability.

[0090] [Third embodiment] [Terminal device processing] The processing contents of the conference supporting device according to the third embodiment will be described with reference to Fig. 16. Fig. 16 is a flowchart showing the flow of processing of the conference supporting device according to the third embodiment. Fig. 16 shows processing executed by the conference supporting device 10 of each participant when a remote conference is being held.

[0091] In the third embodiment, when a statement is made in a subconference while communication between the terminal device 12 and the individual conference management unit 38 is restricted in the second embodiment, a process is executed to store the content of the statement.

[0092] The processes from step S100 to step S114 are the same as the processes from step S70 to step S84 shown in FIG. 15, respectively, and therefore will not be described again.

[0093] If the determination in step S114 is Yes, the individual conference manager 38 determines whether the second utterance data was received from a terminal device 12 that belongs to the same subconference as the terminal device 12 that transmitted the first utterance data (step S116). Specifically, the individual conference manager 38 determines whether the second utterance data was received from a terminal device 12 that belongs to the same subconference as the terminal device 12 that transmitted the first utterance data, based on the terminal identification information included in the second utterance data. If it is determined that the second utterance data was received from a terminal device 12 that belongs to the same subconference as the terminal device 12 that transmitted the first utterance data (step S116; Yes), the process proceeds to step S118. If it is not determined that the second utterance data was received from a terminal device 12 that belongs to the same subconference as the terminal device 12 that transmitted the first utterance data (step S116; No), the process proceeds to step S120.

[0094] If the determination in step S116 is Yes, the individual conference manager 38 stores the second utterance data received while communication between the individual conference manager 38 and the terminal device 12 that transmitted the first utterance data is restricted as voice data in the storage unit 22 (step S118). At this time, the main conference manager 36 may store the first utterance data received while communication between the individual conference manager 38 and the terminal device 12 that transmitted the first utterance data is restricted as voice data. The first utterance data and the second utterance data may be stored in the storage unit 22 as separate voice data, or may be stored as one voice data. Then, the process proceeds to step S120.

[0095] The processes in steps S120 and S122 are the same as the processes in steps S86 and S88 shown in FIG. 15, respectively, and therefore will not be described further.

[0096] The playback process of audio data according to the third embodiment will be described with reference to Fig. 17. Fig. 17 is a flowchart showing the flow of the playback process according to the third embodiment. Fig. 17 shows the process executed by the terminal device 12 of the user who spoke in the main conference after the speech in the main conference has ended.

[0097] The control unit 60 transmits a playback request corresponding to the operation information input to the input unit 50 to the conference supporting device 10 via the communication unit 58 (step S130). Specifically, the user can select, as the playback request, audio data from only the main conference room, audio data from only the sub-conference room, or audio data from both the main conference room and the sub-conference room. Then, the process proceeds to step S132.

[0098] The control unit 60 acquires the audio data corresponding to the playback request from the conference supporting device 10 via the communication unit 58 (step S132), and then proceeds to step S134.

[0099] The control unit 60 causes the acquired audio data to be output from the audio output unit 54 (step S134).

[0100] As described above, the third embodiment allows a user to play back audio data of conversations that took place while the user's terminal device was prohibited from transmitting the first utterance data and the second utterance data while the user was speaking in the main conference, thereby enabling the user to understand the content of the conversations that took place in the main conference and the subconference while communication was restricted.

[0101] The components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads and usage conditions. This distribution and integration configuration may also be performed dynamically.

[0102] In an embodiment of the present invention, the same conference support device 10 is configured to manage the main conference and the individual conferences, but this does not necessarily have to be achieved using the same conference support device 10. For example, multiple conference support devices 10 may be prepared separately, connected to each other via a communication network, and configured so that one conference support device manages the main conference and the other conference support device manages the individual conferences, and the multiple conference support devices may be configured to operate in cooperation with each other.

[0103] Although the embodiments of the present invention have been described above, the present invention is not limited to the contents of these embodiments. Furthermore, the above-described components include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-described components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the above-described embodiments.

[0104] This disclosure will contribute to the realization of the SDG "Build resilient infrastructure, promote inclusive and sustainable industrialization, and foster innovation," and includes matters that contribute to value creation through IoT solutions. [Explanation of symbols]

[0105] 1. Meeting support system 10. Meeting support equipment 12 Terminal Equipment 20,58 Communications Department 22,56 Storage section 24,60 Control unit 30 Meeting Information Acquisition Unit 32 Participant information acquisition department 34 Type information acquisition unit 36 Main Conference Management Department 38 Individual Conference Management Department 50 Input section 52 Display section 54 Audio output section

Claims

1. A conference support device that manages a main conference and a plurality of individual conferences to which terminal devices participating in the main conference belong, a main conference management unit that sets the main conference and manages connections of terminal devices that participate in the main conference; an individual conference management unit that associates the plurality of individual conferences with the main conference and manages connections of terminal devices that participate in each of the plurality of individual conferences; Including, The main conference management unit When first utterance data indicating the content of a utterance made to the main conference is received from the terminal device, the first utterance data is transmitted to the individual conference management unit; The individual conference management unit when receiving the first utterance data from the main conference management unit, transmitting the first utterance data to the terminal devices participating in each of the plurality of individual conferences; Furthermore, when the individual conference management unit receives second utterance data indicating utterance content for one individual conference from one terminal device participating in the one individual conference, the individual conference management unit transmits the second utterance data to the terminal device participating in the one individual conference. Conference support equipment.

2. the first utterance data includes terminal identification information for uniquely identifying the plurality of terminal devices; The individual conference management unit Identifying the terminal identification information included in the first utterance data, and transmitting the first utterance data and the second utterance data to a terminal device other than the terminal device that transmitted the first utterance data. The conference support device according to claim 1 .

3. further comprising a storage unit that stores the first utterance data and the second utterance data; The individual conference management unit prohibiting transmission and reception between the terminal device transmitting the first utterance data and the individual conference management unit while the first utterance data is being transmitted, and storing individual conference data indicative of the content of the utterances made in the individual conference during which transmission and reception is prohibited in a storage unit; after the terminal device that transmitted the first utterance data has finished speaking in the main conference, transmitting the individual conference data to the terminal device that transmitted the first utterance data in response to a request from the terminal device that transmitted the first utterance data; The conference support device according to claim 2 .

4. a conference information acquisition unit that acquires type information from the terminal device; the main conference management unit receives the first utterance data when the identification information acquired by the conference information acquisition unit is main conference information. The conference support device according to claim 1 .

5. A conference support method executed by a computer of a conference support device that manages a main conference and a plurality of individual conferences to which terminal devices participating in the main conference belong, comprising: setting the main conference and managing connections of terminal devices participating in the main conference; a step of associating the plurality of individual conferences with the main conference and managing connections of terminal devices participating in each of the plurality of individual conferences; a step of transmitting, when receiving first utterance data indicating content of a utterance made to the main conference from the terminal device, the first utterance data to the individual conference; a step of transmitting, when the first utterance data is received in the individual conferences, the first utterance data to the terminal devices participating in each of the plurality of individual conferences; a step of, when receiving second utterance data indicating utterance content for one individual conference from one terminal device participating in the one individual conference, transmitting the second utterance data to the terminal device participating in the one individual conference; A meeting support method including:

Images