Remote conferencing management system, remote conferencing management method, and program

Abstract

This remote conference management system comprises: a determination unit that determines whether or not a first terminal being used by a first user for participating in a remote conference and a second terminal being used by a second user for participating in the remote conference are within a predetermined range; and a control unit that performs, when the determination unit determines that the first terminal and the second terminal are within the predetermined range, control such that one of the first terminal and the second terminal is used to allow the user of the other terminal to participate in the remote conference.

Description

Technical Field

[0001] The present invention relates to a remote conference management system, a remote conference management method, and a program.

Background Art

[0002] Patent Document 1 describes a technique in which a mobile terminal is brought into a vehicle on which vehicle equipment is mounted, and when the vehicle starts running, the mobile terminal starts communication with the vehicle equipment that detects an event such as a collision, and performs notification processing regarding the event via a communication network. [Prior Art Document] [Patent Document] Patent Document 1 Japanese Patent Application Laid-Open No. 2019-33427 Problems to be Solved

[0003] By the way, there is a problem that it is not easy for participants in a remote conference to appropriately select a means of participating in the remote conference. The present application aims to improve the operability for selecting a means of participating in a remote conference in order to solve the above problem. And, by extension, it contributes to the development of a sustainable transportation system by further improving traffic safety. General Disclosure

[0004] [[ID=2५]] In a first aspect of the present invention, a remote conference management system is provided. The remote conference management system includes a determination unit that determines whether a first terminal used by a first user to participate in a remote conference and a second terminal used by a second user to participate in the remote conference are within a predetermined range. The remote conference management system includes a control unit that, when the determination unit determines that the first terminal and the second terminal are within a predetermined range, performs control to allow the user of the other terminal to participate in the remote conference through one of the first terminal and the second terminal.

[0005] In the above remote conference management system, the control unit may preferentially select the terminal with a better communication status among the first terminal and the second terminal as the terminal for allowing the user of the other terminal to participate in the remote conference.

[0006] Any of the above remote conferencing management systems may include a terminal information acquisition unit that acquires first information indicating the locations of the first terminal and the second terminal. The determination unit may determine, based on the first information, whether or not the first terminal and the second terminal are located within a predetermined range.

[0007] In any of the above remote conferencing management systems, the terminal information acquisition unit may further acquire second information indicating at least one of the direction of movement and acceleration of the first terminal and the second terminal. The determination unit may determine, based on the first information and the second information, whether or not the first terminal and the second terminal are within a predetermined range.

[0008] Any of the above remote conferencing management systems may include a connection status information acquisition unit that acquires third information indicating the connection status of wireless communication between the first terminal and the second terminal. The determination unit may determine, based on the third information, whether the first terminal and the second terminal are within a predetermined range.

[0009] In any of the above remote conferencing management systems, if the determination unit determines that the first terminal and the second terminal are within a predetermined range, the control unit may present at least one of the first user and the second user to participate in the remote conferencing through one of the first terminals.

[0010] In any of the remote conferencing management systems described above, the control unit may select the first terminal as the terminal for allowing the user of the second terminal to participate in the remote conferencing if the first terminal is a terminal installed on a mobile device.

[0011] In any of the above remote conferencing management systems, the mobile entity may be a vehicle.

[0012] A second embodiment of the present invention provides a remote conference management method. The remote conference management method includes a step of determining whether a first terminal used by a first user to participate in a remote conference and a second terminal used by a second user to participate in the remote conference are located within a predetermined range. If it is determined that the first terminal and the second terminal are located within a predetermined range, the remote conference management method includes a step of controlling one of the terminals to allow the user of the other terminal to participate in the remote conference.

[0013] In a third aspect of the present invention, a program is provided. The program causes a computer to function as the remote conferencing management system described in any of the above.

[0014] It should be noted that the above summary of the invention does not enumerate all of its features. Furthermore, subcombinations of these features may also constitute an invention. [Brief explanation of the drawing]

[0015] [Figure 1] A conceptual diagram of the configuration of the remote conferencing system 5 in one embodiment is shown. [Figure 2] An example of the functional configuration of the remote conferencing management system 110 provided by the remote conferencing server 100 is shown. [Figure 3] The functional configuration of the communication terminal 20 equipped on the mobile device 10 is shown together with the display device 22, microphone 60, and imaging device 28. [Figure 4] An example of a sequence executed by user terminal 80, communication terminal 20, remote conferencing server 100, and user terminal 120 is shown. [Figure 5] A schematic example of the screen 500 displayed on the display device 22 or user terminal 120 is shown. [Figure 6] A schematic example of the screen 600 displayed on the display device 22 or user terminal 120 is shown. [Figure 7]This flowchart shows the processing procedure related to the process by which the communication terminal 20 identifies the speaker. [Figure 8] A schematic example of a screen 800 displayed on the display device 22 or user terminal 120 is shown. [Figure 9] The configuration of mobile body 910 as a modified example of mobile body 10 is shown. [Figure 10] An example using Computer 2000 is shown. [Modes for carrying out the invention]

[0016] The present invention will be described below through embodiments, but these embodiments are not intended to limit the scope of the claims. Furthermore, not all combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0017] Figure 1 conceptually shows the configuration of a remote conferencing system 5 in one embodiment. The remote conferencing system 5 comprises a mobile unit 10, a remote conferencing server 100, and a user terminal 120. In this embodiment, the mobile unit 10 is a vehicle.

[0018] The mobile unit 10 comprises a communication terminal 20, display devices 22a, 22b, 22c, and 22d, seats 11a, 11b, 11c, and 11d, an imaging device 28, microphones 60a, 60b, 60c, and 60d, and antennas 26a, 26b, and 26c.

[0019] Display devices 22a, 22b, 22c, and 22d are installed at positions corresponding to seats 11a, 11b, 11c, and 11d, respectively. In this embodiment, display devices 22a, 22b, 22c, and 22d may be collectively referred to as "display device 22". Display device 22 displays images for remote conferencing. Display device 22 also has a function to acquire instructions from user 90 via touch operation or the like.

[0020] The microphones 60a, 60b, 60c, and 60d are provided at positions corresponding to the seats 11a, 11b, 11c, and 11d, respectively. For example, the microphones 60a and 60b corresponding to the seats 11a and 11b, which are front seats, are provided near the display devices 22a and 22b, respectively. The microphones 60c and 60d corresponding to the seats 11c and 11d, which are rear seats, are also provided near the positions of the headrests of the seats 11c and 11d. The microphones 60a, 60b, 60c, and 60d may be collectively referred to as "microphone 60".

[0021] The remote conference server 100 can communicate with the communication terminal 20 and the user terminal 120. The remote conference server 100 manages the participants who participate in the remote conference. The communication terminal 20 and the user terminal 120 can communicate with each other.

[0022] The communication terminal 20 cooperates with the remote conference server 100 to provide functions for the passengers in the mobile body 10 to participate in the remote conference. The display device 22 and the microphone 60 are connected to the communication terminal 20. The communication terminal 20 causes the display device 22 to display the conference image received under the management of the remote conference server 100. The communication terminal 20 is connected to the microphone 60. The communication terminal 20 transmits the sound obtained from the microphone 60 as audio data to the user terminal 120, which is the terminal used by the participants in the conference.

[0023] Users 90a and 90b are passengers of the mobile device 10. User 90a is the user of user terminal 70. User 90b is the user of user terminal 80. User terminals 70 and 80 are, for example, smartphones. User terminals 70 and 80 may be portable terminals other than smartphones. User terminals 70 and 80 may be computing devices with communication functions. User terminals 70 and 80 may be mobile phones, PDAs, tablet computers, notebook computers, laptop computers, wearable computers, etc.

[0024] Antennas 26a, 26b, and 26c are fixedly installed at different positions within the mobile body 10. In this embodiment, antennas 26a, 26b, and 26c may be collectively referred to as "antenna 26". The communication terminal 20 performs wireless communication with a user terminal 70 or user terminal 80 held by a user 90 within the mobile body 10 via antennas 26. For example, the communication terminal 20 performs wireless communication using UWB (Ultra-Wide Band) with a user terminal 70 held by user 90a via antennas 26. For example, the communication terminal 20 calculates the position of the user terminal 70 based on the signal received during wireless communication using UWB with the user terminal 70, and determines which of seats 11a, 11b, 11c, and 11d user 90a is seated in based on the detected position of the user terminal 70.

[0025] User 90c is located outside the mobile device 10. User 90c is a user of user terminal 120. User terminal 120 may be, for example, a smartphone. User terminal 120 may be a computing device having communication capabilities. User terminal 120 may be a mobile device such as a mobile phone, PDA, tablet computer, notebook computer, laptop computer, or wearable computer, or it may be a desktop computer.

[0026] Here, we assume that users 90a, 90b, and 90c are participating in the same remote conference managed by the remote conference server 100. User 90a is participating in the remote conference using communication terminal 20, user 90b is participating in the remote conference using user terminal 80, and user 90c is participating in the remote conference using user terminal 120.

[0027] While user 90 is participating in a remote conference, communication terminal 20, user terminal 80, and user terminal 120 transmit current location information and movement speed information to the remote conference server 100 at predetermined time intervals. For example, communication terminal 20, user terminal 80, and user terminal 120 each transmit current location information calculated based on signals received from GNSS satellites to the remote conference server 100 at predetermined time intervals. User terminals 80 and 120 further transmit current movement speed information calculated from the time change of current location information to the remote conference server 100 at predetermined time intervals. Communication terminal 20 transmits current location information calculated from the time change of current location information, or movement speed information based on the vehicle speed information of the moving object 10, to the remote conference server 100 at predetermined time intervals.

[0028] The remote conferencing server 100 determines, based on the current location information and movement speed information received from the communication terminal 20, user terminal 80, and user terminal 120, that the communication terminal 20 and user terminal 80 are within a predetermined range and are being used to participate in the same remote conferencing while moving. In this case, the remote conferencing server 100 prompts users 90a and / or 90b through the communication terminal 20 and / or user terminal 80 so that both users 90 within the mobile body 10 can participate in the conference through either the communication terminal 20 or user terminal 80. For example, the remote conferencing server 100 prompts users 90a and / or 90b through the communication terminal 20 and / or user terminal 80 so that user 90b of user terminal 80 can participate in the conference through the communication terminal 20. This allows the communication terminal 20 to prompt users 90a and 90b to participate in the remote conferencing through a single communication line.

[0029] The communication terminal 20 detects the number of people on board the mobile body 10 based on images of the mobile body 10 captured by the imaging device 28, or information from seat sensors provided on each of the seats 11. When a user 90b attempts to join a conference through the communication terminal 20, the communication terminal 20 accepts the user 90b as a participant in the remote conference via the communication terminal 20, provided that the number of participants in the remote conference via the communication terminal 20 does not exceed the number of people detected within the mobile body 10. This prevents more users than the number of people on board the mobile body 10 from joining the remote conference. Consequently, even if a large number of participants join the remote conference via the communication terminal 20, the amount of data transmitted to the communication terminal 20 will not exceed the predetermined data transmission limit set for the remote conference.

[0030] When user 90 participates in a remote conference via communication terminal 20, communication terminal 20 obtains information from user 90, via display device 22, that identifies the user 90 seated in the corresponding seat 11, and stores this information as seating information. Alternatively, communication terminal 20 identifies the seat 11 where user 90 is seated based on signals received from the user terminal via wireless communication using UWB, and also receives information from the user terminal that identifies the user 90 seated in seat 11, and stores this information as seating information. Communication terminal 20 obtains the voice emitted by user 90 via microphone 60. Based on the voice obtained via microphone 60 and the seating information, communication terminal 20 identifies the current speaker user 90 from among the users 90 accepted as participants in the remote conference. Communication terminal 20 displays on display device 22 and user terminal 120 that the identified speaker is speaking. This allows each participant in the remote conference to be informed of who is speaking, even when a single communication terminal 20 accepts multiple users 90 as participants in the remote conference.

[0031] Figure 2 shows an example of the functional configuration of the remote conferencing management system 110 provided by the remote conferencing server 100. The remote conferencing server 100 is responsible for processing user 90 logins to remote conferencing, and processing for communication between user terminals 80, communication terminal 20, and user terminal 120 used by users to participate in remote conferencing.

[0032] The remote conferencing management system 110 may be implemented by one or more computers. The remote conferencing management system 110 comprises a determination unit 210, a control unit 220, a terminal information acquisition unit 230, and a connection status information acquisition unit 240.

[0033] The determination unit 210 determines whether the first terminal used by the first user to participate in the remote conference and the second terminal used by the second user to participate in the remote conference are located within a predetermined range. The first terminal is, for example, a communication terminal 20, and the second terminal is, for example, a user terminal 80. If the determination unit 210 determines that the first terminal and the second terminal are located within a predetermined range, the control unit 220 controls one of the terminals to allow the user of the other terminal to participate in the remote conference.

[0034] The control unit 220 may prioritize selecting the terminal with better communication status from the first terminal and the second terminal as the terminal for allowing the user of the other terminal to participate in the remote conference. If the first terminal is a terminal installed on the mobile unit 10, the control unit 220 may select the first terminal as the terminal for allowing the user of the second terminal to participate in the remote conference.

[0035] The terminal information acquisition unit 230 acquires first information indicating the locations of the first terminal and the second terminal. Based on the first information, the determination unit 210 determines whether the first terminal and the second terminal are located within a predetermined range.

[0036] The terminal information acquisition unit 230 may further acquire second information indicating at least one of the direction of movement and acceleration of the first terminal and the second terminal. Based on the first information and the second information, the determination unit 210 may determine whether the first terminal and the second terminal are within a predetermined range.

[0037] The connection status information acquisition unit 240 may acquire third information indicating the connection status of wireless communication between the first terminal and the second terminal. Based on the third information, the determination unit 210 may determine whether the first terminal and the second terminal are within a predetermined range.

[0038] If the determination unit 210 determines that the first terminal and the second terminal are within a predetermined range, the control unit 220 may prompt at least one of the first and second users to participate in the remote conference through one of the first and second terminals.

[0039] Figure 3 shows the functional configuration of the communication terminal 20 provided by the mobile device 10, along with the display device 22, microphone 60, and imaging device 28. The communication terminal 20 includes a participant information acquisition unit 310, a control unit 320, a speaker identification unit 330, a seating information acquisition unit 340, a person identification unit 350, a user number determination unit 360, a reception unit 370, a storage unit 380, and a communication unit 390.

[0040] The storage unit 380 is configured to include a non-volatile storage medium. The non-volatile storage medium may be a semiconductor memory such as flash memory. The storage unit 380 stores information necessary for the communication terminal 20 to perform processing. The storage unit 380 may store programs to be executed by the processor in the communication terminal 20.

[0041] The communication unit 390 is responsible for communication between the mobile device 10 and the outside world. Specifically, the communication unit 390 is responsible for communication between the remote conferencing server 100 and the user terminal 120. The communication unit 390 can also be responsible for communication between the mobile device 10 and the user terminal 80.

[0042] The imaging device 28 is positioned to capture images of the occupants seated in the seats 11. The imaging device 28 may include one or more imaging devices.

[0043] The communication terminal 20 is mounted on the mobile device 10 and performs processing for conducting a remote conference. The person identification unit 350 identifies the number of people on board the mobile device 10. The user number determination unit 360 determines the number of users that the communication terminal 20 can accept as participants in the remote conference, according to the number of people identified by the person identification unit 350. The user number determination unit 360 may further determine the number of users that the communication terminal 20 can accept as participants in the remote conference, depending on the communication status of the communication terminal 20.

[0044] The participant information acquisition unit 310 acquires information that one or more users participating in the remote conference using mobile terminals are located within the mobile device 10 when a first user is participating in the remote conference using a communication terminal 20. When the participant information acquisition unit acquires information that one or more users participating in the remote conference using mobile terminals are located within the mobile device 10, the control unit 320 may control the communication terminal 20 to allow at least some of the one or more users to participate in the remote conference, up to the number of users determined by the user number determination unit 360.

[0045] The communication unit 390 continuously transmits the current location information of the communication terminal 20 to the remote conferencing management system that manages the remote conferencing. The communication unit 390 further continuously transmits at least one of the movement speed information and acceleration information of the communication terminal 20 to the remote conferencing management system that manages the remote conferencing.

[0046] The passenger identification unit 350 identifies the number of people riding in the mobile body 10 based on information detected by seating sensors provided on the seat seats of the mobile body 10. The passenger identification unit 350 may also identify the number of people riding in the mobile body 10 by analyzing image information from within the mobile body 10 acquired by an imaging device provided in the mobile body 10.

[0047] The reception unit 370 accepts multiple users within the mobile device 10 as participants in the remote conference. The speaker identification unit 330 identifies the speaker from among the multiple users accepted by the reception unit 370 as participants in the remote conference. The control unit 320 controls the system to highlight the user identified as the speaker by the speaker identification unit 330 on the remote conference system.

[0048] The speaker identification unit 330 identifies the speaker from among multiple users based on at least one of (i) image information within the mobile body 10 acquired by the imaging device 28 mounted on the mobile body 10, and (ii) voice information acquired by the microphone 60 installed on the mobile body 10.

[0049] The seating information acquisition unit 340 acquires the seating positions of multiple users within the mobile unit 10. The speaker identification unit 330 identifies the speaker from among the multiple users based on the sound source direction detected from the audio information acquired by the multiple microphones 60 mounted on the mobile unit 10, the installation positions of the multiple microphones 60 within the mobile unit 10, and the seating positions of the multiple users acquired by the seating information acquisition unit 340.

[0050] The speaker identification unit 330 identifies the speaker from among multiple users based on the voice information acquired by each of the microphones 60 mounted on each of the multiple seats of the mobile body 10 and the seating positions of multiple users acquired by the seating information acquisition unit 340.

[0051] The seating information acquisition unit 340 acquires the seating positions of multiple users by acquiring information specifying the seating position of the mobile unit 10 from each of the multiple users when the reception unit 370 accepts multiple users as participants in a meeting. The seating information acquisition unit 340 may acquire the seating positions of multiple users based on the position of the user terminal identified from the wireless signal received from the user terminal held by each of the multiple users when the reception unit 370 accepts multiple users as participants in a meeting. For example, the seating information acquisition unit 340 may calculate the position of the user terminal 70 based on the UWB communication signal received through the antenna 26 between it and the user terminal 70, and acquire the position of user 90a based on the calculated position of the user terminal 70.

[0052] Figure 4 shows an example of a sequence of processes executed by user terminal 80, communication terminal 20, remote conferencing server 100, and user terminal 120. At the start of this sequence, it is assumed that there is no user 90 participating in the remote conferencing using communication terminal 20.

[0053] In S402, the passenger identification unit 350 of the communication terminal 20 identifies the number of people on board the mobile vehicle 10. The passenger identification unit 350 of the communication terminal 20 identifies the number of people on board the mobile vehicle 10 based on the image of the mobile vehicle 10 captured by the imaging device 28 and the information from the seat sensors provided on each of the seats 11.

[0054] In S404, the user count determination unit 360 determines the number of users that the reception unit 370 can accept, according to the number of people identified by the person identification unit 350. For example, the user count determination unit 360 determines the number of people identified by the person identification unit 350 as the upper limit of the number of users that the reception unit 370 can accept.

[0055] Here, the user count determination unit 360 may determine the number of users that the reception unit 370 can accept, according to the communication status of the communication unit 390 acquired by the connection status information acquisition unit 240. The worse the communication status of the communication unit 390 acquired by the connection status information acquisition unit 240, the fewer users the reception unit 370 can accept that the user count determination unit 360 may determine.

[0056] In S406, when user 90a performs an operation to log in to the remote conference using the communication terminal 20, the reception unit 370 accepts user 90a as a participant in the remote conference and performs login processing such as sending the authentication information entered during login to the remote conference server 100.

[0057] In S408, when user 90b uses user terminal 80 to log in to a remote conference in which user 90a is participating, user terminal 80 performs login processing, such as sending the authentication information entered during login to the remote conference server 100. Similarly, in S410, when user 90c uses user terminal 120 to log in to a remote conference in which user 90a is participating, user terminal 120 performs login processing, such as sending the authentication information entered during login to the remote conference server 100.

[0058] In the login process of S406, the seating information acquisition unit 340 acquires the seating position of user 90 within the mobile body 10. For example, as part of the login process, the seating information acquisition unit 340 displays a screen on the display device 22a for specifying the seating position of user 90a, and acquires information indicating the seating position entered by user 90a through the display device 22a. For example, the seating position may be information that identifies the seat 11 in which user 90a is seated. The seating information acquisition unit 340 maintains information as seating information that associates user identification information that identifies user 90a with seat identification information that identifies the seat 11 in which user 90a is seated.

[0059] Alternatively, the seating information acquisition unit 340 identifies the seat 11 where user 90 is seated based on signals received via wireless communication using UWB. For example, the seating information acquisition unit 340 calculates the distance between antenna 26a and user terminal 70, the distance between antenna 26b and user terminal 70, and the distance between antenna 26c and user terminal 70 based on TOA (Time of Arrival), and calculates the position of user terminal 70 by triangulation based on the calculated distances. The seating information acquisition unit 340 may calculate the signal arrival angle from user terminal 70 to each antenna 26 based on the difference in the time at which each antenna 26 receives the signal transmitted from user terminal 70, and calculate the position of user terminal 70 based on the calculated signal arrival angle and the position of each antenna 26. Based on the calculated position of user terminal 70 and the position of seat 11, the seating information acquisition unit 340 identifies the seat 11 where user 90a, who possesses user terminal 70, is seated. The seating information acquisition unit 340 may store as seating information information that associates user identification information of user 90a, which is pre-associated with the user terminal 70, with seat identification information that identifies the seat 11 in which user 90a is seated. The seating information acquisition unit 340 may receive user identification information that identifies user 90a from the user terminal 70 and store as seating information information that associates the received user identification information with seat identification information that identifies the seat 11 in which user 90a is seated. The login process may be performed using the user terminal 70, and the seating information acquisition unit 340 may receive user identification information entered into the user terminal 70 as part of the login process.

[0060] In S420, once the communication terminal 20 has completed logging into the remote conference, the communication terminal 20 begins the process of periodically transmitting location information and speed information to the remote conference server 100. For example, the communication unit 390 transmits the current location information calculated based on signals received from GNSS satellites, and the speed information calculated from the time change of the current location information or the vehicle speed information of the moving object 10, to the remote conference server 100 at predetermined time intervals. Similarly, once the user terminals 80 and 120 have completed logging into the remote conference, they begin the process of periodically transmitting location information and speed information to the remote conference server 100 (S422 and S424). The user terminals 80 and 120 may transmit the current speed information calculated from the time change of the current location information to the remote conference server 100. The terminal information acquisition unit 230 of the remote conference server 100 receives location information and speed information from the user terminals 80, 20, and 120. In steps S420, S422, and S424, user terminal 80, communication terminal 20, and user terminal 120 may start a process to periodically transmit acceleration information to the remote conferencing server 100 in addition to, or instead of, the speed of movement.

[0061] In S430, the determination unit 210 of the remote conferencing server 100 identifies the terminals being used by users 90 on the same mobile vehicle 10 to participate in the same remote conferencing. For example, the terminal information acquisition unit 230 identifies the terminals being used by users 90 on the same mobile vehicle 10 to participate in the same remote conferencing based on the location information and speed information received from user terminal 80, communication terminal 20, and user terminal 120, respectively, and the identification information of the terminals used by already logged-in users. For example, the determination unit 210 determines that user terminal 80 and communication terminal 20 are being used by users 90 on the same mobile vehicle 10 to participate in the same remote conferencing if the location of user terminal 80 and the location of communication terminal 20 are within a predetermined range, and the difference between the movement speed of user terminal 80 and the movement speed of communication terminal 20 is smaller than a predetermined value.

[0062] Here, the connection status information acquisition unit 240 may acquire information from the user terminal 80, the communication terminal 20, and the user terminal 120 indicating the connection status with other terminals via wireless communication, and based on the acquired information, identify the terminals used by the user 90 riding on the same mobile vehicle 10 to participate in the same remote conference. For example, if the user terminal 80 and the communication terminal 20 are connected by WiFi®, the communication unit 390 of the communication terminal 20 and the user terminal 80 transmit identification information and connection status information of one or more terminals connected by WiFi® to the remote conference server 100. The WiFi® connection status may include information indicating the received signal strength from other terminals. The connection status information acquisition unit 240 acquires the information transmitted from the communication terminal 20 and the user terminal 80, and the determination unit 210 identifies the terminals used by the user 90 riding on the same mobile vehicle 10 to participate in the same remote conference based on this information. The determination unit 210 may identify terminals connected via WiFi (registered trademark) and with a received signal strength higher than a predetermined value as terminals used by users 90 on the same mobile vehicle 10 to participate in the same remote conference.

[0063] In S432, the control unit 220 of the remote conferencing server 100 performs processing to send a switching suggestion to the communication terminal 20 and the user terminal 80 recommending that they switch the terminals from which they are participating in the remote conferencing. For example, the control unit 220 sends information to the communication terminal 20 and the user terminal 80 indicating that users 90a and 90b are participating in the same remote conferencing and that it is recommended that user 90b participate in the remote conferencing through the communication terminal 20. As a result, the participant information acquisition unit 310 of the communication terminal 20 acquires information that there is one or more users 90 participating in the remote conferencing using the user terminal 80 within the mobile device 10.

[0064] Here, the control unit 220 acquires the communication status of user terminal 80 and user terminal 120, and may designate the terminal with the best communication status as the terminal from which to participate in the remote conference. This allows multiple users 90 to participate in the remote conference through the communication line of the terminal with the most stable communication status.

[0065] In S434, the control unit 320 of the communication terminal 20 performs processing to switch the terminal from which a participant is joining the remote conference, and in S436, the communication terminal 20 sends a notification to the remote conference server 100 indicating that the switching of the participating terminal is complete. For example, in S434, the control unit 320 displays a screen on the display device 22 for receiving an operation from user 90b to acknowledge joining the remote conference through the communication terminal 20. When the communication terminal 20 detects that user 90b has performed the acknowledgment operation on the display device 22b, in S436, the communication terminal 20 sends a notification to the remote conference server 100 indicating that user 90b has joined the remote conference from the communication terminal 20.

[0066] Figure 4 illustrates a case in S434 where the control unit 320 performs processing to switch the terminals participating in the remote conference. However, the control unit 320 may only perform the processing to switch the terminals participating in the remote conference if the number of users does not exceed the number of users determined by the user count determination unit 360.

[0067] Figure 5 schematically shows an example of a screen 500 displayed on the display device 22 or user terminal 120. Screen 500 shows the case where user 90a is participating in the remote conference from the communication terminal 20, user 90b is participating in the remote conference from user terminal 80, and user 90c is participating in the remote conference from user terminal 120.

[0068] Screen 500 includes objects 510, 520, and 530. Object 510 includes, for example, a facial image of user 90a captured by the imaging device 28. Object 520 includes a facial image of user 90b captured by the user terminal 80. Object 530 includes a facial image of user 90c captured by the user terminal 120.

[0069] Figure 6 schematically shows an example of a screen 600 displayed on the display device 22 or the user terminal 120. Screen 600 shows the case where users 90a and 90b are participating in the remote conference from the communication terminal 20, and user 90c is participating in the remote conference from the user terminal 120.

[0070] Screen 600 includes objects 610, 620, and 630. Object 610 includes, for example, facial images of users 90a and 90b captured by the imaging device 28. Object 620 includes the same facial images of users 90a and 90b captured by the imaging device 28 as object 610. Object 630 includes a facial image of user 90c captured by the user terminal 120 and transmitted from the user terminal 120.

[0071] Thus, with respect to the communication terminal 20, even when multiple users 90 participate in a remote conference through a single communication terminal 20, the multiple users 90 can be treated as separate participants. This makes it possible for other participants to know that multiple users 90 are participating in the remote conference from the communication terminal 20.

[0072] In the example shown in Figure 6, the control unit 320 uses the image captured by the imaging device 28 as both the participant image of user 90a and the participant image of user 90b. In other configurations, the control unit 320 may extract the face images of user 90a and user 90b from the image captured by the imaging device 28, and process the extracted face image of user 90a as the participant image of user 90a and the extracted face image of user 90b as the participant image of user 90b.

[0073] Figure 7 is a flowchart showing the processing steps related to the process by which the communication terminal 20 identifies the speaker. The processing steps in this flowchart are repeatedly executed, for example, when the communication terminal 20 is working in cooperation with the remote conferencing server 100 to perform processing related to remote conferencing.

[0074] In S702, the speaker identification unit 330 detects voice from the information collected by the microphone 60. In S704, the speaker identification unit 330 identifies the speaker based on the voice information detected in S702 and the seating information.

[0075] For example, if user 90b is the speaker, the volume of the voice detected by microphone 60b will be louder than the volume of the voice detected by the other microphones 60a, 60c, and 60d. Therefore, the speaker identification unit 330 determines that the speaker is seated in the seat 11b located at the position corresponding to microphone 60b. The speaker identification unit 330 refers to the seating information and identifies user 90b, who is identified by the user identification information associated with the seat identification information of seat 11b, as the speaker.

[0076] In S706, the communication unit 390 transmits speaker information to the user terminal 120. For example, the communication unit 390 transmits the speaker identification information in the remote conferencing system to the user terminal 120 as speaker information.

[0077] In S708, the control unit 320 highlights the speaker's object. For example, if user 90b is the speaker, the control unit 320 highlights object 620. In this case, the control unit 320 does not highlight object 610.

[0078] Figure 8 schematically shows an example of a screen 800 displayed on the display device 22 or the user terminal 120. Screen 800 shows a case where users 90a and 90b are participating in a remote conference from the communication terminal 20, user 90c is participating in a remote conference from the user terminal 120, and user 90b is speaking.

[0079] Screen 800 includes objects 810, 820, and 830. Objects 810 and 830 are identical to objects 610 and 630, respectively. Object 820 includes, for example, facial images of users 90a and 90b captured by the imaging device 28. Since user 90b is identified as the speaker, object 820 is highlighted to indicate that user 90b is the speaker.

[0080] Thus, with the communication terminal 20, even when multiple users are participating in a remote conference through a single communication terminal 20, it is possible to inform other conference participants who the speaker is.

[0081] Figure 9 shows the configuration of mobile body 910 as a modified version of mobile body 10. Mobile body 910 differs from mobile body 10 in that it is equipped with a microphone array 960 instead of a microphone 60. Therefore, the differences from mobile body 10 will be explained here.

[0082] The microphone array 960 includes multiple microphones. The speaker identification unit 330 of the communication terminal 20 uses the microphone array 960 to localize the sound source.

[0083] For example, if the distance from the sound source to each microphone is different, the time it takes for sound waves to reach each microphone will be different, and the difference in arrival time to each microphone will be a function of the position of the sound source. The speaker identification unit 330 utilizes this principle to determine the direction of the sound source based on the audio signals acquired by the multiple microphones included in the microphone array 960. Based on the estimated direction of the sound source, the installation position of the microphone array within the mobile body 10, and the seating information acquired by the seating information acquisition unit 340, the speaker identification unit 330 can identify the speaker from among the users 90.

[0084] In the above embodiment, the speaker identification unit 330 identifies the speaker using the voice acquired by multiple microphones. Instead of using the voice acquired by multiple microphones, or in addition to using the voice acquired by multiple microphones, the speaker identification unit 330 may identify the speaker by recognizing the voice of the user 90. For example, the speaker identification unit 330 may acquire the speaker's voiceprint in advance when performing login processing, and associate the acquired voiceprint with the user 90's identification information and store it as voiceprint information. When a remote conference is taking place, the speaker identification unit 330 may extract a voiceprint from the voice acquired by microphones installed in the mobile device 10, and identify the speaker from among the multiple users 90 based on the extracted voiceprint and the voiceprint information.

[0085] In other configurations, the speaker identification unit 330 may identify the speaker from the facial images of multiple users 90 captured by the imaging device 28. For example, the speaker identification unit 330 may identify the speaker among the multiple users 90 based on the mouth movements in the facial images. In addition to the mouth movements in the facial images, the speaker identification unit 330 may further identify the speaker among the multiple users 90 based on the voice collected by the microphone.

[0086] In the embodiments described above, the mobile body 10 was assumed to be a vehicle. However, any mobile body other than a vehicle can be used as the mobile body 10. For example, the mobile body 10 may be transportation equipment such as a train, aircraft, or ship.

[0087] According to the remote conferencing system 5 of the embodiment described above, if multiple users 90 are participating in the same remote conferencing from different terminals within the mobile body 10, they can be prompted to join the conference through a single terminal. This allows multiple users to participate in the remote conferencing through a single communication line. Furthermore, the communication terminal 20 determines the upper limit of users who can participate in the remote conferencing through the communication terminal 20, according to the number of people on board the mobile body 10 and the communication status of the communication terminal 20, so that the communication volume of the communication terminal 20 does not exceed the predetermined communication volume set for the remote conferencing. In addition, when multiple users are participating in the remote conferencing through the mobile body 10, the current speaker is identified based on the voice acquired through multiple microphones installed on the mobile body 10 and seating information, and the display device 22 and user terminal 120 are shown that the identified speaker is speaking. This allows each participant in the remote conferencing to know who is speaking, even if a single communication terminal 20 is accepting multiple users 90 as participants in the remote conferencing.

[0088] Figure 10 shows an example of a computer 2000 in which multiple embodiments of the present invention may be embodied in whole or in part. A program installed on the computer 2000 can cause the computer 2000 to function as the remote conferencing management system 110 according to the embodiment, or as a part of the system, or as a device such as a communication terminal 20, or as a part of the device, to perform operations associated with the system or a part of the system, or as a device or as a part of the device, and / or to perform a process or a stage of the process according to the embodiment. Such a program may be executed by the CPU 2012 to cause the computer 2000 to perform specific operations associated with some or all of the processing procedures and blocks in the block diagram described herein.

[0089] The computer 2000 according to this embodiment includes a CPU 2012 and RAM 2014, which are interconnected by a host controller 2010. The computer 2000 also includes a ROM 2026, flash memory 2024, communication interface 2022, and input / output chip 2040. The ROM 2026, flash memory 2024, communication interface 2022, and input / output chip 2040 are connected to the host controller 2010 via an input / output controller 2020.

[0090] CPU2012 operates according to the programs stored in ROM2026 and RAM2014, thereby controlling each unit.

[0091] The communication interface 2022 communicates with other electronic devices via a network. The flash memory 2024 stores programs and data used by the CPU 2012 in the computer 2000. The ROM 2026 stores boot programs and / or hardware-dependent programs of the computer 2000, such as those executed by the computer 2000 upon activation. The input / output chip 2040 may also connect various input / output units, such as keyboards, mice, and monitors, to the input / output controller 2020 via input / output ports such as serial ports, parallel ports, keyboard ports, mouse ports, monitor ports, USB ports, and HDMI® ports.

[0092] The program is provided via a computer-readable storage medium such as a CD-ROM, DVD-ROM, or memory card, or via a network. RAM2014, ROM2026, or flash memory 2024 are examples of computer-readable storage media. The program is installed in flash memory 2024, RAM2014, or ROM2026 and executed by CPU2012. The information processing described within these programs is read by computer 2000, resulting in coordination between the program and the various types of hardware resources described above. The apparatus or method may be configured to realize the operation or processing of information in accordance with the use of computer 2000.

[0093] For example, when communication is performed between computer 2000 and an external device, CPU 2012 may execute a communication program loaded into RAM 2014 and, based on the processing described in the communication program, instruct the communication interface 2022 to perform communication processing. Under the control of CPU 2012, the communication interface 2022 reads the transmission data stored in the transmit buffer processing area provided in the recording medium such as RAM 2014 and flash memory 2024, sends the read transmission data to the network, and writes the received data received from the network to the receive buffer processing area provided on the recording medium.

[0094] Furthermore, CPU2012 may read all or necessary parts of a file or database stored on a recording medium such as flash memory 2024 into RAM2014, and perform various types of processing on the data in RAM2014. CPU2012 then writes the processed data back to the recording medium.

[0095] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and subjected to information processing. The CPU2012 may perform various types of processing on the data read from RAM2014, including various types of operations, information processing, conditional judgments, conditional branching, unconditional branching, information retrieval / replacement, etc., as described herein and specified by the program's instruction sequence, and write the results back to RAM2014. The CPU2012 may also retrieve information in files, databases, etc., within the recording medium. For example, if multiple entries are stored in the recording medium, each having an attribute value of a first attribute associated with an attribute value of a second attribute, the CPU2012 may search among the multiple entries for an entry that matches the condition where the attribute value of the first attribute is specified, read the attribute value of the second attribute stored in that entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies the predetermined condition.

[0096] The programs or software modules described above may be stored on or near computer-readable storage media on computer 2000. Recording media such as hard disks or RAM provided within a server system connected to a dedicated communication network or the Internet can be used as computer-readable storage media. Programs stored on computer-readable storage media may be provided to computer 2000 via the network.

[0097] A program installed on computer 2000, which causes computer 2000 to function as a remote conferencing management system 110, may interact with CPU 2012, etc., to cause computer 2000 to function as each part of the remote conferencing management system 110. The information processing described in these programs is read by computer 2000 and functions as each part of the remote conferencing management system 110, which is a concrete means of cooperation between software and the various hardware resources described above. Then, by realizing the calculation or processing of information according to the purpose of use of computer 2000 in this embodiment, a unique remote conferencing management system 110 suited to the purpose of use is constructed.

[0098] A program installed on computer 2000, which causes computer 2000 to function as a communication terminal 20, may interact with CPU 2012, etc., to cause computer 2000 to function as various parts of the communication terminal 20. The information processing described in these programs is read by computer 2000 and functions as various parts of the communication terminal 20, which are concrete means of collaboration between software and the various hardware resources described above. Then, by realizing the calculation or processing of information according to the purpose of use of computer 2000 in this embodiment, a communication terminal 20 specific to the purpose of use is constructed.

[0099] Various embodiments have been described with reference to block diagrams, etc. In a block diagram, each block may represent (1) a stage in a process in which an operation is performed, or (2) a part of a device that has the role of performing an operation. A particular stage and part may be implemented by a dedicated circuit, a programmable circuit supplied with computer-readable instructions stored on a computer-readable storage medium, and / or a processor supplied with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuit may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. The programmable circuit may include reconfigurable hardware circuits, including logic AND, logic OR, logic XOR, logic NAND, logic NOR, and other logic operations, flip-flops, registers, memory elements such as field-programmable gate arrays (FPGAs), programmable logic arrays (PLAs), etc.

[0100] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by a suitable device, and as a result, a computer-readable storage medium having instructions stored therein constitutes at least part of a product containing instructions that can be executed to provide a means for performing an operation specified in a processing procedure or block diagram. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disk read-only memory (CD-ROM), digital multipurpose disc (DVD), Blu-ray (RTM) disc, memory stick, integrated circuit card, etc.

[0101] Computer-readable instructions may include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, Java®, C++, and traditional procedural programming languages ​​such as the C programming language or similar languages.

[0102] Computer-readable instructions may be provided locally or via a wide area network (WAN), such as a local area network (LAN) or the internet, to a processor or programmable circuit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, and may be executed to provide a means for performing the described processing procedure or the operation specified in the block diagram. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, and the like.

[0103] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications or improvements can be made to the above embodiments. It will be clear from the claims that such modified or improved forms may also be included in the technical scope of the present invention.

[0104] It should be noted that the execution order of operations, procedures, steps, and stages in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not explicitly stated as "before," "prior to," etc., and that these can be performed in any order unless the output of a previous process is used in a later process. Even if the operation flow in the claims, specifications, and drawings is described using phrases such as "first," "next," etc. for convenience, this does not mean that it is mandatory to perform the operations in that order. [Explanation of symbols]

[0105] 5. Remote conferencing system 10 Mobile Units 11 seats 20 Communication terminals 22 Display device 26 Antennas 28 Imaging device 60 Microphones 70 User terminals 80 User terminals 90 users 100 Remote conferencing servers 110 Remote Conferencing Management System 120 user terminals 380 Storage section 390 Communications Department 210 Judgment Department 220 Control Unit 230 Terminal Information Acquisition Unit 240 Connection status information acquisition unit 310 Participant information acquisition department 320 Control Unit 330 Speaker Identification Unit 340 Seating Information Acquisition Unit 350 Number of people identification department 360 User Count Determination Unit 370 Reception Department 500 screens 510 objects 520 objects 530 objects 600 screens 610 objects 620 objects 630 objects 800 screens 810 objects 820 objects 830 objects 910 Mobile Unit 960 Microphone Array 2000 Computer 2010 Host Controller 2012 CPU 2014 RAM 2020 Input / Output Controller 2022 Communication Interface 2024 Flash Memory 2026 ROM 2040 Input / Output Chip

Claims

1. A determination unit that determines whether a first terminal used by a first user to participate in a remote conference and a second terminal used by a second user to participate in the said remote conference are located within a predetermined range, When the determination unit determines that the first terminal and the second terminal are within a predetermined range, the control unit transmits information to the user of the other terminal recommending that the user of the other terminal participate in the remote conference through one of the first terminals. A remote conferencing management system equipped with [features / equipment].

2. The control unit selects the terminal with the better communication status from the first terminal and the second terminal as the terminal for allowing the user of the other terminal to participate in the remote conference, and transmits information to the user of the other terminal recommending that the user of the other terminal participate in the remote conference through the terminal with the better communication status. The remote conferencing management system according to claim 1.

3. Terminal information acquisition unit acquires first information indicating the locations of the first terminal and the second terminal. Furthermore, Based on the first information, the determination unit determines whether the first terminal and the second terminal are located within a predetermined range. The remote conferencing management system according to claim 1 or 2.

4. The terminal information acquisition unit further acquires second information indicating at least one of the movement direction and acceleration of the first terminal and the second terminal, The determination unit determines, based on the first information and the second information, whether the first terminal and the second terminal are located within a predetermined range. The remote conferencing management system according to claim 3.

5. Connection status information acquisition unit acquires third information indicating the connection status of wireless communication between the first terminal and the second terminal. Furthermore, Based on the third piece of information, the determination unit determines whether the first terminal and the second terminal are located within a predetermined range. The remote conferencing management system according to claim 1 or 2.

6. The control unit, if the first terminal is a terminal installed on a mobile device, selects the first terminal as the terminal for allowing the user of the second terminal to participate in the remote conference. The remote conferencing management system according to claim 1.

7. The aforementioned moving object is a vehicle. The remote conferencing management system according to claim 6.

8. A computer determines whether a first terminal used by a first user to participate in a remote conference and a second terminal used by a second user to participate in the remote conference are located within a predetermined range. When the computer determines that the first terminal and the second terminal are within a predetermined range, it transmits information to the user of the other terminal via one of the first terminals, recommending that the user of the other terminal participate in the remote conference. A remote conferencing management method that includes the following features.

9. A program for causing a computer to function as a remote conferencing management system according to claim 1 or 2.

Images