Communication terminal, method, and program

Abstract

A communication terminal, which is mounted on a mobile body and is used for performing a teleconference, is provided with: a reception unit that receives a plurality of users in the mobile body as participants of the teleconference; a speaker specifying unit that specifies a speaker from among a plurality of users accepted as participants of the teleconference by the accepting unit; and a control unit that performs control for highlighting the user identified as the speaker by the speaker identification unit on the teleconference system.

Description

TECHNICAL FIELD

[0001] The present application relates to a communication terminal, a method, and a program. BACKGROUND

[0002] Patent Literature 1 describes a technology in which when a portable terminal is taken into a vehicle on which a vehicle device is mounted and the vehicle starts running, the portable terminal starts communicating with the vehicle device that detects an event such as a collision and performs notification processing related to the event via a communication network.

[0003] Patent Literature 1: Japanese Patent Application Publication No. 2019-33427 SUMMARY

[0004] However, when a plurality of users participate in a remote conference on one line by communication terminals provided on a moving body, there is a problem that it is not easy to properly prompt who among the plurality of users is speaking to other participants. To solve the above problem, the object of the present application is to improve operability in a case where a plurality of users participate in a remote conference by communication terminals provided on a moving body. In addition, it will further improve the safety of traffic and contribute to the development of a sustainable transport system.

[0005] In a first mode of the present application, a communication terminal is provided. The communication terminal is mounted on a moving body and used for conducting a remote conference. The communication terminal has an acceptance unit that accepts a plurality of users in the moving body as participants of the remote conference. The communication terminal has a speaker determination unit that determines a speaker from among the plurality of users accepted by the acceptance unit as participants of the remote conference. The communication terminal has a control unit that performs control for highlighting the user determined by the speaker determination unit as the speaker on a remote conference system.

[0006] In the above communication terminal, the speaker determination unit can determine the speaker from among the plurality of users based on at least one of (i) image information of the moving body acquired by an imaging device mounted on the moving body and (ii) sound information acquired by a microphone provided on the moving body.

[0007] In any of the above communication terminals, a seating information acquisition unit that acquires seating positions of the plurality of users in the moving body can also be provided. The speaker determination unit can determine the speaker from among the plurality of users based on a sound source direction detected from sound information acquired by a plurality of microphones mounted on the moving body, the arrangement positions of the plurality of microphones in the moving body, and the seating positions of the plurality of users acquired by the seating information acquisition unit.

[0008] In any of the above communication terminals, a seating information acquisition section that acquires seating positions of the plurality of users in the mobile body can also be provided. The speaker determination section can determine a speaker from among the plurality of users based on sound information acquired by each microphone mounted corresponding to each seat of the mobile body and the seating positions of the plurality of users acquired by the seating information acquisition section.

[0009] In any of the above communication terminals, when the reception section receives the plurality of users as participants of a conference, the seating information acquisition section can acquire the seating positions of the plurality of users by acquiring information that specifies seat positions of the mobile body from each of the plurality of users.

[0010] In any of the above communication terminals, when the reception section receives the plurality of users as participants of a conference, the seating information acquisition section can acquire the seating positions of the plurality of users based on positions of user terminals held by the plurality of users respectively, determined from wireless signals received from the user terminals.

[0011] In any of the above communication terminals, the mobile body can be a vehicle.

[0012] In a second aspect of the present application, a method is provided. The method is a method executed by a communication terminal mounted on a mobile body and used for conducting a remote conference. The method has a step of receiving a plurality of users in the mobile body as participants of the remote conference. The method has a step of determining a speaker from among the plurality of users received as participants of the remote conference. The method has a step of conducting control for highlighting a user determined as the speaker from among the plurality of users on a remote conference system.

[0013] In a third aspect of the present application, a program is provided. The program is for causing a computer to function as any of the above communication terminals.

[0014] In addition, the above summary of the present application does not list all features of the present application. Furthermore, sub-combinations of these feature groups can also be part of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 The configuration of the remote conference system 5 in an embodiment is conceptually shown.

[0016] Figure 2 An example of the functional configuration of the remote conference management system 110 provided by the remote conference server 100 is shown.

[0017] Figure 3 The functional configuration of the communication terminal 20 provided by the mobile body 10 is shown together with the display device 22, the microphone 60, and the imaging device 28.

[0018] Figure 4 An example of a sequence executed by the user terminal 80, the communication terminal 20, the remote conference server 100, and the user terminal 120 is shown.

[0019] Figure 5 An example of a screen 500 displayed on the display device 22 or the user terminal 120 is schematically shown.

[0020] Figure 6 An example of a screen 600 displayed on the display device 22 or the user terminal 120 is schematically shown.

[0021] Figure 7 is a flowchart showing a process procedure relating to a process of determining a speaker by the communication terminal 20.

[0022] Figure 8 An example of a screen 800 displayed on the display device 22 or the user terminal 120 is schematically shown.

[0023] Figure 9 A structure of a mobile body 910 that is a modification example of the mobile body 10 is shown.

[0024] Figure 10 An example of a computer 2000 is shown. DETAILED DESCRIPTION

[0025] Hereinafter, the present application will be described by embodiments of the application, but the following embodiments do not limit the claimed application. Furthermore, all combinations of features described in the embodiments are not necessarily essential to the solution of the present application.

[0026] Figure 1 A configuration of a remote conference system 5 in an embodiment is conceptually shown. The remote conference system 5 is provided with a mobile body 10, a remote conference server 100, and a user terminal 120. In the present embodiment, the mobile body 10 is a vehicle.

[0027] The mobile body 10 is provided with a communication terminal 20, a display device 22a, a display device 22b, a display device 22c, and a display device 22d, a seat 11a, a seat 11b, a seat 11c, and a seat 11d, an imaging device 28, a microphone 60a, a microphone 60b, a microphone 60c, and a microphone 60d, an antenna 26a, an antenna 26b, and an antenna 26c.

[0028] The display device 22a, the display device 22b, the display device 22c, and the display device 22d are respectively provided at positions corresponding to the seat 11a, the seat 11b, the seat 11c, and the seat 11d. In the present embodiment, the display device 22a, the display device 22b, the display device 22c, and the display device 22d are sometimes collectively referred to as "display devices 22". The display devices 22 display images for a teleconference. The display devices 22 also have a function of acquiring an instruction from the user 90 by a touch operation or the like.

[0029] The microphone 60a, the microphone 60b, the microphone 60c, and the microphone 60d are respectively provided at positions corresponding to the seat 11a, the seat 11b, the seat 11c, and the seat 11d. For example, the microphone 60a and the microphone 60b corresponding to the seat 11a and the seat 11b which are front-row seats are respectively provided near the display device 22a and the display device 22b. The microphone 60c and the microphone 60d corresponding to the seat 11c and the seat 11d which are rear-row seats are respectively provided near headrest positions of the seat 11c and the seat 11d. The microphone 60a, the microphone 60b, the microphone 60c, and the microphone 60d are sometimes collectively referred to as "microphones 60".

[0030] The teleconference server 100 can communicate with the communication terminal 20 and the user terminal 120. The teleconference server 100 manages participants who participate in a mobile teleconference. The communication terminal 20 and the user terminal 120 can communicate with each other.

[0031] The communication terminal 20 cooperates with the teleconference server 100 to provide a function for a passenger in the mobile body 10 to participate in a teleconference. The display device 22 and the microphone 60 are connected to the communication terminal 20. The communication terminal 20 causes the display device 22 to display a conference image received by management of the teleconference server 100. The communication terminal 20 is connected to the microphone 60. The communication terminal 20 transmits sound data acquired from the microphone 60 to the user terminal 120 which is a terminal used by a participant of a conference.

[0032] The user 90a and the user 90b are passengers of the mobile body 10. The user 90a is a user of the user terminal 70. The user 90b is a user of the user terminal 80. The user terminal 70 and the user terminal 80 are, for example, smartphones. The user terminal 70 and the user terminal 80 can be portable terminals other than smartphones. The user terminal 70 and the user terminal 80 can be computing devices having a communication function. The user terminal 70 and the user terminal 80 can be terminals of mobile phones, PDAs, tablet computers, notebook computers, laptop computers, wearable computers, or the like.

[0033] The antenna 26a, the antenna 26b, and the antenna 26c are fixedly provided in the mobile body 10 at positions different from each other. In the present embodiment, the antenna 26a, the antenna 26b, and the antenna 26c are sometimes collectively referred to as "antenna 26". The communication terminal 20 performs wireless communication between the user terminal 70 or the user terminal 80 held by the user 90 in the mobile body 10 via the antenna 26. For example, the communication terminal 20 performs wireless communication using UWB (Ultra-Wide Band) between the user terminal 70 held by the user 90a via the antenna 26. For example, the communication terminal 20 calculates the position of the user terminal 70 based on a signal received in the wireless communication using UWB between the user terminal 70, and judges which of the seats 11a, 11b, 11c, and 11d the user 90a is seated on based on the detected position of the user terminal 70.

[0034] The user 90c is located outside the mobile body 10. The user 90c is a user of the user terminal 120. For example, the user terminal 120 can be a smartphone. The user terminal 120 can be a computing device having a communication function. The user terminal 120 can be a portable terminal of a mobile phone, a PDA, a tablet computer, a notebook computer, a laptop computer, a wearable computer, or the like, or can be a desktop computer.

[0035] Here, it is assumed that the user 90a, the user 90b, and the user 90c participate in the same teleconference managed by the teleconference server 100. It is assumed that the user 90a participates in the teleconference using the communication terminal 20, the user 90b participates in the teleconference using the user terminal 80, and the user 90c participates in the teleconference using the user terminal 120.

[0036] During the period in which the user 90 participates in the teleconference, the communication terminal 20, the user terminal 80, and the user terminal 120 transmit current position information and moving speed information to the teleconference server 100 at predetermined time intervals. For example, the communication terminal 20, the user terminal 80, and the user terminal 120 transmit current position information calculated based on signals respectively received from GNSS satellites to the teleconference server 100 at predetermined time intervals. The user terminal 80 and the user terminal 120 also transmit current moving speed information calculated from a temporal change in the current position information to the teleconference server 100 at predetermined time intervals. The communication terminal 20 transmits current position information calculated from a temporal change in the current position information or moving speed information based on vehicle speed information of the mobile body 10 to the teleconference server 100 at predetermined time intervals.

[0037] The remote conference server 100 judges that the communication terminal 20 and the user terminal 80 are located within a predetermined range and are participating in the same remote conference while moving, based on the current position information and the moving speed information received from the communication terminal 20, the user terminal 80, and the user terminal 120. In this case, the remote conference server 100 prompts the user 90a and / or the user 90b through the communication terminal 20 and / or the user terminal 80 so that the users 90 of both in the moving body 10 participate in the conference through one of the communication terminal 20 and the user terminal 80. For example, the remote conference server 100 prompts the user 90a and / or the user 90b through the communication terminal 20 and / or the user terminal 80 so that the user 90b of the user terminal 80 participates in the conference through the communication terminal 20. Thereby, the communication terminal 20 can prompt the user 90a and the user 90b to participate in the remote conference through one communication line.

[0038] The communication terminal 20 detects the number of people who get on the moving body 10 based on the image of the moving body 10 taken by the imaging device 28 or the information of the seat sensor provided in each seat 11. When the user 90b attempts to participate in the conference through the communication terminal 20, the communication terminal 20 accepts the user 90b as a participant who participates in the remote conference through the communication terminal 20 with the condition that the number of participants who participate in the remote conference through the communication terminal 20 does not exceed the number of people detected in the moving body 10. Thereby, it is possible to prevent users whose number exceeds the number of people who get on the moving body 10 from participating in the remote conference. Further, by a plurality of participants participating in the remote conference through the communication terminal 20, it is possible to prevent the communication volume of the communication terminal 20 from exceeding the set communication volume which is set in advance for the remote conference.

[0039] When the user 90 participates in the remote conference through the communication terminal 20, the communication terminal 20 acquires information for identifying the user 90 who sits on the corresponding seat 11 from the user 90 through the display device 22 and holds it as seat information. In another method, the communication terminal 20 determines the seat 11 on which the user 90 sits based on a signal received from the user terminal by using wireless communication of UWB, and receives information identifying the user 90 who sits on the seat 11 from the user terminal and holds the information as the seat information. The communication terminal 20 acquires a voice uttered by the user 90 through the microphone 60. The communication terminal 20 determines the user 90 who is a current speaker from among the users 90 accepted as participants of the remote conference based on the voice acquired through the microphone 60 and the seat information. The communication terminal 20 causes the display device 22 and the user terminal 120 to display that the determined speaker is speaking. Thereby, even in a case where a plurality of users 90 are accepted as participants of the remote conference through a single communication terminal 20, it is possible to notify each participant of the remote conference who is speaking.

[0040] Figure 2An example of a functional configuration of a remote conference management system 110 possessed by the remote conference server 100 is shown. The remote conference server 100 is responsible for login processing of the user 90 to a remote conference, processing for communication between the user terminal 80, the communication terminal 20, and the user terminal 120 used by the user to participate in the remote conference, and the like.

[0041] The remote conference management system 110 can be implemented by one or more computers. The remote conference management system 110 is provided with a determination section 210, a control section 220, a terminal information acquisition section 230, and a connection state information acquisition section 240.

[0042] The determination section 210 determines whether a first terminal used by a first user to participate in a remote conference and a second terminal used by a second user to participate in the remote conference exist within a predetermined range. The first terminal is, for example, the communication terminal 20, and the second terminal is, for example, the user terminal 80. When the determination section 210 determines that the first terminal and the second terminal exist within the predetermined range, the control section 220 performs control to cause the user of the other terminal to participate in the remote conference through one of the first terminal and the second terminal.

[0043] The control section 220 can preferentially select a terminal having a good communication state among the first terminal and the second terminal as a terminal for causing the user of the other terminal to participate in the remote conference. When the first terminal is a terminal provided in the mobile body 10, the control section 220 can select the first terminal as a terminal for causing the user of the second terminal to participate in the remote conference.

[0044] The terminal information acquisition section 230 acquires first information indicating the positions of the first terminal and the second terminal. The determination section 210 determines whether the first terminal and the second terminal exist within the predetermined range based on the first information.

[0045] The terminal information acquisition section 230 can also acquire second information indicating at least one of the moving directions and the accelerations of the first terminal and the second terminal. The determination section 210 can determine whether the first terminal and the second terminal exist within the predetermined range based on the first information and the second information.

[0046] The connection state information acquisition section 240 can acquire third information indicating the connection state of wireless communication between the first terminal and the second terminal. The determination section 210 can determine whether the first terminal and the second terminal exist within the predetermined range based on the third information.

[0047] When the determination section 210 determines that the first terminal and the second terminal exist within the predetermined range, the control section 220 can prompt at least one of the first user and the second user to cause the user of the other terminal to participate in the remote conference through one of the first terminal and the second terminal.

[0048] Figure 3The functional configuration of the communication terminal 20 provided in the mobile body 10 is shown together with the display device 22, the microphone 60, and the imaging device 28. The communication terminal 20 includes a participant information acquisition section 310, a control section 320, a speaker determination section 330, a seating information acquisition section 340, a number of people determination section 350, a number of users decision section 360, a reception section 370, a storage section 380, and a communication section 390.

[0049] The storage section 380 includes a nonvolatile storage medium. The nonvolatile storage medium can be a semiconductor memory such as a flash memory. The storage section 380 stores information necessary for the communication terminal 20 to perform processing. The storage section 380 can store a program executed by a processor provided in the communication terminal 20.

[0050] The communication section 390 is responsible for communication with the outside of the mobile object 10. Specifically, the communication section 390 is responsible for communication between the teleconference server 100 and the user terminal 120. The communication section 390 can also be responsible for communication with the user terminal 80.

[0051] The imaging device 28 is disposed at a position at which an occupant seated on the seat 11 can be imaged. The imaging device 28 can include one or more imaging devices.

[0052] The communication terminal 20 is mounted on the mobile body 10 and performs processing for conducting a teleconference. The number of people determination section 350 determines the number of people who are on board the mobile body 10. The number of users decision section 360 decides the number of users who can be accepted as participants in a teleconference by the communication terminal 20, based on the number of people determined by the number of people determination section 350. The number of users decision section 360 can also decide the number of users who can be accepted as participants in a teleconference by the communication terminal 20, based on the communication state of the communication terminal 20.

[0053] When the first user participates in a teleconference using the communication terminal 20, the participant information acquisition section 310 acquires information indicating that one or more users who participate in a teleconference using a mobile terminal are present in the mobile object 10. When the participant information acquisition section acquires information indicating that one or more users who participate in a teleconference using a mobile terminal are present in the mobile object 10, the control section 320 can perform control so that at least some of the one or more users participate in a teleconference by the communication terminal 20 with the number of users decided by the number of users decision section 360 as an upper limit.

[0054] The communication section 390 continuously transmits current position information of the communication terminal 20 to a teleconference management system that manages a teleconference. The communication section 390 also continuously transmits at least one of movement speed information and acceleration information of the communication terminal 20 to the teleconference management system that manages a teleconference.

[0055] The number-of-persons determining section 350 determines the number of persons who ride on the mobile body 10 based on information detected by the seating sensors provided on the seating positions of the mobile body 10. The number-of-persons determining section 350 can determine the number of persons who ride on the mobile body 10 by analyzing image information in the mobile body 10 acquired by the imaging device provided on the mobile body 10.

[0056] The reception section 370 receives a plurality of users in the mobile object 10 as participants of a remote conference. The speaker determining section 330 determines a speaker from among the plurality of users received by the reception section 370 as participants of a remote conference. The control section 320 performs control for highlighting the user determined as a speaker by the speaker determining section 330 on the remote conference system.

[0057] The speaker determining section 330 determines a speaker from among the plurality of users based on at least one of (i) image information in the mobile body 10 acquired by the imaging device 28 mounted on the mobile body 10 and (ii) sound information acquired by the microphone 60 provided on the mobile body 10.

[0058] The seating information acquiring section 340 acquires seating positions of the plurality of users in the mobile body 10. The speaker determining section 330 determines a speaker from among the plurality of users based on a sound source direction detected from sound information acquired by the plurality of microphones 60 mounted on the mobile body 10, seating positions of the plurality of microphones 60 in the mobile body 10, and the seating positions of the plurality of users acquired by the seating information acquiring section 340.

[0059] The speaker determining section 330 determines a speaker from among the plurality of users based on sound information acquired by each microphone 60 mounted in correspondence with each seat of the mobile body 10 and the seating positions of the plurality of users acquired by the seating information acquiring section 340.

[0060] When the reception section 370 receives a plurality of users as participants of a conference, the seating information acquiring section 340 acquires the seating positions of the plurality of users by acquiring information specifying the seating positions of the mobile body 10 from each of the plurality of users. When the reception section 370 receives a plurality of users as participants of a conference, the seating information acquiring section 340 can acquire the seating positions of the plurality of users based on positions of the user terminals determined from wireless signals received from the user terminals held by the plurality of users respectively. For example, the seating information acquiring section 340 can calculate the position of the user terminal 70 based on a signal of UWB communication received by the antenna 26 with the user terminal 70, and acquire the position of the user 90a based on the calculated position of the user terminal 70.

[0061] Figure 4An example of a processing sequence executed by the user terminal 80, the communication terminal 20, the teleconference server 100, and the user terminal 120 is shown. It is assumed that at the start of this sequence, there is no user 90 participating in a teleconference using the communication terminal 20.

[0062] In S402, the number-of-persons determination section 350 of the communication terminal 20 determines the number of persons who are on board the moving body 10. In the number-of-persons determination section 350, the communication terminal 20 determines the number of persons who are on board the moving body 10 based on the image of the moving body 10 taken by the camera 28 and the information of the seat sensor provided in each seat 11.

[0063] In S404, the number-of-users decision section 360 decides the number of users that the reception section 370 can receive in accordance with the number determined by the number-of-persons determination section 350. For example, the number-of-users decision section 360 decides the number determined by the number-of-persons determination section 350 as an upper limit value of the number of users that the reception section 370 can receive.

[0064] Here, the number-of-users decision section 360 can decide the number of users that the reception section 370 can receive in accordance with the communication state of the communication section 390 acquired by the connection state information acquisition section 240. The worse the communication state of the communication section 390 acquired by the connection state information acquisition section 240, the fewer the number of users that the number-of-users decision section 360 can decide that the reception section 370 can receive.

[0065] In S406, when the user 90a uses the communication terminal 20 to perform an operation for logging in to a teleconference, the reception section 370 receives the user 90a as a participant of the teleconference and performs a login process of transmitting the authentication information input at the time of login to the teleconference server 100 or the like.

[0066] In S408, when the user 90b uses the user terminal 80 to perform an operation for logging in to the teleconference in which the user 90a participates, the user terminal 80 performs a login process such as transmitting the authentication information input at the time of login to the teleconference server 100. Similarly, in S410, when the user 90c uses the user terminal 120 to perform an operation for logging in to the teleconference in which the user 90a participates, the user terminal 120 performs a login process such as transmitting the authentication information input at the time of login to the teleconference server 100.

[0067] In the login processing of S406, the seating information acquisition section 340 acquires the seating position of the user 90 in the mobile body 10. For example, as part of the login processing, the seating information acquisition section 340 causes the display device 22a to display a screen for specifying the seating position of the user 90a, and acquires information indicating the seating position input by the user 90a through the display device 22a. For example, the seating position can be information for identifying the seat 11 on which the user 90a is seated. The seating information acquisition section 340 holds information associating the user identification information identifying the user 90a and the seat identification information identifying the seat 11 on which the user 90a is seated as the seating information.

[0068] In another method, the seating information acquisition section 340 determines the seat 11 on which the user 90 is seated based on a signal received through wireless communication using UWB. For example, the seating information acquisition section 340 calculates the distance between the antenna 26a and the user terminal 70, the distance between the antenna 26b and the user terminal 70, and the distance between the antenna 26c and the user terminal 70 based on TOA (Time of Arrival), and calculates the position of the user terminal 70 by three-point measurement based on the calculated respective distances. The seating information acquisition section 340 can calculate the signal arrival angle from the user terminal 70 to each of the antennas 26 based on the time difference of receiving the signal transmitted from the user terminal 70 by each of the antennas 26, and calculate the position of the user terminal 70 based on the calculated signal arrival angle and the position of each of the antennas 26. The seating information acquisition section 340 determines the seat 11 on which the user 90a holding the user terminal 70 is seated based on the calculated position of the user terminal 70 and the position of the seat 11. The seating information acquisition section 340 can hold information associating the user identification information of the user 90a previously associated with the user terminal 70 and the seat identification information identifying the seat 11 on which the user 90a is seated as the seating information. The seating information acquisition section 340 can receive the user identification information identifying the user 90a from the user terminal 70, and hold information associating the received user identification information and the seat identification information identifying the seat 11 on which the user 90a is seated as the seating information. The login processing can be performed using the user terminal 70, and the seating information acquisition section 340 can receive the user identification information input to the user terminal 70 in part of the login processing from the user terminal 70.

[0069] In S420, when the communication terminal 20 completes the login to the remote conference, the communication terminal 20 starts the process of periodically transmitting the position information and the speed information to the remote conference server 100. For example, the communication section 390 transmits the current position information calculated based on the signal received from the GNSS satellite, the current position information calculated based on the time change from the current position information, or the moving speed information of the vehicle speed of the mobile body 10 to the remote conference server 100 at a predetermined time interval. Similarly, when the login to the remote conference is completed, the user terminal 80 and the user terminal 120 start the process of periodically transmitting the position information and the speed information to the remote conference server 100 (S422 and S424). The user terminal 80 and the user terminal 120 can transmit the current moving speed information calculated based on the time change from the current position information to the remote conference server 100. The terminal information acquisition section 230 of the remote conference server 100 receives the position information and the speed information from the user terminal 80, the communication terminal 20, and the user terminal 120. In S420, S422, and S424, the user terminal 80, the communication terminal 20, and the user terminal 120 can start the process of periodically transmitting the respective acceleration information to the remote conference server 100 in addition to or instead of the moving speed.

[0070] In S430, the determination section 210 of the remote conference server 100 determines the terminals used by the user 90 who rides the same mobile body 10 for participating in the same remote conference. For example, the terminal information acquisition section 230 determines the terminals used by the user 90 who rides the same mobile body 10 for participating in the same remote conference based on the position information and the speed information received from each of the user terminal 80, the communication terminal 20, and the user terminal 120 and the identification information of the terminal used by the user who has logged in. For example, when the position of the user terminal 80 and the position of the communication terminal 20 are within a predetermined range and the difference between the moving speed of the user terminal 80 and the moving speed of the communication terminal 20 is less than a predetermined value, the determination section 210 determines that the user terminal 80 and the communication terminal 20 are used by the user 90 who rides the same mobile body 10 for participating in the same remote conference.

[0071] Here, the connection state information acquisition section 240 can also acquire information indicating the connection state with another terminal by wireless communication from the user terminal 80, the communication terminal 20, and the user terminal 120, and determine the terminal used by the user 90 who boards the same moving body 10 to participate in the same remote conference based on the acquired information. For example, when the user terminal 80 and the communication terminal 20 are connected by WiFi (registered trademark), the communication section 390 of the communication terminal 20 and the user terminal 80 transmit the identification information of one or more terminals connected by WiFi (registered trademark) and information indicating the connection state to the remote conference server 100. The connection state of WiFi (registered trademark) can include information indicating the received signal strength from another terminal. The connection state information acquisition section 240 acquires the information transmitted from the communication terminal 20 and the user terminal 80, and the determination section 210 determines the terminal used by the user 90 who boards the same moving body 10 to participate in the same remote conference based on the information. The determination section 210 can determine the terminal connected by WiFi (registered trademark) and having a received signal strength higher than a predetermined value as the terminal used by the user 90 who boards the same moving body 10 to participate in the same remote conference.

[0072] In S432, the control section 220 of the remote conference server 100 performs processing for transmitting a switching prompt for switching the source terminal of participation in the remote conference to the communication terminal 20 and the user terminal 80. For example, the control section 220 transmits information indicating that the user 90a and the user 90b are participating in the same remote conference and recommending the user 90b to participate in the remote conference by the communication terminal 20 to the communication terminal 20 and the user terminal 80. Thereby, the participant information acquisition section 310 of the communication terminal 20 acquires information indicating that one or more users 90 who participate in the remote conference using the user terminal 80 exist in the moving body 10.

[0073] Here, the control section 220 can acquire the communication state of each of the user terminal 80 and the user terminal 120, and use the terminal having a good communication state among the user terminal 80 and the user terminal 120 as the source terminal of participation in the remote conference. Thereby, the plurality of users 90 can participate in the remote conference through the communication line of the terminal having a more stable communication state.

[0074] In S434, the control section 320 of the communication terminal 20 performs processing for switching the participation source terminal for the remote meeting, and in S436, the communication terminal 20 transmits a switching completion notification of the participation source terminal to the remote meeting server 100. For example, in S434, the control section 320 causes the display device 22 to display a screen for receiving an operation of confirming participation in the remote meeting by the communication terminal 20 from the user 90b. When the communication terminal 20 detects that the user 90b has performed a confirmation operation on the display device 22b, in S436, the communication terminal 20 transmits a switching completion notification indicating that the user 90b participates in the remote meeting from the communication terminal 20 to the remote meeting server 100.

[0075] Figure 4 The case where the control section 320 performs processing for switching the participation source terminal for the remote meeting in S434 is exemplified. However, the control section 320 can perform processing for switching the participation source terminal for the remote meeting only when the number of users does not exceed the number of users decided by the number of users deciding section 360.

[0076] Figure 5 An example of a screen 500 displayed on the display device 22 or the user terminal 120 is schematically shown. The screen 500 shows a case where the user 90a participates in the remote meeting from the communication terminal 20, the user 90b participates in the remote meeting from the user terminal 80, and the user 90c participates in the remote meeting from the user terminal 120.

[0077] The screen 500 includes an object 510, an object 520, and an object 530. The object 510 includes, for example, a face image of the user 90a captured by the imaging device 28. The object 520 includes a face image of the user 90b captured by the user terminal 80. The object 530 includes a face image of the user 90c captured by the user terminal 120.

[0078] Figure 6 An example of a screen 600 displayed on the display device 22 or the user terminal 120 is schematically shown. The screen 600 shows a case where the user 90a and the user 90b participate in the remote meeting from the communication terminal 20 and the user 90c participates in the remote meeting from the user terminal 120.

[0079] The screen 600 includes an object 610, an object 620, and an object 630. The object 610 includes, for example, face images of the user 90a and the user 90b captured by the imaging device 28. The object 620 is the same as the object 610, and includes face images of the user 90a and the user 90b captured by the imaging device 28. The object 630 includes a face image of the user 90c captured by the user terminal 120 and transmitted from the user terminal 120.

[0080] As described above, according to the communication terminal 20, even when a plurality of users 90 participate in the remote conference through a single communication terminal 20, the plurality of users 90 can be regarded as separate participants. Thereby, other participants are also able to know that the plurality of users 90 are participating in the remote conference from the communication terminal 20.

[0081] In Figure 6 In the example, the control section 320 uses the image captured by the imaging device 28 as the participant image of the user 90a and the participant image of the user 90b. In another embodiment, the control section 320 can extract the facial image of the user 90a and the facial image of the user 90b from the image captured by the imaging device 28, respectively, process the extracted facial image of the user 90a as the participant image of the user 90a, and process the extracted facial image of the user 90b as the participant image of the user 90b.

[0082] Figure 7 is a flowchart showing a process procedure relating to the process of determining a speaker by the communication terminal 20. For example, when the communication terminal 20 performs a process relating to the remote conference in cooperation with the remote conference server 100, the process of the present flowchart is repeated.

[0083] In S702, the speaker determination section 330 detects a sound from the information collected by the microphones 60. In S704, the speaker determination section 330 determines a speaker based on the sound information detected in S702 and the seating information.

[0084] For example, when the user 90b is a speaker, the volume of the sound detected by the microphone 60b is greater than the volume of the sound detected by the other microphones 60a, 60c, and 60d. Therefore, the speaker determination section 330 judges that the speaker is seated on the seat 11b provided at a position corresponding to the microphone 60b. The speaker determination section 330 refers to the seating information, and determines the user 90b identified by the user identification information associated with the seat identification information of the seat 11b as the speaker.

[0085] In S706, the communication section 390 transmits the speaker information to the user terminal 120. For example, the communication section 390 transmits the identification information of the speaker in the remote conference system to the user terminal 120 as the speaker information.

[0086] In S708, the control section 320 highlights the object of the speaker. For example, when the user 90b is a speaker, the control section 320 highlights the object 620. At this time, the control section 320 does not highlight the object 610.

[0087] Figure 8An example of a screen 800 displayed on the display device 22 or the user terminal 120 is schematically shown. The screen 800 shows a case where the user 90a and the user 90b participate in the remote conference from the communication terminal 20, the user 90c participates in the remote conference from the user terminal 120, and the user 90b is speaking.

[0088] The screen 800 contains an object 810, an object 820, and an object 830. The objects 810 and 830 are the same as the objects 610 and 630, respectively. The object 820 contains, for example, face images of the users 90a and 90b captured by the imaging device 28. Since the user 90b is determined to be the speaker, the object 820 is highlighted to indicate that the user 90b is the speaker.

[0089] As described above, according to the communication terminal 20, it is possible to notify other conference participants who is the speaker even when a plurality of users participate in the remote conference through a single communication terminal 20.

[0090] Figure 9 A configuration of a mobile body 910, which is a modification of the mobile body 10, is shown. The mobile body 910 differs from the mobile body 10 in that the mobile body 910 contains a microphone array 960 instead of the microphone 60. Therefore, differences from the mobile body 10 will mainly be described here.

[0091] The microphone array 960 contains a plurality of microphones. The speaker determination section 330 of the communication terminal 20 uses the microphone array 960 to locate a sound source.

[0092] For example, when the distance from the sound source to each microphone is different, the time at which a sound wave reaches each microphone is different, and the difference in arrival time at each microphone is a function of the sound source position. The speaker determination section 330 utilizes this principle to determine the direction of the sound source based on sound signals acquired by the plurality of microphones contained in the microphone array 960. The speaker determination section 330 can determine the speaker from among the users 90 based on the estimated sound source direction, the set position of the microphone array in the mobile body 10, and the seating information acquired by the seating information acquisition section 340.

[0093] In the above-described embodiment, the speaker determining section 330 determines the speaker using the sound acquired by the plurality of microphones. The speaker determining section 330 can determine the speaker by recognizing the voice of the user 90 instead of using the sound acquired by the plurality of microphones or in addition to using the sound acquired by the plurality of microphones. For example, the speaker determining section 330 can acquire the voiceprint of the speaker in advance in a case where the login processing or the like is performed, and hold the acquired voiceprint in association with the identification information of the user 90 as voiceprint information. The speaker determining section 330 can also extract the voiceprint from the sound acquired by the microphone provided in the mobile body 10 when the teleconference is performed, and determine the speaker from the plurality of users 90 on the basis of the extracted voiceprint and the voiceprint information.

[0094] In another embodiment, the speaker determining section 330 can also determine the speaker on the basis of the face image of each of the plurality of users 90 photographed by the photographing device 28. For example, the speaker determining section 330 can also determine the speaker among the plurality of users 90 on the basis of the movement of the mouth in the face image. In addition to the movement of the mouth in the face image, the speaker determining section 330 can also determine the speaker from the plurality of users 90 on the basis of the sound collected by the microphone.

[0095] In the above-described embodiment, the mobile body 10 is a vehicle. However, any mobile body other than a vehicle can be applied as the mobile body 10. For example, the mobile body 10 can be a transportation device of a train, an airplane, or a ship.

[0096] The teleconference system 5 according to the above-described embodiment, when the plurality of users 90 participate in the same teleconference from different terminals in the mobile body 10, can prompt the participation in the conference through one terminal. Thereby, the plurality of users can be prompted to participate in the teleconference through one communication line. Further, since the communication terminal 20 decides the upper limit of the users participating in the teleconference through the communication terminal 20 on the basis of the number of people on board the mobile body 10 and the communication state of the communication terminal 20, it is possible to prevent the communication volume of the communication terminal 20 from exceeding the set communication volume determined in advance for the teleconference. Further, when the plurality of users participate in the teleconference through the mobile body 10, the current speaker is determined on the basis of the sound acquired by the plurality of microphones provided in the mobile body 10 and the seating information, and the situation where the determined speaker is speaking is displayed on the display device 22 and the user terminal 120. Thereby, even in a case where the plurality of users 90 are accepted as participants of the teleconference through a single communication terminal 20, it is possible to notify each participant of the teleconference who is speaking.

[0097] Figure 10An example of a computer 2000 that can embody all or a part of the embodiments of the present application is shown. A program installed to the computer 2000 can cause the computer 2000 to function as the various devices and the like or the units thereof, the remote conference management system 110 or the units thereof, or the communication terminal 20 and the like involved in the embodiments, perform operations associated with the system or the units thereof, the devices or the units thereof, and the like, and / or perform the procedures or the steps thereof involved in the embodiments. To cause the computer 2000 to perform the processing flows described in the present specification and the specific operations associated with some or all of the functional blocks of the block diagrams, such a program can be executed by the CPU 2012.

[0098] The computer 2000 according to the present embodiment includes the CPU 2012 and the RAM 2014, which are connected to each other through the host controller 2010. The computer 2000 also includes the ROM 2026, the flash memory 2024, the communication interface 2022, and the input / output chip 2040. The ROM 2026, the flash memory 2024, the communication interface 2022, and the input / output chip 2040 are connected to the host controller 2010 via the input / output controller 2020.

[0099] The CPU 2012 acts in accordance with the programs stored in the ROM 2026 and the RAM 2014, thereby controlling the units.

[0100] The communication interface 2022 communicates with other electronic devices via a network. The flash memory 2024 stores programs and data used by the CPU 2012 in the computer 2000. The ROM 2026 stores a boot program and the like executed by the computer 2000 when activated, and / or a program dependent on the hardware of the computer 2000. The input / output chip 2040 can also connect various input / output units such as a keyboard, a mouse, and a monitor to the input / output controller 2020 via input / output ports such as a serial port, a parallel port, a keyboard port, a mouse port, a monitor port, a USB port, an HDMI (registered trademark) port, and the like.

[0101] The programs are provided via a computer-readable storage medium such as a CD-ROM, a DVD-ROM, or a USB memory, or a network. The RAM 2014, the ROM 2026, or the flash memory 2024 is an example of the computer-readable storage medium. The programs are installed to the flash memory 2024, the RAM 2014, or the ROM 2026, and executed by the CPU 2012. The information processing described in these programs is read by the computer 2000, and the cooperation between the programs and the various types of hardware resources described above is achieved. The devices or the methods can be constituted by implementing the operations or the processing of information in compliance with the use of the computer 2000.

[0102] For example, in a case where communication is performed between the computer 2000 and an external device, the CPU 2012 can execute a communication program loaded to the RAM 2014, instruct a communication process to the communication interface 2022 based on a process described in the communication program. The communication interface 2022 reads transmission data saved in a transmission buffer processing area provided in a recording medium such as the RAM 2014 and the flash memory 2024 under the control of the CPU 2012, transmits the read transmission data to a network, and writes reception data received from the network to a reception buffer processing area provided in the recording medium or the like.

[0103] In addition, the CPU 2012 can cause all or a required part of a file or a database saved in a recording medium such as the flash memory 2024 to be read to the RAM 2014, and perform various processes on the data on the RAM 2014. The CPU 2012 then writes back the processed data to the recording medium.

[0104] Various types of programs, data, tables, and various information such as databases can be saved to the recording medium, and applied to information processing. The CPU 2012 can perform various processes described in this specification on data read from the RAM 2014, including various operations specified by an instruction sequence of a program, information processing, condition judgment, condition branching, unconditional branching, search / replacement of information, and the like, and write back the result to the RAM 2014. In addition, the CPU 2012 can search for information in a file, a database, or the like in the recording medium. For example, in a case where a plurality of items each having an attribute value of a first attribute associated with an attribute value of a second attribute are saved in the recording medium, the CPU 2012 can search for an item in which the attribute value of the first attribute coincides with a condition from the plurality of items, read the attribute value of the second attribute saved in the item, and thereby acquire the attribute value of the second attribute associated with the first attribute that satisfies a condition set in advance.

[0105] The program or the software module described above can be saved to a computer-readable storage medium on or near the computer 2000. A recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can be used as the computer-readable storage medium. The program saved in the computer-readable storage medium can be provided to the computer 2000 via a network.

[0106] Programs installed in computer 2000 that enable computer 2000 to function as remote conference management system 110, when executed by the computer, cause operations in CPU 2012 and other systems to operate, causing computer 2000 to function as the various units of remote conference management system 110. The information processing described in these programs is read into computer 2000, allowing computer 2000 to function as specific units that collaborate with the software and the various hardware resources described above, namely, the various units of remote conference management system 110. Furthermore, by utilizing these specific units to perform calculations or processing of information corresponding to the intended use of computer 2000 in this embodiment, a unique remote conference management system 110 corresponding to the intended use is constructed.

[0107] Programs installed in computer 2000 that enable computer 2000 to function as communication terminal 20, when executed by the computer, cause operations in CPU 2012 and other systems to operate, causing computer 2000 to function as the various units of communication terminal 20. Information processing described in these programs is read into computer 2000, allowing computer 2000 to function as specific units that collaborate with the various hardware resources described above, namely, the various units of communication terminal 20. Furthermore, by utilizing these specific units to perform calculations or processing of information corresponding to the intended use of computer 2000 in this embodiment, a unique communication terminal 20 corresponding to the intended use is constructed.

[0108] Various embodiments are described with reference to block diagrams, etc. In the block diagrams, each functional block may represent (1) a step of a process for performing an operation or (2) each unit of a device having the function of performing an operation. Specific steps and each unit may be implemented by a dedicated circuit, a programmable circuit supplied together with computer-readable instructions stored on a computer-readable medium, and / or a processor supplied together with computer-readable instructions stored on a computer-readable medium. The dedicated circuit may include digital and / or analog hardware circuits, and may also include integrated circuits (ICs) and / or discrete circuits. The programmable circuit may include logical AND, logical OR, logical XOR, logical NAND, logical NOR, and other logical operations, flip-flops, registers, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and other reconfigurable hardware circuits including memory elements.

[0109] The computer readable storage medium can include any tangible device that can retain, store, or record instructions for execution by a suitable device, and the computer readable storage medium so charged with the instructions becomes a product that includes at least a portion of a program that implements a unit for executing the operations specified in the processing flow or block diagram. As examples of the computer readable storage medium, there can be included an electric storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, and the like. As more specific examples of the computer readable storage medium, there can be included a floppy disk (registered trademark), a flexible disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an electrically erasable programmable read-only memory (EEPROM), a static random access memory (SRAM), a compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a Blu-ray (RTM) disk, a memory stick, an integrated circuit card, and the like.

[0110] The computer readable instructions can include any one of assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine delegate instructions, microcode, firmware instructions, state setting data, or source code or object code described by any combination of one or more programming languages including an object-oriented programming language such as Smalltalk (registered trademark), JAVA (registered trademark), C++, and the like, and a conventional procedural programming language such as "C" programming language or the like.

[0111] The computer readable instructions can be provided to a processor or programmable circuit of a general purpose computer, a special purpose computer, or other programmable data processing apparatus via a local or local area network (LAN), the Internet, or the like wide area network (WAN), and the computer readable instructions can be executed by the processor or programmable circuit in order to implement a unit for executing the operations specified in the processing flow or block diagram described. As examples of the processor, there can be included a computer processor, a processing unit, a microprocessor, a digital signal processor, a controller, a microcontroller, and the like.

[0112] The above has described the present application by way of embodiments, but the technical scope of the present application is not limited to the range described in the above embodiments. It will be apparent to those skilled in the art that various changes or modifications can be made to the above embodiments. Such changed or modified modes can also be included in the technical scope of the present application as is apparent from the description of the claims.

[0113] Note that the order of execution of each process of the actions, flows, steps, and the like of the apparatuses, systems, programs, and methods shown in the claims, the specification, and the drawings is not particularly indicated as "before," "preceding," or the like, and can be implemented in any order as long as the output of the preceding process is not used in the subsequent process. For the action flows in the claims, the specification, and the drawings, even if described using "first," "next," or the like for convenience, it does not mean that it must be implemented in such an order.

[0114] [Legend of Reference Numerals]

[0115] 5 teleconference system

[0116] 10 moving body

[0117] 11 agent

[0118] 20 communication terminal

[0119] 22 display apparatus

[0120] 26 antenna

[0121] 28 imaging apparatus

[0122] 60 microphone

[0123] 70 user terminal

[0124] 80 user terminal

[0125] 90 user

[0126] 100 teleconference server

[0127] 110 teleconference management system

[0128] 120 user terminal

[0129] 380 storage section

[0130] 390 communication section

[0131] 210 determination section

[0132] 220 control section

[0133] 230 terminal information acquisition section

[0134] 240 connection state information acquisition section

[0135] 310 participant information acquisition section

[0136] 320 control section

[0137] 330 speaker determination section

[0138] 340 Seating information acquisition section

[0139] 350 Number of people determination section

[0140] 360 User number determination section

[0141] 370 Receiving section

[0142] 500 Screen

[0143] 510 Object

[0144] 520 Object

[0145] 530 Object

[0146] 600 Screen

[0147] 610 Object

[0148] 620 Object

[0149] 630 Object

[0150] 800 Screen

[0151] 810 Object

[0152] 820 Object

[0153] 830 Object

[0154] 910 Mobile body

[0155] 960 Microphone array

[0156] 2000 Computer

[0157] 2010 Host controller

[0158] 2012 CPU

[0159] 2014 RAM

[0160] 2020 Input / output controller

[0161] 2022 Communication interface

[0162] 2024 Flash memory

[0163] 2026 ROM

[0164] 2040 Input / output chip

Claims

1. A communication terminal mounted on a mobile body and used for conducting a teleconference, wherein, Possessing: an accepting section that accepts a plurality of users in the moving body as participants of the remote conference; a speaker determining section that determines a speaker from among the plurality of users accepted by the accepting section as participants of the remote conference; and a control section that performs control for highlighting on a remote conference system a user determined by the speaker determining section as the speaker.

2. The communication terminal according to claim 1, wherein the speaker determining section determines the speaker from among the plurality of users based on at least one of (i) image information of the moving body acquired by a camera mounted on the moving body and (ii) sound information acquired by a microphone provided on the moving body.

3. The communication terminal according to claim 1, wherein further possessing a seating information acquiring section that acquires seating positions of the plurality of users in the moving body, the speaker determining section determines the speaker from among the plurality of users based on a sound source direction detected from sound information acquired by a plurality of microphones mounted on the moving body, seating positions of the plurality of microphones in the moving body, and the seating positions of the plurality of users acquired by the seating information acquiring section.

4. The communication terminal according to claim 1, wherein further possessing a seating information acquiring section that acquires seating positions of the plurality of users in the moving body, the speaker determining section determines the speaker from among the plurality of users based on sound information acquired by each microphone mounted in correspondence with a plurality of seats of the moving body respectively and the seating positions of the plurality of users acquired by the seating information acquiring section.

5. The communication terminal according to claim 3 or 4, wherein when the accepting section accepts the plurality of users as participants of the conference, the seating information acquiring section acquires the seating positions of the plurality of users by acquiring information specifying seat positions of the moving body from each of the plurality of users.

6. The communication terminal according to claim 3 or 4, wherein when the accepting section accepts the plurality of users as participants of the conference, the seating information acquiring section acquires the seating positions of the plurality of users based on positions of user terminals held by the plurality of users respectively, determined from wireless signals received from the user terminals.

7. The communication terminal according to any one of claims 1 to 4, wherein the moving body is a vehicle. having:

8. A method executed by a communication terminal mounted on a mobile body and used for conducting a remote conference, wherein a step of accepting a plurality of users in the moving body as participants of the remote conference; a step of determining a speaker from among the plurality of users accepted as participants of the remote conference; and a step of performing control for highlighting on a remote conference system a user determined from among the plurality of users as the speaker.

9. A program, wherein the program is for causing a computer to function as the communication terminal of any one of claims 1 to 4. ​ ​

Images