Communication terminal, method, and program

Abstract

The system is a communication terminal mounted on a mobile body and used for performing a teleconference, and is provided with: a number-of-people specifying unit for specifying the number of people riding on the mobile body; and a user number determination unit that determines the number of users that can be accepted as participants of the teleconference by the communication terminal, on the basis of the number of persons identified by the number identification unit.

Description

Technical Field

[0001] The present invention relates to a communication terminal, method and program. Background Art

[0002] Patent document 1 describes a technology in which, when a portable terminal is brought into a vehicle equipped with a vehicle device and the vehicle starts traveling, the portable terminal starts communicating with the vehicle device that detects an event such as a collision and performs notification processing related to the event via a communication network.

[0003] Patent Document 1: Japanese Patent Application Laid-Open No. 2019-33427 Summary of the Invention

[0004] However, it is difficult to properly manage participants in remote conferences that can be attended using communication devices on mobile devices. To address this issue, the present application aims to improve the operability of managing participants in remote conferences. Furthermore, this application will further improve traffic safety and contribute to the development of sustainable transportation systems.

[0005] In a first aspect of the present invention, a communication terminal is provided. The communication terminal is mounted on a mobile object and is used for conducting a remote conference. The communication terminal includes a number-of-people determination unit that determines the number of people on board the mobile object. The communication terminal also includes a user number determination unit that determines the number of users that the communication terminal can accept as participants in the remote conference based on the number of people determined by the number-of-people determination unit.

[0006] In the above communication terminal, the user number determination unit may further determine the number of users that the communication terminal can accept as participants in the remote conference based on a communication state of the communication terminal.

[0007] Any of the above-mentioned communication terminals may include a participation information acquisition unit that, when a first user participates in the remote conference using the communication terminal, acquires information indicating that one or more users participating in the remote conference using portable terminals are present in the mobile object. The communication terminal may include a control unit that, when the participation information acquisition unit acquires information indicating that one or more users participating in the remote conference using portable terminals are present in the mobile object, controls the participation of at least some of the one or more users in the remote conference via the communication terminal, with the number of users determined by the user number determination unit serving as an upper limit.

[0008] Any of the above-mentioned communication terminals may include a communication unit configured to continuously transmit current location information of the communication terminal to a teleconference management system that manages the teleconference.

[0009] In any of the above communication terminals, the communication unit may continuously send at least one of the moving speed information and the acceleration information of the communication terminal to the remote conference management system.

[0010] In any of the above-mentioned communication terminals, the number of people specifying unit may specify the number of people riding on the moving body based on information detected by a seating sensor provided on a seating seat of the moving body.

[0011] In any of the above-mentioned communication terminals, the number of people determining unit may determine the number of people riding on the moving body by analyzing image information of the moving body acquired by a camera provided in the moving body.

[0012] In any of the above communication terminals, the mobile object may be a vehicle.

[0013] In a second aspect of the present invention, a method is provided. The method is performed by a communication terminal mounted on a mobile object and used for conducting a remote conference. The method includes the step of determining the number of users aboard the mobile object. The method also includes the step of determining the number of users that the communication terminal can accept as participants in the remote conference based on the number of users aboard the mobile object.

[0014] A third aspect of the present invention provides a program that causes a computer to function as any of the above-mentioned communication terminals.

[0015] In addition, the above summary of the present invention does not list all the features of the present invention. In addition, sub-combinations of these feature groups may also be part of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The configuration of the teleconference system 5 in one embodiment is conceptually shown.

[0017] Figure 2 An example of the functional configuration of the teleconference management system 110 included in the teleconference server 100 is shown.

[0018] Figure 3 The functional configuration of the communication terminal 20 included in the mobile object 10 is shown together with the display device 22 , the microphone 60 , and the imaging device 28 .

[0019] Figure 4 An example of a sequence executed by the user terminal 80 , the communication terminal 20 , the teleconference server 100 , and the user terminal 120 is shown.

[0020] Figure 5 An example of a screen 500 displayed on the display device 22 or the user terminal 120 is schematically shown.

[0021] Figure 6 An example of a screen 600 displayed on the display device 22 or the user terminal 120 is schematically shown.

[0022] Figure 7 1 is a flowchart showing a processing procedure related to a process of determining a speaker by the communication terminal 20 .

[0023] Figure 8 An example of a screen 800 displayed on the display device 22 or the user terminal 120 is schematically shown.

[0024] Figure 9 The structure of a moving object 910 as a modified example of the moving object 10 is shown.

[0025] Figure 10 An example of a computer 2000 is shown. DETAILED DESCRIPTION

[0026] Hereinafter, the present invention will be described by way of embodiments of the invention, but the following embodiments do not limit the claimed invention. In addition, all combinations of features described in the embodiments are not necessarily essential for the solution of the present invention.

[0027] Figure 1 The configuration of a teleconference system 5 in one embodiment is conceptually shown. The teleconference system 5 includes a mobile object 10, a teleconference server 100, and a user terminal 120. In this embodiment, the mobile object 10 is a vehicle.

[0028] The mobile body 10 includes a communication terminal 20, a display device 22a, a display device 22b, a display device 22c and a display device 22d, seats 11a, seat 11b, seat 11c and seat 11d, a camera 28, microphones 60a, microphone 60b, microphone 60c and microphone 60d, and antennas 26a, antenna 26b and antenna 26c.

[0029] Display devices 22a, 22b, 22c, and 22d are located at positions corresponding to seats 11a, 11b, 11c, and 11d, respectively. In this embodiment, display devices 22a, 22b, 22c, and 22d are sometimes collectively referred to as "display devices 22." Display devices 22 display images used for teleconferencing. Display devices 22 also have the function of receiving instructions from user 90 through touch operations, etc.

[0030] Microphones 60a, 60b, 60c, and 60d are located at positions corresponding to seats 11a, 11b, 11c, and 11d, respectively. For example, microphones 60a and 60b corresponding to front-row seats 11a and 11b are located near display devices 22a and 22b, respectively. Microphones 60c and 60d corresponding to rear-row seats 11c and 11d are located near the headrests of seats 11c and 11d, respectively. Microphones 60a, 60b, 60c, and 60d are sometimes collectively referred to as "microphones 60."

[0031] The teleconference server 100 can communicate with the communication terminal 20 and the user terminal 120. The teleconference server 100 manages participants in the mobile teleconference. The communication terminal 20 and the user terminal 120 can communicate with each other.

[0032] Communication terminal 20 cooperates with teleconference server 100 to provide a function for passengers in mobile vehicle 10 to participate in a teleconference. Display device 22 and microphone 60 are connected to communication terminal 20. Communication terminal 20 causes display device 22 to display conference images received through the management of teleconference server 100. Communication terminal 20 is connected to microphone 60. Communication terminal 20 transmits the sound acquired from microphone 60 as sound data to user terminal 120, a terminal used by conference participants.

[0033] User 90a and user 90b are passengers of mobile object 10. User 90a is a user of user terminal 70. User 90b is a user of user terminal 80. User terminals 70 and 80 are, for example, smartphones. User terminals 70 and 80 may be portable devices other than smartphones. User terminals 70 and 80 may be computing devices with communication capabilities. User terminals 70 and 80 may be terminals such as mobile phones, PDAs, tablet computers, notebook computers, laptop computers, and wearable computers.

[0034] Antenna 26a, antenna 26b, and antenna 26c are fixedly installed at different positions in the mobile body 10. In this embodiment, antenna 26a, antenna 26b, and antenna 26c are sometimes collectively referred to as "antenna 26." The communication terminal 20 wirelessly communicates with the user terminal 70 or user terminal 80 held by the user 90 in the mobile body 10 via antenna 26. For example, the communication terminal 20 wirelessly communicates with the user terminal 70 held by the user 90a via antenna 26 using UWB (Ultra-Wide Band). For example, the communication terminal 20 calculates the position of the user terminal 70 based on the signal received in the wireless communication using UWB with the user terminal 70, and determines which seat of the user terminal 70, seat 11a, seat 11b, seat 11c, and seat 11d, is the user 90a seated in based on the detected position of the user terminal 70.

[0035] User 90c is located outside mobile object 10. User 90c is a user of user terminal 120. For example, user terminal 120 may be a smartphone. User terminal 120 may be a computing device with communication capabilities. User terminal 120 may be a portable terminal such as a mobile phone, PDA, tablet computer, notebook computer, laptop computer, wearable computer, or a desktop computer.

[0036] Here, it is assumed that user 90a, user 90b, and user 90c participate in the same teleconference managed by teleconference server 100. User 90a participates in the teleconference using communication terminal 20, user 90b participates in the teleconference using user terminal 80, and user 90c participates in the teleconference using user terminal 120.

[0037] While user 90 is participating in a teleconference, communication terminal 20, user terminal 80, and user terminal 120 transmit current location information and movement speed information to teleconference server 100 at predetermined time intervals. For example, communication terminal 20, user terminal 80, and user terminal 120 transmit current location information calculated based on signals received from GNSS satellites to teleconference server 100 at predetermined time intervals. User terminal 80 and user terminal 120 also transmit current movement speed information calculated based on temporal changes in current location information to teleconference server 100 at predetermined time intervals. Communication terminal 20 transmits current location information calculated based on temporal changes in current location information or movement speed information based on vehicle speed information of mobile object 10 to teleconference server 100 at predetermined time intervals.

[0038] Based on the current location information and movement speed information received from communication terminal 20, user terminal 80, and user terminal 120, the teleconference server 100 determines that communication terminal 20 and user terminal 80 are within a predetermined range and are participating in the same teleconference while moving. In this case, the teleconference server 100 prompts user 90a and / or user 90b via communication terminal 20 and / or user terminal 80, allowing both users 90 on the mobile object 10 to participate in the conference via either communication terminal 20 or user terminal 80. For example, the teleconference server 100 prompts user 90a and / or user 90b via communication terminal 20 and / or user terminal 80, allowing user 90b of user terminal 80 to participate in the conference via communication terminal 20. This allows communication terminal 20 to prompt both users 90a and 90b to participate in the teleconference via a single communication line.

[0039] Communication terminal 20 detects the number of people aboard moving object 10 based on images of moving object 10 captured by camera 28 or information from seat sensors installed in each seat 11. When user 90b attempts to join a conference via communication terminal 20, communication terminal 20 accepts user 90b as a participant in the remote conference via communication terminal 20, provided that the number of participants participating in the remote conference via communication terminal 20 does not exceed the number of people detected in moving object 10. This prevents a number of users exceeding the number of people aboard moving object 10 from joining the remote conference. Furthermore, the presence of multiple participants in the remote conference via communication terminal 20 prevents the communication volume of communication terminal 20 from exceeding a preset volume for the remote conference.

[0040] When user 90 participates in a remote conference via communication terminal 20, communication terminal 20 obtains information identifying user 90 seated in a corresponding seat 11 from user 90 via display device 22 and stores this information as seating information. In another method, communication terminal 20 uses UWB wireless communication to determine the seat 11 in which user 90 is seated based on a signal received from the user terminal. It also receives information identifying user 90 seated in seat 11 from the user terminal and stores this information as seating information. Communication terminal 20 obtains the voice of user 90 via microphone 60. Based on the voice and seating information obtained via microphone 60, communication terminal 20 identifies user 90 who is currently speaking from among the users 90 accepted as participants in the remote conference. Communication terminal 20 displays the identified speaker on display device 22 and user terminal 120. This allows each participant in the remote conference to be informed of who is currently speaking, even when multiple users 90 are accepted as participants in the remote conference via a single communication terminal 20.

[0041] Figure 2The teleconference server 100 is responsible for processing user 90 logging into the teleconference and for processing user terminals 80, communication terminals 20, and user terminals 120 used by users to participate in the teleconference to communicate with each other.

[0042] The remote conference management system 110 can be implemented by one or more computers and includes a determination unit 210 , a control unit 220 , a terminal information acquisition unit 230 , and a connection status information acquisition unit 240 .

[0043] Determination unit 210 determines whether a first terminal used by a first user to participate in a remote conference and a second terminal used by a second user to participate in a remote conference are within a predetermined range. For example, the first terminal is communication terminal 20, and the second terminal is user terminal 80. If determination unit 210 determines that the first terminal and the second terminal are within the predetermined range, control unit 220 controls the user of one of the first and second terminals to participate in the remote conference via the other terminal.

[0044] The control unit 220 may preferentially select the terminal with the best communication status among the first terminal and the second terminal as the terminal for enabling the user of the other terminal to participate in the remote conference. When the first terminal is a terminal set in the mobile body 10, the control unit 220 may select the first terminal as the terminal for enabling the user of the second terminal to participate in the remote conference.

[0045] The terminal information acquisition unit 230 acquires first information indicating the locations of the first terminal and the second terminal. The determination unit 210 determines whether the first terminal and the second terminal are within a predetermined range based on the first information.

[0046] The terminal information acquisition unit 230 may also acquire second information indicating at least one of the moving direction and acceleration of the first and second terminals. The determination unit 210 may determine whether the first and second terminals are within a predetermined range based on the first and second information.

[0047] The connection status information acquisition unit 240 may acquire third information indicating the connection status of wireless communication between the first terminal and the second terminal. The determination unit 210 may determine whether the first terminal and the second terminal are within a predetermined range based on the third information.

[0048] When the determination unit 210 determines that the first terminal and the second terminal exist within the predetermined range, the control unit 220 may prompt at least one of the first user and the second user to allow the user of the other terminal to join the remote conference through one of the first terminal and the second terminal.

[0049] Figure 3The functional configuration of the communication terminal 20 included in the mobile object 10 is shown together with the display device 22, the microphone 60, and the camera 28. The communication terminal 20 includes a participant information acquisition unit 310, a control unit 320, a speaker identification unit 330, a seating information acquisition unit 340, a number of people identification unit 350, a user number determination unit 360, a reception unit 370, a storage unit 380, and a communication unit 390.

[0050] Storage unit 380 includes a nonvolatile storage medium. The nonvolatile storage medium may be a semiconductor memory such as flash memory. Storage unit 380 stores information required for communication terminal 20 to execute processing. Storage unit 380 may also store programs executed by a processor included in communication terminal 20.

[0051] The communication unit 390 is responsible for external communication with the mobile object 10. Specifically, the communication unit 390 is responsible for communication between the teleconference server 100 and the user terminal 120. The communication unit 390 may also be responsible for communication with the user terminal 80.

[0052] The imaging device 28 is provided at a position where it can capture an image of an occupant seated on the seat 11. The imaging device 28 may include one or more imaging devices.

[0053] Communication terminal 20 is mounted on mobile object 10 and performs processing for conducting a remote conference. A number-of-people determination unit 350 determines the number of people aboard mobile object 10. User number determination unit 360 determines the number of users that communication terminal 20 can accept as participants in the remote conference based on the number of people determined by number-of-people determination unit 350. User number determination unit 360 can also determine the number of users that communication terminal 20 can accept as participants in the remote conference based on the communication status of communication terminal 20.

[0054] When a first user participates in a remote conference using communication terminal 20, participant information acquisition unit 310 obtains information indicating that one or more users participating in the remote conference using mobile terminals are present in mobile object 10. When participation information acquisition unit obtains information indicating that one or more users participating in the remote conference using mobile terminals are present in mobile object 10, control unit 320 may control the mobile object 10 so that at least some of the one or more users participate in the remote conference using communication terminal 20, with the number of users determined by user number determination unit 360 serving as an upper limit.

[0055] Communication unit 390 continuously transmits the current location information of communication terminal 20 to the remote conference management system managing the remote conference. Communication unit 390 also continuously transmits at least one of the moving speed information and acceleration information of communication terminal 20 to the remote conference management system managing the remote conference.

[0056] The number of people determining unit 350 determines the number of people on the moving object 10 based on information detected by a seating sensor provided on a seat of the moving object 10. The number of people determining unit 350 can determine the number of people on the moving object 10 by analyzing image information of the moving object 10 captured by a camera provided on the moving object 10.

[0057] Accepting unit 370 accepts multiple users in mobile object 10 as participants in the teleconference. Speaker identification unit 330 identifies a speaker from among the multiple users accepted as participants in the teleconference by accepting unit 370. Control unit 320 controls the user identified as the speaker by speaker identification unit 330 to be highlighted on the teleconference system.

[0058] The speaker identification unit 330 identifies a speaker from among a plurality of users based on at least one of (i) image information of the mobile object 10 acquired by the camera 28 mounted on the mobile object 10 and (ii) audio information acquired by the microphone 60 provided on the mobile object 10 .

[0059] The seating information acquisition unit 340 acquires the seating positions of the plurality of users in the mobile object 10. The speaker identification unit 330 identifies the speaker from among the plurality of users based on the sound source direction detected from the sound information acquired by the plurality of microphones 60 mounted on the mobile object 10, the installation positions of the plurality of microphones 60 in the mobile object 10, and the seating positions of the plurality of users acquired by the seating information acquisition unit 340.

[0060] The speaker identification unit 330 identifies a speaker from among the plurality of users based on the voice information acquired by each microphone 60 mounted corresponding to the plurality of seats of the moving object 10 and the seat positions of the plurality of users acquired by the seat information acquisition unit 340 .

[0061] When acceptance unit 370 accepts multiple users as conference attendees, seating information acquisition unit 340 acquires the seating locations of the multiple users by acquiring information specifying the seat location of mobile object 10 from each of the multiple users. When acceptance unit 370 accepts multiple users as conference attendees, seating information acquisition unit 340 may acquire the seating locations of the multiple users based on the positions of the user terminals determined from wireless signals received from the user terminals of the multiple users. For example, seating information acquisition unit 340 may calculate the position of user terminal 70 based on signals received via UWB communication with antenna 26 connected to user terminal 70, and acquire the position of user 90a based on the calculated position of user terminal 70.

[0062] Figure 41. Shown is an example of a processing sequence performed by the user terminal 80, the communication terminal 20, the teleconference server 100, and the user terminal 120. Assume that at the start of this sequence, there is no user 90 participating in the teleconference using the communication terminal 20.

[0063] In S402, the number of people determining unit 350 of the communication terminal 20 determines the number of people on the moving object 10. In the number of people determining unit 350, the communication terminal 20 determines the number of people on the moving object 10 based on the image of the moving object 10 captured by the camera 28 and the information from the seat sensor provided in each seat 11.

[0064] In S404, user number determination unit 360 determines the number of users that reception unit 370 can accept based on the number of people determined by number determination unit 350. For example, user number determination unit 360 determines the number of people determined by number determination unit 350 as the upper limit of the number of users that reception unit 370 can accept.

[0065] Here, user number determination unit 360 may determine the number of users that acceptance unit 370 can accept based on the communication status of communication unit 390 acquired by connection status information acquisition unit 240. The worse the communication status of communication unit 390 acquired by connection status information acquisition unit 240, the smaller the number of users that acceptance unit 370 can accept.

[0066] In S406 , when the user 90a performs an operation to log in to the remote conference using the communication terminal 20 , the accepting unit 370 accepts the user 90a as a participant of the remote conference and performs a login process such as sending the authentication information input during login to the remote conference server 100 .

[0067] In S408, when user 90b uses user terminal 80 to log in to the remote conference in which user 90a is participating, user terminal 80 performs a login process such as transmitting the authentication information input during login to remote conference server 100. Similarly, in S410, when user 90c uses user terminal 120 to log in to the remote conference in which user 90a is participating, user terminal 120 performs a login process such as transmitting the authentication information input during login to remote conference server 100.

[0068] During the login process at S406, the seating information acquisition unit 340 acquires the seated position of the user 90 in the mobile object 10. For example, as part of the login process, the seating information acquisition unit 340 causes the display device 22a to display a screen for specifying the seated position of the user 90a, and acquires information indicating the seated position input by the user 90a via the display device 22a. For example, the seated position may be information identifying the seat 11 where the user 90a is seated. The seating information acquisition unit 340 stores, as seating information, information that associates the user identification information identifying the user 90a with the seat identification information identifying the seat 11 where the user 90a is seated.

[0069] In another method, the seating information acquisition unit 340 determines the seat 11 where the user 90 is seated based on signals received via wireless communication using UWB. For example, the seating information acquisition unit 340 calculates the distances between antenna 26a and user terminal 70, antenna 26b and user terminal 70, and antenna 26c and user terminal 70 based on Time of Arrival (TOA). The unit then calculates the position of user terminal 70 using three-point measurement based on these calculated distances. The seating information acquisition unit 340 can calculate the angle of arrival of the signal from user terminal 70 to each antenna 26 based on the time difference between each antenna 26 receiving the signal transmitted from user terminal 70. The unit then calculates the position of user terminal 70 based on the calculated angle of arrival and the position of each antenna 26. Based on the calculated positions of user terminal 70 and the positions of seat 11, the seating information acquisition unit 340 determines the seat 11 where the user 90a holding the user terminal 70 is seated. The seating information acquisition unit 340 may store, as seating information, information that associates the user identification information of the user 90a previously associated with the user terminal 70 with the seat identification information identifying the seat 11 where the user 90a is seated. The seating information acquisition unit 340 may receive the user identification information identifying the user 90a from the user terminal 70 and store, as seating information, information that associates the received user identification information with the seat identification information identifying the seat 11 where the user 90a is seated. A login process may be performed using the user terminal 70, and the seating information acquisition unit 340 may receive, from the user terminal 70, the user identification information input to the user terminal 70 as part of the login process.

[0070] In S420, when communication terminal 20 completes logging into the remote conference, communication terminal 20 begins periodically transmitting location information and speed information to remote conference server 100. For example, communication unit 390 transmits current location information calculated based on signals received from GNSS satellites, current location information calculated based on temporal changes in current location information, or speed information of mobile object 10 to remote conference server 100 at predetermined time intervals. Similarly, when logging into the remote conference, user terminal 80 and user terminal 120 begin periodically transmitting location information and speed information to remote conference server 100 (S422 and S424). User terminal 80 and user terminal 120 may transmit current speed information calculated based on temporal changes in current location information to remote conference server 100. Terminal information acquisition unit 230 of remote conference server 100 receives location information and speed information from user terminal 80, communication terminal 20, and user terminal 120. In S420 , S422 , and S424 , the user terminal 80 , the communication terminal 20 , and the user terminal 120 may start a process of periodically transmitting their respective acceleration information to the teleconference server 100 in addition to or instead of the movement speed.

[0071] In S430, the determination unit 210 of the teleconference server 100 determines the terminal used by user 90 aboard the same mobile body 10 to participate in the same teleconference. For example, the terminal information acquisition unit 230 determines the terminal used by user 90 aboard the same mobile body 10 to participate in the same teleconference based on the location information and speed information received from each of the user terminal 80, the communication terminal 20, and the user terminal 120, as well as the identification information of the terminal used by the logged-in user. For example, when the location of user terminal 80 and the location of communication terminal 20 are within a predetermined range and the difference between the moving speeds of user terminal 80 and communication terminal 20 is less than a predetermined value, the determination unit 210 determines that user terminal 80 and communication terminal 20 are used by user 90 aboard the same mobile body 10 to participate in the same teleconference.

[0072] Here, the connection status information acquisition unit 240 may also acquire information indicating the connection status with another terminal via wireless communication from the user terminal 80, the communication terminal 20, and the user terminal 120, and determine, based on the acquired information, the terminal used by the user 90 traveling on the same mobile vehicle 10 to participate in the same teleconference. For example, when the user terminal 80 and the communication terminal 20 are connected via WiFi (registered trademark), the communication unit 390 of the communication terminal 20 and the user terminal 80 transmit identification information of one or more terminals connected via WiFi (registered trademark) and information indicating the connection status to the teleconference server 100. The WiFi (registered trademark) connection status may include information indicating the received signal strength from the other terminal. The connection status information acquisition unit 240 acquires the information transmitted from the communication terminal 20 and the user terminal 80, and the determination unit 210 determines, based on this information, the terminal used by the user 90 traveling on the same mobile vehicle 10 to participate in the same teleconference. The determination unit 210 may determine that the terminal connected via WiFi (registered trademark) and having a received signal strength greater than a predetermined value is the terminal used by the user 90 traveling on the same mobile vehicle 10 to participate in the same teleconference.

[0073] In S432, the control unit 220 of the teleconference server 100 performs processing for transmitting a switching prompt to the communication terminal 20 and the user terminal 80, recommending a switch to the participating source terminal of the teleconference. For example, the control unit 220 transmits information to the communication terminal 20 and the user terminal 80 indicating that user 90a and user 90b are participating in the same teleconference and recommending that user 90b participate in the teleconference via the communication terminal 20. As a result, the participant information acquisition unit 310 of the communication terminal 20 acquires information indicating that one or more users 90 participating in the teleconference using the user terminal 80 are present in the mobile object 10.

[0074] Here, control unit 220 can obtain the communication status of each of user terminal 80 and user terminal 120 and use the terminal with the best communication status among user terminal 80 and user terminal 120 as the participating source terminal for the remote conference. This allows multiple users 90 to participate in the remote conference via the communication line of the terminal with the more stable communication status.

[0075] In S434, the control unit 320 of the communication terminal 20 performs processing for switching the participating terminal for the remote conference. In S436, the communication terminal 20 transmits a notification of the completion of the switching of the participating terminal to the remote conference server 100. For example, in S434, the control unit 320 causes the display device 22 to display a screen for accepting a confirmation operation from user 90b to join the remote conference via the communication terminal 20. When the communication terminal 20 detects that user 90b has performed the confirmation operation on the display device 22b, in S436, the communication terminal 20 transmits a notification of the completion of the switching to the remote conference server 100, indicating that user 90b has joined the remote conference via the communication terminal 20.

[0076] Figure 4 In step S434, the control unit 320 performs a process for switching the participating terminal for the remote conference. However, the control unit 320 may perform a process for switching the participating terminal for the remote conference only when the number of users does not exceed the number of users determined by the user number determination unit 360.

[0077] Figure 5 An example of a screen 500 displayed on the display device 22 or the user terminal 120 is schematically shown. The screen 500 shows that the user 90a participates in the remote conference from the communication terminal 20, the user 90b participates in the remote conference from the user terminal 80, and the user 90c participates in the remote conference from the user terminal 120.

[0078] Screen 500 includes objects 510, 520, and 530. Object 510 includes, for example, a facial image of user 90a captured by camera 28. Object 520 includes a facial image of user 90b captured by user terminal 80. Object 530 includes a facial image of user 90c captured by user terminal 120.

[0079] Figure 6 An example of a screen 600 displayed on the display device 22 or the user terminal 120 is schematically shown. The screen 600 shows a case where the user 90a and the user 90b participate in the remote conference from the communication terminal 20 and the user 90c participates in the remote conference from the user terminal 120.

[0080] Screen 600 includes objects 610, 620, and 630. Object 610 includes, for example, facial images of users 90a and 90b captured by camera 28. Object 620, like object 610, also includes facial images of users 90a and 90b captured by camera 28. Object 630 includes a facial image of user 90c captured by and transmitted from user terminal 120.

[0081] As described above, according to communication terminal 20, even when multiple users 90 participate in a remote conference through a single communication terminal 20, multiple users 90 can be considered as individual participants. This allows other participants to know that multiple users 90 are participating in a remote conference from communication terminal 20.

[0082] exist Figure 6 In the example, the control unit 320 uses the image captured by the camera 28 as the participant image of user 90a and the participant image of user 90b. In another embodiment, the control unit 320 may extract the facial image of user 90a and the facial image of user 90b from the image captured by the camera 28, respectively, and process the extracted facial image of user 90a as the participant image of user 90a, and the extracted facial image of user 90b as the participant image of user 90b.

[0083] Figure 7 This is a flowchart showing a process procedure related to speaker identification by the communication terminal 20. For example, when the communication terminal 20 executes a process related to a teleconference in cooperation with the teleconference server 100, the process of this flowchart is repeatedly performed.

[0084] In S702, the speaker determination unit 330 detects a sound from the information collected by the microphone 60. In S704, the speaker determination unit 330 determines a speaker based on the sound information and the seating information detected in S702.

[0085] For example, if user 90b is the speaker, the volume of the sound detected by microphone 60b is higher than the volume of the sound detected by other microphones 60a, 60c, and 60d. Therefore, speaker identification unit 330 determines that the speaker is seated at seat 11b, which is located at the position corresponding to microphone 60b. Speaker identification unit 330 refers to the seating information and identifies user 90b, identified by the user identification information associated with the seat identification information of seat 11b, as the speaker.

[0086] In S706, the communication unit 390 transmits the speaker information to the user terminal 120. For example, the communication unit 390 transmits the identification information of the speaker in the teleconference system to the user terminal 120 as the speaker information.

[0087] In S708 , the control unit 320 highlights the speaker's object. For example, when the user 90 b is the speaker, the control unit 320 highlights the object 620 . In this case, the control unit 320 does not highlight the object 610 .

[0088] Figure 8An example of a screen 800 displayed on the display device 22 or the user terminal 120 is schematically shown. The screen 800 shows that the users 90a and 90b are participating in a remote conference from the communication terminal 20, the user 90c is participating in a remote conference from the user terminal 120, and the user 90b is speaking.

[0089] Screen 800 includes objects 810, 820, and 830. Objects 810 and 830 are identical to objects 610 and 630, respectively. Object 820, for example, includes facial images of user 90a and user 90b captured by camera 28. Since user 90b is identified as the speaker, object 820 is highlighted to indicate that user 90b is the speaker.

[0090] As described above, according to the communication terminal 20 , even when a plurality of users participate in a remote conference through a single communication terminal 20 , it is possible to notify other conference participants who is the speaker.

[0091] Figure 9 The configuration of a mobile object 910 is shown as a modified example of the mobile object 10. The mobile object 910 is different from the mobile object 10 in that the mobile object 910 includes a microphone array 960 instead of the microphone 60. Therefore, the difference from the mobile object 10 will be mainly described here.

[0092] Microphone array 960 includes a plurality of microphones. Speaker identification unit 330 of communication terminal 20 uses microphone array 960 to localize a sound source.

[0093] For example, when the distance from the sound source to each microphone is different, the time it takes for the sound wave to reach each microphone is different, and the difference in arrival time at each microphone is a function of the sound source's position. Speaker identification unit 330 utilizes this principle to determine the direction of the sound source based on the sound signals acquired by the multiple microphones included in microphone array 960. Speaker identification unit 330 can identify the speaker from user 90 based on the estimated sound source direction, the installation position of the microphone array on mobile object 10, and the seating information acquired by seating information acquisition unit 340.

[0094] In the above embodiment, speaker identification unit 330 identifies the speaker using sounds captured by multiple microphones. Instead of or in addition to using sounds captured by multiple microphones, speaker identification unit 330 may identify the speaker by recognizing the voice of user 90. For example, speaker identification unit 330 may pre-acquire the speaker's voiceprint during login processing, etc., and store the acquired voiceprint as voiceprint information in association with user 90's identification information. Speaker identification unit 330 may also extract a voiceprint from sounds captured by microphones installed in mobile device 10 during a remote conference, and identify the speaker from among multiple users 90 based on the extracted voiceprint and voiceprint information.

[0095] In another embodiment, the speaker determination unit 330 may also determine the speaker based on the facial image of each of the plurality of users 90 captured by the camera 28. For example, the speaker determination unit 330 may also determine the speaker from the plurality of users 90 based on the movement of the mouth in the facial image. In addition to the movement of the mouth in the facial image, the speaker determination unit 330 may also determine the speaker from the plurality of users 90 based on the sound collected by the microphone.

[0096] In the embodiment described above, the moving object 10 is a vehicle. However, any moving object other than a vehicle can be applied as the moving object 10. For example, the moving object 10 may be a transport device such as a train, an airplane, or a ship.

[0097] According to the teleconference system 5 of the above-described embodiment, when multiple users 90 participate in the same teleconference from different terminals on a mobile object 10, they can be prompted to join the conference through a single terminal. This allows multiple users to be prompted to join the teleconference via a single communication line. Furthermore, because the communication terminal 20 determines the upper limit of users who can participate in the teleconference via the communication terminal 20 based on the number of people on board the mobile object 10 and the communication status of the communication terminal 20, it is possible to prevent the communication volume of the communication terminal 20 from exceeding the predetermined communication volume for the teleconference. Furthermore, when multiple users participate in the teleconference via the mobile object 10, the current speaker is identified based on the voice and seating information acquired by the multiple microphones installed in the mobile object 10, and the display device 22 and the user terminal 120 display that the identified speaker is speaking. This allows each participant to be informed of who is speaking, even when a single communication terminal 20 accepts multiple users 90 as participants in the teleconference.

[0098] Figure 10An example of a computer 2000 that can fully or partially embody multiple embodiments of the present invention is shown. Programs installed on computer 2000 can cause computer 2000 to function as a remote conference management system 110 or its various units, or various devices such as communication terminal 20 or its various units, according to embodiments, to perform operations associated with the system or its various units, the device or its various units, and / or to perform processes or steps of processes according to embodiments. Such programs can be executed by CPU 2012 to cause computer 2000 to perform the processing described in this specification and specific operations associated with some or all of the functional blocks in the block diagrams.

[0099] The computer 2000 according to this embodiment includes a CPU 2012 and a RAM 2014, which are interconnected via a main controller 2010. The computer 2000 further includes a ROM 2026, a flash memory 2024, a communication interface 2022, and an input / output chip 2040. The ROM 2026, the flash memory 2024, the communication interface 2022, and the input / output chip 2040 are connected to the main controller 2010 via the input / output controller 2020.

[0100] The CPU 2012 operates according to the programs stored in the ROM 2026 and the RAM 2014 , thereby controlling each unit.

[0101] The communication interface 2022 communicates with other electronic devices via a network. The flash memory 2024 stores programs and data used by the CPU 2012 in the computer 2000. The ROM 2026 stores startup programs and the like executed by the computer 2000 when activated, and / or programs dependent on the hardware of the computer 2000. The input / output chip 2040 can also connect various input / output units such as a keyboard, mouse, and monitor to the input / output controller 2020 via input / output ports such as a serial port, a parallel port, a keyboard port, a mouse port, a monitor port, a USB port, and an HDMI (registered trademark) port.

[0102] The program is provided via a computer-readable storage medium such as a CD-ROM, DVD-ROM, or USB flash drive, or via a network. RAM 2014, ROM 2026, or flash memory 2024 are examples of computer-readable storage media. The program is installed into flash memory 2024, RAM 2014, or ROM 2026 and executed by CPU 2012. The information processing described in these programs is read by computer 2000, enabling collaboration between the program and the various types of hardware resources described above. A device or method can be constructed by implementing information manipulation or processing in accordance with the use of computer 2000.

[0103] For example, when communication is performed between the computer 2000 and an external device, the CPU 2012 can execute a communication program loaded into the RAM 2014 and, based on the processing described in the communication program, instruct the communication interface 2022 to perform communication processing. Under the control of the CPU 2012, the communication interface 2022 reads transmission data stored in a transmission buffer area provided in the RAM 2014 and a recording medium such as the flash memory 2024, transmits the read transmission data to the network, and writes reception data received from the network to a reception buffer area provided on the recording medium.

[0104] Furthermore, the CPU 2012 can read all or a required portion of a file or database stored in a recording medium such as the flash memory 2024 into the RAM 2014 and perform various processes on the data in the RAM 2014. The CPU 2012 then writes the processed data back to the recording medium.

[0105] Various types of programs, data, tables, and various information such as databases can be saved to a recording medium and applied to information processing. CPU2012 can perform various processing described in this specification, including various operations specified by the instruction sequence of the program, information processing, conditional judgment, conditional branching, unconditional branching, information retrieval / replacement, etc., on the data read from RAM2014, and write the results back to RAM2014. In addition, CPU2012 can retrieve information in files, databases, etc. in the recording medium. For example, when a plurality of items each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in a recording medium, CPU2012 can retrieve an item that specifies an attribute value of the first attribute and is consistent with the condition from the plurality of items, read the attribute value of the second attribute stored in the item, and thereby obtain the attribute value of the second attribute associated with the first attribute that meets the pre-set condition.

[0106] The programs or software modules described above can be stored in a computer-readable storage medium on or near the computer 2000. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can be used as the computer-readable storage medium. The program stored in the computer-readable storage medium can be provided to the computer 2000 via the network.

[0107] Programs installed in computer 2000 that enable computer 2000 to function as remote conference management system 110, when executed by the computer, cause operations in CPU 2012 and other systems to operate, causing computer 2000 to function as the various units of remote conference management system 110. The information processing described in these programs is read into computer 2000, allowing computer 2000 to function as specific units that collaborate with the software and the various hardware resources described above, namely, the various units of remote conference management system 110. Furthermore, by utilizing these specific units to perform calculations or processing of information corresponding to the intended use of computer 2000 in this embodiment, a unique remote conference management system 110 corresponding to the intended use is constructed.

[0108] Programs installed in computer 2000 that enable computer 2000 to function as communication terminal 20, when executed by the computer, cause operations in CPU 2012 and other systems to operate, causing computer 2000 to function as the various units of communication terminal 20. Information processing described in these programs is read into computer 2000, allowing computer 2000 to function as specific units that collaborate with the various hardware resources described above, namely, the various units of communication terminal 20. Furthermore, by utilizing these specific units to perform calculations or processing of information corresponding to the intended use of computer 2000 in this embodiment, a unique communication terminal 20 corresponding to the intended use is constructed.

[0109] Various embodiments are described with reference to block diagrams, etc. In the block diagrams, each functional block may represent (1) a step of a process for performing an operation or (2) each unit of a device having the function of performing an operation. Specific steps and each unit may be implemented by a dedicated circuit, a programmable circuit supplied together with computer-readable instructions stored on a computer-readable medium, and / or a processor supplied together with computer-readable instructions stored on a computer-readable medium. The dedicated circuit may include digital and / or analog hardware circuits, and may also include integrated circuits (ICs) and / or discrete circuits. The programmable circuit may include logical AND, logical OR, logical XOR, logical NAND, logical NOR, and other logical operations, flip-flops, registers, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and other reconfigurable hardware circuits including memory elements.

[0110] A computer-readable storage medium may include any tangible device capable of storing instructions for execution by an appropriate device. Consequently, a computer-readable storage medium having instructions stored therein constitutes at least a portion of a product containing instructions executable to implement a unit for performing the operations specified in the process flow or block diagram. Examples of computer-readable storage media include electrical storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer-readable storage media include floppy disks (registered trademark), flexible magnetic disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), electrically erasable programmable read-only memories (EEPROM), static random access memories (SRAM), compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), Blu-ray discs (RTMs), memory sticks, integrated circuit cards, and the like.

[0111] Computer-readable instructions may include assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine delegate instructions, microcode, firmware instructions, state setting data, or any source code or object code described in any combination of one or more programming languages ​​including object-oriented programming languages ​​such as Smalltalk (registered trademark), JAVA (registered trademark), C++, and conventional procedural programming languages ​​such as the "C" programming language or similar programming languages.

[0112] Computer-readable instructions are provided to a processor or programmable circuit of a general-purpose computer, a special-purpose computer, or other programmable data processing device via a local area network (LAN) or a wide area network (WAN) such as the Internet. The computer-readable instructions can be executed to implement a unit for performing the operations specified in the processing flow or block diagram. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0113] While the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications or improvements can be made to the above embodiments. It will be apparent from the claims that such modifications or improvements are also within the technical scope of the present invention.

[0114] Regarding the order in which actions, processes, steps, and processes, etc., in the apparatus, system, program, and method described in the claims, specifications, and drawings, it should be noted that unless specifically indicated by "before," "preceding," or the like, these processes may be performed in any order, as long as the output of a previous process is not used in a subsequent process. Even if the process flow in the claims, specifications, and drawings is described using "first," "next," or the like for convenience, this does not necessarily mean that the process must be performed in that order.

[0115] [Description of Reference Numerals]

[0116] 5. Remote conference system

[0117] 10 moving objects

[0118] 11 seats

[0119] 20 communication terminals

[0120] 22 display device

[0121] 26 antennas

[0122] 28 camera devices

[0123] 60 microphones

[0124] 70 user terminals

[0125] 80 user terminals

[0126] 90 users

[0127] 100 remote conference server

[0128] 110 Remote Conference Management System

[0129] 120 user terminals

[0130] 380 Storage Department

[0131] 390 Communications Department

[0132] 210 Judgment Department

[0133] 220 Control Department

[0134] 230 Terminal Information Acquisition Department

[0135] 240 Connection status information acquisition unit

[0136] 310 Participant Information Acquisition Department

[0137] 320 Control Department

[0138] 330 Speaker Identification Unit

[0139] 340 Seating Information Acquisition Department

[0140] 350 Headcount Determination Department

[0141] 360 User Number Determination Department

[0142] 370 Reception Department

[0143] 500 screens

[0144] 510 objects

[0145] 520 objects

[0146] 530 objects

[0147] 600 screens

[0148] 610 Object

[0149] 620 objects

[0150] 630 objects

[0151] 800 screens

[0152] 810 Object

[0153] 820 objects

[0154] 830 objects

[0155] 910 Mobile

[0156] 960 microphone array

[0157] 2000 Computer

[0158] 2010 main controller

[0159] 2012CPU

[0160] 2014 RAM

[0161] 2020 Input / Output Controller

[0162] 2022 Communication Interface

[0163] 2024 Flash Memory

[0164] 2026ROM

[0165] 2040 input / output chips.

Claims

1. A communication terminal mounted on a mobile object and used for remote conferencing, wherein: have: a number determination unit that determines the number of people riding on the moving object; and A user number determination unit determines the number of users that the communication terminal can accept as participants in the remote conference based on the number of people determined by the number of people determination unit.

2. The communication terminal according to claim 1, wherein The user number determination unit further determines the number of users that the communication terminal can accept as participants in the remote conference based on the communication state of the communication terminal.

3. The communication terminal according to claim 2, wherein: Also features: a participation information acquisition unit configured to acquire, when a first user participates in the remote conference using the communication terminal, information indicating that one or more users participating in the remote conference using portable terminals are present in the mobile object; as well as a control unit that, when the participation information acquisition unit acquires information indicating that one or more users participating in the remote conference using a portable terminal are present in the mobile body, controls at least some of the one or more users to participate in the remote conference through the communication terminal, with the number of users determined by the user number determination unit as an upper limit. The communication terminal according to claim 3 , wherein: The device further includes a communication unit configured to continuously transmit current location information of the communication terminal to a teleconference management system that manages the teleconference. The communication terminal according to claim 4 , wherein: The communication unit further continuously transmits at least one of the movement speed information and the acceleration information of the communication terminal to the remote conference management system.

6. The communication terminal according to any one of claims 1 to 5, wherein: The number-of-people specifying unit specifies the number of people riding on the moving object based on information detected by a seating sensor provided on a seating seat of the moving object.

7. The communication terminal according to any one of claims 1 to 5, wherein: The number of people specifying unit specifies the number of people riding on the moving object by analyzing image information of the moving object acquired by a camera provided in the moving object.

8. The communication terminal according to any one of claims 1 to 5, wherein: The mobile object is a vehicle.

9. A method, performed by a communication terminal mounted on a mobile object and used for conducting a remote conference, wherein: have: a step of determining the number of people riding on the moving object; as well as The step of determining the number of users that the communication terminal can accept as participants in the remote conference based on the number of people on board the mobile body.

10. A program, wherein The program is for causing a computer to function as the communication terminal according to any one of claims 1 to 5.

Images