Transmission terminals, programs, transmission methods, transmission systems

Abstract

To provide a transmission terminal that allows users to check information from other locations even when displaying images from the transmission terminal itself. [Solution] A transmission terminal 10 that is connected via a network 2 to a transmission management device 50 that manages a plurality of transmission terminals 10, and transmits with at least one transmission terminal, is characterized by having a transmission means 11 that sends a terminal information request to the transmission management device indicating the number of transmission terminals that are currently transmitting with the transmission terminal, and a display means 18 that displays image data received from the transmission terminals that are currently transmitting on a display device, and displays the number of transmission terminals that are currently transmitting, received from the transmission management device in response to the terminal information request, on the display device.

Description

Technical Field

[0003]

[0001] The present invention relates to a transmission management device that manages a plurality of transmission terminals and is connected via a network to a transmission terminal or the like that performs transmission with at least one transmission terminal.

Background Art

[0002] In a video conference system, a transmission terminal captures images of participants in a video conference and collects sounds such as the speeches of the participants, and transmits the collected images and sounds to a transmission terminal at the destination. The transmission terminal at the destination displays the image on a display or a projector connected to this transmission terminal, or outputs the sound from a speaker. Thereby, each participant at a remote location can hold a meeting in a state close to an actual meeting.

[0003] In such a remote conference system, the number of transmission terminals participating in the conference is not limited to two, and a multi-site video conference can be realized by all the transmission terminals participating in the conference.

[0004] Generally, in a video conference held among multiple sites, a participant using a certain transmission terminal wants to know who at a remote location is participating in the conference. Therefore, each transmission terminal divides the image area of the display of the transmission terminal into the number of transmission terminals participating in the video conference, and displays the images of the participants transmitted from each transmission terminal in each image area.

[0005] However, when the number of transmission terminals participating in the video conference increases, one image area becomes narrow, so it may be difficult for each participant to recognize other participants. Also, when the number of participating sites is large, it may not be possible to display the participants of all sites due to layout limitations.

[0006] To address these inconveniences, transmission terminals are known that have a screen mode for displaying information about transmission terminals participating in a video conference, separate from the screen mode for displaying images transmitted from other transmission terminals (see, for example, Patent Document 1). [Overview of the Initiative] [Problems that the invention aims to solve]

[0007] However, the transmission terminal described in Patent Document 1 requires switching between a screen mode for displaying images transmitted from other transmission terminals and a screen mode for displaying information about transmission terminals participating in the video conference. For example, when the screen is set to display images transmitted from other transmission terminals, it is not possible to check the information of transmission terminals that can be obtained when the screen is set to display information about transmission terminals participating in the video conference.

[0008] In view of the above problems, the present invention aims to provide a transmission terminal that can check information from other locations even when displaying an image on the screen. [Means for solving the problem]

[0009] In view of the above problems, the present invention relates to a transmission terminal that is connected via a network to a transmission management device that manages a plurality of transmission terminals and transmits with at least one transmission terminal, and is characterized by comprising: a transmission means that transmits a terminal information request representing a request to obtain the number of transmission terminals that are currently transmitting to the transmission management device; and a display means that displays image data received from the transmission terminal that is currently transmitting on a display device, and displays the number of transmission terminals that are currently transmitting, received from the transmission management device in response to the terminal information request, on the display device. [Effects of the Invention]

[0010] Even in screen mode that displays images from the transmission terminal, it is possible to provide a transmission terminal that allows users to check information from other locations. [Brief explanation of the drawing]

[0011] [Figure 1] This is an example of a schematic diagram of transmission system 1. [Figure 2] This is an example of a conceptual diagram showing the transmission and reception status of image data, sound data, and various management information in a transmission system. [Figure 3] This is an example of a conceptual diagram explaining the image quality of image data. [Figure 4] This is an example of an external view of a transmission terminal. [Figure 5] This is an example of a hardware configuration diagram for a transmission terminal. [Figure 6] This is a hardware configuration diagram of a transmission management device, UI providing device, relay device, or program providing device. [Figure 7] This is an example of a functional block diagram of each terminal, device, and system that constitutes a transmission system. [Figure 8] This is an example of a conceptual diagram of a terminal authentication management table managed by a transmission management device. [Figure 9] This is an example of a conceptual diagram of a terminal management table managed by a transmission management device. [Figure 10] This is an example of a conceptual diagram of a destination list management table managed by a transmission management device. [Figure 11] This is an example of a sequence diagram showing the preparatory steps involved in getting a transmission terminal ready to participate in a video conference. [Figure 12] This is an example of a sequence diagram showing the conference participation process from when a transmission terminal enters a state of participation in a video conference. [Figure 13] This is an example of a sequence diagram showing data transmission operations between transmission terminals. [Figure 14] This figure shows an example of a screen displayed on the screen in full image display mode. [Figure 15] This figure shows an example of a screen displayed on the screen in full image display mode. [Figure 16]It is a sequence diagram showing a terminal information display operation for displaying terminal information in a terminal information display area when a transmission terminal is switched to an information display mode. [Figure 17] It is a diagram showing an example of a display screen in the full image display mode when the number of reference points is 4. [Figure 18] It is a diagram showing an example of a display screen in which the number of reference points is displayed in the full image display mode. [Figure 19] It is a diagram showing another example of a display screen in which the number of reference points is displayed in the full image display mode. [Figure 20] It is a diagram showing an example of a display screen in which the number of images displayed in the full image display mode is different from the number of reference points displayed in the second area ar2. [Figure 21] It is a diagram showing an example of an auxiliary area for simple display. [Figure 22] It is an example of a sequence diagram showing the procedure for a transmission system to display the number of reference points when the transmission terminal is in the full image display mode. [Figure 23] It is a diagram showing an example of a screen for a user to set the display of the auxiliary area. [Figure 24] It is an example of a flowchart showing the operation procedure for a transmission terminal to switch between a standard auxiliary area and an auxiliary area for simple display when "display according to operation" is selected. [Figure 25] It is an example of a flowchart showing the operation procedure of a transmission management device. [Figure 26] It is an example of a flowchart showing the operation procedure of a transmission terminal. [Figure 27] It is a diagram showing an example of a system configuration diagram when a transmission terminal is adopted as a car navigation device. [Figure 28] It is an example of a configuration diagram of an in-vehicle car navigation device. [Figure 29] It is a diagram showing a display example of a destination list displayed on a car navigation device.

Embodiments for Carrying Out the Invention

[0012] Embodiments of the present invention will be described below with reference to the drawings.

[0013] <<Overall Configuration of the Embodiment>> Figure 1 is a schematic diagram of a transmission system 1 according to an embodiment of the present invention. Figure 2 is a conceptual diagram showing the transmission and reception status of image data, sound data, and various management information in the transmission system. Figure 3 is a conceptual diagram explaining the image quality of the image data.

[0014] Furthermore, the transmission system 1 includes a data provision system that transmits content data unidirectionally from one transmission terminal to multiple transmission terminals via a transmission management device 50, and a communication system that mutually transmits information, emotions, etc., between multiple transmission terminals via the transmission management device 50. The communication system is a system for mutually transmitting information, emotions, etc., between multiple communication terminals (corresponding to "transmission terminals") via a communication management device (corresponding to "transmission management device"), and examples include video conferencing systems and video telephone systems.

[0015] In this embodiment, a video conferencing system is assumed as an example of a communication system, a video conferencing management device as an example of a communication management device, and a video conferencing terminal as an example of a communication terminal. The transmission system 1, transmission management device 50, and transmission terminal 10 will be described accordingly. That is, the transmission terminal and transmission management device 50 of the present invention are applicable not only to video conferencing systems, but also to communication systems or data transmission systems.

[0016] As shown in Figure 1, the transmission system 1 comprises a plurality of transmission terminals 10aa, 10ab, ..., displays 120aa, 120ab, ... for each transmission terminal 10aa, 10ab, ..., a UI providing device 20, an information processing terminal 40, a relay device 30, a transmission management device 50, and a program providing device 90. The transmission terminals 10aa, 10ab, ... transmit content data, for example, by sending and receiving image data and sound data.

[0017] The UI provider 20 is a device that transmits layout information, described later, primarily for displaying the number of locations, to the transmission terminals 10aa, 10ab, ... The UI provider 20 may also be combined with, for example, the transmission management device 50 or the relay device 30.

[0018] Furthermore, the information processing terminal 40 is a device in which the functions of the transmission terminal 10 are implemented by software. Specifically, it may be a PC (Personal Computer), tablet device, smartphone, mobile phone, PDA (Personal Digital Assistant), or e-book reader.

[0019] In the following explanation, any transmission terminal among the multiple transmission terminals 10aa, 10ab, ... will be referred to as "transmission terminal 10," and any display among the multiple displays 120aa, 120ab, ... will be referred to as "display 120." Furthermore, the transmission terminal 10 that initiates the request to start a video conference will be referred to as the "requesting terminal," and the transmission terminal 10 that receives the request will be referred to as the "recipient terminal."

[0020] Furthermore, a transmission terminal 10 participating in a video conference or its location is referred to as a "site." The number of sites is the same as the number of transmission terminals 10 participating in the video conference, but multiple transmission terminals 10 may exist in a single space (for example, one conference room).

[0021] As shown in Figure 2, in the transmission system 1, a management information session sei is established between the requesting terminal and the destination terminal via the transmission management device 50 for sending and receiving various management information.

[0022] Furthermore, between the transmission terminals 10 where the management information session sei has been established, four sessions are established via the relay device 30 for sending and receiving four types of data: high-resolution image data, medium-resolution image data, low-resolution image data, and sound data. In this embodiment, these four sessions are collectively referred to as the "image / sound data session sed".

[0023] In this embodiment, the image data is encoded in a scalable manner. For example, the image data used in this embodiment includes low-resolution image data, medium-resolution image data, and high-resolution image data. As shown in Figure 3(a), the low-resolution image data has a resolution of 320 pixels horizontally and 180 pixels vertically, as shown in Figure 3(b), the medium-resolution image data has a resolution of 640 pixels horizontally and 360 pixels vertically, and the high-resolution image data has a resolution of 1280 pixels horizontally and 720 pixels vertically.

[0024] If the bandwidth through which the image data is transmitted is narrow, only the low-resolution image data that serves as the base image will be transmitted, and the receiving transmission terminal 10 can display only low-quality images based on the transmitted low-resolution image data.

[0025] Furthermore, if the bandwidth through which the image data is transmitted is relatively wide, low-resolution image data and medium-resolution image data that serve as the base image are transmitted, and the receiving transmission terminal 10 can display a medium-quality image based on the transmitted low-resolution and medium-resolution image data.

[0026] Furthermore, if the bandwidth through which the image data is transmitted is very wide, low-resolution image data, medium-resolution image data, and high-resolution image data are transmitted as base image quality, and the receiving transmission terminal 10 can display a high-quality image based on the transmitted low-resolution image data, medium-resolution image data, and high-resolution image data.

[0027] In Figure 1, the relay device 30 relays content data between multiple transmission terminals 10. The transmission management device 50 centrally manages login authentication from the transmission terminals 10, management of the transmission status of the transmission terminals 10, management of the destination list, and the transmission status of the relay device 30. The image data may be a video, a still image, or both.

[0028] Multiple routers 70a, 70b, 70c, 70d, 70ab, and 70cd select the optimal route for image data and sound data. In the following description, any of the routers 70a, 70b, 70c, 70d, 70ab, and 70cd will be referred to as "router 70".

[0029] The program supply device 90 is equipped with an HD (Hard Disk) 204, which will be described later, and stores a transmission terminal control program 1000a for realizing various functions in the transmission terminal 10, and can transmit the transmission terminal control program 1000a to the transmission terminal 10.

[0030] Furthermore, the HD204 of the program supply device 90 also stores programs for the relay device 30 to implement various functions, and can transmit these relay device programs to the relay device 30.

[0031] Furthermore, the HD204 of the program supply device 90 also stores transmission management programs for implementing various functions in the transmission management device 50, and can transmit these transmission management programs to the transmission management device 50.

[0032] Furthermore, the HD204 of the program provider 90 also stores programs for the UI provider that enable various functions to be implemented in the UI provider 20, and can transmit these UI provider programs to the UI provider 20.

[0033] Furthermore, the HD204 of the program supply device 90 also stores programs for the information processing terminal 40 to implement various functions, and can transmit these programs to the information processing terminal 40.

[0034] Transmission terminals 10aa, 10ab, 10ac, ... and router 70a are connected via LAN2a for transmission. Transmission terminals 10ba, 10bb, 10bc, ... and router 70b are connected via LAN2b for transmission.

[0035] Furthermore, LAN2a and LAN2b are connected via a dedicated line 2ab that includes router 70ab, and are located within a designated region A. For example, region A is Japan, LAN2a is located within the Tokyo office, and LAN2b is located within the Osaka office.

[0036] On the other hand, transmission terminals 10ca, 10cb, 10cc, ... and router 70c are connected via LAN2c for transmission. Transmission terminals 10da, 10db, 10dc, ... and router 70d are connected via LAN2d for transmission.

[0037] Furthermore, LAN2c and LAN2d are connected via a dedicated line 2cd that includes router 70cd, and are located within a designated region B. For example, region B is the United States, LAN2c is located within the New York office, and LAN2d is located within the Washington D.C. office. Regions A and B are connected to each other via the Internet 2i from routers 70ab and 70cd, respectively, enabling transmission between them.

[0038] Furthermore, the relay device 30, transmission management device 50, UI providing device 20, information processing terminal 40, and program providing device 90 are transmitted to each transmission terminal 10 via the Internet 2i. Here, the relay device 30, transmission management device 50, UI providing device 20, information processing terminal 40, and program providing device 90 may be installed in region A or region B, or in other regions.

[0039] In this embodiment, the transmission network 2 is configured by LAN2a, LAN2b, dedicated line 2ab, Internet 2i, dedicated line 2cd, LAN2c, and LAN2d. The transmission network 2 may include locations where wireless transmission such as WiFi (Wireless Fidelity) or Bluetooth (registered trademark) is performed, in addition to wired transmission.

[0040] Furthermore, in Figure 1, the four sets of numbers shown under each transmission terminal 10, relay device 30, transmission management device 50, each router 70, UI provider device 20, information processing terminal 40, and program provider device 90 simply represent typical IPv4 IP addresses. For example, the IP address of transmission terminal 10aa is "1.2.1.3". While IPv6 could also be used instead of IPv4, IPv4 will be used for simplicity in this explanation.

[0041] Each transmission terminal 10 may be used not only for video conferencing between multiple business locations or between different rooms within the same business location, but also for video conferencing within the same room, or between outdoor and indoor locations, or between outdoor and outdoor locations. When each transmission terminal 10 is used outdoors, wireless transmission such as via a mobile phone transmission network will be performed.

[0042] <<Hardware configuration of the embodiment>> Figure 4 is an external view of the transmission terminal 10. Hereafter, the longitudinal direction of the transmission terminal 10 will be referred to as the X-axis direction, the direction perpendicular to the X-axis direction in the horizontal plane will be referred to as the Y-axis direction, and the direction perpendicular to both the X-axis and Y-axis directions (vertical direction) will be referred to as the Z-axis direction.

[0043] As shown in Figure 4, the transmission terminal 10 comprises a housing 1100, an arm 1200, and a camera housing 1300. The front wall 1110 of the housing 1100 is provided with an intake surface (not shown) formed by multiple intake holes, and the rear wall 1120 of the housing 1100 is provided with an exhaust surface 1121 formed by multiple exhaust holes. As a result, a cooling fan built into the housing 1100 can draw in outside air from behind the transmission terminal 10 through the intake surface (not shown) and exhaust it to the rear of the transmission terminal 10 through the exhaust surface 1121. A sound-collecting hole 1131 is formed on the right wall 1130 of the housing 1100, allowing sounds such as voices, noises, and background noise to be collected by a built-in microphone 114, which will be described later.

[0044] An operation panel 1150 is formed on the right wall 1130 side of the housing 1100. This operation panel 1150 is equipped with several operation buttons (108a to 108e) described later, a power switch 109 described later, and an alarm lamp 119 described later, as well as a sound output surface 1151 formed by several audio output holes for passing sound output from the built-in speaker 115 described later. In addition, a housing section 1160 is formed on the left wall 1140 side of the housing 1100 as a recess for housing the arm 1200 and the camera housing 1300. Multiple connection ports (1132a to 1132c) for electrically connecting cables to the external device connection I / F 118 described later are provided on the right wall 1130 side of the housing 1100. On the other hand, the left side wall 1140 of the housing 1100 is provided with a connection port (not shown) for electrically connecting the cable 120c for the display 120 to the external device connection I / F 118, which will be described later.

[0045] In the following explanation, "operation button 108" will be used to refer to any operation button (108a to 108e), and "connection port 1132" will be used to refer to any connection port (1132a to 1132c).

[0046] Next, the arm 1200 is attached to the housing 1100 via a torque hinge 1210, and the arm 1200 is configured to rotate vertically relative to the housing 1100 within a tilt angle θ1 of 135 degrees. Figure 4 shows the state when the tilt angle θ1 is 90 degrees.

[0047] The camera housing 1300 is equipped with a built-in camera 112, which will be described later, and can capture images of the user, documents, and the room. A torque hinge 1310 is also formed on the camera housing 1300. The camera housing 1300 is attached to the arm 1200 via the torque hinge 1310. The camera housing 1300 is attached to the arm 1200 via the torque hinge 1310, and is configured to rotate in the up, down, left, and right directions relative to the arm 1200 within a pan angle range of ±180 degrees θ2 and a tilt angle range of ±45 degrees θ3, with the state shown in Figure 4 being 0 degrees.

[0048] Note that the UI provider 20, relay device 30, transmission management device 50, program provider 90, and maintenance system 100 all have the same appearance as typical server computers, so their appearance descriptions are omitted. The information processing terminal 40 has the same appearance as typical PCs, smartphones, tablet terminals, etc., so its appearance description is omitted.

[0049] Figure 5 is an example of a hardware configuration diagram of a transmission terminal. As shown in Figure 5, the transmission terminal 10 includes a CPU (Central Processing Unit) 101 that controls the overall operation of the transmission terminal 10, a ROM (Read Only Memory) 102 that stores programs used to drive the CPU 101, such as an IPL (Initial Program Loader), a RAM (Random Access Memory) 103 used as the work area of ​​the CPU 101, a flash memory 104 that stores various data such as the transmission terminal control program 1000a, image data and sound data, an SSD (Solid State Drive) 105 that controls the reading or writing of various data to the flash memory 104 according to the control of the CPU 101, a media drive 107 that controls the reading or writing (storage) of data to a recording medium 106 such as flash memory, an operation button 108 that is operated when selecting a destination for the transmission terminal 10, a power switch 109 for switching the power of the transmission terminal 10 ON / OFF, and a network I / F (Interface) 111 for transmitting data using the transmission network 2.

[0050] Furthermore, the transmission terminal 10 includes a built-in camera 112 that captures an image of a subject and obtains image data according to the control of the CPU 101, an image sensor I / F 113 that controls the drive of the camera 112, a built-in microphone 114 that acts as a built-in sound collector for inputting sound, a built-in speaker 115 that outputs sound, an audio input / output I / F 116 that processes the input and output of sound signals between the microphone 114 and the speaker 115 according to the control of the CPU 101, a display I / F 117 that transmits image data to an external display 120 according to the control of the CPU 101, an external device connection I / F 118 for connecting various external devices, and bus lines 110 such as an address bus and a data bus for electrically connecting each of the above components.

[0051] Display 120 is a display device composed of liquid crystal or organic EL that displays images of the subject, operation icons, etc. Display 120 is connected to display I / F 117 by cable 120c. Cable 120c may be a cable for analog RGB (VGA) signals, a cable for component video, or a cable for HDMI (High-Definition Multimedia Interface) or DVI (Digital Video Interactive) signals.

[0052] The camera 112 includes a lens and a solid-state image sensor that converts light into electric charge to create an electronic image of the subject. As the solid-state image sensor, a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge Coupled Device) is used.

[0053] The external device connection interface (F118) allows external devices such as external cameras, external microphones, and external speakers to be connected via USB (Universal Serial Bus) cables, etc.

[0054] When an external camera is connected to the external device connection interface 118, the CPU 101 prioritizes driving the external camera over the built-in camera 112. Similarly, when an external microphone or external speaker is connected to the external device connection interface 118, the CPU 101 prioritizes driving the external microphone or external speaker over the built-in microphone 114 and built-in speaker 115, respectively.

[0055] The recording medium 106 is configured to be detachably attached to the transmission terminal 10. Furthermore, the transmission terminal 10 may use an EEPROM (Electrically Erasable and Programmable ROM) or the like instead of flash memory 104, as long as it is a non-volatile memory that reads or writes data according to the control of the CPU 101.

[0056] Furthermore, the transmission terminal control program 1000a may be distributed as an installable or executable file, recorded on a computer-readable recording medium such as the recording medium 106. Alternatively, the transmission terminal control program 1000a may be stored in the ROM 102 instead of the flash memory 104.

[0057] Since the information processing terminal 40 has the same hardware configuration as the transmission terminal 10, its description will be omitted. However, the SSD 105 contains a program for controlling the information processing terminal 40.

[0058] Figure 6 is a hardware configuration diagram of a transmission management device, UI providing device, relay device, or program providing device. As shown in Figure 6, the transmission management device 50 includes a CPU 201 that controls the operation of the entire transmission management device 50, a ROM 202 that stores programs used to drive the CPU 201 such as an IPL, a RAM 203 used as the work area of ​​the CPU 201, an HD 204 that stores various data such as transmission management programs, an HDD (Hard Disk Drive) 205 that controls the reading or writing of various data to the HD 204 according to the control of the CPU 201, a media drive 207 that controls the reading or writing (storage) of data to a recording medium 206 such as flash memory, a display 208 that displays various information such as cursors, menus, windows, characters, or images, a network I / F 209 for data transmission using the transmission network 2, a keyboard 211 with multiple keys for inputting characters, numbers, and various instructions, a mouse 212 for selecting and executing various instructions, selecting processing targets, moving the cursor, etc., and a CD-ROM (Compact Disc Read Only) as an example of a removable recording medium. The system includes a CD-ROM drive 214 that controls the reading or writing of various data to the Memory 213, and bus lines 210 such as an address bus and a data bus for electrically connecting the above components.

[0059] The transmission management program may be distributed as an installable or executable file, recorded on a computer-readable recording medium such as recording media 206 or CD-ROM 213. Alternatively, the transmission management program may be stored in ROM 202 instead of HD 204.

[0060] Furthermore, since the relay device 30 has the same hardware configuration as the transmission management device 50, its description will be omitted. However, the HD204 contains a program for controlling the relay device 30.

[0061] In this case as well, the relay device program may be distributed as an installable or executable file, recorded on a computer-readable recording medium such as recording media 206 or CD-ROM 213. Alternatively, the relay device program may be stored in ROM 202 instead of HD 204.

[0062] Furthermore, since the program supply device 90 has the same hardware configuration as the transmission management device 50, its description will be omitted. However, HD204 contains a program for the program supply device used to control the program supply device 90.

[0063] In this case as well, the program for the program provider may be distributed as an installable or executable file, recorded on a computer-readable recording medium such as the recording medium 206 or CD-ROM 213. Alternatively, the program for the program provider may be stored in ROM 202 instead of HD 204.

[0064] Furthermore, since the UI provider device 20 has the same hardware configuration as the transmission management device 50, its description will be omitted. However, the HD204 stores a program for the UI provider device to control the UI provider device 20 and layout information, which will be described later.

[0065] In this case as well, the UI provisioning device program and layout information may be distributed as installable or executable files on a computer-readable recording medium such as recording media 206 or CD-ROM 213. Alternatively, the UI provisioning device program and layout information may be stored in ROM 202 instead of HD 204.

[0066] Furthermore, CD-R (Compact Disc Recordable), DVD (Digital Versatile Disk), Blu-ray Disc, etc., may be used as the computer-readable recording media mentioned above.

[0067] <<Embodiment Functional Configuration>> Figure 7 is a functional block diagram of each terminal, device, and system that constitutes the transmission system. As shown in Figure 7, the transmission terminal 10, relay device 30, and transmission management device 50 are connected so that data can be transmitted via the transmission network 2. Note that the relay device 30 and program provision device 90 shown in Figure 1 are not directly related to the transmission of video conferencing data and are therefore omitted from Figure 7. As the information processing terminal 40 has the same functions as the transmission terminal 10, the transmission terminal 10 in Figure 7 may be replaced by the information processing terminal 40.

[0068] <Functional Configuration of Transmission Terminals> The transmission terminal 10 includes a transmitting / receiving unit 11, an operation input receiving unit 12, a login request unit 13, an imaging unit 14, an audio input unit 15a, an audio output unit 15b, a display control unit 17, and a storage / reading processing unit 19. Each of these units is a function realized by a CPU 101 that executes a transmission terminal control program 1000a loaded from flash memory 104 onto RAM 103. The transmission terminal 10 also has a storage unit 1000 composed of RAM 103 and flash memory 104.

[0069] (Functional configuration of each transmission terminal) The transmitting / receiving unit 11 consists of a CPU 101 and a network interface 111, and is configured to send and receive various data and information with other transmission terminals 10, relay devices 30, transmission management devices 50, or program provision devices 90 via the transmission network 2.

[0070] The transmitting / receiving unit 11 is configured to begin receiving status information from the transmission management device 50 indicating the status of each transmission terminal 10 as a candidate destination for the video conference, even before the video conference is initiated with the desired destination terminal.

[0071] The status information not only indicates whether each transmission terminal 10 is online or offline, but also shows detailed operational status such as whether it is online and capable of transmission, or whether it is online and currently transmitting.

[0072] Furthermore, the status information represents not only the operating status of each transmission terminal 10, but also various other states such as whether the cable 120c is disconnected from the transmission terminal 10, whether sound is output but no image is output, or whether sound is muted. However, in the following explanation, in order to make the present invention easier to understand, we will explain the case in which the status information represents the operating status of each transmission terminal 10.

[0073] The operation input reception unit 12 consists of a CPU 101, operation buttons 108, and a power switch 109, and is configured to receive various inputs from the user. For example, when the user turns on the power switch 109, the operation input reception unit 12 receives the power ON signal and turns on the power of the transmission terminal 10.

[0074] The login request unit 13 is configured by the CPU 101 and, upon receiving a power-on notification, transmits login request information indicating a login request and the current IP address of the terminal to the transmission management device 50 via the transmission network 2 from the transmission / reception unit 11.

[0075] Furthermore, when the user switches the power switch 109 from the ON state to the OFF state, the transmitting / receiving unit 11 sends status information to the transmission management device 50 indicating that the power has been turned OFF, and then the operation input reception unit 12 completely turns off the power to the transmission terminal 10. This allows the transmission management device 50 to know that the transmission terminal 10 has changed from power ON to power OFF.

[0076] The imaging unit 14 consists of a CPU 101, a camera 112, and an image sensor interface 113. It captures images of a subject and outputs the image data obtained from the capture to the transmission / reception unit 11.

[0077] The sound input section 15a consists of a CPU 101 and an sound input / output interface 116. It receives sound signals representing sounds collected by the microphone 114 and converts the input sound signals into sound data.

[0078] The sound output unit 15b consists of a CPU 101 and an sound input / output interface 116. It converts sound data into sound signals and outputs the converted sound signals to the speaker 115, thereby outputting sound to the speaker 115.

[0079] The display control unit 17 is composed of a CPU 101 and a display I / F 117. It combines image data of different resolutions received by the transmitting / receiving unit 11 and transmits the combined image data to the display 120, thereby displaying the image represented by the image data on the display 120.

[0080] Furthermore, the display control unit 17 transmits the destination list information received by the transmitting / receiving unit 11 from the transmission management device 50 to the display 120, thereby displaying the destination list on the display 120.

[0081] The display control unit 17 has an auxiliary area display unit 18. When the transmission terminal 10 is in the full image display mode described later, the auxiliary area display unit 18 displays the number of locations participating in the meeting in the auxiliary area of ​​the display screen.

[0082] The memory and read processing unit 19 is composed of a CPU 101 and an SSD 105, and is configured to store various data in the memory unit 1000 and to read various data stored in the memory unit 1000.

[0083] The storage unit 1000 stores an identifier (or identification information, hereinafter referred to as "terminal ID") and passwords for identifying the transmission terminal 10. Furthermore, the storage unit 1000 is also used as a reception buffer for image data and sound data received when conducting a video conference with the destination terminal.

[0084] In this embodiment, the terminal ID represents information such as language, characters, symbols, or various markings used to uniquely identify the transmission terminal 10. Alternatively, the terminal ID may be a combination of at least two of the following: language, characters, symbols, and various markings.

[0085] <Functional Configuration of Transmission Management Device> The transmission management device 50 includes a transmitting / receiving unit 51, a terminal authentication unit 52, a state management unit 53, a terminal extraction unit 54, a terminal state acquisition unit 55, and a storage / reading processing unit 59. Each of these units is a function realized by a CPU 201 that executes a transmission management program deployed from the HD 204 onto the RAM 203. The transmission management device 50 also includes a storage unit 5000, which is composed of the HD 204.

[0086] (Terminal authentication management table) The memory unit 5000 has a terminal authentication management DB 5002 which has a terminal authentication management table as shown in Figure 8. In the terminal authentication management table, each password is associated with and managed for each terminal ID of all transmission terminals 10 managed by the transmission management device 50. For example, the terminal authentication management table shown in Figure 8 shows that the terminal ID of transmission terminal 10aa is "01aa" and the password is "aaaa".

[0087] (Terminal management table) Furthermore, the memory unit 5000 has a terminal management DB 5003 which has a terminal management table as shown in Figure 9. Figure 9(a) shows the terminal management table when the transmission terminal 10aa with terminal ID "01aa" is logged in, and Figure 9(b) shows the terminal management table after the transmission terminal 10aa with terminal ID "01aa" has sent the start request information described later.

[0088] In the terminal management table, for each terminal ID of each transmission terminal 10, the identification name of each transmission terminal 10 (hereinafter referred to as "terminal name"), the operating status of each transmission terminal 10, the date and time when the login request information for logging into the transmission system 1 was received by the transmission management device 50, and the IP address of the transmission terminal 10 are associated and managed.

[0089] For example, the terminal management table shown in Figure 9(a) indicates that for transmission terminal 10aa with terminal ID "01aa", the terminal name is "Japan Tokyo Office AA Terminal", the operational status is "Online (Transmission Possible)", the date and time the login request information was received by the transmission management device 50 was "November 10, 2009 at 13:40", and the IP address of transmission terminal 10aa is "1.2.1.3".

[0090] Here, the terminal ID in the terminal management table is uniquely assigned when the transmission terminal 10 is registered with the transmission system 1, before the transmission of the login request information sent in step S22 of Figure 11, which will be described later. The terminal name is registered or changed by the administrator of the transmission management device 50. The terminal name may also be registered or changed in response to a request made from the transmission terminal 10 to the transmission management device 50.

[0091] Furthermore, an operating status of "online (transmittable)" means that the transmission terminal 10 is logged into the transmission system 1 but is not participating in a video conference, "online (transmitting)" means that the transmission terminal 10 is logged into the transmission system 1 but is participating in a video conference, and an operating status of "offline" means that the transmission terminal 10 is not logged into the transmission system 1. A transmission terminal 10 participating in a video conference is a transmission terminal 10 whose operating status is "online (transmitting)".

[0092] (Recipient list management table) Furthermore, the storage unit 5000 has a destination list management DB 5004 which has a destination list management table as shown in Figure 10. The terminal management DB 5003 constitutes the destination list management table storage unit in this invention.

[0093] In the destination list management table, all destination terminal IDs registered as candidate destination terminals are associated and managed with the terminal ID of the requesting terminal that initiates the video conference.

[0094] For example, in the destination list management table shown in Figure 10, the candidate destination terminals for a video conference to be initiated from transmission terminal 10aa, whose terminal ID is "01aa", are shown to be transmission terminal 10ab, whose terminal ID is "01ab", transmission terminal 10ba, whose terminal ID is "01ba", and transmission terminal 10db, whose terminal ID is "01db". Candidate destination terminals are added or deleted in response to requests made from transmission terminal 10 to the transmission management device 50.

[0095] Furthermore, the destination list management table registers candidate destination terminals that can request the start of a video conference, and the transmission terminals 10 that can participate in the video conference are not limited to the transmission terminals 10 registered in the destination list management table.

[0096] (URI of the UI provider) Furthermore, the memory unit 5000 stores the URI (Uniform Resource Identifier) ​​5005 of the UI provider device 20. The URI 5005 of the UI provider device 20 is set by the transmission management device 50 or the administrator of the UI provider device 20.

[0097] (Functional configuration of the transmission management device) The transmitting / receiving unit 51 consists of a CPU 201 and a network interface 209, and is configured to transmit and receive various data and information with other transmission terminals 10 or relay devices 30 and program provision devices 90 via the transmission network 2.

[0098] The terminal authentication unit 52 is configured by the CPU 101 and uses the terminal ID and password contained in the login request information received via the transmission / reception unit 51 as search keys to search the terminal authentication management DB 5002 in the storage unit 5000 and performs terminal authentication by determining whether the same terminal ID and password are managed in the terminal authentication management DB 5002.

[0099] The status management unit 53 is comprised of the CPU 101 and manages the operational status of the requesting terminal that made the login request by associating the terminal ID of the requesting terminal, the operational status, the date and time the login request information was received by the transmission management device 50, and the IP address in the terminal management DB 5003 (see Figures 9(a) and 9(b)).

[0100] Furthermore, the status management unit 53 changes the operating status of the transmission terminal 10 in the terminal management DB 5003 (see Figures 9(a) and 9(b)) from online to offline based on the status information sent from the transmission terminal 10 indicating that the power has been turned off, when the user turns the power switch 109 of the transmission terminal 10 from ON to OFF.

[0101] The terminal extraction unit 54 is configured by the CPU 101 and searches the destination list management DB 5004 (see Figure 10) using the terminal ID of the requesting terminal that made the login request as the search key, and extracts terminal IDs of candidate destination terminals for the requesting terminal.

[0102] Furthermore, the terminal extraction unit 54 searches the destination list management DB 5004 (see Figure 10) using the terminal ID of the requesting terminal that made the login request as the search key, and extracts the terminal IDs of other requesting terminals that have registered the requesting terminal's terminal ID as a candidate destination terminal.

[0103] The terminal status acquisition unit 55 is configured by the CPU 101 and searches the terminal management DB 5003 (see Figures 9(a) and 9(b)) using the terminal ID of the candidate destination terminal extracted by the terminal extraction unit 54 as the search key, and reads the operating status for each detected terminal ID.

[0104] This allows the terminal status acquisition unit 55 to acquire the operational status of candidate destination terminals for the requesting terminal that made the login request. The terminal status acquisition unit 55 also searches the terminal management DB 5003 using the terminal ID extracted by the terminal extraction unit 54 as a search key, and acquires the operational status of the requesting terminal that made the login request.

[0105] The memory and read processing unit 59 is composed of a CPU 101 and an HDD 205, and is configured to store various data in the memory unit 5000 and to read various data stored in the memory unit 5000.

[0106] (Functional Configuration of the UI Provisioning Section) The transmitting / receiving unit 21 consists of a CPU 201 and a network interface 209, and is configured to send and receive various data and information with the transmission terminal 10 via the transmission network 2.

[0107] The server unit 22 is composed of a CPU 101, which receives a layout information request signal from the transmission terminal 10, and the storage / reading processing unit 23 reads the layout information lot from the storage unit 2000 and transmits it to the transmission terminal 10 via the transmitting / receiving unit 21.

[0108] The memory / read processing unit 23 is comprised of the CPU 101 and is responsible for storing various data in the memory unit 2000 and reading various data stored in the memory unit 2000. The memory unit 2000 is comprised of the HDD 205 and mainly stores layout information lot.

[0109] <<Processing or operation of the embodiment>> Next, the operation of the transmission system 1 according to this embodiment will be described. Figure 11 is a sequence diagram showing the preparation phase operation until the transmission terminal 10 is ready to participate in a video conference. Note that Figure 11 shows an example where the transmission terminal 10aa is ready. During the preparation phase operation, various information is sent and received in the management information session sei (see Figure 2).

[0110] First, when a user of the transmission terminal 10aa turns on the power switch 109 of the transmission terminal 10aa, the operation input reception unit 12 receives the power ON signal and turns on the power (step S21). Then, triggered by the acceptance of the power ON signal, the login request unit 13 transmits login request information representing a login request from the transmission reception unit 11 of the transmission terminal 10aa to the transmission management device 50 via the transmission network 2 (step S22).

[0111] The login request information includes a terminal ID and password to identify the transmission terminal 10aa that initiated the login request. These terminal IDs and passwords are read from the storage unit 1000 by the storage / reading processing unit 19.

[0112] Furthermore, when the transmission management device 50 receives login request information sent from the transmission terminal 10aa, the transmission management device 50 can determine the IP address of the transmitting transmission terminal 10ab.

[0113] Next, the terminal authentication unit 52 of the transmission management device 50 uses the terminal ID and password contained in the login request information received via the transmission / reception unit 51 as search keys to search the terminal authentication management DB 5002 (see Figure 8) in the storage unit 5000, and performs terminal authentication by determining whether the same terminal ID and password as the search key are managed in the terminal authentication management DB 5002 (step S23).

[0114] If the terminal authentication unit 52 determines that a terminal ID and password identical to the search key are not managed, that is, if it determines that the login request is not from a transmission terminal 10 with legitimate access rights, the transmission / reception unit 51 transmits the authentication result information, which represents the authentication result obtained by the terminal authentication unit 52, to the transmission terminal 10aa that made the login request via the transmission network 2, and the preparatory operation ends.

[0115] On the other hand, if the terminal authentication unit 52 determines that the same terminal ID and password as the search key are being managed, that is, if it determines that the login request is from a transmission terminal 10 with legitimate usage rights, the status management unit 53 stores the terminal ID of the transmission terminal 10aa, its operating status, the date and time the login request information was received, and the IP address of the transmission terminal 10aa in association with each other in the terminal management DB 5003 (see Figure 9(a)) (step S24).

[0116] As a result, as shown in Figure 9(a), the terminal management table will manage the terminal by associating the terminal ID "01aa", operating status "Online (transmission possible)", reception date and time "2009.11.10.13:40", and the IP address of transmission terminal 10aa "1.2.1.3".

[0117] Next, the transmitting / receiving unit 51 of the transmission management device 50 transmits authentication result information, which represents the authentication result obtained by the terminal authentication unit 52, to the transmission terminal 10aa that requested login via the transmission network 2 (step S25). The transmission management device 50 also transmits the URI 5005 of the UI providing device 20 to the transmission terminal 10, either before or after the authentication result information. This allows the transmission terminal 10 to access the UI providing device 20.

[0118] The URI 5005 of this UI provider device 20 indicates the storage location of the layout information lot in the transmission network 2. For example, it may have a format such as "IP address (or domain name) + folder name + file name", but it is sufficient as long as it is accessible from the UI provider device 20.

[0119] By not having each transmission terminal 10 retain URI 5005, even if the UI provider 20 is replaced or the URI changes, the administrator of the transmission management device 50 only needs to change the URI stored in the transmission management device 50. In other words, changes to all transmission terminals 10 are not required, thus reducing the management burden.

[0120] Next, the terminal extraction unit 54 of the transmission management device 50 searches the destination list management DB 5004 (see Figure 10) using the terminal ID "01aa" of the transmission terminal 10aa that made the login request as the search key, and extracts terminal IDs of candidate destination terminals for the transmission terminal 10aa (step S26).

[0121] Here, the terminal IDs "01ab", "01ba", and "01db" of the destination terminals, transmission terminals 10ab, 10ba, and 10db, respectively, will be extracted for the terminal ID "01aa" of transmission terminal 10aa.

[0122] Next, the terminal status acquisition unit 55 of the transmission management device 50 searches the terminal management DB 5003 (see Figure 9(a)) using the terminal IDs "01ab", "01ba", and "01db" of the candidate destination terminals extracted by the terminal extraction unit 54 as search keys, and acquires the operating status of each transmission terminal 10ab, 10ba, and 10db by reading the operating status "offline", "online (transmission possible)", and "online (transmission in progress)" for each terminal ID extracted by the terminal extraction unit 54 (step S27).

[0123] Next, the transmitting / receiving unit 51 of the transmission management device 50 transmits destination status information to the transmission terminal 10aa via the transmission network 2, which includes the terminal IDs "01ab", "01ba", and "01db" used as search keys in step S27, and the operating status of the corresponding transmission terminals 10ab, 10ba, and 10db, respectively: "offline", "online (transmission possible)", and "online (transmission in progress)" (step S28).

[0124] This allows transmission terminal 10aa to understand the current operating status of each of the candidate destination terminals, transmission terminals 10ab, 10ba, and 10db, which are "offline," "online (transmission possible)," and "online (transmission in progress)."

[0125] Furthermore, the terminal extraction unit 54 of the transmission management device 50 searches the destination list management DB 5004 (see Figure 10) using the terminal ID "01aa" of the transmission terminal 10aa that made the login request as a search key, and extracts the terminal IDs of other transmission terminals that have registered the terminal ID "01aa" of transmission terminal 10aa as a candidate destination terminal (step S29). For example, in the destination list management table shown in Figure 10, the extracted terminal IDs are "01ab", "01ba", and "01db".

[0126] Next, the status management unit 53 of the transmission management device 50 searches the terminal management DB 5003 (see Figure 9(a)) using the terminal ID "01aa" of the transmission terminal 10aa that requested the login as the search key, and obtains the operating status "online (transmission possible)" of the transmission terminal 10aa that requested the login (step S30).

[0127] Next, the transmitting and receiving unit 51 of the transmission management device 50 transmits destination status information, which includes the terminal ID "01aa" and operating status "Online (Transmission Possible)" of transmission terminal 10aa, to the transmission terminals 10ba and 10db that are identified as "Online" in the terminal management DB 5003 (see Figure 9(a)) among the transmission terminals 10ab, 10ba, and 10db identified by the terminal IDs "01ab", "01ba", and "01db" extracted in step S29 (steps S31-1, 2).

[0128] When transmitting destination status information to transmission terminals 10ba and 10db, the transmitting / receiving unit 51 refers to the IP address of the transmission terminal managed in the terminal management table shown in Figure 9(a), based on the respective terminal IDs "01ba" and "01db".

[0129] This allows transmission terminals 10db and 10ba, which can transmit to transmission terminal 10aa that made the login request, to be informed of the terminal ID "01aa" and operating status "online (transmission possible)" of the transmission terminal 10aa that made the login request.

[0130] Furthermore, when step S28 of the preparation phase is completed, the display control unit 17 of the transmission terminal 10aa can use the destination list information displayed on the display 120 to highlight on the display 120 the information of transmission terminals 10db that are participating in the video conference, i.e., transmission terminals 10db whose operating status is "online (transmitting)" or "online (transmittable)", thereby allowing the user to recognize the transmission terminals 10 that are participating in the video conference or transmission terminals 10 that are available to participate.

[0131] Figure 12 is a sequence diagram showing the conference participation process from when the transmission terminal 10 enters a participating state in a video conference. Note that Figure 12 shows an example where the transmission terminal 10aa enters a participating state. During the conference participation process, various types of information are sent and received in the management information session sei (see Figure 2).

[0132] First, when a user presses the operation button 108 to select to participate in a video conference, the operation input reception unit 12 receives a request from the transmission terminal 10db to participate in the video conference (step S41). If the transmission terminal 10 is an information processing terminal 40, the request to participate in the video conference is received by pressing a hard key or a soft key displayed on the touch panel of the information processing terminal 40 instead of the operation button 108.

[0133] In response to this request, the transmitting / receiving unit 11 of the transmission terminal 10aa transmits participation request information to the transmission management device 50, which includes the terminal ID "01aa" of the transmission terminal 10aa and the terminal ID "01db" of the transmission terminal 10db, and indicates that the user intends to participate in the video conference (step S42).

[0134] As a result, the transmitting / receiving unit 51 of the transmission management device 50 can receive the participation request information and confirm the IP address "1.2.1.3" of the transmission terminal 10aa, which is the source of the transmission.

[0135] Next, the status management unit 53 of the transmission management device 50 searches the terminal management DB 5003 using the terminal ID "01aa" of the transmission terminal 10aa included in the participation request information as the search key, and sets the operational status of the transmission terminal 10aa that made the participation request to "online (transmitting)" (step S43). This updates the operational status in the terminal management table (see Figure 9(b)).

[0136] Next, the transmitting / receiving unit 51 of the transmission management device 50 transmits start request information, including the terminal ID "01aa" of the transmission terminal 10aa, to the transmission terminal 10db via the transmission network 2 (step S44). This allows the transmission terminal 10db to determine which transmission terminal 10 has requested to join the video conference.

[0137] For the sake of clarity in understanding the invention, the explanation will be omitted, but in the terminal management DB 5003 shown in Figure 9(b), the operating status of the transmission terminal 10cb is also "online (transmitting)," so the transmitting / receiving unit 51 of the transmission management device 50 transmits start request information including the terminal ID "01aa" of the transmission terminal 10aa to the transmission terminal 10cb via the transmission network 2.

[0138] Next, the transmission terminal 10db transmits reception completion information from the transmitting / receiving unit 11 to the transmission management device 50 via the transmission network 2, indicating that the reception of the start request information has been completed (step S45).

[0139] Next, the transmitting / receiving unit 51 of the transmission management device 50 transmits relay start request information to the relay device 30 via the transmission network 2, indicating a request to start relaying (step S46). The relay start request information includes the IP addresses "1.2.1.3" and "1.3.2.4" of transmission terminals 10aa and 10db, respectively.

[0140] As a result, the relay device 30 establishes a session between transmission terminals 10aa and 10db for transmitting three types of image data—low resolution, medium resolution, and high resolution—as well as audio data (step S47). In this way, transmission terminal 10aa becomes able to participate in a video conference with transmission terminal 10db.

[0141] On the other hand, the transmitting and receiving unit 51 of the transmission management device 50 casts terminal status change information, which includes the terminal ID "01aa" and the operating status "online (transmitting)" of the transmission terminal 10aa, to each transmission terminal 10 registered in the terminal management DB 5003 (see Figure 9(b)) (step S48). As a result, each transmission terminal that receives the terminal status change information can store and display the operating status of the transmission terminal 10aa with terminal ID "01aa" as needed.

[0142] Figure 13 is a sequence diagram showing the data transmission operation between transmission terminals 10. In the data transmission operation shown in Figure 13, image data and sound data are sent and received using the image / sound data session sed (see Figure 2).

[0143] As explained using Figure 12, once a session is established between transmission terminals 10aa and 10db, transmission terminal 10aa transmits the image data obtained by imaging unit 14 and the sound data converted from the sound signal input to sound input unit 15a to relay device 30 via transmission / reception unit 51 (step S60).

[0144] The relay device 30, having received image data and sound data from transmission terminal 10aa, transmits the received image data and sound data to each transmission terminal 10aa, 10db participating in the video conference (steps S61-1, 61-2).

[0145] Each transmission terminal 10aa, 10db that receives image data and sound data from the relay device 30 displays the image represented by the received image data on the display 120 using the display control unit 17, and outputs the sound represented by the received sound data to the speaker 115 using the sound output unit 15b (steps S62-1, S62-2).

[0146] Next, the transmission terminal 10db transmits the image data obtained by the imaging unit 14 and the sound data converted from the sound signal input to the sound input unit 15a to the relay device 30 via the transmitting / receiving unit 51 (step S63).

[0147] The relay device 30, having received image data and sound data from transmission terminal 10db, transmits the received image data and sound data to each transmission terminal 10aa, 10db participating in the video conference (steps S64-1, S64-2).

[0148] Each transmission terminal 10aa, 10db that receives image data and sound data from the relay device 30 displays the image represented by the received image data on the display 120 using the display control unit 17, and outputs the sound represented by the received sound data to the speaker 115 using the sound output unit 15b (steps S65-1, S65-2).

[0149] In this way, the transmission terminals 10aa and 10db can conduct video conferencing via the relay device 30. Note that in Figure 13, for the sake of clarity in understanding this embodiment, the transmission terminals 10 participating in the video conferencing are shown as two terminals 10aa and 10db. However, even if there are three or more transmission terminals 10 participating in the video conferencing, the image data and audio data will be transmitted in the same manner as the data transmission operation shown in Figure 13.

[0150] <<Two display modes of transmission terminal 10>> The image display of the transmission terminal 10 will now be described. In this embodiment, the display control unit 17 has two display modes: a full image display mode and an information display mode.

[0151] First, the full image display mode will be explained based on Figure 14. Figure 14 shows an example of the screen displayed on display 120 in full image display mode. The full image display mode is displayed when a session is established in step S47 of Figure 12 and the transmission or reception of image data and sound data begins. In other words, it is the display mode that is mainly used during video conferencing, out of the two display modes.

[0152] In full image display mode, the display control unit 17 of the transmission terminal 10 displays all images transmitted from each transmission terminal 10 participating in the video conference on the display 120. If displaying all images would reduce the size of a single image area, an image up to a predetermined upper limit (maximum value) will be displayed.

[0153] The display screen shown in Figure 14 is allocated a relatively large first image area 500 and a number of relatively small second image areas 510. For example, each second image area 510 displays an image represented by image data transmitted from each transmission terminal 10 participating in the video conference, while the first image area 500 displays an image represented by image data from the main transmission terminal among the transmission terminals 10 participating in the video conference.

[0154] For example, the display control unit 17 identifies the transmission terminal that sent the audio data representing speech among the transmission terminals 10 participating in the video conference as the primary terminal. Here, audio data representing speech refers to audio data in which an audio segment has been detected for each received audio data, and it has been determined that a voiced segment exists. Note that the detection of an audio segment can be achieved using known techniques (see, for example, Non-Patent Document 1).

[0155] Note that the display screen shown in Figure 14 is merely an example. In full image display mode, it is possible to display each image with all multiple image areas of the same size, or to display only the images of the main terminals mentioned above across the entire display 120.

[0156] <Information display mode> Next, the information display modes will be explained. Users can switch between the two display modes by, for example, operating the operation button 108. If the transmission terminal 10 is an information processing terminal 40, the two display modes can be switched between by pressing the hard keys or soft keys displayed on the touch panel of the information processing terminal 40 instead of the operation button 108.

[0157] Figure 15 shows an example of a screen displayed on the display 120 in full image display mode. As shown in Figure 15, in information display mode, the display control unit 17 of the transmission terminal 10 displays terminal information on the display 120, which includes at least the terminal name and terminal ID of each transmission terminal 10 participating in the video conference.

[0158] The display screen shown in Figure 15 is allocated an image display area 520 for displaying images and a terminal information display area 530 for displaying terminal information. For example, the terminal information display area 530 displays a list of terminal information for transmission terminals 10 participating in the video conference.

[0159] Note that terminals whose names are not displayed in the terminal information shown in the list represent transmission terminals 10 that are not registered as candidate destination terminals for the terminal in the destination list management DB5004 (see Figure 10).

[0160] For example, in the display screen shown in Figure 15, the transmission terminal 10db is registered in the destination list management DB 5004 as a candidate destination terminal for transmission terminal 10aa, so its terminal name is displayed. However, the transmission terminal 10cb is not registered in the destination list management DB 5004 as a candidate destination terminal for transmission terminal 10aa, so its terminal name is not displayed.

[0161] The image display area 520 displays an image representing the image data of the main transmission terminal among the transmission terminals 10 participating in the video conference. For example, the display control unit 17 identifies the transmission terminal that is the source of the audio data representing voice among the received audio data among the transmission terminals 10 participating in the video conference as the main transmission terminal. Alternatively, the display control unit 17 may identify the selected transmission terminal as the main transmission terminal when the operation input reception unit 12 receives confirmation that one transmission terminal has been selected from the transmission terminals 10 participating in the video conference in response to the operation of the operation button 108 by the user. If the transmission terminal 10 is an information processing terminal 40, the transmission terminal selected by pressing a hard key or a soft key displayed on the touch panel of the information processing terminal 40 is identified as the main transmission terminal instead of the operation button 108.

[0162] Figure 16 is a sequence diagram showing the terminal information display operation in which terminal information is displayed in the terminal information display area 530 when the transmission terminal 10 is switched to information display mode. In the terminal information display operation shown in Figure 16, various types of information are transmitted and received in the management information session sei.

[0163] First, the transmission terminal 10 sends a terminal information request signal to the transmission management device 50, which represents a request for terminal information from transmission terminals participating in the video conference (step S70).

[0164] Upon receiving the terminal information request signal, the terminal status acquisition unit 55 of the transmission management device 50 searches the terminal management DB 5003 (see Figure 9(b)) in the storage unit 5000 using the IP address of the source of the terminal information request signal as a search key, and obtains the terminal ID of the transmission terminal that sent the terminal information request signal (step S71).

[0165] Next, the terminal status acquisition unit 55 searches the terminal management DB 5003 to find terminal IDs and terminal names whose operating status is "online (transmitting)" (step S72). In this embodiment, the terminal IDs detected by the terminal status acquisition unit 55 are "01aa", "01cb", and "01db". Note that the operating status of the transmission terminal with terminal ID "01aa" is set to "online (transmitting)" in step S43 (see Figure 9(b)).

[0166] Next, the terminal status acquisition unit 55 uses the terminal ID detected in step S71 (in this embodiment, "01aa") as a search key to search the destination list management DB 5004 (see Figure 10) in the storage unit 5000 and obtains the terminal ID of a candidate destination terminal that is the source of the terminal information request signal (step S73).

[0167] For example, if the terminal ID of the transmission terminal that sent the terminal information request signal is "01aa", the terminal IDs acquired by the terminal status acquisition unit 55 will be "01ab", "01ba", and "01db".

[0168] Next, the terminal status acquisition unit 55 determines whether each terminal ID acquired in step S72 is included in the terminal ID of the sender of the terminal information request signal or the terminal ID acquired in step S73. If it determines that it is included, it sends the terminal ID and terminal name to the sender of the terminal information request signal via the transmit / receive unit 51. If it determines that it is not included, it sends only the terminal ID to the sender of the terminal information request signal via the transmit / receive unit 51 (step S74).

[0169] For example, if the terminal ID of the transmission terminal that sent the terminal information request signal is "01aa", then the transmission terminal aa, which is the source of the terminal information request signal, will receive the terminal IDs "01aa" and "01db" of transmission terminal aa and transmission terminal db, as well as the terminal names "Japan Tokyo Office AA Terminal" and "USA Washington Office DB Terminal", and the terminal ID "01cb" of transmission terminal cb.

[0170] Upon receiving this response, the display control unit 17 of the transmission terminal 10 displays the response content in the terminal information display area 530, as shown in the display screen in Figure 15 (step S75).

[0171] <Display of the number of locations, etc., in full image display mode> In the display screen in the full image display mode shown in Figure 14, when the number of transmission terminals 10 participating in the meeting is small, it is easy for video conference participants to see who else is participating. However, as the number increases, it gradually becomes more difficult to keep track of everyone. This will be explained using a diagram.

[0172] Figure 17(a) shows an example of the display screen in full image display mode when there are four locations. The display screen in Figure 17(a) has a large first image area 171 and three second image areas 172 that are narrower than the first image area 171. Similar to Figure 14, the second image areas 172 display images represented by image data transmitted from each transmission terminal 10 participating in the video conference, and the first image area 171 displays images represented by image data from the main transmission terminal among the transmission terminals 10 participating in the video conference.

[0173] If there are around four locations, it is not impossible for users to identify participants from one of the second image regions. However, as shown in Figure 17(b), as the number of locations increases, it becomes difficult for participants to identify other participants. Figure 17(b) shows an example of the display screen in full image display mode when there are nine locations.

[0174] Furthermore, for practical purposes, an upper limit should be set on the number of locations that the display 120 can simultaneously display (hereinafter referred to as the display limit). However, the number of locations that the transmission terminal 10 can use for video conferencing (referred to as the location limit) may not match the display limit. If the display limit is greater than or equal to the location limit, each transmission terminal 10 can display images from all locations on the display 120. However, if the display limit is less than the location limit, there will be locations that the transmission terminal 10 cannot display.

[0175] As shown in Figure 16, in some cases, the terminal name and terminal ID of all transmission terminals 10 at all locations can be displayed by the user operating the operation button 108 to switch to information display mode, but even in this case, the user still needs to perform some operation.

[0176] Furthermore, while in information display mode, only an image of one main location can be displayed, which is inconvenient because it prevents users from understanding the status of other locations.

[0177] Therefore, the transmission terminal 10 of this embodiment displays the number of locations participating in the video conference while remaining in full image display mode. This allows participants in the video conference to understand the number of locations in real time while remaining in full image display mode.

[0178] <<Display of the number of locations in full image display mode>> Figure 18 shows an example of a display screen showing the number of locations in the full image display mode. The auxiliary area display unit 18 displays the number of locations in the auxiliary area 173 of the full image display mode. In Figure 18, the auxiliary area 173 is a band-shaped margin at the top of the display screen, but it may also be a band-shaped margin at the right edge, left edge, or bottom, or it may be displayed overlapping with the image.

[0179] Auxiliary region 173 is divided into regions 1 ar1 to 6 ar6. Area 1 ar1... The terminal ID of the current terminal is displayed. Users of transmission terminal 10 who wish to participate in the same video conference as this transmission terminal 10 can join the video conference by entering or selecting this terminal ID on the transmission terminal 10 that is querying the system. Second area ar2... The number of bases is displayed. Domain 3, ar3... Displays the meeting time since the session was established. Area 4, ar4... Arbitrary information will be displayed. Area 5 (ar5) displays network bandwidth information (upload and download speeds). Area 6, ar6... The cursor buttons are displayed. The cursor button 1736 has an upper triangular button 1736u and a lower triangular button 1736d, but the upper triangular button 1736u is black and the lower triangular button 1736d is white. The user can only operate the upper triangular button 1736u and cannot operate the lower triangular button 1736d.

[0180] Furthermore, it is preferable that the auxiliary area 173 be displayed semi-transparently. This makes it easier to maintain visibility even when the auxiliary area 173 overlaps with image data transmitted and received between locations.

[0181] The number of locations can be displayed anywhere in areas 1 through 6, and if auxiliary area 173 is not divided, it can be displayed anywhere in auxiliary area 173. However, dividing auxiliary area 173 improves the visibility of the display screen from the user's perspective.

[0182] Here, the cursor button 1736 in the sixth area ar6 is an icon that informs the user that the transmission terminal 10 can switch to the simplified display auxiliary area 173 by operating the operation button 108. Hereafter, the auxiliary area in Figure 18 will be referred to as the "standard auxiliary area" to distinguish it.

[0183] In other words, the initial setting of the transmission terminal 10 is configured to display various transmission status information from the first area ar1 to the sixth area ar6 in Figure 18 in the auxiliary area 173. However, by operating the operation button 108 (or by operating the touch panel, etc.) and pressing the upper triangular button 1736u, the user can switch from the standard auxiliary area 173 to a simplified auxiliary area 173, or subsequently set the auxiliary area 173 not to be displayed. Details are explained in Figure 24.

[0184] Furthermore, if the transmission terminal 10 is an information processing terminal 40, it is possible to prevent the display of transmission status information for the first area ar1 to the sixth area ar6 in the auxiliary area 173 by pressing a hard key or a soft key displayed on the touch panel of the information processing terminal 40 instead of the operation button 108.

[0185] In Figure 18, images of four locations are displayed on display 120. Therefore, the number of locations in the second area is "4". Users can see at a glance how many locations are participating in the video conference.

[0186] Figure 19 shows another example of a display screen showing the number of locations in the full image display mode. Components with the same reference numerals as in Figure 18 perform similar functions, so only the differences will be explained.

[0187] In Figure 19, the display screen is divided into nine regions. The number of locations in the second region, ar2, is also "9". In this way, even as the number of locations increases, users can see at a glance how many locations are participating in the video conference.

[0188] For the sake of explanation, let's assume that the maximum number of images that can be displayed by a transmission terminal 10 is "9". When the maximum number of images that can be displayed by a transmission terminal 10 is "9", there may be cases where 10 or more transmission terminals 10 participate in the video conference. In this case, the number of images displayed by the transmission terminal 10 in the full image display mode remains "9", and the number of transmission terminals 10 that participated in the video conference is displayed as the number of locations in the second area ar2.

[0189] Figure 20 shows an example of a display screen where the number of images displayed in the full image display mode differs from the number of locations displayed in the second area ar2. In Figure 20, the number of images displayed in the full image display mode is "9," but the number of locations in the second area ar2 is "15." In this way, even when "the number of locations participating in the video conference > the display limit," users can grasp the number of locations participating in the video conference at a glance.

[0190] The layout (i.e., arrangement) of this auxiliary area 173 is defined in the layout information lot stored in the memory unit 2000 of the UI providing device 20. The layout information lot is described based on languages ​​and standards such as HTML, JavaScript (registered trademark), XML, and CSS.

[0191] The layout information lot specifies the position of the auxiliary area 173, how the auxiliary area 173 is divided into six areas of predetermined sizes (1st to 6th areas), and the information to be displayed in each of the 1st area ar1 to 6th area ar6. Therefore, the transmission terminal 10, which has acquired the layout information lot from the UI providing device 20, can set and display transmission status information in the 1st area ar1 to 6th area ar6.

[0192] As shown in Figure 17 and other figures, in the full image display mode, the arrangement of image data received from the destination terminal by the transmission terminal 10 and displayed on the display 120 is predetermined by the number of locations, etc. In contrast, the layout information lot is defined separately from the image data layout, so it is not affected by the image data layout and can define the transmission status information and arrangement to be placed in the auxiliary area 173. In other words, the transmission terminal 10 can display the first area ar1 to the sixth area ar6 in the auxiliary area independently of the image data, and the auxiliary area 173 is not affected even if the image data layout is changed.

[0193] <Simplified display auxiliary area 173> Figure 21 shows an example of the simplified display auxiliary area 173. In Figure 21, the number of locations is the same as in Figure 18, but the transmission status information for the first area ar1 to the sixth area ar6 is displayed in a simplified manner.

[0194] In the simplified display auxiliary area 173, information about what is displayed in each area is not shown. Primarily, only the contact ID value is displayed in the first area ar1, only the number of locations in the second area ar2, only the time in the third area ar3, and only the bandwidth in the fifth area ar5.

[0195] The cursor buttons 1736 in area 6 (ar6) have a different display format than the standard auxiliary area 173. In the simplified auxiliary area 173, both the upper triangle button 1736u and the lower triangle button 1736d are black. The user can operate either the upper or lower triangle button. The triangle button below: Switches to the standard auxiliary area 173. Top triangle button: Hides the auxiliary area 173 in simplified display mode (also hides the standard auxiliary area 173).

[0196] In the simplified auxiliary area 173, the area occupied by the display 120 is smaller than that of the standard auxiliary area 173, so that the visibility of image data can be improved while displaying transmission status information in the auxiliary area 173.

[0197] Furthermore, it is preferable that the auxiliary area 173 for simplified display is also displayed semi-transparently, similar to the standard auxiliary area 173.

[0198] <Display Instructions> Figure 22 is an example of a sequence diagram showing the procedure by which the transmission system 1 displays the number of locations when the transmission terminal 10 is in full image display mode. The procedure in Figure 22 is executed after the session is established in step S47 of Figure 12, without the user having to operate the operation buttons 108, etc. In other words, the transmission terminal 10 is initially set to display the auxiliary area 173.

[0199] Alternatively, after the session is established in step S47, the user may press the operation button 108 to perform the action. If the transmission terminal 10 is an information processing terminal 40, the action may be performed by pressing a hard key or a soft key displayed on the touch panel of the information processing terminal 40 instead of the operation button 108.

[0200] First, the transmission terminal 10 specifies the URI of the UI providing device 20 obtained in step S25 of Figure 11 and sends a layout information request signal (an example of a layout information request within the claims) to the UI providing device 20 to request layout information lot (S80). Note that the URI of the UI providing device 20 is not obtained in step S25 of Figure 11, but may also be obtained by the transmission terminal 10 from the transmission management device 50 immediately before step S80, or by the user pressing the operation button 108. Note that if the transmission terminal 10 is an information processing terminal 40, the URI of the UI providing device 20 is obtained by pressing a hard key or a soft key displayed on the touch panel of the information processing terminal 40 instead of the operation button 108.

[0201] The transmitting / receiving unit 21 of the UI providing device 20 receives a layout information request signal, which causes the server unit 22 to refer to the storage unit 2000 based on the specified URI and read out the layout information lot (S81).

[0202] Next, the server unit 22 transmits the read layout information lot to the transmission terminal 10 that sent the layout information request signal via the transmitting / receiving unit 21 (S82). This layout information lot includes layout information for both the standard and simplified display auxiliary areas 173.

[0203] The transmitting / receiving unit 11 of the transmission terminal 10 stores the layout information lot in volatile memory (S83). The volatile memory is, for example, RAM 103. That is, by storing the layout information lot once obtained in the transmission terminal 10, it is not necessary to obtain the layout information lot again from the UI providing device 20 while the power is ON. As a result, even if the transmission terminal 10 erases the auxiliary area 173 once and then displays the auxiliary area 173 again, the layout information lot can be read from the volatile memory of the transmission terminal 10. Therefore, the processing time required to obtain the layout information lot can be omitted, and the time until the auxiliary area 173 is displayed can also be shortened. In addition, since the layout information lot is erased when the power of the transmission terminal 10 is turned OFF by storing it in volatile memory, it is possible to prevent the layout information lot from leaking from the transmission terminal 10.

[0204] Next, the auxiliary area display unit 18 of the transmission terminal 10 transmits a terminal information request signal to the transmission management device 50 (S84). The terminal information request signal is a signal requesting the number of transmission terminals 10 participating in the video conference.

[0205] The transmitting / receiving unit 51 of the transmission management device 50 receives a terminal information request signal, and the terminal status acquisition unit 55 searches the terminal management DB 5003 and calculates the number of terminal IDs whose operating status is "online (transmitting)" (S85). That is, the terminal status acquisition unit 55 identifies the terminal management table (see Figure 9(b)) in which the IP address of the transmission terminal 10 that sent the terminal information request signal is registered, and identifies all terminal IDs whose operating status is "online (transmitting)" from the terminal management table and calculates the number. In this embodiment, the terminal IDs detected by the terminal status acquisition unit 55 are "01aa", "01cb", and "01db", so the number is "3". Note that the operating status of the transmission terminal 10aa with terminal ID "01aa" is set to "online (transmitting)" in step S43 of Figure 12.

[0206] In addition to the number of transmission terminals 10 that are "online (transmitting)", the number of transmission terminals 10 that are "online (transmittable)" may also be determined. This allows each transmission terminal 10 to display the number of logged-in transmission terminals 10 on the display 120. Alternatively, the number of "online (transmitting)" transmission terminals 10 and the number of "online (transmittable)" transmission terminals 10 may be determined separately, and each transmission terminal 10 may display them separately on the display 120. This allows, for example, to determine if there are users who are not participating in the meeting if the two numbers do not match. Next, the terminal status acquisition unit 55 transmits the number of "online (transmitting)" terminal IDs obtained above to the transmission terminal 10 that sent the terminal information request signal via the transmission / reception unit 51 (S86).

[0207] The terminal status acquisition unit 55 may also send the number of online (transmitting) terminal IDs to online transmission terminals 10 other than the transmission terminal 10 that sent the terminal information request signal. For example, if the number of online terminal IDs is sent to transmission terminal 10aa with terminal ID "01aa" in step S86, it will also send it to transmission terminals 10cb and 10db with terminal IDs "01cb" and "01db". As a result, transmission terminals 10cb and 10db with terminal IDs "01cb" and "01db" can display the number of locations in the auxiliary area 173 simply by receiving the layout information lot.

[0208] The transmitting / receiving unit 11 of the transmission terminal 10 receives the number of terminal IDs that are "online (transmitting)", and the auxiliary area display unit 18 displays it on the display 120 (S87). That is, it interprets the layout information lot to determine the position of the standard auxiliary area 173 and the layout of the first area ar1 to the sixth area ar6, sets the transmission status information in the first area ar1 to the sixth area ar6 and displays it on the display 120.

[0209] The terminal ID of the local terminal in the first area ar1 is stored in the memory unit 1000. The number of locations in the second area ar2 is transmitted from the transmission management device 50. The conference time since the session was established in the third area ar3 is measured by the transmission terminal 10 from the time the session was established. The arbitrary information in the fourth area ar4 is, for example, information held by the transmission terminal 10 or the transmission management device 50. The network bandwidth information in the fifth area ar5 is displayed by the transmission terminal 10 by periodically or in real time measuring the communication speed with the relay device 30. The cursor buttons 1736 in the sixth area ar6 are image data such as predetermined icons.

[0210] The displayed auxiliary area 173 can be erased by the user operating the operation button 108. It can also be displayed again by operating the operation button 108.

[0211] As described above, the transmission terminal 10 can display the number of locations participating in the video conference without switching to information display mode. Therefore, users do not need to operate the operation buttons 108, etc. Also, since the transmission terminal 10 remains in full image display mode, users can understand the number of locations while viewing the images of each location. Furthermore, even if the number of locations participating in the video conference exceeds the display limit, users can still accurately understand the number of locations.

[0212] <Display settings for the auxiliary area> The user can set the transmission terminal 10 to either "display according to operation" or "clear after a certain period of time" for the displayed auxiliary area 173.

[0213] Figure 23 shows an example of a screen in which the user configures the display settings for the auxiliary area 173. The screen in Figure 23 is displayed on the display 120 when the user operates, for example, the operation button 108. The screen in Figure 23 displays two radio buttons 221 and 222, and messages 223 that say "Display according to operation" and 224 that say "Clear after a certain period of time" which are associated with radio buttons 221 and 222. The user can select one of the two radio buttons 221 or 222.

[0214] If radio button 221 is selected, the transmission terminal 10 will continue to display the auxiliary area 173, but as shown in Figure 24, it will switch between displaying the standard auxiliary area 173 and the simplified auxiliary area 173, or not display either auxiliary area 173, according to the user's operation.

[0215] If radio button 222 is selected, the transmission terminal 10 displays the auxiliary area 173, then erases it after a certain period of time, and displays it again when a trigger for redisplay is detected. In this case as well, one of the triggers is that the user can display and erase it through operation.

[0216] When radio button 222 is selected, the message 225 "Please set a certain time" becomes visible, and it becomes possible to set a number in the time setting field 226. The user can set the time in the time setting field 226 until the auxiliary area 173 is cleared. A predetermined initial value is set until the user sets it. The user of the transmission terminal 10 can set any time in the time setting field 226. Note that upper and lower limits may be set for the time that can be set in the time setting field 226.

[0217] Figure 24 is an example of a flowchart illustrating the procedure for switching between the standard auxiliary area 173 and the simplified display auxiliary area 173 when "Display according to operation" is selected.

[0218] Figure 24 shows that in step S87 of Figure 22, the auxiliary area display unit 18 starts displaying the standard auxiliary area 173 (S87-1).

[0219] The auxiliary area display unit 18 displays the standard auxiliary area 173 and then determines whether a predetermined time has elapsed (S87-2). The predetermined time may be set in advance, or the time set in "Erase after a certain period of time" in Figure 23 may be used. In addition to the predetermined time, it may also be set by the user.

[0220] After a predetermined time has elapsed (Yes in S87-2), the auxiliary area display unit 18 switches the standard auxiliary area 173 to the simplified auxiliary area 173 (S87-3). The cursor buttons 1736 of the simplified auxiliary area 173 have two black triangular buttons 1736u and d, so the user can recognize that pressing either triangular button will cause a change on the screen.

[0221] The auxiliary area display unit 18 determines whether or not the lower triangular button 1736d has been operated (S87-4).

[0222] If the lower triangular button 1736d is pressed (Yes in S87-4), the auxiliary area display unit 18 switches the simplified auxiliary area 173 to the standard auxiliary area 173 (S87-7). In the standard auxiliary area 173, the lower triangular button 1736d is outlined in white, indicating that nothing happens when pressed, while the upper triangular button 1736u is black, indicating that pressing it will display the simplified auxiliary area 173. In addition, the standard auxiliary area 173 is displayed semi-transparently, allowing the meeting to continue without significantly reducing the visibility of the image data.

[0223] If the lower triangular button 1736d is not operated (No. in S87-4), the auxiliary area display unit 18 determines whether the upper triangular button 1736u has been operated (S87-5). Note that the determination of whether the lower triangular button 1736d or the upper triangular button 1736u has been operated can be made in any order.

[0224] If the upper triangular button 1736u is operated (Yes in S87-5), the auxiliary area display unit 18 does not display the auxiliary area (S87-6). That is, neither the standard auxiliary area nor the simplified auxiliary area is displayed. After this, the simplified auxiliary area may be displayed by operating the lower triangular button 1736d (i.e., return to step S87-3). Note that the upper triangular button 1736u may be designed to do nothing when operated, or it may display the standard auxiliary area when operated. It may also be possible to display only the cursor button 1736 and return to step S87-3 without displaying either the standard or simplified auxiliary area.

[0225] If the standard auxiliary area is displayed in step S87-7, the auxiliary area display unit 18 determines whether or not the upper triangular button 1736u has been operated (S87-8).

[0226] If the upper triangular button 1736u is displayed (Yes in S87-8), the auxiliary area display unit 18 switches the standard auxiliary area 173 to the simplified auxiliary area 173 (S87-3).

[0227] In this way, users can switch the display mode of the auxiliary area by operating the cursor buttons 1736.

[0228] <Operation procedure after displaying the auxiliary area> Next, the operation of the transmission management device 50 and the transmission terminal 10 after the transmission terminal 10 displays the auxiliary area 173 in step S87 of Figure 22 will be described.

[0229] Figure 25 is an example of a flowchart showing the operation procedure of the transmission management device 50. The process in Figure 25 is repeatedly executed, for example, after the transmission management device 50 calculates the number of online terminal IDs in step S85 of Figure 22.

[0230] The terminal status acquisition unit 55 monitors the terminal management table in which the IP address (or terminal ID) of the transmission terminal 10 that sent the terminal information request signal is registered (S85-1).

[0231] The terminal status acquisition unit 55 determines whether the number of terminal IDs that are "online (transmitting)" as transmitted in step S85 of Figure 22 has increased or decreased (S85-2).

[0232] If the result of step S85-2 is No, the process in Figure 25 ends, and the process is repeated from step S85-1.

[0233] If the determination in step S85-2 is Yes, the number of locations has changed, so the terminal status acquisition unit 55 sends the current number of terminal IDs whose operating status is "online (transmitting)" to the transmission terminal 10 (S85-3). As a result, the transmission terminal 10, having acquired the number of terminal IDs that are "online (transmitting)," can always obtain the correct number of locations.

[0234] In addition, in Figure 25, the terminal status acquisition unit 55 may also transmit the number of "online (transmitting)" terminal IDs to the "online (transmitting)" transmission terminals 10 other than the transmission terminal 10 that sent the terminal information request signal. This ensures that all online transmission terminals 10 always maintain the correct number of locations.

[0235] Figure 26 is an example of a flowchart showing the operation procedure of the transmission terminal 10. The process in Figure 26 is repeatedly executed, for example, after the transmission terminal 10 displays the auxiliary area 173 in step S87 of Figure 22. Figure 26 illustrates the case where the auxiliary area 173 is set to "clear after a certain period of time."

[0236] First, the auxiliary area display unit 18 determines whether or not the auxiliary area 173 is currently being displayed (S87-1).

[0237] If the determination in step S87-1 is Yes, the auxiliary area display unit 18 determines whether the elapsed time since the start of display in the auxiliary area 173 has exceeded a certain period of time (S87-2).

[0238] If the determination in step S87-2 is Yes, the auxiliary area display unit 18 erases the auxiliary area 173 (S87-3). This allows participants to participate in the video conference mainly by focusing on the images of each location.

[0239] If the determination in step S87-2 is No, the auxiliary area display unit 18 will continue to display the auxiliary area 173.

[0240] If the determination in step S87-1 is No, the auxiliary area display unit 18 determines whether there is an opportunity to display the auxiliary area 173. This opportunity arises when the number of "online (transmitting)" terminal IDs is received from the transmission management device 50 or when the user performs an operation to display.

[0241] First, the auxiliary area display unit 18 determines whether or not it has received the number of "online (transmitting)" terminal IDs from the transmission management device 50 (S87-4).

[0242] If the determination in step S87-4 is Yes, the auxiliary area display unit 18 displays the auxiliary area 173 again based on the layout information lot stored in RAM 103 (S87-5). This allows the transmission terminal 10 to immediately reflect the changed number of locations in the auxiliary area 173 if there is a change in the number of locations.

[0243] If the determination in step S87-4 is No, the auxiliary area display unit 18 determines whether or not the user has performed a display operation (S87-6).

[0244] If the determination in step S87-6 is Yes, the auxiliary area display unit 18 displays the auxiliary area 173 again based on the layout information lot stored in RAM 103 (S87-5). This allows the user to display the auxiliary area 173 at any time.

[0245] If the determination in steps S87-4 and S87-6 is No, the auxiliary area display unit 18 will not display the auxiliary area 173, leaving it erased.

[0246] Therefore, displaying this information when the number of locations changes makes it easier for users to notice the change. Also, since it disappears after a certain period of time, users can focus on the images of each location while participating in the video conference.

[0247] If a user wants to know more about participating locations after checking the number of locations or seeing changes in their count, they can switch to information display mode. Users can switch to information display mode when the number of locations changes, so there is no need to switch to information display mode just to check the number of locations or participating locations.

[0248] <<Different Variations>> Although the best mode for carrying out the present invention has been described above using examples, the present invention is not limited in any way to these examples, and various modifications and substitutions can be made without departing from the spirit of the present invention.

[0249] For example, the relay device 30, the transmission management device 50, and the program provision device 90 do not have to exist independently, but may be constructed by a single computer. Furthermore, the various means and storage units 1000, 2000, 3000, and 5000 of the relay device 30, the transmission management device 50, and the program provision device 90 may be arbitrarily assigned to multiple computers. In this case, the UI provision device 20 may be included as a function of any of the computers.

[0250] Furthermore, while the image quality of the image data relayed by the relay device 30 was managed by focusing on the image resolution of the image data, this is not the only example of quality management. Other examples of quality management may include focusing on the depth of image quality, the sampling frequency of the sound data, and the bit length of the sound data. Additionally, the sound data may be transmitted and received separately in three resolutions (high resolution, medium resolution, and low resolution).

[0251] Furthermore, while Figures 9(a) and 9(b) show that the IP address of the transmission terminal 10 is managed, this is not the only option. Any information that identifies the transmission terminal 10 on the transmission network 2 is acceptable. For example, the FQDN (Fully Qualified Domain Name) of the transmission terminal 10 could be managed. In this case, a well-known DNS (Domain Name System) server would obtain the IP address corresponding to the FQDN.

[0252] Furthermore, although the above embodiment described a video conferencing system as an example of the transmission system 1, it is not limited to this, and may also be a telephone system such as an IP (Internet Protocol) phone or an internet phone.

[0253] Furthermore, while the above-described transmission system 1 was explained as an example of a video conferencing system, it is not limited to this, and may also be a telephone system such as an IP (Internet Protocol) phone or an internet phone.

[0254] Furthermore, the transmission system 1 may also be a system that includes a car navigation device mounted on a mobile vehicle. Figure 27 shows an example of a system configuration when the transmission terminal 10 is applied to the car navigation device 200. Hereafter, when distinguishing the car navigation device 200, the code of the car navigation device 200 will be represented as 200-i (where i is a natural number).

[0255] In this case, one transmission terminal 10 corresponds to a car navigation system 200-2 installed in automobile 601. The other transmission terminal 10 corresponds to a management terminal 200-1 used by a communicator at the management center 604, or a car navigation system 200-3 installed in another automobile 602. Management terminal 200-1, car navigation system 200-2, and car navigation system 200-3 are connected to each other via network 2. Furthermore, management terminal 200-1, car navigation system 200-2, and car navigation system 200-3 establish a session with the transmission management device 50.

[0256] Figure 28 shows an example of the configuration of a car navigation system. The car navigation system 200 has a configuration in which a GPS receiver 612, a vehicle speed sensor 613, a gyro sensor 614, and road map data 615 are connected to the transmission terminal 10 described above. The transmission terminal 10 also has an integrated display 120. The display 120 may be connected to the transmission terminal 10 by wire or wireless, or it may be detachable from the transmission terminal 10.

[0257] The transmission terminal 10 executes a program and provides the functionality of a car navigation device 200. The GPS receiver 612 acquires GPS satellites and outputs the coordinates of the current location. The vehicle speed sensor 613 is a sensor that detects the speed (wheel rotation speed) of the automobiles 601 and 602. The gyro sensor 614 is a sensor that detects angular velocity, and the direction of travel of the vehicle can be detected by integrating the angular velocity.

[0258] The road map data 615 is data that represents roads as a combination of nodes and links, and may be downloaded from an external source or pre-stored in the vehicle. The road map data is displayed on the display 120.

[0259] Furthermore, a PC for screen sharing may be connected to the transmission terminal 10. In addition, the car navigation device 200 may have a tuner function for receiving radio and TV broadcasts, an audio function for playing music, a function for linking with in-vehicle cameras such as a rearview camera, a browser function for displaying websites, and the like, in addition to the configuration shown in Figure 28.

[0260] The transmission terminal 10 estimates its own position using autonomous navigation, starting from the location information detected by the GPS receiver 612, and accumulating the travel distance detected by the vehicle speed sensor 613 in the direction of travel detected by the gyro sensor 614. The vehicle's position is displayed on the road map using a vehicle marker or the like. It also searches for a route to the destination and guides the driver. In this way, the transmission terminal 10 realizes the function of a car navigation system.

[0261] The car navigation device 200 is connected to the network 2 by a transmission terminal 10. For example, it is connected to the network 2 via a mobile phone line (including wireless communication that covers a relatively wide area, mainly outdoors, using antenna power from a base station) using a mobile phone.

[0262] The transmission terminal 10, acting as a car navigation device 200-2, displays a screen on the display 120 as shown in Figure 29. The destination list is a display screen that includes the terminal ID 1100-2, terminal name 1100-3, and icons 1100-4a to 1100-4c that reflect status information of candidate destination terminals in the destination list frame 1100-1. The occupant selects a destination terminal from the list of destination terminals displayed on the display 120.

[0263] Once a session is established with the destination terminal, the transmission terminal 10 (or car navigation device) can send image data, audio data, etc., to the selected destination terminal, or receive them from the selected destination terminal.

[0264] Also, by the user operating the car navigation device 200 or the transmission terminal 10, the transmission terminal 10 can capture display data such as road maps and TV images. The transmission terminal 10 can transmit this display data to other transmission terminals 10 connected to the network 2.

[0265] Also, when the user selects the management terminal 200-1 used by the communicator of the management center 604 from the list of destination terminals displayed on the display 120, the image data of the communicator is displayed on the display 120 and voice is output from the speaker. Therefore, the user can talk to the communicator like a TV phone.

[0266] Therefore, just by the mounting location being a mobile body, image data, voice data, and display data can be transmitted and received in the same way as the transmission terminal 10 in the office.

[0267] Also, in the above embodiment, image data and audio data have been described as an example of content data, but it is not limited to this, and tactile (touch) data may be used. In this case, the feeling that the user touches on one transmission terminal side is transmitted to the other transmission terminal side.

[0268] Furthermore, the content data may be olfactory (smell) data. In this case, the smell on one transmission terminal side is transmitted to the other transmission terminal side. Also, the content data may be at least one of image data, audio data, tactile data, and olfactory data.

[0269] Also, in the above embodiment, the case of holding a video conference by the transmission system 1 has been described, but it is not limited to this, and it may be used for meetings, general conversations between family members and friends, or one-way information presentation.

Explanation of Reference Numerals

[0270] 1 Transmission system 10 Transmission terminal 11 Transmission and reception unit 12 Operation Input Reception Unit 15a Audio input section 15b Audio Input Section 17 Display Control Unit 18 Auxiliary area display section 19 Memory and Retrieval Processing Unit 30 Relay device 50 Transmission Management Device 51 Transmitter / Receiver 52 Terminal Authentication Section 53 Status Management Department 54 Terminal extraction unit 55 Terminal status acquisition unit 59 Memory and Retrieval Processing Unit 120 displays (display devices) [Prior art documents] [Patent Documents]

[0271] [Patent Document 1] Japanese Patent Publication No. 2013-157995 [Non-patent literature]

[0272] [Non-Patent Document 1] Niimi, "Speech Recognition," Kyoritsu Shuppan, pp. 68-72, 1979.

Claims

1. A transmission terminal that connects via a network to a transmission management device that manages multiple transmission terminals, and transmits data to at least one transmission terminal, A transmission means that transmits a terminal information request, which represents a request to obtain the number of transmission terminals currently transmitting, to the transmission management device, Display means for displaying image data received from the transmission terminal that is currently transmitting on a display device, and for displaying the number of transmission terminals that are currently transmitting, received from the transmission management device in response to the terminal information request, on the display device. A transmission terminal characterized by having the following features.

2. The transmission terminal according to claim 1, characterized in that the number of image data displayed on the display device is within the maximum number of image data that can be displayed on the display device.

3. The transmission terminal according to claim 2, characterized in that, when the number of transmission terminals currently transmitting is greater than the maximum value, the display means displays the image data in a number within the maximum value and also displays the number of transmission terminals currently transmitting on the display device.

4. The transmission terminal according to any one of claims 1 to 3, characterized in that the display means displays the number of transmission terminals that are currently transmitting on the display device based on predetermined arrangement information.

5. The transmission means transmits a placement information request, which represents the request for the placement information, to devices other than the plurality of transmission terminals connected to the network. The transmission terminal according to claim 4, characterized in that the display means displays the number of transmission terminals on the display device in accordance with the arrangement information received in response to the arrangement information request.

6. The transmission management device has location information acquisition means that acquires location information indicating the location where the placement information is stored in the network, The transmitting means transmits a placement information request that represents a request for the placement information stored at the location indicated by the location information. The transmission terminal according to feature 4.

7. It has a storage means for storing the arrangement information received in response to an arrangement information request, The transmission terminal according to claim 4, characterized in that, after the number of transmission terminals currently transmitting transitions from being displayed on the display device to being not displayed, when the number of transmission terminals currently transmitting is to be displayed on the display device, the display means displays the number of transmission terminals currently transmitting on the display device according to the arrangement information read from the storage means.

8. The transmission terminal according to claim 7, characterized in that the arrangement information stored in the storage means is erased when the power of the transmission terminal is turned off.

9. The display means, after a predetermined time has elapsed since the display device displayed the number of transmission terminals that are currently transmitting, will not display the number of transmission terminals that are currently transmitting on the display device. The transmission terminal according to feature 1.

10. The transmission terminal according to claim 9, characterized in that, when the number of transmission terminals currently transmitting is received from the transmission management device while the display device is not displaying the number of transmission terminals currently transmitting, the display means causes the number of transmission terminals currently transmitting received from the transmission management device to be displayed on the display device.

11. It has an operation reception means for receiving operations, When the operation receiving means receives an operation to display the number of transmission terminals that are currently transmitting, the display means displays the number of transmission terminals that are currently transmitting on the display device. The transmission terminal according to feature 1.

12. If the operation receiving means receives an operation that does not display the number of transmission terminals currently transmitting, the display means shall not display the number of transmission terminals currently transmitting on the display device. The transmission terminal according to claim 11, characterized in that it is as described above.

13. The display means displays information about the transmission status, including the number of transmission terminals that are currently transmitting. When the display means is displaying the transmission status information occupying a first area, and the operation receiving means receives a predetermined operation, the display means displays the transmission status information occupying a second area different from the first area. The transmission terminal according to claim 12, characterized in that it is a transmission terminal according to the features described above.

14. The display means displays the identification information of the transmission terminal on the display device. The transmission terminal according to feature 1.

15. The aforementioned identification information is information that other transmission terminals accept as input in order for those other transmission terminals to participate in the same conference as the aforementioned transmission terminal. The transmission terminal according to claim 14.

16. A transmission management device that manages multiple transmission terminals is connected via a network to an information processing device that transmits data to at least one transmission terminal. A transmission step of sending a terminal information request, which represents a request to obtain the number of transmission terminals currently transmitting, to the transmission management device, A program that causes a program to perform a display step of displaying image data received from a transmission terminal that is currently transmitting on a display device, and displaying the number of transmission terminals that are currently transmitting, received from the transmission management device in response to a terminal information request, on the display device.

17. A transmission method that connects to a transmission management device that manages multiple transmission terminals via a network and performs transmission with at least one transmission terminal, The transmission means sends a terminal information request to the transmission management device, which represents a request to obtain the number of transmission terminals and the number of transmission terminals currently transmitting. The display means includes the steps of displaying image data received from the transmission terminal that is currently transmitting on the display device, and displaying the number of transmission terminals that are currently transmitting, received from the transmission management device in response to the terminal information request, on the display device, A transmission method characterized by having the following features.

18. In a transmission system in which a transmission management device that manages multiple transmission terminals and at least one transmission terminal and a transmission terminal that performs transmission can communicate via a network, The aforementioned transmission management device is A calculation means for calculating the number of transmission terminals that are currently transmitting data, The system includes a transmission means that transmits the number of transmission terminals currently transmitting, calculated by the calculation means, to the transmission terminals, The aforementioned transmission terminal is A transmission means that transmits a terminal information request, which represents a request to obtain the number of transmission terminals currently transmitting, to the transmission management device, A transmission system characterized by having a display means that displays image data received from a transmission terminal that is currently transmitting on a display device, and displays the number of transmission terminals that are currently transmitting, received from the transmission management device in response to a terminal information request, on the display device.

Images