Transmission terminal, program, and transmission method
Patent Information
- Application Number
- JP2024111418
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2014-03-31
- Filing Date
- 2024-07-11
- Publication Date
- 2024-11-06
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing video conferencing systems struggle to effectively display participant information when the number of participating terminals increases, leading to difficulties in identifying other participants due to limited screen space and layout constraints.
A transmission terminal equipped with a network connection to a management device that allows for the display of terminal information alongside image data, including the number of participating terminals, even in modes primarily dedicated to image display.
Enables participants to confirm information on other terminals in real-time, maintaining visibility of images while providing a clear overview of the number of participating locations, even when the number exceeds display limits.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a transmission terminal etc. that is connected via a network to a transmission management device that manages a plurality of transmission terminals and performs transmission with at least one transmission terminal. [Background technology]
[0002] A video conference system collects images and sounds by photographing participants in a video conference and collecting their remarks using a transmission terminal, and then transmits the collected images and sounds to a destination transmission terminal. The destination transmission terminal displays images on a display or projector connected to the transmission terminal, and outputs sounds from a speaker. This allows each remote participant to hold a conference in a state close to an actual conference.
[0003] In such a remote teleconferencing system, the number of transmission terminals participating in the conference is not limited to two, and a multi-site video conference can be realized by all transmission terminals participating in the conference.
[0004] Generally, in a video conference held among multiple locations, a participant using a certain transmission terminal wants to know who is participating in the conference at a remote location. Therefore, each transmission terminal divides the image area of the display of the transmission terminal into the number of transmission terminals participating in the video conference, and displays the images of the participants transmitted from each transmission terminal in each image area.
[0005] However, when the number of transmission terminals participating in a video conference increases, the image area becomes smaller, making it difficult for each participant to grasp the other participants. Also, when there are many participating locations, layout limitations may make it impossible to display all participants.
[0006] To address such inconveniences, a transmission terminal is known that has a screen mode for displaying information about transmission terminals participating in a video conference, in addition to a screen mode for displaying images transmitted from other transmission terminals on a display (see, for example, Patent Document 1). Summary of the Invention [Problem to be solved by the invention]
[0007] However, in the transmitting terminal described in Patent Document 1, it is necessary to switch between a screen mode for displaying images transmitted from other transmitting terminals on a display and a screen mode for displaying information of transmitting terminals participating in a video conference. For this reason, for example, in a state in which a screen mode for displaying images transmitted from other transmitting terminals is selected, there is a problem that information of the transmitting terminal that can be grasped in a screen mode for displaying information of the transmitting terminal participating in a video conference cannot be confirmed.
[0008] In view of the above-mentioned problems, an object of the present invention is to provide a transmission terminal that allows a user to check information about other locations even in a screen mode that displays images. [Means for solving the problem]
[0009] In view of the above problems, the present invention is characterized in that a transmission terminal is connected via a network to a transmission management device that manages a plurality of transmission terminals and transmits with at least one transmission terminal, the transmission terminal having a transmission means for transmitting a terminal information request to the transmission management device, the terminal information request representing a request to obtain the number of transmission terminals currently transmitting, and a display means for displaying image data received from the transmission terminal currently transmitting on a display device, and displaying on the display device the number of the transmission terminals currently transmitting received from the transmission management device in response to the terminal information request. Effect of the Invention
[0010] It is possible to provide a transmission terminal that allows the user to check information on other locations even in a screen mode that displays an image of the transmission terminal. [Brief description of the drawings]
[0011] [Figure 1] 1 is a schematic diagram of a transmission system 1 according to an embodiment of the present invention; [Diagram 2] 1 is a conceptual diagram showing an example of a state of transmission and reception of image data, audio data, and various management information in a transmission system; [Diagram 3] 1 is an example of a conceptual diagram illustrating image quality of image data. [Figure 4] FIG. 2 is an example of an external view of a transmission terminal. [Diagram 5] FIG. 2 is a diagram illustrating an example of a hardware configuration of a transmission terminal. [Figure 6] 2 is a hardware configuration diagram of a transmission management device, a UI providing device, a relay device, or a program providing device. [Figure 7] FIG. 2 is an example of a functional block diagram of each terminal, device, and system constituting the transmission system. [Figure 8] 1 is a conceptual diagram of an example of a terminal authentication management table managed by a transmission management device; [Figure 9] 1 is a conceptual diagram of an example of a terminal management table managed by a transmission management device; [Figure 10] 1 is a conceptual diagram of an example of a destination list management table managed by a transmission management device; [Figure 11] 11 is a sequence diagram illustrating an example of a preparation stage operation until a transmission terminal is ready to participate in a video conference; [Figure 12] 11 is a sequence diagram illustrating an example of a conference participation operation until a transmission terminal enters a participating state in a video conference. [Figure 13] FIG. 11 is an example of a sequence diagram showing a data transmission operation between transmission terminals. [Figure 14] FIG. 13 is a diagram showing an example of a screen displayed on a display in a full image display mode. [Figure 15] FIG. 13 is a diagram showing an example of a screen displayed on a display in a full image display mode. [Figure 16]11 is a sequence diagram showing a terminal information display operation for displaying terminal information in a terminal information display area when the transmission terminal is switched to an information display mode. FIG. [Figure 17] FIG. 13 is a diagram showing an example of a display screen in the full image display mode when there are four locations. [Figure 18] FIG. 13 is a diagram showing an example of a display screen on which the number of locations is displayed in a full image display mode. [Figure 19] FIG. 13 is a diagram showing another example of a display screen on which the number of locations is displayed in the full image display mode. [Figure 20] FIG. 13 is a diagram showing an example of a display screen in which the number of images displayed in the all image display mode is different from the number of locations displayed in the second area ar2. [Figure 21] FIG. 13 is a diagram showing an example of an auxiliary area in a simplified display mode. [Figure 22] 11 is a sequence diagram illustrating an example of a procedure in which a transmission system displays the number of locations when a transmission terminal is in a full image display mode. [Figure 23] FIG. 13 is a diagram showing an example of a screen on which a user makes display settings for the auxiliary area. [Figure 24] 11 is a flowchart illustrating an example of an operation procedure in which the transmission terminal switches between a standard auxiliary area and a simplified display auxiliary area when "display according to operation" is selected. [Diagram 25] FIG. 11 is a flowchart illustrating an example of an operation procedure of the transmission management device. [Figure 26] FIG. 11 is a flowchart illustrating an example of an operation procedure of the transmission terminal. [Figure 27] FIG. 1 is a diagram illustrating an example of a system configuration diagram when a transmission terminal is employed as a car navigation device. [Figure 28] FIG. 1 is a diagram illustrating an example of a configuration of a car navigation device mounted on a vehicle. [Figure 29] FIG. 11 is a diagram showing an example of a destination list displayed on a car navigation device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
[0013] <<Overall configuration of the embodiment>> Fig. 1 is a schematic diagram of a transmission system 1 according to an embodiment of the present invention. Fig. 2 is a conceptual diagram showing the state of transmission and reception of image data, sound data, and various management information in the transmission system. Fig. 3 is a conceptual diagram explaining the image quality of image data.
[0014] The transmission system 1 also includes a data providing system that transmits content data in one direction from one transmission terminal to multiple transmission terminals via a transmission management device 50, and a communication system that transmits information, emotions, etc. between multiple transmission terminals via the transmission management device 50. A communication system is a system for transmitting information, emotions, etc. between multiple communication terminals (corresponding to "transmission terminals") via a communication management device (corresponding to "transmission management device"), and examples of such a system include a video conference system and a video telephone system.
[0015] In this embodiment, a transmission system 1, a transmission management device 50, and a transmission terminal 10 will be described assuming a video conference system as an example of a communication system, a video conference management device as an example of a communication management device, and a video conference terminal as an example of a communication terminal. That is, the transmission terminal and the transmission management device 50 of the present invention are applied not only to a video conference system, but also to a communication system or a data transmission system.
[0016] 1, the transmission system 1 includes a plurality of transmission terminals 10aa, 10ab, ..., displays 120aa, 120ab, ... for each of the transmission terminals 10aa, 10ab, ..., a UI providing device 20, an information processing terminal 40, a relay device 30, a transmission management device 50, and a program providing device 90. The transmission terminals 10aa, 10ab, ... are configured to transmit content data, for example, by sending and receiving image data and sound data.
[0017] The UI providing device 20 is a device that transmits layout information (to be described later) for mainly displaying the number of locations to the transmission terminals 10aa, 10ab, .... The UI providing device 20 may be, for example, the transmission management device 50 or the relay device 30.
[0018] The information processing terminal 40 is a device in which the functions of the transmission terminal 10 are realized by software. Specifically, the information processing terminal 40 is, for example, a personal computer (PC), a tablet terminal, a smartphone, a mobile phone, a personal digital assistant (PDA), or an electronic book terminal.
[0019] In the following description, any one of the multiple transmission terminals 10aa, 10ab, ... is represented as a "transmission terminal 10," and any one of the multiple displays 120aa, 120ab, ... is represented as a "display 120." The transmission terminal 10 that requests the start of a video conference is represented as a "request terminal," and the transmission terminal 10 that is the destination of the request is represented as a "destination terminal."
[0020] In addition, the transmitting terminals 10 participating in the video conference or their locations are referred to as “bases.” The number of bases is the same as the number of transmitting terminals 10 participating in the video conference, but multiple transmitting terminals 10 may exist in a space that can be regarded as one (for example, one conference room).
[0021] As shown in FIG. 2, in the transmission system 1, a management information session sei for transmitting and receiving various types of management information is established between a request source terminal and a destination terminal via a transmission management device 50.
[0022] Furthermore, between the transmission terminals 10 where the management information session sei has been established, four sessions for transmitting and receiving each of four types of data, high-resolution image data, medium-resolution image data, low-resolution image data, and sound data, are established via the relay device 30. In this embodiment, these four sessions are collectively referred to as the "image and sound data session sed."
[0023] Here, the image data handled in this embodiment is scalably encoded. For example, the image data handled in this embodiment includes low-resolution image data, medium-resolution image data, and high-resolution image data. As shown in Fig. 3(a), the low-resolution image data has a resolution of 320 pixels horizontally and 180 pixels vertically, as shown in Fig. 3(b), the medium-resolution image data has a resolution of 640 pixels horizontally and 360 pixels vertically, and the high-resolution image data has a resolution of 1280 pixels horizontally and 720 pixels vertically.
[0024] When the bandwidth through which the image data is transmitted is narrow, only low-resolution image data serving as the base image is transmitted, and the receiving transmission terminal 10 can display only the low-quality image based on the transmitted low-resolution image data.
[0025] In addition, when the bandwidth through which the image data is transmitted is relatively wide, low-resolution image data and medium-resolution image data that serve as the base image are transmitted, and the receiving transmission terminal 10 can display an image of medium image quality based on the transmitted low-resolution image data and medium-resolution image data.
[0026] In addition, when the bandwidth through which the image data is transmitted is very wide, low-resolution image data serving as the base image quality, medium-resolution image data, and high-resolution image data are transmitted, and the receiving transmission terminal 10 can display a high-image-quality image based on the transmitted low-resolution image data, medium-resolution image data, and high-resolution image data.
[0027] 1, the relay device 30 relays content data between a plurality of transmission terminals 10. The transmission management device 50 centrally manages login authentication from the transmission terminal 10, management of the transmission state of the transmission terminal 10, management of the destination list, and the transmission status of the relay device 30. Note that the image represented by the image data may be a moving image or a still image, or may be both a moving image and a still image.
[0028] The routers 70a, 70b, 70c, 70d, 70ab, and 70cd select the optimal route for image data and audio data. In the following description, any one of the routers 70a, 70b, 70c, 70d, 70ab, and 70cd will be referred to as a "router 70."
[0029] The program providing device 90 is equipped with an HD (Hard Disk) 204 described later, in which a transmission terminal control program 1000a for realizing various functions in the transmission terminal 10 is stored, and the program providing device 90 is capable of transmitting the transmission terminal control program 1000a to the transmission terminal 10.
[0030] Furthermore, HD 204 of the program providing device 90 also stores a relay device program for causing the relay device 30 to realize various functions, and can transmit the relay device program to the relay device 30.
[0031] Furthermore, the HD 204 of the program providing device 90 also stores a transmission management program for causing the transmission management device 50 to realize various functions, and can transmit the transmission management program to the transmission management device 50.
[0032] Furthermore, the HD 204 of the program providing device 90 also stores a program for the UI providing device for causing the UI providing device 20 to realize various functions, and can transmit the program for the UI providing device to the UI providing device 20 .
[0033] Furthermore, the HD 204 of the program providing device 90 also stores information processing terminal programs for implementing various functions in the information processing terminal 40 , and can transmit the information processing terminal programs to the information processing terminal 40 .
[0034] The transmission terminals 10aa, 10ab, 10ac, ... and the router 70a are connected to each other via a LAN 2a so as to be able to transmit data. The transmission terminals 10ba, 10bb, 10bc, ... and the router 70b are connected to each other via a LAN 2b so as to be able to transmit data.
[0035] Moreover, LAN2a and LAN2b are connected to each other so as to enable communication by a dedicated line 2ab including a router 70ab, and are provided within a predetermined area A. For example, area A is Japan, LAN2a is provided within an office in Tokyo, and LAN2b is provided within an office in Osaka.
[0036] On the other hand, the transmission terminals 10ca, 10cb, 10cc, . . . and the router 70c are connected to each other via a LAN 2c so that they can transmit data. The transmission terminals 10da, 10db, 10dc, . . . and the router 70d are connected to each other via a LAN 2d so that they can transmit data.
[0037] Furthermore, LAN2c and LAN2d are connected to each other so that communication can be performed by a dedicated line 2cd including a router 70cd, and are provided in a predetermined area B. For example, area B is the United States of America, with LAN2c provided in an office in New York and LAN2d provided in an office in Washington, D.C. Areas A and B are connected to each other so that communication can be performed by routers 70ab and 70cd, respectively, via the Internet 2i.
[0038] The relay device 30, the transmission management device 50, the UI providing device 20, the information processing terminal 40, and the program providing device 90 are connected to each transmission terminal 10 via the Internet 2i in a manner enabling communication therebetween. Here, the relay device 30, the transmission management device 50, the UI providing device 20, the information processing terminal 40, and the program providing device 90 may be installed in region A or region B, or may be installed in other regions.
[0039] In this embodiment, the transmission network 2 of this embodiment is configured by the LAN 2a, the LAN 2b, the dedicated line 2ab, the Internet 2i, the dedicated line 2cd, the LAN 2c, and the LAN 2d. The transmission network 2 may include a portion where not only wired transmission but also wireless transmission such as WiFi (Wireless Fidelity) or Bluetooth (registered trademark) is performed.
[0040] 1, four sets of numbers shown under each transmission terminal 10, relay device 30, transmission management device 50, each router 70, UI providing device 20, information processing terminal 40, and program providing device 90 simply indicate general IP addresses in IPv4. For example, the IP address of the transmission terminal 10aa is "1.2.1.3." Although IPv6 may be used instead of IPv4, in order to simplify the explanation, IPv4 will be used in the explanation.
[0041] Each transmission terminal 10 may be used not only for video conferences among a plurality of offices or between different rooms in the same office, but also for video conferences in the same room, or for video conferences between indoors and outdoors or between outdoors and outdoors. When each transmission terminal 10 is used outdoors, wireless transmission is performed using a mobile phone transmission network or the like.
[0042] <<Hardware configuration of the embodiment>> 4 is an external view of the transmission terminal 10. In the following description, the longitudinal direction of the transmission terminal 10 is defined as the X-axis direction, the direction perpendicular to the X-axis direction in a horizontal plane is defined as the Y-axis direction, and the direction perpendicular to the X-axis and Y-axis directions (vertical direction) is defined as the Z-axis direction.
[0043] As shown in Fig. 4, the transmission terminal 10 includes a housing 1100, an arm 1200, and a camera housing 1300. An intake surface (not shown) formed by a plurality of intake holes is provided on a front wall surface 1110 of the housing 1100, and an exhaust surface 1121 formed with a plurality of exhaust holes is provided on a rear wall surface 1120 of the housing 1100. As a result, by driving a cooling fan built into the housing 1100, it is possible to take in outside air behind the transmission terminal 10 through the intake surface (not shown) and exhaust the air to the rear of the transmission terminal 10 through the exhaust surface 1121. A sound pickup hole 1131 is formed on a right wall surface 1130 of the housing 1100, and sounds such as voice, noise, and the like can be picked up by a built-in microphone 114 described later.
[0044] An operation panel 1150 is formed on the right side wall 1130 of the housing 1100. This operation panel 1150 is provided with a plurality of operation buttons (108a to 108e), a power switch 109, and an alarm lamp 119, which will be described later, and also has a sound output surface 1151 formed with a plurality of sound output holes for passing sound output from a built-in speaker 115, which will be described later. In addition, a storage section 1160 is formed on the left side wall 1140 of the housing 1100 as a recess for storing the arm 1200 and the camera housing 1300. A plurality of connection ports (1132a to 1132c) for electrically connecting cables to an external device connection I / F 118, which will be described later, are provided on the right side wall 1130 of the housing 1100. Meanwhile, a left wall surface 1140 of the housing 1100 is provided with a connection port (not shown) for electrically connecting a cable 120c for the display 120 to an external device connection I / F 118 (described later).
[0045] In the following, "operation button 108" will be used to refer to any of the operation buttons (108a to 108e), and "connection port 1132" will be used to refer to any of the connection ports (1132a to 1132c).
[0046] Next, arm 1200 is attached to housing 1100 via torque hinge 1210, and configured so that arm 1200 can rotate up and down within a range of tilt angle θ1 of 135 degrees relative to housing 1100. Fig. 4 shows a state where tilt angle θ1 is 90 degrees.
[0047] The camera housing 1300 is provided with a built-in camera 112, which will be described later, and can capture images of a user, a document, a room, and the like. A torque hinge 1310 is also formed in the camera housing 1300. The camera housing 1300 is attached to the arm 1200 via the torque hinge 1310. The camera housing 1300 is attached to the arm 1200 via the torque hinge 1310, and is configured to be rotatable in the up, down, left, and right directions with respect to the arm 1200 within a pan angle θ2 range of ±180 degrees, with the state shown in FIG. 4 being 0 degrees, and within a tilt angle θ3 range of ±45 degrees.
[0048] The UI providing device 20, the relay device 30, the transmission management device 50, the program providing device 90, and the maintenance system 100 each have the same appearance as a general server computer, and therefore the description of the appearance is omitted. The information processing terminal 40 has the same appearance as a general PC, smartphone, tablet terminal, etc., and therefore the description of the appearance is omitted.
[0049] Fig. 5 is an example of a hardware configuration diagram of a transmission terminal. As shown in Fig. 5, the transmission terminal 10 includes a CPU (Central Processing Unit) 101 that controls the operation of the entire transmission terminal 10, a ROM (Read Only Memory) 102 that stores programs used to drive the CPU 101, such as an IPL (Initial Program Loader), a RAM (Random Access Memory) 103 that is used as a work area for the CPU 101, a transmission terminal control program 1000a, a flash memory 104 that stores various data such as image data and sound data, a solid state drive (SSD) 105 that controls reading or writing of various data from the flash memory 104 under the control of the CPU 101, a media drive 107 that controls reading or writing (storing) of data from a recording medium 106 such as a flash memory, an operation button 108 that is operated when selecting a destination of the transmission terminal 10, a power switch 109 that switches the power of the transmission terminal 10 on and off, and a network I / F (Interface) 111 that transmits data using the transmission network 2.
[0050] The transmission terminal 10 also includes a built-in camera 112 that captures an image of a subject under the control of the CPU 101 to obtain image data, an image sensor I / F 113 that controls the operation of the camera 112, a microphone 114 as a built-in sound collecting device for inputting sound, a built-in speaker 115 that outputs sound, an audio input / output I / F 116 that processes input and output of sound signals between the microphone 114 and the speaker 115 under the control of the CPU 101, a display I / F 117 that transmits image data to an external display 120 under the control of the CPU 101, an external device connection I / F 118 for connecting various external devices, and bus lines 110 such as an address bus and a data bus for electrically connecting the above-mentioned components.
[0051] The display 120 constitutes a display device constituted by a liquid crystal or an organic electroluminescence (EL) displaying an image of a subject, operation icons, etc. The display 120 is connected to the display I / F 117 by a cable 120c. The cable 120c may be a cable for analog RGB (VGA) signals, a cable for component video, or a cable for HDMI (registered trademark) (High-Definition Multimedia Interface) or DVI (Digital Video Interactive) signals.
[0052] The camera 112 includes a lens and a solid-state image sensor that converts light into electric charges to digitize an image of a subject. As the solid-state image sensor, a complementary metal oxide semiconductor (CMOS) or a charge coupled device (CCD) is used.
[0053] The external device connection I / F 118 allows external devices such as an external camera, an external microphone, and an external speaker to be connected via a USB (Universal Serial Bus) cable or the like.
[0054] When an external camera is connected to the external device connection I / F 118, the CPU 101 drives the external camera in priority over the built-in camera 112. When an external microphone or an external speaker is connected to the external device connection I / F 118, the CPU 101 drives the external microphone or the external speaker in priority over the built-in microphone 114 or the built-in speaker 115, respectively.
[0055] The recording medium 106 is configured to be detachable from the transmission terminal 10. The transmission terminal 10 may use an EEPROM (Electrically Erasable and Programmable ROM) or the like instead of the flash memory 104, as long as the EEPROM is a non-volatile memory that reads or writes data under the control of the CPU 101.
[0056] The transmission terminal control program 1000a may be distributed by being recorded in a computer-readable recording medium such as the recording medium 106 as a file in an installable or executable format. The transmission terminal control program 1000a may be stored in the ROM 102 instead of the flash memory 104.
[0057] The information processing terminal 40 has the same hardware configuration as the transmission terminal 10, and therefore the description thereof will be omitted. However, the SSD 105 stores an information processing terminal program for controlling the information processing terminal 40.
[0058] 6 is a hardware configuration diagram of a transmission management device, a UI providing device, a relay device, or a program providing device. As shown in FIG. 6, the transmission management device 50 includes a CPU 201 for controlling the overall operation of the transmission management device 50, a ROM 202 storing a program used to drive the CPU 201, such as an IPL, a RAM 203 used as a work area for the CPU 201, a HD 204 for storing various data such as a transmission management program, a Hard Disk Drive (HDD) 205 for controlling reading or writing of various data from the HD 204 under the control of the CPU 201, a media drive 207 for controlling reading or writing (storing) of data from a recording medium 206, such as a flash memory, a display 208 for displaying various information such as a cursor, a menu, a window, a character, or an image, a network I / F 209 for transmitting data using the transmission network 2, a keyboard 211 having a plurality of keys for inputting characters, numerical values, various instructions, etc., a mouse 212 for selecting and executing various instructions, selecting a processing target, moving the cursor, etc., and a CD-ROM (Compact Disc Read Only Memory) 212 as an example of a removable recording medium. The computer-readable medium includes a CD-ROM drive 214 for controlling the reading and writing of various data from and to a CD-ROM (memory) 213, and bus lines 210 such as an address bus and a data bus for electrically connecting the above-mentioned components.
[0059] The transmission management program may be distributed by being recorded in a computer-readable recording medium such as the recording medium 206 or the CD-ROM 213 as a file in an installable or executable format. The transmission management program may be stored in the ROM 202 instead of the HD 204.
[0060] Moreover, the relay device 30 has the same hardware configuration as the transmission management device 50, and therefore the description thereof will be omitted. However, the HD 204 stores a relay device program for controlling the relay device 30.
[0061] In this case, the program for the relay device may be distributed by being recorded in a computer-readable recording medium such as the recording medium 206 or the CD-ROM 213 as a file in an installable or executable format. The program for the relay device may be stored in the ROM 202 instead of the HD 204.
[0062] Moreover, the program providing device 90 has the same hardware configuration as the transmission management device 50, and therefore the description thereof will be omitted. However, the HD 204 stores a program for the program providing device for controlling the program providing device 90.
[0063] In this case, the program for the program providing device may be distributed by being recorded in a computer-readable recording medium such as the recording medium 206 or the CD-ROM 213 as a file in an installable or executable format. The program for the program providing device may be stored in the ROM 202 instead of the HD 204.
[0064] Moreover, the UI providing device 20 has a hardware configuration similar to that of the transmission management device 50, and therefore the description thereof will be omitted. However, the HD 204 records a UI providing device program for controlling the UI providing device 20 and layout information to be described later.
[0065] In this case, the UI providing device program and the layout information may be distributed by being recorded in a computer-readable recording medium such as the recording medium 206 or the CD-ROM 213 as a file in an installable or executable format. Also, the UI providing device program and the layout information may be stored in the ROM 202 instead of the HD 204.
[0066] As the computer-readable recording medium, a CD-R (Compact Disc Recordable), a DVD (Digital Versatile Disk), a Blu-ray disc, etc. may be used.
[0067] <<Functional configuration of the embodiment>> Fig. 7 is a functional block diagram of each terminal, device, and system constituting the transmission system. As shown in Fig. 7, a transmission terminal 10, a relay device 30, and a transmission management device 50 are connected so as to be able to transmit data via a transmission network 2. Note that the relay device 30 and the program providing device 90 shown in Fig. 1 are not directly related to the transmission of data related to the video conference, and therefore are omitted from Fig. 7. The information processing terminal 40 has the same functions as the transmission terminal 10, and therefore the transmission terminal 10 in Fig. 7 may be the information processing terminal 40.
[0068] <Functional configuration of the transmission terminal> The transmission terminal 10 has a transmission / reception unit 11, an operation input reception unit 12, a login request unit 13, an imaging unit 14, a sound input unit 15a, a sound output unit 15b, a display control unit 17, and a storage / readout processing unit 19. These units are functions realized by a CPU 101 that executes a transmission terminal control program 1000a expanded from the flash memory 104 onto the RAM 103. The transmission terminal 10 also has a storage unit 1000 that is configured by the RAM 103 and the flash memory 104.
[0069] (Functional configuration of the transmission terminal) The transmission / reception unit 11 is composed of a CPU 101 and a network I / F 111, and is configured to transmit and receive various data and information with other transmission terminals 10, relay devices 30, transmission management devices 50, or program providing devices 90 via the transmission network 2.
[0070] The transmitter / receiver unit 11 is configured to start receiving, from the transmission management device 50, each status information indicating the status of each transmission terminal 10 as a candidate for the connection destination of the video conference, even before starting a video conference with a desired destination terminal.
[0071] The status information not only indicates the operating status of each transmission terminal 10, whether it is online or offline, but also indicates detailed operating status such as whether it is online and ready for transmission, whether it is online and transmitting, and so on.
[0072] In addition, the status information indicates not only the operating status of each transmission terminal 10 but also various other states, such as the cable 120c being disconnected from the transmission terminal 10, outputting sound but not outputting images, or not outputting sound (MUTE). In the following explanation, in order to make the present invention easier to understand, a case will be described in which the status information indicates the operating status of each transmission terminal 10.
[0073] The operation input reception unit 12 is composed of a CPU 101, an operation button 108, and a power switch 109, and is configured to receive various inputs from a user. For example, when a user turns on the power switch 109, the operation input reception unit 12 receives the power-on command and turns on the power of the transmission terminal 10.
[0074] The login request unit 13 is composed of a CPU 101, and upon receiving a power-on signal, transmits login request information indicating a request for login and the current IP address of the terminal from the transmitting / receiving unit 11 to the transmission management device 50 via the transmission network 2.
[0075] In addition, when the user turns the power switch 109 from ON to OFF, the transmitting / receiving unit 11 transmits state information to the transmission management device 50 to the effect that the power is to be turned OFF, and then the operation input receiving unit 12 turns the power of the transmission terminal 10 completely OFF. This allows the transmission management device 50 to know that the power of the transmission terminal 10 has been turned OFF from ON.
[0076] The imaging unit 14 is composed of a CPU 101, a camera 112, and an imaging element I / F 113, and is configured to capture an image of a subject and output image data obtained by capturing the image to the transmitting / receiving unit 11.
[0077] The sound input unit 15a is composed of a CPU 101 and a sound input / output I / F 116, and is configured to receive a sound signal representing a sound collected by a microphone 114 and convert the input sound signal into sound data.
[0078] The sound output unit 15b is constituted by the CPU 101 and a sound input / output I / F 116, and converts sound data into a sound signal and outputs the converted sound signal to the speaker 115, thereby causing the speaker 115 to output sound.
[0079] The display control unit 17 is composed of a CPU 101 and a display I / F 117, and combines image data of different resolutions received by the transmission / reception unit 11, and transmits the combined image data to the display 120, thereby causing the image represented by the image data to be displayed on the display 120.
[0080] Furthermore, the display control unit 17 transmits information about the destination list received by the transmitting / receiving unit 11 from the transmission management device 50 to the display 120, thereby causing the display 120 to display the destination list.
[0081] The display control unit 17 has an auxiliary area display unit 18. When the transmission terminal 10 is in a full image display mode described later, the auxiliary area display unit 18 displays the number of locations participating in the conference in the auxiliary area of the display screen.
[0082] The storage / read processing unit 19 is constituted by a CPU 101 and an SSD 105 , and is configured to store various data in the storage unit 1000 and to read out various data stored in the storage unit 1000 .
[0083] The storage unit 1000 stores an identifier (or identification information, hereinafter referred to as a "terminal ID") and a password for identifying the transmission terminal 10. Furthermore, the storage unit 1000 is also used as a receiving buffer for image data and audio data received when a video conference is held with a destination terminal.
[0084] The terminal ID in the present embodiment indicates information such as language, characters, symbols, or various marks used to uniquely identify the transmission terminal 10. The terminal ID may also be information that combines at least two of the language, characters, symbols, and various marks.
[0085] <Functional configuration of the transmission management device> The transmission management device 50 has a transmitting / receiving unit 51, a terminal authentication unit 52, a status management unit 53, a terminal extraction unit 54, a terminal status acquisition unit 55, and a storage / readout processing unit 59. Each of these units is a function realized by a CPU 201 that executes a transmission management program loaded from the HD 204 onto the RAM 203. The transmission management device 50 also has a storage unit 5000 configured by the HD 204.
[0086] (Terminal authentication management table) A terminal authentication management DB 5002 having a terminal authentication management table as shown in Fig. 8 is constructed in the storage unit 5000. In the terminal authentication management table, each password is associated with each terminal ID of all transmission terminals 10 managed by the transmission management device 50 and managed. For example, the terminal authentication management table shown in Fig. 8 indicates that the terminal ID of the transmission terminal 10aa is "01aa" and the password is "aaaa".
[0087] (Terminal Management Table) In addition, the storage unit 5000 has a terminal management DB 5003 having a terminal management table as shown in Fig. 9. Fig. 9(a) shows the terminal management table in a state where a transmission terminal 10aa with a terminal ID of "01aa" has logged in, and Fig. 9(b) shows the terminal management table in a state after the transmission terminal 10aa with a terminal ID of "01aa" has transmitted start request information, which will be described later.
[0088] In the terminal management table, for each terminal ID of each transmission terminal 10, the identification name of each transmission terminal 10 (hereinafter referred to as the "terminal name"), the operating status of each transmission terminal 10, the reception date and time when the login request information for logging in to the transmission system 1 was received by the transmission management device 50, and the IP address of the transmission terminal 10 are associated and managed.
[0089] For example, the terminal management table shown in FIG. 9(a) indicates that for a transmission terminal 10aa with a terminal ID of "01aa", the terminal name is "AA terminal, Tokyo office, Japan", the operating status is "online (transmission possible)", the date and time when the login request information was received by the transmission management device 50 was "13:40 on Nov. 10, 2009", and the IP address of the transmission terminal 10aa is "1.2.1.3".
[0090] Here, the terminal ID in the terminal management table is uniquely assigned when the transmission terminal 10 is registered in the transmission system 1 before transmitting login request information transmitted in step S22 of Fig. 11 described later, and the terminal name is registered or changed by the administrator of the transmission management device 50. Note that the terminal name may be registered or changed in response to a request made from the transmission terminal 10 to the transmission management device 50.
[0091] Furthermore, the operating status "online (transmitting)" indicates that the transmission terminal 10 is logged in to the transmission system 1 and is not participating in a video conference, "online (transmitting)" indicates that the transmission terminal 10 is logged in to the transmission system 1 and is participating in a video conference, and the operating status "offline" indicates that the transmission terminal 10 is not logged in to the transmission system 1. A transmission terminal 10 participating in a video conference is a transmission terminal 10 whose operating status is "online (transmitting)".
[0092] (Destination list management table) Furthermore, in the storage unit 5000, a destination list management DB 5004 having a destination list management table as shown in Fig. 10 is constructed. The terminal management DB 5003 constitutes the destination list management table storage unit in the present invention.
[0093] In the destination list management table, the terminal ID of a request source terminal that requests the start of a video conference is associated with the terminal IDs of all destination terminals that are registered as candidates for the destination terminal and managed.
[0094] 10, the candidates of the destination terminals to which the transmission terminal 10aa with the terminal ID "01aa" requests to start a video conference are the transmission terminal 10ab with the terminal ID "01ab," the transmission terminal 10ba with the terminal ID "01ba," and the transmission terminal 10db with the terminal ID "01db." The destination terminal candidates are added or deleted in response to a request made from the transmission terminal 10 to the transmission management device 50.
[0095] In addition, the destination list management table registers candidates for destination terminals that request the start of a video conference, and the transmission terminals 10 that can participate in a video conference are not limited to the transmission terminals 10 registered in the destination list management table.
[0096] (URI of UI providing device) Furthermore, the storage unit 5000 stores a uniform resource identifier (URI) 5005 of the UI providing device 20. The URI 5005 of the UI providing device 20 is set by the transmission management device 50 or an administrator of the UI providing device 20, or the like.
[0097] (Functional configuration of the transmission management device) The transmitting / receiving unit 51 is configured by a CPU 201 and a network I / F 209, and transmits and receives various data and information to and from other transmission terminals 10, relay devices 30, and program providing devices 90 via the transmission network 2.
[0098] The terminal authentication unit 52 is composed of the CPU 101, and performs terminal authentication by searching the terminal authentication management DB 5002 in the memory unit 5000 using the terminal ID and password contained in the login request information received via the transmission / reception unit 51 as search keys, and determining whether the same terminal ID and password are managed in the terminal authentication management DB 5002.
[0099] The status management unit 53 is composed of the CPU 101, and in order to manage the operating status of the requesting terminal that has made the login request, stores and manages in the terminal management DB 5003 (see Figures 9(a) and (b)) the operating status, the date and time when the login request information was received by the transmission management device 50, and the IP address in association with the terminal ID of the requesting terminal.
[0100] In addition, when a user turns the state of the power switch 109 of the transmission terminal 10 from ON to OFF, the state management unit 53 changes the operating state of the transmission terminal 10 in the terminal management DB 5003 (see Figures 9(a) and (b)) from online to offline based on state information sent from the transmission terminal 10 indicating that the power is to be turned OFF.
[0101] The terminal extraction unit 54 is configured by the CPU 101, and searches the destination list management DB 5004 (see FIG. 10) using the terminal ID of the requesting terminal that made the login request as a search key, and extracts the terminal IDs of candidate destination terminals for the requesting terminal.
[0102] In addition, the terminal extraction unit 54 searches the destination list management DB 5004 (see FIG. 10) using the terminal ID of the requesting terminal that made the login request as a search key, and also extracts the terminal IDs of other requesting terminals that have registered the terminal ID of the requesting terminal as a candidate destination terminal.
[0103] The terminal status acquisition unit 55 is configured by the CPU 101, and searches the terminal management DB 5003 (see Figures 9(a) and (b)) using the terminal IDs of the candidate destination terminals extracted by the terminal extraction unit 54 as a search key, and reads out the operating status for each detected terminal ID.
[0104] This allows the terminal status acquisition unit 55 to acquire the operating status of candidate destination terminals for the request source terminal that has made the login request. The terminal status acquisition unit 55 also searches the terminal management DB 5003 using the terminal ID extracted by the terminal extraction unit 54 as a search key, and acquires the operating status of the request source terminal that has made the login request.
[0105] The storage / readout processing unit 59 is constituted by the CPU 101 and the HDD 205 , and is configured to store various data in the storage unit 5000 and to read out various data stored in the storage unit 5000 .
[0106] (Functional configuration of UI provision section) The transmitting / receiving unit 21 is configured with a CPU 201 and a network I / F 209, and transmits and receives various data and information to and from the transmission terminal 10 via the transmission network 2.
[0107] The server unit 22 is constituted by the CPU 101, and receives a layout information request signal from the transmission terminal 10. The storage / read processing unit 23 transmits the layout information lot read from the storage unit 2000 to the transmission terminal 10 via the transmission / reception unit 21.
[0108] The storage / read processing unit 23 is configured by the CPU 101, and stores various data in the storage unit 2000 and reads out various data stored in the storage unit 2000. The storage unit 2000 is configured by the HDD 205, and mainly stores layout information lots.
[0109] <<Processing or Operation of the Embodiment>> Next, the operation of the transmission system 1 according to the present embodiment will be described. Fig. 11 is a sequence diagram showing a preparatory operation until the transmission terminal 10 is in a preparatory state for participating in a video conference. Fig. 11 shows an example in which the transmission terminal 10aa is in the preparatory state. In the preparatory operation, various information is transmitted and received through the management information session sei (see Fig. 2).
[0110] First, when a user of the transmission terminal 10aa turns on the power switch 109 of the transmission terminal 10aa, the operation input reception unit 12 receives the power-on command and turns on the power (step S21). Then, upon receiving the power-on command, the login request unit 13 transmits login request information representing a login request from the transmitting / receiving unit 11 of the transmission terminal 10aa to the transmission management device 50 via the transmission network 2 (step S22).
[0111] The login request information includes a terminal ID and a password for identifying the transmission terminal 10aa that has made the login request. The terminal ID and the password are read from the storage unit 1000 by the storage / read processing unit 19.
[0112] When the login request information transmitted from the transmission terminal 10aa is received by the transmission management device 50, the transmission management device 50 can know the IP address of the transmission terminal 10ab, which is the sender.
[0113] Next, the terminal authentication unit 52 of the transmission management device 50 searches the terminal authentication management DB 5002 (see Figure 8) in the memory unit 5000 using the terminal ID and password contained in the login request information received via the transmission / reception unit 51 as search keys, and performs terminal authentication by determining whether the terminal ID and password identical to the search key are managed in the terminal authentication management DB 5002 (step S23).
[0114] Here, if the terminal authentication unit 52 determines that the same terminal ID and password as the search key are not managed, that is, if it determines that the login request is not from a transmission terminal 10 with valid usage authority, the transmitter / receiver unit 51 transmits authentication result information representing the authentication result obtained by the terminal authentication unit 52 to the transmission terminal 10aa that made the login request via the transmission network 2, and the preparatory stage operation is terminated.
[0115] On the other hand, if the terminal authentication unit 52 determines that the same terminal ID and password as the search key are being managed, that is, if it determines that the login request is from a transmission terminal 10 with valid usage authority, the status management unit 53 associates and stores in the terminal management DB 5003 (see Figure 9 (a)) the terminal ID of the transmission terminal 10aa, the operating status, the reception date and time when the login request information was received, and the IP address of the transmission terminal 10aa (step S24).
[0116] As a result, as shown in FIG. 9A, the terminal management table manages the terminal ID "01aa", the operation state "online (transmission possible)", the reception date and time "2009.11.10.13:40", and the IP address "1.2.1.3" of the transmission terminal 10aa in association with each other.
[0117] Next, the transmitting / receiving unit 51 of the transmission management device 50 transmits authentication result information indicating the authentication result obtained by the terminal authentication unit 52 to the transmission terminal 10aa that has requested the login via the transmission network 2 (step S25). In addition, the transmission management device 50 transmits the URI 5005 of the UI providing device 20 to the transmission terminal 10 together with or before or after the authentication result information. This allows the transmission terminal 10 to access the UI providing device 20.
[0118] The URI 5005 of the UI providing device 20 indicates the storage location of the layout information lot in the transmission network 2. For example, it has a format such as "IP address (or domain name) + folder name + file name", but it is sufficient if the UI providing device 20 can be accessed.
[0119] Since each transmission terminal 10 does not hold the URI 5005, even if the UI providing device 20 is replaced or the URI is changed, the administrator of the transmission management device 50 only needs to change the URI stored in the transmission management device 50. In other words, since it is not necessary to change all the transmission terminals 10, the management load can be reduced.
[0120] Next, the terminal extraction unit 54 of the transmission management device 50 searches the destination list management DB 5004 (see FIG. 10) using the terminal ID “01aa” of the transmission terminal 10aa that has made the login request as a search key, and extracts terminal IDs of candidate destination terminals for the transmission terminal 10aa (step S26).
[0121] In this example, the terminal IDs "01ab", "01ba", and "01db" of the transmission terminals 10ab, 10ba, and 10db, which are destination terminals for the terminal ID "01aa" of the transmission terminal 10aa, are extracted.
[0122] Next, the terminal status acquisition unit 55 of the transmission management device 50 searches the terminal management DB 5003 (see Figure 9 (a)) using the terminal IDs "01ab", "01ba", and "01db" of the destination terminal candidates extracted by the terminal extraction unit 54 as search keys, and acquires the operating status of each of the transmission terminals 10ab, 10ba, and 10db by reading out the operating status "offline", "online (transmission possible)", and "online (transmitting)" for each terminal ID extracted by the terminal extraction unit 54 (step S27).
[0123] Next, the transmitter / receiver unit 51 of the transmission management device 50 transmits destination status information including the terminal IDs "01ab", "01ba", and "01db" used as the search keys in step S27 and the operating statuses of the corresponding transmission terminals 10ab, 10ba, and 10db, "offline", "online (transmission possible)", and "online (transmitting)", to the transmission terminal 10aa via the transmission network 2 (step S28).
[0124] This allows the transmission terminal 10aa to know the current operating status of each of the transmission terminals 10ab, 10ba, and 10db, which are candidates for the destination terminal, namely "offline," "online (transmission possible)," or "online (transmitting)."
[0125] Furthermore, the terminal extraction unit 54 of the transmission management device 50 searches the destination list management DB 5004 (see FIG. 10) using the terminal ID "01aa" of the transmission terminal 10aa that has made the login request as a search key, and extracts the terminal IDs of other transmission terminals that have registered the terminal ID "01aa" of the transmission terminal 10aa as a candidate for a destination terminal (step S29). For example, in the destination list management table shown in FIG. 10, the extracted terminal IDs are "01ab", "01ba", and "01db".
[0126] Next, the status management unit 53 of the transmission management device 50 searches the terminal management DB 5003 (see FIG. 9(a)) using the terminal ID “01aa” of the transmission terminal 10aa that has made the login request as a search key, and obtains the operating status of the transmission terminal 10aa that has made the login request as “online (transmission possible)” (step S30).
[0127] Next, the transmitter / receiver unit 51 of the transmission management device 50 transmits destination status information including the terminal ID "01aa" and operating status "online (transmission possible)" of the transmission terminal 10aa acquired in step S30 to the transmission terminals 10ba and 10db whose operating status is "online" in the terminal management DB 5003 (see Figure 9(a)) among the transmission terminals 10ab, 10ba, and 10db identified by the terminal IDs "01ab," "01ba," and "01db" extracted in step S29 (steps S31-1, 2).
[0128] When transmitting destination status information to the transmission terminals 10ba and 10db, the transmission / reception unit 51 refers to the IP addresses of the transmission terminals managed in the terminal management table shown in Figure 9(a) based on each terminal ID "01ba" and "01db".
[0129] This makes it possible to communicate the terminal ID "01aa" of the transmission terminal 10aa that made the login request and its operating status "online (transmission possible)" to the transmission terminals 10db and 10ba that can transmit to the transmission terminal 10aa that made the login request.
[0130] Furthermore, when step S28 of the preparation stage operation is completed, the display control unit 17 of the transmission terminal 10aa highlights on the display 120 the information of the destination list displayed on the display 120, which is the terminal participating in the video conference, i.e., the information of the transmission terminal 10db whose operating status is “online (transmitting)” or “online (transmittable)”, thereby enabling the user to recognize the transmission terminals 10 that are participating in the video conference or the transmission terminals 10 that can participate.
[0131] Fig. 12 is a sequence diagram showing a conference participation operation until the transmission terminal 10 enters a participating state for participating in a video conference. Fig. 12 shows an example in which the transmission terminal 10aa enters a participating state. In the conference participation operation, various information is transmitted and received in the management information session sei (see Fig. 2).
[0132] First, when a user presses the operation button 108 to select participation in a video conference, the operation input receiving unit 12 receives a request to participate in a video conference with the transmission terminal 10db (step S41). When the transmission terminal 10 is the information processing terminal 40, the request to participate in a video conference is received by pressing a hard key or a soft key displayed on a touch panel of the information processing terminal 40 instead of the operation button 108.
[0133] In response to this request, the transmitter / receiver unit 11 of the transmission terminal 10aa transmits participation request information to the transmission management device 50, which includes the terminal ID "01aa" of the transmission terminal 10aa and the terminal ID "01db" of the transmission terminal 10db and indicates an intention to participate in the video conference (step S42).
[0134] As a result, the transmitting / receiving unit 51 of the transmission management device 50 can receive the participation request information and confirm the IP address "1.2.1.3" of the transmission terminal 10aa which is the sender.
[0135] Next, the status management unit 53 of the transmission management device 50 searches the terminal management DB 5003 using the terminal ID “01aa” of the transmission terminal 10aa included in the participation request information as a search key, and sets the operation status of the transmission terminal 10aa that has requested participation to “online (transmitting)” (step S43). This updates the operation status in the terminal management table (see FIG. 9(b)).
[0136] Next, the transmitting / receiving unit 51 of the transmission management device 50 transmits start request information including the terminal ID "01aa" of the transmission terminal 10aa to the transmission terminal 10db via the transmission network 2 (step S44). This enables the transmission terminal 10db to know which transmission terminal 10 has requested to participate in the video conference.
[0137] In order to make the invention easier to understand, the explanation will be omitted, but since the operating status of the transmission terminal 10cb is also “online (transmitting)” in the terminal management DB 5003 shown in FIG. 9(b), the transmitter / receiver unit 51 of the transmission management device 50 transmits start request information including the terminal ID “01aa” of the transmission terminal 10aa to the transmission terminal 10cb via the transmission network 2.
[0138] Next, the transmission terminal 10db transmits reception completion information indicating that reception of the start request information has been completed to the transmission management device 50 from the transmitting / receiving unit 11 via the transmission network 2 (step S45).
[0139] Next, the transmitter / receiver 51 of the transmission management device 50 transmits relay start request information representing a request to start relaying to the relay device 30 via the transmission network 2 (step S46). The relay start request information includes the IP addresses “1.2.1.3” and “1.3.2.4” of the transmission terminal 10aa and the transmission terminal 10db, respectively.
[0140] As a result, the relay device 30 establishes a session for transmitting three types of image data, namely, low resolution, medium resolution, and high resolution, and sound data between the transmission terminals 10aa and 10db (step S47). In this way, the transmission terminal 10aa becomes able to participate in a video conference with the transmission terminal 10db.
[0141] Meanwhile, the transmitter / receiver 51 of the transmission management device 50 casts terminal status change information including the terminal ID "01aa" and the operating status "online (transmitting)" of the transmission terminal 10aa to each transmission terminal 10 registered in the terminal management DB 5003 (see FIG. 9(b)) (step S48). As a result, each transmission terminal that receives the terminal status change information can store and display the operating status of the transmission terminal 10aa with the terminal ID "01aa" as necessary.
[0142] Fig. 13 is a sequence diagram showing a data transmission operation between the transmission terminals 10. In the data transmission operation shown in Fig. 13, image data and audio data are transmitted and received in the image / audio data session sed (see Fig. 2).
[0143] As described using FIG. 12, when a session is established between the transmission terminals 10aa and 10db, the transmission terminal 10aa transmits image data captured by the imaging unit 14 and sound data converted from a sound signal input to the sound input unit 15a to the relay device 30 via the transmission / reception unit 51 (step S60).
[0144] The relay device 30, which has received the image data and audio data from the transmission terminal 10aa, transmits the received image data and audio data to each of the transmission terminals 10aa and 10db participating in the video conference (steps S61-1 and 61-2).
[0145] Each transmission terminal 10aa, 10db that receives image data and sound data from the relay device 30 causes the display control unit 17 to display the image represented by the received image data on the display 120, and also causes the sound output unit 15b to output the sound represented by the received sound data to the speaker 115 (steps S62-1, S62-2).
[0146] Next, the transmission terminal 10db transmits image data captured by the imaging unit 14 and sound data converted from the sound signal input to the sound input unit 15a to the relay device 30 via the transmission / reception unit 51 (step S63).
[0147] The relay device 30, which has received the image data and audio data from the transmission terminal 10db, transmits the received image data and audio data to each of the transmission terminals 10aa and 10db participating in the video conference (steps S64-1 and S64-2).
[0148] Each transmission terminal 10aa, 10db that receives image data and sound data from the relay device 30 causes the display control unit 17 to display the image represented by the received image data on the display 120, and also causes the sound output unit 15b to output the sound represented by the received sound data to the speaker 115 (steps S65-1, S65-2).
[0149] In this manner, the transmission terminals 10aa and 10db can hold a video conference via the relay device 30. In Fig. 13, in order to facilitate understanding of the present embodiment, the transmission terminals 10 participating in the video conference are two transmission terminals 10aa and 10db, but even if the transmission terminals 10 participating in the video conference are three or more, the image data and audio data are transmitted in the same manner as in the data transmission operation shown in Fig. 13.
[0150] <<Two display modes of the transmission terminal 10>> The following describes image display on the transmission terminal 10. In this embodiment, the display control unit 17 has two display modes: an all-image display mode and an information display mode.
[0151] First, the full image display mode will be described with reference to Fig. 14. Fig. 14 is a diagram showing an example of a screen displayed on the display 120 in the full image display mode. The full image display mode is displayed when a session is established in step S47 in Fig. 12 and transmission or reception of image data and audio data is started. In other words, of the two display modes, this is the display mode that is primarily used during a video conference.
[0152] In the all image display mode, the display control unit 17 of the transmitting terminal 10 displays all images transmitted from each transmitting terminal 10 participating in the video conference on the display 120. If displaying all images results in a small image area, a predetermined upper limit (maximum value) of images is displayed.
[0153] 14 is assigned one relatively wide first image area 500 and a plurality of relatively narrow second image areas 510. For example, an image represented by image data transmitted from each transmitting terminal 10 participating in the video conference is displayed in each second image area 510, and an image represented by image data of a main transmitting terminal among the transmitting terminals 10 participating in the video conference is displayed in the first image area 500.
[0154] For example, the display control unit 17 identifies, among the transmission terminals 10 participating in the video conference, the transmission terminal that is the source of the sound data representing voice among the received sound data, as the main terminal. Here, the sound data representing voice refers to sound data that is determined to have a voiced section by detecting a voice section in each piece of received sound data. The detection of the voice section can be realized using a known technique (for example, see Non-Patent Document 1).
[0155] Note that the display screen shown in FIG. 14 is just one example, and in the full image display mode, it is possible to display multiple images with all image areas of the same size, or to display only the images of the main terminals listed above across the entire display 120.
[0156] <Information display mode> Next, the information display mode will be described. The user can alternate between two display modes by operating, for example, the operation button 108. If the transmission terminal 10 is the information processing terminal 40, the user can alternate between the two display modes by pressing a hard key of the information processing terminal 40 or a soft key displayed on a touch panel instead of the operation button 108.
[0157] Fig. 15 is a diagram showing an example of a screen displayed on the display 120 in the full image display mode. As shown in Fig. 15, in the information display mode, the display control unit 17 of the transmitting terminal 10 causes the display 120 to display terminal information including at least the terminal name and terminal ID of each transmitting terminal 10 participating in the video conference.
[0158] 15, an image display area 520 for displaying an image and a terminal information display area 530 for displaying terminal information are allocated. For example, the terminal information of the transmission terminals 10 participating in the video conference is displayed in a list in the terminal information display area 530.
[0159] In addition, among the terminal information displayed in the list, those that do not display the terminal name represent transmission terminals 10 that are not registered as candidates for the destination terminal of the own terminal in the destination list management DB 5004 (see FIG. 10).
[0160] For example, in the display screen shown in FIG. 15, the name of the transmission terminal 10db is displayed because the transmission terminal 10db is registered in the destination list management DB 5004 as a candidate destination terminal for the transmission terminal 10aa, but the name of the transmission terminal 10cb is not displayed because the transmission terminal 10cb is not registered in the destination list management DB 5004 as a candidate destination terminal for the transmission terminal 10aa.
[0161] In the image display area 520, an image represented by image data of a main transmitting terminal among the transmitting terminals 10 participating in the video conference is displayed. For example, the display control unit 17 specifies, among the transmitting terminals 10 participating in the video conference, a transmitting terminal that is a source of sound data representing voice among the received sound data, as a main transmitting terminal. In addition, when the operation input receiving unit 12 receives a selection of one transmitting terminal from the transmitting terminals 10 participating in the video conference in response to an operation of the operation button 108 by a user, the display control unit 17 may specify the selected transmitting terminal as a main transmitting terminal. Note that, when the transmitting terminal 10 is the information processing terminal 40, the transmitting terminal selected by pressing a hard key or a soft key displayed on a touch panel of the information processing terminal 40 instead of the operation button 108 is specified as a main transmitting terminal.
[0162] Fig. 16 is a sequence diagram showing a terminal information display operation for displaying terminal information in the terminal information display area 530 when the transmission terminal 10 is switched to the information display mode. In the terminal information display operation shown in Fig. 16, various information is transmitted and received through the management information session sei.
[0163] First, the transmission terminal 10 transmits a terminal information request signal representing a request for terminal information of the transmission terminals participating in the video conference to the transmission management device 50 (step S70).
[0164] The terminal status acquisition unit 55 of the transmission management device 50 that receives the terminal information request signal searches the terminal management DB 5003 (see Figure 9 (b)) in the memory unit 5000 using the IP address of the sender of the terminal information request signal as a search key, and acquires the terminal ID of the transmission terminal that sent the terminal information request signal (step S71).
[0165] Next, the terminal status acquisition unit 55 searches the terminal management DB 5003 to search for terminal IDs and terminal names whose operating status is "online (transmitting)" (step S72). In this embodiment, the terminal IDs detected by the terminal status acquisition unit 55 are "01aa", "01cb", and "01db". Note that the operating status of the transmission terminal with the terminal ID "01aa" is set to "online (transmitting)" in step S43 (see FIG. 9(b)).
[0166] Next, the terminal status acquisition unit 55 searches the destination list management DB 5004 (see FIG. 10) in the memory unit 5000 using the terminal ID detected in step S71 (in this embodiment, "01aa") as a search key, and acquires the terminal IDs of candidates for the destination terminal that is the source of the terminal information request signal (step S73).
[0167] For example, if the terminal ID of the transmission terminal that transmitted the terminal information request signal is "01aa", the terminal IDs acquired by the terminal status acquisition unit 55 will be "01ab", "01ba", and "01db".
[0168] Next, the terminal status acquisition unit 55 determines whether each terminal ID acquired in step S72 is included in the terminal ID of the sender of the terminal information request signal or the terminal ID acquired in step S73, and if it determines that they are included, it makes the transceiver unit 51 respond with the terminal ID and terminal name to the sender of the terminal information request signal, and if it determines that they are not included, it makes the transceiver unit 51 respond with only the terminal ID to the sender of the terminal information request signal (step S74).
[0169] For example, if the terminal ID of the transmitting terminal that sent the terminal information request signal is "01aa," then the terminal IDs "01aa," "01db," and terminal names "Japan Tokyo Office, AA Terminal" and "USA Washington Office, DB Terminal" of transmitting terminal aa and transmitting terminal db, as well as the terminal ID "01cb" of transmitting terminal cb are transmitted to transmitting terminal aa, which is the sender of the terminal information request signal.
[0170] Upon receiving this response, the display control unit 17 of the transmission terminal 10 displays the response content in the terminal information display area 530 as shown in the display screen of FIG. 15 (step S75).
[0171] <Displaying the number of locations in full image display mode> In the display screen in the full image display mode as shown in Fig. 14, when the number of transmission terminals 10 participating in the conference is small, the participants of the video conference can easily know who is participating as the other participants, but as the number increases, it becomes increasingly difficult to know. The following will be explained with reference to the figure.
[0172] Fig. 17(a) shows an example of a display screen in the full image display mode when the number of locations is four. The display screen in Fig. 17(a) has a wide first image area 171 and three second image areas 172 narrower than the first image area 171. As in Fig. 14, the second image area 172 displays an image represented by image data transmitted from each transmitting terminal 10 participating in the video conference, and the first image area 171 displays an image represented by image data of a main transmitting terminal among the transmitting terminals 10 participating in the video conference.
[0173] If the number of locations is about four, it is not impossible for a user to identify a participant from one of the second image areas. However, as shown in Fig. 17(b), as the number of locations increases, it becomes difficult for participants to identify other participants. Fig. 17(b) shows an example of a display screen in full image display mode when the number of locations is nine.
[0174] In addition, from the viewpoint of practicality, an upper limit (hereinafter referred to as the display upper limit number) should be set for the number of locations that the display 120 can simultaneously display, but the number of locations with which the transmitting terminal 10 can hold a video conference (hereinafter referred to as the display upper limit number) may not match the display upper limit number. If the display upper limit number is greater than or equal to the location upper limit number, each transmitting terminal 10 can display images of all locations on the display 120, but if the display upper limit number is less than the location upper limit number, there will be locations that the transmitting terminal 10 cannot display.
[0175] As shown in FIG. 16, a user may be able to display the terminal names and terminal IDs of the transmission terminals 10 at all bases by operating the operation button 108 to switch to the information display mode, but even in this case, the user is required to perform some kind of operation.
[0176] Also, during the information display mode, an image of only one main location can be displayed, which causes the inconvenience of being unable to grasp the situation at other locations.
[0177] Therefore, the transmission terminal 10 of the present embodiment displays the number of locations participating in the video conference while remaining in the full image display mode, allowing the participants in the video conference to grasp the number of locations in real time while remaining in the full image display mode.
[0178] <<Displaying the number of locations in full image display mode>> Fig. 18 is a diagram showing an example of a display screen on which the number of locations is displayed in the full image display mode. The auxiliary area display unit 18 displays the number of locations in the auxiliary area 173 in the full image display mode. In Fig. 18, the auxiliary area 173 is a strip-shaped blank area at the top of the display screen, but it may be a strip-shaped blank area at the right edge, left edge, or bottom, or may be displayed so as to overlap with the image.
[0179] The auxiliary region 173 is divided into a first region ar1 to a sixth region ar6. The first area ar1 displays the terminal ID of the own terminal. A user of the transmission terminal 10 who wishes to participate in the same video conference as the transmission terminal 10 can participate in the video conference by inputting or selecting the terminal ID into the transmission terminal 10 that is currently inquiring. Second area ar2: The number of locations is displayed. Third area ar3: Displays the conference time since the session was established. Fourth area ar4: Any information is displayed. Fifth area ar5: Network bandwidth information (upstream and downstream communication speeds) is displayed. The sixth area ar6 displays cursor buttons. The cursor button 1736 has an upper triangle button 1736u and a lower triangle button 1736d, but the upper triangle button 1736u is black and the lower triangle button 1736d is white. The user can only operate the upper triangle button 1736u, and cannot operate the lower triangle button 1736d.
[0180] It is preferable that the auxiliary area 173 is displayed semi-transparently, so that visibility can be easily maintained even if the auxiliary area 173 overlaps with image data transmitted between locations.
[0181] The number of locations may be displayed in any of the first to sixth areas, or may be displayed anywhere in auxiliary area 173 if auxiliary area 173 is not divided. However, dividing auxiliary area 173 increases the visibility of the display screen from the user's perspective.
[0182] Here, the cursor button 1736 in the sixth area ar6 is an icon that notifies the user that the transmission terminal 10 can switch to the simplified display auxiliary area 173 by operating the operation button 108. Hereinafter, the auxiliary area in FIG. 18 will be distinguished by being referred to as the "standard auxiliary area."
[0183] That is, in the initial setting of the transmission terminal 10, the auxiliary area 173 is set to display various pieces of transmission status information of the first area ar1 to the sixth area ar6 in Fig. 18, but the user can operate the operation button 108 (or operate a touch panel, etc.) to press the upper triangle button 1736u to switch from the standard auxiliary area 173 to the simplified display auxiliary area 173, or set not to display the auxiliary area 173 thereafter. Details will be described with reference to Fig. 24.
[0184] When the transmission terminal 10 is an information processing terminal 40, it is possible not to display the information on the transmission status of the first area ar1 to the sixth area ar6 in the auxiliary area 173 by pressing a hard key of the information processing terminal 40 or a soft key displayed on the touch panel instead of the operation button 108.
[0185] 18, images of four locations are displayed on the display 120. Therefore, the number of locations in the second area is "4." The user can know at a glance the number of locations participating in the video conference.
[0186] Fig. 19 is a diagram showing another example of a display screen in which the number of locations is displayed in the full image display mode. Since components with the same reference numerals as those in Fig. 18 perform similar functions, only differences will be described.
[0187] 19, the display screen is divided into nine regions. The number of locations in the second region ar2 is also "9." In this way, even if the number of locations increases, the user can grasp at a glance the number of locations participating in the video conference.
[0188] For the sake of explanation, it is assumed that the display upper limit number of the transmitting terminal 10 is "9". When the display upper limit number of the transmitting terminal 10 is "9", there is a case where 10 or more transmitting terminals 10 participate in the video conference. In this case, the transmitting terminal 10 displays the number of transmitting terminals 10 participating in the video conference as the number of locations in the second area ar2 while keeping the number of images displayed in the full image display mode as "9".
[0189] Fig. 20 shows an example of a display screen in which the number of images displayed in the full image display mode differs from the number of locations displayed in the second area ar2. In Fig. 20, the number of images displayed in the full image display mode is "9", but the number of locations in the second area ar2 is "15". In this way, even when "the number of locations participating in the video conference > the maximum number to display", the user can know at a glance the number of locations participating in the video conference.
[0190] The layout (i.e., arrangement) of this auxiliary area 173 is defined in layout information lot stored in the storage unit 2000 of the UI providing device 20. The layout information lot is written based on languages and standards such as HTML, JavaScript (registered trademark), XML, and CSS.
[0191] The layout information lot determines the position of the auxiliary area 173, the division of the auxiliary area 173 into a first area to a sixth area of a fixed size, and the information to be displayed in each of the first area ar1 to the sixth area ar6. Therefore, the transmitting terminal 10 that acquires the layout information lot from the UI providing device 20 can set and display information on the transmission status in the first area ar1 to the sixth area ar6.
[0192] 17 and other figures, in the full image display mode, the layout of the image data received by the transmitting terminal 10 from the destination terminal on the display 120 is determined in advance based on the number of locations, etc. In contrast, the layout information lot is defined separately from the layout of the image data, so that the information and layout of the transmission status to be placed in the auxiliary area 173 can be defined without being affected by the layout of the image data. In other words, the transmitting terminal 10 can display the first area ar1 to the sixth area ar6 in the auxiliary area independently of the image data, and even if the layout of the image data is changed, the auxiliary area 173 is not affected.
[0193] <Simple display auxiliary area 173> Fig. 21 is a diagram showing an example of the simplified display auxiliary area 173. In Fig. 21, the number of points is the same as in Fig. 18, but the information on the transmission states of the first area ar1 to the sixth area ar6 is displayed in a simplified manner.
[0194] In the simplified display auxiliary area 173, information about what is displayed in each area is not displayed; mainly, only the contact ID value is displayed in the first area ar1, only the number of locations is displayed in the second area ar2, only the time is displayed in the third area ar3, and only the bandwidth is displayed in the fifth area ar5.
[0195] The display form of the cursor button 1736 in the sixth area ar6 is different from that in the standard auxiliary area 173. In the simplified display auxiliary area 173, both the upper triangle button 1736u and the lower triangle button 1736d are black. The user can operate either the upper or lower triangle button. Down triangle button: Switch to standard auxiliary area 173. Upper triangle button: Do not display the simplified display auxiliary area 173 (and do not display the standard auxiliary area 173 either).
[0196] The simplified display auxiliary area 173 occupies a smaller area of the display 120 than the standard auxiliary area 173, so that the visibility of the image data can be improved while still displaying information about the transmission status in the auxiliary area 173.
[0197] It is preferable that the simplified display auxiliary area 173 is displayed semi-transparently, like the standard auxiliary area 173.
[0198] <Display procedure> Fig. 22 is an example of a sequence diagram showing a procedure in which the transmission system 1 displays the number of locations when the transmission terminal 10 is in the full image display mode. The procedure in Fig. 22 is executed after the session is established in step S47 in Fig. 12, even if the user does not operate the operation button 108 or the like. That is, the initial setting of displaying the auxiliary area 173 is set in the transmission terminal 10.
[0199] After the session is established in step S47, the user may press the operation button 108. When the transmission terminal 10 is the information processing terminal 40, the user may press a hard key or a soft key displayed on a touch panel of the information processing terminal 40 instead of the operation button 108.
[0200] First, the transmission terminal 10 specifies the URI of the UI providing device 20 acquired in step S25 of Fig. 11 and transmits a layout information request signal (an example of a layout information request in the claims) for requesting layout information lot to the UI providing device 20 (S80). Note that the URI of the UI providing device 20 may not be acquired in step S25 of Fig. 11, but may be acquired by the transmission terminal 10 from the transmission management device 50 immediately before step S80, or may be acquired by pressing the operation button 108 by the user. Note that when the transmission terminal 10 is the information processing terminal 40, the URI of the UI providing device 20 is acquired by pressing a hard key or a soft key displayed on a touch panel of the information processing terminal 40 instead of the operation button 108.
[0201] The transmitting / receiving unit 21 of the UI providing device 20 receives the layout information request signal, and in response to this, the server unit 22 refers to the storage unit 2000 based on the specified URI and reads out the layout information lot (S81).
[0202] Next, the server unit 22 transmits the read layout information lot to the transmission terminal 10 that transmitted the layout information request signal via the transmission / reception unit 21 (S82). This layout information lot includes the layout information of both the standard and simplified display auxiliary areas 173.
[0203] The transmitting / receiving unit 11 of the transmitting terminal 10 stores the layout information lot in a volatile memory (S83). The volatile memory is, for example, the RAM 103. That is, by storing the layout information lot once obtained in the transmitting terminal 10, it is not necessary to obtain the layout information lot again from the UI providing device 20 while the power is ON. As a result, even when the transmitting terminal 10 erases the auxiliary area 173 once and then displays the auxiliary area 173 again, the layout information lot can be read from the volatile memory of the transmitting terminal 10. Therefore, the processing time for obtaining the layout information lot can be omitted, and the time until the auxiliary area 173 is displayed can also be shortened. Furthermore, if the layout information lot is stored in a volatile memory, the layout information lot is erased by turning off the power of the transmitting terminal 10, so that it is possible to prevent the layout information lot from leaking from the transmitting terminal 10.
[0204] Next, the auxiliary area display unit 18 of the transmission terminal 10 transmits a terminal information request signal to the transmission management device 50 (S84). The terminal information request signal is a signal requesting the number of transmission terminals 10 participating in the video conference.
[0205] The transmitting / receiving unit 51 of the transmission management device 50 receives the terminal information request signal, and the terminal status acquisition unit 55 searches the terminal management DB 5003 and calculates the number of terminal IDs whose operating status is "online (transmitting)" (S85). That is, the terminal status acquisition unit 55 identifies the terminal management table (see FIG. 9(b)) in which the IP address of the transmission terminal 10 that transmitted the terminal information request signal is registered, and identifies all terminal IDs whose operating status is "online (transmitting)" from the terminal management table to determine the number. In this embodiment, the terminal IDs detected by the terminal status acquisition unit 55 are "01aa", "01cb", and "01db", so the number is "3". The operating status of the transmission terminal 10aa whose terminal ID is "01aa" is set to "online (transmitting)" in step S43 of FIG. 12.
[0206] In addition to the number of "online (transmitting)" transmission terminals 10, the number of "online (transmitting)" transmission terminals 10 may be obtained. This allows each transmission terminal 10 to display the number of logged-in transmission terminals 10 on the display 120. Also, the number of "online (transmitting)" transmission terminals 10 and the number of "online (transmitting)" transmission terminals 10 may be obtained separately, and each transmission terminal 10 may display them separately on the display 120. This allows, for example, if the two numbers do not match, to know that there is a user who is not participating in the conference. Next, the terminal status acquisition unit 55 transmits the number of "online (transmitting)" terminal IDs acquired above to the transmission terminal 10 that transmitted the terminal information request signal via the transmission / reception unit 51 (S86).
[0207] The terminal status acquisition unit 55 may transmit the number of "online (transmitting)" terminal IDs to online transmission terminals 10 other than the transmission terminal 10 that transmitted the terminal information request signal. For example, when the number of online terminal IDs is transmitted to the transmission terminal 10aa with the terminal ID "01aa" in step S86, it also transmits the number to the transmission terminals 10cb and 10db with the terminal IDs "01cb" and "01db". This allows the transmission terminals 10cb and 10db with the terminal IDs "01cb" and "01db" to display the number of locations in the auxiliary area 173 simply by receiving the layout information lot.
[0208] The transmitter / receiver 11 of the transmission terminal 10 receives the number of "online (transmitting)" terminal IDs, and the auxiliary area display unit 18 displays it on the display 120 (S87). That is, the layout information lot is interpreted to determine the position of the standard auxiliary area 173 and the layout of the first area ar1 to the sixth area ar6, and transmission status information is set in the first area ar1 to the sixth area ar6 and displayed on the display 120.
[0209] The terminal ID of the own terminal in the first area ar1 is stored in the storage unit 1000. The number of locations in the second area ar2 is transmitted from the transmission management device 50. The conference time from the establishment of the session in the third area ar3 is measured by the transmitting terminal 10 from the establishment of the session. The arbitrary information in the fourth area ar4 is, for example, information possessed by the transmitting terminal 10 or the transmission management device 50. The network bandwidth information in the fifth area ar5 is measured by the transmitting terminal 10 periodically or in real time and displayed as the communication speed with the relay device 30. The cursor button 1736 in the sixth area ar6 is image data such as a predetermined icon.
[0210] The displayed auxiliary area 173 can be erased by the user operating the operation button 108. It can also be displayed by operating the operation button 108 again.
[0211] In this manner, the transmission terminal 10 can display the number of locations participating in the video conference without switching to the information display mode. Therefore, the user does not need to operate the operation button 108 or the like. Furthermore, since the transmission terminal 10 remains in the full image display mode, the user can grasp the number of locations while viewing the images of each location. Furthermore, even if the number of locations participating in the video conference exceeds the upper limit of the number that can be displayed, the user can accurately grasp the number of locations.
[0212] <Display settings for auxiliary area> The user can set the transmission terminal 10 to either "display according to the operation" or "disappear after a certain period of time has passed" for the displayed auxiliary area 173.
[0213] Fig. 23 is a diagram showing an example of a screen on which a user sets the display settings of the auxiliary area 173. The screen of Fig. 23 is displayed on the display 120 when the user operates, for example, the operation button 108. The screen of Fig. 23 displays two radio buttons 221 and 222, and a message 223 that reads "Display according to operation" and a message 224 that reads "Delete after a certain period of time has elapsed" that correspond to the radio buttons 221 and 222. The user can select one of the two radio buttons 221 and 222.
[0214] When the radio button 221 is selected, the transmission terminal 10 keeps the auxiliary area 173 displayed, but as shown in FIG. 24, depending on the user's operation, the transmission terminal 10 may switch between the standard auxiliary area 173, the simplified display auxiliary area 173, or neither auxiliary area 173 may be displayed.
[0215] When the radio button 222 is selected, the transmission terminal 10 displays the auxiliary area 173, erases it after a certain time has elapsed, and displays it again when a trigger for redisplaying it is detected. In this case, too, one of the triggers is the user's operation to display or erase it.
[0216] When the radio button 222 is set, a message 225 saying "Please set a certain time" becomes visible, and a number can be set in a time setting field 226. In the time setting field 226, the user can set the time until the auxiliary area 173 is erased. A predetermined initial value is set until the user sets a time. The user of the transmission terminal 10 can set any time in the time setting field 226. An upper and lower limit for the time that can be set in the time setting field 226 may be set.
[0217] FIG. 24 is a flowchart illustrating an example of an operation procedure in which the transmission terminal 10 switches between the standard auxiliary area 173 and the simplified display auxiliary area 173 when "display according to operation" is selected.
[0218] FIG. 24 starts when the auxiliary area display unit 18 displays the standard auxiliary area 173 in step S87 in FIG. 22 (S87-1).
[0219] The auxiliary area display unit 18 determines whether a predetermined time has elapsed after displaying the standard auxiliary area 173 (S87-2). The predetermined time may be determined in advance, or may be the time set in "Clear after a certain time has elapsed" in Fig. 23. The time may be set by the user separately from the certain time.
[0220] When a predetermined time has elapsed (Yes in S87-2), the auxiliary area display unit 18 switches (S87-3) from the standard auxiliary area 173 to the simplified display auxiliary area 173. The upper and lower triangular buttons 1736u, d of the cursor button 1736 in the simplified display auxiliary area 173 are both black, so that the user can recognize that a change will occur on the screen regardless of which triangular button is pressed.
[0221] The auxiliary region display unit 18 judges whether or not the lower triangle button 1736d has been operated (S87-4).
[0222] When the lower triangular button 1736d is operated (Yes in S87-4), the auxiliary area display unit 18 switches the simplified display auxiliary area 173 to the standard auxiliary area 173 (S87-7). In the standard auxiliary area 173, the lower triangular button 1736d is outlined in white, indicating that pressing it will cause nothing to happen, while the upper triangular button 1736u is black, indicating that pressing it will display the simplified display auxiliary area 173. In addition, the standard auxiliary area 173 is displayed semi-transparently, allowing the conference to continue without impairing the visibility of the image data as much as possible.
[0223] If the lower triangle button 1736d is not operated (No in S87-4), the auxiliary region display unit 18 judges whether the upper triangle button 1736u is operated (S87-5). Note that the judgment of whether the lower triangle button 1736d or the upper triangle button 1736u is operated may be made first.
[0224] If the upper triangular button 1736u is operated (Yes in S87-5), the auxiliary area display unit 18 does not display the auxiliary area (S87-6). That is, neither the standard auxiliary area nor the simplified display auxiliary area is displayed. After this, the simplified display auxiliary area may be displayed by operating the lower triangular button 1736d (that is, returning to step S87-3). Note that the upper triangular button 1736u may be designed so that nothing happens when it is operated, or the standard auxiliary area may be displayed upon operation. It may be possible to control the display of neither the standard auxiliary area nor the simplified display auxiliary area, but to display only the cursor button 1736, and to return to step S87-3.
[0225] If the standard auxiliary area is displayed in step S87-7, the auxiliary area display unit 18 determines whether or not the upper triangular button 1736u has been operated (S87-8).
[0226] When the upper triangular button 1736u is displayed (Yes in S87-8), the auxiliary area display unit 18 switches the standard auxiliary area 173 to the simplified display auxiliary area 173 (S87-3).
[0227] In this way, the user can operate the cursor button 1736 to switch the display mode of the auxiliary area.
[0228] <Operation procedure after displaying the auxiliary area> Next, the operations of the transmission management device 50 and the transmission terminal 10 after the transmission terminal 10 displays the auxiliary area 173 in step S87 of FIG. 22 will be described.
[0229] Fig. 25 is an example of a flowchart showing the operation procedure of the transmission management device 50. The process of Fig. 25 is repeatedly executed after the transmission management device 50 calculates the number of online terminal IDs in step S85 of Fig. 22, for example.
[0230] The terminal status acquisition unit 55 monitors the terminal management table in which the IP address (or terminal ID) of the transmission terminal 10 that has transmitted the terminal information request signal is registered (S85-1).
[0231] The terminal status acquisition unit 55 determines whether or not the number of terminal IDs that are in the "online (transmitting)" state and that were transmitted in step S85 of FIG. 22 has increased or decreased (S85-2).
[0232] If the determination in step S85-2 is No, the process in FIG. 25 ends, and the process is repeated from step S85-1.
[0233] If the determination in step S85-2 is Yes, the number of locations has changed, so the terminal status acquisition unit 55 transmits the current number of terminal IDs whose operating status is "online (transmitting)" to the transmission terminal 10 (S85-3). In this way, once the transmission terminal 10 has acquired the number of terminal IDs whose operating status is "online (transmitting)", it can always acquire the correct number of locations.
[0234] 25, the terminal status acquisition unit 55 may also transmit the number of "online (transmitting)" terminal IDs to the "online (transmitting)" transmitting terminals 10 other than the transmitting terminal 10 that transmitted the terminal information request signal. This allows all online transmitting terminals 10 to always hold the correct number of locations.
[0235] Fig. 26 is an example of a flowchart illustrating an operation procedure of the transmission terminal 10. The process of Fig. 26 is repeatedly executed after the transmission terminal 10 displays the auxiliary area 173 in step S87 of Fig. 22, for example. Fig. 26 describes a case where the auxiliary area 173 is set to "disappear after a certain period of time has elapsed."
[0236] First, the auxiliary area display unit 18 determines whether or not the auxiliary area 173 is being displayed (S87-1).
[0237] If the determination in step S87-1 is Yes, the auxiliary area display unit 18 determines whether or not the time that has elapsed since the start of displaying the auxiliary area 173 has exceeded a certain period of time (S87-2).
[0238] If the determination in step S87-2 is Yes, the auxiliary area display unit 18 erases (S87-3) the auxiliary area 173. This allows the participants to participate in the video conference while focusing mainly on the images of the respective locations.
[0239] If the determination in step S87-2 is No, the auxiliary area display unit 18 keeps the auxiliary area 173 displayed.
[0240] If the determination in step S87-1 is No, the auxiliary area display unit 18 determines whether or not there is an opportunity to display the auxiliary area 173. This opportunity occurs when the number of terminal IDs that are “online (transmitting)” is received from the transmission management device 50 or when the user performs an operation to display it.
[0241] First, the auxiliary area display unit 18 determines whether or not the number of terminal IDs that are "online (transmitting)" has been received from the transmission management device 50 (S87-4).
[0242] If the determination in step S87-4 is Yes, the auxiliary area display unit 18 displays the auxiliary area 173 again (S87-5) based on the layout information lot stored in the RAM 103. In this way, when the number of locations has changed, the transmission terminal 10 can immediately reflect the changed number of locations in the auxiliary area 173.
[0243] If the determination in step S87-4 is No, the auxiliary area display unit 18 determines whether or not the user has performed an operation to display (S87-6).
[0244] If the determination in step S87-6 is Yes, the auxiliary area display unit 18 displays the auxiliary area 173 again (S87-5) based on the layout information lot stored in the RAM 103. This allows the user to display the auxiliary area 173 at any timing.
[0245] If the determinations in steps S87-4 and S87-6 are No, the auxiliary area display unit 18 keeps the auxiliary area 173 hidden and does not display it.
[0246] Therefore, by displaying the change in the number of locations, users can easily notice the change. In addition, the image disappears after a certain period of time, so users can participate in the video conference by focusing on the images of each location.
[0247] If the user wants to know more about the participating locations by checking the change in the number of locations or the number, he or she can switch to the information display mode. Since the user can switch to the information display mode when the number of locations has changed, there is no need for the user to switch to the information display mode to check the number of locations or the participating locations.
[0248] <<Suitable Modifications>> The above describes the best mode for carrying out the present invention using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.
[0249] For example, the relay device 30, the transmission management device 50, and the program providing device 90 may be constructed by a single computer, rather than being independent of each other. Also, each means and the storage units 1000, 2000, 3000, and 5000 of the relay device 30, the transmission management device 50, and the program providing device 90 may be arbitrarily assigned to a plurality of computers. In this case, the UI providing device 20 may be included as a function of any of the computers.
[0250] Furthermore, as an example of the image quality of the image data relayed by the relay device 30, the image resolution of the image data is managed, but the quality is not limited to this, and other examples of the quality may be managed by focusing on the image quality depth of the image data, the sampling frequency of the sound of the sound data, the bit length of the sound of the sound data, etc. Also, the sound data may be transmitted and received by dividing it into data of three different resolutions (high resolution, medium resolution, and low resolution).
[0251] 9(a) and 9(b), the IP address of the transmission terminal 10 is managed, but the present invention is not limited to this, and any information for identifying the transmission terminal 10 on the transmission network 2 may be managed, for example, the FQDN (Fully Qualified Domain Name) of the transmission terminal 10. In this case, the IP address corresponding to the FQDN is obtained by a well-known DNS (Domain Name System) server.
[0252] In the above embodiment, a video conference system has been described as an example of the transmission system 1, but the present invention is not limited to this and may be a telephone system such as an IP (Internet Protocol) telephone or an Internet telephone.
[0253] Further, in the above-mentioned transmission system 1, a video conference system has been described as an example of the transmission system 1, but the present invention is not limited to this and may be a telephone system such as an IP (Internet Protocol) telephone or an Internet telephone.
[0254] The transmission system 1 may be a system including a car navigation device mounted on a moving object. Fig. 27 is a diagram showing an example of a system configuration in which the transmission terminal 10 is applied to a car navigation device 200. Hereinafter, when distinguishing between the car navigation devices 200, the reference numerals of the car navigation devices 200 are represented as 200-i (i is a natural number).
[0255] In this case, one transmission terminal 10 corresponds to a car navigation device 200-2 mounted on an automobile 601. The other transmission terminal 10 corresponds to a management terminal 200-1 used by a communicator of a management center 604, or a car navigation device 200-3 mounted on another automobile 602. The management terminal 200-1, the car navigation device 200-2, and the car navigation device 200-3 are connected to each other via a network 2. In addition, the management terminal 200-1, the car navigation device 200-2, and the car navigation device 200-3 establish a session by the transmission management device 50.
[0256] 28 is a diagram showing an example of the configuration of a car navigation device. The car navigation device 200 has a configuration in which a GPS receiver 612, a vehicle speed sensor 613, a gyro sensor 614, and road map data 615 are connected to the transmission terminal 10 described above. The transmission terminal 10 also has an integrated display 120. The display 120 may be connected to the transmission terminal 10 by wire or wirelessly, or may be detachable from the transmission terminal 10.
[0257] The transmission terminal 10 executes a program to provide the function of the car navigation device 200. The GPS receiver 612 captures GPS satellites and outputs the coordinates of the current position. The vehicle speed sensor 613 is a sensor that detects the speed (wheel rotation speed) of the automobiles 601, 602. The gyro sensor 614 is a sensor that detects angular velocity, and by integrating the angular velocity, the traveling direction of the vehicle can be detected.
[0258] The road map data 615 is data that represents roads as a combination of nodes and links, and may be downloaded from an external source or may be stored in the vehicle in advance. The road map data is displayed on the display 120.
[0259] A PC for screen sharing may be connected to the transmission terminal 10. In addition to the configuration illustrated in Fig. 28, the car navigation device 200 may have a tuner function for receiving radio and TV broadcasts, an audio function for playing music, a linking function with an in-vehicle camera such as a rearview camera, a browser function for displaying websites, and the like.
[0260] The transmission terminal 10 estimates its own vehicle position by autonomous navigation that accumulates the travel distance detected by the vehicle speed sensor 613 in the travel direction detected by the gyro sensor 614, starting from the position information detected by the GPS receiver 612. The own vehicle position is displayed on a road map using a vehicle mark or the like. In addition, the transmission terminal 10 searches for a route to the destination and provides guidance to the driver. In this way, the transmission terminal 10 realizes a function as a car navigation system.
[0261] The car navigation device 200 is connected to the network 2 by the transmission terminal 10. For example, the car navigation device 200 is connected to the network 2 by using a mobile phone to connect to the network 2 via a mobile phone line (including wireless communication that uses antenna power from a base station to cover a relatively wide area, mainly outdoors) or the like.
[0262] The transmission terminal 10 as the car navigation device 200-2 displays a screen as shown in Fig. 29 on the display 120. The destination list is a display screen including a destination terminal candidate terminal ID 1100-2, a terminal name 1100-3, and icons 1100-4a to 1100-4c reflecting status information in a destination list frame 1100-1. The passenger selects a destination terminal from the list of destination terminals displayed on the display 120.
[0263] When a session with the destination terminal is established, the transmission terminal 10 (or the car navigation device) can transmit image data, audio data, and the like to the selected destination terminal, or receive image data, audio data, and the like from the selected destination terminal.
[0264] In addition, the user can operate the car navigation device 200 or the transmission terminal 10 to import display data such as road maps and TV images into the transmission terminal 10. The transmission terminal 10 can transmit this display data to other transmission terminals 10 connected to the network 2.
[0265] Furthermore, when a user selects the management terminal 200-1 used by the communicator of the management center 604 from the list of destination terminals displayed on the display 120, image data of the communicator is displayed on the display 120 and sound is output from the speaker. Thus, the user can talk with the communicator as if on a TV phone.
[0266] Therefore, even if the installation location is a mobile body, image data, audio data, and display data can be transmitted and received in the same way as the transmission terminal 10 in an office.
[0267] In the above embodiment, image data and sound data are described as an example of content data, but the present invention is not limited to this and may be tactile data. In this case, the sensation of a user touching one transmission terminal is transmitted to the other transmission terminal.
[0268] Furthermore, the content data may be smell data. In this case, the smell (odor) of one transmission terminal side is transmitted to the other transmission terminal side. The content data may be at least one of image data, sound data, tactile data, and smell data.
[0269] In addition, in the above embodiment, a case where a video conference is held using the transmission system 1 is described, but this is not limited to this, and the system may be used for meetings, general conversations between family members or friends, etc., or for presenting information in one direction. [Explanation of symbols]
[0270] 1. Transmission System 10 Transmission Terminal 11 Transmitter / receiver 12 Operation input reception section 15a Sound input section 15b Sound input section 17 Display control section 18 Auxiliary area display section 19 Storage and readout processing section 30 Relay Device 50 Transmission management device 51 Transmitter / receiver 52 Terminal authentication unit 53 Status Management Unit 54 Terminal Extraction Unit 55 Terminal status acquisition unit 59 Memory / read processing section 120 Display (display device) [Prior art documents] [Patent documents]
[0271] [Patent Document 1] JP 2013-157995 A [Non-patent literature]
[0272] [Non-Patent Document 1] Niimi, "Speech Recognition", Kyoritsu Shuppan, pp.68-72, 1979
Claims
1. A transmission terminal that performs a video conference with a plurality of locations and transmits and receives image and audio data to and from the plurality of locations participating in the video conference, a transmitting / receiving unit that receives the image and the sound data transmitted from the plurality of locations; a display control means for displaying the received image and the number of locations participating in the video conference on a display screen; The display screen includes a plurality of second image areas each displaying the received image, and a first image area that is larger than the second image area, and the first image area displays the image transmitted from a location that is a transmission source of the sound data indicating a voice. Transmission terminal.
2. A transmission terminal as described in claim 1, wherein, when the number of participants taking part in the video conference exceeds a predetermined upper limit, images of a number of locations equal to the upper limit are displayed on the display screen.
3. A transmission terminal as described in claim 1 or 2, wherein the first image area displays an image of the location from which the sound data from which the voice section was detected was transmitted.
4. A transmission terminal for conducting a video conference with a plurality of locations and transmitting and receiving image and audio data to and from the plurality of locations participating in the video conference, a transmitting / receiving unit that receives the image and the sound data transmitted from the plurality of locations; a display control unit that displays the received image and the number of locations participating in the video conference on a display screen; The display screen includes a plurality of second image areas each displaying the received image, and a first image area that is larger than the second image area, and the first image area displays the image transmitted from a location that is a transmission source of the sound data indicating a voice. program.
5. A transmission method performed by a transmission terminal which performs a video conference with a plurality of locations and transmits and receives image and audio data to and from the plurality of locations participating in the video conference, comprising: receiving the image and sound data transmitted from the plurality of locations; displaying the received image and the number of locations participating in the video conference on a display screen; The display screen includes a plurality of second image areas each displaying the received image, and a first image area that is larger than the second image area, and the first image area displays the image transmitted from a location that is a transmission source of the sound data indicating a voice. Transmission method.
