TRANSMISSION SYSTEM, FIRST TRANSMISSION TERMINAL, TRANSMISSION METHOD, DISPLAY METHOD, AND PROGRAM

Abstract

To solve a problem in which, conventionally, image communication or voice communication is only restricted in consideration of an environment around a destination terminal, and image communication is not restricted in consideration of the image communication state of the destination terminal itself, and therefore, the conventional request source terminal performs useless image communication when the destination terminal cannot support image communication.SOLUTION: A management system 50 creates a predetermined communication control message for causing each terminal to control image communication on the basis of image communication state information indicating the image communication state of the terminal, and sends a communication control message for the terminal that controls the image communication. As a result, a transmission management system can cause a predetermined terminal to perform image communication according to the image communication state of the counterpart terminal, thereby preventing unnecessary image communication.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an invention for managing communication of content data for conversation among a plurality of transmission terminals. [Background technology]

[0002] One example of a transmission system that transmits and receives content data between multiple transmission terminals via a relay device is a video conference system that conducts video conferences and the like via a communication network such as the Internet. The need for such video conference systems has increased in recent years with the reduction in travel expenses and travel time. In such video conference systems, multiple video conference terminals, which are an example of transmission terminals, are used. Then, a video conference can be realized by transmitting and receiving image data and audio data between these video conference terminals.

[0003] However, there are environments that are inappropriate for video conferencing, such as private places such as a home. As a measure for video conferencing in such environments, a transmission terminal has been proposed that acquires the environmental state of the other transmission terminal and restricts image communication or audio communication (see Patent Document 1). Summary of the Invention [Problem to be solved by the invention]

[0004] However, the transmission terminal disclosed in Patent Document 1 only restricts image communication or voice communication in consideration of the surrounding environment of the other transmission terminal, and does not restrict image communication in consideration of the image communication state of the other transmission terminal itself. In other words, the transmission terminal disclosed in Patent Document 1 does not restrict image communication in consideration of the image communication state of the other transmission terminal, such as a state in which the other transmission terminal has a voice communication function but does not have an image communication function, or a state in which the other transmission terminal has an image communication function but does not perform image communication. Therefore, for example, even if the transmission terminal performs image communication when the other transmission terminal is in the image communication state described above, a problem occurs in that the other transmission terminal performs useless image communication because the other transmission terminal cannot handle image communication. [Means for solving the problem]

[0005] The invention of claim 1 is a transmission management system that manages image communication among a plurality of transmission terminals capable of at least voice communication, characterized in having a terminal management means that manages image communication status information indicating the image communication status of each transmission terminal for each terminal identification information for identifying each of the plurality of transmission terminals, a creation means that creates a predetermined communication control message for causing a predetermined transmission terminal to control image communication based on the image communication status information, and a transmission means that transmits the created communication control message to a transmission terminal among the plurality of transmission terminals that is to control the image communication.

[0006] The invention of claim 6 is a transmission system having a transmission management system as described in claim 2 or 3 and the transmission terminal, characterized in that the transmission terminal has a receiving means for receiving a communication control message transmitted from the transmission management system, and a display control means for displaying a display message based on the received communication control message on a specified display means.

[0007] The invention according to claim 10 is a program for causing the transmission management system to function as each of the means according to any one of claims 1 to 5.

[0008] The invention according to claim 11 is a program providing system that provides the program according to claim 10 to the transmission management system via a communication network.

[0009] The invention according to claim 12 is a maintenance system for performing maintenance of the transmission management system according to any one of claims 1 to 5. Effect of the Invention

[0010] As described above, according to the present invention, the transmission management system creates a predetermined communication control message for causing a predetermined transmission terminal to control image communication based on image communication status information indicating the image communication status of each transmission terminal, and transmits the communication control message to the transmission terminal that controls the image communication. As a result, the transmission management system can cause the predetermined transmission terminal to perform image communication according to the image communication status of the other transmission terminal, thereby preventing unnecessary image communication. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a schematic diagram of a transmission system according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a conceptual diagram showing the state of transmission and reception of image data, audio data, and various management information in the transmission system. [Diagram 3] FIG. 3 is a conceptual diagram illustrating the image quality of image data. [Figure 4] FIG. 4 is an external view of the terminal according to this embodiment. [Diagram 5] FIG. 5 is a hardware configuration diagram of a terminal according to this embodiment. [Figure 6] FIG. 6 is a hardware configuration diagram of the management system, the relay device, or the program providing system according to the present embodiment. [Figure 7]FIG. 7 is a functional block diagram of each terminal, device, and system constituting the transmission system according to this embodiment. [Figure 8] FIG. 8 is a functional configuration diagram of the final narrowing down unit. [Figure 9] FIG. 9 is a functional configuration diagram of the primary narrowing down unit. [Figure 10] FIG. 10 is a conceptual diagram showing a change quality management table. [Figure 11] FIG. 11 is a conceptual diagram showing a relay device management table. [Figure 12] FIG. 12 is a conceptual diagram showing a terminal authentication management table. [Figure 13] FIG. 13 is a conceptual diagram showing the terminal management table. [Figure 14] FIG. 14 is a conceptual diagram showing a destination list management table. [Figure 15] FIG. 15 is a conceptual diagram showing a session management table. [Figure 16] FIG. 16 is a conceptual diagram showing an address priority management table. [Figure 17] FIG. 17 is a conceptual diagram showing a transmission speed priority management table. [Figure 18] FIG. 18 is a conceptual diagram showing a quality control table. [Figure 19] FIG. 19 is a sequence diagram showing a process for managing state information indicating the operating state of each relay device. [Figure 20] FIG. 20 is a sequence diagram showing a preparatory process for starting remote communication between terminals. [Figure 21] FIG. 21 is a sequence diagram showing a process of narrowing down relay devices. [Figure 22] FIG. 22 is a process flow diagram showing a process for narrowing down relay devices. [Diagram 23] FIG. 23 is a diagram showing a calculation state of priority points when narrowing down relay devices. [Figure 24] FIG. 24 is a sequence diagram showing a process in which a transmission terminal selects a relay device according to the first embodiment of the present invention. [Diagram 25] FIG. 25 is a process flow diagram showing a process for selecting a relay device at a transmission terminal. [Figure 26] FIG. 26 is a sequence diagram showing the process of creating, transmitting and receiving a communication control message. [Figure 27] FIG. 27 is a flow diagram mainly showing a process for creating a communication control message. [Figure 28] FIG. 28 is a flow chart showing in detail a part of the process for creating a communication control message. [Figure 29] FIG. 29 is a diagram showing an example of a video conference screen displayed on a display of the transmission terminal. [Diagram 30] FIG. 30 shows an example of a video conference screen displayed on a display of the transmission terminal. [Diagram 31] FIG. 31 is a diagram showing an example of a video conference screen displayed on a display on the transmission terminal side. [Diagram 32] FIG. 32 shows an example of a video conference screen displayed on a display of the transmission terminal. [Diagram 33] FIG. 33 is a sequence diagram showing a process of transmitting and receiving image data and audio data between transmission terminals. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the present invention will be described with reference to Fig. 1 to Fig. 33. In this embodiment, a transmission terminal performs voice communication by transmitting voice data to a transmission terminal on the other side, and performs image communication by transmitting image data to the transmission terminal on the other side.

[0013] <<Overall configuration of the embodiment>> Fig. 1 is a schematic diagram of a transmission system 1 according to an embodiment of the present invention. Fig. 2 is a conceptual diagram showing the state of transmission and reception of image data, audio data, and various management information in the transmission system. Fig. 3 is a conceptual diagram explaining the image quality of image data.

[0014] The transmission system also includes a data providing system that transmits content data in one direction from one transmission terminal to another transmission terminal via a transmission management system, and a communication system that transmits information, emotions, etc. between multiple transmission terminals via a transmission management system. This communication system is a system for transmitting information, emotions, etc. between multiple communication terminals (corresponding to "transmission terminals") via a communication management system (corresponding to "transmission management system"), and examples of this include a video conference system and a video telephone system.

[0015] In this embodiment, a transmission system, a transmission management system, and a transmission terminal are described assuming a television conference system as an example of a communication system, a television conference management system as an example of a communication management system, and a television conference terminal as an example of a communication terminal. That is, the transmission terminal and the transmission management system of the present invention are not only applied to television conference systems, but also to communication systems or transmission systems. Note that, although a "television conference" is described in this embodiment, it is sometimes called a "video conference," and the two are the same thing.

[0016] First, the transmission system 1 shown in FIG. 1 is constructed by a plurality of transmission terminals (10aa, 10ab,...), displays (120aa, 120ab,...) as display means for each transmission terminal (10aa, 10ab,...), a plurality of relay devices (30a, 30b, 30c, 30d), a transmission management system 50, a program providing system 90, and a maintenance system 100.

[0017] The multiple terminals 10 perform transmission by sending and receiving image data and audio data as examples of content data.

[0018] In the following, a "transmission terminal" is simply referred to as a "terminal", and a "transmission management system" is simply referred to as a "management system". Any of the multiple terminals (10aa, 10ab,...) is referred to as a "terminal 10", any of the multiple displays (120aa, 120ab,...) is referred to as a "display 120", and any of the multiple relay devices (30a, 30b, 30c, 30d) is referred to as a "relay device 30". Furthermore, a terminal that is a request source that requests the start of a video conference is referred to as a "request source terminal", and a terminal that is a destination (relay destination) that is the request destination is referred to as a "destination terminal".

[0019] 2, in the transmission system 1, a management information session sei for transmitting and receiving various management information is established between the request source terminal and the destination terminal via the management system 50. In addition, four sessions for transmitting and receiving each of four types of data, high-resolution image data, medium-resolution image data, low-resolution image data, and audio data, are established between the request source terminal and the destination terminal via the relay device 30. Here, these four sessions are collectively shown as an image / audio data session sed.

[0020] Here, the image resolution of the image data handled in this embodiment will be described. As shown in FIG. 3(a), there is a low-resolution image consisting of 160 pixels horizontally and 120 pixels vertically, which is a base image; as shown in FIG. 3(b), there is a medium-resolution image consisting of 320 pixels horizontally and 240 pixels vertically; and as shown in FIG. 3(c), there is a high-resolution image consisting of 640 pixels horizontally and 480 pixels vertically. Among these, when passing through a narrowband path, low-quality image data consisting only of low-resolution image data which is a base image is relayed. When the bandwidth is relatively wide, low-quality image data consisting of low-resolution image data which is a base image and medium-quality image data consisting of medium-resolution image data is relayed. Also, when the bandwidth is very wide, high-quality image data consisting of low-resolution image data which is a base image, medium-resolution image data, and high-resolution image data is relayed.

[0021] 1 relays content data between a plurality of terminals 10. A management system 50 centrally manages login authentication from the terminals 10, management of call status of the terminals 10, management of a destination list, and the like, as well as the communication status of the relay device 30. Note that the images in the image data may be moving images or still images, or may be both moving images and still images.

[0022] The multiple routers (70a, 70b, 70c, 70d, 70ab, 70cd) select the optimal route for image data and audio data. In the following, any one of the routers (70a, 70b, 70c, 70d, 70ab, 70cd) is referred to as a "router 70."

[0023] The program providing system 90 includes a hard disk (HD) 204 described below, which stores terminal programs for causing the terminal 10 to realize various functions (or for causing the terminal 10 to function as various means), and can transmit the terminal programs to the terminal 10. The HD 204 of the program providing system 90 also stores relay device programs for causing the relay device 30 to realize various functions (or for causing the relay device 30 to function as various means), and can transmit the relay device programs to the relay device 30. The HD 204 of the program providing system 90 also stores transmission management programs for causing the management system 50 to realize various functions (or for causing the management system 50 to function as various means), and can transmit the transmission management programs to the management system 50.

[0024] The maintenance system 100 is a computer for maintaining, managing, or repairing at least one of the terminal 10, the relay device 30, the management system 50, and the program providing system 90. For example, when the maintenance system 100 is installed in Japan and the terminal 10, the relay device 30, the management system 50, or the program providing system 90 is installed outside the country, the maintenance system 100 remotely performs maintenance such as maintenance, management, or repair of at least one of the terminal 10, the relay device 30, the management system 50, and the program providing system 90 via the communication network 2. In addition, the maintenance system 100 performs maintenance such as management of the model number, serial number, sales destination, maintenance inspection, or failure history of at least one of the terminal 10, the relay device 30, the management system 50, and the program providing system 90 without going through the communication network 2.

[0025] Meanwhile, the terminals (10aa, 10ab, 10ac,...), the relay device 30a, and the router 70a are communicatively connected by the LAN 2a. The terminals (10ba, 10bb, 10bc,...), the relay device 30b, and the router 70b are communicatively connected by the LAN 2b. Furthermore, the LAN 2a and the LAN 2b are communicatively connected by a dedicated line 2ab including the router 70ab, and are constructed within a predetermined region A. For example, the region A is Japan, the LAN 2a is constructed within an office in Tokyo, and the LAN 2b is constructed within an office in Osaka.

[0026] On the other hand, the terminals (10ca, 10cb, 10cc, . . .), the relay device 30c, and the router 70c are communicatively connected by the LAN 2c. The terminals 10d (10da, 10db, 10dc, . . .), the relay device 30d, and the router 70d are communicatively connected by the LAN 2d. The LAN 2c and the LAN 2d are communicatively connected by a dedicated line 2cd including the router 70cd, and are constructed in a predetermined region B. For example, the region B is the United States of America, the LAN 2c is constructed in an office in New York, and the LAN 2d is constructed in an office in Washington DC. The regions A and B are communicatively connected from the routers (70ab, 70cd) via the Internet 2i, respectively.

[0027] Furthermore, the management system 50 and the program providing system 90 are communicatively connected to the terminal 10 and the relay device 30 via the Internet 2i. The management system 50 and the program providing system 90 may be installed in the area A or area B, or may be installed in an area other than these.

[0028] In this embodiment, the communication network 2 of this embodiment is constructed by the LAN 2a, the LAN 2b, the dedicated line 2ab, the Internet 2i, the dedicated line 2cd, the LAN 2c, and the LAN 2d. The communication network 2 may include locations where not only wired communication but also wireless communication such as WiFi (Wireless Fidelity) and Bluetooth (registered trademark) is performed.

[0029] 1, four sets of numbers shown under each terminal 10, each relay device 30, management system 50, each router 70, and program providing system 90 simply indicate general IPv4 IP addresses. For example, the IP address of terminal 10aa is "1.2.1.3." Also, IPv6 may be used instead of IPv4, but for the sake of simplicity, IPv4 will be used in the following description.

[0030] Each terminal 10 may be used not only for calls between multiple offices or between different rooms in the same office, but also for calls within the same room, or calls between indoors and outdoors or between outdoors and outdoors. When each terminal 10 is used outdoors, wireless communication such as a mobile phone communication network is performed.

[0031] <<Hardware configuration of the embodiment>> Next, the hardware configuration of this embodiment will be described. Fig. 4 is an external view of the terminal 10 according to this embodiment. In the following description, the longitudinal direction of the terminal 10 is defined as the X-axis direction, the direction perpendicular to the X-axis direction in a horizontal plane is defined as the Y-axis direction, and the direction perpendicular to the X-axis and Y-axis directions (vertical direction) is defined as the Z-axis direction.

[0032] As shown in Fig. 4, the terminal 10 includes a housing 1100, an arm 1200, and a camera housing 1300. A front wall surface 1110 of the housing 1100 is provided with an intake surface (not shown) formed with a plurality of intake holes, and a rear wall surface 1120 of the housing 1100 is provided with an exhaust surface 1121 formed with a plurality of exhaust holes. This allows the outside air behind the terminal 10 to be taken in through the intake surface (not shown) by driving a cooling fan built into the housing 1100, and to be exhausted to the rear of the terminal 10 through the exhaust surface 1121. A sound pickup hole 1131 is formed in a right wall surface 1130 of the housing 1100, and sounds such as voice, sounds, and noise can be picked up by a built-in microphone 114 described later.

[0033] An operation panel 1150 is formed on the right side wall 1130 of the housing 1100. This operation panel 1150 is provided with a plurality of operation buttons (108a to 108e), a power switch 109, and an alarm lamp 119, which will be described later, and also has a sound output surface 1151 formed with a plurality of sound output holes for passing sound output from a built-in speaker 115, which will be described later. In addition, a storage section 1160 is formed on the left side wall 1140 of the housing 1100 as a recess for storing the arm 1200 and the camera housing 1300. A plurality of connection ports (1132a to 1132c) for electrically connecting cables to an external device connection I / F 118, which will be described later, are provided on the right side wall 1130 of the housing 1100. Meanwhile, a left wall surface 1140 of the housing 1100 is provided with a connection port (not shown) for electrically connecting a cable 120c for the display 120 to an external device connection I / F 118 (described later).

[0034] In the following, "operation button 108" will be used to refer to any of the operation buttons (108a to 108e), and "connection port 1132" will be used to refer to any of the connection ports (1132a to 1132c).

[0035] Next, arm 1200 is attached to housing 1100 via torque hinge 1210, and configured so that arm 1200 can rotate up and down within a range of tilt angle θ1 of 135 degrees relative to housing 1100. Fig. 4 shows a state where tilt angle θ1 is 90 degrees.

[0036] The camera housing 1300 is provided with a built-in camera 112, which will be described later, and can capture images of a user, a document, a room, and the like. A torque hinge 1310 is also formed in the camera housing 1300. The camera housing 1300 is attached to the arm 1200 via the torque hinge 1310. The camera housing 1300 is attached to the arm 1200 via the torque hinge 1310, and is configured to be rotatable in the up, down, left, and right directions with respect to the arm 1200 within a pan angle θ2 range of ±180 degrees, with the state shown in FIG. 4 being 0 degrees, and within a tilt angle θ3 range of ±45 degrees.

[0037] In addition, since the relay device 30, the management system 50, and the program providing system 90 each have the same appearance as a general server computer, a description of their appearance will be omitted.

[0038] Fig. 5 is a hardware configuration diagram of the terminal 10 according to the present embodiment of the present invention. As shown in Fig. 5, the terminal 10 according to the present embodiment includes a CPU (Central Processing Unit) 101 for controlling the operation of the entire terminal 10, a ROM (Read Only Memory) 102 for storing programs used to drive the CPU 101, such as an IPL (Initial Program Loader), a RAM (Random Access Memory) 103 for use as a work area for the CPU 101, a flash memory 104 for storing various data such as terminal programs, image data, and audio data, a solid state drive (SSD) 105 for controlling the reading or writing of various data from the flash memory 104 under the control of the CPU 101, a media drive 107 for controlling the reading or writing (storage) of data from a recording medium 106 such as a flash memory, an operation button 108 operated when selecting a destination of the terminal 10, a power switch 109 for switching the power of the terminal 10 ON / OFF, and a network I / F (Interface) 111 for transmitting data using the communication network 2.

[0039] The terminal 10 also includes a built-in camera 112 that captures an image of a subject under the control of the CPU 101 to obtain image data, an image sensor I / F 113 that controls the operation of the camera 112, a built-in microphone 114 that inputs voice, a built-in speaker 115 that outputs voice, an audio input / output I / F 116 that processes input and output of audio signals between the microphone 114 and the speaker 115 under the control of the CPU 101, a display I / F 117 that transmits image data to an external display 120 under the control of the CPU 101, an external device connection I / F 118 for connecting various external devices, an alarm lamp 119 that notifies of abnormalities in various functions of the terminal 10, and a bus line 110 such as an address bus and a data bus for electrically connecting the above components as shown in Fig. 5. Note that some models of the terminal 10 do not include the camera 112 and the image sensor I / F 113.

[0040] The display 120 is a display unit configured with liquid crystal or organic electroluminescence (EL) that displays an image of a subject, operation icons, etc. The display 120 is also connected to the display I / F 117 by a cable 120c. This cable 120c may be a cable for analog RGB (VGA) signals, a cable for component video, or a cable for HDMI (registered trademark) (High-Definition Multimedia Interface) or DVI (Digital Video Interactive) signals.

[0041] The camera 112 includes a lens and a solid-state imaging element that converts light into electric charges to digitize an image (video) of a subject. As the solid-state imaging element, a complementary metal oxide semiconductor (CMOS), a charge coupled device (CCD), or the like is used.

[0042] External devices such as an external camera, an external microphone, and an external speaker can be connected to the external device connection I / F 118 by a USB (Universal Serial Bus) cable, etc. When an external camera is connected, the external camera is driven in priority over the built-in camera 112 under the control of the CPU 101. Similarly, when an external microphone or an external speaker is connected, the external microphone or the external speaker is driven in priority over the built-in microphone 114 or the built-in speaker 115, respectively, under the control of the CPU 101.

[0043] The recording medium 106 is detachable from the terminal 10. In addition, as long as it is a non-volatile memory that reads or writes data under the control of the CPU 101, it is not limited to the flash memory 104, and an EEPROM (Electrically Erasable and Programmable ROM) or the like may be used.

[0044] Furthermore, the terminal program may be distributed by being recorded in a computer-readable recording medium such as the recording medium 106 as a file in an installable or executable format. The terminal program may be stored in the ROM 102 instead of the flash memory 104.

[0045] 6 is a hardware configuration diagram of a management system according to the present embodiment of the present invention. The management system 50 includes a CPU 201 for controlling the operation of the entire management system 50, a ROM 202 storing a program used to drive the CPU 201 such as an IPL, a RAM 203 used as a work area for the CPU 201, a HD 204 for storing various data such as a transmission management program, a Hard Disk Drive (HDD) 205 for controlling reading or writing of various data from the HD 204 under the control of the CPU 201, a media drive 207 for controlling reading or writing (storing) of data from a recording medium 206 such as a flash memory, a display 208 for displaying various information such as a cursor, a menu, a window, a character, or an image, a network I / F 209 for transmitting data using a communication network 2, a keyboard 211 having a plurality of keys for inputting characters, numerical values, various instructions, etc., a mouse 212 for selecting and executing various instructions, selecting a processing target, moving the cursor, etc., a CD-ROM (Compact Disc Read Only Memory) 212 as an example of a removable recording medium, and a USB 213 for connecting the USB 213 to the USB 213. The computer-readable medium includes a CD-ROM drive 214 that controls the reading and writing of various data from and to a CD-ROM (CD-ROM) 213, and bus lines 210 such as an address bus and a data bus for electrically connecting the above components as shown in FIG.

[0046] The transmission management program may be distributed by being recorded in a computer-readable recording medium such as the recording medium 206 or the CD-ROM 213 as a file in an installable or executable format. The transmission management program may be stored in the ROM 202 instead of the HD 204.

[0047] Moreover, the relay device 30 has the same hardware configuration as the management system 50, and therefore a description thereof will be omitted. However, a relay device program for controlling the relay device 30 is recorded on the HD 204. In this case, too, the relay device program may be recorded in a computer-readable recording medium such as the recording medium 206 or the CD-ROM 213 as a file in an installable or executable format and distributed. Moreover, the relay device program may be stored in the ROM 202 instead of the HD 204.

[0048] Furthermore, the program provision system 90 and the maintenance system 100 have the same hardware configuration as the management system 50, and therefore a description thereof will be omitted. However, a program provision program for controlling the program provision system 90 is recorded on the HD 204. In this case as well, the program provision program may be recorded in a computer-readable recording medium such as the recording medium 206 or CD-ROM 213 as a file in an installable or executable format and distributed. Furthermore, the program for the program provision system may be stored in the ROM 202 instead of the HD 204.

[0049] As another example of the removable recording medium, the information may be recorded on a computer-readable recording medium such as a CD-R (Compact Disc Recordable), a DVD (Digital Versatile Disk), or a Blu-ray disc and provided.

[0050] <<Functional configuration of the embodiment>> Next, the functional configuration of this embodiment will be described. In this embodiment, when a delay occurs in receiving image data at the terminal 10 as the destination (relay destination), the relay device 30 changes the image resolution of the image data and then transmits the image data to the terminal 10 as the relay destination.

[0051] Fig. 7 is a functional block diagram of each terminal, device, and system constituting the transmission system 1 of the present embodiment. In Fig. 7, a terminal 10, a relay device 30, and a management system 50 are connected so as to be able to perform data communication via a communication network 2. In addition, the program providing system 90 shown in Fig. 1 is omitted in Fig. 7 because it is not directly related to the communication of the video conference.

[0052] <Device Functionality Configuration> The terminal 10 has a transmission / reception unit 11, an operation input reception unit 12, a login request unit 13, an imaging unit 14, an audio input unit 15a, an audio output unit 15b, a final selection unit 16, a display control unit 17, a delay detection unit 18, a storage / readout processing unit 19, and a message creation unit 20. Each of these units is a function or a means that is realized by operating any of the components shown in Fig. 5 by an instruction from the CPU 101 according to a terminal program expanded from the flash memory 104 onto the RAM 103. The terminal 10 also has a memory unit 1000 constructed by the RAM 103 shown in Fig. 5 and the flash memory 104 shown in Fig. 5.

[0053] (Device functional configuration) Next, each functional configuration of the terminal 10 will be described in detail with reference to Fig. 5 and Fig. 7. In the following, when describing each functional configuration of the terminal 10, the relationship between each component shown in Fig. 5 and the main components for realizing each functional configuration of the terminal 10 will also be described.

[0054] The transmitting / receiving unit 11 of the terminal 10 shown in FIG. 7 is realized by the command from the CPU 101 shown in FIG. 5 and the network I / F 111 shown in FIG. 5, and transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 2. Before starting a call with a desired destination terminal, the transmitting / receiving unit 11 starts receiving each state information indicating the state of each terminal as a destination candidate from the management system 50. Note that this state information indicates not only the operating state of each terminal 10 (online or offline state), but also detailed states such as whether the terminal 10 is online but can still make a call, whether the terminal is in a call, whether the terminal is away from its desk, etc. Also, this state information indicates not only the operating state of each terminal 10, but also various states such as the cable 120c being disconnected from the terminal 10, the terminal 10 outputting sound but not outputting an image, or the terminal 10 not outputting sound (MUTE). Below, a case where the state information indicates an operating state will be described as an example.

[0055] The operation input reception unit 12 is realized by commands from the CPU 101 shown in Fig. 5, and the operation buttons 108 and power switch 109 shown in Fig. 5, and receives various inputs from the user. For example, when the user turns on the power switch 109 shown in Fig. 5, the operation input reception unit 12 shown in Fig. 7 receives the power ON command and turns on the power.

[0056] 5, and automatically transmits login request information indicating a login request and the current IP address of the requesting terminal from the transmitting / receiving unit 11 to the management system 50 via the communication network 2 when the power is turned ON. When the user turns the power switch 109 from ON to OFF, the transmitting / receiving unit 11 transmits status information to the management system 50 indicating that the power is to be turned OFF, and then the operation input receiving unit 12 turns the power completely OFF. This allows the management system 50 to know that the terminal 10 has been turned OFF from ON.

[0057] The imaging unit 14 is realized by commands from the CPU 101 shown in Fig. 5, and the camera 112 and the image sensor I / F 113 shown in Fig. 5, captures an image of a subject, and outputs image data obtained by capturing the image. Note that some models of the terminal 10 do not include the imaging unit 14.

[0058] The audio input unit 15a is realized by commands from the CPU 101 shown in Fig. 5 and the audio input / output I / F 116 shown in Fig. 5, and inputs audio data related to the audio signal after the user's voice is converted into an audio signal by the microphone 114. The audio output unit 15b is realized by commands from the CPU 101 shown in Fig. 5 and the audio input / output I / F 116 shown in Fig. 5, and outputs an audio signal related to the audio data to the speaker, causing the speaker 115 to output the audio.

[0059] In order to perform a final narrowing-down process that narrows down multiple relay devices 30 to one relay device 30, the final narrowing-down unit 16 realizes a measurement unit 16a, a calculation unit 16b, and a final selection unit 16c as shown in FIG. 8 based on instructions from the CPU 101 shown in FIG. 5.

[0060] Of these, the measurement unit 16a measures the reception date and time when the pre-transmission information is received by the transmission / reception unit 11 for each piece of pre-transmission information described below that is received by the transmission / reception unit 11. For each piece of pre-transmission information whose reception date and time is measured by the measurement unit 16a, the calculation unit 16b calculates the required time (T) from transmission to reception of the pre-transmission information based on the difference between the measured reception time and the transmission date and time included in the pre-transmission information. The final selection unit 16c selects one relay device by selecting the relay device 30 that relayed the pre-transmission information that required the shortest required time among the required times calculated by the calculation unit 16b.

[0061] The display control unit 17 is realized by commands from the CPU 101 shown in Fig. 5 and the display I / F 117 shown in Fig. 5, and performs control for combining received image data with different resolutions and transmitting the combined image data to the display 120, as described below. The display control unit 17 can also transmit information on the destination list received from the management system 50 to the display 120, and display the destination list on the display 120. Furthermore, the display control unit 17 causes the display 120 to display screens such as those shown in Figs. 29 to 32. The screens in Figs. 29 to 32 will be described later.

[0062] The delay detection unit 18 is realized by a command from the CPU 101 shown in FIG. 5, and detects the delay time (ms) of image data or audio data transmitted from another terminal 10 via the relay device 30.

[0063] 5 and the SSD 105 shown in Fig. 5, or is realized by an instruction from the CPU 101, and performs processing such as storing various data in the storage unit 1000 and reading various data stored in the storage unit 1000. A terminal ID (Identification) for identifying the terminal 10, a password, etc. are stored in this storage unit 1000.

[0064] The message creation unit 20 is realized by an instruction from the CPU 101 shown in Fig. 5, reads pre-stored image communication status information from the storage unit 1000, and creates an image communication status message (m) including this image communication status information. This image communication status information and the image communication status message (m) will be described later.

[0065] The storage unit 1000 also stores image communication status information, which will be described later. Furthermore, image data and audio data received during a call with a destination terminal are overwritten and stored in the storage unit 1000 each time the data is received. Of these, images are displayed on the display 120 using the image data before being overwritten, and audio is output from the speaker 115 using the audio data before being overwritten.

[0066] The terminal ID in this embodiment and the relay device ID described later indicate identification information such as language, characters, symbols, or various marks used to uniquely identify the terminal 10 and the relay device 30. The terminal ID and the relay device ID may be identification information that combines at least two of the above languages, characters, symbols, and various marks.

[0067] <Functional configuration of relay device> The relay device 30 has a transmitting / receiving unit 31, a state detecting unit 32, a data quality checking unit 33, a change quality managing unit 34, a data quality changing unit 35, and a storing / reading processing unit 39. Each of these units is a function or a functioning means realized by any of the components shown in Fig. 6 operating in response to an instruction from the CPU 201 in accordance with the relay device program loaded from the HD 204 onto the RAM 203. The relay device 30 also has a memory unit 3000 constructed by the RAM 203 shown in Fig. 6 and / or the HD 204 shown in Fig. 6. Fig. 10 is a conceptual diagram showing a change quality management table.

[0068] (Change quality control table) In the storage unit 3000, a change quality management DB (Data Base) 3001 configured by a change quality management table as shown in Fig. 10 is constructed. In the change quality management table, the IP address of the terminal 10 as the relay destination (destination) of the image data and the image quality of the image data relayed by the relay device 30 to this relay destination are associated and managed.

[0069] (Functional configuration of relay device) Next, a detailed description will be given of each functional configuration of the relay device 30. In the following, in describing each functional configuration of the relay device 30, the relationship between each component shown in FIG. 6 and the main components for realizing each functional configuration of the relay device 30 will also be described.

[0070] The transceiver unit 31 of the relay device 30 shown in Figure 7 is realized by instructions from the CPU 201 shown in Figure 6 and the network I / F 209 shown in Figure 6, and transmits and receives various data (or information) with other terminals, devices, or systems via the communication network 2.

[0071] The status detection unit 32 is realized by an instruction from the CPU 201 shown in Fig. 6, and detects the operation status of the relay device 30 having this status detection unit 32. The operation status includes "online", "offline", "busy", and "temporarily suspended".

[0072] The data quality confirmation unit 33 is realized by an instruction from the CPU 201 shown in Figure 6, and confirms the image quality of the relayed image data by searching the change quality management DB 3001 (see Figure 10) using the IP address of the destination terminal as a search key and extracting the image quality of the corresponding image data to be relayed.

[0073] The change quality management unit 34 is realized by an instruction from the CPU 201 shown in Fig. 6, and changes the contents of the change quality management DB 3001 based on quality information (described later) sent from the management system 50. For example, during a video conference between a request source terminal (terminal 10aa) with a terminal ID of "01aa" and a destination terminal (terminal 10db) with a terminal ID of "01db" by transmitting and receiving high-quality image data, if a delay in receiving image data occurs at the destination terminal (terminal 10db) due to a request source terminal (terminal 10bb) and a destination terminal (terminal 10ca) holding another video conference starting a video conference via the communication network 2, the relay device 30 lowers the image quality of the image data that has been relayed up until now from high image quality to medium image quality. In such a case, the contents of the change quality management DB 3001 are changed based on the quality information indicating medium image quality so that the image quality of the image data relayed by the relay device 30 is lowered from high image quality to medium image quality.

[0074] The data quality change unit 35 is realized by a command from the CPU 201 shown in FIG. 6, and changes the image quality of the image data sent from the source terminal based on the contents of the changed quality management DB 3001 described above.

[0075] The storage / readout processing unit 39 is realized by instructions from the CPU 201 shown in FIG. 6 and the HDD 205 shown in FIG. 6, and performs processing such as storing various data in the memory unit 3000 and reading out various data stored in the memory unit 3000.

[0076] <Management system functional configuration> The management system 50 has a transmission / reception unit 51, a terminal authentication unit 52, a status management unit 53, a terminal extraction unit 54, a terminal status extraction unit 55, a primary screening unit 56, a session management unit 57, a quality determination unit 58, a storage / readout processing unit 59, a delay time management unit 60, and a message creation unit 61. Each of these units is a function or a functioning means realized by any of the components shown in Fig. 6 operating in response to an instruction from the CPU 201 in accordance with the management system program loaded from the HD 204 onto the RAM 203. The management system 50 also has a memory unit 5000 constructed by the HD 204 shown in Fig. 6.

[0077] FIG. 8 is a functional configuration diagram of the final narrowing down section. FIG. 9 is a functional configuration diagram of the primary narrowing down section. FIG. 11 is a conceptual diagram showing a relay device management table. FIG. 12 is a conceptual diagram showing a terminal authentication management table. FIG. 13 is a conceptual diagram showing a terminal management table. FIG. 14 is a conceptual diagram showing a destination list management table. FIG. 15 is a conceptual diagram showing a session management table. FIG. 16 is a conceptual diagram showing an address priority management table. FIG. 17 is a conceptual diagram showing a transmission speed priority management table. FIG. 18 is a conceptual diagram showing a quality management table.

[0078] (Relay device management table) In the storage unit 5000, a relay device management DB 5001 is constructed, which is configured by a relay device management table as shown in Fig. 11. In this relay device management table, for each relay device ID of each relay device 30, the operation state of each relay device 30, the reception date and time when the status information indicating the operation state was received by the management system 50, the IP address of the relay device 30, and the maximum data transmission rate (Mbps) of the relay device 30 are associated and managed. For example, in the relay device management table shown in Fig. 11, it is shown that the operation state of the relay device 30a with the relay device ID "111a" is "online", the date and time when the status information was received by the management system 50 is "13:00 on Nov. 10, 2009", the IP address of this relay device 30a is "1.2.1.2", and the maximum data transmission rate of this relay device 30a is "100Mbps".

[0079] (Terminal authentication management table) Furthermore, the storage unit 5000 has constructed a terminal authentication management DB 5002 that is configured by a terminal authentication management table as shown in Fig. 12. In this terminal authentication management table, each password is associated with each terminal ID of all terminals 10 managed by the management system 50. For example, the terminal authentication management table shown in Fig. 12 indicates that the terminal ID of the terminal 10aa is "01aa" and the password is "aaaa".

[0080] (Terminal Management Table) In addition, the storage unit 5000 has a terminal management DB 5003 configured by a terminal management table as shown in Fig. 13. In this terminal management table, for each terminal ID of each terminal 10, the destination name when each terminal 10 is the destination, the operation state of each terminal 10, the image communication state described later, the reception date and time when the login request information described later is received by the management system 50, and the IP address of the terminal 10 are associated and managed. For example, the terminal management table shown in Fig. 13 indicates that the terminal 10aa with the terminal ID "01aa" has the terminal name "Japan Tokyo Office AA Terminal", the operation state "online (transmitting)", the image communication state "ON", the date and time when the login request information was received by the management system 50 is "November 10, 2009, 13:40", and the IP address of this terminal 10aa is "1.2.1.3".

[0081] Here, the image communication status information will be explained. The image communication status information indicates the status of the terminal 10 regarding image communication, and there are the following four patterns. ON: Terminal 10 is in image communication OFF: The terminal 10 is in a state where image communication is stopped (however, the terminal 10 has an image communication function, which is the imaging unit 14 for performing image communication). NA: A state in which the terminal 10 does not have an image communication function (the terminal 10 does not have an image communication function, which is the imaging unit 14 for performing image communication). DC: Image communication status is unknown (the image communication status is not sent from the terminal 10 because the terminal 10 is offline or the terminal 10 is in the initial state). Among these, the storage unit 1000 initially stores image communication status information indicating DC, and the imaging unit 14 overwrites the image communication status information indicating ON or OFF from DC. The image communication status information indicating NA is stored in advance in the storage unit 1000 at the time of shipment of the terminal 10 from the factory.

[0082] (Destination list management table) Furthermore, in the storage unit 5000, a destination list management DB 5004 is constructed, which is configured by a destination list management table as shown in Fig. 14. In this destination list management table, all the terminal IDs of destination terminals registered as candidates for destination terminals are managed in association with the terminal ID of a request source terminal requesting the start of a call in a video conference. For example, in the destination list management table shown in Fig. 14, it is shown that the candidates for destination terminals (terminal 10db) to which a request source terminal (terminal 10aa) with a terminal ID of "01aa" can request the start of a call in a video conference are the terminal 10ab with a terminal ID of "01ab", the terminal 10ba with a terminal ID of "01ba", and the terminal 10bb with a terminal ID of "01bb". The candidates for destination terminals are updated by being added or deleted in response to an addition or deletion request from any request source terminal to the management system 50.

[0083] (Session management table) In addition, in the storage unit 5000, a session management DB 5005 configured by a session management table as shown in Fig. 15 is constructed. In this session management table, for each selection session ID used to execute a session for selecting a relay device 30, the relay device ID of the relay device 30 used to relay image data and audio data, the terminal ID of the request source terminal, the terminal ID of the destination terminal, the reception delay time (ms) when the image data is received at the destination terminal, and the reception date and time when the delay information indicating this delay time is sent from the destination terminal and received by the management system 50 are associated and managed. For example, in the session management table shown in FIG. 15, the relay device 30a (relay device ID "111a") selected in the session executed using the selection session ID "se1" relays image data and audio data between a request source terminal (terminal 10aa) with a terminal ID of "01aa" and a destination terminal (terminal 10db) with a terminal ID of "01db", and the delay time of the image data at the destination terminal (terminal 10db) at "14:00 on November 10, 2009" is "200 (ms)". When a video conference is held between two terminals 10, the reception date and time of the delay information may be managed based on the delay information transmitted from the request source terminal instead of the destination terminal. However, when a video conference is held between three or more terminals 10, the reception date and time of the delay information is managed based on the delay information transmitted from the terminal 10 that receives the image data and audio data.

[0084] (Address priority management table) Furthermore, the storage unit 5000 has a priority management DB 5006 configured by an address priority management table as shown in FIG. 16. In this address priority management table, the more "same" in the four sets of dot address parts of a general IPv4 IP address in any terminal 10 and any relay device 30, the higher the address priority point is managed in association with the sameness or difference of the dot address. This "sameness" means that the dot address parts are the same, and "differentness" means that the dot address parts are different. For example, in the address priority management table shown in FIG. 16, in the case of an IP address in which three values ​​from the top to the bottom of the dot address are the same, the address priority point is "5". In the case of an IP address in which two values ​​from the top to the bottom of the dot address are the same, the address priority point is "3". In this case, whether or not the value of the lowest dot address is the same does not affect the priority. In the case of an IP address in which the highest value of the dot address is the same and the second highest value is different, the address priority point is "1". In this case, whether the third highest and lowest dot address values ​​are the same does not affect the priority. If the most significant value of the dot address is different for an IP address, the address priority point is "0". In this case, whether the second highest, third highest, and lowest dot address values ​​are the same does not affect the priority.

[0085] (Transmission speed priority management table) The priority management DB 5006 constructed in the storage unit 5000 also includes a transmission speed priority management table as shown in FIG. 17. In this transmission speed priority management table, the maximum data transmission speed and the transmission speed priority are associated and managed so that the larger the value of the maximum data transmission speed (Mbps) in the relay device 30, the higher the transmission speed priority point. For example, in the transmission speed priority management table shown in FIG. 17, when the maximum data transmission speed in the relay device 30 is 1000 Mbps or more, the transmission speed priority point is "5". When the maximum data transmission speed in the relay device 30 is 100 Mbps or more and less than 1000 Mbps, the transmission speed priority point is "3". When the maximum data transmission speed in the relay device 30 is 10 Mbps or more and less than 100 Mbps, the transmission speed priority point is "1". When the maximum data transmission speed in the relay device 30 is less than 10 Mbps, the transmission speed priority point is "0".

[0086] (Quality control table) Furthermore, in the storage unit 5000, a quality control DB 5007 configured by a quality control table as shown in Fig. 18 is constructed. In this quality control table, the delay time of the image data and the image quality of the image data (image quality) are managed in association with each other so that the longer the delay time (ms) of the image data at the request source terminal or the destination terminal is, the lower the image quality of the image data relayed by the relay device 30 is.

[0087] (Functional configuration of the management system) Next, a detailed description will be given of each functional configuration of the management system 50. In the following, when describing each functional configuration of the management system 50, the relationship between each component shown in FIG. 6 and the main components for realizing each functional configuration of the management system 50 will also be described.

[0088] The transmission / reception unit 51 is executed by instructions from the CPU 201 shown in FIG. 6 and the network I / F 209 shown in FIG. 6, and transmits and receives various data (or information) with other terminals, devices or systems via the communication network 2.

[0089] The terminal authentication unit 52 is realized by an instruction from the CPU 201 shown in FIG. 6, and performs terminal authentication by searching the terminal authentication management DB 5002 in the memory unit 5000 using the terminal ID and password included in the login request information received via the transmission / reception unit 51 as search keys, and determining whether the same terminal ID and password are managed in the terminal authentication management DB 5002.

[0090] 6, and in order to manage the operating state of a requesting terminal that has made a login request, the status management unit 53 stores and manages the terminal ID of the requesting terminal, the operating state of the requesting terminal, the reception date and time when the login request information was received by the management system 50, and the IP address of the requesting terminal in a terminal management DB 5003 (see FIG. 13) in association with each other. When the user turns the state of the power switch 109 of the terminal 10 from ON to OFF, the status management unit 53 changes the operating state shown as online in the terminal management DB 5003 (see FIG. 13) to offline based on the status information sent from the terminal 10 indicating that the power is to be turned OFF.

[0091] The terminal extraction unit 54 is realized by an instruction from the CPU 201 shown in Fig. 6, and searches the destination list management DB 5004 (see Fig. 14) using the terminal ID of the requesting terminal that has made the login request as a key, and extracts the terminal ID by reading out the terminal IDs of candidates for destination terminals that can communicate with the requesting terminal. The terminal extraction unit 54 also searches the destination list management DB 5004 (see Fig. 14) using the terminal ID of the requesting terminal that has made the login request as a key, and extracts the terminal IDs of other requesting terminals that have registered the terminal ID of the requesting terminal as a candidate for a destination terminal.

[0092] Furthermore, the terminal extraction unit 54 searches the destination list management DB 5004 using the terminal ID of the terminal that transmitted the image communication status message (m) as the terminal ID of the request source terminal, thereby extracting all the terminal IDs of the corresponding destination terminals.

[0093] The terminal status extraction unit 55 is realized by an instruction from the CPU 201 shown in Fig. 6, and searches the terminal management DB 5003 (see Fig. 13) using the terminal ID of the destination terminal candidate extracted by the terminal extraction unit 54 as a search key, and reads out the operation status for each terminal ID extracted by the terminal extraction unit 54. This allows the terminal status extraction unit 55 to obtain the operation status of the destination terminal candidate that can communicate with the request source terminal that has made the login request. The terminal status extraction unit 55 also searches the terminal management DB 5003 using the terminal ID extracted by the terminal extraction unit 54 as a search key, and obtains the operation status of the request source terminal that has made the login request.

[0094] Furthermore, the terminal status extraction unit 55 extracts corresponding image communication status information by searching the terminal management DB 5003 (see FIG. 13) using the terminal ID of the destination terminal as a search key. The terminal status extraction unit 55 also refers to the terminal management DB 5003 (see FIG. 13) to extract the IP addresses of the terminals 10 in image communication, the terminals 10 in image communication stopped, or the terminals 10 having an image communication function.

[0095] Primary narrowing down unit 56 is realized by an instruction from CPU 201 shown in Fig. 6, and performs a primary narrowing down process before the final narrowing down process in order to support the final narrowing down process of ultimately narrowing down a plurality of relay devices 30 to one relay device 30. To this end, primary narrowing down unit 56 realizes selection session ID generation unit 56a, terminal IP address extraction unit 56b, primary selection unit 56c, and priority determination unit 56d as shown in Fig. 9 by an instruction from CPU 201 shown in Fig. 5.

[0096] Of these, the selection session ID generating unit 56a generates a selection session ID used to execute a session for selecting a relay device 30. The terminal IP address extracting unit 56b searches the terminal management DB 5003 (see FIG. 13) based on the terminal ID of the request source terminal and the terminal ID of the destination terminal included in the start request information sent from the request source terminal, to extract the IP address of each corresponding terminal 10. The primary selecting unit 56c selects a relay device 30 by selecting the relay device ID of a relay device 30 whose operating status is "online" from among the relay devices 30 managed in the relay device management DB 5001 (see FIG. 11).

[0097] Further, the primary selection unit 56c searches the relay device management DB 5001 (see FIG. 11) based on the IP address of the request source terminal and the IP address of the destination terminal extracted by the terminal IP address extraction unit 56b, and investigates whether each dot address of the IP address of the selected relay device 30 is the same as or different from each dot address of the IP addresses of the request source terminal and the destination terminal. Furthermore, the primary selection unit 56c further selects the relay device 30 by selecting the top two relay devices 30 with the highest points among the integrated points obtained by integrating the higher point for the terminal 10 in the address priority points and the transmission speed priority points for each relay device. Note that in this embodiment, the top two relay devices 30 with the highest points are selected, but the present invention is not limited to this, and if it is possible to narrow down as many relay devices 30 as possible, the top three or more relay devices 30 with the highest points may be selected.

[0098] The priority determination unit 56d refers to the priority management DB 5006 (see FIG. 16) and determines address priority points for each relay device 30 surveyed by the primary selection unit 56c. The priority determination unit 56d also searches the priority management DB 5006 (see FIG. 17) based on the maximum data transmission speed of each relay device 30 managed in the relay device management DB 5001 (see FIG. 11) to determine transmission speed priority points for each relay device 30 narrowed down by the first narrowing down process by the primary selection unit 56c.

[0099] 6, session management unit 57 associates and stores and manages the selection session ID generated by selection session ID generation unit 56a, the terminal ID of the request source terminal, and the terminal ID of the destination terminal in session management DB 5005 (see FIG. 15) of storage unit 5000. Also, session management unit 57 stores and manages the relay device ID of relay device 30 finally selected by final selection unit 16c of terminal 10 for each selection session ID in session management DB 5005 (see FIG. 15).

[0100] The quality determining unit 58 determines the image quality of the image data to be relayed by the relay device 30 by searching the quality management DB 5007 (see FIG. 18) using the delay time as a search key and extracting the image quality of the corresponding image data.

[0101] The storage / read processing unit 59 is executed by instructions from the CPU 201 shown in FIG. 6 and the HDD 205 shown in FIG. 6, and performs processing such as storing various data in the memory unit 5000 and reading out various data stored in the memory unit 5000.

[0102] The delay time management unit 60 is realized by an instruction from the CPU 201 shown in Figure 6, and searches the terminal management DB 5003 (see Figure 13) using the IP address of the destination terminal as a search key to extract the corresponding terminal ID, and further stores and manages the delay time indicated in the delay information in the delay time field of the record containing the extracted terminal ID in the session management table of the session management DB 5005 (see Figure 15).

[0103] The message creation unit 61 creates a communication control message according to the image communication status of each terminal 30. The process of creating this communication control message will be described in detail later with reference to Fig. 28. The message creation unit 61 also acquires image communication status information related to the sender terminal 10 and the IP address of the terminal 10 from the image communication status message (m) sent from the terminal 10. Furthermore, the message creation unit 61 overwrites and rewrites the image communication status information acquired from the image communication status message (m) in the attribute portion of the image communication status corresponding to the IP address acquired from the image communication status message (m) in the terminal management DB 5003 (see Fig. 13).

[0104] The message creation unit 61 also extracts the terminal ID corresponding to the IP address acquired from the image communication status message (m) from the terminal management DB 5003 (see FIG. 13). Furthermore, the message creation unit 61 refers to the image communication status information extracted from the terminal management DB 5003 (see FIG. 13) to determine whether the request source terminal and all destination terminals each have an image communication function.

[0105] <<Processing or Operation of the Embodiment>> Next, a processing method in the transmission system 1 according to the present embodiment will be described with reference to Figs. 19 to 33. Fig. 19 is a sequence diagram showing a process of managing state information indicating the state of each relay device 30 transmitted from each relay device 30 to the management system 50. Fig. 20 is a sequence diagram showing a process of preparation for starting a call between a plurality of terminals 10. Fig. 21 is a sequence diagram showing a process of narrowing down the relay devices 30. Fig. 22 is a process flow diagram showing a process of narrowing down the relay devices 30. Fig. 23 is a diagram showing a calculation state of priority points when narrowing down the relay devices 30. Fig. 24 is a sequence diagram showing a process of the terminal 10 selecting the relay device 30. Fig. 25 is a process flow diagram showing a process of selecting the relay device 30 at the terminal. Fig. 33 is a sequence diagram showing a process of transmitting and receiving image data and audio data between terminals.

[0106] First, a process of managing the status information indicating the status of each relay device 30 transmitted from each relay device 30 to the management system 50 will be described with reference to FIG. 19. First, in each relay device 30, the status detection unit 32 shown in FIG. 7 periodically detects the operating status of the relay device 30 (steps S1-1 to S1-4). Then, in order to have the management system 50 manage the operating status of each relay device 30 in real time, the transmission / reception unit 31 of each relay device 30 periodically transmits each piece of status information to the management system 50 via the communication network 2 (steps S2-1 to S2-4). Each piece of status information includes the relay device ID of each relay device 30 and the operating status detected by the status detection unit 32 of the relay device 30 related to each of these relay device IDs. In this embodiment, the relay devices (30a, 30b, 30d) are operating normally and are "online", while the relay device 30c is operating but is "offline" due to some malfunction in the program for executing the relay operation of the relay device 30c.

[0107] Next, in the management system 50, the transmitting / receiving unit 51 receives each state information sent from each relay device 30, and stores and manages the state information for each relay device ID in the relay device management DB 5001 (see FIG. 11) of the storage unit 5000 via the storage / read processing unit 59 (steps S3-1 to S3-4). As a result, in the relay device management table as shown in FIG. 11, any of the operation states of "online", "offline", or "out of order" is stored and managed for each relay device ID. At this time, the reception date and time when the state information was received by the management system 50 is also stored and managed for each relay device ID. Note that, when no state information is sent from the relay device 30, the field portion of the operation state and the field portion of the reception date and time in each record of the relay device management table shown in FIG. 11 are blank, or show the operation state and the reception date and time at the time of the previous reception, respectively.

[0108] Next, a description will be given of a transmission and reception process of each piece of management information in a preparation stage before starting a call between the terminal 10aa and the terminal 10db, with reference to Fig. 20. In Fig. 20, various pieces of management information are transmitted and received by the management information session sei shown in Fig. 2.

[0109] First, when a user turns on the power switch 109 shown in FIG. 5, the operation input reception unit 12 shown in FIG. 7 receives the power ON and turns on the power (step S21). Then, the login request unit 13, triggered by the reception of the power ON, automatically transmits login request information indicating a login request from the transmission / reception unit 11 to the management system 50 via the communication network 2 (step S22). This login request information includes a terminal ID and a password for identifying the terminal 10aa, which is the request source, which is the terminal itself. These terminal ID and password are data that are read from the storage unit 1000 via the storage / read processing unit 19 and sent to the transmission / reception unit 11. When the login request information is transmitted from the terminal 10aa to the management system 50, the management system 50, which is the receiving side, can grasp the IP address of the terminal 10ab, which is the transmitting side.

[0110] Next, the terminal authentication unit 52 of the management system 50 searches the terminal authentication management DB 5002 (see FIG. 12) in the storage unit 5000 using the terminal ID and password included in the login request information received via the transmission / reception unit 51 as search keys, and performs terminal authentication by determining whether the same terminal ID and password are managed in the terminal authentication management DB 5002 (step S23). If the same terminal ID and password are managed by the terminal authentication unit 52 and it is determined that the login request is from a terminal 10 with valid usage authority, the status management unit 53 stores the terminal ID of the terminal 10aa, the operating status, the date and time when the login request information was received, and the IP address of the terminal 10aa in the terminal management DB 5003 (see FIG. 13) in association with each other (step S24). As a result, in the terminal management table shown in FIG. 13, the terminal ID "01aa" is managed in association with the operating state "online", the reception date and time "2009.11.10.13:40", and the IP address "1.2.1.3" of the terminal 10aa.

[0111] Then, the transmitting / receiving unit 51 of the management system 50 transmits authentication result information indicating the authentication result obtained by the terminal authentication unit 52 to the requesting terminal (terminal 10aa) that made the login request via the communication network 2 (step S25). In this embodiment, a case where the terminal authentication unit 52 judges that the terminal has a valid usage right will be described below.

[0112] The terminal extraction unit 54 of the management system 50 searches the destination list management DB 5004 (see FIG. 14) using the terminal ID "01aa" of the request source terminal (terminal 10aa) that made the login request as a search key, and reads out and extracts the terminal IDs of candidate destination terminals that can communicate with the request source terminal (terminal 10aa) (step S26). Here, the terminal IDs ("01ab", "01ba", "01db") of the destination terminals (terminals 10ab, 10ba, 10db) corresponding to the terminal ID "01aa" of the request source terminal (terminal 10aa) are extracted.

[0113] Next, the terminal status extraction unit 55 searches the terminal management DB 5003 (see FIG. 13) using the terminal IDs ("01ab", "01ba", "01db") of the destination terminal candidates extracted by the terminal extraction unit 54 as search keys, and obtains the operating status of each of the terminals (10ab, 10ba, 10db) by reading out the operating status ("offline", "online", "online") for each terminal ID extracted by the terminal extraction unit 54 (step S27).

[0114] Next, the transmitting / receiving unit 51 transmits destination status information including the terminal IDs ("01ab", "01ba", "01db") used as the search keys in step S27 above and the operation statuses ("offline", "online", "online") of the corresponding destination terminals (terminals 10ab, 10ba, 10db) to the request source terminal (terminal 10aa) via the communication network 2 (step S28). This allows the request source terminal (terminal 10aa) to grasp the current operation statuses ("offline", "online", "online") of the terminals (10ab, 10ba, 10db) that are candidates for destination terminals that can communicate with the request source terminal (terminal 10aa).

[0115] Furthermore, the terminal extraction unit 54 of the management system 50 searches the destination list management DB 5004 (see FIG. 14) using the terminal ID "01aa" of the request source terminal (terminal 10aa) that has made the login request as a search key, and extracts the terminal IDs of other request source terminals that have registered the terminal ID "01aa" of the request source terminal (terminal 10aa) as a destination terminal candidate (step S29). In the destination list management table shown in FIG. 14, the terminal IDs of the other extracted request source terminals are "01ab", "01ba", and "01db".

[0116] Next, the terminal status extraction unit 55 of the management system 50 searches the terminal management DB 5003 (see FIG. 13) using the terminal ID “01aa” of the requesting terminal (terminal 10aa) that made the login request as a search key, and obtains the operating status of the requesting terminal (terminal 10aa) that made the login request (step S30).

[0117] Then, the transmitting / receiving unit 51 transmits destination status information including the terminal ID "01aa" of the request source terminal (terminal 10aa) acquired in the above step S30 and the operating status "online" to the terminals (10ba, 10db) whose operating status is "online" in the terminal management DB 5003 (see FIG. 13) among the terminals (10ab, 10ba, 10db) related to the terminal IDs ("01ab", "01ba", "01db") extracted in the above step S29 (steps S31-1, 2). Note that when the transmitting / receiving unit 51 transmits the destination status information to the terminals (10ba, 10db), it refers to the IP addresses of the terminals managed in the terminal management table shown in FIG. 13 based on the respective terminal IDs ("01ba", "01db"). This makes it possible to communicate the terminal ID "01aa" and the operating status "online" of the requesting terminal (terminal 10aa) that made the login request to each of the other destination terminals (terminals 10db, 10ba) that can communicate with the requesting terminal (terminal 10aa) that made the login request.

[0118] On the other hand, in the same manner as in step 21 above, when a user turns on the power switch 109 shown in FIG. 6 in another terminal 10, the operation input receiving unit 12 shown in FIG. 7 receives the power ON and performs processing similar to the processing in steps S22 to S31-1, 2 above, so that a description thereof will be omitted.

[0119] Next, the process of narrowing down the relay devices 30 will be described with reference to FIG. 21. In FIG. 21, various pieces of management information are transmitted and received by the management information session sei shown in FIG. 21. In this embodiment, the request source terminal (terminal 10aa) can make a call with at least one of the terminals (10ba, 10db) whose operating status is online among the terminals 10 as destination candidates, based on the terminal status information received in step S32. Hereinafter, a case will be described where the user of the request source terminal (terminal 10aa) selects to start a call with the destination terminal (terminal 10db).

[0120] First, when a user presses the operation button 108 shown in Fig. 5 to select the terminal 10db, the operation input reception unit 12 shown in Fig. 7 receives a request to start a call with the destination terminal (terminal 10db) (step S41). Then, the transmission / reception unit 11 of the request source terminal (terminal 10aa) transmits start request information indicating a desire to start a call, which includes the terminal ID "01aa" of the terminal 10aa and the terminal ID "01db" of the destination terminal (terminal 10db), to the management system 50 (step S42). As a result, the transmission / reception unit 51 of the management system 50 receives the start request information and can grasp the IP address "1.2.1.3" of the request source terminal (terminal 10aa) that is the sender.

[0121] Then, based on the terminal ID "01aa" of the request source terminal (terminal 10aa) and the terminal ID "01db" of the destination terminal (terminal 10db) included in the start request information, the status management unit 53 changes the operation status field of the record including the terminal ID "01aa" and the terminal ID "01db" in the terminal management table of the terminal management DB 5003 (see FIG. 13) to "in call" (step S43). Note that in this state, the request source terminal (terminal 10aa) and the destination terminal (terminal 10db) have not started a call, but are in a call state, and when another terminal 10 tries to call the request source terminal (terminal 10aa) or the destination terminal (terminal 10db), a notification sound or display indicating a so-called in-call state is output.

[0122] Next, a process of executing a session for selecting a relay device 30 will be described. First, the selection session ID generating unit 56a shown in Fig. 9 generates a selection session ID used for executing a session for selecting a relay device 30 (step S44). Then, the session managing unit 57 stores and manages the selection session ID "se1" generated in the above step S44, the terminal ID "01aa" of the request source terminal (terminal 10aa), and the terminal ID "01db" of the destination terminal (terminal 10db) in a session management DB 5005 (see Fig. 15) of the storage unit 5000 in association with each other (step S45).

[0123] Next, the primary narrowing-down unit 56 of the management system 50 shown in FIG. 7 performs primary narrowing-down of the relay devices 30 for relaying the call between the requesting terminal (terminal 10aa) and the destination terminal (terminal 10db) based on the relay device management DB 5001, the terminal management DB 5003, and the priority management DB 5006 (step S46).

[0124] Here, the process in step S46 will be described in more detail with reference to Fig. 9 and Fig. 22. First, the terminal IP address extraction unit 56b shown in Fig. 9 searches the terminal management DB 5003 (see Fig. 13) based on the terminal ID "01aa" of the request source terminal (terminal 10aa) and the terminal ID "01db" of the destination terminal (terminal 10db) contained in the start communication information sent from the request source terminal (terminal 10aa), thereby extracting the IP addresses ("1.2.1.3", "1.3.2.4") of the corresponding terminals (10aa, 10db) (step S46-1).

[0125] Next, the primary selection unit 56c selects each of the relay device IDs (111a, 111b, 111d) of the relay devices (30a, 30b, 30d) that are in the “online” state among the operation states of the relay devices 30 managed in the relay device management DB 5001 (see FIG. 11) (step S46-2). In addition, the primary selection unit 56c searches the relay device management DB 5001 (see FIG. 11) based on the IP address "1.2.1.3" of the requesting terminal (terminal 10aa) and the IP address "1.3.2.4" of the destination terminal (terminal 10db) extracted in step S46-1 above, and investigates whether each dot address in the IP addresses ("1.2.1.2.", "1.2.2.2", "1.3.2.2") of the relay devices (30a, 30b, 30d) selected in step S46-2 above is the same as or different from each dot address in the IP addresses ("1.2.1.3", "1.3.2.4") of the requesting terminal (terminal 10aa) and the destination terminal (terminal 10db) above (step S46-3).

[0126] Next, the priority determination unit 56d refers to the priority management DB 5006 (see FIG. 16) and determines the address priority points for each relay device (30a, 30b, 30d) investigated in step S46-3 (step S46-4). If the result of this determination process is expressed in a table, it will be as shown in FIG. 23. Note that FIG. 23 is a diagram showing the calculation state of the priority points when performing the narrowing down process of the relay device 30. In this FIG. 23, the address priority points, the transmission speed priority points, and the integrated points are shown for each relay device ID. In addition, the address priority points further show the points for the request source terminal (terminal 10aa) and the points for the destination terminal (terminal 10db) of each relay device 30. The integrated point is the sum of the higher of the two address priority points and the transmission speed priority points.

[0127] In this embodiment, the IP address "1.2.1.2" of the relay device 30a is "same.same.same.different" with respect to the IP address "1.2.1.3" of the request source terminal (terminal 10aa), so the address priority point is "5" as shown in Fig. 23. Also, as shown in Fig. 1, the IP address "1.2.1.2" of the relay device 30a is "same.different.different.different" with respect to the IP address "1.3.2.4" of the destination terminal (terminal 10db), as shown in Fig. 16, so the address priority point is "1" as shown in Fig. 23. Also, the IP address "1.2.2.2" of the relay device 30b is "same.same.different.different" with respect to the IP address "1.2.1.3" of the request source terminal (terminal 10aa), so the address priority point is "3". Moreover, since the IP address "1.2.2.2" of the relay device 30b is "same.different.same.different" to the IP address "1.3.2.4" of the destination terminal (terminal 10db), the address priority point is "1." Furthermore, the IP address "1.3.2.2" of the relay device 30d is "same.different.different.different" to the IP address "1.2.1.3" of the request source terminal (terminal 10aa), the address priority point is "1." Furthermore, the IP address "1.3.2.2" of the relay device 30d is "same.same.same.different" to the IP address "1.3.2.4" of the destination terminal (terminal 10db), the address priority point is "5."

[0128] Next, returning to FIG. 22, the priority determination unit 56d searches the priority management DB 5006 (see FIG. 17) based on the maximum data transmission rate of each relay device 30 managed in the relay device management DB 5001 (see FIG. 11), thereby determining the transmission rate priority points for each relay device (30a, 30b, 30d) narrowed down by the first narrowing down process in the above step S46-2 (step S46-5). In this embodiment, as shown in FIG. 11, the maximum data transmission rate of the relay device 30a is 100 (Mbps), so with reference to the transmission rate priority shown in FIG. 17, the transmission rate priority is 3 points. Similarly, when the maximum data transmission rate of the relay device 30b is calculated, it is 1000 (Mbps), so the transmission rate priority is 5 points. Similarly, when the maximum data transmission rate of the relay device 30d is calculated, it is 10 (Mbps), so the transmission rate priority is 1 point.

[0129] Next, the primary selection unit 56c selects the top two relay devices 30 with the highest points among the integrated points obtained by integrating the highest address priority points of the terminals (10aa, 10db) and the transmission speed priority points for each relay device (30a, 30b, 30d) (step 46-6). In this embodiment, as shown in Fig. 23, the relay device IDs (111a, 111b, 111d) have integrated points of "8", "8", and "6", respectively, so the relay device 30a associated with the relay device ID "111a" and the relay device 30b associated with the relay device ID "111b" are selected.

[0130] When the narrowing down process in step S46 is completed, the transmitting / receiving unit 51 shown in Fig. 7 transmits relay device narrowing down information for conveying the number of the narrowed down relay devices 30 to the destination terminal (terminal 10db) via the communication network 2 (step S47). This relay device narrowing down information includes the number of relay devices 30 narrowed down in step 46, "2", the terminal ID "01aa" of the request source terminal (terminal 10aa), and the selection session ID "se1". As a result, the terminal 10db can grasp the number of relay devices 30 and which terminal 10 has requested to start a call in the execution of the session in the selection session ID "se1", and can also grasp the IP address "1.1.1.2" of the management system 50 which is the source of the relay device narrowing down information.

[0131] Then, the terminal 10db transmits reception completion information indicating that reception of the relay device selection information has been completed to the management system 50 from the transmitting / receiving unit 11 via the communication network 2 (step S48). This reception completion information includes the session ID "se1". This enables the management system 50 to grasp the IP address "1.3.2.4" of the destination terminal (terminal 10db) which is the transmission source, as well as the completion of transmission of the number of relay devices executed with the session ID "se1".

[0132] 24 and 25, a process in which the destination terminal (terminal 10db) selects the relay device 30 will be described. In FIG. 24, various pieces of management information are transmitted and received through the management information session sei shown in FIG.

[0133] First, before starting a video conference call, the management system 50 transmits pre-relay request information requesting relay in advance to each of the relay devices (30a, 30b) narrowed down in step S46 (steps S61-1, 2). This pre-relay request information includes the session ID "se1", the IP address "1.2.1.3" of the request source terminal (terminal 10aa), and the IP address "1.3.2.4" of the destination terminal (terminal 10db). This allows each relay device (30a, 30b) to know which selection session it belongs to, what the request source terminal is, and what the destination terminal is, and also to know the IP address "1.1.1.2" of the management system 50 that is the sender of the pre-relay request information.

[0134] Next, each of the relay devices (30a, 30b) transmits pre-transmission request information indicating that the relay device (30a, 30b) as its own device transmits pre-transmission information including a ping (Packet Internet Groper) (described later) to the request source terminal (terminal 10aa) recognized in the above steps S61-1, 2 via the communication network 2 from the transmission / reception unit 31 before starting a call (steps S62-1, 2). This pre-transmission information includes the session ID "se1". As a result, the request source terminal (terminal 10aa) is able to recognize that the pre-transmission information is to be transmitted to each of the relay devices (30a, 30b) in the selection process of the relay device 30 executed with the session ID "se1", and is able to recognize the IP addresses ("1.2.1.2", "1.2.2.2") of the relay devices (30a, 30b) that are the transmission sources of the pre-transmission request information.

[0135] Note that the management system 50 does not directly notify the requesting terminal of the IP address of the destination terminal, but instead notifies the relay device 30a of the IP address of the destination terminal as in step S61-1 above, and then requests the relay device 30a to transmit pre-transmission request information to itself (relay device 30a) as in step S61-2 above, in order to ensure security by preventing each terminal 10 from being informed of the IP addresses of the other terminals 10.

[0136] Next, the request source terminal (terminal 10aa) transmits pre-transmission information from the transmission / reception unit 11 to each relay device (30a, 30b) via the communication network 2 (steps S63-1, 2). This pre-transmission information is transmitted to the destination terminal (terminal 10db) via each relay device (30a, 30b) in place of the image data and voice data prior to transmission of the image data and voice data, and is used to measure the time required from transmission of the request source terminal (terminal 10aa) to reception of the destination terminal (terminal 10db). In addition, this pre-transmission information includes a ping for confirming that the request source terminal (terminal 10aa), the relay devices (30a, 30b), and the destination terminal (terminal 10db) are connected to be able to communicate, the transmission date and time when the pre-transmission information was transmitted from the request source terminal (terminal 10aa), and the session ID "se1". As a result, each relay device (30a, 30b) can recognize that pre-transmission information has been sent during the execution of a session with the selection session ID "se1", and can also recognize the IP address "1.2.1.3" of the requesting terminal (terminal 10aa) that is the sender of this pre-transmission information.

[0137] Next, each relay device (30a, 30b) relays the pre-transmission information to the IP address "1.3.2.4" of the destination terminal (terminal 10db) included in the pre-relay request information received in steps S61-1, 2 (steps S64-1, 2). This allows the destination terminal (terminal 10db) to know that the pre-transmission information has been sent in the execution of the session with session ID "se1", and also to know the IP addresses ("1.2.1.2", "1.2.2.2") of the relay devices (30a, 30b) that are the senders (relay sources) of the pre-transmission information.

[0138] Next, the final narrowing down unit 16 of the destination terminal (terminal 10db) narrows down to one relay device 30 that will ultimately relay image data and audio data in the video conference call, based on the pre-transmission information (step S65).

[0139] Here, the process in step S65 will be described in more detail with reference to FIG. 8 and FIG. 25. First, the measurement unit 16a of the final narrowing down unit 16 shown in FIG. 8 measures the reception date and time when the pre-transmission information relayed by each relay device (30a, 30b) is received by the transmission / reception unit 11 of the terminal 10db (step S65-1). Next, the calculation unit 16b calculates the required time from transmission to reception of each pre-transmission information based on the difference between the reception date and time and the transmission date and time included in the pre-transmission information for each pre-transmission information whose reception time has been measured (step S65-2). Next, the final selection unit 16c judges whether or not all of the pre-transmission information corresponding to the number "2" of relay devices 30 to be relayed has been received in the execution of the session in the session ID "se1" (step S65-3). Then, if all of the pre-transmission information has not been received (NO), the final selection unit 16c judges whether or not a predetermined time (here, one minute) has elapsed since the pre-transmission information was received by the terminal 10db (step S65-4). Furthermore, if the predetermined time has not elapsed (NO), the process returns to step S65-1. On the other hand, if all the information has been received in step S65-3 (YES), or if the predetermined time has elapsed in step S65-4 (YES), the final selection unit 16c selects one relay device 30 that has relayed the pre-transmission information that required the shortest required time among the required times calculated by the calculation unit 16b so far (step S65-5). In this embodiment, an example is shown in which the relay device 30a is selected because the pre-transmission information relayed by the relay device 30a required a shorter time from transmission to reception than the pre-transmission information relayed by the relay device 30b.

[0140] Next, the destination terminal (terminal 10db) transmits selection information indicating that the relay device 30a has been selected from the transmitting / receiving unit 11 to the management system 50 via the communication network 2 (step S66). This selection information includes the session ID "se1" and the relay device ID "111a" of the selected relay device 30a. This allows the management system 50 to know that the relay device 30a has been selected in the execution of the session with the session ID "se1", and also to know the IP address "1.3.2.4" of the destination terminal (terminal 10db) that is the source of the selection information.

[0141] Next, the session management unit 57 of the management system 50 stores and manages the relay device ID "111a" of the relay device 30a finally selected as the one in the session management table of the session management DB 5005 (see FIG. 15) in the relay device ID field of the record including the session ID "se1" (step S67). Then, the transmission / reception unit 51 of the management system 50 transmits relay start request information indicating a request to start relaying to the relay device 30a via the communication network 2 (step S68). This relay start request information includes the IP addresses ("1.2.1.3", "1.3.2.4") of the request source terminal (terminal 10aa) and destination terminal (terminal 10db) to be relayed. As a result, the relay device 30a can recognize that the relay device 30a, which is the own device, has been selected, and establishes a session for communicating three types of image data, low resolution, medium resolution, and high resolution, and voice data between the terminals (10aa, 10db) (step S69). Therefore, the terminals (10aa, 10db) can start a video conference call.

[0142] In the above step S47, the management system 50 transmits the relay device selection information to the destination terminal (terminal 10db), and then the destination terminal (terminal 10db) performs the relay device selection process (step S65) through steps S48 to S64-1 and 2. However, this is not limited to the above, and in the above step S47, the management system 50 may transmit the relay device selection information to the request source terminal (terminal 10aa), and the sender and receiver of each piece of information may be switched between the request source terminal (terminal 10aa) and the destination terminal (terminal 10db) until steps S64-1 and 2. This allows the request source terminal (terminal 10aa) to perform the relay device selection process instead of the above step S65, and also to transmit the selection information instead of the above step S66.

[0143] Next, with reference to Figs. 26 to 28, a process will be described in which, in a case where a video conference is held at multiple locations (multiple terminals 10), a communication control message is created and transmitted to each terminal 10, and then processing based on the communication control message is performed at each terminal 10. Fig. 26 shows a case where a video conference is held by three terminals, terminal 10aa, terminal 10ba, and terminal 10db. Fig. 26 is a sequence diagram showing the process of creating and transmitting / receiving a communication control message. Fig. 27 is a flow diagram mainly showing the process of creating a communication control message. Fig. 28 is a flow diagram showing in detail a part of the process of creating a communication control message. In this embodiment, a case where three terminals (10aa, 10ba, 10db) as multiple locations hold a conference will be described.

[0144] First, as shown in FIG. 26, the message creation unit 20 of each terminal 10 (terminal 10aa in this case) reads out pre-stored image communication status information from the storage unit 1000, and creates an image communication status message (m) including this image communication status information (step S71). This image communication status message (m) includes the IP address "1.2.1.3" of this terminal 10aa. Note that a destination name may be included instead of the IP address. This image communication status message (m) can be created in, for example, an XML (eXtensible Markup Language) format that complies with XMPP (eXtensible Messaging and Presence Protocol) of RFC3921. Note that communication control messages and display control messages, which will be described later, are also created in the same format.

[0145] Next, the transmitting / receiving unit 11 of the terminal 10aa transmits the image communication status message (m) created in the above step S71 to the management system 50 via the communication network 2 (step S72). As a result, the transmitting / receiving unit 51 of the management system 50 receives the image communication status message (m).

[0146] Next, the message creation unit 61 of the management system 50 creates a communication control message according to the image communication state of each terminal 30 (step S73). This step S73 will be described in detail with reference to FIG.

[0147] First, the message creation unit 61 acquires image communication status information related to the source terminal 10aa and the IP address "1.2.1.3" of the terminal 10aa from the image communication status message (m) received by the transmission / reception unit 51 (step S73-1). Then, the message creation unit 61 overwrites and rewrites the image communication status information acquired in the above step S73-1 in the attribute portion of the image communication status corresponding to the IP address acquired in the above step S73-1 in the terminal management DB 5003 (see FIG. 13) (step S73-2).

[0148] Next, message creation unit 61 extracts the terminal ID corresponding to the IP address acquired in step S73-1 from terminal management DB 5003 (see FIG. 13) (step S73-3).

[0149] Next, the terminal extraction unit 54 searches the destination list management DB 5004 using the terminal ID extracted in step S73-3 as the terminal ID of the request source terminal to extract all the terminal IDs of the corresponding destination terminals (step S73-4).Then, the terminal status extraction unit 55 searches the terminal management DB 5003 (see FIG. 13) based on the terminal ID of the destination terminal to extract the corresponding image communication status information (step S73-5).

[0150] Next, the message creation unit 61 refers to the image communication status information extracted in the above step S73-5 to determine whether the request source terminal and all the destination terminals each have an image communication function (step S73-6). If neither the request source terminal nor all the destination terminals have an image communication function (NO), the process of step S73 ends. In this case, the message transmission process of step S74 described later (see Figs. 26 and 27) is not performed.

[0151] On the other hand, if either the request source terminal or any of all the destination terminals has an image communication function (YES), the message creation unit 61 creates a communication control message (described later) according to the image communication state of each terminal 10 (step S73-7).

[0152] Next, the above step S73-7 will be described in more detail with reference to FIG. 28. First, the message creation unit 61 refers to the image communication state extracted in the above step S73-5 and judges whether two or more terminals 10 are in image communication among the multiple terminals (10aa, 10ba, 10db) in the multi-site conversation (step S101). If it is judged in this step S101 that the multiple terminals 10 are in image communication (YES), the message creation unit 61 further judges whether there is a terminal 10 in which image communication is stopped among the remaining terminals 10 (step S102). If it is judged in this step S103 that there is a terminal 10 in which image communication is stopped (YES), the terminal state extraction unit 55 refers to the terminal management DB 5003 (see FIG. 13) and extracts the IP address of the terminal 10 in which image communication is stopped that is judged to exist in the above step S102 (step S103). Then, the message creation unit 61 creates a communication control message (M3) indicating that a plurality of terminals are in image communication (step S104). Specifically, the communication control message (M3) includes the IP address of the destination extracted in step S103 and the IP address of the management system 50, which is the sender, in addition to the message "MESSAGE_IMAGE_ON".

[0153] On the other hand, if it is determined in step S101 that multiple terminals 10 are not in image communication (NO), the message creation unit 61 refers to the image communication state extracted in step S73-5 and determines whether only a single terminal 10 is in image communication (step S105). If it is determined in step S105 that only a single terminal 10 is in image communication (YES), the message creation unit 61 further refers to the image communication state extracted in step S73-5 and determines whether all terminals 10 other than the terminal 10 in image communication do not have an image communication function (step S106). If it is determined in step S106 that all terminals 10 other than the terminal 10 in image communication do not have an image communication function (YES), the terminal status extraction unit 55 refers to the terminal management DB 5003 (see FIG. 13) and extracts the IP address of the single terminal 10 determined in step S105 to be in image communication (step S107). Then, the message creation unit 61 creates a communication control message (M4) for forcibly stopping the image communication (step S108). Specifically, the communication control message (M4) includes the IP address of the destination extracted in step S107 and the IP address of the management system 50, which is the sender, in addition to the message "DATA_IMAGE_OFF".

[0154] On the other hand, if it is not determined in step S106 that all terminals 10 other than the terminal 10 in image communication do not have the image communication function (NO), that is, if it is determined that at least one of the terminals 10 other than the terminal 10 in image communication has the image communication function, the message creation unit 61 further refers to the image communication state extracted in step S73-5 and determines whether there is a terminal 10 in which image communication is stopped among the terminals 10 having the image communication function (step S109). If it is determined in step S109 that there is a terminal 10 in which image communication is stopped (YES), the terminal state extraction unit 55 refers to the terminal management DB 5003 (see FIG. 13) and extracts the IP address of the terminal 10 in image communication determined in step S105 to be present (step S110). Then, the message creation unit 61 creates a communication control message (M1) indicating that the other multiple terminals are in the image communication stopped state (step S111). Specifically, the communication control message (M1) includes the IP address of the destination extracted in step S110 above and the IP address of the management system 50 that is the sender, in addition to the message "MESSAGE_IMAGE_OFF".

[0155] Furthermore, the terminal status extraction unit 55 refers to the terminal management DB 5003 (see FIG. 13) and extracts the IP address of the terminal 10 that is determined to be in image communication stop in the above step S109 (step S112). Then, the message creation unit 61 creates a communication control message (M2) indicating that another single terminal is in image communication (step S113). Specifically, the communication control message (M2) includes the IP address of the destination extracted in step S112 and the IP address of the management system 50 that is the sender, in addition to the message "MESSAGE_IMAGE_ON".

[0156] On the other hand, (1) if it is determined by the above step S102 that there is no terminal 10 that is not performing image communication among the remaining terminals 10 (NO), (2) if it is determined by the above step S105 that only a single terminal 10 is not performing image communication (NO), that is, if there is no terminal 10 performing image communication, or (3) if it is determined by the above step S109 that there is no terminal 10 that is performing image communication (NO), the terminal status extraction unit 55 refers to the terminal management DB 5003 (see FIG. 13) and extracts the IP address of the terminal 10 that is determined by the above step S73-6 to have the image communication function (step S114). Then, the message creation unit 61 creates a display control message (M5) to erase a predetermined display message displayed on each display 120 of the multiple terminals 10 (step S115). Specifically, the display control message (M5) includes the IP address of the destination extracted in step S114 and the IP address of the management system 50 that is the sender, in addition to the message "MESSAGE_IMAGE_ON".

[0157] As a result of the above, the message creating unit 61 creates a communication control message (M1, M2, M3, or M4) or a display control message (M5).

[0158] Next, the transmitting / receiving unit 51 of the management system 50 transmits, via the communication network 2, to the three terminals (10aa, 10ba, 10db) at multiple locations, a communication control message (M1, M2, M3, or M4) or a display control message (M5) created according to the image communication state of each of these terminals and the image communication state of the conversation partner's terminal (step S74). Note that in FIG. 26, in step S74, the transmission from the management system 50 to the terminal ba is represented as step S74-1, the transmission from the management system 50 to the terminal db is represented as step S74-2, and the transmission from the management system 50 to the terminal aa is represented as step S74-3.

[0159] Specifically, when the step S103 is executed, the transmitting / receiving unit 51 of the management system 50 transmits the communication control message (M3) created in the step S104 to the IP address extracted in the step S103. When the step S107 is executed, the transmitting / receiving unit 51 transmits the communication control message (M4) created in the step S108 to the IP address extracted in the step S107. When the step S110 is executed, the transmitting / receiving unit 51 transmits the communication control message (M1) created in the step S111 to the extracted IP address. When the step S112 is executed, the transmitting / receiving unit 51 transmits the communication control message (M2) created in the step S113 to the IP address extracted in the step S112. When the step S114 is executed, the transmitting / receiving unit 51 transmits the communication control message (M5) created in the step S115 to the IP address extracted in the step S114.

[0160] As a result, each of the terminals (10aa, 10ba, 10db) performs processing based on the contents of the communication control message (M1, M2, M3, or M4) or the display control message (M5) (steps S75-1 to S75-3).

[0161] Next, the process performed in steps S75-1 to S75-3 will be described with reference to FIG. 7 and FIG. 29 to FIG. 32. FIG. 29 to FIG. 32 are diagrams showing an example of a video conference screen displayed on the display 120 of the terminal 10. Among them, the video conference screen P1 displayed on the display 120 includes a main screen P2, a sub-screen P3, and a sub-screen P4. The two sub-screens below the sub-screen P4 and three sub-screens to the left of the bottom sub-screen are included. In addition, at the bottom of the video conference screen P1, a message display area P5 for displaying a display message, a "screen switching" button P6 that is pressed with a mouse pointer or the like when switching between images displayed on the main screen and the sub-screens, and a camera icon P7 that indicates whether the power of the camera 112 of the terminal is turned on and image communication is being performed from the terminal side or whether the power of the camera 112 of the terminal is turned off and image communication is not being performed from the terminal side are displayed. When an "X" is displayed on the camera icon P7, it means that the power of the camera 112 of the own terminal is on and the user can now turn it off. On the other hand, when an "X" is not displayed on the camera icon P7, it means that the power of the camera 112 of the own terminal is off and the user can now turn it on.

[0162] First, in the above step S104, when a communication control message (M3) is created to the effect that other terminals are suspending image communication, the display control unit 17 of the terminal 10 displays a display message such as "Images from other locations are being received. The camera of the terminal is off" in the message display area P5 of the display 120, as shown in Fig. 29. Note that the main image of the other party is displayed on the main screen P2, and the image of the terminal itself is not displayed on the sub-screen P3, although the image of the terminal itself is normally displayed on the sub-screen P3, and furthermore, the sub image of the other party is displayed on the sub-screen P4.

[0163] As a result, by looking at the video conference image P1 shown in Fig. 29, the user can recognize that image data is not being sent from his / her own terminal, and that image data is being sent from another terminal. If the user wishes to send image data from his / her own terminal, he / she can start sending image data by pressing the camera icon P7. When the camera icon P7 is pressed, an "X" is displayed on the camera icon P7.

[0164] Furthermore, when a communication control message (M4) for forcibly stopping image communication is created in step S108, the imaging unit 14 of the terminal 10 stops imaging and the transmitting / receiving unit 11 stops transmitting image data (stops image communication) in accordance with a command from the CPU 101. Furthermore, the display control unit 17 makes all screens P2, P3, P4, etc. of the video conference screen P1 invisible and erases the "x" from the camera icon P7, as shown in Fig. 30. Note that the imaging unit 14 includes not only the camera 112 shown in Fig. 5 but also an external camera connected to the external device connection I / F.

[0165] As a result, by looking at the video conference image P1 shown in Figure 30, the user can recognize that no image data is being sent from his / her own terminal, that the camera 112 of his / her own terminal has stopped capturing images, and that no image data is being sent from other terminals.

[0166] Furthermore, in the above step S111, when a communication control message (M1) is created to the effect that all other terminals are suspending image communication, the display control unit 17 of the terminal 10 displays a display message such as "No images are being received from other locations. The camera of the terminal itself is on" in the message display area P5 of the display 120 as shown in Fig. 31. In Fig. 31, nothing is displayed on the main screen P2 and the sub-screen P4, and the image of the terminal itself is displayed on the sub-screen P3.

[0167] As a result, by looking at the video conference screen P1 shown in Figure 31, the user can recognize that only his / her own terminal is sending image data and that no image data has been sent from any of the other terminals.

[0168] Furthermore, when a communication control message (M2) is created in step S113 to the effect that another single terminal is engaged in image communication, the display control unit 17 of the terminal 10 displays a display message such as "Images are being received from one other location. The camera of the terminal itself is off" in the message display area P5 as shown in Fig. 32. In Fig. 32, an image of the other party at one location is displayed on the main screen P2, and nothing is displayed on the sub-screens P3 and P4.

[0169] As a result, by looking at the video conference screen P1 shown in Figure 32, the user can recognize that no image data is being sent from his / her own terminal and that image data is being sent from only one of the multiple remote terminals.

[0170] In this way, when the image of Figure 31 is displayed on one terminal 10 and the image of Figure 32 is displayed on the other terminal 10, each user can choose to perform image communication with each other or not perform image communication with each other by matching the image communication status on both sides.

[0171] Furthermore, in the above step S115, if a display control message (M5) to the effect that the display message is to be erased is created, the display control unit 17 of the terminal 10 erases the display message displayed in the message display area P5.

[0172] Note that the terminal 10 having the image communication function can transmit an image communication status message not only by the process of step S72 in Fig. 26, but also by pressing the camera icon P7 shown in Fig. 29 during a video conference. Specifically, when the user selects the camera icon P7 with an "x" displayed as shown in Fig. 29, the message creation unit 20 of the terminal 10 creates an image communication status message indicating that image communication is stopped. Also, when the user selects the camera icon P7 without an "x" displayed as shown in Fig. 30, the message creation unit 20 of the terminal 10 creates an image communication status message indicating that image communication is in progress.

[0173] Next, a process of transmitting and receiving image data and audio data to perform a video conference call between a request source terminal and a destination terminal will be described with reference to Figures 7 and 33. Note that the one-way process of transmitting image data and audio data from the terminal 10aa to the terminal 10db and the reverse process of transmitting image data and audio data from the terminal 10db to the terminal 10aa involve the same processes as those for transmitting and receiving image data and audio data and detecting a delay time, which will be described later, and therefore only the one-way communication will be described and the reverse communication will be omitted.

[0174] First, the request source terminal (terminal 10aa) transmits image data of a subject captured by the imaging unit 14a and audio data of audio input by the audio input unit 15a to the relay device 30a from the transmission / reception unit 11 via the communication network 2 by the image / audio data session sed shown in FIG. 2 (step S81). In this embodiment, high-quality image data consisting of three resolutions, low resolution, medium resolution, and high resolution, as shown in FIG. 3, and audio data are transmitted. As a result, the relay device 30a receives the image data and audio data of the above three resolutions at the transmission / reception unit 31. Then, the data quality confirmation unit 33 searches the change quality management DB 3001 (see FIG. 10) using the IP address "1.3.2.4" of the destination terminal (terminal 10db) as a search key, and extracts the image quality of the corresponding image data to be relayed, thereby confirming the image quality of the image data to be relayed (step S82). In this embodiment, since the confirmed image data has a "high image quality" and is the same as the image data received by the transmitting / receiving unit 31, the relay device 30a transfers the image data with the same image quality and the audio data with the same sound quality to the destination terminal (terminal 10db) through the image / audio data session sed (step S83). As a result, the destination terminal (terminal 10db) receives high-image-quality image data consisting of three resolutions, low resolution, medium resolution, and high resolution, and audio data through the transmitting / receiving unit 11. Then, the display control unit 17 combines the image data of the three image qualities to display an image on the display 120, and the audio output unit 15b outputs audio based on the audio data.

[0175] Next, the delay detection unit 18 of the terminal 10db detects the delay time of the image data received by the transmission / reception unit 11 at regular time intervals (for example, every second) (step S84). In this embodiment, the following description will be given for the case where the delay time is 200 (ms).

[0176] The transmitter / receiver 11 of the destination terminal (terminal 10db) transmits delay information indicating a delay time of "200 (ms)" to the management system 50 via the communication network 2 through the management information session sei shown in Fig. 2 (step S85). This allows the management system 50 to grasp the delay time and the IP address "1.3.2.4" of the terminal 10db that is the sender of the delay information.

[0177] Next, the delay time management unit 60 of the management system 50 searches the terminal management DB 5003 (see FIG. 13) using the IP address "1.3.2.4" of the destination terminal (terminal 10db) as a search key to extract the corresponding terminal ID "01db", and further stores and manages the delay time "200 (ms)" indicated in the delay information in the delay time field of the record for the terminal ID "01db" in the session management table of the session management DB 5005 (see FIG. 15) (step S86).

[0178] Next, the quality determination unit 58 searches the quality control DB 5007 (see FIG. 18) using the above delay time “200 (ms)” as a search key, and extracts the image quality of the corresponding image data as “medium image quality,” thereby determining the image quality to be “medium image quality” (step S87).

[0179] Next, the transmitting / receiving unit 51 searches the relay device management DB 5001 (see FIG. 11) using the relay device ID "111a" associated with the terminal ID "01db" in the session management table of the session management DB (see FIG. 15) as a search key, and extracts the IP address "1.2.1.2" of the corresponding relay device 30a (step S88). Then, the transmitting / receiving unit 51 transmits quality information indicating the image quality "medium quality" of the image data determined in the above step 87 to the relay device 30a via the communication network 2 through the management information session sei shown in FIG. 2 (step S89). This quality information includes the IP address "1.3.2.4" of the destination terminal (terminal 10db) used as the search key in the above step S86. As a result, in the relay device 30a, the change quality management unit 34 stores and manages in the change quality management DB 3001 (see Figure 10) the IP address "1.3.2.4" of the destination terminal 10 (here, terminal 10db) and the image quality "medium image quality" of the relayed image data in association with each other (step S90).

[0180] Next, the terminal 10aa continues to transmit high-quality image data consisting of three resolutions, low resolution, medium resolution, and high resolution, and audio data to the relay device 30a through the image and audio data session sed in the same manner as in step S81 (step S91). As a result, in the relay device 30a, the data quality confirmation unit 33 uses the IP address "1.3.2.4" of the destination terminal (terminal 10db) as a search key to search the change quality management DB 3001 (see FIG. 10) and extracts the corresponding image quality "medium image quality" of the image data to be relayed in the same manner as in step S82 (step S92). In this embodiment, the confirmed image quality of the image data is "medium image quality", which is lower than the image quality "high image quality" of the image data received by the transmission / reception unit 31, so the data quality change unit 35 changes the image quality of the image data by suppressing the image quality of the image data from "high image quality" to "medium image quality" (step S93).

[0181] Then, the transmitting / receiving unit 31 transmits the image data, the image quality of which has been changed to "medium image quality", and the audio data, the sound quality of which has not been changed, to the terminal 10db via the communication network 2 by the image / audio data session sed (step S94). As a result, the destination terminal (terminal 10db) receives the medium image quality image data consisting of low resolution and medium resolution, and the audio data at the transmitting / receiving unit 11. Then, the display control unit 17 combines the image data of the two resolutions to display an image on the display 120, and the audio output unit 15b outputs audio based on the audio data.

[0182] In this way, if a delay occurs in reception at the destination terminal (terminal 10db) receiving the image data, the relay device 30a can change the image quality so as not to cause discomfort to people participating in the video conference.

[0183] <<Main Effects of This Embodiment>> As described above, according to this embodiment, the management system 50 creates a predetermined communication control message for causing a predetermined terminal 10 to control image communication based on image communication status information indicating the image communication status of each terminal 10, and transmits the communication control message to the terminal 10 that controls the image communication. This allows the transmission management system to cause a predetermined transmission terminal to perform image communication according to the image communication status of the other terminal 10, thereby preventing unnecessary image communication. This also has the effect of avoiding unnecessary power consumption, communication costs, etc.

[0184] Even if it is possible to grasp the LAN environment of the communication network 2, such as the IP address of the relay device 30, it is difficult to grasp the environment of the entire Internet 2i, so first, from the graspable environmental information, the number of relay devices 30 that relay image data and audio data is narrowed down to two or more. Then, before actually transmitting and receiving image data and audio data between the multiple terminals 10, pre-transmission information is transmitted and received in place of the image data and audio data, thereby achieving the effect of narrowing down to one relay device 30 that can actually relay the pre-transmission information the fastest.

[0185] That is, by selecting relay devices 30 to which two or more top IP addresses close to any of the IP addresses of the terminal 10 are assigned, two or more candidates for the relay device 30 to be finally used can be left. As a result, by actually transmitting and receiving pre-transmission information between the request source terminal and the destination terminal via each of the candidate relay devices 30, it is possible to narrow down the two or more candidate relay devices 30 to the relay device 30 that relayed the pre-transmission information with the shortest required time for transmission and reception. This provides the effect of realizing transmission and reception of image data or audio data of the highest possible quality under the current environment of the communication network 2.

[0186] In addition, in this embodiment, when narrowing down the relay devices 30, not only is a relay device 30 having an IP address close to the IP address of the terminal 10 that will be holding the video conference preferentially selected, but two or more relay devices 30 are selected in consideration of the maximum data transmission speed of each relay device 30. This provides the advantage of being able to narrow down candidates for relay devices 30 that are suited to the actual environment of the communication network 2.

[0187] Furthermore, in this embodiment, when narrowing down the relay devices 30, the relay devices 30 are narrowed down to those whose operating status is online, so that it is possible to narrow down the candidates for relay devices 30 that are more suited to the actual environment of the communication network 2.

[0188] [Supplementary description of the embodiment] The relay device 30, the management system 50, the program providing system 90, and the maintenance system 100 in the above-described embodiment may be constructed by a single computer, or may be constructed by a plurality of computers in which each unit (function or means) is divided and arbitrarily assigned. In addition, when the program providing system 90 is constructed by a single computer, the program transmitted by the program providing system 90 may be divided into a plurality of modules and transmitted, or may be transmitted without being divided. Furthermore, when the program providing system 90 is constructed by a plurality of computers, the plurality of modules may be transmitted from each computer in a divided state.

[0189] In addition, recording media such as CD-ROMs on which the terminal programs, relay device programs, or transmission management programs of the above embodiments are stored, as well as HD204 on which these programs are stored, and program providing system 90 equipped with this HD204, are all used as program products when the terminal programs, relay device programs, and transmission management programs are provided to users, etc., domestically or overseas.

[0190] Furthermore, in the above embodiment, the quality of the image data relayed by the relay device 30 is managed by focusing on the image resolution of the image data using the change quality management table shown in Fig. 10 and the quality management table shown in Fig. 18, but the quality is not limited to this, and other examples of the quality may be managed by focusing on the image depth of the image data, the sampling frequency of the sound in the sound data, the bit length of the sound in the sound data, etc. Also, the sound data may be transmitted and received by dividing it into data of three different resolutions (high resolution, medium resolution, and low resolution).

[0191] In addition, in Figs. 11, 13, and 15, the reception date and time are managed, but this is not limiting, and it is sufficient that at least the reception time of the reception date and time is managed.

[0192] Furthermore, in the above embodiment, the IP address of the relay device is managed in FIG. 11, and the IP address of the terminal is managed in FIG. 13. However, the present invention is not limited to this. If the relay device identification information is for identifying the relay device 30 on the communication network 2, or the terminal identification information is for identifying the terminal 10 on the communication network 2, the FQDN (Fully Qualified Domain Name) of each may be managed. In this case, the IP address corresponding to the FQDN is acquired by a well-known DNS (Domain Name System) server. Note that the "relay device identification information for identifying the relay device 30 on the communication network 2" may be expressed as "relay device connection destination information indicating the connection destination to the relay device 30 on the communication network 2" or "relay device destination information indicating the destination to the relay device 30 on the communication network 2". Similarly, the "terminal identification information for identifying the terminal 10 on the communication network 2" may be expressed as "terminal connection destination information indicating the connection destination to the terminal 10 on the communication network 2" or "terminal destination information indicating the destination to the terminal 10 on the communication network 2".

[0193] In the above embodiment, a video conference system has been described as an example of the transmission system 1, but the present invention is not limited thereto and may be a telephone system such as an IP (Internet Protocol) telephone or an Internet telephone. The transmission system 1 may also be a car navigation system. In this case, for example, one of the terminals 10 corresponds to a car navigation device mounted on a car, and the other terminal 10 corresponds to a management terminal or a management server of a management center that manages the car navigation, or a car navigation device mounted on another car. The transmission system 1 may also be a communication system for a mobile phone. In this case, for example, the terminal 10 corresponds to a mobile phone.

[0194] In addition, in the above embodiment, image data and audio data have been described as examples of content data, but the present invention is not limited to this and may be tactile (touch) data. In this case, the sensation of contact by the user on one terminal side is transmitted to the other terminal side. Furthermore, the content data may be olfactory (smell) data. In this case, the smell (odor) on one terminal side is transmitted to the other terminal side. Furthermore, the content data may be at least one of image data, audio data, tactile data, and olfactory data.

[0195] In addition, in the above embodiment, a case where a video conference is held using the transmission system 1 has been described, but this is not limited to this, and the system may also be used for meetings, general conversations between family members or friends, etc., or for the presentation of information in one direction. [Explanation of symbols]

[0196] 1. Transmission System 10 Transmission Terminal 11 Transmitting / receiving unit (an example of a receiving means, an example of a communication stopping means) 14 Imaging unit (an example of an imaging means) 16 Final Selection Section 16a Measurement section 16b Calculation part 16c Final Selection 17 Display control unit (an example of a display control means) 18 Delay detection unit 20 Message creation unit (an example of a creation means) 30 Relay Device 31 Transmitter / receiver 32 Status detection unit 33 Data Quality Check Department 34 Change Quality Control Department 35 Data Quality Change Division 50 Transmission Management System 51 Transmitting / receiving unit (an example of a transmitting means) 52 Terminal authentication unit 53 Status Management Unit 54 Terminal Extraction Unit 55 Terminal state extraction unit 56 Primary Filtering Section 56a Selection session ID generation unit 56b Terminal IP address extraction unit 56c Primary Selection Section 56d Priority determination section 57 Session Management Unit 58 Quality Decision Department 60 Delay Time Management Unit 61 Message creation unit (an example of a creation means) 70 Router 90 Program Delivery System 100 Maintenance System 101 CPU (an example of a control means) 120 Display (an example of a display means) 1000 storage section 3000 storage section 3001 Change Quality Control DB 5000 storage section 5001 Relay device management DB 5002 Terminal authentication management DB 5003 Terminal management DB (an example of terminal management means) 5004 Destination list management DB (an example of destination list management means) 5005 Session Management DB 5006 Priority management DB 5007 Quality control DB [Prior art documents] [Patent documents]

[0197] [Patent Document 1] JP 2008-131412 A

Claims

1. A transmission system including a first transmission terminal for transmitting and receiving image data between a second transmission terminal and a management system, The management system includes: a management means for managing first status information in a case where an imaging means of the second transmission terminal is turned on in the second transmission terminal while image data is being transmitted and received between the first transmission terminal and the second transmission terminal; a transmission means for transmitting, when an imaging means is turned off in the second transmission terminal, second status information changed from the first status information due to the imaging means of the second transmission terminal being turned off, to the first transmission terminal; The first transmission terminal, a receiving means for receiving the second status information while a first icon indicating that communication of image data acquired by an imaging means of the first transmission terminal has started is displayed on a display screen; a display control means for displaying, on a display screen of the second transmission terminal, a state indicating that communication of image data captured by the imaging means of the second transmission terminal is stopped, based on the second status information; the display control means displays, on a display screen, a second icon indicating that communication of image data captured by the imaging means of the first transmission terminal has been stopped when the imaging means of the first transmission terminal is turned off; A transmission system characterized in that the first transmission terminal starts communicating captured image data by accepting an operation on the second icon, while changing the display from the second icon to the first icon using the display control means.

2. 2. The transmission system according to claim 1, wherein said display control means displays said status on a display screen by displaying a message.

3. 3. The transmission system according to claim 1, wherein the image data is image data used in a video conference.

4. A transmission system as described in any one of claims 1 to 3, characterized in that the display screen includes a portion that displays image data captured by an imaging means of the first transmission terminal, and a portion that displays the first icon or the second icon.

5. 5. A transmission system according to any one of claims 1 to 4, characterized in that the first icon is a first camera icon, and the second icon is a second camera icon that has a different appearance from the first camera icon.

6. A first transmission terminal for transmitting and receiving image data between a second transmission terminal, a receiving means for receiving, when an imaging means is turned off in the second transmission terminal while transmitting and receiving image data to and from the second transmission terminal, second status information indicating that the imaging means of the second transmission terminal is off, which is changed from first status information indicating that the imaging means of the second transmission terminal is on due to the imaging means of the second transmission terminal being turned off, from a management system connected via a network; a display control means for displaying, on a display screen of the second transmission terminal, a state indicating that communication of image data captured by the imaging means of the second transmission terminal is stopped, based on the second status information, when the second status information is received while a first icon indicating that communication of image data captured by the imaging means of the first transmission terminal has started is being displayed on a display screen, the display control means displays, on a display screen, a second icon indicating that communication of image data captured by the imaging means of the first transmission terminal has been stopped when the imaging means of the first transmission terminal is turned off; The first transmission terminal is characterized in that it starts communicating captured image data by accepting an operation on the second icon, while changing the display from the second icon to the first icon using the display control means.

7. A transmission method executed by a transmission system including a first transmission terminal that transmits and receives image data between a second transmission terminal and a management system, comprising: The management system includes: a management step of managing first status information in the second transmission terminal when an imaging means of the second transmission terminal is turned on while image data is being transmitted and received between the first transmission terminal and the second transmission terminal; a transmission step of transmitting, when an imaging means is turned off in the second transmission terminal, second status information changed from the first status information due to the imaging means of the second transmission terminal being turned off, to the first transmission terminal; The first transmission terminal, a receiving step of receiving the second status information while a first icon indicating that communication of image data acquired by an imaging means of the first transmission terminal has started is displayed on a display screen; a display control step of displaying, on a display screen of the second transmission terminal, a state indicating that communication of image data captured by an imaging means of the second transmission terminal is stopped, based on the second status information; a display control step of displaying, on a display screen, a second icon indicating that communication of image data captured by the imaging means of the first transmission terminal has been stopped when the imaging means of the first transmission terminal is turned off; A transmission method characterized in that the first transmission terminal starts communicating captured image data by accepting an operation on the second icon, while changing the display from the second icon to the first icon on the display screen by the display control step.

8. A display method executed by a first transmission terminal which transmits and receives image data to and from a second transmission terminal, comprising: displaying on a display screen a first icon indicating that communication of image data acquired by an imaging means of the first transmission terminal has started; a display control step of receiving, when an imaging means of the second transmission terminal is turned off, second status information indicating that the imaging means of the second transmission terminal is off, which is changed from first status information indicating that the imaging means of the second transmission terminal is on due to the imaging means of the second transmission terminal being turned off, from a management system connected via a network, and, when the second status information is received while the first icon is being displayed on a display screen, displaying, on the display screen of the second transmission terminal, a status indicating that communication of image data captured by the imaging means of the second transmission terminal is stopped, based on the second status information; a display control step of displaying, on a display screen, a second icon indicating that communication of image data captured by the imaging means of the first transmission terminal has been stopped when the imaging means of the first transmission terminal is turned off; A display method characterized by starting communication of captured image data by accepting an operation on the second icon, while changing the display from the second icon to the first icon in the display control step.

9. A first transmission terminal for transmitting and receiving image data to and from a second transmission terminal, a receiving means for receiving, when an imaging means is turned off in the second transmission terminal while transmitting and receiving image data to and from the second transmission terminal, second status information indicating that the imaging means of the second transmission terminal is off, which is changed from first status information indicating that the imaging means of the second transmission terminal is on due to the imaging means of the second transmission terminal being turned off, from a management system connected via a network; a display control means for displaying, on a display screen of the second transmission terminal, a state indicating that communication of the image data captured by the imaging means of the second transmission terminal is stopped based on the second status information when the second status information is received while a first icon indicating that communication of the image data captured by the imaging means of the first transmission terminal has started is being displayed on a display screen of the second transmission terminal; A program that functions as The display control means further displays, on a display screen, a second icon indicating that communication of image data captured by the imaging means of the first transmission terminal has been stopped when the imaging means of the first transmission terminal is turned off; The program is characterized in that it causes the first transmission terminal to start communicating captured image data by accepting an operation on the second icon, while changing the display from the second icon to the first icon using the display control means.

10. 10. The program according to claim 9, wherein the display control means displays the state on a display screen by displaying a message.

11. 11. The program according to claim 9, wherein the image data is image data used in a video conference.

12. The program according to any one of claims 9 to 11, characterized in that the display screen includes a portion for displaying image data captured by an imaging means of the first transmission terminal, and a portion for displaying the first icon or the second icon.

13. 13. The program according to claim 9, wherein the first icon is a first camera icon, and the second icon is a second camera icon that is displayed differently from the first camera icon.

Images