Transmission terminal, transmission method, transmission system, and program

Abstract

To solve the following problem in a conventional technology: in using materials stored in a computer or the like other than a transmission terminal in a meeting, the computer must be connected to a transmission system to perform authentication through a transmission control system and determination of a display data destination, resulting in increased processing load through the transmission management system to connect a computer which is a transmission source of image data and voice data and is other than the transmission terminal, to the transmission system.SOLUTION: A destination of a relay device for transmitting image data and voice data is stored in a transmission terminal, and display data of materials from an external input device which is not managed by a transmission management system are accepted and the display data are transmitted to the relay device indicated by the stored destination. Accordingly, a partner in a convection is enabled to refer to the materials without performing a series of processing of making the external input device login the transmission management system or selecting a relay device. Furthermore, a load of managing the external input device by the transmission management system can be reduced.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a transmission terminal that transmits and receives display data of a screen to be shared with other transmission terminals. [Background technology]

[0002] In recent years, with the demand for reducing business trip expenses and time, transmission systems for video conferences and the like via communication networks such as the Internet have become widespread. In such transmission systems, video conferences can be realized by transmitting and receiving image data and audio data between multiple transmission terminals. In addition, the recent improvement of broadband environments has made it possible to transmit and receive high-quality image data and high-quality audio data, making it easier to understand the situation of the other party in a video conference and improving the quality of communication through conversation.

[0003] However, when a video conference is being held, it may be necessary not only to transmit audio data and image data to inform the other parties of the video conference of their status, but also to transmit display data on a screen showing materials to be used in the conference to the other parties so that they can refer to the materials.

[0004] Conventionally, whether sending voice data, image data, or display data, the data or information was sent to a distribution device via a network, and the distribution device, upon receiving the data, distributed the data or information to a transmission terminal. In this way, video conference participants could hear the other person's voice, see the other person's face, and refer to materials during the video conference (see Patent Document 1). Summary of the Invention [Problem to be solved by the invention]

[0005] However, when materials managed by a computer other than the transmission terminal are used in a conference, the conventional technology requires that the computer be managed by the transmission management system. In this way, when materials managed by a computer other than the transmission terminal that is the source of image data or audio data are used, a problem occurs in that the processing load on the transmission management system increases. [Means for solving the problem]

[0006] The invention of claim 1 is a transmission terminal that transmits image data and display data of a screen to be shared with another transmission terminal via a specified relay device, the transmission terminal comprising: a storage means for storing relay device information regarding the relay device to which the transmission terminal transmits the image data; a receiving means for receiving the display data from an external input device connected to the transmission terminal; and a transmitting means for transmitting the display data received by the receiving means to the relay device indicated by the relay device information stored in the storage means.

[0007] The invention of claim 3 is a transmission method executed by a transmission terminal that transmits image data and display data of a screen to be shared with another transmission terminal via a specified relay device, wherein the transmission terminal has a storage means for storing relay device information regarding the relay device to which the transmission terminal transmits the image data, and the transmission terminal executes a receiving step of receiving the display data from an external input device connected to the transmission terminal, and a transmitting step of transmitting the display data received in the receiving step to the relay device indicated by the relay device information stored in the storage means.

[0008] The invention according to claim 4 is characterized in that the transmission terminal performs each of the steps according to claim 3. Effect of the Invention

[0009] According to the present invention, a transmission terminal stores relay device information relating to a relay device that relays image data, and transmits display data displayed by an external input device to a relay device related to the stored relay device information. This allows the external input device to transmit display data without logging in to the transmission management system, thereby preventing an increase in the load on the transmission management system. [Brief description of the drawings]

[0010] [Figure 1] 1 is a schematic diagram of a transmission system according to an embodiment of the present invention; [Diagram 2] 1 is an external view of a transmission terminal according to an embodiment of the present invention; [Diagram 3] FIG. 2 is a hardware configuration diagram of a transmission terminal according to an embodiment of the present invention. [Figure 4] 1 is a hardware configuration diagram of a transmission management system, a relay device, a program providing server, or an external input device according to an embodiment of the present invention. [Diagram 5] 1 is a functional block diagram of each terminal, device, and system constituting a transmission system according to an embodiment of the present invention. [Figure 6] FIG. 2 is a conceptual diagram illustrating the image quality of image data. [Figure 7] FIG. 13 is a conceptual diagram illustrating a change quality management table. [Figure 8] FIG. 13 is a conceptual diagram illustrating a relay device management table. [Figure 9] FIG. 13 is a conceptual diagram illustrating a terminal authentication management table. [Figure 10] FIG. 13 is a conceptual diagram illustrating a terminal management table. [Figure 11] FIG. 13 is a conceptual diagram illustrating a destination list management table. [Figure 12] FIG. 13 is a conceptual diagram illustrating a session management table. [Figure 13] FIG. 13 is a conceptual diagram illustrating an address priority management table. [Figure 14] FIG. 13 is a conceptual diagram showing a transmission speed priority management table. [Figure 15]FIG. 13 is a conceptual diagram showing a quality control table. [Figure 16] FIG. 11 is a sequence diagram showing a process of managing state information indicating the operating state of each relay device. [Figure 17] FIG. 11 is a sequence diagram showing a preparation process for starting communication between transmission terminals. [Figure 18] FIG. 11 is a sequence diagram showing a process of narrowing down relay devices. [Figure 19] FIG. 11 is a process flow diagram showing a process for narrowing down relay devices. [Figure 20] FIG. 13 is a diagram showing a calculation state of priority points when performing a narrowing-down process for relay devices. [Figure 21] FIG. 11 is a sequence diagram showing a process in which a transmission terminal selects a relay device. [Figure 22] FIG. 11 is a process flow diagram showing a process of selecting a relay device at a transmission terminal. [Figure 23] 11 is a sequence diagram showing a process of transmitting and receiving image data and audio data between transmission terminals. FIG. [Figure 24] FIG. 2 is a functional block diagram of an external input device. [Diagram 25] 11 is a sequence diagram showing a process of displaying display data displayed by an external input device on a transmission terminal that is a conference partner. FIG. [Figure 26] 4 is an example of a screen displayed by an external input device. [Figure 27] 4 is an example of a screen on which the transmission terminal displays image data and display data. [Figure 28] FIG. 11 is a flow chart showing a process in which the external input device installs a display data acquisition unit. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] <<Overall configuration of the embodiment>> Hereinafter, an embodiment of the present invention will be described with reference to Fig. 1 to Fig. 28. Fig. 1 is a schematic diagram of a transmission system 1 according to an embodiment of the present invention, and the outline of this embodiment will be described first with reference to Fig. 1.

[0012] The transmission system also includes a data providing system that transmits content data in one direction from one transmission terminal to another transmission terminal via a transmission management system, and a communication system that transmits information, emotions, etc. between multiple transmission terminals via a transmission management system. This communication system is a system for transmitting information, emotions, etc. between multiple communication terminals (corresponding to "transmission terminals") via a communication management system (corresponding to "transmission management system"). Examples of such a communication system include a television or video conference system, a television telephone system, an audio conference system, an audio telephone system, and a PC (Personal Computer) screen sharing system.

[0013] In this embodiment, a transmission system, a transmission management system, and a transmission terminal will be described assuming a television or video conference system as an example of a communication system, a television or video conference management system as an example of a communication management system, and a television or video conference terminal as an example of a communication terminal. That is, the transmission terminal and the transmission management system of the present invention are applied not only to a television or video conference system, but also to a communication system or a transmission system.

[0014] The transmission system 1 shown in FIG. 1 is constructed by a plurality of transmission terminals (10aa, 10ab, ..., 10db), displays (120aa, 120ab, ..., 120db) for each transmission terminal (10aa, 10ab, ..., 10db), a plurality of relay devices (30a, 30b, 30c, 30d), a transmission management system 50, a program provision system 90, and a maintenance system 100.

[0015] In this embodiment, the term "transmission terminal 10" is used to refer to any of the transmission terminals (10aa, 10ab, ..., 10db), the term "display 120" is used to refer to any of the displays (120aa, 120ab, ..., 120db), and the term "relay device 30" is used to refer to any of the relay devices (30a, 30b, 30c, 30d).

[0016] A transmission terminal 10 transmits and receives image data, audio data, and the like to and from other transmission terminals 10. In this embodiment, a case where the image data is a moving image will be described, but the image data may be not only a moving image but also a still image. The image data may include both a moving image and a still image. A relay device 30 relays image data and audio data between a plurality of transmission terminals 10. A transmission management system 50 centrally manages the transmission terminals 10 and the relay devices 30.

[0017] The external input device 40 is connected to the transmission terminal 10 and transmits display data for displaying material data to the transmission terminal 10. The material data here refers to data used, for example, by word processing software, spreadsheet software, presentation software, or the like.

[0018] In addition, the multiple routers (70a, 70b, ..., 70f) shown in FIG. 1 select the optimal route for image data and audio data. In this embodiment, "router 70" is used to indicate any one of the routers (70a, 70b, ..., 70f). The program providing system 90 includes a hard disk (HD) (not shown) that stores a program for a transmission terminal for enabling the transmission terminal 10 to realize various functions or various means, and can transmit the program for a transmission terminal to the transmission terminal 10. The HD of the program providing system 90 also stores a program for a relay device for enabling the relay device 30 to realize various functions or various means, and can transmit the program for a relay device to the relay device 30. The HD of the program providing system 90 also stores a program for transmission management for enabling the transmission management system 50 to realize various functions or various means, and can transmit the program for transmission management to the transmission management system 50.

[0019] Moreover, the transmission terminal 10aa, the transmission terminal 10ab, the relay device 30a, and the router 70a are communicatively connected by a LAN 2a. The transmission terminal 10ba, the transmission terminal 10bb, the relay device 30b, and the router 70b are communicatively connected by a LAN 2b. Moreover, the LAN 2a and the LAN 2b are communicatively connected by a dedicated line 2ab including the router 70c, and are constructed in a predetermined region A. For example, the region A is Japan, the LAN 2a is constructed in an office in Tokyo, and the LAN 2b is constructed in an office in Osaka.

[0020] On the other hand, the transmission terminal 10ca, the transmission terminal 10cb, the relay device 30c, and the router 70d are communicatively connected by the LAN 2c. The transmission terminal 10da, the transmission terminal 10db, the relay device 30d, and the router 70e are communicatively connected by the LAN 2d. The LAN 2c and the LAN 2d are communicatively connected by a dedicated line 2cd including the router 70f, and are constructed in a predetermined region B. For example, the region B is America, the LAN 2c is constructed in an office in New York, and the LAN 2d is constructed in an office in Washington DC. The regions A and B are communicatively connected from the routers (70c, 70f) via the Internet 2i, respectively.

[0021] The transmission management system 50 and the program providing system 90 are communicatively connected to the transmission terminal 10 and the relay device 30 via the Internet 2i. The transmission management system 50 and the program providing system 90 may be installed in the area A or area B, or may be installed in an area other than these.

[0022] In this embodiment, the communication network 2 of the embodiment is constructed by the LAN 2a, the LAN 2b, the dedicated line 2ab, the Internet 2i, the dedicated line 2cd, the LAN 2c, and the LAN 2d.

[0023] In addition, in Fig. 1, four sets of numbers shown under each transmission terminal 10, each relay device 30, transmission management system 50, each router 70, and program providing system 90 simply indicate IP addresses in general IPv4. For example, the IP address of the transmission terminal 10aa is "1.2.1.3". Also, IPv6 may be used instead of IPv4. For the sake of simplicity, the following description will be given using IPv4. <<Hardware Configuration of the Embodiment>> Next, the hardware configuration of this embodiment will be described.

[0024] 2 is an external view of the transmission terminal according to the present embodiment. In the following description, the longitudinal direction of the transmission terminal 10 is defined as the X-axis direction, the direction perpendicular to the X-axis direction in a horizontal plane is defined as the Y-axis direction, and the direction perpendicular to the X-axis and Y-axis directions (vertical direction) is defined as the Z-axis direction.

[0025] 2, the transmission terminal 10 includes a housing 1100, an arm 1200, and a camera housing 1300. A front wall surface 1110 of the housing 1100 is provided with an intake surface (not shown) formed with a plurality of intake holes, and a rear wall surface 1120 of the housing 1100 is provided with an exhaust surface 1121 formed with a plurality of exhaust holes. This allows the outside air behind the transmission terminal 10 to be taken in through the intake surface (not shown) by driving a cooling fan built into the housing 1100, and to be exhausted to the rear of the transmission terminal 10 through the exhaust surface 1121. A sound pickup hole 1131 is formed in a right wall surface 1130 of the housing 1100, and sounds such as voice, noise, and the like can be picked up by a built-in microphone 114 described later.

[0026] An operation panel 1150 is formed on the right side wall 1130 of the housing 1100. This operation panel 1150 is provided with a plurality of operation buttons (108a to 108e), a power switch 109, and an alarm lamp 119, which will be described later, and also has a sound output surface 1151 formed with a plurality of sound output holes for passing sound output from a built-in speaker 115, which will be described later. In addition, a storage section 1160 is formed on the left side wall 1140 of the housing 1100 as a recess for storing the arm 1200 and the camera housing 1300. A plurality of connection ports (1132a to 1132c) for electrically connecting cables to an external device connection I / F 118, which will be described later, are provided on the right side wall 1130 of the housing 1100. Meanwhile, a left wall surface 1140 of the housing 1100 is provided with a connection port (not shown) for electrically connecting a cable 120c for the display 120 to an external device connection I / F 118 (described later).

[0027] In the following, "operation button 108" will be used to refer to any of the operation buttons (108a to 108e), and "connection port 1132" will be used to refer to any of the connection ports (1132a to 1132c).

[0028] Next, arm 1200 is attached to housing 1100 via torque hinge 1210, and configured so that arm 1200 can rotate up and down within a range of tilt angle θ1 of 135 degrees relative to housing 1100. Fig. 2 shows a state where tilt angle θ1 is 90 degrees.

[0029] The camera housing 1300 is provided with a built-in camera 1021, and can capture images of a user, a document, a room, and the like. The camera housing 1300 is also formed with a torque hinge 1310. The camera housing 1300 is attached to the arm 1200 via the torque hinge 1310. The camera housing 1300 is attached to the arm 1200 via the torque hinge 1310, and is configured to be rotatable in the up, down, left, and right directions with respect to the arm 1200 within a pan angle θ2 range of ±180 degrees, with the state shown in FIG. 2 being 0 degrees, and within a tilt angle θ3 range of ±45 degrees.

[0030] In addition, since the relay device 30, the management system 50, and the program providing system 90 each have the same appearance as a general server computer, a description of their appearance will be omitted.

[0031] 3 is a hardware configuration diagram of the transmission terminal 10 according to an embodiment of the present invention. As shown in FIG. 3, the transmission terminal 10 of the present embodiment includes a CPU (Central Processing Unit) 101 that controls the operation of the entire transmission terminal 10, a ROM (Read Only Memory) 102 that stores a program for the transmission terminal, a RAM (Random Access Memory) 103 used as a work area for the CPU 101, a flash memory 104 that stores various data such as image data and audio data, a solid state drive (SSD) 105 that controls reading or writing of various data from the flash memory 104 under the control of the CPU 101, a media drive 107 that controls reading or writing (storing) of data from a recording medium 106 such as a flash memory, an operation button 108 that is operated when selecting a destination of the transmission terminal 10, a power switch 109 for switching ON / OFF of the power of the transmission terminal 10, a network I / F 111 for transmitting data using a communication network 2 described later, a CCD (Charge Coupled Device) 104 for capturing an image of a subject and obtaining image data under the control of the CPU 101, and a 3D printer 106 for storing the image data. The terminal 10 includes a camera 112 (camera 112), an image sensor I / F 113 that controls the operation of the camera 112, a microphone 114 that inputs audio, a speaker 115 that outputs audio, an audio input / output I / F 116 that processes input and output of audio signals between the microphone 114 and the speaker 115 under the control of the CPU 101, a display I / F 117 that transmits image data to an external display 120 under the control of the CPU 101, an external device I / F 118 that transmits and receives various data to and from external devices, an alarm lamp 119 that notifies of abnormalities in various functions of the terminal 10, and bus lines 110 such as an address bus and a data bus for electrically connecting the above-mentioned components as shown in FIG.

[0032] The display 120 is a display unit configured with liquid crystal or organic electroluminescence (EL) that displays an image of a subject, operation icons, etc. The display 120 is also connected to the display I / F 117 by a cable 120c. This cable 120c may be a cable for analog RGB (VGA) signals, a cable for component video, or a cable for HDMI (High-Definition Multimedia Interface) or DVI (Digital Video Interactive) signals.

[0033] The camera 112 includes a lens and a solid-state imaging element that converts light into electric charges to digitize an image (video) of a subject. As the solid-state imaging element, a complementary metal oxide semiconductor (CMOS), a charge coupled device (CCD), or the like is used.

[0034] External devices such as an external camera, an external microphone, and an external speaker can be electrically connected to the external device connection I / F 118 by a USB (Universal Serial Bus) cable or the like inserted into a connection port 1132 of the housing 1100 shown in Fig. 2. When an external camera is connected, the external camera is driven in priority over the built-in camera 112 under the control of the CPU 101. Similarly, when an external microphone or an external speaker is connected, the external microphone or the external speaker is driven in priority over the built-in microphone 114 or the built-in speaker 115, respectively, under the control of the CPU 101.

[0035] The recording medium 106 is detachably attached to the transmission terminal 10. As long as it is a non-volatile memory that reads or writes data under the control of the CPU 101, it is not limited to the flash memory 104, but may be an EEPROM (Electrically Erasable and Programmable ROM) or the like. The camera 112 is a solid-state imaging element that converts light into electric charges to digitize an image (video) of a subject, and is not limited to a CCD, but may be a CMOS (Complementary Metal Oxide Semiconductor) or the like, as long as it captures an image of the subject. The display 120 is made of a liquid crystal or organic EL that displays an image of the subject, operation icons, and the like.

[0036] Furthermore, the transmission terminal program may be distributed by being recorded in a computer-readable recording medium such as the recording medium 106 in the form of an installable or executable file.

[0037] 4 is a hardware configuration diagram of a transmission management system according to an embodiment of the present invention. The transmission management system 50 includes a CPU 201 for controlling the overall operation of the transmission management system 50, a ROM 202 storing a transmission management program, a RAM 203 used as a work area for the CPU 201, a HD (Hard Disk) 204 for storing various data, a HDD (Hard Disk Drive) 205 for controlling the reading or writing of various data from the HD 204 under the control of the CPU 201, a media drive 207 for controlling the reading or writing (storage) of data from a recording medium 206 such as a flash memory, a display 208 for displaying various information such as a cursor, a menu, a window, a character, or an image, a network I / F 209 for transmitting data using a communication network 2 described later, a keyboard 211 having multiple keys for inputting characters, numbers, various instructions, etc., a mouse 212 for selecting and executing various instructions, selecting a processing target, moving the cursor, etc., a CD-ROM (Compact Disc Read Only Memory) as an example of a removable recording medium, The computer-readable medium includes a CD-ROM drive 214 that controls reading and writing of data from and to a CD-ROM (read-only memory) 213, an external device I / F 215 that transmits and receives information to and from external devices, and bus lines 210 such as an address bus and a data bus for electrically connecting the above-mentioned components as shown in FIG. 4.

[0038] The transmission management program may be distributed by being recorded in a computer-readable recording medium such as the recording medium 206 or CD-ROM 213 in the form of an installable or executable file.

[0039] Moreover, the external input device 40 has the same hardware configuration as the transmission management system 50, and therefore a description thereof will be omitted. However, the ROM 202 stores a program for the external input device for controlling the external input device 40. In this case as well, the program for the external input device may be distributed by being recorded in a computer-readable recording medium such as the recording medium 206 or CD-ROM 213 as a file in an installable or executable format.

[0040] Moreover, the relay device 30 has the same hardware configuration as the transmission management system 50, and therefore a description thereof will be omitted. However, a relay device program for controlling the relay device 30 is recorded in the ROM 202. In this case as well, the relay device program may be recorded in a computer-readable recording medium such as the recording medium 206 or CD-ROM 213 as a file in an installable or executable format and distributed.

[0041] Moreover, since the program providing system 90 has the same hardware configuration as the transmission management system 50, a description thereof will be omitted. However, a program providing program for controlling the program providing system 90 is recorded in the ROM 202. In this case, too, the program providing program may be recorded in the form of an installable or executable file on a computer-readable recording medium such as the recording medium 206 or CD-ROM 213 and distributed.

[0042] The maintenance system 100 is a computer for maintaining, managing, or repairing at least one of the terminal 10, the relay device 30, the management system 50, and the program providing system 90. For example, when the maintenance system 100 is installed domestically and the terminal 10, the relay device 30, the management system 50, or the program providing system 90 is installed overseas, the maintenance system 100 remotely performs maintenance such as maintaining, managing, or repairing at least one of the terminal 10, the relay device 30, the management system 50, the authentication system 80, and the program providing system 90 via the communication network 2.

[0043] In addition, the maintenance system 100 performs maintenance such as managing model numbers, serial numbers, sales destinations, maintenance inspections, or failure histories of at least one of the terminals 10, relay devices 30, management systems 50, and program provision systems 90 without going through the communication network 2.

[0044] Furthermore, the program provision system 90 and the maintenance system 100 have the same hardware configuration as the management system 50, and therefore a description thereof will be omitted. However, a program provision program for controlling the program provision system 90 is recorded on the HD 204. In this case as well, the program provision program may be recorded in a computer-readable recording medium such as the recording medium 206 or CD-ROM 213 as a file in an installable or executable format and distributed. Furthermore, the program for the program provision system may be stored in the ROM 202 instead of the HD 204.

[0045] As another example of the removable recording medium, the information may be recorded on a computer-readable recording medium such as a CD-R (Compact Disc Recordable), a DVD (Digital Versatile Disk), or a Blu-ray disc. Next, the functional configuration of this embodiment will be described. Fig. 5 is a functional block diagram of each terminal, device, and system constituting the transmission system 1 of this embodiment. In Fig. 5, the transmission terminal 10, the relay device 30, and the transmission management system 50 are connected so as to be able to perform data communication via the communication network 2. The external input device 40 is connected so as to be able to transmit and receive data to and from the transmission terminal 10. Fig. 24 is a functional block diagram of the external input device 40 constituting the transmission system 1 of this embodiment. In addition, the program providing system 90 shown in Fig. 1 is omitted in Fig. 5 because it is not directly related to the communication of the video conference.

[0046] <Functional configuration of the transmission terminal> The transmission terminal 10 includes a transmission / reception unit 11, an operation input reception unit 12, a login request unit 13, an imaging unit 14a, an image display control unit 14b, an audio input unit 15a, an audio output unit 15b, a selection processing unit 16, a delay detection unit 17, an external information transmission / reception unit 18, and a storage / readout processing unit 19. Each of these units is a function or means realized by any of the components shown in Fig. 3 operating according to an instruction from the CPU 201 in accordance with a program stored in the ROM 202. The transmission terminal 10 also includes a storage unit 1000 constructed by the SSD 105 shown in Fig. 3.

[0047] (Each functional part of the transmission terminal) Next, each unit of the transmission terminal will be described in detail. The transmission / reception unit 11 of the transmission terminal 10 is realized by the network I / F 111 shown in FIG. 3, and transmits and receives various data (information) to and from other terminals, devices, or systems via the communication network 2. The operation input reception unit 12 is realized by the operation button 108 and the power switch 109 shown in FIG. 3, and receives various inputs from the user. For example, when the user turns on the power switch 109 shown in FIG. 3, the operation input reception unit 12 shown in FIG. 5 receives the power ON and turns on the power. The login request unit 13 is realized by a command from the CPU 101 shown in FIG. 3, and automatically transmits login request information indicating a login request and the current IP address of the transmission terminal 10ab to the transmission management system 50 via the communication network 2 from the transmission / reception unit 11 upon receiving the power ON.

[0048] The imaging unit 14a is realized by the camera 112 and the imaging element I / F 113 shown in Fig. 3, captures an image of a subject, and outputs image data obtained by capturing the image. The image display control unit 14b is realized by the display I / F 117 shown in Fig. 3, and performs control for transmitting image data to the external display 120. The audio input unit 15a is realized by the microphone 114 and the audio input / output I / F 116 shown in Fig. 3, inputs the user's voice, converts the voice into an audio signal, and outputs audio data related to the audio signal. The audio output unit 15b is realized by the speaker 115 and the audio input / output I / F 116 shown in Fig. 3, and converts the audio signal related to the audio data into audio and outputs it.

[0049] In order to perform a final narrowing process to narrow down the number of relay devices 30 to one relay device 30, the selection processing unit 16 realizes a measurement unit 16a, a calculation unit 16b, and a selection unit 16c by an instruction from the CPU 101 shown in FIG. 3. Among them, the measurement unit 16a measures the reception date and time when the pre-transmission information is received by the transmission / reception unit 11 for each piece of pre-transmission information described later received by the transmission / reception unit 11. For each piece of pre-transmission information whose reception date and time is measured by the measurement unit 16a, the calculation unit 16b calculates the required time from transmission to reception of the pre-transmission information based on the difference between the measured reception time and the transmission date and time included in the pre-transmission information. The selection unit 16c selects the relay device 30 that has relayed the pre-transmission information that required the shortest required time among the required times calculated by the calculation unit 16b, thereby finally selecting one relay device.

[0050] The delay detection unit 17 is realized by an instruction from the CPU 101 shown in FIG. 3, and detects a delay time (ms) of image data or audio data transmitted from another transmission terminal 10 via the relay device 30. The external information transmission / reception unit 18 transmits and receives data to and from an external device via the external device I / F 215. The storage / read processing unit 19 is executed by the SSD 105 shown in FIG. 3, and performs processing to store various data in the storage unit 1000 and to read various data stored in the storage unit 1000. The storage unit 1000 stores a terminal ID (Identification) for identifying the transmission terminal 10, a password, image data, audio data, a relay device ID for identifying the relay device 30 that transmits various data, an IP address of a destination terminal, and the like. The storage unit 1000 also stores a display data acquisition unit 451 and a display data transmission unit 452 that are transmitted to the external input device 40 and operate on the external input device 40. The display data acquisition unit 451 is for the external input device 40 to acquire display data, and the display data transmission unit 452 is for transmitting the display data acquired by the display data acquisition unit 451 to the transmission terminal 10. The display data represents image data of an image displayed on the screen of a display device in a format such as JPEG (Joint Photographic Experts Group) or Bitmap, and drawing commands in a format such as GDI (Graphics Device Interface).

[0051] The terminal ID in this embodiment and the relay device ID described later indicate identification information such as language, characters, symbols, or various marks used to uniquely identify the transmission terminal 10 and the relay device 30, respectively. The terminal ID and the relay device ID may be identification information that combines at least two of the above languages, characters, symbols, and various marks. In the following description, the transmission terminal 10 as a request source requesting the start of a video conference is referred to as a "request source terminal 10A," and the transmission terminal 10 as a destination to which the request is made is referred to as a "destination terminal 10B."

[0052] <Functional configuration of external input device> As shown in Fig. 24, the external input device 40 has a transmission / reception unit 41, a connection detection unit 42, an installation determination unit 43, a program acquisition unit 44, an operation input reception unit 46, a display control unit 47, a mounting unit 48, and a storage / readout processing unit 49. Each of these units is a function or means realized by any of the components shown in Fig. 4 operating by an instruction from the CPU 201 according to the program stored in the ROM 202. The external input device 40 also has a storage unit 4000 constructed by the HDD 205 shown in Fig. 4. The external input device 40 also has an OS (Operating System) such as Windows (registered trademark) installed, and thus has the function of executing a program when connected to another device.

[0053] <Functional parts of external input device> Next, each unit of the external input device 40 will be described in detail. The transmission / reception unit 41 of the external input device 40 is realized by the network I / F 209 shown in FIG. 4, and transmits and receives various data (information) to and from the transmission terminal 10. The connection detection unit 42 detects that data transmission and reception with the external device is possible through the external device input / output I / F 215. The installation determination unit 43 determines whether or not the display data acquisition unit 451 and the display data transmission unit 452 are installed in the external input device 40. The program acquisition unit 44 acquires the display data acquisition unit 451 and the display data transmission unit 452 from the storage unit 1000 of the transmission terminal 10 connected via the transmission / reception unit 41, and installs them. The operation input acceptance unit 46 accepts input by a user's operation. The display control unit 47 displays an image read by a storage / read processing unit 49 (described later) on the display device 400. The mounting unit 48 mounts the storage units of various devices connected to the external input device 40. 4, and performs processing such as storing various data in the storage unit 4000 and reading out various data stored in the storage unit 4000. The storage unit 4000 stores material data and the like.

[0054] <Functional configuration of relay device> Next, the functions or means of the relay device 30 will be described. The relay device 30 has a transmitting / receiving unit 31, a state detecting unit 32, a data quality checking unit 33, a change quality managing unit 34, a data quality changing unit 35, and a storage / reading processing unit 39. Each of these units is a function or means realized by any of the components shown in Fig. 4 operating according to an instruction from the CPU 201 in accordance with the program stored in the ROM 202. The relay device 30 also has a storage unit 3000 constructed by the HD 204 shown in Fig. 4.

[0055] (Change quality control table) In the storage unit 3000, a change quality management DB 3001 is constructed, which is configured by a change quality management table as shown in Fig. 7. In the change quality management table, the IP address of the transmission terminal 30 as the relay destination of the image data and the image quality of the image data relayed by the relay device 30 to this relay destination are associated and managed.

[0056] Here, the image resolution of the image data handled in this embodiment will be described. As shown in FIG. 6(a), there is a low-resolution image consisting of 160 pixels horizontally and 120 pixels vertically, which is a base image; as shown in FIG. 6(b), there is a medium-resolution image consisting of 320 pixels horizontally and 240 pixels vertically; and as shown in FIG. 6(c), there is a high-resolution image consisting of 640 pixels horizontally and 480 pixels vertically. Among these, when passing through a narrowband path, low-quality image data consisting only of low-resolution image data which is a base image is relayed. When the bandwidth is relatively wide, low-quality image data consisting of low-resolution image data which is a base image and medium-quality image data consisting of medium-resolution image data is relayed. Also, when the bandwidth is very wide, high-quality image data consisting of low-resolution image data which is a base image, medium-resolution image data, and high-resolution image data is relayed. For example, in the change quality management table shown in FIG. 7, when the relay device 30 relays image data to the destination terminal 10db having the IP address "1.3.2.4", the image quality (image quality) of the relayed image data is "high quality".

[0057] <Functional parts of the relay device> Next, a detailed description will be given of each functional configuration of the relay device 30. In the following, when describing each part of the relay device 30, the relationship between each part of the relay device 30 and the main components for realizing each part of the relay device 30 among the components shown in FIG.

[0058] The transmitting / receiving unit 31 of the relay device 30 shown in Fig. 5 is realized by the network I / F 209 shown in Fig. 4, and transmits and receives various data (information) to and from other terminals, devices, or systems via the communication network 2. The status detection unit 32 is realized by commands from the CPU 201 shown in Fig. 4, and detects the operating status of the relay device 30 having this status detection unit 32. The operating status may be "ONLINE", "OFFLINE", or "out of order".

[0059] The data quality confirmation unit 33 is realized by an instruction from the CPU 201 shown in Fig. 4, and searches the change quality management table (see Fig. 7) using the IP address of the destination terminal 10B as a search key, and extracts the corresponding image quality of the image data to be relayed, thereby confirming the image quality of the image data to be relayed. The change quality management unit 34 is realized by an instruction from the CPU 201 shown in Fig. 4, and changes the contents of the change quality management DB 3001 based on quality information (described later) sent from the transmission management system 50. For example, during a video conference by transmitting and receiving high-quality image data between the request source terminal 10aa with the terminal ID "01aa" and the destination terminal 10db with the terminal ID "01db", if a delay in receiving image data occurs at the destination terminal 10db due to a request source terminal bb and the destination terminal ca starting a video conference via the communication network 2, the relay device 30 needs to lower the image quality of the image data that has been relayed so far from high image quality to medium image quality. In such a case, the contents of the change quality management DB 3001 are changed based on the quality information indicating medium image quality so that the image quality of the image data relayed by the relay device 30 is lowered from high image quality to medium image quality.

[0060] 4, and changes the image quality of the image data sent from the source terminal 10 based on the changed contents of the changed quality management DB 3001. The storage / read processing unit 39 is realized by the HDD 205 shown in FIG. 4, and performs processing to store various data in the storage unit 3000 and to read out various data stored in the storage unit 3000.

[0061] <Functional configuration of the transmission management system> Next, the functions or means of the transmission management system 50 will be described. The transmission management system 50 has a transmitting / receiving unit 51, a terminal authentication unit 52, a status management unit 53, a terminal extraction unit 54, a terminal status acquisition unit 55, a selection unit 56, a session management unit 57, a quality determination unit 58, a storage / readout processing unit 59, and a delay time management unit 60. Each of these units is a function or means realized by any of the components shown in Fig. 4 operating according to an instruction from the CPU 201 in accordance with the program stored in the ROM 202. The transmission management system 50 also has a storage unit 5000 constructed by the HD 204 shown in Fig. 4.

[0062] (Relay device management table) In the storage unit 5000, a relay device management DB 5001 is constructed, which is configured by a relay device management table as shown in Fig. 8. In this relay device management table, the operation state of each relay device 30, the reception date and time when the state information indicating the operation state was received by the transmission management system 50, the IP address of the relay device 30, and the maximum data transmission rate (Mbps) of the relay device 30 are associated and managed for each relay device ID of each relay device 30. For example, in the relay device management table shown in Fig. 8, it is shown that the operation state of the relay device 30a with the relay device ID "111a" is "ONLINE", the date and time when the state information was received by the transmission management system 50 is "13:00 on Nov. 10, 2009", the IP address of this relay device 30a is "1.2.1.2", and the maximum data transmission rate of this relay device 30a is 100 Mbps.

[0063] (Terminal authentication management table) Furthermore, the storage unit 5000 stores a terminal authentication management DB 5002 that is configured by a terminal authentication management table as shown in Fig. 9. In this terminal authentication management table, each password is associated with each terminal ID of all the transmission terminals 10 managed by the transmission management system 50. For example, the terminal authentication management table shown in Fig. 9 indicates that the terminal ID of the transmission terminal 10aa is "01aa" and the password is "aaaa".

[0064] (Terminal Management Table) The storage unit 5000 also has a terminal management DB 5003 configured by a terminal management table as shown in Fig. 10. In this terminal management table, the operation state of each transmission terminal 10, the reception date and time when login request information described later was received by the transmission management system 50, and the IP address of the transmission terminal 10 are associated and managed for each terminal ID of each transmission terminal 10. For example, the terminal management table shown in Fig. 10 indicates that the operation state of the transmission terminal 10aa with the terminal ID "01aa" is "ONLINE", the date and time when the login request information was received by the transmission management system 50 is "13:40 on Nov. 10, 2009", and the IP address of this transmission terminal 10aa is "1.2.1.3".

[0065] (Destination list management table) Furthermore, the storage unit 5000 has a destination list management DB 5004 configured by a destination list management table as shown in Fig. 11. In this destination list management table, the terminal IDs of the destination terminals 10B registered as candidates for the destination terminals 10B are all associated and managed with the terminal ID of the request source terminal 10A requesting the start of a video conference. For example, the destination list management table shown in Fig. 11 indicates that the destination terminals 10B candidates to which the request source terminal 10aa with the terminal ID "01aa" can request the start of a video conference are the transmission terminal 10ab with the terminal ID "01ab", the transmission terminal 10ba with the terminal ID "01ba", and the transmission terminal 10db with the terminal ID "01db". The destination terminals 10B candidates are updated by being added or deleted in response to an addition or deletion request from the request source terminal 10A to the transmission management system 50.

[0066] (Session management table) 12 is stored in the storage unit 5000. In the session management table, the relay device ID of the relay device 30 used to relay the image data and audio data, the terminal ID of the request source terminal 10A, the terminal ID of the destination terminal 10B, the reception delay time (ms) when the image data is received at the destination terminal 10B, and the reception date and time when the delay information indicating the delay time is sent from the destination terminal 10B and received by the transmission management system 50 are associated and managed for each selection session ID used to execute a session for selecting the relay device 30. For example, in the session management table shown in FIG. 12, the relay device 30a (relay device ID "111a") selected in the session executed using the selection session ID "se1" relays image data and audio data between the request source terminal 10aa with the terminal ID "01aa" and the destination terminal 10db with the terminal ID "01db", and the delay time of the image data at the destination terminal 10db at "14:00 on Nov. 10, 2009" is 200 (ms). When a video conference is held between two transmission terminals 10, the reception date and time of the delay information may be managed based on the delay information transmitted from the request source terminal 10A instead of the destination terminal 10B. However, when a video conference is held between three or more transmission terminals 10, the reception date and time of the delay information is managed based on the delay information transmitted from the transmission terminal 10 on the receiving side of the image data and audio data.

[0067] (Address priority management table) Furthermore, the storage unit 5000 has a priority management DB 5006 configured by an address priority management table as shown in FIG. 13. In this address priority management table, the addresses are managed in association with each other so that the address priority points are higher depending on whether the four dot address parts of a general IPv4 IP address are the same or different. For example, in the address priority management table shown in FIG. 13, when the three values ​​from the top to the bottom of the dot address are the same for an IP address, the address priority point is "5". When the two values ​​from the top to the bottom of the dot address are the same for an IP address, the address priority point is "3". In this case, whether the values ​​of the lowest dot addresses are the same or not does not affect the priority. When the values ​​of the highest dot addresses are the same and the second highest dot addresses are different, the address priority point is "1". In this case, whether the values ​​of the third highest dot addresses and the lowest dot addresses are the same or not does not affect the priority. When the values ​​of the highest dot addresses are different for an IP address, the address priority point is "0". In this case, whether the second, third, and lowest dot addresses have the same value has no bearing on the priority.

[0068] (Transmission speed priority management table) The priority management DB 5006 constructed in the storage unit 5000 also includes a transmission speed priority management table as shown in FIG. 14. In this transmission speed priority management table, the transmission speed priority points are associated and managed so that they are higher according to the value of the maximum data transmission speed (Mbps) in the relay device 30. For example, in the transmission speed priority management table shown in FIG. 14, when the maximum data transmission speed in the relay device 30 is 1000 Mbps or more and less than 1000 Mbps, the transmission speed priority point is "5". When the maximum data transmission speed in the relay device 30 is 100 Mbps or more and less than 1000 Mbps, the transmission speed priority point is "3". When the maximum data transmission speed in the relay device 30 is 10 Mbps or more and less than 100 Mbps, the transmission speed priority point is "1". When the maximum data transmission speed in the relay device 30 is less than 10 Mbps, the transmission speed priority point is "0".

[0069] (Quality control table) Furthermore, the storage unit 5000 has a quality control DB 5007 configured by a quality control table as shown in Fig. 15. In this quality control table, the image quality (image quality) of the image data relayed by the relay device 30 is associated and managed according to the delay time (ms) of the image data in the request source terminal 10A or the destination terminal 10B.

[0070] (Each functional part of the transmission management system) Next, a detailed description will be given of each functional unit of the transmission management system 50. In the following, when describing each unit of the transmission management system 50, the relationship between each unit and the main components for realizing each unit of the transmission management system 50 among the components shown in FIG. 4 will also be described.

[0071] The transmission / reception unit 51 is executed by the network I / F 209 shown in Fig. 4, and transmits and receives various data (information) to and from other terminals, devices, or systems via the communication network 2. The terminal authentication unit 52 uses the terminal ID and password included in the login request information received via the transmission / reception unit 51 as search keys, searches the terminal authentication management DB 5002 in the storage unit 5000, and determines whether the same terminal ID and password are managed in the terminal authentication management DB 5002, thereby performing terminal authentication. The status management unit 53 manages the operating status of the request source terminal 10A that has made the login request by storing and managing the terminal ID of the request source terminal 10A, the operating status of the request source terminal 10A, the reception date and time when the login request information was received by the transmission management system 50, and the IP address of the request source terminal 10A in a terminal management table (see Fig. 11) in association with each other.

[0072] The terminal extraction unit 54 searches the destination list management table (see FIG. 11) using the terminal ID of the request source terminal 10A that has made the login request as a key, and extracts the terminal ID by reading out the terminal IDs of candidates for destination terminals 10B that can communicate with the request source terminal 10A. The terminal extraction unit 54 also searches the destination list management table (see FIG. 11) using the terminal ID of the request source terminal 10A that has made the login request as a key, and extracts the terminal IDs of other request source terminals 10A that have registered the terminal ID of the request source terminal 10A as a candidate for destination terminal 10B.

[0073] The terminal status acquisition unit 55 searches the terminal management DB table (see FIG. 10) using the terminal IDs of the candidates for the destination terminal 10B extracted by the terminal extraction unit 54 as a search key, and reads out the operating status for each terminal ID extracted by the terminal extraction unit 54. This allows the terminal status acquisition unit 55 to acquire the operating status of the candidates for the destination terminal 10B that can communicate with the request source terminal 10A that has made the login request. The terminal status acquisition unit 55 also searches the terminal management table (see FIG. 10) using the terminal IDs extracted by the terminal extraction unit 54 as a search key, and acquires the operating status of the request source terminal 10A that has made the login request.

[0074] The narrowing down unit 56 has a selection session ID generating unit 56a, a terminal IP address extracting unit 56b, a primary selecting unit 56c, and a priority determining unit 56d in order to support a final narrowing down process for narrowing down a plurality of relay devices 30 to one relay device 30, and to perform a primary narrowing down process before the final narrowing down process. Among them, the selection session ID generating unit 56a generates a selection session ID used for executing a session for selecting a relay device 30. The terminal IP address extracting unit 56b extracts the IP address of each corresponding transmission terminal 10 by searching the terminal management table (see FIG. 10) based on the terminal ID of the request source terminal 10A and the terminal ID of the destination terminal 10B included in the start request information sent from the request source terminal 10A. The primary selecting unit 56c selects a relay device 30 by selecting the relay device ID of the relay device 30 whose operation state is "ONLINE" from among the relay devices 30 managed in the relay device management table (see FIG. 8).

[0075] Further, the primary selection unit 56c searches the relay device management table (see FIG. 8) based on the IP address of the request source terminal 10A and the IP address of the destination terminal 10B extracted by the terminal IP address extraction unit 56b, and investigates whether each dot address in the IP address of the selected relay device 30 is the same as or different from each dot address in the IP addresses of the request source terminal 10A and the destination terminal 10B. Furthermore, the primary selection unit 56c further selects relay devices 30 by selecting, for each relay device, the top two relay devices 30 with the highest points out of the integrated points obtained by integrating the higher address priority points for the transmitting terminal 10 and the transmission speed priority points.

[0076] In this embodiment, the top two relay devices 30 with the highest points are selected, but this is not limited to this. If it is possible to narrow down as many relay devices 30 as possible, the top three or more relay devices 30 with the highest points may be selected.

[0077] The priority determination unit 56d refers to the priority management table (see FIG. 13) and determines address priority points for each relay device 30 surveyed by the primary selection unit 56c. The priority determination unit 56d also searches the priority management table (see FIG. 14) based on the maximum data transmission speed of each relay device 30 managed in the relay device management table (see FIG. 8) to determine transmission speed priority points for each relay device 30 narrowed down by the primary selection unit 56c.

[0078] Session management unit 57 stores and manages the selection session ID generated by selection session ID generation unit 56a, the terminal ID of the request source terminal, and the terminal ID of the destination terminal in a session management table (see FIG. 12) in storage unit 5000. In addition, session management unit 57 stores and manages the relay device ID of the relay device 30 finally selected by selection unit 16c of transmission terminal 10 for each selection session ID in the session management table (see FIG. 12).

[0079] The quality determination unit 58 searches the quality management table (see FIG. 15) using the delay time as a search key, and extracts the image quality of the corresponding image data, thereby determining the image quality of the image data to be relayed by the relay device 30. The storage / read processing unit 59 is executed by the HDD 205 shown in FIG. 4, and performs processing to store various data in the storage unit 5000 and to read various data stored in the storage unit 5000. The delay time management unit 60 searches the terminal management table (see FIG. 10) using the IP address of the destination terminal 10B as a search key, extracts the corresponding terminal ID, and further stores and manages the delay time indicated by the delay information in the delay time field of the record including the extracted terminal ID in the session management table (see FIG. 12). <<Processing and Operation of the Embodiment>> The above is an explanation of the configuration and functions (or means) of the transmission system 1 according to this embodiment, and next, a processing method in the transmission system 1 according to this embodiment will be explained using FIG. 16 to FIG. 23 and FIG. 25 to FIG. 28. FIG. 16 is a sequence diagram showing a process of managing state information indicating the state of each relay device 30 transmitted from each relay device 30 to the transmission management system 1. FIG. 17 is a sequence diagram showing a process of preparation for starting communication between a plurality of transmission terminals 10. FIG. 18 is a sequence diagram showing a process of narrowing down the relay devices 30. FIG. 19 is a process flow diagram showing a process of narrowing down the relay devices 30. FIG. 20 is a diagram showing a calculation state of points when narrowing down the relay devices 30. FIG. 21 is a sequence diagram showing a process of the transmission terminal 10 selecting the relay device 30. FIG. 22 is a process flow diagram showing a process of the transmission terminal 10 selecting the relay device 30. FIG. 23 is a sequence diagram showing a process of transmitting and receiving image data and audio data between the transmission terminals 10. FIG. 25 is a sequence diagram showing a process of displaying the display data displayed by the external input device 40 on the transmission terminal 10, which is the other party in the conference. FIG. 26 is an example of a screen displayed by the external input device 40. FIG. 27 is an example of a screen displayed by the transmission terminal 10 displaying the image data and the display data. FIG. 28 is a flow diagram showing a process in which the external input device installs the display data acquisition unit.

[0080] First, a process of managing the status information indicating the status of each relay device 30 transmitted from each relay device 30 to the transmission management system 50 will be described with reference to FIG. 16. First, in each relay device 30, the status detection unit 32 shown in FIG. 5 periodically detects the operation status of the relay device 30 (S1-1 to S1-4). Then, in order to have the transmission management system 50 manage the operation status of each relay device 30 in real time, the transmission / reception unit 31 of each relay device 30 periodically transmits each status information to the transmission management system 50 via the communication network 2 (steps S2-1 to S2-4). Each status information includes the relay device ID of each relay device 30 and the operation status detected by the status detection unit 32 of the relay device 30 related to each relay device ID. In this embodiment, the relay devices (30a, 30b, 30d) are operating normally and are "ONLINE", while the relay device 30c is operating but is "OFFLINE" due to some malfunction in the program for executing the relay operation of the relay device 30c.

[0081] Next, in the transmission management system 50, the transmitting / receiving unit 51 receives each piece of status information sent from each relay device 30, and stores and manages the status information for each relay device ID in the relay device management table (see FIG. 8) of the storage unit 5000 via the storage / read processing unit 59 (steps S3-1 to S3-4). As a result, in the relay device management table as shown in FIG. 8, any of the operating states of "ON line", "OFF line", or "out of order" is stored and managed for each relay device ID. At this time, the reception date and time when the status information was received by the transmission management system 50 is also stored and managed for each relay device ID. Note that when no status information is sent from the relay device 30, the field portion of the operating state and the field portion of the reception date and time in each record of the relay device management table shown in FIG. 8 are blank, or show the operating state and reception date and time at the time of the previous reception, respectively.

[0082] Next, with reference to FIG. 17, a preparation process before starting communication between the transmission terminal 10aa and the transmission terminal 10db will be described. First, when a user turns on the power switch 109 shown in FIG. 3, the operation input reception unit 12 shown in FIG. 5 receives the power ON and turns on the power (step S21). Then, the login request unit 13, upon receiving the power ON, automatically transmits login request information indicating a login request from the transmission / reception unit 11 to the transmission management system 50 via the communication network 2 (step S22). This login request information includes a terminal ID and a password for identifying the transmission terminal 10aa, which is the request source, as the request source. The terminal ID and password are data that are read from the storage unit 1000 via the storage / read processing unit 19 and sent to the transmission / reception unit 11. When the login request information is transmitted from the transmission terminal 10aa to the transmission management system 50, the transmission management system 50, which is the receiving side, can grasp the IP address of the transmission terminal 10ab, which is the transmitting side.

[0083] Next, the terminal authentication unit 52 of the transmission management system 50 searches the terminal authentication management table (see FIG. 9) in the storage unit 5000 using the terminal ID and password included in the login request information received via the transmission / reception unit 51 as search keys, and performs terminal authentication by determining whether the same terminal ID and password are managed in the terminal authentication management DB 5002 (step S23). If the terminal authentication unit 52 determines that the login request is from a transmission terminal 10 with valid usage authority because the same terminal ID and password are managed by the terminal authentication unit 52, the status management unit 53 stores the terminal ID of the transmission terminal 10aa, the operating status, the reception date and time when the login request information was received, and the IP address of the transmission terminal 10aa in the terminal management table (see FIG. 10) in association with each other (step S24). As a result, in the terminal management table shown in FIG. 10, the transmission terminal ID "01aa" is managed in association with the operation state "ONLINE", the reception date and time "2009.11.10.13:40", and the terminal IP address "1.2.1.3".

[0084] Then, the transmitting / receiving unit 51 of the transmission management system 50 transmits authentication result information indicating the authentication result obtained by the terminal authentication unit 52 to the request source terminal 10aa that has made the login request via the communication network 2 (step S25). In this embodiment, a case where the terminal authentication unit 52 judges that the terminal has a valid use right will be described below.

[0085] The terminal extraction unit 54 of the transmission management system 50 searches the destination list management table (see FIG. 11) using the terminal ID "01aa" of the request source terminal 10aa that has made the login request as a search key, and reads out and extracts the terminal IDs of candidates for the destination terminal 10B that can communicate with the request source terminal 10aa (step S26). Here, the terminal IDs "01ab", "01ba", and "01db" of the destination terminals (10ab, 10ba, 10db) corresponding to the terminal ID "01aa" of the request source terminal 10aa are extracted.

[0086] Next, the terminal status acquisition unit 55 searches the terminal management table (see FIG. 10) using the terminal IDs ("01ab", "01ba", "01db") of the candidates for the destination terminal 10B extracted by the terminal extraction unit 54 as search keys, and acquires the operating status of each of the transmission terminals (10ab, 10ba, 10db) by reading out the operating status ("OFFLINE", "ONLINE", "ONLINE") for each terminal ID extracted by the terminal extraction unit 54 (step S27).

[0087] Next, the transmitting / receiving unit 51 transmits destination status information including the terminal ID ("01ab", "01ba", "01db") used as the search key in step S27 above and the operation status ("OFFLINE", "ONLINE", "ONLINE") of the corresponding destination terminal (10ab, 10ba, 10db) to the request source terminal 10aa via the communication network 2 (step S28). This allows the request source terminal 10aa to grasp the current operation status ("OFFLINE", "ONLINE", "ONLINE") of each of the transmission terminals (10ab, 10ba, 10db) that are candidates for the destination terminal 10B that can communicate with the request source terminal 10aa.

[0088] Furthermore, the terminal extraction unit 54 of the transmission management system 50 searches the destination list management table (see FIG. 11) using the terminal ID "01aa" of the request source terminal 10aa that has made the login request as a search key, and extracts the terminal IDs of other request source terminals 10A that have registered the terminal ID "01aa" of the request source terminal 10aa as a candidate for the destination terminal 10B (step S29). In the destination list management table shown in FIG. 11, the terminal IDs of the other request source terminals 10A that are extracted are "01ab", "01ba", and "01db".

[0089] Next, the terminal status acquisition unit 55 of the transmission management system 50 searches the terminal management table (see FIG. 10) using the terminal ID “01aa” of the request source terminal 10aa that made the login request as a search key, and acquires the operating status of the request source terminal 10aa that made the login request (step S30).

[0090] Then, the transmitting / receiving unit 51 transmits destination status information including the terminal ID "01aa" and the operating status "ONLINE" of the request source terminal 10aa acquired in the above step S30 to the transmitting terminals (10ba, 10db) having the operating status "ONLINE" in the terminal management table (see FIG. 10) among the transmitting terminals (10ab, 10ba, 10db) related to the terminal IDs ("01ab", "01ba", and "01db") extracted in the above step S29 (steps S31-1, S31-2). When the transmitting / receiving unit 51 transmits the destination status information to the transmitting terminals (10ba, 10db), the transmitting terminals refer to the IP addresses of the terminals managed in the terminal management table shown in FIG. 10 based on the respective terminal IDs ("01ba", "01db"). This makes it possible to communicate the terminal ID "01aa" and operating status "ONLINE" of the request source terminal 10aa that made the login request to each of the other destination terminals (10db, 10ba) that can communicate with the request source terminal 10aa that made the login request.

[0091] Meanwhile, in another transmission terminal 10, similarly to step S21 above, when a user turns on the power switch 109 shown in FIG. 4, the operation input receiving unit 12 shown in FIG. 5 receives the power ON and performs processing similar to the processing of steps S22 to S31-1 and 31-2 above, so a description thereof will be omitted.

[0092] Next, the process of narrowing down the relay devices 30 will be described with reference to Fig. 18. In this embodiment, the request source terminal 10aa can communicate with at least one of the transmission terminals (10ba, 10db) whose operating status is ONLINE, among the transmission terminals 10 as destination candidates, based on the destination status information received in step S28. Hereinafter, a case will be described where the user of the request source terminal 10aa selects to start communication with the destination terminal 10db.

[0093] First, when a user presses the operation button 108 shown in Fig. 3 to select the transmission terminal 10db, the operation input reception unit 12 shown in Fig. 5 receives a request to start communication with the transmission terminal 10db (step S41). Then, the transmission / reception unit 11 of the transmission terminal 10aa transmits start request information indicating a desire to start communication, including the terminal ID "01aa" of the request source terminal 10aa and the terminal ID "01db" of the destination terminal 10db, to the transmission management system 50 (step S42). As a result, the transmission / reception unit 51 of the transmission management system 50 receives the start request information and also knows the IP address "1.2.1.3" of the request source terminal 10aa, which is the sender. Then, based on the terminal ID "01aa" of the request source terminal 10aa and the terminal ID "01db" of the destination terminal 10db included in the start request information, the status management unit 53 changes the operation status field of the record including the terminal ID "01aa" and the terminal ID "01db" in the terminal management table (see FIG. 10) to "in call" (step S43). Note that in this state, the request source terminal 10aa and the destination terminal 10db have not started communication (call), but are in a call state, and when another transmission terminal 10 tries to communicate with the request source terminal 10aa or the destination terminal 10db, a voice or display indicating a so-called in-call state is output.

[0094] Next, a process of executing a session for selecting a relay device 30 will be described in steps S44 to S48 and steps S61-1 to 66. First, selection session ID generating unit 56a generates a selection session ID used for executing a session for selecting a relay device 30 (step S44). Then, session managing unit 57 associates and stores and manages the selection session ID "se1" generated in step S44, the terminal ID "01aa" of request source terminal 10aa, and the terminal ID "01db" of destination terminal 10db in a session management table (see FIG. 12) in storage unit 5000 (step S45).

[0095] Next, the narrowing-down unit 56 of the transmission management system 50 performs a primary narrowing-down of the relay devices 30 for relaying communication between the request source terminal 10aa and the destination terminal 10db based on the relay device management DB 5001, the terminal management DB 5003, and the priority management DB 5006 (step S46).

[0096] Here, the process in step S46 will be described in more detail with reference to Fig. 19. First, the terminal IP address extraction unit 56b searches the terminal management table (see Fig. 10) based on the terminal ID "01aa" of the request source terminal 10aa and the terminal ID "01db" of the destination terminal 10db included in the start communication information sent from the request source terminal 10aa, and extracts the IP addresses ("1.2.1.3", "1.3.2.4") of the corresponding transmission terminals (10aa, 10db) (step S46-1). Next, the primary selection unit 56c selects the relay device IDs (111a, 111b, 111d) of the relay devices (30a, 30b, 30d) that are "ONLINE" among the operation states of the relay devices 30 managed in the relay device management table (see Fig. 8) (step S46-2). In addition, the primary selection unit 56c searches the relay device management table (see FIG. 8) based on the IP address "1.2.1.3" of the request source terminal 10aa and the IP address "1.3.2.4" of the destination terminal 10db extracted in step S46-1 above, and investigates whether each dot address in the IP addresses ("1.2.1.2.", "1.2.2.2", "1.3.2.2") of the relay devices (30a, 30b, 30d) selected in step S46-2 above is the same as or different from each dot address in the IP addresses ("1.2.1.3", "1.3.2.4") of the request source terminal 10aa and the destination terminal 10db (step S46-3).

[0097] Next, the priority determination unit 57c refers to the priority management table (see FIG. 13) and determines the address priority points for each relay device (30a, 30b, 30d) investigated in step S46-3 (step S46-4). The result of this determination process is shown in a table in FIG. 20. FIG. 20 is a diagram showing the calculation state of the priority points when narrowing down the relay devices 30. In FIG. 20, the address priority points, transmission speed priority points, and integrated points are shown for each relay device ID. The address priority points further show the points for the request source terminal 10aa and the points for the destination terminal 10db for each relay device 30. The integrated point is the sum of the higher of the two address priority points and the transmission speed priority point.

[0098] In this embodiment, the IP address "1.2.1.2" of the relay device 30a is "same.same.same.different" to the IP address "1.2.1.3" of the request source terminal 10aa, so the address priority point is "5" as shown in Fig. 20. Also, the IP address "1.2.1.2" of the relay device 30a is "same.different.different.different" to the IP address "1.3.2.4" of the destination terminal 10db, so the address priority point is "1". Also, the IP address "1.2.2.2" of the relay device 30b is "same.same.different.different" to the IP address "1.2.1.3" of the request source terminal 10aa, so the address priority point is "3". Furthermore, since the IP address "1.2.2.2" of relay device 30b is "same.different.same.different" to the IP address "1.3.2.4" of destination terminal 10db, the address priority point is "1." Furthermore, the IP address "1.3.2.2" of relay device 30d is "same.different.different.different" to the IP address "1.2.1.3" of request source terminal 10aa, the address priority point is "1." Furthermore, since the IP address "1.3.2.2" of relay device 30d is "same.same.same.different" to the IP address "1.3.2.4" of destination terminal 10db, the address priority point is "5."

[0099] Next, returning to FIG. 19, the priority determination unit 57d searches the priority management table (see FIG. 14) based on the maximum data transmission rate of each relay device 30 managed in the relay device management table (see FIG. 8), thereby determining the transmission rate priority points for each relay device (30a, 30b, 30d) narrowed down by the first narrowing down process in the above step S46-2 (step S46-5). In this embodiment, as shown in FIG. 8, the maximum data transmission rate of the relay device 30a is 100 (Mbps), so with reference to the transmission rate priority shown in FIG. 14, the transmission rate priority is 3 points. Similarly, when the maximum data transmission rate of the relay device 30b is calculated, it is 1000 (Mbps), so the transmission rate priority is 5 points. Similarly, when the maximum data transmission rate of the relay device 30d is calculated, it is 10 (Mbps), so the transmission rate priority is 1 point.

[0100] Next, the primary selection unit 56c selects the top two relay devices 30 with the highest points among the integrated points obtained by integrating the address priority points of the transmission terminals (10aa, 10db) and the transmission speed priority points for each relay device (30a, 30b, 30d) (step 46-6). In this embodiment, as shown in Fig. 20, the relay device IDs (111a, 111b, 111d) have integrated points of "8", "8", and "6", respectively, so the relay device 30a associated with the relay device ID "111a" and the relay device 30b associated with the relay device ID "111b" are selected.

[0101] When the narrowing down process in step S46 is completed, the transmitting / receiving unit 51 shown in Fig. 5 transmits relay device narrowing down information for conveying the number of narrowed down relay devices 30 to the destination terminal 10db via the communication network 2 (step S47). This relay device narrowing down information includes the number of relay devices 30 narrowed down in step S46, "2", the terminal ID "01aa" of the request source terminal 10aa, and the selection session ID "se1". As a result, the transmission terminal 10db can grasp the number of relay devices 30 and which transmission terminal 10 has requested to start a video conference in the execution of the session in the selection session ID "se1", and can also grasp the IP address "1.1.1.2" of the transmission management system 50 that is the source of the relay device narrowing down information.

[0102] Then, the transmission terminal 10db transmits reception completion information indicating that reception of the relay device selection information has been completed from the transmitting / receiving unit 11 to the transmission management system 50 via the communication network 2 (step S48). This reception completion information includes the session ID "se1". This enables the transmission management system 50 to grasp the IP address "1.3.2.4" of the destination terminal 10db, which is the source, as well as the fact that transmission of the number of relay devices executed with the session ID "se1" has been completed.

[0103] Next, the process of the destination terminal 10aa selecting the relay device 30 will be described with reference to FIG. 21. First, before starting the video conference, the transmission management system 50 transmits pre-relay request information requesting relay in advance to each of the relay devices (30a, 30b) narrowed down in step S46 (steps S61-1, 61-2). This pre-relay request information includes the session ID "se1", the IP address "01aa" of the request source terminal 10aa, and the destination terminal 10db. This allows the relay devices (30a, 30b) to know which selection session it belongs to, what the request source terminal 10A is, and what the destination terminal 10B is, and also to know the IP address "1.1.1.2" of the transmission management system 50 that is the sender of the pre-relay request information.

[0104] Next, each of the relay devices (30a, 30b) transmits pre-transmission request information indicating that the relay device (30a, 30b) as its own device transmits pre-transmission information including a ping (Packet Internet Groper) (described later) to the request source terminal 10aa recognized in the above steps S61-1, 61-2 via the communication network 2 from the transmission / reception unit 31 before the start of the video conference (steps S62-1, 62-2). This pre-transmission information includes the session ID "se1". As a result, the request source terminal 10aa is able to recognize that the pre-transmission information is to be transmitted to each of the relay devices (30a, 30b) in the selection process of the relay device 30 executed with the session ID "se1", and is able to recognize the IP addresses ("1.2.1.2", "1.2.2.2") of the relay devices (30a, 30b) that are the transmission sources of the pre-transmission request information.

[0105] The transmission management system 50 does not directly notify the request source terminal 10ba of the IP address of the destination terminal 10db, but notifies the relay device 10aa of the IP address of the destination terminal 10db as in step S61-1 above, and then requests the relay device 10aa to transmit pre-transmission request information to itself (relay device 10aa) as in step 61-2 above. This is to ensure security by preventing each transmission terminal 10 from being informed of the IP addresses of the other transmission terminals 10.

[0106] Next, the request source terminal 10aa transmits pre-transmission information from the transmission / reception unit 11 to the relay devices (30a, 30b) via the communication network 2 (steps S63-1, 63-2). This pre-transmission information is transmitted to the destination terminal 10db via the relay devices (30a, 30b) in place of the image data and voice data prior to transmission of the image data and voice data, and is used to measure the time required from transmission by the request source terminal 10aa to reception by the destination terminal db. This pre-transmission information also includes a ping for confirming that the request source terminal 10aa, the relay devices (30a, 30b), and the destination terminal 10db are connected to be able to communicate with each other, the transmission date and time when the pre-transmission information was transmitted from the request source terminal 10aa, and the session ID "se1". As a result, each relay device (30a, 30b) can recognize that pre-transmission information has been sent during the execution of a session with the selection session ID "se1", and can also recognize the IP address "1.2.1.3" of the requesting terminal 10aa, which is the sender of this pre-transmission information.

[0107] Next, each relay device (30a, 30b) relays the pre-transmission information to the IP address "1.3.2.4" of the destination terminal 10db included in the pre-relay request information received in the above steps S61-1, 61-2 (steps S64-1, 64-2). As a result, the destination terminal 10db can recognize that the pre-transmission information has been sent in the execution of the session with the session ID "se1", and can also recognize the IP addresses ("1.2.1.2", "1.2.2.2") of the relay devices (30a, 30b) that are the senders (relay sources) of the pre-transmission information.

[0108] Next, the selection processing unit 16 of the destination terminal 10db narrows down the selection to one relay device 30 that will ultimately relay the image data and audio data in the video conference based on the pre-transmission information (step 65).

[0109] Here, the process in step S65 will be described in more detail with reference to FIG. 5 and FIG. 22. First, the measurement unit 16a of the selection processing unit 16 shown in FIG. 5 measures the reception date and time when the transmission / reception unit 11 of the transmission terminal 10db receives each piece of pre-transmission information relayed by each relay device (30a, 30b) (step S65-1). Next, the calculation unit 16b calculates the required time from transmission to reception of each piece of pre-transmission information based on the difference between the reception date and time and the transmission date and time included in the pre-transmission information for each piece of pre-transmission information whose reception time has been measured (step S65-2). Next, the selection unit 16c judges whether or not all pieces of pre-transmission information corresponding to the number "2" of relay devices 30 to be relayed have been received in the execution of the session with the session ID "se1" (step S65-3). Then, if all pieces of pre-transmission information have not been received (NO), the selection unit 16c judges whether or not a predetermined time (here, one minute) has elapsed since the transmission terminal 10db received the pre-transmission information (step S65-4). Furthermore, if the predetermined time has not elapsed (NO), the process returns to step S65-1. On the other hand, if all the information has been received in step S65-3 (YES), or if the predetermined time has elapsed in step S65-4 (YES), the selection unit 16c selects one relay device 30 that has relayed the pre-transmission information that required the shortest required time among the required times calculated by the calculation unit 16b so far (step S65-5). In this embodiment, an example is shown in which the relay device 30a is selected because the pre-transmission information relayed by the relay device 30a required a shorter time from transmission to reception than the pre-transmission information relayed by the relay device 30b.

[0110] In the above embodiment, the relay device 30a is narrowed down on the destination terminal 10db side, but this is not limited to the above. The destination terminal 10db may transmit all required time information indicating the time required from transmission to reception of the pre-transmission information to the request source terminal 10aa or the transmission management system 50, so that one relay device 30a is finally narrowed down on the request source terminal 10aa side or the transmission management system 50 side.

[0111] Next, the destination terminal 10db transmits selection information indicating that the relay device 30a has been selected from the transmitting / receiving unit 11 to the transmission management system 50 via the communication network 2 (step S66). This selection information includes the session ID "se1" and the relay device ID "111a" of the selected relay device 30a. This allows the transmission management system 50 to know that the relay device 30a has been selected in the execution of the session with the session ID "se1", and also to know the IP address "1.3.2.4" of the transmission terminal 10db which is the source of the selection information.

[0112] Next, the session management unit 57 of the transmission management system 50 stores and manages the relay device ID "111a" of the relay device 30a finally selected as the one in the relay device ID field of the record including the selection session ID "se1" in the session management table of the session management DB 5005 (see FIG. 12) (step S67-1), and the transmission / reception unit 51 transmits the relay device ID "111a" and the IP address "1.3.2.4" of the destination terminal 10db to the request source terminal 10aa (step S67-21). Then, the transmission / reception unit 51 of the transmission management system 50 transmits relay start request information indicating a request to start relaying to the relay device 30a via the communication network 2 (step S68). This relay start request information includes the IP addresses ("1.2.1.3", "1.3.2.4") of the request source terminal 10aa and the destination terminal 10db to be relayed. As a result, the relay device 30a establishes a session for communicating three types of image data, low resolution, medium resolution, and high resolution, and audio data between the transmission terminals (10aa, 10db) (step S69). As a result, the transmission terminals (10aa, 10db) can start a video conference.

[0113] In the above step S47, the transmission management system 50 transmits the relay device selection information to the destination terminal 10db, and then the destination terminal 10db performs the relay device selection process (step S65) through steps S48 to S64-1 and 64-2. However, this is not limited to the above, and in the above step S47, the transmission management system 50 may transmit the relay device selection information to the request source terminal 10aa, and the sender and receiver of each piece of information may be switched between the request source terminal 10aa and the destination terminal 10db until steps S64-1 and 64-2. This allows the request source terminal 10aa to perform the relay device selection process instead of the above step S65, and also to transmit the selection information instead of the above step S66.

[0114] Next, a process of transmitting and receiving image data and audio data to hold a video conference between the request source terminal 10aa and the destination terminal 10db will be described with reference to FIG. 5 and FIG. 23. First, the request source terminal 10aa transmits image data of a subject captured by the imaging unit 14a and audio data of audio input by the audio input unit 15a from the transmitting and receiving unit 11 via the communication network 2 to the relay device 30a (step S81). In this embodiment, high-quality image data consisting of three resolutions, low resolution, medium resolution, and high resolution, as shown in FIG. 6, and audio data are transmitted. As a result, the relay device 30a receives the image data and audio data of the above three resolutions at the transmitting and receiving unit 31. Then, the data quality confirmation unit 33 searches the change quality management table (see FIG. 7) using the IP address "1.3.2.4" of the destination terminal 10db as a search key, and extracts the image quality of the image data to be relayed, thereby confirming the image quality of the image data to be relayed (step S82). In this embodiment, since the image quality of the confirmed image data is "high quality" and is the same as the image quality of the image data received by the transmitting / receiving unit 31, the image data with its image quality and the audio data with its sound quality are transferred to the destination terminal 10db (step S83). As a result, the destination terminal 10db receives the image data and audio data at the transmitting / receiving unit 11, and the image display control unit 14b displays an image based on the image data on the display 120, and the audio output unit 15b outputs audio based on the audio data.

[0115] Next, the delay detection unit 17 of the transmission terminal 10db detects the delay time of the image data received by the transmission / reception unit 11 at regular time intervals (for example, every second) (step S84). In this embodiment, the following description will be given for the case where the delay time is 200 (ms).

[0116] The transmitter / receiver 11 of the destination terminal 10db transmits delay information indicating the delay time "200 (ms)" to the transmission management system 50 via the communication network 2 (step S85). This allows the transmission management system 50 to grasp the delay time and the IP address "1.3.2.4" of the transmission terminal 10db that is the sender of the delay information.

[0117] Next, the delay time management unit 60 of the transmission management system 50 searches the terminal management table (see FIG. 10) using the IP address "1.3.2.4" of the destination terminal 10db as a search key to extract the corresponding terminal ID "01db", and further stores and manages the delay time "200 (ms)" indicated in the delay information in the delay time field of the record for the terminal ID "01db" in the session management table (see FIG. 12) of the session management DB 5005 (step S86).

[0118] Next, the quality determination unit 58 searches the quality control table (see FIG. 15) using the delay time “200 (ms)” as a search key, and extracts the image quality of the corresponding image data, “medium image quality,” thereby determining the image quality to be “medium image quality” (step S87).

[0119] Next, the transmitting / receiving unit 51 searches the relay device management DB table (see FIG. 8) using the relay device ID "111a" associated with the terminal ID "01db" in the session management table (see FIG. 12) as a search key, and extracts the IP address "1.2.1.2" of the corresponding relay device 30a (step S88). Then, the transmitting / receiving unit 51 transmits quality information indicating the image quality "medium quality" of the image data determined in the above step S87 to the relay device 30a via the communication network 2 (step S89). This quality information includes the IP address "1.3.2.4" of the destination terminal 10db used as the search key in the above step S86. As a result, in the relay device 30a, the change quality management unit 34 stores and manages the IP address "1.3.2.4" of the destination transmission terminal 10 (here, the destination terminal 10db) and the image quality "medium image quality" of the relayed image data in the change quality management table (see Figure 7) in association with each other (step S90).

[0120] Next, the transmission terminal 10aa continues to transmit high-quality image data consisting of three types of image quality, low quality, medium quality, and high quality, and voice data to the relay device 30a in the same manner as in step S81 above (step S91). As a result, in the relay device 30a, the data quality confirmation unit 33 uses the IP address "1.3.2.4" of the destination terminal 10db as a search key to search the change quality management table (see FIG. 7) and extracts the corresponding image quality "medium quality" of the image data to be relayed in the same manner as in step S82 above, thereby confirming the image quality of the image data to be relayed (step S92). In this embodiment, since the confirmed image quality of the image data is "medium quality", which is lower than the image quality "high quality" of the image data received by the transmission / reception unit 31, the data quality change unit 35 changes the image quality of the image data by suppressing the image quality of the image data from "high quality" to "medium quality" (step S93). Then, the transmitting / receiving unit 31 transmits the image data, the image quality of which has been changed to "medium image quality," and the audio data, the audio quality of which has not been changed, to the transmission terminal 10db via the communication network 2 (step S94). In this way, when a delay in reception occurs at the destination terminal 10db receiving the image data, the relay device 30a can change the image quality so as not to give a sense of incongruity to the people participating in the video conference.

[0121] Next, a process of sharing the entire screen displaying the material data stored in the storage unit 4000 of the external input device 40 after the relay device 30 is determined will be described with reference to Fig. 25. Here, an example will be described in which information displayed by the external input device 40aa connected to the transmission terminal 10aa is displayed on the transmission terminal 10db, which is the destination terminal.

[0122] As described above, once the relay device 30 has been determined, the relay device ID "111a" and the IP address "1.3.2.4" of the destination terminal 10db sent by the transmission management system 50 in S67-21 are received by the transmitter / receiver unit 11 of the transmission terminal 10aa, and the received relay device ID "111a" and IP address "1.3.2.4" are stored in the memory unit 1000 by the storage / read processing unit 19 (step S67-22).

[0123] When the external input device 40aa and the transmission terminal 10aa are connected to each other, the connection detection unit 42 of the external input device 40aa detects this (step S70). When the connection detection unit 42 detects that the external input device 40aa and the transmission terminal 10aa are connected to each other, the installation determination unit 43 determines whether or not the display data transmission unit 452 has already been installed, as shown in FIG. 28 (step S71). If it is determined in step S71 that the display data transmission unit 452 has not been installed, the program acquisition unit 44 acquires and installs the data transmission unit 452 stored in the transmission terminal 10aa (step S72). When the display data transmission unit 452 is installed in step S72, the external input device 40aa requests the transmission terminal 10aa for permission to allow the display data acquisition unit 451 to execute processing (step S73). When the transmission terminal 10aa permits the external input device 40aa to execute the process of the display data acquisition unit 451, the display data acquisition unit 451 acquires display data (step S74). Next, the display data transmission unit 452 transmits the display data acquired by the display data acquisition unit 451 to the transmission terminal 10aa (step S75).

[0124] If it is determined in step S71 that the display data transmission unit 452 is installed, the process proceeds to step S73 and subsequent steps.

[0125] When the external information transmitting / receiving unit 18 of the destination transmission terminal 10aa receives the display data, the storage / read processing unit 19 acquires the relay device ID "111a" and the IP address "1.3.2.4" of the destination transmission terminal 10db stored in the storage unit 1000 (step S77). Then, the transmitting / receiving unit 11 transmits the display data and the IP address "1.3.2.4" of the destination transmission terminal 10db to the relay device 30 indicated by the relay device ID "111a" acquired in step S77 (step S78). When the relay device 30 receives the display data transmitted from the transmission terminal 10aa in step S78, the relay device 30 changes the quality of the display data based on the IP address "1.3.2.4" of the transmission terminal 10db (step S79) and transmits the display data to the transmission terminal 10db (step S80). The details of the process of step S77 are the same as the process of changing the quality of the audio data and image data (steps S81 to S94) described above, and therefore the description will be omitted. When the transmitter / receiver 11 of the transmission terminal 10db receives the display data transmitted from the relay device 30, the image display control unit 14b displays it. In the example shown in FIG. 27, an image displayed on the external input device 40aa is displayed on the left side of the screen based on the display data, and image data captured by the imaging unit 14a of the transmission terminal 10aa and transmitted by the transmitter / receiver 11 is displayed on the upper right part of the screen. In addition, image data captured by the imaging unit 14a of the transmission terminal 10db is displayed on the lower right part of the screen.

[0126] <<Main Effects of the Embodiment>> As described above, according to the present embodiment, the relay device ID of the relay device 30 that relays the audio data and image data is stored in the storage unit 1000 of the transmission terminal 10aa, so that the display data of the material data stored in the storage unit 4000 of the external input device 40 that is not managed by the transmission management system 50 can be transmitted to the transmission terminal 10db that is the other party in the conference. As a result, the transmission management system 50 does not need to perform a process of authenticating the external input device 40, and therefore the management load can be reduced.

[0127] In addition, even when it is desired to share a screen displayed on an external input device 40 that does not include a display data acquisition unit 451 or a display data transmission unit 452, the display data acquisition unit 451 and the display data transmission unit 452 are provided when a connection with the transmission terminal 40 is made, so that the screen can be shared. Even if it is possible to obtain the environment of LANs 2a to 2d, such as the IP addresses of relay devices 30, in the communication network 2, it is difficult to obtain the environment of the Internet 2i, so first, from the available environmental information, the relay devices 30 that relay image data and audio data are narrowed down to two or more. Then, before actually transmitting and receiving image data and audio data between the multiple transmission terminals 10, pre-transmission information is transmitted and received in place of the image data and audio data, thereby achieving the effect of narrowing down to one relay device 30 that can actually relay the pre-transmission information the fastest.

[0128] That is, by selecting relay devices 30 to which two or more top IP addresses close to any of the IP addresses of the transmission terminal 10 are assigned, two or more candidates for the relay device 30 to be finally used can be left. As a result, by actually transmitting and receiving pre-transmission information between the request source terminal 10A and the destination terminal 10B via each candidate relay device 30, it is possible to narrow down the two or more candidate relay devices 30 to the relay device 30 that relayed the pre-transmission information with the shortest required time for transmission and reception. This provides the effect of realizing transmission and reception of image data or audio data of the highest possible quality under the current environment of the communication network 2.

[0129] In addition, in this embodiment, when narrowing down the relay devices 30, not only is a relay device 30 having an IP address close to the IP address of the transmission terminal 10 that will be holding the video conference preferentially selected, but two or more relay devices 30 are selected in consideration of the maximum data transmission speed of each relay device 30. This provides the advantage of being able to narrow down candidates for relay devices 30 that are suited to the actual environment of the communication network 2.

[0130] Furthermore, in this embodiment, when narrowing down the relay devices 30, the relay devices 30 are narrowed down from among those whose operating status is ONLINE, so that it is possible to narrow down candidates for relay devices 30 that are more suited to the actual environment of the network 2.

[0131] <<Supplementary embodiment>> The process of acquiring the display data may use a mirror driver. The mirror driver can transmit the generated display data directly to the USB driver without going through a program. In this case, the mirror driver generates the display data, and the USB driver transmits the generated display data to the transmission terminal 10 via the external device I / F 118. In this case, since the program does not need to acquire the display data, it is possible to reduce resources for executing the program.

[0132] In the above embodiment, the program acquires image data generated by the display driver and transmits it to the transmission terminal 10. However, the program may acquire drawing commands generated by the GDI and transmit them to the transmission terminal 10aa. Since the drawing commands have a smaller capacity than image data, it is possible to reduce the network load. The display data acquisition unit 451 may acquire only the display data displayed on a specific screen among multiple screens on the virtual display, and the display data transmission unit 452 may transmit the display data. As a result, image data displayed on screens other than the specific screen is not shared with the conference partner, and therefore security can be improved by, for example, preventing highly confidential materials from being shown.

[0133] Furthermore, the transmission management system 50, the program providing system 90, and the maintenance system 100 in each of the above embodiments may be constructed by a single computer, or may be constructed by multiple computers in which each section (function or means) is divided and arbitrarily assigned. Furthermore, when the program providing system 90 is constructed by a single computer, the program transmitted by the program providing system 90 may be divided into multiple modules and transmitted, or may be transmitted without being divided. Furthermore, when the program providing system 90 is constructed by multiple computers, the multiple modules may be transmitted from each computer in a divided state.

[0134] In addition, the recording medium on which the program for the transmission terminal, the program for the relay device, and the program for transmission management of the above-mentioned embodiment are stored, as well as the HD204 on which these programs are stored, and the program providing system 90 equipped with this HD204 are all used as program products when the program for the transmission terminal, the program for the relay device, and the program for transmission management are provided to users, etc., domestically or overseas.

[0135] Furthermore, in the above embodiment, the quality control table shown in FIG. 7 and the quality control table shown in FIG. 15 are used to manage the image quality of the image data relayed by the relay device 30 by focusing on the image resolution of the image data as an example of the image quality of the image data. However, this is not limited to this, and other examples of quality may be managed by focusing on the image quality depth of the image data, the sampling frequency of the audio in the audio data, the bit length of the audio in the audio data, etc.

[0136] In addition, in Figs. 8, 10, and 12, the reception date and time are managed, but this is not limiting, and it is sufficient that at least the reception time of the reception date and time is managed.

[0137] Further, in the above embodiment, the IP address of the relay device is managed in FIG. 8 and the IP address of the transmission terminal is managed in FIG. 10, but the present invention is not limited thereto. If the relay device identification information is for identifying the relay device 30 on the communication network 2 or the terminal identification information is for identifying the transmission terminal 10 on the communication network 2, the FQDN (Fully Qualified Domain Name) of each may be managed. In this case, the IP address corresponding to the FQDN is acquired by a well-known DNS (Domain Name System) server. Note that the "relay device identification information for identifying the relay device 30 on the communication network 2" may be expressed as "relay device connection destination information indicating the connection destination to the relay device 30 on the communication network 2" or "relay device destination information indicating the destination to the relay device 30 on the communication network 2". Similarly, the "terminal identification information for identifying the transmission terminal 10 on the communication network 2" may be expressed as "terminal connection destination information indicating the connection destination to the transmission terminal 10 on the communication network 2" or "terminal destination information indicating the destination to the transmission terminal 10 on the communication network 2".

[0138] In the above embodiment, a video conference system has been described as an example of the transmission system 1, but the present invention is not limited thereto and may be a telephone system such as an IP (Internet Protocol) telephone or an Internet telephone. The transmission system 1 may be a car navigation system. In this case, for example, one of the transmission terminals 30 corresponds to a car navigation device mounted on a car, and the other of the transmission terminals 30 corresponds to a management terminal or a management server of a management center that manages the car navigation, or a car navigation device mounted on another car.

[0139] Furthermore, in the process of acquiring display data, instead of acquiring the entire screen, only a portion displayed in an area (window) of the screen may be acquired as described below. Fig. 26 shows an example of a screen displayed by the external input device 40aa. In this example, areas (windows) (a) and (b) are displayed on the screen. Of these, the area to be shared with the transmission terminal 10db is pointed to by manipulating the pointer shown in (c) in Fig. 27 with the mouse 212. At this time, the display data related to the pointed area (a) is acquired by the display data acquisition unit 451.

[0140] Furthermore, the external input device may be provided in advance with a display data acquisition unit 451 and a display data transmission unit 452 to be used by the external input device and stored in the storage unit 1000 of the transmission terminal 10. This makes it possible to omit the process of transmitting and receiving the display data acquisition unit 451 and the display data transmission unit 452, thereby reducing the load on the transmission terminal 10 and the external input device 40.

[0141] Furthermore, although the external input device 40 has been described as displaying data on the display device 400, the display device 400 may be a device separate from the external input device 40, or may be provided in the external input device 40.

[0142] In addition, in the process in step S71 described above, the installation determination unit 43 may determine whether or not the display data acquisition unit 451 and the display data transmission unit 452 are already installed. In this case, if it is determined in step S71 that the display data transmission unit 452 is not installed, the program acquisition unit 44 acquires the display data acquisition unit 451 and the data transmission unit 452 stored in the transmission terminal 10aa and installs them (step S73).

[0143] Moreover, the external input device 40aa may be provided with the display data acquisition unit 451 in advance, thereby eliminating the need for processing by the installation determination unit 43. The display data acquired by the display data acquisition unit 451 may be written to the storage unit 1000 of the mounted transmission terminal 10aa, thereby eliminating the need for the display data transmission unit 452. [Explanation of symbols]

[0144] 1. Transmission System 10 Transmission Terminal 11 Transmitting / receiving unit (an example of a display data transmitting means) 12 Operation input reception section 13 Login request section 14a Imaging section 14b Image display control unit 15a Audio input section 15b Audio output section 16 Selection processing section 16a Measurement section 16b Calculation part 16c Selection section 17 Delay detection unit 18 External information transmission / reception unit (an example of a display data receiving means) 19 Storage and readout processing section 20 Detection unit 30 Relay Device 31 Transmitter / receiver 32 Status detection unit 33 Data Quality Check Department 34 Change Quality Control Department 35 Data Quality Change Division 40 External input device 41 Transmitter / receiver 42 Connection detection unit 43 Installation Judgment Department 44 Program Acquisition Department 451 Display data acquisition unit 452 Display data transmission unit 46 Operation input reception section 47 Display control section 48 Mounting section 49 Memory / readout processing section 50 Transmission Management System 51 Transmitter / receiver 52 Terminal authentication unit 53 Status Management Unit 54 Terminal Extraction Unit 55 Terminal status acquisition unit 56 Narrowing section 56a Selection session ID generator 56b Terminal IP address extraction unit 56c Selection section 56d Priority determination section 57 Session Management Unit 58 Quality Decision Department 60 Delay Time Management Unit 100 Maintenance System 400 display device 1000 Storage unit (an example of a destination information storage means) 3000 storage section 3001 Change Quality Control DB 4000 storage section 4001 Material management DB 5000 storage section 5001 Relay device management DB 5002 Terminal authentication management DB 5003 Terminal management DB 5004 Destination list management database 5005 Session Management DB 5006 Priority management DB 5007 Quality control DB [Prior art documents] [Patent documents]

[0145] [Patent Document 1] US6,760,749B1

Claims

1. A transmission terminal for transmitting and receiving image data, a login request unit that transmits a login request based on the identification information to the communication management system; a transmission / reception unit that, after being authenticated by the communication management system based on the login request, starts communication related to communication using the image data with another transmission terminal logged in to the communication management system, and transmits and receives the image data using the established session; a display data receiving unit that receives display data displayed on a display screen of an external input device when the external input device is connected to the other transmission terminal while the communication related to the communication is being performed with the other transmission terminal; having After the permission is granted, the transmission / reception unit transmits the display data using the session, thereby enabling a display screen of the external input device related to the display data to be shared with the other transmission terminal. A transmission terminal comprising:

2. The display control unit further displays the display data displayed on the display screen of the external input device on a display unit. The transmission terminal according to claim 1.

3. The display control unit causes the image data received from the other transmission terminal to be displayed on the display unit. The transmission terminal according to claim 2.

4. The permission is permission to execute a process to acquire the display data displayed on a display screen of the external input device. A transmission terminal according to any one of claims 1 to 3.

5. The transceiver unit receives the request for permission, the display data receiving unit receives the display data if the request for permission is granted. A transmission terminal according to any one of claims 1 to 4.

6. The external input device is an external input device that is not logged in to the communication management system. A transmission terminal according to any one of claims 1 to 5.

7. A transmission method executed in a transmission terminal that transmits and receives image data, comprising: a login request step of transmitting a login request based on the identification information to the communication management system; a transmitting / receiving step of starting communication related to communication using the image data with another transmission terminal logged in to the communication management system after being authenticated by the communication management system based on the login request, and transmitting / receiving the image data using the established session; a display data receiving step of receiving display data displayed on a display screen of an external input device when the external input device is connected to the other transmission terminal while the communication related to the communication is being performed with the other transmission terminal; Including, the transmitting and receiving step transmits the display data using the session after the permission is granted, thereby enabling a display screen of the external input device related to the display data to be shared with the other transmission terminal. A transmission method comprising:

8. A transmission system comprising a transmission terminal, another transmission terminal, and a communication management system, The transmission terminal includes: a login request unit that transmits a login request based on the identification information to the communication management system; a transmission / reception unit that, after being authenticated by the communication management system based on the login request, starts communication related to communication using image data with another transmission terminal logged in to the communication management system, and transmits and receives the image data using the established session; a display data receiving unit that receives display data displayed on a display screen of an external input device when the external input device is connected to the other transmission terminal while the communication related to the communication is being performed with the other transmission terminal; having After the permission is granted, the transmission / reception unit transmits the display data using the session, thereby enabling a display screen of the external input device related to the display data to be shared with the other transmission terminal. A transmission system comprising:

9. A computer comprising: a login request means for transmitting a login request based on the identification information to the communication management system; a transmitting / receiving means for starting communication related to communication using image data with another transmission terminal logged in to the communication management system after being authenticated by the communication management system based on the login request, and transmitting / receiving the image data using the established session; a display data receiving means for receiving display data displayed on a display screen of an external input device when the external input device is connected to the other transmission terminal while the communication related to the communication is being performed with the other transmission terminal; The function of the device is as follows: the transmitting / receiving means transmits the display data using the session after the permission is granted, thereby enabling a display screen of the external input device related to the display data to be shared with the other transmission terminal. A program characterized by:

10. The display control means further comprises a display control means for displaying the display data displayed on the display screen of the external input device on a display unit. The program according to claim 9.

11. The display control means causes the image data received from the other transmission terminal to be displayed on the display unit. The program according to claim 10.

12. The permission is permission to execute a process to acquire the display data displayed on a display screen of the external input device. The program according to any one of claims 9 to 11.

13. The transmitting / receiving means receives the request for permission, the display data receiving means receives the display data when the request for permission is granted.

13. A program according to any one of claims 9 to 12.

14. The external input device is an external input device that is not logged in to the communication management system.

14. The program according to any one of claims 9 to 13.

Images