Relay device

Abstract

To enable communication to be easily performed by using an arranged communication terminal, appropriately grasping the location of each employee and connecting a communication line, for example, in a company.SOLUTION: A video or like by terminal recording part 132 records video record data obtained by associating terminal identification information from each of a plurality of PCs with video data photographed through a camera part in a video or the like by terminal storage part 113. A first video or the like provision part 133 provides a communication terminal of a provision request source with the video record data of each PC of the video or the like by terminal storage part 113 in response to a provision request from one PC among the plurality of PCs. A first connection processing part 134 connects a communication line between a PC of a connection request source and a PC specified by the terminal identification information included in a connection request in response to the connection request including the terminal identification information from the PC of a request source to enable telephone conversation by video and voices.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present invention relates to a relay device, such as a UTM (Unified Threat Management) device, that relays communications between different networks or within a predetermined network. [Background technology]

[0002] Web conferencing, in which participants in different locations hold a conference via the Internet using communication terminals such as personal computers (PCs) and tablet PCs, is widely used. Web conferencing is a system in which audio from a microphone built into or connected to a communication terminal and video from a camera are transmitted to a conference server via the Internet, and participants share this information using their own communication terminals, thereby holding a real-time conference. As disclosed in, for example, Patent Document 1 described below, improvements have been made to web conferencing to make it easier for a variety of users to use.

[0003] When holding a web conference, the person convening the conference sends a meeting notification specifying the date and time in advance to the participants by email, and the conference body is formed on that date and time, and the conference can be held. During the web conference, a so-called conference app can be used to view a list of the members and participants who have been invited, and members who have not yet joined (prospective participants) can be individually called from within the web conference. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-282596 Summary of the Invention [Problem to be solved by the invention]

[0005] Web conferences, such as those described above, are extremely effective for meetings traditionally held in conference rooms, i.e., when multiple participants gather at a set date and time to discuss something. Web conferences make it easy to hold a meeting because participants can join the discussion (web conference) from their own locations using their own communication devices. However, web conferences require mutual authentication between the organizer and participants to prevent the contents of the meeting from being leaked to the outside world, and discussions cannot take place unless both parties intend to discuss the matter.

[0006] In other words, to participate in a web conference, you need to be invited by the organizer and be authenticated by the organizer, so hosting a web conference and participating in a web conference are time-consuming. For this reason, web conferences are not suitable for casual interactions (conversations) such as confirming something with a person in charge, sharing schedules, or asking for advice.

[0007] For example, it may happen that you want to confirm something with a person in charge and go to their desk, only to find that they are working from home and not there that day. In this case, you have to call the person in question, but making a phone call is a hassle. It is also common for the person you urgently need to confirm something with to be in the company but not at their desk. For example, if the person in question belongs to the development department and commutes back and forth between their desk and the laboratory, there is a need to accurately track their location and contact them quickly and reliably.

[0008] In view of the above, the object of this invention is to enable a group of multiple members (employees), such as a company, to utilize communication terminals located within the group to properly grasp the location of each member and connect communication lines to easily communicate with each other. [Means for solving the problem]

[0009] In order to solve the above problem, the relay device of the invention described in claim 1 comprises: A relay device to which a plurality of communication terminals are connected and which relays communications between the communication terminals, each of the plurality of communication terminals is provided with a camera unit, a display unit, a microphone, and a speaker, and is capable of communication using video and audio; a terminal-specific video etc. recording means for recording video recording data, which associates terminal identification information from each of the plurality of communication terminals with video data captured by the camera unit, in a terminal-specific video etc. storage unit; a first providing means for providing the video recording data for each of the communication terminals in the terminal-specific video etc. storage unit to the communication terminal that has made the provision request in response to a provision request for the video recording data from one of the communication terminals; a first connection processing means for connecting a communication line between the communication terminal that is a source of a provision request and a communication terminal identified by the terminal identification information included in the connection request in response to a connection request including the terminal identification information from the communication terminal that is a source of a provision request and to which the video recording data is provided from the first providing means; Equipped with The first connection processing means is characterized in that it enables communication using video and audio between communication terminals connected to a communication line.

[0010] According to the relay device of claim 1, a communication terminal connected to the relay device is equipped with a camera unit, a display unit, a microphone, and a speaker, and is capable of video and audio communication. A terminal-specific video etc. recording means records video record data from each of the multiple communication terminals, in which terminal identification information and video data captured by the camera unit are associated, in a terminal-specific video etc. storage unit. A first providing means provides the video record data for each communication terminal in the terminal-specific video etc. storage unit to the communication terminal that made the provision request in response to a provision request from one of the multiple communication terminals.

[0011] In response to a connection request including terminal identification information from the communication terminal that has made the provision request, the first connection processing means connects a communication line between the communication terminal that has made the provision request and the communication terminal identified by the terminal identification information included in the connection request. As a result, between the communication terminals connected by the communication line, video data captured by the camera unit and audio data collected by the microphone are transmitted to the other party. Therefore, between the communication terminals connected by the communication line, video based on the transmitted video data is displayed on the display unit, and audio based on the transmitted audio data is emitted from the speaker, enabling a call using video and audio. [Effects of the Invention]

[0012] According to this invention, in a group consisting of multiple members (employees) of a company or the like, the location of each member can be properly grasped using the communication terminals provided, and communication lines can be connected to easily communicate with each other. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram for explaining an example of the configuration of a communication system configured using a UTM device to which an embodiment of a relay device according to the present invention is applied; [Figure 2] 1 is a diagram for explaining the arrangement of a PC (Personal Computer) which is a communication terminal connected to a UTM device according to an embodiment. [Figure 3] FIG. 10 is a diagram illustrating a case where a call is made using a PC at home while working from home and a PC at one's desk at work. [Figure 4] FIG. 10 is a diagram for explaining a case where a call is made using a PC at home while telecommuting and a PC at a location where the intended contact is located at work. [Figure 5] FIG. 10 is a diagram for explaining a case where a communication line is connected between PCs and a call is being made, and a user joins the call. [Figure 6] FIG. 10 is a diagram for explaining a process for recognizing a person using a PC. [Figure 7] FIG. 2 is a block diagram illustrating a configuration example of a PC according to an embodiment. [Figure 8] FIG. 1 is a block diagram illustrating a configuration example of a UTM device according to an embodiment. [Figure 9] 3 is a diagram illustrating an example of data stored in an employee information storage unit of a UTM device according to an embodiment. FIG. [Figure 10] 3 is a diagram illustrating an example of data stored in a terminal information storage unit of a UTM device according to an embodiment; [Figure 11] 3 is a diagram for explaining an example of data stored in a terminal-specific video etc. storage unit of a UTM device according to an embodiment; FIG. [Figure 12] 10A and 10B are diagrams illustrating an example of data stored in a face recognition data storage unit of a UTM device according to an embodiment. [Figure 13] 10 is a diagram illustrating an example of data stored in a clothing recognition data storage unit of a UTM device according to an embodiment. FIG. [Figure 14] 3 is a diagram illustrating an example of data stored in a voice recognition data storage unit of a UTM device according to an embodiment. FIG. [Figure 15] 3 is a diagram illustrating an example of data stored in a location history storage unit of a UTM device according to an embodiment; FIG. [Figure 16] FIG. 10 is a sequence diagram illustrating the process of making a call using a PC at home while working from home and a PC at one's desk at work. [Figure 17] FIG. 10 is a diagram for explaining the process of making a call using a PC at home while telecommuting and a PC at a location where the intended contact is located at work. [Figure 18] This is a sequence diagram continuing from FIG. DETAILED DESCRIPTION OF THE INVENTION

[0014] An embodiment of a relay device according to the present invention will be described below with reference to the drawings. In the embodiment described below, an example will be described in which the relay device according to the present invention is applied to a UTM device.

[0015] [Communication system configuration example] FIG. 1 is a diagram illustrating an example of the configuration of a communication system that uses a UTM device 1 to which an embodiment of a relay device according to the present invention is applied. In FIG. 1, a wide area network 8 shown in the center includes an IP (Internet Protocol) network 81 and an external telephone network 82. The IP network 81 refers to a computer network interconnected using Internet Protocol Suite technology, and is equivalent to the so-called "Internet," and is also called a WAN (Wide Area Network). The external telephone network 82 includes a public switched telephone network, a mobile phone network, etc., and primarily provides voice communication services.

[0016] A LAN system is configured by connecting a plurality of PCs 3(1), 3(2), 3(3), ... to a UTM device 1 connected to an IP network 81 via a LAN (Local Area Network) 2. In this embodiment, the LAN is formed within a specific company. As is well known, each of the PCs 3(1), 3(2), 3(3), ... is an information processing device, and as will be described later, also functions as a communication terminal.

[0017] The UTM device 1 integrates multiple different security functions into a single piece of hardware, enabling centralized security management of networks, etc. In other words, the UTM device 1 efficiently and comprehensively protects LAN systems formed within a company from threats such as computer viruses and hacking. As shown in Fig. 1, the UTM device 1 of this embodiment also functions as a relay device that monitors, controls, and manages communications between an IP network (WAN) 81 and a LAN 2, and communications between terminals (PCs) connected to the LAN 2.

[0018] Each of PCs 3(1), 3(2), 3(3), ... can access web pages stored on a web server provided on IP network 81 via UTM device 1, and can send / receive e-mails and chats via IP network 81. Furthermore, each of PCs 3(1), 3(2), 3(3), ... is equipped with a camera unit, a display unit, a microphone, and a speaker, as will be described in detail later, and can also make calls using video and audio (audio-video communication) with a target party via UTM device 1. The party to the call can be a communication terminal such as PC 10(1) (described later) connected to IP network 81, or a communication terminal such as PCs 3(1), 3(2), 3(3), ... other than the PC itself connected to LAN 2.

[0019] A telephone control device 4, which is connected to an external telephone network 82 and an IP network 81, is connected to a plurality of internal telephone terminals (hereinafter referred to as telephone terminals) 6(1), 6(2), 6(3), ... via an internal telephone network 5, thereby constituting a business phone system. The telephone control device 4 controls connections between internal and external lines and connections within internal lines. In other words, the telephone control device 4 is connected to a plurality of telephone terminals 6(1), 6(2), 6(3), ..., and performs so-called call control, such as connecting, disconnecting, and transferring communications between internal and external lines and within internal lines. The telephone control device 4 is also connected to the IP network 81 because it enables the use of IP telephone services using VoIP technology through the IP network 81.

[0020] As shown in FIG. 1, the UTM device 1 and the telephone control device 4 are connected via a LAN 2 and can transmit and receive data between them. In this embodiment, when a voice call is made through telephone terminals 6(1), 6(2), 6(3), etc., the telephone control device 4 generates voice recognition data that associates an employee number (user identification information) with voice data. The employee number is information associated with each pre-registered telephone terminal 6(1), 6(2), 6(3), etc. (each extension number), and can also be identified from information such as the caller's name recognized from the call voice. The associated voice data is voice data collected through telephone terminals 6(1), 6(2), 6(3), etc. The voice recognition data generated by the telephone control device 4 serves as information for identifying the employees who use telephone terminals 6(1), 6(2), 6(3), etc. and PCs 3(1), 3(2), 3(3), etc. The voice recognition data formed by the telephone control device 4 is provided to the UTM device 1 via the LAN 2 and is used in the UTM device 1.

[0021] Each of PCs 3(1), 3(2), 3(3), ... and telephone terminals 6(1), 6(2), 6(3), ... is installed on a desk in a company office. It is predetermined which employee will use which PC and which telephone terminal, such as PC 3(1) and telephone terminal 6(1) being used by person X, and PC 3(2) and telephone terminal PC6(2) being used by person A. Therefore, each of telephone terminals 6(1), 6(2), 6(3), ... is assigned an extension number, and the extension number is associated with the user (employee number or name) and managed by telephone control device 4. As described above, when a call is made using each of telephone terminals 6(1), 6(2), 6(3), ..., the employee's employee number can be identified and voice recognition data can be generated.

[0022] As shown in FIG. 1, the wide area network 8 is connected to an outside telephone terminal 9(1) and a PC 10(1), and also to a smartphone 11(1) via a mobile phone network base station BS(1). In FIG. 1, only one outside telephone terminal 9(1), one PC 10(1), one smartphone 11(1), and one base station BS(1) are shown. However, in reality, many outside telephone terminals are connected, such as outside telephone terminals 9(2), 9(3), ..., PCs 10(2), 10(3), ..., and smartphones 11(2), 11(3), .... Of course, there are also many base stations, such as base station (1) and base station (2). The outside telephone terminals 9(1), 9(2), 9(3), ... may be connected to the wide area network 8 like fixed telephone terminals installed in homes, or may be connected via other telephone control devices and used to form a business phone system.

[0023] External telephone terminals 9(1), 9(2), 9(3), ... and smartphones 11(1), 11(2), 11(3), ... are capable of making calls to and from telephone terminals 6(1), 6(2), 6(3), ... connected to telephone control device 4. In this embodiment, PCs 10(1), PCs 10(2), 10(3), ... are used at home by users (employees) of PCs 3(1), 3(2), 3(3), ... when they work from home.

[0024] In the communication system of this embodiment configured as described above, conversations (calls) can be easily conducted using PCs, without the hassle of conventional web conferences, where users must first obtain mutual approval using PCs, then connect a communication line and start a discussion. In this case, calls are made during business hours, and even if the call recipient is not at their desk, their current location can be identified and the call can be started. In this embodiment, the UTM device 1 establishing a communication line between PCs and conducting a conversation using video and audio is referred to as "conducting a call."

[0025] [Example of PC placement] FIG. 2 is a diagram for explaining the arrangement of PCs 3(1), 3(2), 3(3), ... and PC 10(1), which are communication terminals connected to the UTM device 1 of the embodiment. FIG. 2(A) shows PC 10(1) installed at the home (Shinjuku Ward, Tokyo) of Mr. A, an employee of a company that uses the communication system shown in FIG. 1. That is, in this embodiment, PC 10(1) is a communication terminal used at home by Mr. A, who is telecommuting. FIG. 2(B) also shows the arrangement of PCs in a development department area Ar1 and a laboratory area Ar2, which are used by development department staff in this example, at a company (Yokohama City, Kanagawa Prefecture) that uses the communication system shown in FIG. 1.

[0026] As shown in Figure 2(B), the development department area Ar1 is located on the third floor of the company building, and the laboratory area Ar2 is located on the second floor of the company building. In the development department area Ar1, desks dk(1) to dk(6) used by development department employees (department staff) are located, and PCs 3(1) to 3(6) are installed on desks dk(1) to dk(6). In this example, desk dk(2) and PC 3(2) located in the development department area Ar1 are the desk and PC used by Mr. A at work.

[0027] Furthermore, the development department employees may frequently move back and forth between their desks in the development department area Ar1 and the laboratory area Ar2 to conduct experiments using equipment such as simulators installed in the laboratory area Ar2. As shown in Figure 2(B), PC3(11) to PC3(14) used by the development department employees are also installed in the laboratory area Ar2.

[0028] As shown in Figure 2, when PC3(1), 3(2), ... and PC10(1) are installed, it is possible to quickly initiate calls via the PC. Specifically, it is possible to quickly and easily make calls via the PC such as (a) a call from person B in the office to person A working from home, (b) a call from person A working from home to person C in the office, and (c) an interruption to a call between person A and person C.

[0029] In FIG. 2(A), the area indicated by the dotted line drawn from PC 10(1) represents the area that can be photographed by the camera unit installed in PC 10(1). Similarly, in FIG. 2(B), the area indicated by the dotted line drawn from PC 3(2) and PC 3(11) represents the area that can be photographed by the camera unit installed in PC 3(2) and PC 3(11). The photographed area is the area in which image recognition of people appearing in the video captured by the camera unit is possible. In this way, image recognition is performed using the video data captured by the camera unit 311 installed in PC 3. This allows the location of the target development department employee to be determined, enabling a call with the target development department employee using PC 3 via PC 10 to be initiated quickly and reliably.

[0030] [Calls made via PC] Below, we will explain in detail the cases mentioned above: (a) contacting person B who is in the office with person A who is working from home, (b) contacting person C who is working from home with person A who is in the office, and (c) interrupting a call between person A and person C.

[0031] <(a) A message from B, who is at the office, to A, who is working from home> 3 is a diagram for explaining communication between a PC at home and a PC at one's desk at work while telecommuting. PCs 10(1), 10(2), ... used at home while telecommuting and PCs 3(1), 3(2), ... used at work are equipped with a display 307, speaker 309, camera unit 311, and microphone 313, as will be described in detail later. Speaker 309 in PC 10(1), PC 3(2), etc. is provided in an appropriate location, such as on the side or front of PC 10(1), PC 3(2), etc., and the speaker is not shown in FIGS. 3 to 5.

[0032] As shown in Figure 3(A), person A, who works from home, turns on PC10(1) at home, accesses UTM device 1 at work, and after authentication is obtained, begins the process of transmitting video data captured by camera unit 311 to UTM device 1. UTM device 1 records the video data from PC10(1) in buffer memory and sequentially transmits this data to PC3(2) on desk dk(2) that person A uses at work. As a result, as shown in Figure 3(A), video of person A working from home using PC10(1) at home is displayed on display 307 of PC3(2) on desk dk(2) that person A uses at work.

[0033] Normally, PCs 3(1) to 3(6) installed on desks dk(1) to dk(6) in the office are turned on when the employee to whom they are assigned arrives at work. However, PC 3(n) on desk dk(n) of an employee working from home is turned on by another employee who arrives at the office. Therefore, if the image of person A is displayed on display 307 of PC 3(2) on desk dk(2) in the office, it will be clear that person A is working from home.

[0034] In the situation shown in Figure 3(A), for example, suppose that Person B of the Administration Department wants to check with Person A of the Development Department and goes to Person A's desk dk(2), but Person A is working from home and not at desk dk(2). In this case, Person B can tell that Person A is working from home because Person A is displayed on display 307 of PC3(2) on Person A's desk dk(2). Therefore, as shown in Figure 3(B), Person B addresses Person A by name, such as "Person A," while facing PC3(2) where Person A's image is displayed on display 307.

[0035] The voice of person B calling out to "person A" is collected by microphone 313 of PC3(2), and if it is determined by voice recognition, for example, that the voice is addressed to "person A," it is transmitted from PC3(2) to PC10(1) at person A's home via UTM device 1. The voice of person B calling from PC2(3) is emitted from speaker 309 of PC10(1) of person A, who is working from home, and when person A responds, a communication line is established between PC10(1) and PC3(2) via UTM device 1. This allows person A to talk using PC10(1) and person B using PC3(2) without either person A or B having to perform any complicated operations.

[0036] When the call ends, PC10(1) no longer picks up the audio of person A, and PC3(2) no longer picks up the audio of person B. When the UTM device 1 detects that this state has continued for a certain period of time or longer, both video and audio communication between PC10(1) and PC3(2) ends. In this case, the UTM device 1 returns to a unidirectional communication state (the state in Figure 3(A)) in which video data of the video captured by the camera unit 311 of PC10(1) is sent to PC3(2).

[0037] <(a) A, who is working from home, contacts C, who is in the office> 4 is a diagram illustrating a case where communication is performed using a PC at home while telecommuting and a PC at a location where the intended contact is located at work. As explained using FIG. 3, video data of video of person A taken by camera unit 311 of PC10(1) used by person A while telecommuting is transmitted to UTM device 1, buffered, and provided to PC3(2) used by person A at work. As a result, the video of person A is displayed on display 307 of PC3(2).

[0038] When PC3(1) to PC3(6) located in the company's development department area Ar1 and PC3(11) to PC3(14) located in the laboratory area Ar2 are powered on, they begin the process of providing video data of images captured by camera unit 311 to UTM device 1. As a result, UTM device 1 buffers each of the video data of images captured by camera unit 311 of the PCs connected to it via LAN2, and becomes able to provide each of these to other PCs.

[0039] In this case, suppose that Person A, who is working from home, has a consultation with Person C, who is in the development department of the same department. Person A uses PC10(1) at home to request the UTM device 1 to provide images captured by the camera units 311 of PCs 3(1) to 3(6) in the development department area Ar1 and PCs 3(11) to 3(14) in the laboratory area Ar2. Based on the buffered image data from PCs 3(1) to 3(6) and PCs 3(11) to 3(14), the UTM device 1 creates a display list of images using the image data from the camera units 311 of each PC, and provides this to PC 10(1), which is the requestor.

[0040] The display list, for example, as shown on display 307 of PC10(1) in Fig. 4, displays multiple small images based on video data from PCs connected to the UTM device 1 on a single screen. Fig. 4 shows a case in which four images based on video data from PC3(11) to PC3(14) located in the laboratory area Ar2 are displayed on display 307 of PC10(1). Of course, by scrolling the screen, it is also possible to display images based on video data from PC3(1) to PC3(6) located in the development department area Ar1.

[0041] Person A refers to the display list displayed on display 307 of PC10(1) that he uses to find out which PC's video includes the person C he is interested in. As shown on display 307 of PC10(1) in Figure 4, in this example, it can be confirmed that the person C is included in the video based on the video data from PC3(13). Note that multiple videos are displayed on display 307 of PC10(1), and it may be difficult to identify who is in the video. In this case, by performing a predetermined operation, such as clicking once on the desired small video portion, the desired small video can be displayed in full screen on display 307. In this case, it becomes possible to see the details of the desired video, making it easier to identify the person in the video.

[0042] In the example shown in Figure 4, suppose that user A of PC10(1) has confirmed from the display list on the display of PC10(1) that user C is captured in the camera image of PC3(13) located in laboratory area Ar2. By performing a predetermined operation such as double-clicking on the portion of the camera image of PC3(13) displayed on display 307 of PC10(1), user A requests UTM device 1 to establish a communication line with PC3(13), the source of the camera image.

[0043] As a result, a communication line is connected via the UTM device 1 between PC10(1) used by A and PC3(11) where C is being captured by the camera unit 311, enabling them to communicate with each other. To this end, A calls out "C-san" into the microphone 313 of PC10(1). As a result, a voice calling out "C-san" is emitted from the speaker 309 of PC3(13), making C aware that A is requesting the execution of a call, and the call is then made via PC10(1) and PC3(13).

[0044] As mentioned above, since Mr. A is working from home, the camera image from PC10(1) is displayed on the display 307 of PC3(2), which Mr. A uses when he is at work. However, when a communication line is connected between Mr. A's PC10(1) and PC3(13) and a call becomes possible, the UTM device 1 stops the supply of the camera image from PC10(1) to PC3(2). This prevents Mr. A, who is currently talking to Mr. C, from being spoken to through PC3(2). In this example, when the call between Mr. A using PC10(1) and Mr. C using PC3(13) ends, the UTM device 1 resumes the supply of the camera image from PC10(1) to PC3(2).

[0045] In this way, when telecommuting, person A can use his / her PC 10(1) to identify a PC equipped with a camera capturing a target person (employee) from a display list of camera images from each PC connected to the UTM device 1. A communication line can be established between this identified PC and the PC 10(1) used by person A via the UTM device 1, allowing for a call using video and audio. While FIG. 4 illustrates an example in which person A telecommutes and calls person C, the present invention is not limited to this. A PC equipped with a camera capturing a target person can be identified between PCs connected to LAN2, and from a PC connected to LAN2 to PC 10(1) connected to the IP network 81. When establishing a communication line with this identified PC, the same process as described with reference to FIG. 4 can be used.

[0046] <(c) Interrupting a call between A and C> FIG. 5 is a diagram illustrating a case where a call is interrupted when a communication line is connected between PCs and a call is in progress. Note that interrupting does not mean interrupting the current call, but rather joining the current call later. As explained with reference to FIG. 4, assume that the UTM device 1 connects a communication line between PC 10(1) used by person A working from home and PC 3(13) near person C in a company laboratory, and that person A and person C are engaged in a call. Now, assume that person D, also working from home, needs to have a discussion with person C. Similar to the case of person A explained with reference to FIG. 4, person D uses his / her own PC 10(2) to request the UTM device 1 to provide a display list, which is a list of images captured by the camera unit 311 of the PC connected to the UTM device 1.

[0047] As explained using Figure 4, the UTM device 1 forms a display list of camera images from each PC and provides it to PC10(2) used by Mr. D. Mr. D checks the display list of camera images displayed on display 307 of PC10(2), selects the camera image from PC3(13) equipped with camera unit 311 capturing Mr. C, and requests that a communication line be established between PC3(13). In addition to the communication line already established between PC10(1) and PC3(13), the UTM device 1 establishes a communication line between PC10(2) and PC3(13), and also establishes a communication line between PC10(2) and PC10(1).

[0048] However, in this example, person A, who uses PC10(1), and person C, who is near PC3(13), have already started a call via PC10(1) and PC3(13). Therefore, the UTM device 1 forms display information X1 indicating that person A is currently on a call with person C, and provides this to PC10(2) by combining it with camera footage from PC3(13), which is located near person C and whose camera unit 311 captures person C. As a result, as shown on display 307 of PC10(2) in Figure 5, person D can recognize from the information displayed on display 307 of PC10(2) that three people, including person C and person A, are now on a call.

[0049] Similarly, the UTM device 1 forms display information X2 indicating that person D has joined the call, synthesizes this with camera footage from PC3(13), which is located near person C and whose camera unit 311 captures person C, and provides this to PC10(1). Furthermore, the UTM device 1 forms display information X3 indicating that person D has joined the call, synthesizes this with camera footage from PC10(1), whose camera unit 311 captures person A, and provides this to PC3(13).

[0050] As a result, as shown on display 307 of PC10(1) in Fig. 5, person A can recognize from the information displayed on display 307 of PC10(1) that a call is in progress with three people, including person C and person D. Also, as shown on display 307 of PC3(13) in Fig. 5, person C can recognize from the information displayed on display 307 of PC3(13) that a call is in progress with three people, including person A and person D.

[0051] That is, in the example shown in Figure 5, A and C, who were initially on the call, can clearly understand that D joined the call later. Furthermore, D, who joined the call later, can clearly understand that C, who was the first party to be connected, was already on the call with A. Note that although the case where three parties make a call has been explained here, the same processing can be applied to cases where more than three parties make a call.

[0052] In the example described with reference to FIG. 5, PC3(13) whose camera unit 311 captures C, with whom D wants to have a discussion, is identified, and when the communication line is connected, the person who was talking earlier and the person who joined the call later are notified by display. However, this is not limited to this. In order to identify the PC whose camera unit 311 captures C, with whom D wants to have a discussion, information about people who are already talking via a connected communication line may be displayed when the display list requested from the UTM device 1 is provided. In this case, an indication that the person is currently talking to A is included in the image from the camera unit 311 of PC3(13) capturing C. Similarly, an indication that the person is currently talking to C is included in the image from the camera unit 311 of PC10(1) capturing A.

[0053] In this way, it becomes possible to know where the person you want to talk to is and whether that person is currently on a call. After that, when you specify the other party's PC and connect the communication line, display information indicating the presence of people who have already been on the call and the presence of people who will join the call later can be displayed in the manner described using Figure 5. This allows both people who were already on the call and people who joined the call later to clearly know who are participating in the call.

[0054] [Knowing the location of the intended recipient in advance] In the example described using Figures 4 and 5, in order to identify a PC equipped with a camera unit 311 capturing an image of the person (employee) with whom a call is desired, a display list of images captured by the camera unit 311 of each PC was provided from the UTM device 1. The images from the camera unit 311 of each PC that make up this display list were checked, and the location of the target employee (person) was identified. As described using Figure 2, if the locations where development department employees work are limited to two locations, the development department area Ar1 and the laboratory area Ar2, the location of the target employee can be easily identified by using the display list.

[0055] However, when the locations where development department employees work span a wider area, using a display list to check their location can be time-consuming and laborious. Therefore, the UTM device 1 of this embodiment determines in advance which PC's camera unit 311 each employee is captured on, and manages the information in association with each PC. This allows users of the UTM device 1 to more quickly determine the location of the desired employee.

[0056] FIG. 6 is a diagram for explaining the process of recognizing a person using a PC. Here, we will specifically explain the process of recognizing a person using PC3(11) and PC3(12) installed opposite each other in the laboratory area Ar2 shown in FIG. 2. In FIG. 6, the triangular area indicated by the dotted line drawn out from PC3(11) is the imaging area in which the camera unit 311 mounted on PC3(11) can capture images sufficient to recognize a person. Similarly, in FIG. 6, the triangular area indicated by the dotted line drawn out from PC3(12) is the imaging area in which the camera unit 311 mounted on PC3(12) can capture images sufficient to recognize a person.

[0057] In Figure 6, squares labeled E to K represent people present in each shooting area, and the arrows attached to each square labeled with the letter indicate the direction each person's face is facing. Therefore, E, F, and G are each located within the shooting area of ​​the camera unit 311 of PC3(11) and are facing toward the camera unit 311 of PC3(11). H is located within the shooting area of ​​the camera unit 311 of PC3(12) and is facing toward the camera unit 311 of PC3(12). In contrast, I is located within the shooting area of ​​the camera unit 311 of PC3(11), but is facing away from the camera unit 311 of PC3(11). J and K are located within the shooting area of ​​the camera unit 311 of PC3(12), but are facing away from the camera unit 311 of PC3(12).

[0058] In this embodiment, the UTM device 1, which receives and aggregates video data captured by the camera unit 311 of each PC, is provided with facial recognition data that associates each employee's facial image data with an employee number in order to recognize each employee. The facial image data is, for example, image data of a photo taken when the employee joined the company or a photo attached to a resume, and the employee number is unique identification information assigned to each employee when they joined the company. The UTM device 1 performs image recognition using the video data from the camera unit 311 of PC3(11) and pre-prepared facial recognition data. This makes it possible to identify that E, F, and G are located in the area captured by the camera unit 311 of PC3(11). Similarly, the UTM device 1 performs image recognition using the video data from the camera unit 311 of PC3(12) and pre-prepared facial recognition data. This makes it possible to identify that H is located in the area captured by the camera unit 311 of PC3(12).

[0059] However, because Mr. I has his back to the camera unit 311 of PC3(11), facial recognition cannot identify that Mr. I is in the shooting area of ​​PC3(11)'s camera unit 311. Similarly, because Mr. J and Mr. K have their backs to the camera unit 311 of PC3(12), facial recognition cannot identify that Mr. J and Mr. K are in the shooting area of ​​PC3(12)'s camera unit 311. Therefore, other information for identifying individuals is used to identify people who could not be identified by facial recognition.

[0060] In this embodiment, clothing recognition data, voice recognition data, and location history data are used as information other than facial recognition data to identify individuals who could not be identified by facial recognition. Clothing recognition data is data generated, for example, when an employee uses an employee ID card (IC card) to register attendance, and is generated by associating the employee number read from the employee ID card with a card reader and a full-body image of the employee taken at that time. For this reason, although not shown in FIG. 1, for example, a card reader and a camera unit are installed at the company's employee entrance. The employee number read from the employee ID card by the card reader and the image data of the employee image taken by the camera unit are associated with each other, and are provided to the UTM device 1 for storage.

[0061] In addition, in the communication system shown in FIG. 1, when a call is made or received using telephone terminals 6(1), 6(2), 6(3), ..., voice recognition data is formed by the telephone control device 4 and provided to the UTM device 1. Each of telephone terminals 6(1), 6(2), 6(3), ... is a so-called internal telephone terminal, and the employee who mainly uses it is determined. The identification information of telephone terminals 6(1), 6(2), 6(3), ... is associated with an employee number and managed by the telephone control device 4. Furthermore, if voice signals from telephone terminals 6(1), 6(2), 6(3), ... are recognized, the employee may mention their own name or affiliation, which can be used to identify the employee. Therefore, when a call is made using telephone terminals 6(1), 6(2), 6(3), ..., the telephone control device 4 forms voice recognition data (e.g., voiceprint data) from the employee number of the employee making the call and the call voice, and then associates these to form voice recognition data. The voice recognition data formed by the telephone control device 4 is provided to the UTM device 1 via the LAN 2 and stored therein.

[0062] Location history data is time-series data indicating when and where each employee was, using the facial recognition data, clothing recognition data, and voice recognition data described above. It is generated by the UTM device 1. Consider the case of Mr. C as a specific example of location history data. Assume that the employee's location is tracked every 10 minutes from 9:00 AM, when work starts, to 6:00 PM, when work ends. In this case, the location history data indicates Mr. C's location on February 8, 2024, such as in front of PC3(3) at his desk from 9:00 AM to 9:50 AM, in front of PC3(13) in the laboratory from 10:00 AM to 11:50 AM, and in front of PC3(13) in the laboratory from 1:00 PM to 3:30 PM. The location history data is also accumulated by date, allowing information to be obtained for each employee, such as where they are typically located between 10:00 AM and 11:00 AM on Tuesdays.

[0063] As a result, even if an employee cannot be identified by facial recognition data, the clothing recognition data can be compared with the image captured by the PC's camera unit 311 to identify the employee based on the shape, pattern, hairstyle, etc. of the clothing. In this case, since the clothing recognition data includes a full-body image of each employee, the height and weight of each employee can be estimated, and the height and weight of the employee captured by the camera unit 311 can be compared and used to identify the employee. In addition, the voice recognition data can be compared with the voice collected by the PC's microphone 313 to identify the employee.

[0064] Also, suppose that even using face recognition data, clothing recognition data, and voice recognition data, it is not possible to identify the employee present in the area photographed by the camera unit 311 of PC3(11). In this case, it is possible to use location history data to estimate that the employee who is most likely to be present in the area photographed by the camera unit 311 of PC3(11) on a given day and at a given time period is the employee currently present in the area photographed by the camera unit 311 of PC3(11). In this way, it is possible to identify an employee based on the pattern of the employee's clothing captured by the PC's camera unit 311 and the employee's voice collected by the PC's microphone 313, or based on the employee's height and pattern of the clothing captured by the PC's camera unit 311. It is also possible to identify an employee based on the employee's height captured by the PC's camera unit 311 and the employee's voice collected by the PC's microphone 313.

[0065] In this way, by combining the employee's face, clothing, etc. (including physical features such as height), voice (voice), and location history of the employee, it is possible to identify each employee who is present in the image capture area of ​​each PC's camera unit 311. By managing these identified employees in association with their PCs, it is possible to specify an employee and identify the PC that is present in the image capture area of ​​the camera unit 311.

[0066] [Example of configuration of major devices that make up a communication system] Next, as explained using Figures 3 to 6, we will explain an example configuration of PC3(1), 3(2), 3(3), ..., PC10(1), ..., UTM device 1, which are the main devices that enable easy communication with target employees through the communication system.

[0067] Since PCs 3(1), 3(2), 3(3),... are basically configured in the same way, they will be collectively referred to as PC3 below, except when it is necessary to show them separately. Also, since PCs 10(1), 10(2),... are basically configured in the same way, they will be collectively referred to as PC10 below, except when it is necessary to show them separately. Further, the PC3 and 10 used in this embodiment are commercially available existing PCs and have basically the same configuration. Therefore, in the following, PC3 will be taken as an example to explain its configuration, but PC10 has the same configuration.

[0068] <Configuration Example of PC3 and 10> FIG. 7 is a block diagram for explaining the configuration example of the PCs 3 and 10 of the embodiment. The connection terminal 301T constitutes a connection end to LAN2. The LAN I / F (interface) 301 realizes the connection function to LAN2. That is, the LAN I / F �01 realizes the function of converting the data addressed to the own device transmitted through LAN2 into a form processable by the own device and capturing it. Also, the LAN I / F 301 realizes the function of converting the data transmitted from the own device into a transmission form and transmitting it to LAN2.

[0069] In the case of PC10, the connection terminal 301T constitutes a connection end to the home gateway device, and the LAN I / F 301 realizes the function of converting the data addressed to the own device transmitted through the home gateway device into a form processable by the own device and capturing it. Also, in the case of PC10, the LAN I / F 301 realizes the function of converting the data transmitted from the own device into a transmission form and transmitting it to the IP network (WAN) via the gateway device.

[0070] [[ID=IS]] Although not shown, the control unit 302 is a microprocessor including a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), and non-volatile memory, and realizes the function of controlling each unit of the PC 3. The storage device 303 is a device unit including a recording medium such as an HDD (Hard Disk Drive) or SSD (Solid State Drive) and its driver, and performs operations such as recording, reading, changing, and deleting various data from the recording medium. The storage device 303 stores and holds necessary data and programs, and is also used as a working area for temporarily storing intermediate data generated in various processes.

[0071] The operation unit 304 receives instructions input from the user, converts them into electrical signals, and supplies them to the control unit 302. Specifically, the operation unit 304 includes a pointing device such as a keyboard or a mouse. The USB I / F 305 is a USB-standard interface to which various USB devices, such as a USB memory stick or a printer or scanner that can be connected via USB, can be connected.

[0072] The display controller 306 and display 307 constitute a display output interface, and the audio output processing unit 308 and speaker 309 constitute an audio output interface. The video input processing unit 310 and camera unit 311 constitute a video input interface, and the audio input processing unit 312 and microphone 313 constitute an audio input interface. These constitute a user interface, and provide information to and receive information from the user.

[0073] With such a configuration, the PCs 3 and 10 can perform various information processes by executing various application software in the control unit 302. For example, it is possible to create a document using word processing software, create a table or graph using spreadsheet software, refer to a web page provided on a server on the WAN using a browser, or send and receive e-mails using e-mail software. Further, it is possible to send and receive chat messages using chat software or perform a video and voice call using call software.

[0074] Furthermore, the PCs 3 and 10 of this embodiment are equipped with simple call software. When the PCs 3 and 10 are connected to the UTM device 1, they can execute the simple call software, start shooting with the camera unit 311, and provide the video data obtained by shooting to the UTM device 1. Also, when the simple call software functions in the PCs 3 and 10 and a communication line is connected between the PCs 3 and 10 and a target partner via the UTM device 1 in response to an instruction from the UTM device 1, the PCs 3 and 10 can perform a call using video and voice.

[0075] <Configuration example of UTM device 1> FIG. 8 is a block diagram for explaining a configuration example of the UTM device 1 of the embodiment. The connection terminal 101T constitutes a connection end to the IP network (WAN) 81. The WAN I / F (Interface) 101 is a part that performs communication processing through the IP network 81. That is, the communication I / F 101 converts a signal addressed to itself transmitted through the IP network 81 into a signal in a form that can be processed by itself and captures this signal. Also, the WAN I / F 101 converts a signal transmitted from itself into a signal in a transmission form and sends it out to the IP network 81 for transmission to a partner. Therefore, when communicating with a partner connected to the IP network 81, it is performed through the connection terminal 101T and the communication I / F 101.

[0076] The control unit 102 is a microprocessor (not shown) that includes a CPU, ROM, RAM, non-volatile memory, etc., and controls each unit of the UTM device 1. The storage device 103 is a device unit that includes a recording medium, such as an HDD (Hard Disk Drive) or SSD (Solid State Drive), and its driver, and records, reads, modifies, and deletes various data from the recording medium. In addition to storing and holding necessary data and programs, the storage device 103 is also used as a working area for temporarily storing intermediate data generated in various processes.

[0077] The connection terminal 104T constitutes the connection terminal with the LAN2. The LANI / F (Interface) 104 is a part that processes communications via the LAN2. That is, the LANI / F 104 converts signals sent to the device itself via the LAN2 into signals in a format that can be processed by the device itself and then receives these signals. The LANI / F 104 also converts signals to be sent from the device itself into signals in a format for transmission, and sends them to the LAN2 to be transmitted to the other party. Therefore, when communication is performed with PCs 3(1), 3(2), 3(3), ..., 3(n) connected to the LAN2, it is performed via the connection terminal 104T and the LANI / F 104.

[0078] The information storage unit 110 is configured with seven storage units shown in FIG. 8 that store various information used for processing. The seven storage units that make up the information storage unit 110 can be created on different HDDs or SSDs, or, if storage capacity allows, can be configured on a single HDD or SSD. Of course, the storage units can also be distributed across multiple HDDs or SSDs. By using the information in each storage unit of the information storage unit 110, it becomes possible to manage video data from each PC and provide it to the PC, or to identify the location of employees. Each storage unit will be described below.

[0079] FIG. 9 is a diagram illustrating an example of data stored in the employee information storage unit 111 of the UTM device 1. The employee information storage unit 111 stores and holds information about each employee, and as shown in FIG. 9, has storage areas for employee number, name, affiliation, password, and other information. The employee number is identification information that can uniquely identify each employee, and is assigned and registered when the employee joins the company. The name is the name (first and last name) of each employee, and includes information related to the name, such as reading and alphabetical notation. The affiliation is information indicating the department to which each employee belongs.

[0080] In Figure 9, "Development Section" is listed as an example of an affiliation, but in reality, detailed information about the affiliation, such as "XX Headquarters XX Department XX Section XX Group," is registered. The password is authentication information (authentication string) assigned to each employee and is used for authentication processing when connecting to the UTM device 1 via the IP network 81 or LAN 2. Other information that can be registered and managed includes various information about each employee, such as each employee's home address, home telephone number, mobile phone number, and extension number.

[0081] Fig. 10 is a diagram illustrating an example of data stored in the terminal information storage unit 112 of the UTM device 1 according to the embodiment. The terminal information storage unit 112 stores and holds information that uniquely identifies the PC used by each employee, and as shown in Fig. 10, has areas for storing an employee number, name, IP (Internet Protocol) address, and MAC (Media Access Control) address. The employee number is the same information as the employee number in the employee information storage unit 111, and is identification information that can uniquely identify the employee who uses the PC. The name is the name of the employee who uses the PC.

[0082] An IP address is information assigned to a PC used by an employee, and is identification information for identifying the location of that PC on the IP network 81. A MAC address is identification information that is originally assigned to each PC and is unique to that PC. If an employee can be identified by their employee number using the data stored in the terminal information storage unit 112, the PC used by that employee can be identified, and if a PC can be identified by its IP address or MAC address, the employee using that PC can also be identified.

[0083] Fig. 11 is a diagram illustrating an example of data stored in the terminal-specific video etc. storage unit 113 of the UTM device 1. The terminal-specific video etc. storage unit 113 mainly realizes the function of buffering video data obtained through the camera unit 311 of each PC, and as shown in Fig. 11, has storage areas for No., IP address, MAC address, video data, and employee identification information. No. is a sequence number assigned in order to PCs that connect to the UTM device 1. The IP address and MAC address are information provided by the PC that connects to the UTM device 1, and as described above, are identification information that can uniquely identify the PC.

[0084] The video data in the terminal-specific video etc. storage unit 113 is obtained through the camera unit 311 of each PC connected to the UTM device 1 and is provided from each PC. In other words, the video data storage area in the terminal-specific video etc. storage unit 113 corresponds to an area for buffering video data from each PC, and can store the most recently provided multiple frames of video data. In this way, the basic configuration of the terminal-specific video etc. storage unit 113 is to store and hold video recording data, which is information that associates terminal identification information with video data. The video data stored in the video data storage area in the terminal-specific video etc. storage unit 113 is used to form the above-mentioned display list, which is a list of video data from each PC, and is also sent and received between PCs connected via a communication line.

[0085] The employee identification information in the terminal-specific video etc. storage unit 113 is the employee number of an employee who is recognized as being located in the image capture area of ​​each PC's camera unit 311. That is, as explained using Fig. 6, facial recognition, clothing recognition, etc. is performed on the image data captured by the PC's camera unit 311, and voice recognition is performed on the voice data collected by the microphone 313, and the employee number of the recognized employee is stored and held. The employee identification information in the terminal-specific video etc. storage unit 113 makes it possible to identify the employee who is located in the image capture area of ​​each PC's camera unit 311 by their employee number.

[0086] FIG. 12 is a diagram illustrating an example of data stored in the facial recognition data storage unit 114 of the UTM device 1. The facial recognition data storage unit 114 stores and holds image data of still images of each employee's face to be compared with the facial portions of people included in the video data acquired through the camera unit 311 of each PC, and includes storage areas for employee numbers, names, facial recognition data, and other information. The employee number is identification information that uniquely identifies each employee, and the name is the name of the employee identified by the employee number. The facial recognition data is image data of still images of each employee's face used for facial recognition, as described above. Other information can include various necessary information such as gender, age, and date of employment. The data stored in the facial recognition data storage unit 114 is generated and stored when each employee joins the company.

[0087] FIG. 13 is a diagram illustrating an example of data stored in the clothing recognition data storage unit of the UTM device according to the embodiment. The clothing recognition data storage unit 115 stores and retains image data, such as a still image of the entire body of each employee, to be compared with the clothing of the person included in the video data acquired through the camera unit 311 of each PC. It includes storage areas for employee numbers, names, clothing recognition data, height, and other information. The employee number is identification information that uniquely identifies each employee, and the name is the name of the employee identified by the employee number. As described above, the clothing recognition data is image data, such as a still image of the entire body of each employee, used to recognize clothing. The height is the height of the employee, the subject of the image, estimated based on information such as the focal length and angle of view of the camera used to capture the full-body image of each employee. Other information may include various other necessary information, such as the date and time of the image.

[0088] As described above, the data to be stored in the clothing recognition data storage unit 115 is generated and stored each day when an employee arrives at work (when registering attendance). This is because clothing often varies from day to day. That is, when an employee arrives at work each day, they hold their employee card over a card reader to register their attendance. At this time, the employee number read from the employee card is associated with full-body image data of the employee taken at the time of attendance registration to form registration data, which is then registered in the clothing recognition data storage unit 115. The employee name required at this time can be obtained from the employee information storage unit 111 based on the employee number, and the employee's height can be estimated (calculated) using information such as the focal length and angle of view of the camera used to photograph the employee, as described above. Note that clothing recognition data is used in this example because it also includes the employee's estimated height.

[0089] FIG. 14 is a diagram illustrating an example of data stored in the voice recognition data storage unit 116 of the UTM device 1. The voice recognition data storage unit 116 stores and holds pre-collected voice data of each employee to be compared with the employee's spoken voice (voice data) obtained through the microphone 313 of each PC, and includes storage areas for employee numbers, names, voice recognition data, and other information. The employee number is identification information that uniquely identifies each employee, and the name is the name of the employee identified by the employee number. In this embodiment, as described above, the voice recognition data is employee voice data collected by the telephone control device 4 through subordinate telephone terminals 6(1), 6(2), etc. Other information can include various necessary information, such as the date the voice data was collected. The employee data shown in FIG. 14 is generated by the telephone control device 4, provided to the UTM device 1 via the LAN 2, and stored in the voice recognition data storage unit 116.

[0090] 15 is a diagram illustrating an example of data stored in the location history storage unit 117 of the UTM device 1. The location history storage unit 117 stores and holds time-series data indicating when and where each employee was located, as recognized using the face recognition data, clothing recognition data, etc., and voice recognition data described above, and has storage areas for employee number, name, recognition date and time, and terminal identification information. The employee number is identification information that can uniquely identify each employee, and the name is the name of the employee identified by the employee number. The recognition date and time indicates the date and time (year, month, day, and time) when each employee's location was recognized using the face recognition data, clothing recognition data, etc., and voice recognition data.

[0091] The terminal identification information is information that identifies a PC equipped with a camera unit 311 that captured the video data used to recognize the employee's location and a microphone 313 that collected the audio data, and is, for example, an IP address or a MAC address. For ease of explanation, in FIG. 15, the terminal identification information is substituted with the reference symbols shown in the figure. The data stored in the location history storage unit 117 is generated by the UTM device 1. By aggregating the data stored in the location history storage unit 117, it is possible to obtain information for each employee, such as where they are most likely to be located by day of the week and time period.

[0092] Next, we will explain each processing unit of the UTM device 1. Although not shown, the security function unit 120 is configured to include, for example, a P2P countermeasure unit, an HP access control unit, an anti-virus unit, an anti-mail unit, an IPS / IDS unit, and a firewall unit. In other words, the security function unit 120 realizes the function of prohibiting P2P connections with parties that have not implemented security measures or with malicious parties (function as a P2P countermeasure unit). Note that "P2P" stands for "Peer to Peer," meaning that equal parties communicate directly with each other via the Internet.

[0093] Furthermore, the security function unit 120 realizes a function of prohibiting access to homepages that fall into a desired category by selecting the desired homepage category in advance (a function as a homepage access control unit). The security function unit 120 also verifies (virus checks) responses to web pages (a function as an anti-virus unit). More specifically, the security function unit 120 realizes a function of monitoring communications when browsing web pages and verifying (checking) whether images to be viewed or files to be downloaded contain viruses. The security function unit 120 also realizes a function of blocking received e-mails that contain unwanted advertisements or viruses (a function as an e-mail anti-virus unit).

[0094] The security function unit 120 also functions to prevent inappropriate intrusions and notify users of inappropriate intrusions (functioning as an IPS / IDS unit). Here, IPS is an abbreviation for Intrusion Prevention System, and IDS is an abbreviation for Intrusion Detection System. The security function unit 120 can defend against attacks by so-called malware such as worms and Trojan horses. The security function unit 120 also determines whether to supply data to LAN2 based on the data communication status, the software used, etc., and realizes a function to protect the system from attacks and unauthorized access from external networks (functioning as a firewall unit).

[0095] Relay processing unit 130 is configured with multiple processing units as shown in Figure 8, and as explained using Figures 2 to 5, it realizes the function of connecting a communication line to a target party and enabling a call via a PC in response to extremely simple instructions. Each unit that makes up relay processing unit 130 will be explained below.

[0096] The authentication processing unit 131 performs authentication processing by requiring PC 10 connecting through the IP network 81 or PC 3 connecting through LAN 2 to enter an employee number and password, and allows only legitimate employee PCs to connect, while rejecting all other connections. The authentication processing unit 131 assigns a number to the authenticated PC 3, 10, and uses the IP address and MAC address from that PC 3, 10 to secure a storage area (Figure 11) for that PC in the terminal-specific video etc. storage unit 113. PC 3, 10 authenticated by the authentication processing unit 131 establishes a link with the UTM device 1 and begins processing to send video data of video captured through its own camera unit 311 to the UTM device 1.

[0097] The terminal-specific video recording unit 132 sequentially records video data transmitted from each of the PCs 3 and 10 connected to the UTM device 1 in the video data storage area of ​​the terminal-specific video storage unit 113 corresponding to each PC. That is, the unit 132 records video recording data, which is information associating terminal identification information with video data. As described above, the video data storage area of ​​the terminal-specific video storage unit 113 is capable of recording the latest multiple frames of video data. In response to a display list request from the PCs 3 and 10 connected to the UTM device 1 via the IP network 81 or LAN 2, the first video providing unit 133 creates a display list using the video data from each PC in the terminal-specific video storage unit 113. The first video providing unit 133 provides the created display list to the requesting PC.

[0098] The display list formed by the first video etc. providing unit 133 is a list of small videos based on video data from each PC, as shown on the display 307 of PC 10(1) in FIG. 4. That is, the display list associates the video data of the small videos for displaying a list of videos based on the video data from each PC with the terminal identification information of the providing PC. In this embodiment, the terminal identification information is an IP address or MAC address, but any information that can uniquely identify each PC will suffice. As described above, based on the display list, it is possible to identify in which PC's camera unit 311 the employee with whom the user wishes to make a call is located.

[0099] When the first connection processing unit 134 receives a connection request including terminal identification information from PC3, 10 that has received a display list from the first video etc. providing unit 133, it connects a communication line between the requesting PC3, 10 and the requested PC3, 10. This allows the user of the requesting PC3, 10 and an employee near the requested PC3, 10 to communicate with each other through the communication line. That is, as described with reference to FIG. 4, when PC10(1) that has received the display list requests a connection with PC3(13) in the laboratory area Ar2, the first connection processing unit 134 connects a communication line between PC10(1) and PC3(13). This allows Mr. A, the user of PC10(1), to quickly initiate a call with Mr. C, the desired employee near PC3(13).

[0100] Assume that there is a PC 10 that has connected to the UTM device 1 through the IP network 81. That is, this is the case when an employee has connected to the UTM device 1 using a home PC (external PC) 10 due to telecommuting. When the home PC 10 is authenticated by the authentication processing unit 131, the home terminal identification unit 135 references the data stored in the terminal information storage unit 112 based on the employee number used in the authentication process for the PC, and identifies the home PC (internal PC) 3 at work of the employee using the home PC 10. That is, as explained with reference to FIG. 3, when employee A, the user of PC 10(1), connects to the UTM device 1 using his / her home PC 10(1), the home terminal identification unit 135 performs processing to identify PC 3(2) that employee A, the user of PC 10(1), uses at his / her desk at work.

[0101] The second video etc. providing unit 136 performs processing to provide the video data, which is provided from the home PC 10 that has been authenticated and connected to the UTM device 1 and buffered in the terminal-specific video etc. storage unit 113, and the terminal identification information of the home PC 10, to the identified desk PC 3. As a result, as explained using FIG. 3(A), the video of the user of the home PC 10 is displayed on the display 307 of the desk PC 3, and other employees who come to work can recognize that the employee using the desk PC 3 is working from home.

[0102] When a call is made to the user of the home PC 10 through the user's desk PC 3, the second connection processing unit 137 connects a communication line between the requesting user's desk PC 3 and the requested home PC 10 in response to a connection request from the user's desk PC 3 that includes the terminal identification information of the home PC 10. This allows video data obtained through both camera units 311 and audio data obtained through both microphones 313 to be transmitted and received between the home PC 10 and the user's desk PC 3, enabling a call to be made.

[0103] The person recognition unit 138 uses video data from the camera unit 311 and audio data from the microphone 313 of the PC 3 connected to LAN 2 to recognize a person present in the image capture area of ​​the camera unit 311 of the PC 3. Furthermore, the person recognition unit 138 updates the information (employee number) identifying the recognized person (employee) in the employee recognition information field of the terminal-specific video and image storage unit 113. The person recognition unit 138 first performs person recognition using the video data from the camera unit 311 of the PC 3 connected to LAN 2 and the facial recognition data in the facial recognition data storage unit 114. If the person is not recognized using the facial recognition data, the person recognition unit 138 performs person recognition using the clothing and other recognition data. If the person is not recognized using the facial recognition data, the person recognition unit 138 performs person recognition using the audio data from the microphone 313 of the PC 3 connected to LAN 2 and the audio data in the audio recognition data storage unit 116. Furthermore, if necessary, the person recognition unit 138 also uses information from the location history storage unit 117 to determine whether the recognized person is a person who frequently visits the recognized location, thereby further improving the accuracy of recognition.

[0104] In response to a search request including an employee's name from PC3 connected to LAN2 or PC10 connected to IP network 81, connection destination identification unit 139 first references the data stored in employee information storage unit 111 and identifies the employee number of the target employee. Next, based on the identified employee number, connection destination identification unit 139 references the employee recognition information in the data stored in terminal-specific video etc. storage unit 113 and identifies PC3 where the target employee is located in the shooting area of ​​camera unit 311. Connection destination identification unit 139 provides video from the video data captured by camera unit 311 of the identified PC3 to PC3 or PC10 that issued the request, selects the PC whose camera unit 311 is capturing the target employee, and enables connection of a communication line between that PC and the selected PC.

[0105] The connection information display processing unit 140 processes the display information to display information indicating the other party of the call on the video image based on the video data captured by the camera unit 311 of the PC 3, 10 that is already in a call via a connected communication line. Specifically, as described with reference to Fig. 5, if the PC 3, 10 that requested the connection is already in a call, the PC 3, 10 that originated the connection request is provided with a video image including information about the other party with whom the call is already being made. Furthermore, the PC 3, 10 that is already in a call is provided with a video image including information about the PC 3, 10 that has newly requested a connection. Furthermore, in the display list described with reference to Fig. 4, information about the other party of the call can be included in the video image from the PC that is already in a call via a connected communication line.

[0106] [Summary of Processing Performed in the Communication System of the Embodiment] Next, with reference to sequence diagrams, the processing performed in the communication system of this embodiment, centered on the UTM device 1 of the embodiment described with reference to Figures 8 to 15, will be summarized. Below, the cases where communication is performed using a PC at home and a PC at one's desk at work, and the cases where communication is performed using a PC at home and a PC at a target office, will be described.

[0107] <When communicating using a PC at home while working from home and a PC at your desk at work> Fig. 16 is a sequence diagram for explaining processing centered on the UTM device 1 when communication is performed using a PC at home while telecommuting and a PC at one's desk at work. Similar to the example described above with reference to Fig. 3, Fig. 16 also shows an example in which employee A telecommutes using PC10(1) and makes a call with B via PC3(2) that A uses at work. In this case, the UTM device 1 functions as follows: authentication processing unit 131, terminal-specific video recording unit 132, desk terminal identification unit 135, second video providing unit 136, and second connection processing unit 137.

[0108] Person A turns on PC 10(1) at home and uses a specified URL to access the company's UTM device 1 and send a connection request (step S1). The authentication processing unit 131 of the UTM device 1 requests PC 10(1), the source of the connection request, to enter an employee number and password for authentication processing (step S2). In response to the request from the UTM device 1, Person A enters his employee number and password into PC 10(1) and sends them to the UTM device 1 (step S3).

[0109] In the UTM device 1, the authentication processing unit 131 functions to perform authentication processing to determine whether the transmitted employee number and password match those stored in the employee information storage unit 111 (step S4). If they match, the authentication processing unit 131 authenticates and permits the connection (step S5). However, if the transmitted employee number and password do not match those stored in the employee information storage unit 111, the authentication processing unit 131 does not perform authentication and rejects the connection.

[0110] If authentication is successful and the connection is permitted, PC10(1), which has made the connection request, activates its own camera unit 311 and begins the process of transmitting the camera image (image data) obtained by shooting to the UTM device 1 (step S6). In response to this, in the UTM device 1, the terminal-specific image recording unit 132 functions to secure a recording area for the image data etc. from PC10(1) in the terminal-specific image storage unit 113, and begins the process of recording the terminal identification information and image data from PC10(1) (step S7).

[0111] Next, in the UTM device 1, the user's desk terminal identification unit 135 functions to identify the terminal identification information, which is the IP address or MAC address, of PC3 that Mr. A, who uses PC10(2), uses at his desk in the office (step S8). In step S4, the terminal identification information of PC3 that Mr. A uses at his desk in the office can be identified by referring to the terminal information storage unit 112 based on the employee number from PC10(2). In this embodiment, as explained using FIG. 3, it is assumed that the PC that Mr. A uses at his office is PC3(2).

[0112] Thereafter, the second video providing unit 136 starts providing the video data from PC10(1) buffered in the terminal-specific video etc. storage unit 113 to PC3(2) used by Mr. A at his own desk in the office, which was identified by the own desk terminal identifying unit 135 (step S9). As a result, as explained using Figure 3, the video of Mr. A working from home, captured by the camera unit 311 of PC10(1), is displayed on the display 307 of PC3(2) in the office, making it possible for other employees who come to work to recognize that Mr. A is working from home.

[0113] After this, suppose that person B comes to person A's desk at the company to check with person A, but finds out that person A is working from home. In this case, person B requests that PC3(2) call out to person A as "person A" (step S10). In response to the call, PC3(2) requests UTM device 1 to call PC10(1), which is providing video to PC3(2) (step S11).

[0114] In this case, the second connection processing unit 137 functions in the UTM device 1 to request a call from PC10(1) (step S12). When the call request (communication request) arrives from the UTM device 1, PC10(1) notifies person A, the user of PC10(1), that there is a call request, for example by emitting a predetermined ringtone. In response, when person A instructs PC10(1) to respond to the call request, PC10(1) sends an "OK message" to the UTM device 1 indicating that it will respond to the call request.

[0115] Note that the instruction given by user A to PC10 to respond to the call request may be a voice instruction such as "I'll respond," or may be by selecting a predetermined icon displayed on the display 307 of PC10(1) or by operating a predetermined key on the operation unit 304. In the UTM device 1, when the "OK message" from PC10(1) is received, the second connection processing unit 137 functions to send the "OK message" to the requesting PC3(2) (step S14). Furthermore, the second connection processing unit 137 connects a communication line between the requesting PC3(2) and the requesting PC10(1) (step S15). In this case, a process is started to provide PC10(2) with video data from the camera unit 311 of PC3(2) buffered in the terminal-specific video etc. storage unit 113.

[0116] As a result, video data from PC10(1) is provided to PC3(2) via terminal-specific video etc. storage unit 113, and video data from PC3(2) is provided to PC10(1) via terminal-specific video etc. storage unit 113. Therefore, between PC3(2) and PC10(1), video data from both camera units 311 and audio data from both microphones 313 are sent and received over the connected communication line, and a video and audio call is conducted. After this, when the call ends, a predetermined end operation is performed from PC3(2) or PC10(1), and the state returns to the state before the communication line was connected. In other words, the provision of camera video and collected audio from PC3(2) ends, and the state returns to a unidirectional communication state in which camera video is provided from PC10(1) to PC3(2).

[0117] <Communication using a home PC and a company PC> 17 and 18 are diagrams for explaining processing centered on the UTM device 1 when communication is performed using a PC at home while telecommuting and a PC at a location where the desired company contact is located. Similar to the example described above with reference to FIG. 4, FIGS. 17 and 18 also illustrate an example in which employee A telecommutes using PC 10(1) and needs to talk with C, who is at the office. That is, this is an example in which A and C communicate via PC 10(1) used by A telecommuting and a PC nearby C, who is at the office. For simplicity of explanation, PCs 3(1) to 3(6) connected to LAN 2 are omitted from FIGS. 17 and 18.

[0118] PC10(1) and the like connected to the UTM device 1 via the IP network 81, and PC3(1), ..., 3(n) connected to the UTM device 1 via LAN 2, each request a connection to the UTM device 1, and are authenticated using an employee number and password. The process up to this authentication is performed between each PC 3, 10 and the UTM device 1 in the same manner as the process from step S1 to step S5 described using FIG. 16. Once authentication is achieved and connection to the UTM device 1 is permitted, PC3, 10 starts the process of transmitting video data (camera video) captured by its own camera unit 311 to the UTM device 1 (steps S21 to S25).

[0119] In response to this, in the UTM device 1, the terminal-specific video etc. recording unit 132 functions to start recording the terminal identification information and video data in the terminal-specific video etc. storage unit 113 (step S26). That is, in step S26, the terminal-specific video etc. recording unit 132 secures a recording area for video data etc. from PC10(1), PC3(11) to PC3(14) in the terminal-specific video etc. storage unit 113. Then, the terminal-specific video etc. recording unit 132 starts recording the terminal identification information and video data from PC10(1), PC3(11) to PC3(14) in the recording area for each PC secured in the terminal-specific video etc. storage unit 113. The processing of step S26 starts buffering the camera video from each PC 3, 10 using the video data storage area in the terminal-specific video etc. storage unit 113.

[0120] When person A, who is working from home, needs to discuss something with person C, person A performs an operation to request PC 10(1) to provide a display list (step S27). As described above, this display list is a list of images based on video data from each PC. In other words, this display list, as shown on display 307 of PC 10(1) in Fig. 4, associates the video data of small images for displaying a list of images based on video data from each PC with the terminal identification information of the PC that provided the images.

[0121] Upon receiving the operation input, PC 10(1) forms a list request (a request to provide a display list) and transmits it to the UTM device 1 (step S28). Upon receiving the list request, the UTM device 1 causes the first video etc. providing unit 133 to function, forms a display list using the data stored in the terminal-specific video etc. storage unit 113 (step S29), and provides the formed display list to the requesting PC 10(1) (step S30).

[0122] PC10(1) receives the display list from UTM device 1, displays it on display 307, and provides it to user A (step S31). Then, proceeding to the processing of Fig. 8, user A of PC10(1) refers to the display list displayed on display 307 of PC10(1), scrolling the display if necessary, etc., to find the video in which C appears. If found, user A performs a predetermined operation, such as double-clicking on the portion of the video displayed on display 307, to select and input the video (step S32).

[0123] As described above, each small video included in the display list is associated with the terminal identification information (IP address or MAC address) of the PC equipped with the camera unit 311 that captured that small video. Therefore, PC10(1) forms a connection request including the terminal identification information associated with the selected small video and sends this to the UTM device 1 (step S33). In the UTM device 1 that receives the connection request from PC10(1), the first connection processing unit 134 functions to extract the terminal identification information from the connection request from PC10(1) and identify the PC to connect to (step S34). Here again, as explained using FIG. 4, it is assumed that the PC to connect to is PC3(13) installed in the laboratory area Ar2.

[0124] In this case, the first connection processing unit 134 of the UTM device 1 forms and transmits a connection request to PC3(2) (step S35). When PC3(2) receives the connection request from the UTM device 1, if it is in an operating state in which it can connect a communication line, it forms an "OK message" and transmits this to the UTM device 1 (step S36). When the UTM device 1 receives the "OK message" from PC3(2), the first connection processing unit 134 functions to connect a communication line between the requesting PC10(1) and the requested PC3(13) (step S37).

[0125] As a result, the camera video from PC10(2) buffered in the terminal-specific video etc. storage unit 113 is supplied to PC3(13), and the camera video from PC3(13) is supplied to PC10(1). Similarly, the audio collected by the microphone 313 of PC10(2) is supplied to PC3(13), and the audio collected by the microphone 313 of PC3(13) is supplied to PC10(1). In this way, video and audio communication is possible between PC10(1) and PC3(13).

[0126] Although not shown in Figures 17 and 18, before the call starts, the camera image from PC10(1) used by Mr. A, who is working from home, is displayed on display 307 of PC3(2) installed at Mr. A's desk at the office. However, as mentioned above, once a communication line is connected between PC10(1) and PC3(13) and the call starts, the provision of the camera image from PC10(1) to PC3(2) is interrupted. This prevents PC3(2) from calling Mr. A, who is using PC10(1).

[0127] <Use of the person recognition unit 138 and the connection destination identification unit 139> 4, 17, and 18, Mr. A, an employee of the requesting PC 10(1), uses the display list to identify PC 3(2) having a camera unit 311 capturing the target person (employee) and establishes a communication line with the identified PC 3(2). However, this is not limited to this. The UTM device 1 includes a person recognition unit 138 and a connection destination identification unit 139. The person recognition unit 138 can identify employees in the area captured by the camera unit 311 using video data, facial recognition data, and clothing recognition data from the camera unit 311 of PC 3, and also using audio data and audio recognition data from the microphone 313 of PC 3. In this case, person recognition can also be performed using location history data. The recognition result by the person recognition unit 138 is updated in the employee identification information field of the terminal-specific video storage unit 113.

[0128] Therefore, if the name and department of the target employee are input from the requesting PC 3, 10, the employee number of the target employee can be identified based on the data stored in the employee information storage unit 111. The connection destination identification unit 139 functions using this identified employee number, and by referencing the data stored in the terminal-specific video etc. storage unit 113, it is possible to uniquely determine which PC 3, 10 the target employee is near. Therefore, by connecting a communication line between the requesting PC 3, 10 that has input the name and department of the target employee and the PC 3, 10 identified by the connection destination identification unit 139, a call can be made.

[0129] <Using the connection information display processing unit> Furthermore, when an attempt is made to interrupt a call with a PC 3, 10 that is already connected to a communication line and in a call as the other party, the connection information display processing unit 140 comes into operation. In this case, the PC 3, 10 of the employee who interrupts the call later displays information indicating the other party who has a communication line connected to the PC 3, 10 that is the connection request destination. Furthermore, the PC 3, 10 that is already connected to a communication line and in a call as the other party displays information indicating the newly connected other party. This makes it possible to reliably identify who is talking when a call is being made using the PC 3, 10. In other words, not only two-party calls but also multi-party calls are possible.

[0130] When the UTM device 1 connects a communication line, it associates the terminal identification information of the PC connected to the communication line and stores and manages it in, for example, the storage device 103. Therefore, by referring to the data stored in the terminal information storage unit 112 based on the terminal identification information of the information in the storage device 103, it is possible to identify the name of the employee who is using the PC 3, 10, and display this as connection destination information. Therefore, the terminal information storage unit 112 may store and retain not only the terminal identification information of the PC 3 used in the company, but also the employee number and terminal identification information provided by the PC 10 when the PC 3, 10 connects to the UTM device 1 in a manner that allows them to be distinguished from the PC 3. In this embodiment, the terminal identification information is an IP address or a MAC address, but any information that can uniquely identify each PC 3, 10 may also be used.

[0131] 17 and 18, a case has been described in which PC10(1) connected to UTM device 1 via IP network 81 calls PC3(13) connected to UTM device 1 via LAN2, and a communication line is connected between the two. However, this is not limited to this. Even when a communication line is connected between PC10 connected to UTM device 1 via IP network 81, or when a communication line is connected between PC3 connected via LAN2, the communication line can be connected in the same manner as the processing described using FIGS. 17 and 18. Of course, even when PC3 connected to UTM device 1 via LAN2 calls PC10 connected to UTM device 1 via IP network 81, the communication line can be connected in the same manner as the processing described using FIGS. 17 and 18.

[0132] [Effects of the embodiment] In the communication system of the above-described embodiment, the UTM device 1 functions to accurately track the location of employees using PCs 3 installed in the company or PCs 10 used when telecommuting, and enables easy communication by connecting a communication line. In other words, within a communication system centered around the UTM device 1, a communication line can be connected and a call can be made simply by calling the desired PC or by selecting the desired PC and calling it. Therefore, calls can be made without sending an invitation email or requiring authentication when joining a conference, as is the case with conventional web conferences. Moreover, calls can be made not only between two parties but also between multiple parties.

[0133] [Modifications, etc.] In the above-described embodiment, camera images from each PC 3, 10 are buffered via the terminal-specific image storage unit 113 so that they can be transmitted to the required destination. After the communication line is connected, audio data collected by the microphone 313 can be sent and received between the PCs connected to the communication line via the UTM device 1. In this case, the audio data can also be buffered via the terminal-specific image storage unit 113 of the UTM device 1 and transmitted to the destination, just like the camera images.

[0134] Furthermore, in the above-described embodiment, an example has been described in which a relay device of the present invention is provided to a UTM device 1 that has been widely adopted in companies and the like in recent years, but the present invention is not limited to this. It is also possible to provide a dedicated relay server device equipped with the functions of the relay device of the present invention, or to install the functions of the relay device of the present invention in a so-called gateway device. In short, the relay device of the present invention can be applied to any device equipped with the function of relaying communications between different networks or communications within a specified network.

[0135] In the above-described embodiment, the images transmitted between PCs are basically moving images, including the small images in the display list. However, this is not limited to this. For example, still images may be captured at predetermined intervals, such as every few seconds or every few tens of seconds, through the camera units 311 of the PCs 3 and 10, and these images may be transmitted between the PCs. [Explanation of symbols]

[0136] 1...UTM device, 101T...connection end, 101...WAN I / F, 102...control unit, 103...storage device, 104...LAN I / F, 104T...connection end, 110...information storage unit, 111...employee information storage unit, 112...terminal information storage unit, 113...terminal-specific video etc. storage unit, 114...face recognition data storage unit, 115...clothing etc. recognition data storage unit, 116...voice recognition data storage unit, 117...location history storage unit, 120...security function unit, 130...relay processing unit, 131...authentication processing unit, 132...terminal-specific video etc. recording unit, 133...first video providing unit, 134...first connection processing unit, 135...own desk terminal identification unit, 136...second video providing unit, 137...second connection processing unit, 138...person recognition unit, 139...connection destination identification unit, 140...connection information display processing unit, 2...LAN, 3, 3(1), 3(2), 3(3)...PC, 301T...connection end, 301...LAN I / F, 302...control unit, 303...storage device, 304...operation unit, 305...USB I / F, 306...display controller, 307...display, 308...audio output processing unit, 309...speaker, 310...video input processing unit, 311...camera unit, 312...audio input processing unit, 313...microphone, 4...telephone control device, 5...internal telephone network, 6(1), 6(2), 6(3)...telephone terminal, 8...wide area network, 81...IP network, 82...external telephone network, 9(1)...external telephone, 10, 10(1)...PC, 11(1)...smartphone, BS1...base station

Claims

1. A relay device to which a plurality of communication terminals are connected and which relays communications between the communication terminals, each of the plurality of communication terminals is provided with a camera unit, a display unit, a microphone, and a speaker, and is capable of communication using video and audio; a terminal-specific video etc. recording means for recording video recording data, which associates terminal identification information from each of the plurality of communication terminals with video data captured by the camera unit, in a terminal-specific video etc. storage unit; a first providing means for providing the video recording data for each of the communication terminals in the terminal-specific video etc. storage unit to the communication terminal that has made the provision request in response to a provision request for the video recording data from one of the communication terminals; a first connection processing means for connecting a communication line between the communication terminal that is a request source for provision and a communication terminal identified by the terminal identification information included in the connection request in response to a connection request including the terminal identification information from the communication terminal that is a request source for provision and to which the video recording data is provided from the first providing means; Equipped with A relay device that enables communication using video and audio between communication terminals connected to a communication line by said first connection processing means.

2. The relay device according to claim 1, a terminal information storage unit that stores and holds the terminal identification information of the internal communication terminal connected to the device itself through a LAN (Local Area Network) and the user identification information of the user who uses the communication terminal in association with each other; a terminal identification means for referring to the data stored in the terminal information storage unit based on user identification information from the external communication terminal connected to the device via a WAN (Wide Area Network), and identifying the internal communication terminal used by the user identified by the user identification information; a second providing means for providing the video record data from the external communication terminal recorded in the terminal-specific video etc. storage unit to the internal communication terminal identified by the terminal identifying means; a second connection processing means for connecting a communication line between the internal communication terminal that is a communication request source and the external communication terminal identified by the terminal identification information included in the communication request in response to a communication request from the internal communication terminal that includes the terminal identification information of the external communication terminal; A relay device comprising:

3. 3. The relay device according to claim 1, a user information storage unit that stores and holds information about a user and user identification information in association with each other; a person recognition unit that recognizes a person included in an image formed from video data captured through the camera unit of the communication terminal and identifies the user identification information of the person; a user identification information recording means for recording the user identification information identified by the person recognition means in the terminal-specific video etc. storage unit in association with the terminal identification information of the communication terminal; a request receiving means for receiving a search request including information about the target user from the communication terminal connected to the device; a user identification information acquiring means for referencing data stored in the user information storage unit based on information about the user included in the search request received through the request receiving means, and acquiring the corresponding user identification information; a terminal identification information acquisition means for referencing the data stored in the storage unit for storing images etc. by terminal based on the user identification information acquired by the user information acquisition means, and acquiring terminal identification information of a communication terminal capturing an image including a person specified by the user identification information; Equipped with A relay device characterized in that the first connection processing means connects a communication line between the communication terminal that originated the search request and the communication terminal identified by the terminal identification information acquired by the terminal identification information acquisition means.

4. The relay device according to claim 3, The person recognition means The communication terminal includes one or both of an image recognition processing means for performing image recognition based on an image formed by video data captured through the camera unit of the communication terminal and predetermined image recognition information, and a voice recognition processing means for performing voice recognition based on voice data collected by the microphone of the communication terminal and predetermined voice recognition information, the predetermined image recognition information is user-specific face recognition data that associates user identification information of each user of the plurality of communication terminals with face recognition data prepared in advance, and / or user identification information acquired in association with the user identification information and clothing recognition data that can recognize the clothing of the user; The predetermined voice recognition information is voice recognition data that associates user identification information of each user of the plurality of communication terminals with pre-collected voice data of each user. A relay device characterized by:

5. 3. The relay device according to claim 1, A connection information display processing means for processing an image captured through a camera unit of a communication terminal connected to the communication line to include information about the other party of the communication. A relay device comprising:

Images