Space sharing system, space sharing method, and program

Abstract

To share space to facilitate smoother communication.SOLUTION: A space sharing system 1 having a first imaging device 11A and a first display device 17A, and a second imaging device 11B and a second display device 17B, which are arranged in different spaces includes an information acquisition unit 21 that acquires video and audio from the first imaging device 11A and the second imaging device 11B, an output control unit 27 that outputs the video and audio acquired from the first imaging device 11A to the second display device 17B and outputs the video and audio acquired from the second imaging device 11B to the first display device 17A, an operation reception unit 22 that accepts operations on a display screen of the second display device 17B, and an identification unit 23 that identifies an information terminal device corresponding to the operation position on the display screen. The information acquisition unit 21 acquires video and audio from a third imaging device 31 included in the identified information terminal device, and the output control unit 27 outputs the video and audio acquired from the third imaging device 31 to the second display device 17B.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a space sharing system, a space sharing method, and a program. [Background technology]

[0002] 2. Description of the Related Art A system is known in which a virtual space is shared between geographically separated spaces by using an image capture device and a display device (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2004-056161 A Summary of the Invention [Problem to be solved by the invention]

[0004] Such a space sharing system is installed, for example, in different offices of the same company. By using the space sharing system, there is an effect of facilitating mutual communication. However, when calling a person in another space using the space sharing system, it is necessary to set up a call by the space sharing system in advance or to call the person by outputting a voice via a display device. In addition, since communication is performed via a display device provided in the space sharing system, the person who is called needs to move in front of the display device.

[0005] The present disclosure aims to enable smoother communication by sharing a space. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, the space-sharing system according to the present disclosure is a space-sharing system including a first imaging device and a first display device arranged in a first space, and a second imaging device and a second display device arranged in a second space, and includes an information acquisition unit that acquires video images and picked up audio by the first imaging device and the second imaging device, an output control unit that outputs the video images and picked up audio by the first imaging device to the second display device and outputs the video images and picked up audio by the second imaging device to the first display device, an operation reception unit that accepts operations on a display screen of the second display device, and an identification unit that identifies an information terminal device corresponding to the operation position on the display screen of the second display device accepted by the operation reception unit, wherein the information acquisition unit acquires video images and picked up audio by a third imaging device provided in the information terminal device identified by the identification unit, and the output control unit outputs the video images and picked up audio by the third imaging device provided in the information terminal device identified by the identification unit to the second display device.

[0007] The space sharing method according to the present disclosure is a space sharing method for a space sharing system including a first imaging device and a first display device arranged in a first space, and a second imaging device and a second display device arranged in a second space, in which the space sharing system executes the following steps: acquiring video captured and audio picked up by the first imaging device and the second imaging device; outputting the video captured and audio picked up by the first imaging device to the second display device and outputting the video captured and audio picked up by the second imaging device to the first display device; determining that an operation on a display screen of the second display device has been accepted; when an operation on the display screen of the second display device has been accepted, identifying an information terminal device corresponding to the operation position on the display screen of the second display device where the operation was accepted; acquiring video captured and audio picked up by a third imaging device provided in the identified information terminal device; and outputting the video captured and audio picked up by the third imaging device to the second display device.

[0008] The program of the present disclosure causes a computer that controls a space sharing system to perform the following steps: acquiring video captured and audio picked up by a first imaging device and a second imaging device; outputting the video captured and audio picked up by the first imaging device to a second display device and outputting the video captured and audio picked up by the second imaging device to the first display device; determining that an operation on a display screen of the second display device has been accepted; if an operation on the display screen of the second display device has been accepted, identifying an information terminal device corresponding to the operation position on the display screen of the second display device where the operation was accepted; acquiring video captured and audio picked up by a third imaging device provided in the identified information terminal device; and outputting the video captured and audio picked up by the third imaging device to the second display device. Effect of the Invention

[0009] According to the present disclosure, communication can be carried out more smoothly by sharing a space. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram showing an example of the configuration of a space sharing system according to the first embodiment. [Diagram 2] FIG. 2 is a schematic diagram showing an example of installation of the space sharing device, in which the device is installed in the first space. [Diagram 3] FIG. 3 is a schematic diagram showing an example of installation of the space sharing device in the first space. [Figure 4] FIG. 4 is a schematic diagram showing an example of installation of the space sharing device in the second space. [Diagram 5] FIG. 5 is a block diagram showing an example of the configuration of the space sharing device according to the first embodiment. [Figure 6] FIG. 6 is a block diagram showing an example of the configuration of an information terminal device according to the first embodiment. [Figure 7] FIG. 7 is a flowchart showing the space sharing method according to the first embodiment. [Figure 8] FIG. 8 is a diagram showing an example of a display screen of the space sharing device according to the first embodiment. [Figure 9] FIG. 9 is a diagram showing another example of the display screen of the space sharing device according to the first embodiment. [Figure 10] FIG. 10 is a block diagram showing an example of the configuration of the space sharing device according to the second embodiment. [Figure 11] FIG. 11 is a flowchart showing a space sharing method according to the second embodiment. [Figure 12] FIG. 12 is a diagram showing an example of a display screen of the space sharing device according to the second embodiment. [Figure 13] FIG. 13 is a flowchart showing a space sharing method according to the third embodiment. [Figure 14] FIG. 14 is a diagram showing an example of a display screen of the space sharing device according to the third embodiment. [Figure 15] FIG. 15 is a diagram showing an example of a display screen of the space sharing device according to the third embodiment. [Figure 16] FIG. 16 is a schematic diagram showing another example of installation of the space sharing device, which is installed in the first space. [Figure 17] FIG. 17 is a diagram showing an example of a display screen of the space sharing device according to the fourth embodiment. [Figure 18] FIG. 18 is a flowchart showing a space sharing method according to the fourth embodiment. [Figure 19] FIG. 19 is a flowchart showing a space sharing method according to the fifth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of a space sharing system, a space sharing method, and a program according to the present invention will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the following preferred embodiments.

[0012] [First embodiment] (Space sharing system) 1 is a schematic diagram showing a configuration example of a space sharing system according to a first embodiment. The space sharing system 1 constantly shares video and audio in real time in a plurality of real spaces. For example, the space sharing system 1 can be applied to remote offices to give the impression that the office is shared in a pseudo manner.

[0013] The space sharing system 1 includes a space sharing device 10 arranged in each of a plurality of spaces, and at least one information terminal device 30 arranged in each space. The number of spaces shared by the space sharing system 1 is not limited.

[0014] In the following description, as an example, the space sharing system 1 will be described as sharing two spaces, a first space A and a second space B. A first space sharing device 10A is arranged in the first space A. In the first space A, an information terminal device 30A 1 , information terminal device 30A 2 The information terminal device 30A 1 , information terminal device 30A 2 is connected to the first space sharing device 10A and the second space sharing device 10B so as to be able to communicate data with them. The second space sharing device 10B is disposed in the second space B. In the second space B, an information terminal device 30B 1 , information terminal device 30B 2 etc. are arranged. 1 , information terminal device 30B 2 is connected to the first space sharing device 10A and the second space sharing device 10B so as to be able to perform data communication. 1 , information terminal device 30A 2 , information terminal device 30B 1 , information terminal device 30B 2are connected via an in-house network or the like. When it is not particularly necessary to distinguish between the first space sharing device 10A and the second space sharing device 10B, they will be described as the space sharing device 10. When it is not particularly necessary to distinguish between the information terminal devices 30, they will be described as the information terminal device 30. Similarly, in the description of each component, when it is not particularly necessary to distinguish between the components provided in the first space sharing device 10A and the components provided in the second space sharing device 10B, the reference numerals will not be marked with A or B, and when it is necessary to distinguish between the components provided in the first space sharing device 10A and the components provided in the second space sharing device 10B, the reference numerals will be marked with A or B.

[0015] An example of installation of the first space sharing device 10A and the information terminal device 30A in the first space A will be described with reference to Fig. 2. Fig. 2 is a schematic diagram showing an example of installation of the space sharing devices, for example, in a first space such as an office. The first space sharing device 10A is preferably disposed in a position where it can capture an image of the entire space A or the range of departments where space sharing is useful. In the first space A, the information terminal device 30A 1 or information terminal device 30A 14 The person u1, the person u3 to the person u6, the person u9, the person u10, the person u12 to the person u14 are arranged on the information terminal device 30A. 1 , information terminal device 30A 3 or information terminal device 30A 6 , information terminal device 30A 9 , information terminal device 30A 10 , information terminal device 30A 12 or information terminal device 30A 14 , respectively. In other words, the person ua is a person who is using the space sharing system 1. Also, the person u1, the person u3 to the person u6, the person u9, the person u10, the person u12 to the person u14 are using the information terminal device 30A. 1 , information terminal device 30A 3 or information terminal device 30A 6 , information terminal device 30A 9 , information terminal device 30A 10 , information terminal device 30A 12or information terminal device 30A 14 Each person logs in to and uses the information terminal device 30 using their own account information. That is, each information communication terminal 30 and the person using the information terminal device 30 are linked.

[0016] Space sharing in the first space A and the second space B will be described with reference to FIG. 3 and FIG. 4. FIG. 3 is a schematic diagram showing an example of installation of the space sharing device, and is a diagram showing the device installed in the first space. FIG. 4 is a schematic diagram showing an example of installation of the space sharing device, and is a diagram showing the device installed in the second space. Also, FIG. 3 and FIG. 4 show a state in which communication is performed using the space sharing system 1. As shown in FIG. 3, a person ua present in the first space A faces the first display device 17A and uses the space sharing system 1. The first space sharing device 10A arranged in the first space A outputs video and audio of the second space B. When a person ub faces the second display device 17B, the first space sharing device 10A displays an image of the person ub located in the second space B in almost life-size. As shown in FIG. 4, a person ub present in the second space B faces the second display device 17B and uses the space sharing system 1. The second space sharing device 10B arranged in the second space B outputs video and audio of the first space A. When the person ua faces the first display device 17A, the second space sharing device 10B displays an image of the person ua located in the first space A at approximately life-size. In this manner, a virtual space is shared between the first space A and the second space B. The person ua present in the first space A and the person ub present in the second space B communicate with each other using the space sharing system 1. The communication using the space sharing system 1 may be referred to as a phone call.

[0017] (Space sharing device) 5 is a block diagram showing a configuration example of a space sharing device according to the first embodiment. The space sharing device 10 is a general-purpose server device including a distributed type realized by a computer, or a personal computer. The space sharing device 10 controls space sharing in different spaces. In this embodiment, the space sharing device 10 controls space sharing between a first space A and a second space B.

[0018] In the space sharing device 10, a space sharing control device 20 described later may centrally control each imaging device 11 and each display device 17 installed in different spaces. The space sharing device 10 may be configured such that the space sharing control device 20 is arranged in each space and connected to each other via a network. In this embodiment, the space sharing device 10 is described as being arranged in each of the first space A and the second space B.

[0019] The space sharing device 10 includes an imaging device 11 , an operation unit 13 , a display device 17 , a communication unit 19 , and a space sharing control device 20 .

[0020] The imaging device 11 captures an image of the space in which it is placed and collects an audio. More specifically, for example, a first imaging device 11A placed in a first space A captures an image of the first space A and collects an audio. For example, a second imaging device 11B placed in a second space B captures an image of the second space B and collects an audio. The imaging device 11 acquires the captured image and collected audio as information by an information acquisition unit 21. The imaging device 11 is placed in the vicinity of the display device 17. The imaging device 11 includes a camera 111 and a microphone 112.

[0021] The camera 111 captures an image of the space in which it is placed. The camera 111 is placed, for example, around the display device 17 or in front of the display device 17. The camera 111 captures an image of a person facing the display device 17 placed in the same space. More specifically, for example, the camera 111 of the first imaging device 11A placed in the first space A captures an image of the first space A. For example, the camera 111 of the second imaging device 11B placed in the second space B captures an image of the second space B. The camera 111 outputs the captured image to the image acquisition unit 211 of the space sharing control device 20.

[0022] The microphone 112 collects audio in the space in which it is placed. The microphone 112 is placed, for example, around the display device 17 or in front of the display device 17. The microphone 112 collects audio spoken by a person facing the display device 17 placed in the same space. More specifically, for example, the microphone 112 of the first imaging device 11A placed in the first space A collects audio in the first space A. For example, the microphone 112 of the second imaging device 11B placed in the second space B collects audio in the second space B. The microphone 112 outputs audio information related to the collected audio to the audio acquisition unit 212 of the space sharing control device 20.

[0023] The operation unit 13 is an input device capable of inputting various operations to the space sharing device 10. The operation unit 13 outputs an operation signal corresponding to the received input operation to the space sharing control device 20. The operation unit 13 includes, for example, a touch panel, buttons, switches, etc., arranged on top of the display unit 171. The operation unit 13 can accept, for example, a key input operation, a click operation, or a touch operation.

[0024] In this embodiment, the operation unit 13 is a touch panel arranged on the display unit 171. The operation unit 13 can accept a touch operation on an information terminal device 30 or a person in another space being displayed on the display screen of the display unit 171 as a person designation operation for calling the information terminal device 30 or the person. More specifically, for example, the operation unit 13 provided on the display unit 171 of the first display device 17A arranged in the first space A can accept a person designation operation for calling the information terminal device 30 or the person present in the second space B. For example, the operation unit 13 provided on the display unit 171 of the second display device 17B arranged in the second space B can accept a person designation operation for calling the information terminal device 30 or the person present in the first space A.

[0025] A predetermined range including a range in which the information terminal device 30 or a person included in the displayed image is detected may be set as a touch range selectable in the person designation operation.

[0026] The storage unit 14 stores information such as the contents of calculations and programs of the space sharing device 10. The storage unit 14 is, for example, a storage device such as a random access memory (RAM) or a solid state drive (SSD).

[0027] The storage unit 14 stores information indicating the information terminal device 30 arranged in the imaging range of the imaging device 11 and the account user of each information terminal device 30. The storage unit 14 acquires and stores operation status information indicating whether the information terminal device 30 is operating or the attendance / non-attendance information of each account user. More specifically, for example, the storage unit 14 of the first space sharing device 10A arranged in the first space A stores information indicating the information terminal device 30 arranged in the imaging range of the first imaging device 11A and the account user of each information terminal device 30. The storage unit 14 also stores operation status information of the information terminal device 30 in the first space A and attendance / non-attendance information of each account user. For example, the storage unit 14 of the second space sharing device 10B arranged in the second space B stores information indicating the information terminal device 30 arranged in the imaging range of the second imaging device 11B and the account user of each information terminal device 30. The storage unit 14 also stores the operation state information of the information terminal device 30 in the second space B and the attendance / non-attendance information of each account user.

[0028] The display device 17 displays an image of the other party's space that shares the space and outputs audio. The display of the image and the output of the audio of the display device 17 are controlled by the output control unit 27. The display device 17 includes a display unit 171 and an audio output unit 172.

[0029] The display unit 171 displays various images including characters and images. The display unit 171 is, for example, a display including a liquid crystal display, an organic EL (Electro-Luminescence), and the like. The display unit 171 displays an image of the other side's space that shares the space. More specifically, for example, the display unit 171 of the first display device 17A arranged in the first space A displays an image of the second space B. For example, the display unit 171 of the second display device 17B arranged in the second space B displays an image of the first space A. The display of the display unit 171 is controlled by the display control unit 271.

[0030] The audio output unit 172 is an audio output device that outputs audio of the other party's space that shares the space. More specifically, for example, the audio output unit 172 of the first display device 17A arranged in the first space A outputs audio of the second space B. For example, the audio output unit 172 of the second display device 17B arranged in the second space B outputs audio of the first space A. The audio output of the audio output unit 172 is controlled by the audio output control unit 272.

[0031] The communication unit 19 is a communication unit for performing wired communication or wireless communication. The communication unit 19 performs communication by a communication method using, for example, Wi-Fi or a mobile phone line, and is connected to an intranet or the Internet. The communication of the communication unit 19 is controlled by a communication control unit 29. The communication unit 19 performs communication of video and audio data with other space sharing devices 10 and information terminal devices 30.

[0032] (Space sharing control device) The space sharing control device 20 controls each part of the space sharing device 10. The space sharing control device 20 has, for example, a control device such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), and a storage device such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The space sharing control device 20 may be realized by, for example, an integrated circuit such as an ASIC or an FPGA. The space sharing control device 20 may be realized by a combination of hardware and software. The space sharing control device 20 executes a program that controls the operation of the space sharing device 10 according to the present disclosure.

[0033] The space sharing control device 20 includes an information acquisition unit 21, an operation reception unit 22, a specification unit 23, an output control unit 27, and a communication control unit 29 as functional blocks realized by circuits or programs being executed.

[0034] In this embodiment, the space sharing control device 20 controls the imaging device 11 and the display device 17 arranged in each space. The space sharing control device 20 is assumed to be arranged in each of the first space A and the second space B.

[0035] The information acquisition unit 21 acquires video captured and audio collected by the imaging device 11 in the space in which the space sharing device 10 is arranged. More specifically, for example, the information acquisition unit 21 of the first space sharing device 10A arranged in the first space A acquires video captured and audio collected by the first imaging device 11A arranged in the first space A. For example, the information acquisition unit 21 of the second space sharing device 10B arranged in the second space B acquires video captured and audio collected by the second imaging device 11B arranged in the second space B. The information acquisition unit 21 includes a video acquisition unit 211 and an audio acquisition unit 212.

[0036] The information acquisition unit 21 acquires, via the communication unit 19 and the communication control unit 29, video captured by an imaging device (third imaging device) 31 included in the information terminal device 30 identified by the identification unit 23 and audio collected by the imaging device (third imaging device) 31.

[0037] The video acquisition unit 211 acquires a video captured by the camera 111. More specifically, the video acquisition unit 211 acquires a video captured by the camera 111 of people and the like present around the display device 17 arranged in the same space. More specifically, for example, the video acquisition unit 211 of the first space sharing device 10A arranged in the first space A acquires a video captured of people and the like present around the first display device 17A. For example, the video acquisition unit 211 of the second space sharing device 10B arranged in the second space B acquires a video captured of people and the like present around the second display device 17B. The video data acquired by the imaging of the camera 111 is a moving image.

[0038] The voice acquisition unit 212 acquires voice detected by the microphone 112. The voice acquisition unit 212 acquires voice such as voice spoken by a person around the display device 17 arranged in the same space. More specifically, for example, the voice acquisition unit 212 of the first space sharing device 10A arranged in the first space A acquires voice such as voice spoken by a person present in the vicinity of the first display device 17A. For example, the voice acquisition unit 212 of the second space sharing device 10B arranged in the second space B acquires voice such as voice spoken by a person present in the vicinity of the second display device 17B.

[0039] The operation reception unit 22 acquires an operation signal corresponding to an operation on the operation unit 13 from the operation unit 13. The operation reception unit 22 outputs a control signal corresponding to the acquired operation signal to each unit.

[0040] The operation reception unit 22 receives an operation on the display screen of the display device 17. The operation reception unit 22 receives a touch operation on the information terminal device 30 or person in another space displayed on the display screen of the display unit 171 as a person designation operation for calling the information terminal device 30 or person. More specifically, for example, the operation reception unit 22 of the first space sharing device 10A arranged in the first space A receives a person designation operation on the information terminal device 30 or person in the second space B displayed on the display unit 171 of the first display device 17A. For example, the operation reception unit 22 of the second space sharing device 10B arranged in the second space B receives a person designation operation on the information terminal device 30 or person in the first space A displayed on the display unit 171 of the second display device 17B.

[0041] When a person designation operation is accepted, the identification unit 23 identifies the information terminal device 30 to be called. When a person designation operation is accepted, the identification unit 23 identifies the information terminal device 30 corresponding to the person designation operation.

[0042] A case where an information terminal device 30 is specified on the display screen and a person specification operation is performed will be described. The identification unit 23 identifies the information terminal device 30 that exists within the range imaged by the imaging device 11 arranged in another space, which corresponds to the operation position on the display screen of the display device 17 accepted by the operation acceptance unit 22. More specifically, for example, the identification unit 23 of the first space sharing device 10A arranged in the first space A identifies the information terminal device 30 that exists within the range imaged by the second imaging device 11B, which corresponds to the operation position on the display screen of the first display device 17A accepted by the operation acceptance unit 22. For example, the identification unit 23 of the second space sharing device 10B arranged in the second space B identifies the information terminal device 30 that exists within the range imaged by the first imaging device 11A, which corresponds to the operation position on the display screen of the second display device 17B accepted by the operation acceptance unit 22.

[0043] A case where a person is specified on the display screen and a person specification operation is performed will be described. The specification unit 23 specifies the information terminal device 30 associated with a person present in a range captured by the imaging device 11 arranged in another space, which corresponds to the operation position on the display screen of the display device 17 accepted by the operation acceptance unit 22. More specifically, for example, the specification unit 23 of the first space sharing device 10A arranged in the first space A specifies a person present in the second space B present in a range captured by the second imaging device 11B corresponding to the operation position on the display screen of the first display device 17A accepted by the operation acceptance unit 22. For example, the specification unit 23 of the second space sharing device 10B arranged in the second space B specifies a person present in the first space A present in a range captured by the first imaging device 11A corresponding to the operation position on the display screen of the second display device 17B accepted by the operation acceptance unit 22.

[0044] The positions of the information terminal devices 30 within the imaging range of the imaging device 11 are associated in advance. For example, in the case of fixed seats where work is performed at a fixed seat, the coordinate range of the image and the person sitting in the seat are associated in advance and stored in the storage unit 14. The person and the logged-in information terminal device 30 are associated from the seat corresponding to the coordinate range in which the person is recognized from the image. For example, in the case of free address seats where seats can be freely selected, the position information of the information terminal device 30 is obtained from a beacon device or the like provided in each information terminal device 30. The obtained position information of the information terminal device 30 is associated with the coordinates in the image.

[0045] Furthermore, the identification unit 23 judges whether the identified information terminal device 30 can be called. The identification unit 23 judges whether the identified information terminal device 30 can be called from the operation state information of the information terminal device 30 stored in the storage unit 14, information on the running application, or the attendance / non-attendance information of each account user. When the information terminal device 30 is in an operating state, indicates that the account user has attended work, and is not communicating with other users through an application for communicating online with other users, the identification unit 23 judges that the identified information terminal device 30 can be called.

[0046] The output control unit 27 outputs the video captured by the imaging device 11 arranged in the other space and the collected audio received by the communication control unit 29 to the display device 17. More specifically, for example, the output control unit 27 of the space sharing device 10A arranged in the first space A outputs the video captured by the second imaging device 11B and the collected audio received by the communication control unit 29 to the first display device 17A arranged in the first space A. For example, the output control unit 27 of the space sharing device 10B arranged in the second space B outputs the video captured by the first imaging device 11A and the collected audio received by the communication control unit 29 to the second display device 17B arranged in the second space B.

[0047] When the information terminal device 30 is identified by the identification unit 23, the output control unit 27 outputs the video captured by the imaging device (third imaging device) 31 of the information terminal device 30 to the display device 17. More specifically, for example, when the information terminal device 30 is identified by the identification unit 23, the output control unit 27 of the first space sharing device 10A arranged in the first space A outputs the video captured by the third imaging device 31 present in the second space B and the collected audio to the first display device 17A. For example, when the information terminal device 30 is identified by the identification unit 23, the output control unit 27 of the second space sharing device 10B arranged in the second space B outputs the video captured by the third imaging device 31 present in the first space A and the collected audio to the second display device 17B.

[0048] The display control unit 271 controls the display of the image on the display unit 171. For example, the display control unit 271 controls the display unit 171 to display the image captured in the other space. When a person is captured in the image captured in the other space, the display control unit 271 displays the image including the person present in the other space. More specifically, for example, the display control unit 271 of the first space sharing device 10A arranged in the first space A displays the image captured in the second space B. For example, the display control unit 271 of the second space sharing device 10B arranged in the second space B displays the image captured in the first space A. In addition, the display control unit 271 controls the display unit 171 to display the image captured by the imaging device 31 of the information terminal device 30 identified by the identification unit 23 on the display unit 171.

[0049] The audio output control unit 272 controls the output of audio in the audio output unit 172. For example, the audio output control unit 272 controls the audio output unit 172 to output audio collected in another space. When the audio collected in another space includes the audio of a person present in the other space, the audio output control unit 272 outputs audio including the audio of the person present in the other space. More specifically, for example, the audio output control unit 272 of the first space sharing device 10A arranged in the first space A outputs audio collected in the second space B. For example, the audio output control unit 272 of the second space sharing device 10B arranged in the second space B outputs audio collected in the first space A.

[0050] The communication control unit 29 controls the communication unit 19 to control transmission and reception of data including video and audio between the other space sharing device 10 and the information terminal device 30. More specifically, the communication control unit 29 controls, for example, the communication unit 19 to control communication between the other space sharing device 10 and the information terminal device 30. The communication control unit 29 controls, for example, the communication unit 19 to receive video captured in another space and audio collected from the other space sharing device 10, and receives video captured by the imaging device 31 of the information terminal device 30 from the information terminal device 30. The communication control unit 29 controls, for example, the communication unit 19 to transmit video captured by the imaging device 11 and audio collected to the other space sharing device 10 and the information terminal device 30.

[0051] More specifically, for example, the communication control unit 29 of the first space sharing device 10A arranged in the first space A controls the video and audio of the first space A acquired by the information acquisition unit 21 to be outputtable on the second display device 17B and the information terminal device 30 arranged in the second space B, and controls the video and audio captured by the second imaging device 11B and the imaging device 31 to be received from the second space sharing device 10B and the information terminal device 30 arranged in the second space B. For example, the communication control unit 29 of the space sharing device 10B arranged in the second space B controls the video and audio of the second space B acquired by the information acquisition unit 21 to be outputtable on the first display device 17A and the information terminal device 30 arranged in the first space A, and controls the video and audio captured by the first imaging device 11A and the imaging device 31 to be received from the first space sharing device 10A and the information terminal device 30 arranged in the first space A.

[0052] When the communication control unit 29 receives a person designation operation, it controls the communication unit 19 to transmit call information to the designated information terminal device 30 by voice, a message, or the like.

[0053] (Information terminal device) FIG. 6 is a block diagram showing a configuration example of an information terminal device according to the first embodiment. The information terminal device 30 is used in a space shared by the space sharing system 1. In this embodiment, the information terminal device 30 is, for example, a terminal device used by a person present in the first space A and the second space B. The information terminal device 30 is, for example, an information terminal device such as a general-purpose desktop or notebook personal computer, a smartphone, or a tablet terminal used by a person. The information terminal device 30 is connected to the space sharing device 10 via a network so as to be able to communicate data. More specifically, the information terminal device 30 arranged in the first space A is connected to the first space sharing device 10A arranged in the first space A via a network so as to be able to communicate data. The information terminal device 30 arranged in the second space B is connected to the second space sharing device 10B arranged in the second space B via a network so as to be able to communicate data.

[0054] The information terminal device 30 acquires the account user information of the information terminal device 30 based on the account information of the person who is logged in to the information terminal device 30 .

[0055] The information terminal device 30 includes an imaging device 31 , an operation unit 33 , a display unit 37 , an audio output unit 38 , a communication unit 39 , and a terminal control device 40 .

[0056] The imaging device 31 captures images and sounds around the information terminal device 30. The imaging device 31 includes a camera 311 and a microphone 312.

[0057] Camera 311 captures an image in a direction facing information terminal device 30. Camera 311 captures an image in a direction facing display device 37 of information terminal device 30. In other words, camera 311 captures an image in a direction in which a person using information terminal device 30 is present. Camera 311 is disposed, for example, near display device 37. Camera 311 outputs the captured image to image acquisition unit 411 of terminal control device 40.

[0058] The microphone 312 picks up sounds around the information terminal device 30. The microphone 312 may pick up sounds from a direction facing the display device 37 of the information terminal device 30. In other words, the microphone 312 picks up sounds uttered by a person using the information terminal device 30. The microphone 312 outputs audio information related to the picked up sounds to the audio acquisition unit 412 of the terminal control device 40.

[0059] The operation unit 33 is an input device capable of inputting various operations to the information terminal device 30. The operation unit 33 outputs an operation signal corresponding to the received input operation to the terminal control device 40. The operation unit 33 includes, for example, a keyboard, a mouse, a touch panel arranged over the display unit 37, buttons, switches, etc. The operation unit 33 can accept, for example, a key input operation, a click operation, or a touch operation.

[0060] The display unit 37 displays various images including characters and images. The display unit 37 is, for example, a display including a liquid crystal display, an organic EL display, etc. The display unit 37 displays an image of the other party's space that shares the space. In this embodiment, when a call is made from an information terminal device 30 that exists in another space, the display unit 37 displays an image captured by the imaging device 11 of the space sharing device 10 that exists in the other space. The display unit 37 may display an image captured by the imaging device 11 of the space sharing device 10 in the same space. The display of the display unit 37 is controlled by the display control unit 471.

[0061] The audio output unit 38 is an audio output device that outputs audio of the other space that shares the space. In this embodiment, when the audio output unit 38 is called by the information terminal device 30 that exists in another space, the audio output unit 38 outputs audio picked up by the imaging device 11 of the space sharing device 10 that exists in the other space. The audio output unit 38 may output audio picked up by the imaging device 11 of the space sharing device 10 in the same space. The audio output control unit 472 controls the audio output of the audio output unit 38.

[0062] The communication unit 39 is a communication unit for performing wired communication or wireless communication. The communication unit 39 performs communication by a communication method using, for example, Wi-Fi or a mobile phone line, and is connected to an intranet or the Internet. The communication of the communication unit 39 is controlled by a communication control unit 49. The communication unit 39 performs communication of data such as video and audio with the space sharing device 10 arranged in the space in which the information terminal device 30 exists. The communication unit 39 performs communication of data such as account user information and operation state information of the information terminal device 30 with the space sharing device 10 arranged in the space in which the information terminal device 30 exists.

[0063] (Terminal Control Device) The terminal control device 40 controls each part of the information terminal device 30. The terminal control device 40 has, for example, a control device such as a CPU or MPU, and a storage device such as a RAM or ROM. The terminal control device 40 executes a program that controls the operation of the information terminal device 30 according to the present disclosure. The terminal control device 40 may be realized by an integrated circuit such as an ASIC or an FPGA. The terminal control device 40 loads a stored program into a memory and executes instructions included in the program. The terminal control device 40 includes an internal memory such as the above-mentioned RAM, and the internal memory is used for temporary storage of data in the terminal control device 40. The terminal control device 40 may be realized by a combination of hardware and software.

[0064] The terminal control device 40 has an information acquisition unit 41, an operation acceptance unit 42, an output control unit 47, and a communication control unit 49 as functional blocks realized by circuits or programs being executed.

[0065] The information acquisition unit 41 acquires the video captured by the imaging device 31 and the audio collected by the imaging device 31. The information acquisition unit 41 includes a video acquisition unit 411 and an audio acquisition unit 412.

[0066] The video acquisition unit 411 acquires video captured by the camera 311. More specifically, the video acquisition unit 411 acquires video captured by the camera 311 of a person facing the information terminal device 30, etc. The video data acquired by capturing an image by the camera 311 is a moving image.

[0067] The voice acquiring unit 412 acquires the voice detected by the microphone 312. The voice acquiring unit 412 acquires voice such as a voice spoken by a person present in the vicinity of the information terminal device 30.

[0068] The operation reception unit 42 acquires an operation signal corresponding to an operation on the operation unit 33 from the operation unit 33. The operation reception unit 42 outputs a control signal corresponding to the acquired operation signal to each unit.

[0069] The output control unit 47 may cause the display device 17 to output the video captured by the space sharing device 10 in the other space and the audio collected. The output control unit 47 may also cause the display device 17 to output the video captured by the space sharing device 10 in the same space and the audio collected. More specifically, the output control unit 47 may cause the display device 17 to output the video captured by the imaging device 11 arranged in the other space or the same space and received by the communication control unit 49.

[0070] The output control unit 47 may output the video captured by the imaging device 31 and the audio collected by the imaging device 31 to the display unit 37 and the audio output unit 38. The output control unit 47 includes a display control unit 471 and an audio output control unit 472.

[0071] The display control unit 471 controls the display of the image on the display unit 37. For example, the display control unit 471 may control the display unit 37 to display the image captured by the space sharing device 10 in another space. The display control unit 471 may also control the display unit 37 to display the image captured by the space sharing device 10 in the same space.

[0072] The audio output control unit 472 controls the output of audio in the audio output unit 38. For example, the audio output control unit 472 may control the audio output unit 38 to output audio picked up by a space sharing device 10 in another space. The audio output control unit 472 may also output audio picked up by a space sharing device 10 in the same space.

[0073] The communication control unit 49 controls the communication unit 39 to control communication of data such as video and audio with the space sharing device 10 arranged in the other space. More specifically, the communication control unit 49 controls, for example, the communication unit 39 to control communication with the space sharing device 10 arranged in the other space. The communication control unit 49 controls, for example, the communication unit 39 to receive video and audio from the space sharing device 10 arranged in the other space. The communication control unit 49 controls, for example, the communication unit 39 to transmit video and audio of a person facing the display device 37 of the information terminal device 30, acquired by the information acquisition unit 41, to the space sharing device 10 arranged in the other space. The communication control unit 49 may control the communication unit 39 to control communication of data such as video and audio with the space sharing device 10 arranged in the same space.

[0074] (Space sharing method) Fig. 7 is a flowchart showing a space sharing method according to the first embodiment. The process shown in Fig. 7 is executed while the space is shared by the space sharing system 1. While the space is shared, the first space sharing device 10A arranged in the first space A outputs video and audio of the second space B. The second space sharing device 10B arranged in the second space B outputs video and audio of the first space A. Fig. 7 shows the process executed by each of the space sharing devices 10 arranged in the first space A and the second space B.

[0075] 7, the space sharing control device 20 associates the position information of the information terminal device 30 present in the space in which the space sharing device 10 is placed with the position of the image acquired by the image acquisition unit 211. The position of the information terminal device 30 in the image acquired by the image acquisition unit 211 corresponds to the operation position of the operation unit 13.

[0076] The space sharing control device 20 determines whether or not an operation has been performed on the display screen of the display device 17 of the space sharing device 10 (step S101). More specifically, the space sharing control device 20 determines whether or not an operation has been received on the display screen of the display device 17 by the operation receiving unit 22. When the space sharing control device 20 determines that an operation has been performed on the display screen of the display device 17 of the space sharing device 10 (Yes in step S101), the space sharing control device 20 proceeds to step S102. When the space sharing control device 20 does not determine that an operation has been performed on the display screen of the display device 17 of the space sharing device 10 (No in step S101), the space sharing control device 20 ends the processing of this flowchart.

[0077] When it is determined that an operation has been performed on the display screen of the display device 17 of the space sharing device 10 (Yes in step S101), the space sharing control device 20 determines whether or not the operation is an operation to specify the information terminal device 30 or person in the other space being displayed (step S102). More specifically, the space sharing control device 20 determines whether or not a specification operation to specify the information terminal device 30 or person in the other space being displayed on the display screen of the display device 17 has been received by the operation receiving unit 22. When the space sharing control device 20 determines that the operation is an operation to specify the information terminal device 30 or person in the other space being displayed (Yes in step S102), the process proceeds to step S103. When the space sharing control device 20 does not determine that the operation is an operation to specify the information terminal device 30 or person in the other space being displayed (No in step S102), the process of this flowchart ends.

[0078] When it is determined that the operation is an information terminal device 30 or person designation operation of another space being displayed (Yes in step S102), the space sharing control device 20 determines whether or not the designated information terminal device 30 can be called (step S103). More specifically, the space sharing control device 20 uses the identification unit 23 to identify the information terminal device 30 corresponding to the person designation operation. The space sharing control device 20 uses the identification unit 23 to determine whether or not the identified information terminal device 30 can be called from the operation state information of the information terminal device 30 stored in the storage unit 14, or the attendance / non-attendance information of each account user and the operation state of the application. When the space sharing control device 20 determines that the designated information terminal device 30 can be called (Yes in step S103), the process proceeds to step S104. When the space sharing control device 20 does not determine that the designated information terminal device 30 can be called (No in step S103), the process proceeds to step S109.

[0079] When it is determined that the designated information terminal device 30 can be called (Yes in step S103), the space sharing control device 20 calls the designated information terminal device 30 (step S104). More specifically, the space sharing control device 20 transmits call information by voice, message, or the like to the designated information terminal device 30 via the communication control unit 29. The space sharing control device 20 proceeds to step S105.

[0080] The space sharing control device 20 determines whether or not there has been a response (step S105). More specifically, the space sharing control device 20 determines whether or not there has been a response from the called information terminal device 30 by the communication control unit 29. If the space sharing control device 20 determines that there has been a response (Yes in step S105), it proceeds to step S106. If the space sharing control device 20 does not determine that there has been a response (No in step S105), it ends the processing of this flowchart.

[0081] When it is determined that there is a response (Yes in step S105), the space sharing control device 20 starts outputting the video captured by the imaging device 31 of the information terminal device 30 and the collected audio (step S106). More specifically, the space sharing control device 20 causes the output control unit 27 to output the video captured by the imaging device 31 of the called information terminal device 30 and the collected audio received by the communication control unit 29 to the display device 17. In other words, a call between the person using the space sharing device 10 and the person using the information terminal device 30 becomes possible. The space sharing control device 20 proceeds to step S107.

[0082] The space sharing control device 20 determines whether to end the call (step S107). More specifically, the space sharing control device 20 determines to end the call, for example, when an operation to end the call is accepted by the operation acceptance unit 22, or when an operation to end the call is accepted in the information terminal device 30 that responded. When the space sharing control device 20 determines to end the call (Yes in step S107), it proceeds to step S108. When the space sharing control device 20 does not determine to end the call (No in step S107), it executes the process of step S107 again. The end of the call in step S107 refers to the end of the call between the space sharing device 10 and the information terminal device 30.

[0083] When it is determined that the call is to be ended (Yes in step S107), the space sharing control device 20 stops outputting the video captured by the imaging device 31 of the information terminal device 30 and the collected audio (step S108). More specifically, the space sharing control device 20 causes the output control unit 27 to stop outputting the video captured by the imaging device 31 of the called information terminal device 30 and the audio to the display device 17.

[0084] When it is not determined that the designated information terminal device 30 can be called (No in step S103), the space sharing control device 20 displays a message indicating that the call is not possible (step S109). More specifically, the space sharing control device 20 controls the display unit 171 by the display control unit 271 to display a message video indicating that the call is not possible, superimposed on the video of the other space captured by the designated information terminal device 30.

[0085] In this way, the space sharing system 1 allows the first space A and the second space B to virtually share a space, and allows the information terminal device 30 to participate in the sharing of the space.

[0086] A case where a person ub present in the second space B communicates with a person present in the first space A will be described with reference to Figs. 8 and 9. Fig. 8 is a diagram showing an example of a display screen by the space sharing device according to the first embodiment. Fig. 9 is a diagram showing another example of a display screen by the space sharing device according to the first embodiment. As shown in Fig. 4, the person ub faces the second display device 17B in the second space B. The person ub checks the image of the first space A displayed on the second display device 17B.

[0087] The dashed circle in FIG. 8 indicates the person u14 or the information terminal device 30A used by the person u14 in the image displayed on the second display device 17B. 14 8, when a person ub in front of the second display device 17B touches the area indicated by the dashed circle in step S102, the information terminal device 30A 14 A pop-up window MW showing the account user name "u14-san" logged in to the information terminal device 30A is displayed. The area indicated by the dashed line may or may not be displayed on the second display device 17B. When the person ub performs a touch operation on the dashed circular area or the pop-up window MW, or a long touch operation on the dashed circular area, in step S103, 14If it is determined in step S103 that the information terminal device 30A is callable, then in step S104, 14 In step S105, the person u14 is called via the information terminal device 30A. 14 9, in step S106, the information terminal device 30A 14 The image and sound captured by the first imaging device 31 are output to the second display device 17B. At this time, the image captured by the first imaging device 11A may be displayed as the background, or may not be displayed. The sound acquired by the first imaging device 11A may be output at a lowered sound level, or may be muted. In FIG. 9, the information terminal device 30A is displayed as a window W in the image captured by the first imaging device 11A. 14 An image captured by the imaging device 31 is displayed.

[0088] (effect) As described above, in this embodiment, the information terminal device 30 present within the range imaged by the first imaging device 11A corresponding to the operation position on the display screen of the second display device 17B is identified. In this embodiment, when the information terminal device 30 is identified, the video imaged by the third imaging device 31 provided in the identified information terminal device 30 and the audio collected are output to the second display device 17B. According to this embodiment, when calling a person present in another space using the space sharing system 1, the person can be called without setting a call by the space sharing system in advance or outputting audio via the display device 17 to call the person. According to this embodiment, the person who is called can respond using the information terminal device 30 that is being used without moving to the front of the display device 17. In this way, according to this embodiment, the space can be shared and communication can be performed more smoothly.

[0089] [Second embodiment] A space sharing device 10X according to the second embodiment will be described with reference to Figs. 10 to 12. Fig. 10 is a block diagram showing a configuration example of a space sharing device according to the second embodiment. Fig. 11 is a flowchart showing a space sharing method according to the second embodiment. Fig. 12 is a diagram showing an example of a display screen by the space sharing device according to the second embodiment. In the following description, components similar to those in the space sharing device 10 are given the same or corresponding reference symbols, and detailed description thereof will be omitted. The same applies to the following embodiments. The space sharing device 10X differs from the first embodiment in that it includes a person detection unit 24X and in the processing of an output control unit 27X.

[0090] In this embodiment, when calling a person via the information terminal device 30 in the other space, restrictions on output of video or audio captured in the other space are controlled depending on whether or not there is a call with the other person in the other space.

[0091] The person detection unit 24X detects a person from the video acquired by the information acquisition unit 21. The person detection unit 24X detects the whole body, upper body, face, etc. of a person from the video acquired by the video acquisition unit 211. The person detection unit 24X detects a person who is facing the display device 17, that is, a person facing the display device 17, in the vicinity of the display device 17. The method by which the person detection unit 24X detects a person or face from the video can be a known method and is not limited. In other words, the person detection unit 24X detects a person in front of the display device 17 to use the space sharing device 10X. The person detection unit 24X transmits or shares the result of detecting a person to the space sharing device 10X in another space.

[0092] When a person facing the display device 17 is not detected based on the detection result of the person detection unit 24X, the output control unit 27X restricts the output of the video captured by the imaging device 11 or the audio collected, and outputs the video captured by the imaging device 31 of the information terminal device 30 and the audio to the display device 17. More specifically, when a person facing the display device 17 in the space sharing device 10X in the other space, that is, a person making a call using the space sharing device 10X in the other space, is not detected based on the detection result of the person detection unit 24X in the space sharing device 10X in the other space, the output control unit 27X restricts the output of the video captured by the imaging device 11 in the space sharing device 10X in the other space and outputs the video captured by the third imaging device 31 and the audio collected to the display device 17 arranged in the other space.

[0093] When a person facing the display device 17 is detected based on the detection result of the person detection unit 24X, the output control unit 27X outputs, to the display device 17, the video captured and the audio collected by the imaging device 31 of the information terminal device 30 in addition to the video captured and the audio collected by the imaging device 11. More specifically, when a person facing the display device 17 in the space sharing device 10X in the other space, that is, a person making a call using the space sharing device 10X in the other space, is detected, the output control unit 27X outputs, to the display device 17, the video captured and the audio collected by the third imaging device 31 in addition to the video captured and the audio collected by the imaging device 11 in the space sharing device 10X in the other space.

[0094] The limitation of the video output by the output control unit 27X includes, for example, stopping the video output (turning off the video display), reducing the brightness and saturation of the video, blurring the video, and the like.

[0095] The restriction on the audio output by the output control unit 27X includes, for example, stopping the audio output (turning the audio OFF), lowering the volume, and the like.

[0096] Specifically, for example, when a person facing the first display device 17A is not detected near the first display device 17A based on the detection result of the person detection unit 24X of the first space sharing device 10X arranged in the first space A, the output control unit 27X of the second space sharing device 10B arranged in the second space B limits the output of the image captured by the first imaging device 11A or the sound collected, and outputs the image captured by the third imaging device 31 and the sound collected to the second display device 17B. When a person facing the first display device 17A is detected near the first display device 17A, the output control unit 27X of the second space sharing device 10B outputs the image captured by the third imaging device 31 and the sound collected to the second display device 17B in addition to the image captured by the first imaging device 11A and the sound collected.

[0097] Next, a space sharing method will be described with reference to Fig. 11. The processes in steps S201 to S205 and S213 in the flowchart shown in Fig. 11 are similar to those in steps S101 to S105 and S109 in the flowchart shown in Fig. 7.

[0098] The space sharing control device 20X judges whether or not a person facing the display device 17 in the space sharing control device 20X of the other space exists (step S206). More specifically, the space sharing control device 20X acquires a detection result of a person facing the display device 17 in the other space by the person detection unit 24X in the space sharing control device 20X of the other space. The person facing the display device 17 in the other space may be a caller. When the space sharing control device 20X judges that a person facing the display device 17 in the other space exists (Yes in step S206), the process proceeds to step S207. When the space sharing control device 20X does not judge that a person facing the display device 17 in the other space exists (No in step S206), the process proceeds to step S210.

[0099] When it is determined that there is a person facing the display device 17 in the other space (Yes in step S206), the space sharing control device 20X continues outputting the video and audio received from the space sharing control device 20X in the other space, and starts outputting the video and audio received from the information terminal device 30 (step S207). In other words, when a call is being made with the space sharing control device 20X in the other space, the space sharing control device 20X continues the call, and starts a call with the information terminal device 30 that has responded to the call. The space sharing control device 20X proceeds to step S208.

[0100] The space sharing control device 20X determines whether to end the call (step S208). More specifically, the space sharing control device 20X determines to end the call when, for example, the operation receiving unit 22 receives an operation to end the call, or when the information terminal device 30 that responded receives an operation to end the call. When the space sharing control device 20X determines to end the call (Yes in step S208), it proceeds to step S209. When the space sharing control device 20X does not determine to end the call (No in step S208), it executes the process of step S208 again.

[0101] When it is determined that the call is to be ended (Yes in step S208), the space sharing control device 20X continues output of the video and audio received from the space sharing control device 20X in the other space, and ends output of the video and audio received from the information terminal device 30 (step S209). In other words, the space sharing control device 20X continues space sharing with the space sharing control device 20X in the other space, and stops space sharing with the information terminal device 30 that responded to the call, by the output control unit 27X.

[0102] When it is not determined that there is a person facing the display device 17 in the other space (No in step S206), the space sharing control device 20X restricts the output of the video or audio received from the space sharing control device 20X in the other space, and starts the output of the video and audio received from the information terminal device 30 (step S210). In other words, when a call with the space sharing control device 20X in the other space is not being made, the space sharing control device 20X restricts the output of the video or audio received from the space sharing control device 20 in the other space by the output control unit 27X, and starts a call with the information terminal device 30 that has responded to the call. The space sharing control device 20X proceeds to step S211.

[0103] The space sharing control device 20X determines whether to end the call (step S211). More specifically, the space sharing control device 20X determines to end the call when, for example, the operation receiving unit 22 receives an operation to end the call, or when the information terminal device 30 that responded receives an operation to end the call. When the space sharing control device 20X determines to end the call (Yes in step S211), it proceeds to step S212. When the space sharing control device 20X does not determine to end the call (No in step S211), it executes the process of step S211 again.

[0104] When it is determined that the call is to be ended (Yes in step S211), the space sharing control device 20X releases the restriction on the output of the video or audio received from the space sharing control device 20X in the other space, and ends the output of the video and audio received from the information terminal device 30 (step S212). In other words, the space sharing control device 20X releases the restriction on space sharing with the space sharing control device 20X in the other space, by the output control unit 27X, and stops space sharing with the information terminal device 30 that responded to the call.

[0105] Fig. 12 is an example of a display screen of the second display device 17B when it is determined in step S206 that there is a person facing the display device 17 in the other space. In Fig. 12, a person ua is detected as a person facing the first display device 17A. The second display device 17B displays the image captured by the imaging device 11 of the space sharing control device 20X in the space A, which is the other space, as well as the image of the information terminal device 30A that has been called. 14 At this time, the second display device 17B displays the image captured by the imaging device 31 of the space sharing control device 20X in the space A and the collected audio and the image captured by the imaging device 11 of the information terminal device 30A. 14 12, the image captured by the imaging device 31 of the information terminal device 30 to which the call was made is preferably displayed at a position that avoids the range of the detected person, in other words, at a position that does not overlap the range of the detected person, or at a position that does not overlap too much with the range of the detected person. 14 A window W of an image captured by the imaging device 31 is displayed in a position that does not overlap with the range of the detected person ua.

[0106] In addition, in the case where it is not determined in step S206 that there is a person facing the display device 17 in the space, the display screen of the second display device 17B is displayed as shown in Fig. 9. At this time, the second display device 17B is restricted in outputting the image captured by the imaging device 11 of the space sharing control device 20X in the space A or the sound collected, and the information terminal device 30A 14 The image captured by the imaging device 31 and the sound collected are output.

[0107] (effect) As described above, in this embodiment, when a person facing the first display device 17A is not detected near the first display device 17A, the output of the image captured by the first imaging device 11A or the audio collected is limited, and the image captured by the third imaging device 31 and the audio collected are output to the second display device 11B. In this embodiment, when a person facing the first display device 17A is detected near the first display device 17A, the image captured by the third imaging device 31 and the audio collected are output to the second display device 17B in addition to the image captured by the first imaging device 11A and the audio collected. According to this embodiment, the output of the image and audio on the other space side can be limited according to the detection result of the caller in the space of the called party.

[0108] [Third embodiment] A space sharing control device 20X according to the third embodiment will be described with reference to Figs. 13 to 15. Fig. 13 is a flowchart showing a space sharing method according to the third embodiment. Fig. 14 is a diagram showing an example of a display screen by the space sharing device according to the third embodiment. Fig. 15 is a diagram showing an example of a display screen by the space sharing device according to the third embodiment. In this embodiment, the processing of a person detection unit 24X and an output control unit 27X differs from that of the second embodiment.

[0109] The person detection unit 24X detects a person and the position of the person's face from the video acquired by the information acquisition unit 21. More specifically, for example, when a person ub present in the second space B calls a person u14 in the first space A, the person detection unit 24X detects the face of the person ub from the camera video of the second imaging device 11B on the second space B side, and calculates the position of the face of the person ub relative to the second display device 17B. The person detection unit 24X may detect the face of the person from the video captured by the third imaging device 31 and received by the communication control unit 29.

[0110] The person detection unit 24X may further detect a parameter indicating the size of the person's face, such as the width of the person's face, from the video acquired by the information acquisition unit 21. More specifically, for example, when the person ub present in the second space B calls the person u14 in the first space A, the person detection unit 24X may detect a parameter indicating the size of the person's face, such as the width of the person's face, from the camera video of the second imaging device 11B on the second space B side. In addition, the person detection unit 24X may detect a parameter indicating the size of the person's face, such as the width of the person's face, from the video imaged by the third imaging device 31 received by the communication control unit 29.

[0111] The output control unit 27X outputs the video captured by the third imaging device 31 acquired by the information acquisition unit 21 to the display device 17 in accordance with the position of the face of the person detected from the video captured by the imaging device 11. The output control unit 27X causes the image captured by the third imaging device 31 to be displayed in accordance with the position of the face of the person ub using the space sharing device 10 on the display device 17.

[0112] More specifically, for example, the output control unit 27X of the second space sharing device 10B arranged in the second space B outputs the image captured by the third imaging device 31 present in the first space received by the communication control unit 29 to the second display device 17B, aligned with the position of the face of a person detected from the image captured by the second imaging device 11B, in this case person ub, the user of the space sharing device 10B.

[0113] In addition, when there is a user of the first space sharing device 10A placed in the first space A, the output control unit 27X of the first space sharing device 10A placed in the first space A may output the image captured by the third imaging device 31 received by the communication control unit 29 to the first display device 17A, aligned with the position of the person's face detected from the image captured by the first imaging device 11A.

[0114] The output control unit 27X may adjust the size of the person's face relative to the display device 17 according to the size of the face, such as the width of the face of the person detected by the person detection unit 24X, and display the adjusted size on the display unit 171. The adjustment of the image captured by the third imaging device 31 to be displayed on the display unit 171 may be performed by any process such as cropping the image or enlarging the display size. By performing such a process, the sense of distance between the person using the space sharing device 10X in another space and the person using the information terminal device 30 is matched, and the awkwardness of sharing the space can be reduced.

[0115] Next, a space sharing method will be described with reference to Fig. 13. The processes in steps S301 to S305 and steps S308 to S310 in the flowchart shown in Fig. 13 are similar to those in steps S101 to S105 and steps S106 to S109 in the flowchart shown in Fig. 7.

[0116] When it is determined that there is a response (Yes in step S305), the space sharing control device 20X detects the position of the person and the face (step S306). More specifically, the space sharing control device 20X detects the person facing the display device 17 and the position of the person's face from the video acquired by the information acquisition unit 21 using the person detection unit 24X. The detected face position here is the position of the face in the up-down direction. The space sharing control device 20X proceeds to step S307.

[0117] The space sharing control device 20X starts outputting the image captured by the imaging device 31 of the information terminal device 30 and the collected sound (step S307). More specifically, the space sharing control device 20X outputs the image captured by the imaging device 31 of the called information terminal device 30 and the collected sound received by the communication control device 29 to the display device 17 by the output control unit 27X. The space sharing control device 20X outputs the image captured by the imaging device 31 of the called information terminal device 30 to the display device 17 by the output control unit 27X in accordance with the position of the face of the person detected from the image captured by the imaging device 11. The space sharing control device 20X proceeds to step S308. Displaying in accordance with the position of the face of the person detected from the image captured by the imaging device 11 means that the position of the image captured by the imaging device 31 of the called information terminal device 30 in the height direction on the display device 17 is set to a position equivalent to the position of the face of the person detected from the image captured by the imaging device 11 in the height direction.

[0118] 14 and 15, the second display device 17B in the second space B displays the first space A. The dashed line L in the figure indicates the position of the face of person ub who exists in the second space B. FIG. 14 shows a state in which person ub has called person u14 who exists in the first space A. FIG. 15 shows a state in which person ub has called person u14 while talking to person ua who exists in the first space A.

[0119] (effect) As described above, in this embodiment, the image captured by the third imaging device 31 is output to the second display device 17B in accordance with the position of the face of the person detected from the image captured by the second imaging device 11B. According to this embodiment, the position of the face of the called person can be displayed in accordance with the position of the caller's face on the display device of the calling party. According to this embodiment, it is possible to have a conversation with a person in a distant space in a state similar to a face-to-face conversation in real space.

[0120] In this embodiment, by adjusting the width of the face, the people in the first space A and the people in the second space B can converse with the same sense of distance, that is, with the same face size.

[0121] [Fourth embodiment] A space sharing control device 20X according to the fourth embodiment will be described with reference to Figs. 16 to 18. Fig. 16 is a schematic diagram showing another example of installation of the space sharing device, and is a diagram showing the device installed in the first space. Fig. 17 is a diagram showing an example of a display screen of the space sharing device according to the fourth embodiment. Fig. 18 is a flowchart showing a space sharing method according to the fourth embodiment. In this embodiment, the processing of the identification unit 23 is different from that in the first embodiment.

[0122] In this embodiment, it is possible to call a person who exists in a place other than the space where the space sharing device 10 is placed, such as another floor, another business, or at home. A person who exists in a place other than the space where the space sharing device 10 is placed is assumed to be associated with a fixed seat in the space where the space sharing device 10 is placed. For this reason, a range centered on the seat displayed on the display screen of the display device 17 is set as a person corresponding range of the person associated with that seat. When using the space sharing device 10 to call a person who does not exist in the range captured by the imaging device 11 of another space sharing device 10 in another space, in other words, a person who exists in a place other than the other space, the call becomes possible by specifying the person corresponding range on the display screen of the display device 17.

[0123] The identification unit 23 identifies the information terminal device 30 used by a person associated with the operation position on the display screen of the display device 17 accepted by the operation acceptance unit 22. The identification unit 23 identifies the information terminal device 30 used by a person corresponding to the person corresponding range that is the operation position on the display screen of the display device 17 accepted by the operation acceptance unit 22. More specifically, for example, the identification unit 23 of the first space sharing device 10A arranged in the first space A identifies the information terminal device 30 used by a person in the person corresponding range that corresponds to the operation position on the display screen of the first display device 17A accepted by the operation acceptance unit 22. For example, the identification unit 23 of the second space sharing device 10B arranged in the second space B identifies the information terminal device 30 used by a person in the person corresponding range that corresponds to the operation position on the display screen of the second display device 17B accepted by the operation acceptance unit 22.

[0124] In FIG. 16 and FIG. 17, the person u11 is working in a place other than the first space A, such as another floor, another business, or at home. The person u11 is not present in the first space A, and the information terminal device 30 used by the person u11 is not placed either. In FIG. 17, the position indicated by the dashed line is registered as a fixed seat, and no person is present at that seat. In FIG. 17, the person u11 who is not present in the first space A may be displayed on the fixed seat of the person u11 indicated by the dashed line, so that it is clear that the person u11 is not present in the first space A. For example, an avatar or the like indicating the person u11 may be displayed on the fixed seat of the person u11. In this state, by performing a touch operation on the dashed circular range on the display screen of the second display device 17B, the information terminal device 30 of the person u11 associated with that position is called up. By performing a touch operation on the dashed circular range on the display screen of the second display device 17B, the person u11 can be called up if the person u11 is working in another place and logged in to the information terminal device 30. When the person u11 is not logged in to the information terminal device 30 because he / she is on holiday or on the move, a call is not made to the information terminal device 30 of the person u11, and a message indicating that the call is not possible is displayed.

[0125] Next, a space sharing method will be described with reference to Fig. 18. The processes in steps S401, S404 to S409 in the flowchart shown in Fig. 18 are similar to those in steps S101, S104 to S109 in the flowchart shown in Fig. 7.

[0126] When it is determined that an operation has been performed on the display screen of the display device 17 of the space sharing device 10 (Yes in step S401), the space sharing control device 20 determines whether or not the operation is a designation operation for the person corresponding range of the other space being displayed (step S402). More specifically, the space sharing control device 20 determines whether or not a designation operation for designating the person corresponding range of the other space being displayed has been received on the display screen of the display device 17 by the operation receiving unit 22. When the space sharing control device 20 determines that the operation is a designation operation for the person corresponding range of the other space being displayed (Yes in step S402), it proceeds to step S403. When the space sharing control device 20 does not determine that the operation is a designation operation for the person corresponding range of the other space being displayed (No in step S402), it ends the processing of this flowchart.

[0127] When it is determined that the operation is a designation operation for the person corresponding range of the other space being displayed (Yes in step S402), the space sharing control device 20 determines whether or not the information terminal device 30 used by the person corresponding to the designated range can be called (step S403). More specifically, the space sharing control device 20 uses the identification unit 23 to identify the information terminal device 30 used by the person corresponding to the designation operation of the person corresponding range. The space sharing control device 20 uses the identification unit 23 to determine whether or not the identified information terminal device 30 can be called from the operation state information of the information terminal device 30 stored in the storage unit 14, or the attendance / non-attendance information of each account user and the operation state of the application. When the space sharing control device 20 determines that the information terminal device 30 used by the person corresponding to the designated range can be called (Yes in step S403), the process proceeds to step S404. When the space sharing control device 20 does not determine that the information terminal device 30 used by the person corresponding to the designated range can be called (No in step S403), the process proceeds to step S409.

[0128] (effect) As described above, in this embodiment, it is possible to identify and call an information terminal device 30 that is associated with an operation position on the display screen of the display device and is used by a person who does not exist in another space. According to this embodiment, it is possible to call a person who does not exist in another space by using the space sharing device 10.

[0129] [Fifth embodiment] A space sharing control device 20X according to the fifth embodiment will be described with reference to Fig. 19. Fig. 19 is a flowchart showing a space sharing method according to the fifth embodiment. In this embodiment, the process in the specification unit 23 is different from that in the second embodiment.

[0130] When a person is detected at the operation position on the display screen of the second display device 17B accepted by the operation accepting unit 22, the identifying unit 23 identifies the information terminal device 30 used by the detected person, and when a person is not detected at the operation position, the identifying unit 23 identifies the information terminal device 30 used by the person associated with the operation position.

[0131] Next, the space sharing method will be described with reference to Fig. 19. The processes of steps S501, S504, S505, S507, S511, and S512 in the flowchart shown in Fig. 19 are similar to those of steps S201, S204, S205, S208, and S213 in the flowchart shown in Fig. 11. The processes of steps S506 and S508 in the flowchart shown in Fig. 19 are similar to those of steps S106 and S108 in the flowchart shown in Fig. 7. The processes of steps S509 and S510 in the flowchart shown in Fig. 19 are similar to those of steps S402 and S403 in the flowchart shown in Fig. 18.

[0132] The space sharing control device 20X determines whether or not a person has been detected within the operation range (step S502). If the space sharing control device 20X determines that a person has been detected within the operation range (Yes in step S502), the process proceeds to step S503. If the space sharing control device 20X does not determine that a person has been detected within the operation range (No in step S502), the process proceeds to step S509.

[0133] The space sharing control device 20X judges whether the information terminal device 30 associated with the person can be called (step S503). More specifically, the space sharing control device 20X uses the identification unit 23 to identify the information terminal device 30 corresponding to the person designation operation. The space sharing control device 20X uses the identification unit 23 to judge whether the identified information terminal device 30 can be called from the operation state information of the information terminal device 30 stored in the storage unit 14, or the attendance / non-attendance information of each account user and the operation state of the application. When the space sharing control device 20X judges that the information terminal device 30 associated with the person can be called (Yes in step S503), the process proceeds to step S504. When the space sharing control device 20 does not judge that the information terminal device 30 associated with the person can be called (No in step S503), the process proceeds to step S512.

[0134] (effect) As described above, in this embodiment, when a person is detected at the operation position on the display screen of the display device 17, the information terminal device 30 used by the detected person can be identified, and when a person is not detected at the operation position, the information terminal device 30 used by the person associated with the operation position can be identified. According to this embodiment, a person who is not in the space can be called by using the space sharing device 10.

[0135] In the above description, the space sharing control device 20 has been described as being disposed in the first space A and the second space B, but in the case of centralized control of the imaging devices 11 and the display devices 17 installed in the first space A and the second space B, the space sharing control device 20 has the functions of the space sharing control device 20 disposed in the first space A described above and the functions of the space sharing control device 20 disposed in the second space B. In this case, the space sharing control device 20 is as follows.

[0136] The information acquisition unit 21 acquires the video captured by the first imaging device 11A and the second imaging device 11B and the audio collected. The output control unit 27 outputs the video captured by the first imaging device 11A and the audio collected by the second imaging device 11B to the second display device 17B, and outputs the video captured by the second imaging device 11B and the audio collected by the first display device 17A. The operation reception unit 22 accepts an operation on the display screen of the second display device 17B. The identification unit 23 identifies an information terminal device 30 that exists in the range captured by the first imaging device 11A corresponding to the operation position on the display screen of the second display device 17B accepted by the operation reception unit 22. The information acquisition unit 21 acquires the video captured by the third imaging device 31 provided in the information terminal device 30 identified by the identification unit 23. When the information terminal device 30 is identified by the identification unit 23, the output control unit 27 outputs the video captured by the third imaging device 31 and the audio collected by the second display device 17B. [Explanation of symbols]

[0137] 1. Space sharing system 10 Space sharing device 10A First space sharing device 10B Second space sharing device 11 Imaging device 11A First Imaging Device 11B Second imaging device 111 Camera 112 Microphone 13 Control section 17 Display device 17A First display device 17B Second display device 171 Display section 172 Audio output section 19 Communications Department 20 Control unit (space sharing control device) 21 Information Acquisition Department 211 Video Acquisition Unit 212 Voice Acquisition Unit 22 Operation reception section 23 Specific section 27 Output control section 271 Display control section 272 Audio output control unit 29 Communication control section 30 Information terminal device 31 Imaging device (third imaging device) 311 Camera 312 Microphone 33 Operation section 37 Display section 38 Audio output section 39 Communications Department 40 Control unit (information terminal control device) 41 Information Acquisition Department 411 Video Acquisition Unit 412 Voice Acquisition Unit 42 Operation reception section 47 Output control section 471 Display control unit 472 Audio output control unit 49 Communication control section A first space B Second space

Claims

1. A space sharing system including a first imaging device and a first display device disposed in a first space, and a second imaging device and a second display device disposed in a second space, an information acquisition unit that acquires images captured by the first imaging device and the second imaging device and sounds collected by the first imaging device and the second imaging device; an output control unit that outputs the image captured by the first imaging device and the sound collected by the second imaging device to the second display device, and outputs the image captured by the second imaging device and the sound collected by the first display device; an operation receiving unit that receives an operation on a display screen of the second display device; an identification unit that identifies an information terminal device corresponding to the operation position on the display screen of the second display device accepted by the operation acceptance unit; Equipped with The information acquisition unit acquires video captured by a third imaging device included in the information terminal device identified by the identification unit, and audio collected by the third imaging device, The output control unit outputs, to the second display device, a video image captured by the third imaging device included in the information terminal device identified by the identification unit. Space sharing system.

2. the identification unit identifies an information terminal device present in a range captured by the first imaging device, the information terminal device corresponding to the operation position on the display screen of the second display device accepted by the operation acceptance unit; The space sharing system according to claim 1 .

3. the identification unit identifies an information terminal device used by a person associated with the operation position on the display screen of the second display device accepted by the operation acceptance unit. The space sharing system according to claim 1 .

4. a person detection unit that detects a person from the video image acquired by the information acquisition unit; Further equipped with the identification unit, when a person is detected at an operation position on the display screen of the second display device accepted by the operation acceptance unit, identifies an information terminal device used by the detected person, and, when a person is not detected at the operation position, identifies an information terminal device used by the person associated with the operation position. The space sharing system according to claim 1 .

5. a person detection unit that detects a person from the video image acquired by the information acquisition unit; Further equipped with When a person facing the first display device is not detected near the first display device based on the detection result of the person detection unit, the output control unit limits output of the video captured by the first imaging device or the audio picked up by the first imaging device and outputs the video captured by the third imaging device and the audio picked up to the second display device, and when a person facing the first display device is detected near the first display device, the output control unit outputs the video captured by the third imaging device and the audio picked up to the second display device in addition to the video captured by the first imaging device and the audio picked up to the second display device. The space sharing system according to any one of claims 1 to 3.

6. a person detection unit that detects a person and a position of the person's face from the video image acquired by the information acquisition unit; Further equipped with The output control unit outputs, to the second display device, the image captured by the third imaging device, which is acquired by the information acquisition unit, in accordance with a position of a face of a person detected from the image captured by the second imaging device. The space sharing system according to any one of claims 1 to 3.

7. A space sharing method for a space sharing system including a first imaging device and a first display device arranged in a first space, and a second imaging device and a second display device arranged in a second space, comprising: A step of acquiring images captured by the first imaging device and the second imaging device and sounds collected by the first imaging device and the second imaging device; A step of outputting the image captured by the first imaging device and the sound collected by the second imaging device to a second display device, and outputting the image captured by the second imaging device and the sound collected by the first display device; determining that an operation on a display screen of the second display device has been accepted; when an operation on the display screen of the second display device is accepted, identifying an information terminal device corresponding to an operation position on the display screen of the second display device where the operation is accepted; acquiring a video image captured by a third imaging device included in the identified information terminal device and a sound captured by the third imaging device; outputting the image captured by the third imaging device and the sound collected by the third imaging device to the second display device; The space sharing system executes the space sharing method.

8. A step of acquiring images captured by the first imaging device and the second imaging device and sounds collected by the first imaging device and the second imaging device; A step of outputting the image captured by the first imaging device and the sound collected by the second imaging device to a second display device, and outputting the image captured by the second imaging device and the sound collected by the first display device; determining that an operation on a display screen of the second display device has been accepted; when an operation on the display screen of the second display device is accepted, identifying an information terminal device corresponding to an operation position on the display screen of the second display device where the operation is accepted; acquiring a video image captured by a third imaging device included in the identified information terminal device and a sound captured by the third imaging device; outputting the image captured by the third imaging device and the sound collected by the third imaging device to the second display device; A program that runs on a computer that controls a space sharing system.

Images