Space sharing system, space sharing method, and program

Abstract

A space sharing system 1 including a first imaging device 11A and a first display device 17A and a second imaging device 11B and a second display device 17B arranged in different spaces. The space sharing system 1 comprises: an information acquisition unit 21 for acquiring video and voice from the first imaging device 11A and the second imaging device 11B; an output control unit 27 for outputting the video and voice acquired from the first imaging device 11A to the second display device 17B, and outputting the video and voice acquired from the second imaging device 11B to the first display device 17A; an operation reception unit 22 for receiving an operation on the display screen of the second display device 17B; and an identification unit 23 for identifying an information terminal device corresponding to the operation position on the display screen. The information acquisition unit 21 acquires the video and voice acquired from a third imaging device 31 included in the identified information terminal device, and the output control unit 27 outputs the video and voice acquired from the third imaging device 31 to the second display device 17B.

Description

Space sharing system, space sharing method and program

[0001] The present invention relates to a space sharing system, a space sharing method, and a program.

[0002] 2. Description of the Related Art A system is known in which a virtual space is shared between geographically separated spaces using an image capture device and a display device (see, for example, Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2004-056161

[0004] Such a space-sharing system is installed, for example, in different offices within the same company. The use of a space-sharing system has the effect of facilitating mutual communication. However, when using a space-sharing system to call a person in another space, it is necessary to set up a call using the space-sharing system in advance or to call the person by outputting a voice message via a display device. Furthermore, since communication is performed via a display device provided in the space-sharing system, the person being called must move in front of the display device.

[0005] The present disclosure aims to enable smoother communication by sharing a space.

[0006] In order to solve the above-mentioned problems and achieve the object, the space-sharing system of the present disclosure is a space-sharing system comprising a first imaging device and a first display device arranged in a first space, and a second imaging device and a second display device arranged in a second space, and comprising: an information acquisition unit that acquires video captured and audio picked up by the first imaging device and the second imaging device; an output control unit that outputs the video captured and audio picked up by the first imaging device to the second display device and outputs the video captured and audio picked up by the second imaging device to the first display device; an operation reception unit that accepts operations on a display screen of the second display device; and an identification unit that identifies an information terminal device corresponding to the operation position on the display screen of the second display device accepted by the operation reception unit, wherein the information acquisition unit acquires video captured and audio picked up by a third imaging device provided in the information terminal device identified by the identification unit, and the output control unit outputs the video captured and audio picked up by the third imaging device provided in the information terminal device identified by the identification unit to the second display device.

[0007] The space-sharing method according to the present disclosure is a space-sharing method for a space-sharing system including a first imaging device and a first display device arranged in a first space, and a second imaging device and a second display device arranged in a second space, wherein the space-sharing system executes the following steps: acquiring video captured and audio picked up by the first imaging device and the second imaging device; outputting the video captured and audio picked up by the first imaging device to the second display device, and outputting the video captured and audio picked up by the second imaging device to the first display device; determining that an operation on the display screen of the second display device has been accepted; if an operation on the display screen of the second display device has been accepted, identifying an information terminal device corresponding to the operation position on the display screen of the second display device where the operation was accepted; acquiring video captured and audio picked up by a third imaging device provided in the identified information terminal device; and outputting the video captured and audio picked up by the third imaging device to the second display device.

[0008] The program of the present disclosure causes a computer that controls a space-sharing system to perform the following steps: acquiring video captured and audio picked up by a first imaging device and a second imaging device; outputting the video captured and audio picked up by the first imaging device to a second display device and outputting the video captured and audio picked up by the second imaging device to the first display device; determining that an operation on the display screen of the second display device has been accepted; if an operation on the display screen of the second display device has been accepted, identifying an information terminal device that corresponds to the operation position on the display screen of the second display device where the operation was accepted; acquiring video captured and audio picked up by a third imaging device provided in the identified information terminal device; and outputting the video captured and audio picked up by the third imaging device to the second display device.

[0009] According to the present disclosure, communication can be carried out more smoothly by sharing a space.

[0010] FIG. 1 is a schematic diagram showing an example of the configuration of a space sharing system according to a first embodiment. FIG. 2 is a schematic diagram showing an example of an installation of a space sharing device, where the device is installed in a first space. FIG. 3 is a schematic diagram showing an example of an installation of a space sharing device, where the device is installed in a first space. FIG. 4 is a schematic diagram showing an example of an installation of a space sharing device, where the device is installed in a second space. FIG. 5 is a block diagram showing an example of the configuration of a space sharing device according to the first embodiment. FIG. 6 is a block diagram showing an example of the configuration of an information terminal device according to the first embodiment. FIG. 7 is a flowchart showing a space sharing method according to the first embodiment. FIG. 8 is a diagram showing an example of a display screen displayed by the space sharing device according to the first embodiment. FIG. 9 is a diagram showing another example of a display screen displayed by the space sharing device according to the first embodiment. FIG. 10 is a block diagram showing an example of the configuration of a space sharing device according to a second embodiment. FIG. 11 is a flowchart showing a space sharing method according to the second embodiment. FIG. 12 is a diagram showing an example of a display screen displayed by the space sharing device according to the second embodiment. FIG. 13 is a flowchart showing a space sharing method according to a third embodiment. FIG. 14 is a diagram showing an example of a display screen displayed by the space sharing device according to the third embodiment. FIG. 15 is a diagram showing an example of a display screen displayed by the space sharing device according to the third embodiment. Fig. 16 is a schematic diagram showing another example of installation of the space sharing device, and is a diagram showing installation in a first space. Fig. 17 is a diagram showing an example of a display screen of the space sharing device according to the fourth embodiment. Fig. 18 is a flowchart showing a space sharing method according to the fourth embodiment. Fig. 19 is a flowchart showing a space sharing method according to the fifth embodiment.

[0011] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A space sharing system, a space sharing method, and a program according to the present invention will be described in detail below with reference to the accompanying drawings. However, the present invention is not limited to the following embodiments.

[0012] [First embodiment] (Space sharing system) Fig. 1 is a schematic diagram showing an example of the configuration of a space sharing system according to a first embodiment. The space sharing system 1 constantly shares video and audio in real time in multiple real spaces. When applied to, for example, remote offices, the space sharing system 1 gives the impression of sharing the office in a pseudo-like manner.

[0013] The space sharing system 1 includes a space sharing device 10 arranged in each of a plurality of spaces, and at least one information terminal device 30 arranged in each space. The number of spaces shared by the space sharing system 1 is not limited.

[0014] In the following description, as an example, the space sharing system 1 will be described as sharing two spaces, a first space A and a second space B. A first space sharing device 10A is arranged in the first space A. An information terminal device 30A is also arranged in the first space A. １ , information terminal device 30A ２ The information terminal device 30A is arranged. １ , information terminal device 30A ２ is connected to the first space sharing device 10A and the second space sharing device 10B so as to be able to communicate data with each other. The second space sharing device 10B is disposed in the second space B. The information terminal device 30B is disposed in the second space B. １ , information terminal device 30B ２ The information terminal device 30B is arranged. １ , information terminal device 30B ２ are connected to the first space sharing device 10A and the second space sharing device 10B so as to be able to communicate data with each other. Specifically, the first space sharing device 10A, the second space sharing device 10B, the information terminal device 30A １ , information terminal device 30A ２ , information terminal device 30B １ , information terminal device 30B ２ are connected via an in-house network or the like. When there is no particular need to distinguish between the first space sharing device 10A and the second space sharing device 10B, they will be described as space sharing devices 10. When there is no particular need to distinguish between the information terminal devices 30, they will be described as information terminal devices 30. Similarly, in the description of each component, when there is no particular need to distinguish between the components provided in the first space sharing device 10A and the components provided in the second space sharing device 10B, the reference numerals will not be appended with A or B, and when there is a need to distinguish between the components provided in the first space sharing device 10A and the components provided in the second space sharing device 10B, the reference numerals will be appended with A or B.

[0015] An example of installation of the first space sharing device 10A and the information terminal device 30A in the first space A will be described with reference to Fig. 2. Fig. 2 is a schematic diagram showing an example of installation of the space sharing device, for example, in a first space such as an office. The first space sharing device 10A is preferably placed in a position where it can capture an image of the entire space A and the range of departments where space sharing is useful. In the first space A, the information terminal device 30A １ or information terminal device 30A １４ The person ua faces the first display device 17A. The person u1, the person u3 to the person u6, the person u9, the person u10, the person u12 to the person u14 are located in front of the information terminal device 30A. １ , information terminal device 30A ３ or information terminal device 30A ６ , information terminal device 30A ９ , information terminal device 30A １０ , information terminal device 30A １２ or information terminal device 30A １４ , respectively. In other words, the person ua is a person who is using the space sharing system 1. Furthermore, the person u1, the person u3 to the person u6, the person u9, the person u10, the person u12 to the person u14 are using the information terminal device 30A. １ , information terminal device 30A ３ or information terminal device 30A ６ , information terminal device 30A ９ , information terminal device 30A １０ , information terminal device 30A １２ or information terminal device 30A １４ Each person logs in to and uses the information terminal device 30 using their own account information. In other words, each information communication terminal 30 is linked to the person using the information terminal device 30.

[0016] Space sharing in a first space A and a second space B will be described using Figures 3 and 4. Figure 3 is a schematic diagram showing an example of installation of a space-sharing device, installed in the first space. Figure 4 is a schematic diagram showing an example of installation of a space-sharing device, installed in the second space. Figures 3 and 4 also show a state in which communication is taking place using the space-sharing system 1. As shown in Figure 3, a person ua present in the first space A is using the space-sharing system 1 while facing the first display device 17A. The first space-sharing device 10A arranged in the first space A outputs video and audio of the second space B. When a person ub faces the second display device 17B, the first space-sharing device 10A displays an image of the person ub located in the second space B at approximately life-size. As shown in Figure 4, a person ub present in the second space B is using the space-sharing system 1 while facing the second display device 17B. The second space-sharing device 10B arranged in the second space B outputs video and audio of the first space A. When person ua faces the first display device 17A, the second space sharing device 10B displays an image of person ua located in the first space A at approximately life-size. In this way, a pseudo-space is shared between the first space A and the second space B. Person ua present in the first space A and person ub present in the second space B communicate using the space sharing system 1. Communication using the space sharing system 1 can also be referred to as a phone call.

[0017] (Space sharing device) Fig. 5 is a block diagram showing an example of the configuration of a space sharing device according to the first embodiment. The space sharing device 10 is a general-purpose server device, including a distributed type, realized by a computer, or a personal computer. The space sharing device 10 controls space sharing in different spaces. In this embodiment, the space sharing device 10 controls space sharing between a first space A and a second space B.

[0018] The space sharing device 10 may have a space sharing control device 20 (described later) that centrally controls the imaging devices 11 and the display devices 17 installed in different spaces. The space sharing device 10 may have a configuration in which the space sharing control device 20 is arranged in each space and connected to each other via a network. In this embodiment, the space sharing device 10 is described as being arranged in each of the first space A and the second space B.

[0019] The space sharing device 10 includes an imaging device 11 , an operation unit 13 , a display device 17 , a communication unit 19 , and a space sharing control device 20 .

[0020] The imaging device 11 captures images of the space in which it is placed and collects audio. More specifically, for example, a first imaging device 11A placed in a first space A captures images of the first space A and collects audio. For example, a second imaging device 11B placed in a second space B captures images of the second space B and collects audio. The imaging device 11 acquires the captured images and collected audio as information by an information acquisition unit 21. The imaging device 11 is placed in the vicinity of the display device 17. The imaging device 11 includes a camera 111 and a microphone 112.

[0021] The camera 111 captures an image of the space in which it is placed. The camera 111 is placed, for example, around the display device 17 or in front of the display device 17. The camera 111 captures an image of a person facing the display device 17 placed in the same space. More specifically, for example, the camera 111 of the first imaging device 11A placed in the first space A captures an image of the first space A. For example, the camera 111 of the second imaging device 11B placed in the second space B captures an image of the second space B. The camera 111 outputs the captured image to the image acquisition unit 211 of the space sharing control device 20.

[0022] The microphone 112 picks up sound in the space in which it is placed. The microphone 112 is placed, for example, around the display device 17 or in front of the display device 17. The microphone 112 picks up sound spoken by a person facing the display device 17 placed in the same space. More specifically, for example, the microphone 112 of the first imaging device 11A placed in the first space A picks up sound in the first space A. For example, the microphone 112 of the second imaging device 11B placed in the second space B picks up sound in the second space B. The microphone 112 outputs audio information related to the picked-up sound to the audio acquisition unit 212 of the space sharing control device 20.

[0023] The operation unit 13 is an input device capable of inputting various operations to the space sharing device 10. The operation unit 13 outputs an operation signal corresponding to the received input operation to the space sharing control device 20. The operation unit 13 includes, for example, a touch panel, buttons, switches, etc., superimposed on the display unit 171. The operation unit 13 can accept, for example, a key input operation, a click operation, or a touch operation.

[0024] In this embodiment, the operation unit 13 is a touch panel disposed on the display unit 171. The operation unit 13 can accept a touch operation on an information terminal device 30 or a person in another space that is being displayed on the display screen of the display unit 171 as a person designation operation for calling the information terminal device 30 or the person. More specifically, for example, the operation unit 13 provided on the display unit 171 of the first display device 17A disposed in the first space A can accept a person designation operation for calling an information terminal device 30 or a person present in the second space B. For example, the operation unit 13 provided on the display unit 171 of the second display device 17B disposed in the second space B can accept a person designation operation for calling an information terminal device 30 or a person present in the first space A.

[0025] A predetermined range including the range in which the information terminal device 30 or person included in the displayed video is detected may be set as the touch range selectable in the person designation operation.

[0026] The storage unit 14 stores information such as the calculation contents and programs of the space sharing device 10. The storage unit 14 is, for example, a storage device such as a RAM (Random Access Memory) or an SSD (Solid State Drive).

[0027] The memory unit 14 stores information indicating the information terminal devices 30 located within the imaging range of the imaging device 11 and the account users of each information terminal device 30. The memory unit 14 acquires and stores operation status information indicating whether the information terminal devices 30 are operating or whether each account user has arrived at work. More specifically, for example, the memory unit 14 of the first space sharing device 10A located in the first space A stores information indicating the information terminal devices 30 located within the imaging range of the first imaging device 11A and the account users of each information terminal device 30. The memory unit 14 also stores operation status information of the information terminal devices 30 in the first space A and the attendance status information of each account user. For example, the memory unit 14 of the second space sharing device 10B located in the second space B stores information indicating the information terminal devices 30 located within the imaging range of the second imaging device 11B and the account users of each information terminal device 30. The storage unit 14 also stores the operating status information of the information terminal device 30 in the second space B and the attendance / non-attendance information of each account user.

[0028] The display device 17 displays an image of the other party's space that is sharing the space and outputs audio. The display of the image and the output of the audio of the display device 17 are controlled by the output control unit 27. The display device 17 includes a display unit 171 and an audio output unit 172.

[0029] The display unit 171 displays various types of video including text and images. The display unit 171 is, for example, a display including a liquid crystal display, an organic EL (Electro-Luminescence), or the like. The display unit 171 displays a video of the other party's space that shares the space. More specifically, for example, the display unit 171 of the first display device 17A arranged in the first space A displays a video of the second space B. For example, the display unit 171 of the second display device 17B arranged in the second space B displays a video of the first space A. The display of the display unit 171 is controlled by the display control unit 271.

[0030] The audio output unit 172 is an audio output device that outputs audio from the other party's space that shares the space. More specifically, for example, the audio output unit 172 of the first display device 17A arranged in the first space A outputs audio from the second space B. For example, the audio output unit 172 of the second display device 17B arranged in the second space B outputs audio from the first space A. The audio output of the audio output unit 172 is controlled by the audio output control unit 272.

[0031] The communication unit 19 is a communication unit for performing wired communication or wireless communication. The communication unit 19 performs communication using a communication method using, for example, Wi-Fi or a mobile phone line, and is connected to an intranet or the Internet. The communication of the communication unit 19 is controlled by a communication control unit 29. The communication unit 19 communicates video and audio data with other space sharing devices 10 and information terminal devices 30.

[0032] (Space sharing control device) The space sharing control device 20 controls each part of the space sharing device 10. The space sharing control device 20 has, for example, a control device such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), and a storage device such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The space sharing control device 20 may be realized by, for example, an integrated circuit such as an ASIC or an FPGA. The space sharing control device 20 may be realized by a combination of hardware and software. The space sharing control device 20 executes a program that controls the operation of the space sharing device 10 according to the present disclosure.

[0033] The space sharing control device 20 includes an information acquisition unit 21, an operation reception unit 22, an identification unit 23, an output control unit 27, and a communication control unit 29 as functional blocks realized by circuits and executed programs.

[0034] In this embodiment, the space sharing control device 20 controls the imaging device 11 and the display device 17 arranged in each space. The space sharing control device 20 is assumed to be arranged in each of the first space A and the second space B.

[0035] The information acquisition unit 21 acquires video captured and audio collected by the imaging device 11 in the space in which the space sharing device 10 is placed. More specifically, for example, the information acquisition unit 21 of the first space sharing device 10A placed in the first space A acquires video captured and audio collected by the first imaging device 11A placed in the first space A. For example, the information acquisition unit 21 of the second space sharing device 10B placed in the second space B acquires video captured and audio collected by the second imaging device 11B placed in the second space B. The information acquisition unit 21 includes a video acquisition unit 211 and an audio acquisition unit 212.

[0036] The information acquisition unit 21 acquires, via the communication unit 19 and the communication control unit 29, the video captured and the audio collected by the imaging device (third imaging device) 31 provided in the information terminal device 30 identified by the identification unit 23.

[0037] The video acquisition unit 211 acquires video captured by the camera 111. More specifically, the video acquisition unit 211 acquires video captured by the camera 111 of people and the like present in the vicinity of the display device 17 arranged in the same space. More specifically, for example, the video acquisition unit 211 of the first space sharing device 10A arranged in the first space A acquires video captured of people and the like present in the vicinity of the first display device 17A. For example, the video acquisition unit 211 of the second space sharing device 10B arranged in the second space B acquires video captured of people and the like present in the vicinity of the second display device 17B. The video data acquired by imaging with the camera 111 is a moving image.

[0038] The audio acquisition unit 212 acquires audio detected by the microphone 112. The audio acquisition unit 212 acquires audio such as audio spoken by a person in the vicinity of the display device 17 arranged in the same space. More specifically, for example, the audio acquisition unit 212 of the first space sharing device 10A arranged in the first space A acquires audio such as audio spoken by a person present in the vicinity of the first display device 17A. For example, the audio acquisition unit 212 of the second space sharing device 10B arranged in the second space B acquires audio such as audio spoken by a person present in the vicinity of the second display device 17B.

[0039] The operation reception unit 22 acquires an operation signal corresponding to an operation on the operation unit 13 from the operation unit 13. The operation reception unit 22 outputs a control signal corresponding to the acquired operation signal to each unit.

[0040] The operation acceptance unit 22 accepts operations on the display screen of the display device 17. The operation acceptance unit 22 accepts a touch operation on an information terminal device 30 or a person in another space that is being displayed on the display screen of the display unit 171 as a person designation operation for calling the information terminal device 30 or the person. More specifically, for example, the operation acceptance unit 22 of the first space sharing device 10A arranged in the first space A accepts a person designation operation on an information terminal device 30 or a person in the second space B that is being displayed on the display unit 171 of the first display device 17A. For example, the operation acceptance unit 22 of the second space sharing device 10B arranged in the second space B accepts a person designation operation on an information terminal device 30 or a person in the first space A that is being displayed on the display unit 171 of the second display device 17B.

[0041] When a person designation operation is accepted, the identification unit 23 identifies the information terminal device 30 to be called. When a person designation operation is accepted, the identification unit 23 identifies the information terminal device 30 corresponding to the person designation operation.

[0042] A case will be described where an information terminal device 30 is specified on the display screen and a person designation operation is performed. The identification unit 23 identifies the information terminal device 30 that exists within the range captured by the imaging device 11 disposed in another space and corresponds to the operation position on the display screen of the display device 17 accepted by the operation acceptance unit 22. More specifically, for example, the identification unit 23 of the first space sharing device 10A disposed in the first space A identifies the information terminal device 30 that exists within the range captured by the second imaging device 11B and corresponds to the operation position on the display screen of the first display device 17A accepted by the operation acceptance unit 22. For example, the identification unit 23 of the second space sharing device 10B disposed in the second space B identifies the information terminal device 30 that exists within the range captured by the first imaging device 11A and corresponds to the operation position on the display screen of the second display device 17B accepted by the operation acceptance unit 22.

[0043] A case will be described in which a person designation operation is performed by specifying a person on the display screen. The identification unit 23 identifies an information terminal device 30 associated with a person present within the range captured by the imaging device 11 disposed in another space, corresponding to the operation position on the display screen of the display device 17 accepted by the operation acceptance unit 22. More specifically, for example, the identification unit 23 of the first space sharing device 10A disposed in the first space A identifies a person present within the range captured by the second imaging device 11B, corresponding to the operation position on the display screen of the first display device 17A accepted by the operation acceptance unit 22. For example, the identification unit 23 of the second space sharing device 10B disposed in the second space B identifies a person present within the range captured by the first imaging device 11A, corresponding to the operation position on the display screen of the second display device 17B accepted by the operation acceptance unit 22.

[0044] The position of each information terminal device 30 within the imaging range of the imaging device 11 is associated in advance. For example, in the case of fixed seats where work is performed at a specific seat, the coordinate range of the image and the person sitting in the seat are associated in advance and stored in the storage unit 14. The person and the logged-in information terminal device 30 are associated from the seat corresponding to the coordinate range in which the person is recognized from the image. For example, in the case of free address seating where seats can be freely selected, the position information of the information terminal device 30 is obtained from a beacon device or the like provided in each information terminal device 30. The acquired position information of the information terminal device 30 is associated with the coordinates in the image.

[0045] Furthermore, the identification unit 23 determines whether the identified information terminal device 30 can be called. The identification unit 23 determines whether the identified information terminal device 30 can be called based on the operation status information of the information terminal device 30, information on running applications, or attendance / non-attendance information of each account user stored in the storage unit 14. The identification unit 23 determines that the identified information terminal device 30 can be called when the information terminal device 30 is in an operating state, indicates that the account user is at work, and is not communicating with other users via an application for communicating online with other users, for example.

[0046] The output control unit 27 outputs the video captured and the audio collected by the imaging device 11 disposed in the other space, which are received by the communication control unit 29, to the display device 17. More specifically, for example, the output control unit 27 of the space sharing device 10A disposed in the first space A outputs the video captured and the audio collected by the second imaging device 11B, which are received by the communication control unit 29, to the first display device 17A disposed in the first space A. For example, the output control unit 27 of the space sharing device 10B disposed in the second space B outputs the video captured and the audio collected by the first imaging device 11A, which are received by the communication control unit 29, to the second display device 17B disposed in the second space B.

[0047] When the identification unit 23 identifies the information terminal device 30, the output control unit 27 outputs the video captured and the audio collected by the imaging device (third imaging device) 31 of the information terminal device 30 to the display device 17. More specifically, for example, when the identification unit 23 identifies the information terminal device 30, the output control unit 27 of the first space sharing device 10A arranged in the first space A outputs the video captured and the audio collected by the third imaging device 31 present in the second space B to the first display device 17A. For example, when the identification unit 23 identifies the information terminal device 30, the output control unit 27 of the second space sharing device 10B arranged in the second space B outputs the video captured and the audio collected by the third imaging device 31 present in the first space A to the second display device 17B.

[0048] The display control unit 271 controls the display of the image on the display unit 171. For example, the display control unit 271 controls the display unit 171 to display an image captured of another space. When a person is captured in the image captured of the other space, the display control unit 271 displays an image including the person present in the other space. More specifically, for example, the display control unit 271 of the first space sharing device 10A arranged in the first space A displays an image captured of the second space B. For example, the display control unit 271 of the second space sharing device 10B arranged in the second space B displays an image captured of the first space A. Furthermore, the display control unit 271 controls the display unit 171 to display an image captured by the imaging device 31 of the information terminal device 30 identified by the identification unit 23 on the display unit 171.

[0049] The audio output control unit 272 controls the output of audio in the audio output unit 172. For example, the audio output control unit 272 controls the audio output unit 172 to output audio collected in another space. When the audio collected in another space includes the audio of a person present in the other space, the audio output control unit 272 outputs audio including the audio of the person present in the other space. More specifically, for example, the audio output control unit 272 of the first space sharing device 10A arranged in the first space A outputs the audio collected in the second space B. For example, the audio output control unit 272 of the second space sharing device 10B arranged in the second space B outputs the audio collected in the first space A.

[0050] The communication control unit 29 controls the communication unit 19 to control transmission and reception of data including video and audio between the other space sharing device 10 and the information terminal device 30. More specifically, the communication control unit 29 controls, for example, the communication unit 19 to control communication between the other space sharing device 10 and the information terminal device 30. The communication control unit 29 controls, for example, the communication unit 19 to receive video captured in another space and audio collected from the other space sharing device 10, and to receive video captured and audio collected by the imaging device 31 of the information terminal device 30 from the information terminal device 30. The communication control unit 29 controls, for example, the communication unit 19 to transmit video captured and audio collected by the imaging device 11 to the other space sharing device 10 and the information terminal device 30.

[0051] More specifically, for example, the communication control unit 29 of the first space sharing device 10A arranged in the first space A controls the transmission of video and audio of the first space A acquired by the information acquisition unit 21 to the second display device 17B and the information terminal device 30 arranged in the second space B so that they can be output, and the reception of video captured by the second imaging device 11B and the imaging device 31 from the second space sharing device 10B and the information terminal device 30 arranged in the second space B. For example, the communication control unit 29 of the space sharing device 10B arranged in the second space B controls the transmission of video and audio of the second space B acquired by the information acquisition unit 21 to the first display device 17A and the information terminal device 30 arranged in the first space A so that they can be output, and the reception of video captured by the first imaging device 11A and the imaging device 31 from the first space sharing device 10A and the information terminal device 30 arranged in the first space A.

[0052] When the communication control unit 29 receives the person designation operation, it controls the communication unit 19 to transmit call information by voice or message to the designated information terminal device 30 .

[0053] (Information Terminal Device) FIG. 6 is a block diagram showing an example of the configuration of an information terminal device according to the first embodiment. The information terminal device 30 is used in a space shared by the space sharing system 1. In this embodiment, the information terminal device 30 is, for example, a terminal device used by a person present in the first space A and the second space B. The information terminal device 30 is, for example, an information terminal device such as a general-purpose desktop or notebook personal computer, a smartphone, or a tablet terminal used by the person. The information terminal device 30 is connected to the space sharing device 10 via a network so as to be able to communicate data therewith. More specifically, the information terminal device 30 arranged in the first space A is connected to the first space sharing device 10A arranged in the first space A via the network so as to be able to communicate data therewith. The information terminal device 30 arranged in the second space B is connected to the second space sharing device 10B arranged in the second space B via the network so as to be able to communicate data therewith.

[0054] The information terminal device 30 acquires account user information of the information terminal device 30 based on the account information of the person who is logged in to the information terminal device 30 .

[0055] The information terminal device 30 includes an imaging device 31 , an operation unit 33 , a display unit 37 , an audio output unit 38 , a communication unit 39 , and a terminal control device 40 .

[0056] The imaging device 31 captures images and collects sounds around the information terminal device 30. The imaging device 31 includes a camera 311 and a microphone 312.

[0057] The camera 311 captures an image in a direction facing the information terminal device 30. The camera 311 captures an image in a direction facing the display device 37 of the information terminal device 30. In other words, the camera 311 captures an image in a direction in which a person using the information terminal device 30 is present. The camera 311 is disposed, for example, near the display device 37. The camera 311 outputs the captured image to the image acquisition unit 411 of the terminal control device 40.

[0058] The microphone 312 picks up sounds around the information terminal device 30. The microphone 312 may also pick up sounds from a direction facing the display device 37 of the information terminal device 30. In other words, the microphone 312 picks up sounds spoken by a person using the information terminal device 30. The microphone 312 outputs audio information related to the picked-up sounds to the audio acquisition unit 412 of the terminal control device 40.

[0059] The operation unit 33 is an input device capable of inputting various operations to the information terminal device 30. The operation unit 33 outputs an operation signal corresponding to the received input operation to the terminal control device 40. The operation unit 33 includes, for example, a keyboard, a mouse, a touch panel overlaid on the display unit 37, buttons, switches, etc. The operation unit 33 can accept, for example, a key input operation, a click operation, or a touch operation.

[0060] The display unit 37 displays various types of video including text and images. The display unit 37 is, for example, a display including a liquid crystal display, an organic EL display, etc. The display unit 37 displays a video of the space of the other party sharing the space. In this embodiment, when a call is received from an information terminal device 30 present in another space, the display unit 37 displays a video captured by the imaging device 11 of the space sharing device 10 present in the other space. The display unit 37 may also display a video captured by the imaging device 11 of the space sharing device 10 present in the same space. The display on the display unit 37 is controlled by a display control unit 471.

[0061] The audio output unit 38 is an audio output device that outputs audio from the other space that shares the space. In this embodiment, when a call is made from an information terminal device 30 that exists in another space, the audio output unit 38 outputs audio picked up by the imaging device 11 of the space sharing device 10 that exists in the other space. The audio output unit 38 may also output audio picked up by the imaging device 11 of the space sharing device 10 in the same space. The audio output of the audio output unit 38 is controlled by the audio output control unit 472.

[0062] The communication unit 39 is a communication unit for performing wired communication or wireless communication. The communication unit 39 performs communication using a communication method using, for example, Wi-Fi or a mobile phone line, and is connected to an intranet or the Internet. The communication of the communication unit 39 is controlled by a communication control unit 49. The communication unit 39 communicates data such as video and audio with the space sharing device 10 arranged in the space where the information terminal device 30 exists. The communication unit 39 communicates data such as account user information and operation status information of the information terminal device 30 with the space sharing device 10 arranged in the space where the information terminal device 30 exists.

[0063] (Terminal Control Device) The terminal control device 40 controls each part of the information terminal device 30. The terminal control device 40 has, for example, a control device such as a CPU or MPU, and a storage device such as a RAM or ROM. The terminal control device 40 executes a program that controls the operation of the information terminal device 30 according to the present disclosure. The terminal control device 40 may be realized by, for example, an integrated circuit such as an ASIC or FPGA. The terminal control device 40 loads a stored program into memory and executes instructions included in the program. The terminal control device 40 includes an internal memory such as the RAM described above, and the internal memory is used for temporary storage of data in the terminal control device 40, etc. The terminal control device 40 may be realized by a combination of hardware and software.

[0064] The terminal control device 40 has an information acquisition unit 41, an operation reception unit 42, an output control unit 47, and a communication control unit 49 as functional blocks realized by circuits and programs to be executed.

[0065] The information acquisition unit 41 acquires the video captured by the imaging device 31 and the audio collected by the imaging device 31. The information acquisition unit 41 includes a video acquisition unit 411 and an audio acquisition unit 412.

[0066] The video acquisition unit 411 acquires video captured by the camera 311. More specifically, the video acquisition unit 411 acquires video captured by the camera 311 of a person facing the information terminal device 30, etc. The video data acquired by the camera 311 is a moving image.

[0067] The voice acquisition unit 412 acquires the voice detected by the microphone 312. The voice acquisition unit 412 acquires voice such as a voice spoken by a person present in the vicinity of the information terminal device 30.

[0068] The operation reception unit 42 acquires an operation signal corresponding to an operation on the operation unit 33 from the operation unit 33. The operation reception unit 42 outputs a control signal corresponding to the acquired operation signal to each unit.

[0069] The output control unit 47 may output video captured and audio collected by a space sharing device 10 in another space to the display device 17. The output control unit 47 may also output video captured and audio collected by a space sharing device 10 in the same space to the display device 17. More specifically, the output control unit 47 may output video captured and audio collected by an imaging device 11 arranged in another space or the same space, which have been received by the communication control unit 49, to the display device 17.

[0070] The output control unit 47 may output the video captured by the imaging device 31 and the audio collected by the imaging device 31 to the display unit 37 and the audio output unit 38. The output control unit 47 includes a display control unit 471 and an audio output control unit 472.

[0071] The display control unit 471 controls the display of the video on the display unit 37. For example, the display control unit 471 may control the display unit 37 to display the video captured by the space sharing device 10 in another space. The display control unit 471 may also control the display unit 37 to display the video captured by the space sharing device 10 in the same space.

[0072] The audio output control unit 472 controls the output of audio in the audio output unit 38. For example, the audio output control unit 472 may control the audio output unit 38 to output audio picked up by a space sharing device 10 in another space. The audio output control unit 472 may also output audio picked up by a space sharing device 10 in the same space.

[0073] The communication control unit 49 controls the communication unit 39 to control communication of data such as video and audio with a space sharing device 10 located in another space. More specifically, the communication control unit 49 controls, for example, the communication unit 39 to control communication with a space sharing device 10 located in another space. The communication control unit 49 controls, for example, the communication unit 39 to receive video and audio from a space sharing device 10 located in another space. The communication control unit 49 controls, for example, the communication unit 39 to transmit video and audio of a person facing the display device 37 of the information terminal device 30, acquired by the information acquisition unit 41, to the space sharing device 10 located in the other space. The communication control unit 49 may also control the communication unit 39 to control communication of data such as video and audio with a space sharing device 10 located in the same space.

[0074] (Space sharing method) Fig. 7 is a flowchart showing a space sharing method according to the first embodiment. The processing shown in Fig. 7 is executed while space is being shared by the space sharing system 1. While space is being shared, the first space sharing device 10A arranged in the first space A outputs video and audio of the second space B. The second space sharing device 10B arranged in the second space B outputs video and audio of the first space A. Fig. 7 shows processing executed by each of the space sharing devices 10 arranged in the first space A and the second space B.

[0075] 7 , the space sharing control device 20 associates the position information of the information terminal device 30 present in the space where the space sharing device 10 is placed with the position of the image acquired by the image acquisition unit 211. The position of the information terminal device 30 in the image acquired by the image acquisition unit 211 corresponds to the operation position of the operation unit 13.

[0076] The space sharing control device 20 determines whether or not an operation has been performed on the display screen of the display device 17 of the space sharing device 10 (step S101). More specifically, the space sharing control device 20 determines whether or not an operation has been received on the display screen of the display device 17 by the operation receiving unit 22. If the space sharing control device 20 determines that an operation has been performed on the display screen of the display device 17 of the space sharing device 10 (Yes in step S101), the space sharing control device 20 proceeds to step S102. If the space sharing control device 20 does not determine that an operation has been performed on the display screen of the display device 17 of the space sharing device 10 (No in step S101), the space sharing control device 20 ends the processing of this flowchart.

[0077] When it is determined that an operation has been performed on the display screen of the display device 17 of the space sharing device 10 (Yes in step S101), the space sharing control device 20 determines whether the operation is an operation to specify an information terminal device 30 or a person in another space that is being displayed (step S102). More specifically, the space sharing control device 20 determines whether the operation receiving unit 22 has received a specification operation to specify an information terminal device 30 or a person in another space that is being displayed on the display screen of the display device 17. When the space sharing control device 20 determines that the operation is an operation to specify an information terminal device 30 or a person in another space that is being displayed (Yes in step S102), the process proceeds to step S103. When the space sharing control device 20 does not determine that the operation is an operation to specify an information terminal device 30 or a person in another space that is being displayed (No in step S102), the process of this flowchart ends.

[0078] If it is determined that the operation is a designation operation for an information terminal device 30 or a person in another space being displayed (Yes in step S102), the space sharing control device 20 determines whether the designated information terminal device 30 can be called (step S103). More specifically, the space sharing control device 20 uses the identification unit 23 to identify the information terminal device 30 corresponding to the person designation operation. The space sharing control device 20 uses the identification unit 23 to determine whether the identified information terminal device 30 can be called based on the operation status information of the information terminal device 30 stored in the storage unit 14, or the attendance / non-attendance information and application operation status of each account user. If the space sharing control device 20 determines that the designated information terminal device 30 can be called (Yes in step S103), the process proceeds to step S104. If the space sharing control device 20 does not determine that the designated information terminal device 30 can be called (No in step S103), the process proceeds to step S109.

[0079] If it is determined that the designated information terminal device 30 can be called (Yes in step S103), the space sharing control device 20 calls the designated information terminal device 30 (step S104). More specifically, the space sharing control device 20 transmits call information by voice, message, or the like to the designated information terminal device 30 via the communication control unit 29. The space sharing control device 20 proceeds to step S105.

[0080] The space sharing control device 20 determines whether or not there has been a response (step S105). More specifically, the space sharing control device 20 determines whether or not there has been a response from the called information terminal device 30 using the communication control unit 29. If the space sharing control device 20 determines that there has been a response (Yes in step S105), it proceeds to step S106. If the space sharing control device 20 does not determine that there has been a response (No in step S105), it ends the processing of this flowchart.

[0081] If it is determined that there has been a response (Yes in step S105), the space sharing control device 20 starts outputting the video captured by the imaging device 31 of the information terminal device 30 and the collected audio (step S106). More specifically, the space sharing control device 20 causes the output control unit 27 to output the video captured by the imaging device 31 of the called information terminal device 30 and the collected audio received by the communication control unit 29 to the display device 17. In other words, a state is created in which a call can be made between the person using the space sharing device 10 and the person using the information terminal device 30. The space sharing control device 20 proceeds to step S107.

[0082] The space sharing control device 20 determines whether to end the call (step S107). More specifically, the space sharing control device 20 determines to end the call, for example, when the operation receiving unit 22 receives an operation to end the call, or when the information terminal device 30 that responded receives an operation to end the call. When the space sharing control device 20 determines to end the call (Yes in step S107), it proceeds to step S108. When the space sharing control device 20 does not determine to end the call (No in step S107), it executes the process of step S107 again. The end of the call in step S107 refers to the end of the call between the space sharing device 10 and the information terminal device 30.

[0083] When it is determined that the call is to be ended (Yes in step S107), the space sharing control device 20 stops outputting the video captured by the imaging device 31 of the information terminal device 30 and the audio collected (step S108). More specifically, the space sharing control device 20 causes the output control unit 27 to stop outputting the video captured by the imaging device 31 of the called information terminal device 30 and the audio to the display device 17.

[0084] If it is not determined that the designated information terminal device 30 can be called (No in step S103), the space sharing control device 20 displays a message that the call is not possible (step S109). More specifically, the space sharing control device 20 controls the display unit 171 via the display control unit 271 to display a message video indicating that the call is not possible, superimposed on the video of the other space captured by the designated information terminal device 30.

[0085] In this way, the space sharing system 1 allows the first space A and the second space B to virtually share a space, and allows the information terminal device 30 to participate in the space sharing.

[0086] 8 and 9, a case will be described in which person ub present in second space B communicates with a person present in first space A. FIG. 8 is a diagram showing an example of a display screen by the space sharing device according to the first embodiment. FIG. 9 is a diagram showing another example of a display screen by the space sharing device according to the first embodiment. As shown in FIG. 4, person ub faces the second display device 17B in second space B. Person ub checks the image of first space A displayed on the second display device 17B.

[0087] The dashed circle in FIG. 8 represents the person u14 or the information terminal device 30A used by the person u14 in the image displayed on the second display device 17B. １４ 8, in step S102, when a person ub in front of the second display device 17B touches the area indicated by the dashed circle, the information terminal device 30A １４ A pop-up window MW showing the account user name "u14" logged in to the information terminal device 30A is displayed. The area indicated by the dashed line may or may not be displayed on the second display device 17B. When the person ub performs a touch operation on the dashed circular area or the pop-up window MW, or a long touch operation on the dashed circular area, in step S103, １４If it is determined in step S103 that the information terminal device 30A is callable, the information terminal device 30A is called in step S104. １４ In step S105, the person u14 is called via the information terminal device 30A. １４ 9, in step S106, the information terminal device 30A １４ The image and sound captured by the first imaging device 31 are output to the second display device 17B. At this time, the image captured by the first imaging device 11A may be displayed as a background, or may not be displayed at all. The sound captured by the first imaging device 11A may be output at a low sound level or may be muted. In FIG. 9, the information terminal device 30A is displayed as a window W in the image captured by the first imaging device 11A. １４ The image captured by the imaging device 31 is displayed.

[0088] (Effects) As described above, in this embodiment, the information terminal device 30 located within the range captured by the first imaging device 11A corresponding to the operation position on the display screen of the second display device 17B is identified. In this embodiment, when the information terminal device 30 is identified, the video captured by the third imaging device 31 provided in the identified information terminal device 30 and the audio collected by the third imaging device 31 are output to the second display device 17B. According to this embodiment, when calling a person located in another space using the space-sharing system 1, the call can be made without setting up a call via the space-sharing system in advance or calling the person by outputting audio via the display device 17. According to this embodiment, the called person can respond using the information terminal device 30 they are using without having to move to the display device 17. In this way, according to this embodiment, space can be shared and communication can be carried out more smoothly.

[0089] [Second embodiment] A space sharing device 10X according to a second embodiment will be described using Figs. 10 to 12. Fig. 10 is a block diagram showing an example of the configuration of a space sharing device according to the second embodiment. Fig. 11 is a flowchart showing a space sharing method according to the second embodiment. Fig. 12 is a diagram showing an example of a display screen of the space sharing device according to the second embodiment. In the following description, components similar to those in the space sharing device 10 are given the same or corresponding reference numerals, and detailed description thereof will be omitted. The same applies to the following embodiments. The space sharing device 10X differs from the first embodiment in that it includes a person detection unit 24X and in the processing of an output control unit 27X.

[0090] In this embodiment, when calling a person via the information terminal device 30 in another space, the restriction on the output of video or audio captured in the other space is controlled depending on whether or not a call is being made with another person in the other space.

[0091] The person detection unit 24X detects people from the video acquired by the information acquisition unit 21. The person detection unit 24X detects the whole body, upper body, or face of a person from the video acquired by the video acquisition unit 211. The person detection unit 24X detects people in the vicinity of the display device 17 who are facing the display device 17, that is, people who are facing the display device 17. The method by which the person detection unit 24X detects people and faces from the video can be any known method and is not limited to this. In other words, the person detection unit 24X detects people who are in front of the display device 17 in order to use the space sharing device 10X. The person detection unit 24X transmits or shares the person detection result to a space sharing device 10X in another space.

[0092] If a person facing the display device 17 is not detected based on the detection result of the person detection unit 24X, the output control unit 27X restricts the output of the video captured by the imaging device 11 or the audio collected by the imaging device 11, and outputs the video captured and the audio collected by the imaging device 31 of the information terminal device 30 to the display device 17. More specifically, if a person facing the display device 17 in the space sharing device 10X in the other space, that is, a person making a call using the space sharing device 10X in the other space, is not detected based on the detection result of the person detection unit 24X in the space sharing device 10X in the other space, the output control unit 27X restricts the output of the video captured or the audio collected by the imaging device 11 in the space sharing device 10X in the other space, and outputs the video captured and the audio collected by the third imaging device 31 to the display device 17 arranged in the other space.

[0093] When a person facing the display device 17 is detected based on the detection result of the person detection unit 24X, the output control unit 27X outputs, to the display device 17, the video imaged and the audio collected by the imaging device 31 of the information terminal device 30 in addition to the video imaged and the audio collected by the imaging device 11. More specifically, when the output control unit 27X detects a person facing the display device 17 in the space sharing device 10X in the other space, that is, a person making a call using the space sharing device 10X in the other space, the output control unit 27X outputs, to the display device 17, the video imaged and the audio collected by the third imaging device 31 in addition to the video imaged and the audio collected by the imaging device 11 in the space sharing device 10X in the other space.

[0094] The restriction on the video output by the output control unit 27X may be, for example, stopping the video output (turning off the video display), reducing the brightness and saturation of the video, or blurring the video.

[0095] The restriction on the audio output by the output control unit 27X includes, for example, stopping the audio output (turning the audio OFF) or reducing the volume.

[0096] Specifically, for example, when no person facing the first display device 17A is detected near the first display device 17A based on the detection result of the person detection unit 24X of the first space sharing device 10X arranged in the first space A, the output control unit 27X of the second space sharing device 10B arranged in the second space B limits the output of the video captured or audio collected by the first imaging device 11A and outputs the video captured and audio collected by the third imaging device 31 to the second display device 17B. When a person facing the first display device 17A is detected near the first display device 17A, the output control unit 27X of the second space sharing device 10B outputs the video captured and audio collected by the third imaging device 31 to the second display device 17B in addition to the video captured and audio collected by the first imaging device 11A.

[0097] Next, the space sharing method will be described with reference to Fig. 11. The processes in steps S201 to S205 and step S213 in the flowchart shown in Fig. 11 are similar to the processes in steps S101 to S105 and step S109 in the flowchart shown in Fig. 7.

[0098] The space sharing control device 20X determines whether or not a person is facing the display device 17 of the space sharing control device 20X in the other space (step S206). More specifically, the space sharing control device 20X acquires the detection result of the person facing the display device 17 in the other space by the person detection unit 24X in the space sharing control device 20X in the other space. The person facing the display device 17 in the other space may be the caller. If the space sharing control device 20X determines that a person is facing the display device 17 in the other space (Yes in step S206), the process proceeds to step S207. If the space sharing control device 20X does not determine that a person is facing the display device 17 in the other space (No in step S206), the process proceeds to step S210.

[0099] When it is determined that there is a person facing the display device 17 in the other space (Yes in step S206), the space sharing control device 20X continues outputting the video and audio received from the space sharing control device 20X in the other space, and starts outputting the video and audio received from the information terminal device 30 (step S207). In other words, when a call is in progress with the space sharing control device 20X in the other space, the space sharing control device 20X continues the call and starts the call with the information terminal device 30 that has answered the call. The space sharing control device 20X proceeds to step S208.

[0100] The space sharing control device 20X determines whether to end the call (step S208). More specifically, the space sharing control device 20X determines to end the call, for example, when the operation receiving unit 22 receives an operation to end the call, or when the information terminal device 30 that responded receives an operation to end the call. If the space sharing control device 20X determines to end the call (Yes in step S208), the process proceeds to step S209. If the space sharing control device 20X does not determine to end the call (No in step S208), the process of step S208 is executed again.

[0101] When it is determined that the call is to be ended (Yes in step S208), the space sharing control device 20X continues outputting the video and audio received from the space sharing control device 20X in the other space, and ends outputting the video and audio received from the information terminal device 30 (step S209). In other words, the space sharing control device 20X continues space sharing with the space sharing control device 20X in the other space, and stops space sharing with the information terminal device 30 that responded to the call, by using the output control unit 27X.

[0102] If it is not determined that there is a person facing the display device 17 in the other space (No in step S206), the space sharing control device 20X restricts the output of video or audio received from the space sharing control device 20X in the other space, and starts output of video and audio received from the information terminal device 30 (step S210). In other words, if a call with the space sharing control device 20X in the other space is not in progress, the space sharing control device 20X restricts the output of video or audio received from the space sharing control device 20 in the other space by the output control unit 27X, and starts a call with the information terminal device 30 that has responded to the call. The space sharing control device 20X proceeds to step S211.

[0103] The space sharing control device 20X determines whether to end the call (step S211). More specifically, the space sharing control device 20X determines to end the call, for example, when the operation receiving unit 22 receives an operation to end the call, or when the information terminal device 30 that responded receives an operation to end the call. If the space sharing control device 20X determines to end the call (Yes in step S211), the process proceeds to step S212. If the space sharing control device 20X does not determine to end the call (No in step S211), the process of step S211 is executed again.

[0104] When it is determined that the call is to be ended (Yes in step S211), the space sharing control device 20X removes the restriction on the output of the video or audio received from the space sharing control device 20X in the other space, and ends the output of the video and audio received from the information terminal device 30 (step S212). In other words, the space sharing control device 20X removes the restriction on space sharing with the space sharing control device 20X in the other space, by the output control unit 27X, and stops space sharing with the information terminal device 30 that responded to the call.

[0105] 12 is an example of the display screen of the second display device 17B when it is determined in step S206 that a person is present facing the display device 17 in another space. In FIG. 12, person ua is detected as the person facing the first display device 17A. In addition to the image captured by the imaging device 11 of the space sharing control device 20X in space A, which is another space, the second display device 17B also displays the image of the information terminal device 30A that was called. １４ At this time, the second display device 17B displays the image captured by the imaging device 31 of the space sharing control device 20X in the space A and the collected audio and the image captured by the imaging device 11 of the information terminal device 30A. １４ 12, the image captured by the image capturing device 31 of the information terminal device 30 from which the call was made is preferably displayed at a position that avoids the range of the detected person, in other words, at a position that does not overlap the range of the detected person, or at a position that does not overlap too much with the range of the detected person. １４ The window W of the image captured by the image capturing device 31 is displayed in a position that does not overlap with the range of the detected person ua.

[0106] Furthermore, if it is determined in step S206 that there is no person facing the display device 17 in the space, the display screen of the second display device 17B is displayed as shown in Fig. 9. At this time, the second display device 17B is limited in outputting the video captured by the imaging device 11 of the space sharing control device 20X in space A or the audio collected, and the information terminal device 30A １４ The image captured by the imaging device 31 and the sound collected are output.

[0107] (Effects) As described above, in the present embodiment, when a person facing the first display device 17A is not detected near the first display device 17A, the output of the video imaged or the audio collected by the first imaging device 11A is limited, and the video imaged and the audio collected by the third imaging device 31 are output to the second display device 11B. In the present embodiment, when a person facing the first display device 17A is detected near the first display device 17A, the video imaged and the audio collected by the third imaging device 31 are output to the second display device 17B in addition to the video imaged and the audio collected by the first imaging device 11A. According to the present embodiment, the output of the video and audio from the other space can be limited depending on the detection result of the caller in the space of the called party.

[0108] [Third embodiment] A space sharing control device 20X according to a third embodiment will be described using Fig. 13 to Fig. 15. Fig. 13 is a flowchart showing a space sharing method according to the third embodiment. Fig. 14 is a diagram showing an example of a display screen by the space sharing device according to the third embodiment. Fig. 15 is a diagram showing an example of a display screen by the space sharing device according to the third embodiment. In this embodiment, the processing of a person detection unit 24X and an output control unit 27X differs from that of the second embodiment.

[0109] The person detection unit 24X detects a person and the position of the person's face from the video acquired by the information acquisition unit 21. More specifically, for example, when person ub present in the second space B calls person u14 in the first space A, the person detection unit 24X detects the face of person ub from the camera video of the second imaging device 11B on the second space B side, and calculates the position of the face of person ub relative to the second display device 17B. The person detection unit 24X may detect the person's face from the video captured by the third imaging device 31 and received by the communication control unit 29.

[0110] The person detection unit 24X may further detect parameters indicating the size of the person's face, such as the width of the person's face, from the video acquired by the information acquisition unit 21. More specifically, for example, when person ub present in the second space B calls person u14 in the first space A, the person detection unit 24X may detect parameters indicating the size of the person's face, such as the width of the person's face, from the camera video of the second imaging device 11B on the second space B side. Furthermore, the person detection unit 24X may detect parameters indicating the size of the person's face, such as the width of the person's face, from the video captured by the third imaging device 31 received by the communication control unit 29.

[0111] The output control unit 27X outputs the video captured by the third imaging device 31 acquired by the information acquisition unit 21 to the display device 17, aligned with the position of the face of the person detected from the video captured by the imaging device 11. The output control unit 27X displays the image captured by the third imaging device 31, aligned with the position of the face of the person ub who is using the space sharing device 10 on the display device 17.

[0112] More specifically, for example, the output control unit 27X of the second space sharing device 10B arranged in the second space B outputs the image captured by the third imaging device 31 present in the first space received by the communication control unit 29 to the second display device 17B, aligning it with the position of the face of a person detected from the image captured by the second imaging device 11B, in this case person ub, who is the user of the space sharing device 10B.

[0113] Furthermore, when there is a user of the first space sharing device 10A placed in the first space A, the output control unit 27X of the first space sharing device 10A placed in the first space A may output the image captured by the third imaging device 31 received by the communication control unit 29 to the first display device 17A, aligned with the position of the person's face detected from the image captured by the first imaging device 11A.

[0114] The output control unit 27X may adjust the size of the person's face relative to the display device 17 according to the size of the person's face, such as the width of the face, detected by the person detection unit 24X. Specifically, in the second display device 17B, the size of the person's face included in the video captured by the third imaging device 31 is adjusted to be equivalent to the size of the face of the user of the first space sharing device 10A, and the adjusted size is displayed on the display unit 171. The video captured by the third imaging device 31 to be displayed on the display unit 171 can be adjusted by any process, such as cropping the video or enlarging the display size. By performing such processing, the sense of distance between the person using the space sharing device 10X in another space and the person using the information terminal device 30 becomes consistent, thereby reducing the sense of discomfort felt when sharing a space.

[0115] Next, a space sharing method will be described with reference to Fig. 13. The processes in steps S301 to S305 and steps S308 to S310 in the flowchart shown in Fig. 13 are similar to the processes in steps S101 to S105 and steps S106 to S109 in the flowchart shown in Fig. 7.

[0116] When the space sharing control device 20X determines that there has been a response (Yes in step S305), it detects the position of the person and their face (step S306). More specifically, the space sharing control device 20X detects the person facing the display device 17 and the position of the person's face from the video acquired by the information acquisition unit 21 using the person detection unit 24X. The detected position of the face here is the position of the face in the up-down direction. The space sharing control device 20X proceeds to step S307.

[0117] The space sharing control device 20X starts outputting the video captured by the imaging device 31 of the information terminal device 30 and the collected audio (step S307). More specifically, the space sharing control device 20X causes the output control unit 27X to output the video captured by the imaging device 31 of the called information terminal device 30 and the collected audio received by the communication control unit 29 to the display device 17. The space sharing control device 20X causes the output control unit 27X to output the video captured by the imaging device 31 of the called information terminal device 30 to the display device 17 in accordance with the position of the person's face detected from the video captured by the imaging device 11. The space sharing control device 20X proceeds to step S308. Displaying the video in accordance with the position of the person's face detected from the video captured by the imaging device 11 means adjusting the height direction position of the video captured by the imaging device 31 of the called information terminal device 30 on the display device 17 to a position equivalent to the height direction position of the person's face detected from the video captured by the imaging device 11.

[0118] 14 and 15, the second display device 17B in the second space B displays the first space A. The dashed line L in the figure indicates the position of the face of person ub who is present in the second space B. FIG. 14 shows a state in which person ub has called person u14 who is present in the first space A. FIG. 15 shows a state in which person ub has called person u14 while talking to person ua who is present in the first space A.

[0119] (Effects) As described above, in this embodiment, the image captured by the third imaging device 31 is output to the second display device 17B in accordance with the position of the face of the person detected from the image captured by the second imaging device 11B. According to this embodiment, the position of the face of the called person can be displayed on the display device of the calling party in accordance with the position of the caller's face. According to this embodiment, it is possible to have a conversation with a person in a distant space in a state similar to that of a face-to-face conversation in real space.

[0120] In this embodiment, by adjusting the width of the faces, the people in the first space A and the second space B can converse with the same sense of distance, that is, with the same face size.

[0121] [Fourth embodiment] A space sharing control device 20X according to a fourth embodiment will be described with reference to Figs. 16 to 18. Fig. 16 is a schematic diagram showing another example of installation of a space sharing device, and is a diagram showing installation in a first space. Fig. 17 is a diagram showing an example of a display screen of the space sharing device according to the fourth embodiment. Fig. 18 is a flowchart showing a space sharing method according to the fourth embodiment. In this embodiment, the processing of the identification unit 23 differs from that of the first embodiment.

[0122] In this embodiment, it is possible to call a person who is present in a location other than the space where the space sharing device 10 is located, such as on another floor, at another business location, or at home. A person who is present in a location other than the space where the space sharing device 10 is located is assumed to be associated with a fixed seat in the space where the space sharing device 10 is located. Therefore, the range centered on the seat displayed on the display screen of the display device 17 is set as the person correspondence range of the person associated with that seat. When using the space sharing device 10 to call a person in another space who is not present in the range captured by the imaging device 11 of another space sharing device 10, in other words, a person who is present in a location other than the other space, the call can be made by specifying the person correspondence range on the display screen of the display device 17.

[0123] The identification unit 23 identifies the information terminal device 30 used by a person associated with the operation position on the display screen of the display device 17 accepted by the operation acceptance unit 22. The identification unit 23 identifies the information terminal device 30 used by a person corresponding to a person correspondence range, which is the operation position on the display screen of the display device 17 accepted by the operation acceptance unit 22. More specifically, for example, the identification unit 23 of the first space sharing device 10A arranged in the first space A identifies the information terminal device 30 used by a person in the person correspondence range corresponding to the operation position on the display screen of the first display device 17A accepted by the operation acceptance unit 22. For example, the identification unit 23 of the second space sharing device 10B arranged in the second space B identifies the information terminal device 30 used by a person in the person correspondence range corresponding to the operation position on the display screen of the second display device 17B accepted by the operation acceptance unit 22.

[0124] In FIGS. 16 and 17 , person u11 is working in a location other than the first space A, such as another floor, another business location, or at home. Person u11 is not present in the first space A, and the information terminal device 30 used by person u11 is not located there. In FIG. 17 , the positions indicated by dashed lines are registered as fixed seats, and no person is seated there. In FIG. 17 , a display indicating that person u11 is not present in the first space A may be displayed at the fixed seat indicated by dashed lines for person u11. For example, an avatar representing person u11 may be displayed at the fixed seat. In this state, by touching the dashed circular area on the display screen of the second display device 17B, the information terminal device 30 of person u11 associated with that location is called up. By touching the dashed circular area on the display screen of the second display device 17B, person u11 can be called up if he or she is working in another location and logged in to the information terminal device 30. If the person u11 is not logged in to the information terminal device 30 because he / she is on vacation or traveling, a call to the information terminal device 30 of the person u11 is not made, and a message indicating that the call is not possible is displayed.

[0125] Next, the space sharing method will be described with reference to Fig. 18. The processes in steps S401, S404 to S409 in the flowchart shown in Fig. 18 are similar to those in steps S101, S104 to S109 in the flowchart shown in Fig. 7.

[0126] When it is determined that an operation has been performed on the display screen of the display device 17 of the space sharing device 10 (Yes in step S401), the space sharing control device 20 determines whether the operation is a designation operation for the person correspondence range of the other space being displayed (step S402). More specifically, the space sharing control device 20 determines whether the operation receiving unit 22 has received a designation operation for designating the person correspondence range of the other space being displayed on the display screen of the display device 17. When the space sharing control device 20 determines that the designation operation is a designation operation for the person correspondence range of the other space being displayed (Yes in step S402), the process proceeds to step S403. When the space sharing control device 20 does not determine that the designation operation is a designation operation for the person correspondence range of the other space being displayed (No in step S402), the process of this flowchart ends.

[0127] If it is determined that the operation is a designation operation for the person corresponding range of the other space being displayed (Yes in step S402), the space sharing control device 20 determines whether the information terminal device 30 used by the person corresponding to the designated range can be called (step S403). More specifically, the space sharing control device 20 uses the identification unit 23 to identify the information terminal device 30 used by the person corresponding to the designation operation of the person corresponding range. The space sharing control device 20 uses the identification unit 23 to determine whether the identified information terminal device 30 can be called based on the operation status information of the information terminal device 30 stored in the storage unit 14, or the attendance / non-attendance information of each account user and the operation status of applications. If the space sharing control device 20 determines that the information terminal device 30 used by the person corresponding to the designated range can be called (Yes in step S403), the process proceeds to step S404. If the space sharing control device 20 does not determine that the information terminal device 30 used by the person corresponding to the designated range can be called (No in step S403), the process proceeds to step S409.

[0128] (Effects) As described above, in this embodiment, it is possible to identify and call an information terminal device 30 that is associated with an operation position on the display screen of the display device and is being used by a person who is not in another space. According to this embodiment, it is possible to call a person who is not in another space using the space sharing device 10.

[0129] Fifth Embodiment A space sharing control device 20X according to a fifth embodiment will be described with reference to Fig. 19. Fig. 19 is a flowchart showing a space sharing method according to the fifth embodiment. In this embodiment, the processing in the identification unit 23 differs from that in the second embodiment.

[0130] If a person is detected at the operation position on the display screen of the second display device 17B accepted by the operation accepting unit 22, the identifying unit 23 identifies the information terminal device 30 used by the detected person, and if no person is detected at the operation position, the identifying unit 23 identifies the information terminal device 30 used by the person associated with the operation position.

[0131] Next, a space sharing method will be described with reference to Fig. 19. The processes of steps S501, S504, S505, S507, S511, and S512 in the flowchart shown in Fig. 19 are the same as those of steps S201, S204, S205, S208, and S213 in the flowchart shown in Fig. 11. The processes of steps S506 and S508 in the flowchart shown in Fig. 19 are the same as those of steps S106 and S108 in the flowchart shown in Fig. 7. The processes of steps S509 and S510 in the flowchart shown in Fig. 19 are the same as those of steps S402 and S403 in the flowchart shown in Fig. 18.

[0132] The space sharing control device 20X determines whether a person has been detected within the operation range (step S502). If the space sharing control device 20X determines that a person has been detected within the operation range (Yes in step S502), the process proceeds to step S503. If the space sharing control device 20X does not determine that a person has been detected within the operation range (No in step S502), the process proceeds to step S509.

[0133] The space sharing control device 20X determines whether the information terminal device 30 associated with the person can be called (step S503). More specifically, the space sharing control device 20X uses the identification unit 23 to identify the information terminal device 30 corresponding to the person designation operation. The space sharing control device 20X uses the identification unit 23 to determine whether the identified information terminal device 30 can be called based on the operation status information of the information terminal device 30 stored in the storage unit 14, or the attendance / non-attendance information of each account user and the operation status of the application. If the space sharing control device 20X determines that the information terminal device 30 associated with the person can be called (Yes in step S503), the process proceeds to step S504. If the space sharing control device 20 does not determine that the information terminal device 30 associated with the person can be called (No in step S503), the process proceeds to step S512.

[0134] (Effects) As described above, in this embodiment, when a person is detected at the operation position on the display screen of the display device 17, the information terminal device 30 used by the detected person can be identified, and when a person is not detected at the operation position, the information terminal device 30 used by the person associated with the operation position can be identified. According to this embodiment, it is possible to call a person who is not in a space using the space sharing device 10.

[0135] In the above description, the space sharing control device 20 is described as being disposed in each of the first space A and the second space B, but in the case of centralized control of the imaging devices 11 and the display devices 17 installed in the first space A and the second space B, the space sharing control device 20 has the functions of the space sharing control device 20 disposed in the first space A described above and the functions of the space sharing control device 20 disposed in the second space B. In this case, the space sharing control device 20 is as follows.

[0136] The information acquisition unit 21 acquires video images and collected audio by the first imaging device 11A and the second imaging device 11B. The output control unit 27 outputs the video images and collected audio by the first imaging device 11A to the second display device 17B, and outputs the video images and collected audio by the second imaging device 11B to the first display device 17A. The operation acceptance unit 22 accepts operations on the display screen of the second display device 17B. The identification unit 23 identifies an information terminal device 30 present in the range imaged by the first imaging device 11A corresponding to the operation position on the display screen of the second display device 17B accepted by the operation acceptance unit 22. The information acquisition unit 21 acquires video images and collected audio by a third imaging device 31 included in the information terminal device 30 identified by the identification unit 23. When the identification unit 23 identifies the information terminal device 30, the output control unit 27 outputs the video images and collected audio by the third imaging device 31 to the second display device 17B.

[0137] The space sharing system, space sharing method, and program of the present disclosure can be used, for example, in a communication system that constantly shares video and audio in real time across multiple real spaces.

[0138] 1 Space sharing system 10 Space sharing device 10A First space sharing device 10B Second space sharing device 11 Imaging device 11A First imaging device 11B Second imaging device 111 Camera 112 Microphone 13 Operation unit 17 Display device 17A First display device 17B Second display device 171 Display unit 172 Audio output unit 19 Communication unit 20 Control unit (space sharing control device) 21 Information acquisition unit 211 Video acquisition unit 212 Audio acquisition unit 22 Operation acceptance unit 23 Identification unit 27 Output control unit 271 Display control unit 272 Audio output control unit 29 Communication control unit 30 Information terminal device 31 Imaging device (third imaging device) 311 Camera 312 Microphone 33 Operation unit 37 Display unit 38 Audio output unit 39 Communication unit 40 Control unit (information terminal control device) 41 Information acquisition unit 411 Video acquisition unit 412 Audio acquisition unit 42 Operation reception unit 47 Output control unit 471 Display control unit 472 Audio output control unit 49 Communication control unit A First space B Second space

Claims

1. A space-sharing system comprising a first imaging device and a first display device arranged in a first space, and a second imaging device and a second display device arranged in a second space, comprising: an information acquisition unit that acquires video captured and audio picked up by the first imaging device and the second imaging device; an output control unit that outputs the video captured and audio picked up by the first imaging device to the second display device, and outputs the video captured and audio picked up by the second imaging device to the first display device; an operation reception unit that accepts operations on a display screen of the second display device; and an identification unit that identifies an information terminal device corresponding to the operation position on the display screen of the second display device accepted by the operation reception unit, wherein the information acquisition unit acquires video captured and audio picked up by a third imaging device provided in the information terminal device identified by the identification unit, and the output control unit outputs the video captured and audio picked up by the third imaging device provided in the information terminal device identified by the identification unit to the second display device.

2. The space sharing system according to claim 1, wherein the identification unit identifies an information terminal device present in the range imaged by the first imaging device that corresponds to the operation position on the display screen of the second display device accepted by the operation acceptance unit.

3. The space sharing system according to claim 1, wherein the identification unit identifies an information terminal device used by a person associated with the operation position on the display screen of the second display device accepted by the operation acceptance unit.

4. The space sharing system of claim 1, further comprising a person detection unit that detects a person from the image acquired by the information acquisition unit, wherein the identification unit, when a person is detected at an operation position on the display screen of the second display device accepted by the operation acceptance unit, identifies an information terminal device used by the detected person, and, when a person is not detected at the operation position, identifies an information terminal device used by the person associated with the operation position.

5. A space sharing system as described in any one of claims 1 to 3, further comprising a person detection unit that detects a person from the image acquired by the information acquisition unit, wherein the output control unit, when no person facing the first display device in the vicinity of the first display device is detected based on the detection result of the person detection unit, limits the output of the image captured by the first imaging device or the audio picked up by the third imaging device and outputs the image captured by the third imaging device and the audio picked up to the second display device, and when a person facing the first display device in the vicinity of the first display device is detected, outputs the image captured by the third imaging device and the audio picked up to the second display device in addition to the image captured by the first imaging device.

6. A space sharing system as described in any one of claims 1 to 3, further comprising a person detection unit that detects a person and the position of the person's face from the image acquired by the information acquisition unit, wherein the output control unit outputs the image captured by the third imaging device acquired by the information acquisition unit to the second display device in accordance with the position of the person's face detected from the image captured by the second imaging device.

7. A space sharing method for a space sharing system including a first imaging device and a first display device arranged in a first space, and a second imaging device and a second display device arranged in a second space, the space sharing method comprising the steps of: acquiring video captured and audio picked up by the first imaging device and the second imaging device; outputting the video captured and audio picked up by the first imaging device to the second display device, and outputting the video captured and audio picked up by the second imaging device to the first display device; determining that an operation on a display screen of the second display device has been accepted; when an operation on the display screen of the second display device has been accepted, identifying an information terminal device corresponding to the operation position on the display screen of the second display device where the operation was accepted; acquiring video captured and audio picked up by a third imaging device provided in the identified information terminal device; and outputting the video captured and audio picked up by the third imaging device to the second display device.

8. A program that causes a computer that controls a space sharing system to operate the following steps: acquiring video images and picked up audio by a first imaging device and a second imaging device; outputting the video images and picked up audio by the first imaging device to a second display device and outputting the video images and picked up audio by the second imaging device to the first display device; determining that an operation on a display screen of the second display device has been accepted; when an operation on the display screen of the second display device has been accepted, identifying an information terminal device corresponding to the operation position on the display screen of the second display device at which the operation was accepted; acquiring video images and picked up audio by a third imaging device provided in the identified information terminal device; and outputting the video images and picked up audio by the third imaging device to the second display device.

Images