Information processing system, information processing device, program, information processing method and room

Abstract

To provide an information processing system for performing effective performance to support communications of a plurality of users being in the same space.SOLUTION: In an information processing system including a video display device installed in a space to display video, a sensor device possessed by a user being in the space to output a signal for detecting position information of the user possessing it, and an information processing device connected to the video display device and the sensor device in a data communicable manner, the information processing device includes user information storage means for storing the position information of each of a plurality of users being in the space detected on the basis of a signal output from each of the sensor devices possessed by the plurality of users being the space in association with the users, and environmental performance means for controlling environmental performance to support communications among the plurality of uses by means of video displayed by the video display device on the basis of the position information of each of the plurality of users.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing system, an information processing device, a program, an information processing method, and a room. [Background technology]

[0002] There has long been known technology that, based on the detection results of a motion sensor, can prepare an environment for office meetings or presentations so that devices such as lighting equipment, projectors, retractable screens, and blinds can project images, without the user having to perform complicated operations (see, for example, Patent Document 1). Summary of the Invention [Problem to be solved by the invention]

[0003] With the recent spread of cloud computing, users can now use remote conference services, for example. By using remote conference services, users can hold remote conferences with users in different locations while saving on transportation costs and travel time. It is expected that such remote conferences will become more and more popular in the future.

[0004] Therefore, a conference where users gather in the same place (meet in person) is required to provide a different value from a remote conference. However, Patent Document 1 does not describe such a problem.

[0005] An object of the embodiment of the present invention is to provide an information processing system that effectively produces a presentation to support communication between a plurality of users in the same space. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, claim 1 of the present application provides an information processing system having an image display device that displays an image installed in a space, a sensor device carried by a user in the space and that outputs a signal for detecting location information of the user, and an information processing device connected to the image display device and the sensor device so as to be capable of data communication, wherein the information processing device has a user information storage means that associates and stores location information of each of a plurality of users in the space detected based on signals respectively output from the sensor devices carried by the plurality of users in the space, and an environmental production means that controls an environmental production that supports communication between the plurality of users through images displayed by the image display device, based on the location information of each of the plurality of users. Effect of the Invention

[0007] According to an embodiment of the present invention, it is possible to provide an effective performance to support communication between a plurality of users in the same space. [Brief description of the drawings]

[0008] [Figure 1] 1 is a configuration diagram of an example of an information processing system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram for explaining an example of a conference room according to the embodiment; [Diagram 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer according to the present embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of a hardware configuration of a smartphone according to the embodiment of the present invention. [Diagram 5] FIG. 2 is a diagram illustrating a hardware configuration of an example of a projector according to the embodiment. [Figure 6] FIG. 2 is a functional configuration diagram of an example of an information processing device according to the present embodiment. [Figure 7] 3 is a diagram illustrating an example of a configuration of a table used in the information processing system according to the embodiment. [Figure 8]10 is a flowchart illustrating an example of a processing procedure of the information processing system according to the embodiment. [Figure 9] FIG. 13 is an image diagram showing an example of spotlight processing on a user who is in an utterance state. [Figure 10] 13 is a flowchart of an example of a procedure for spotlighting a user who is in an utterance state. [Figure 11] FIG. 13 is an image diagram showing an example of spotlight processing according to the distance between users. [Figure 12] FIG. 13 is an image diagram showing an example of a specific example of spotlight processing according to the distance between users. [Figure 13] 13 is a flowchart of an example of a spotlight processing procedure according to a distance between users. [Figure 14] 13 is a flowchart of another example of a spotlight processing procedure according to the distance between users. [Figure 15] 13 is a flowchart illustrating an example of a procedure for a sound output process according to a distance between users. [Figure 16] 10 is a flowchart illustrating an example of a procedure for outputting a report in the information processing system according to the embodiment; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In this embodiment, a conference room will be described as an example of a space where multiple users gather, but the present invention is not limited to a conference room and can be applied to various spaces such as rooms for seminars and lectures, meeting spaces, etc.

[0010] [First embodiment] <System configuration> FIG. 1 is a configuration diagram of an example of an information processing system according to the present embodiment. FIG. 2 is a diagram for explaining an example of a conference room according to the present embodiment. In the information processing system 1 of FIG. 1, an information processing device 10, an image display device 12, a sensor device 14, a speaker 16, a camera 18, an operation device 20, a light 30, a speaker 32, an odor output device 34, and a sensor device 36 are connected by wire or wirelessly so as to be able to communicate with each other via a network 40 such as the Internet or a LAN. The operation device 20 has a microphone 22, a tag 24, and an operation button 26. Note that a plurality of types of operation devices 20 with different shapes and functions may be prepared in the conference room. For example, the operation device 20 may have a configuration including a touch panel, a gyro sensor, a tactile sensor, or the like. The operation devices 20 may have different sizes and textures.

[0011] The conference room is provided with a video display device 12, a sensor device 14, a speaker 16, a camera 18, and an operation device 20. The approach through which the user passes before entering the conference room is configured to include a light 30, a speaker 32, an odor output device 34, and a sensor device 36. Although an example in which the information processing device 10 is provided outside the conference room is shown in FIG. 1, it may be provided inside the conference room.

[0012] A sensor device 36 on the approach receives a beacon (radio wave) transmitted from a tag of a user passing through the approach as a signal for detecting the user's location information and notifies the information processing device 10. A light 30 on the approach outputs light under the control of the information processing device 10. A speaker 32 on the approach outputs sound under the control of the information processing device 10. An odor output device 34 on the approach outputs an odor under the control of the information processing device 10.

[0013] The information processing device 10 detects the user's location information from a signal notified by the sensor device 36, and performs approach direction processing to reset the user's mood with light, sound, and scent according to the detected user's location information. It is desirable to make the approach wide enough for only one person to pass at a time, so that it is time for users entering or leaving the conference room to return to their individuality. The approach direction processing for users entering the conference room and the approach direction processing for users leaving the conference room may be different.

[0014] The sensor device 14 in the conference room receives radio waves emitted from the tags of the users in the conference room as signals for detecting the user's location information, and notifies the information processing device 10. The sensor device 14 may be a sensor of a positioning system that can output a signal for detecting the user's location information, and may correspond to a dedicated tag, a smartphone, or various BLE sensors as the measurement target. The information processing device 10 detects the location information of each user in the conference room based on the signals for detecting the user's location information notified from one or more sensor devices 14.

[0015] The operation device 20 in the conference room is an example of a measurement target by the positioning system. For example, the sensor device 14 in the conference room can receive radio waves transmitted from the tag 24 of the operation device 20 and notify the information processing device 10 of a signal for detecting the position information of the user holding the operation device 20 in the conference room, as shown in Fig. 2. Note that the tag 24 built into the operation device 20 is just an example, and tags of other types may be used.

[0016] The camera 18 in the conference room captures images of the conference room and notifies the information processing device 10 of the captured image data. For example, a Kinect (registered trademark) video camera can be used as the camera 18. When Kinect (registered trademark) is used as the camera 18, the movement and posture of the user can be recognized by combining the detection results of a distance image sensor, an infrared sensor, and an array microphone with posture recognition technology. Posture recognition technology is a technology that estimates the posture of the user.

[0017] The multiple image display devices 12 in the conference room are projectors, and display images on walls (front wall, rear wall, right wall, left wall, etc.) and floor, which are examples of surfaces that divide the conference room, as shown in FIG. 2, under the control of the information processing device 10. By using the multiple image display devices 12, the conference room can project images onto the entire walls and floor. The multiple image display devices 12 in the conference room may be display devices embedded in the walls and floor of the conference room. By using the display devices embedded in the walls and floor, the conference room can display images on the walls and floor. In the following, an example in which the image display devices 12 are projectors will be described. The speaker 16 in the conference room outputs sound under the control of the information processing device 10. The shape of the conference room shown in FIG. 2 is merely an example, and other shapes may be used. In addition, the conference room does not need to be divided entirely by surfaces such as walls, floors, and ceilings, and some surfaces may be open and not divided.

[0018] The operation device 20 can be used by, for example, holding it in the user's hand, but may also be in the form of a wearable device worn by the user, such as a watch or glasses. The microphone 22 of the operation device 20 converts the user's voice into an electrical signal. The tag 24 is an example of an object to be measured by the positioning system. The operation button 26 accepts operations from the user. The operation device 20 transmits the electrical signal converted from the user's voice as an output signal to the information processing device 10. The operation device 20 transmits the user's operation accepted from the operation button 26 to the information processing device 10 as an operation signal.

[0019] Based on the user's position information detected by a signal notified from the sensor device 14, the output signal from the microphone 22 of the operation device 20, and the operation signal from the operation button 26 of the operation device 20, the information processing device 10 performs an environmental design process as described below to support communication between multiple users in the same space, with the aim of improving teamwork among the users in the conference room.

[0020] Furthermore, the information processing device 10 supports, for example, the generation of ideas in a team through environmental production processing as described below. In the information processing system 1 according to this embodiment, the teamwork of multiple users in a conference room is improved by controlling the environmental production using images and sounds as described below. Furthermore, the information processing device 10 achieves both maximization of individual creativity and realization of diversity through teamwork through environmental production processing as described below. For example, the information processing device 10 supports the divergence of ideas by stimulating the brains of individuals through environmental production processing as described below, thereby improving the quality of idea generation.

[0021] A conference room that can control the environmental production process described below is a space where digital environmental production (environmental effects) can be used to achieve effects such as users drinking alcohol, getting drunk, feeling better, helping ideas, and promoting communication. Such a space may be referred to as a digital alcohol space below. A digital alcohol space is a space that maximizes teamwork created by individuals (users) gathering together and transforms it into creative motivation. In addition, a digital alcohol space not only makes the work itself more efficient, but also improves the emotions and motivation of the workers themselves through environmental effects, supporting creativity.

[0022] The configuration of the information processing system 1 shown in Fig. 1 is an example. For example, the information processing device 10 may be realized by a single computer or multiple computers, or may be realized by using a cloud service. In addition, the information processing device 10 may be, for example, a PJ (Projector), an IWB (Interactive White Board: a white board having an electronic blackboard function capable of mutual communication), an output device such as a digital signage, a HUD (Head Up Display) device, an industrial machine, an imaging device, a sound collecting device, a medical device, a network home appliance, an automobile (Connected Car), a notebook PC (Personal Computer), a mobile phone, a smartphone, a tablet terminal, a game machine, a PDA (Personal Digital Assistant), a digital camera, a wearable PC, or a desktop PC.

[0023] <Hardware configuration> "computer" The information processing device 10 is realized by, for example, a computer 500 having the hardware configuration shown in FIG.

[0024] Fig. 3 is a diagram showing an example of a hardware configuration of a computer 500 according to this embodiment. As shown in Fig. 3, the computer 500 includes a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, a RAM (Random Access Memory) 503, an HD 504, an HDD (Hard Disk Drive) controller 505, a display 506, an external device connection I / F (Interface) 508, a network I / F 509, a data bus 510, a keyboard 511, a pointing device 512, a DVD-RW (Digital Versatile Disk Rewritable) drive 514, and a media I / F 516.

[0025] Of these, the CPU 501 controls the overall operation of the computer 500. The ROM 502 stores programs used to drive the CPU 501, such as the IPL. The RAM 503 is used as a work area for the CPU 501. The HD 504 stores various data such as programs. The HDD controller 505 controls the reading and writing of various data from and to the HD 504 under the control of the CPU 501.

[0026] The display 506 displays various information such as a cursor, a menu, a window, characters, or an image. The external device connection I / F 508 is an interface for connecting various external devices. In this case, the external devices are, for example, a USB (Universal Serial Bus) memory or a printer. The network I / F 509 is an interface for data communication using the network 40. The data bus 510 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 501.

[0027] The keyboard 511 is a type of input means having a plurality of keys for inputting characters, numbers, various instructions, etc. The pointing device 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The DVD-RW drive 514 controls reading and writing of various data from a DVD-RW 513 as an example of a removable recording medium. Note that the medium is not limited to a DVD-RW, and may be a DVD-R or the like. The media I / F 516 controls reading and writing (storing) of data from a recording medium 515 such as a flash memory.

[0028] 《Smartphone》 The operation device 20 may be realized by, for example, a smartphone 600 having the hardware configuration shown in FIG.

[0029] Fig. 4 is a diagram showing an example of a hardware configuration of a smartphone 600 according to this embodiment. As shown in Fig. 4, the smartphone 600 includes a CPU 601, a ROM 602, a RAM 603, an EEPROM 604, a CMOS sensor 605, an image sensor I / F 606, an acceleration / direction sensor 607, a media I / F 609, and a GPS receiver 611.

[0030] Of these, the CPU 601 controls the overall operation of the smartphone 600. The ROM 602 stores the CPU 601 and programs used to drive the CPU 601, such as the IPL. The RAM 603 is used as a work area for the CPU 601. The EEPROM 604 reads and writes various data, such as smartphone programs, under the control of the CPU 601.

[0031] The CMOS (Complementary Metal Oxide Semiconductor) sensor 605 is a type of built-in imaging means that captures an image of a subject (mainly a self-portrait) under the control of the CPU 601 to obtain image data. Note that imaging means such as a CCD (Charge Coupled Device) sensor may be used instead of the CMOS sensor 605. The imaging element I / F 606 is a circuit that controls the driving of the CMOS sensor 605. The acceleration / azimuth sensor 607 is one of various sensors such as an electronic magnetic compass or gyrocompass that detects geomagnetism, and an acceleration sensor.

[0032] A media I / F 609 controls reading and writing (storing) of data from and to a recording medium 608 such as a flash memory. A GPS receiving unit 611 receives GPS signals from GPS satellites.

[0033] The smartphone 600 also includes a long-distance communication circuit 612, a CMOS sensor 613, an image sensor I / F 614, a microphone 615, a speaker 616, an audio input / output I / F 617, a display 618, an external device connection I / F 619, a short-distance communication circuit 620, an antenna 620a of the short-distance communication circuit 620, and a touch panel 621.

[0034] Of these, the long-distance communication circuit 612 is a circuit that communicates with other devices via the network 40. The CMOS sensor 613 is a type of built-in imaging means that captures an image of a subject and obtains image data under the control of the CPU 601. The image sensor I / F 614 is a circuit that controls the driving of the CMOS sensor 613. The microphone 615 is a built-in circuit that converts sound into an electrical signal. The speaker 616 is a built-in circuit that converts an electrical signal into physical vibrations to produce sound such as music or voice.

[0035] The audio input / output I / F 617 is a circuit that processes input and output of audio signals between the microphone 615 and the speaker 616 under the control of the CPU 601. The display 618 is a type of display means such as a liquid crystal or organic EL (Electro Luminescence) that displays an image of a subject, various icons, and the like.

[0036] The external device connection I / F 619 is an interface for connecting various external devices. The short-range communication circuit 620 is a communication circuit such as NFC (Near Field Communication) or Bluetooth (registered trademark). The touch panel 621 is a type of input means that allows a user to operate the smartphone 600 by pressing the display 618.

[0037] The smartphone 600 also includes a bus line 610. The bus line 610 is an address bus, a data bus, or the like for electrically connecting the components such as the CPU 601 shown in FIG.

[0038] Projector A projector 700, which is an example of the image display device 12, may be realized by, for example, the hardware configuration shown in FIG.

[0039] Fig. 5 is a diagram showing an example of a hardware configuration of a projector 700 according to this embodiment. As shown in Fig. 5, the projector 700 includes a CPU 701, a ROM 702, a RAM 703, a media I / F 707, an operation unit 708, a power switch 709, a bus line 710, a network I / F 711, an LED (Light Emitting Diode) drive circuit 714, an LED light source 715, a projection device 716, a projection lens 717, an external device connection I / F 718, a fan drive circuit 719, and a cooling fan 720.

[0040] Of these, the CPU 701 controls the overall operation of the projector 700. The ROM 702 stores programs used to drive the CPU 701. The RAM 703 is used as a work area for the CPU 701. The media I / F 707 controls reading and writing (storing) of data to a recording medium 706 such as a flash memory.

[0041] The operation unit 708 has various keys, buttons, LEDs, etc., and is used by the user to perform various operations other than turning on / off the power of the projector 700. For example, the operation unit 708 accepts instruction operations such as operations to adjust the size of the projected image, operations to adjust the color tone, operations to adjust the focus, operations to adjust the keystone, etc., and outputs the contents of the accepted operations to the CPU 701.

[0042] The power switch 709 is a switch for switching ON / OFF the power supply of the projector 700. The bus line 710 is an address bus, a data bus, or the like for electrically connecting the components such as the CPU 701 shown in Fig. 5. The network I / F 711 is an interface for data communication using a communication network 9 such as the Internet.

[0043] The LED drive circuit 714 controls turning on and off the LED light source 715 under the control of the CPU 701. When the LED light source 715 is turned on under the control of the LED drive circuit 714, it irradiates the projection light to the projection device 716. The projection device 716 modulates the projection light from the LED light source 715 by a spatial light modulation method based on image data provided via an external device connection I / F 718 or the like, and projects the modulated light as an image onto the projection surface of the screen through a projection lens 717. As the projection device 716, for example, a liquid crystal panel or a DMD (Digital Micromirror Device) or the like is used.

[0044] The LED drive circuit 714, LED light source 715, projection device 716, and projection lens 717 function as a whole as a projection section (projection means) that projects a projection image onto a projection surface based on image data.

[0045] The external device connection I / F 718 is directly connected to a PC (Personal Computer) and receives control signals and image data from the PC. The external device connection I / F 718 is also an interface for connecting various external devices (such as a stick PC 730). The fan drive circuit 719 is connected to the CPU 701 and the cooling fan 720, and drives / stops the cooling fan 720 based on a control signal from the CPU 701. The cooling fan 720 rotates to exhaust air from inside the projector 700 and cool the inside of the projector 700.

[0046] Furthermore, when the CPU 701 is supplied with power from the power source, it starts up according to a control program previously stored in the ROM 702, and sends a control signal to an LED drive circuit 714 to turn on an LED light source 715, and sends a control signal to a fan drive circuit 719 to rotate a cooling fan 720 at a predetermined rated speed. When the supply of power from the power supply circuit of the projector 700 begins, the projection device 716 enters a state in which it can display an image, and furthermore, power is supplied from the power supply circuit to various other components. When the power switch 709 of the projector 700 is turned OFF, a power OFF signal is sent from the power switch 709 to the CPU 701.

[0047] When the CPU 701 detects the power OFF signal, it sends a control signal to the LED drive circuit 714 to turn off the LED light source 715. After that, when a predetermined time has elapsed, the CPU 701 sends a control signal to the fan drive circuit 719 to stop the cooling fan 720, ends its own control process, and finally sends an instruction to the power supply circuit to stop the supply of power from the power source.

[0048] <Functional configuration> The information processing device 10 according to the present embodiment is realized by, for example, a functional configuration as shown in Fig. 6. Fig. 6 is a functional configuration diagram of an example of the information processing device according to the present embodiment. Note that, in the functional configuration of Fig. 6, configurations that are not necessary for the description of the present embodiment are appropriately omitted.

[0049] The information processing device 10 in Figure 6 has a receiving unit 50, a user detection unit 52, a speech volume measurement unit 54, a speech content recognition unit 56, an environmental production unit 58, a video output control unit 60, a sound output control unit 62, a light control unit 64, an odor output control unit 66, and a memory unit 68.

[0050] The receiving unit 50 receives signals for detecting the position information of the user from the sensor devices 14 and 36. The receiving unit 50 receives the image capturing results of the camera 18. The receiving unit 50 also receives the output signal of the user's voice converted by the microphone 22 of the operation device 20, and the user's operation signal accepted by the operation button 26.

[0051] The user detection unit 52 detects the position information of a user in the conference room or the position information of a user passing through the approach from a signal for detecting the user's position information received from the sensor device 14 or 36. The speech amount measurement unit 54 measures the speech amount of each user in the conference room from an output signal of the user's voice received from the operation device 20.

[0052] The speech content recognition unit 56 performs voice recognition on the contents of speech of each user in the conference room from the output signal of the user's voice received from the operation device 20. The environment production unit 58 determines the environment production of the conference room (digital alcohol space) and the approach production of the approach based on the user's position information detected by the user detection unit 52, the user's speech volume measured by the speech volume measurement unit 54, and the user's speech content voice-recognized by the speech content recognition unit 56.

[0053] The environmental production unit 58 controls the video output control unit 60, the sound output control unit 62, the light control unit 64, and the odor output control unit 66 so that the determined environmental production and approach production are executed. The video output control unit 60 controls the environmental production by the video displayed by the video display device 12 based on the control by the environmental production unit 58. The sound output control unit 62 controls the environmental production and approach production by the sound output by the speakers 16 and 32 based on the control by the environmental production unit 58. The light control unit 64 controls the approach production by the light output by the light 30 based on the control by the environmental production unit 58. The odor output control unit 66 controls the approach production by the odor output by the odor output device 34 based on the control by the environmental production unit 58.

[0054] The storage unit 68 stores, for example, a table as shown in Fig. 7. Fig. 7 is a configuration diagram of an example of a table used in the information processing system according to this embodiment. Note that the table does not necessarily have to be in a table format as shown in Fig. 7, and it is sufficient if similar information can be stored and managed.

[0055] FIG. 7(a) is a configuration diagram of an example of an image display device information storage unit. The image display device information management storage unit manages items such as an image display device ID in association with wall and floor projection range information. The image display device ID is an example of identification information for identifying the image display device 12. The wall projection range information is information indicating which range of which wall in the conference room each image display device 12 can project onto. The floor projection range information is information indicating which range of the floor in the conference room each image display device 12 can project onto. In addition, when the image display device 12 is a projector, it is sufficient that at least either the wall or floor projection range information is associated with the image display device ID.

[0056] When the video display device 12 is a display, wall and floor display range information is associated with the video display device ID and managed in the video display device information storage unit. The wall display range information is information indicating which wall and which range each display installed in the conference room can display on. The floor display range information is information indicating which floor and which range each display installed in the conference room can display on. When the video display device 12 is a display, it is sufficient that at least either the wall or floor display range information is associated with the video display device ID.

[0057] 7(b) is a diagram showing an example of a configuration of a user information storage unit. The user information storage unit has items such as a user ID, a tag ID, location information, and speech state information. The user ID is an example of identification information for identifying a user. The tag ID is an example of identification information for identifying the tag 24 of the operation device 20. The location information is information on the location of a detected user.

[0058] The speech state information is information for distinguishing between a speaking user and a non-speaking user, which will be described later. The speech state information "True" indicates that the user is speaking, which will be described later. Also, the speech state information "False" indicates that the user is not speaking, which will be described later. Note that the location information is updated every time the user's location information is detected, so the latest location information within the space is associated with the user ID and managed.

[0059] Also, Fig. 7(c) is a configuration diagram of an example of a log storage unit. The log storage unit stores the user information shown in Fig. 7(b) as log information from the start to the end of the conference. The log storage unit has items such as date and time, user ID, tag ID, location information, speech state information, and speech amount information. Note that the speech amount information indicates the speech amount measured by the speech amount measurement unit 54, but the size of the speech amount may be indicated by a level as in Fig. 7(c) or may be indicated by a numerical value such as a measured value.

[0060] Also, for example, in the case of the log storage unit of FIG. 7(c), it can be seen that "userA" and "userB" had already participated at "17:00", and that "userC" joined late at "17:01".

[0061] <Processing> The information processing system 1 according to this embodiment realizes a digital alcohol space, for example, in the procedure shown in Fig. 8. Fig. 8 is a flowchart showing an example of the processing procedure of the information processing system according to this embodiment.

[0062] In the information processing system 1 according to the present embodiment, when a user passing through the approach is detected in step S10, the process proceeds to step S12, where the environmental production unit 58 controls the light control unit 64, the sound output control unit 62, and the odor output control unit 66, respectively, to perform an approach production process in which the light 30, the speaker 32, and the odor output device 34 on the approach reset the user's feelings with light, sound, and scent. As the approach production process, for example, a production process for lowering the amount of light on the approach 30, a production process for outputting environmental music with a relaxing effect by the speaker 32, and a production process for outputting a scent with a relaxing effect by the odor output device 34 are executed. In addition, for example, in the approach production process, a user may be identified in a passage through which a user passes when entering or leaving the digital alcohol space, and the approach production process may be made different for each user. In such a case, user information that associates a user ID with the contents of the approach production process is managed in advance, and the approach production process can be made different for each user by determining the approach production process based on the identified user ID and the managed user information. The approach performance process may include at least one of light, sound, and scent performance processes, and may involve controlling at least one of the light 30, speaker 32, and scent output device 34 on the approach.

[0063] Furthermore, in the information processing system 1 according to this embodiment, when a user in the conference room is detected in step S14, the process proceeds to step S16, where the video display device 12 and the speaker 16 in the conference room are controlled to perform an environmental production process using video and sound as described below. The environmental production process includes, for example, spotlight processing for a user who is speaking, spotlight processing according to the distance between users, and sound output processing according to the distance between users, as described below.

[0064] In the following, as examples of the environmental production processing using video and audio, a spotlight processing for a user who is speaking, a spotlight processing according to the distance between users, a sound output processing according to the distance between users, and a report processing will be described. Note that the environmental production processing according to this embodiment is not limited to the above processing examples.

[0065] For example, the process may be one that judges the team status of users in a conference room based on information representing the status of multiple users in the conference room, such as a signal notified from sensor device 14, and shifts the team status in an appropriate direction based on the judgment result.

[0066] Spotlight processing for users in speaking states Here, an example of spotlight processing for a user in an utterance state will be described with reference to Fig. 9 and Fig. 10. Fig. 9 is an image diagram showing an example of spotlight processing for a user in an utterance state. Fig. 10 is a flowchart showing an example of a procedure for spotlight processing for a user in an utterance state.

[0067] The spotlight processing for a user in an utterance state is, for example, processing for dividing a plurality of users in a conference room into users who are uttering and users who are not uttering, and shining a spotlight on an area 1000 on a wall behind the speaking user as seen by the users who are not uttering, as shown in Fig. 9. Note that the spotlight processing may be processing for making the area 1000 brighter than the areas other than the area 1000, or processing for making the color of the area 1000 more noticeable than the colors of the areas other than the area 1000. Furthermore, the spotlight processing for a user in an utterance state may be processing for shining a spotlight on an area at the feet of the speaking user.

[0068] In a conference room with an environment created as shown in Fig. 9, users other than the one currently speaking can visually distinguish the one currently speaking, so it is expected that the other users will pay attention to the one currently speaking. Note that Fig. 9 shows one image display device 12 out of multiple image display devices 12 capable of projecting images onto the walls and floor of the conference room, and the other image display devices 12 are omitted.

[0069] When there are multiple non-speaking users, a spotlight may be shone on each of the areas 1000 on the wall behind the speaking user as seen from the perspective of each of the non-speaking users, or one of the areas 1000 may be selected to be shone with a spotlight.

[0070] Furthermore, if there are multiple image display devices 12 that can shine a spotlight on area 1000, the spotlight may be shone on area 1000 from multiple image display devices 12, or the image display device 12 to shine the spotlight on may be selected according to priority.

[0071] The spotlight process shown in FIG. 9 can be realized, for example, by the process procedure in FIG. 10. In step S20, when the speech amount measurement unit 54 measures the speech amount of the user in the conference room, the environment production unit 58 proceeds to step S22. In step S22, the environment production unit 58 identifies the user who is speaking based on the speech amount of the user measured by the speech amount measurement unit 54. Here, the user who is speaking is the user who has measured the speech amount if there is only one user. If there are multiple users who have measured the speech amount, the user who is speaking may be further identified based on a predetermined logic from the user who has measured the speech amount. The predetermined logic includes, for example, a logic in which the user who has the largest measured speech amount is the user who is speaking, or a logic in which the user with a high priority associated with a user ID is the user who is speaking, but may be another logic.

[0072] Proceeding to step S24, the environment production unit 58 acquires the position information of the speaking user and the non-speaking user from, for example, the user information storage unit of Fig. 7(b). Proceeding to step S26, the environment production unit 58 controls the video output control unit 60 based on the position information of the speaking user and the position information of the non-speaking user so that the area 1000 behind the speaking user as seen by the non-speaking user is brightly illuminated by the video display device 12. Note that, when the video display device 12 is a display, the video output control unit 60 controls so that the display content displayed by the display is brightened. Note that it is not necessary for the entire display content to be brightened, and it is sufficient to control so that at least a part of the display content is brightened.

[0073] The ambience production unit 58 repeats the process of steps S24 to S30 until the user in the conference room finishes speaking or the currently speaking user changes. When the currently speaking user changes, the ambience production unit 58 returns to step S22, and after identifying the currently speaking user, repeats the process of steps S24 to S30. When the user in the conference room finishes speaking, the ambience production unit 58 ends the process of the flowchart in FIG. 10.

[0074] Spotlight processing according to the distance between users Here, an example of spotlight processing according to the distance between users will be described with reference to Figs. 11 to 13. Fig. 11 is an image diagram showing an example of spotlight processing according to the distance between users. Fig. 12 is an image diagram showing a specific example of spotlight processing according to the distance between users. Fig. 13 is a flowchart showing an example of a spotlight processing procedure according to the distance between users.

[0075] The spotlight process according to the distance between users is a process of changing the size of a spotlight area 1000 that is illuminated on the feet of users based on the distance between multiple users in a conference room, as shown in Fig. 11. Note that the spotlight process may be a process of making the color of the area 1000 more noticeable than the color of the area other than the area 1000, in addition to a process of making the area 1000 brighter than the color of the area other than the area 1000.

[0076] The process of changing the size of the area 1000 of the spotlight shining on the feet of the users based on the distance between the users in the conference room controls the video output control unit 60 to make the area 1000 larger, for example, as shown in Fig. 12(A) as the distance between the users increases. Also, the process of changing the size of the area 1000 of the spotlight shining on the feet of the users based on the distance between the users in the conference room controls the video output control unit 60 to make the area 1000 smaller, for example, as shown in Fig. 12(B) as the distance between the users decreases.

[0077] Note that the example in Figure 12 is just one example, and the video output control unit 60 may be controlled to make the area 1000 smaller the greater the distance between the users, and to make the area 1000 larger the closer the distance between the users.

[0078] The spotlight processing shown in Figures 11 and 12 can be realized, for example, by the procedure shown in Figure 13. In step S40, when the user detection unit 52 detects multiple users in the conference room, the environment production unit 58 proceeds to step S42.

[0079] In step S42, the ambience production unit 58 acquires position information of the detected users from, for example, the user information storage unit in Fig. 7(b). In step S44, the ambience production unit 58 calculates the distances between the users in the conference room from the position information of the detected users.

[0080] Proceeding to step S46, the environmental production unit 58 controls the video output control unit 60 to change the size of the area 1000 that illuminates the feet of the users, for example, as shown in Fig. 12, based on the calculated distance between the users. The environmental production unit 58 repeats the processes of steps S42 to S48 until the conference ends. When the users are no longer detected and the conference ends, the environmental production unit 58 ends the process of the flowchart in Fig. 13.

[0081] Fig. 14 is a flowchart of another example of the spotlight processing procedure according to the distance between users. The spotlight processing according to the distance between users shown in Fig. 14 is a process of changing the amount of light in an area 1000 that illuminates the feet of users based on the distance between multiple users in a conference room.

[0082] The process of changing the amount of light in area 1000 that illuminates the feet of users based on the distance between multiple users in a conference room controls the video output control unit 60 so that, for example, the greater the distance between the users, the lower the amount of light in area 1000, and the closer the distance between the users, the higher the amount of light in area 1000.

[0083] In addition, the process of changing the amount of light in area 1000 that illuminates the feet of users based on the distance between multiple users in the conference room may control the video output control unit 60 so that, for example, the greater the distance between the users, the greater the amount of light in area 1000, and the closer the distance between the users, the lower the amount of light in area 1000.

[0084] In step S60, when the user detection unit 52 detects multiple users in the conference room, the environmental production unit 58 proceeds to step S62. In step S62, the environmental production unit 58 acquires position information of the detected multiple users from, for example, the user information storage unit of Fig. 7(b). In step S64, the environmental production unit 58 calculates the distance between the multiple users in the conference room from the position information of the detected multiple users.

[0085] Proceeding to step S66, the environmental production unit 58 controls the video output control unit 60 so as to change the amount of light in the area 1000 illuminating the feet of the users, based on the calculated distance between the users. The environmental production unit 58 repeats the processes of steps S62 to S68 until the conference ends. When the users are no longer detected and the conference ends, the environmental production unit 58 ends the process of the flowchart in FIG.

[0086] "Sound output processing according to the distance between users" Next, an example of sound output processing according to the distance between users will be described. Fig. 15 is a flowchart of an example of a sound output processing procedure according to the distance between users. The sound output processing according to the distance between users shown in Fig. 15 is a processing for changing the volume of sound output to a conference room based on the distance between multiple users in the conference room.

[0087] The process of changing the volume of the sound output to the conference room based on the distance between multiple users in the conference room controls the sound output control unit 62 so that, for example, the greater the distance between the users, the higher the volume of the sound output, and the closer the distance between the users, the lower the volume of the sound output.

[0088] In addition, the process of changing the volume of the sound output to the conference room based on the distance between multiple users in the conference room may control the sound output control unit 62 so that, for example, the greater the distance between the users, the lower the volume of the sound output, and the closer the distance between the users, the higher the volume of the sound output.

[0089] In step S80, when the user detection unit 52 detects multiple users in the conference room, the environmental production unit 58 proceeds to step S82. In step S82, the environmental production unit 58 acquires position information of the detected multiple users from, for example, the user information storage unit of Fig. 7(b). In step S84, the environmental production unit 58 calculates the distance between the multiple users in the conference room from the position information of the detected multiple users.

[0090] Proceeding to step S86, the environmental production unit 58 controls the sound output control unit 62 so as to change the volume of the sound output from the speaker 16 in the conference room based on the calculated distance between the users. The environmental production unit 58 repeats the processes of steps S82 to S94 until a user in the conference room speaks or the conference ends.

[0091] When a user in the conference room speaks, the environment production unit 58 proceeds to step S90, controls the sound output control unit 62 to lower the volume of the sound output, and maintains the lowered state of the volume of the sound output until the user in the conference room stops speaking. When the user in the conference room stops speaking, the volume of the sound output returns to the volume in step S86. Then, when the user is no longer detected and the conference ends, the environment production unit 58 ends the processing of the flowchart in FIG.

[0092] Report Processing Next, an example of a report process in which an environment is produced according to an operation received from a user and the user's location information, and a report is output after the end of a meeting is described. Fig. 16 is a flowchart showing an example of a report output process procedure of the information processing system according to this embodiment.

[0093] In step S100, the environment production unit 58 accepts a conference time (activity time) designation from the user and starts the conference. The environment production unit 58 repeats the processes of steps S102 to S112 until it accepts an operation from the user, a user in the conference room speaks, or the conference time ends.

[0094] When an operation from the user is accepted, the environmental production unit 58 proceeds to step S104, and acquires the user's position information from, for example, the user information storage unit of Fig. 7(b). In step S106, the environmental production unit 58 performs environmental production by the image display device 12 and the speaker 16 in accordance with the operation accepted from the user and the user's position information.

[0095] For example, in step S106, based on the operation received from the user and the user's position information, it may be possible to project information / video content onto the walls or floor of the digital alcohol space, select the information / video content projected onto the walls or floor, move or zoom in / out the selected information / video content, and so on.

[0096] In addition, when a user in the conference room speaks, the environment production unit 58 proceeds to step S110, and projects additional information and video content onto the walls and floor of the digital alcohol space in accordance with the content of the user's speech recognized by voice recognition from the output signal of the user's voice received from the operation device 20.

[0097] The additional projection onto the walls and floor of the digital alcohol space in step S110 does not necessarily have to be information or video content corresponding to the user's speech, but may be random information or video content. By additionally projecting random information or video content onto the walls and floor of the digital alcohol space, the environment production unit 58 can control the concentration and divergence of the thoughts of the users participating in the conference.

[0098] Furthermore, the information processing system 1 according to this embodiment may display different content (for example, a right-brain display or a left-brain display) depending on the walls or floor of the digital alcohol space to which the additional projection is made, even if the same information / video content is displayed. This feedback may be used to control the concentration and divergence of thoughts of users participating in a meeting.

[0099] When the user is no longer detected and the conference ends, the environment production unit 58 proceeds to step S114 and outputs the state of the conference, such as the user's movement trajectory in the conference room, as a report. For example, when outputting the user's movement trajectory in the conference room as a report, the information in the log storage unit in FIG. 7(c) is referenced, and a drawing process is performed to map the position information of each user from the start to the end of the conference as a trajectory onto image data or video data of the space in which the user communicated, thereby creating image data or video data that visualizes the user's movement trajectory. The image data or video data of the space may be data captured by the camera 18 installed in the space, or may be image data or video data stored in advance. In this way, the environment production unit 58 can output the created image data or video data as a report.

[0100] In addition, by referring to the speech state managed in the log storage unit, when the position information of the date and time in the speech state is drawn as a trajectory, the trajectory may be drawn in a different color or shape from when the user is not in the speech state, so that the trajectory can be distinguished. Similarly, the color or shape of the trajectory to be drawn may be changed according to the amount of speech measured.

[0101] By referring to such a report, it is possible to look back on the extent to which the environmental design stimulated communication between multiple users.

[0102] The present invention is not limited to the above-mentioned specifically disclosed embodiment, and various modifications and changes are possible without departing from the scope of the claims. Note that the information processing system 1 described in the present embodiment is only one example, and it goes without saying that there are various system configuration examples according to the application and purpose.

[0103] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and a device such as an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), or a conventional circuit module designed to execute each function described above.

[0104] The devices described in the example are merely one of several computing environments for implementing the embodiments disclosed herein. In one embodiment, the information processing apparatus 10 includes a plurality of computing devices, such as a server cluster. The plurality of computing devices are configured to communicate with each other via any type of communication link, including a network 40, a shared memory, and the like, and perform the processes disclosed herein.

[0105] Furthermore, the disclosed processing steps can be combined in various ways in the information processing device 10. The elements of the information processing device 10 may be integrated into one device or may be separated into multiple devices. [Explanation of symbols]

[0106] 1. Information Processing Systems 10. Information processing device 12 Image display device 14 Sensor device 16 Speakers 18 Camera 20 Operation Device 22. Mike 24 Tags 26 Operation buttons 30 Light 32 Speaker 34 Odor Output Device 36 Sensor Device 50 Receiving unit 52 User detection unit 54 Speech volume measurement unit 56 Speech content recognition unit 58 Environmental Direction Department 60 Video output control section 62 Sound output control section 64 Light control section 66 Odor output control unit 68 Memory section [Prior art documents] [Patent documents]

[0107] [Patent Document 1] JP 2016-32205 A

Claims

1. An information processing system having an image display device that displays an image installed in a space, a sensor device that is carried by a user in the space and outputs a signal for detecting position information of the user, and an information processing device that is connected to the image display device and the sensor device so as to be capable of data communication, The information processing device includes: a user information storage means for storing position information of each of a plurality of users in the space, the position information being detected based on signals output from the sensor devices carried by the users in the space, in association with the users; an environmental production means for controlling an environmental production that supports communication between the plurality of users by using images displayed by the image display device, based on position information of each of the plurality of users; An information processing system having the above configuration.

2. The information processing device includes: a speech volume measuring means for measuring the speech volume of each of the plurality of users based on an output signal from a microphone; The environmental production means identifies a user who is currently speaking from among the plurality of users based on a result of the measurement by the speech amount measurement means, and controls the environmental production so that a user who is currently speaking can be visually distinguished from a user who is not currently speaking.

2. The information processing system according to claim 1,

3. The environmental production means controls the environmental production so that an area behind the speaking user as seen by other users is brighter than other areas.

3. The information processing system according to claim 2,

4. The environmental production means controls a size or a light amount of an image displayed by the image display device based on a distance between the plurality of users calculated from position information of each of the plurality of users.

4. The information processing system according to claim 1, wherein:

5. The information processing device includes: a sound output control means for outputting sounds for the plurality of users from a speaker; The environmental production means controls the acoustic production of the sound based on the distance between the plurality of users.

5. The information processing system according to claim 4,

6. the user information storage means stores position information of each of the plurality of users in the space in association with the user based on radio waves from an operation device carried by each of the plurality of users; 6. The information processing system according to claim 1, wherein:

7. The environmental production means changes the environmental production by the image displayed by the image display device on one or more surfaces that divide the space.

7. The information processing system according to claim 1, wherein:

8. The information processing device includes: The system further includes a speech content recognition unit that recognizes speech contents of the plurality of users based on an output signal from a microphone, The environmental production means controls the environmental production according to the contents of the utterances of the plurality of users based on a result of the voice recognition by the utterance content recognition means.

8. The information processing system according to claim 1, wherein:

9. the user information storage means stores changes in position information of each of a plurality of users in the space as log information, The environment production means outputs a report according to a trajectory of position information of each of a plurality of users in the space based on the log information.

9. The information processing system according to claim 1, wherein:

10. An image display device that displays an image installed in a space, and an information processing device that is connected to a sensor device that is carried by a user in the space and outputs a signal for detecting position information of the user, the information processing device being capable of data communication with the sensor device, a user information storage means for storing position information of each of a plurality of users in the space, the position information being detected based on signals output from the sensor devices carried by the users in the space, in association with the users; an environmental production means for controlling an environmental production that supports communication between the plurality of users by using images displayed by the image display device, based on position information of each of the plurality of users; An information processing device having the above configuration.

11. An image display device that displays an image installed in a space, and an information processing device that is connected to a sensor device that is carried by a user in the space and outputs a signal for detecting the position information of the user, so as to be capable of data communication; a user information storage means for storing position information of each of a plurality of users in the space, the position information being detected based on signals output from the sensor devices carried by the users in the space, in association with the users; an environmental production means for controlling an environmental production that supports communication between the plurality of users by using images displayed by the image display device, based on position information of each of the plurality of users; A program to function as a

12. An information processing method for an information processing system having an image display device that displays an image installed in a space, a sensor device that is carried by a user in the space and outputs a signal for detecting position information of the user, and an information processing device that is connected to the image display device and the sensor device so as to be capable of data communication, storing position information of each of a plurality of users in the space, the position information being detected based on signals output from the sensor devices carried by the users in the space, in association with the users; a step of controlling an environmental presentation that supports communication between the plurality of users by using images displayed by the image display device, based on position information of each of the plurality of users; An information processing method comprising the steps of:

13. A room comprising an information processing system according to any one of claims 1 to 9.

Images