Image display method, image display system, and program

Abstract

To provide an image display method, an image display system, and a program that make it easy for a user to notice a displayed first image and provide a feeling that the user himself / herself has noticed the image to enhance a sense of immersion.SOLUTION: The information processing device 30 determines whether or not the action of the user P to the object 100 disposed in the space S satisfies the first condition based on the detection result of the at least one sensor 10, and when determining that the action of the user P satisfies the first condition, displays the first image 200 in the first region of the space S that can be included in the visual field of the user by the action.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image display method, an image display system, and a program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, systems have been known that display images in a space to give a user a sense of immersion in the space.

[0003] For example, the video display system disclosed in Patent Document 1 generates an image of a virtual camera at a specific camera position and displays it on a display screen. The video display system detects the position of a real object on the display screen using a sensor, moves the camera position, and makes an object appear in the image based on the position of the real object on the display screen. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-219820 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the video display system of Patent Document 1 makes objects appear in the video based on the position of a real person, regardless of which part of the video displayed on the display screen the person is looking at. When the person is moving around a little, the objects appear less. As a result, it is difficult for the person to notice the objects, making it difficult to achieve a sense of immersion. [Means for solving the problem]

[0006] The image display method disclosed herein is an image display method in which an information processing device determines whether a user's action relative to an object placed in a space satisfies a first condition based on the detection result of at least one sensor, and if it is determined that the user's action satisfies the first condition, displays a first image in a first area of ​​the space that may be included in the user's field of view due to the action.

[0007] The image display system of the present disclosure is an image display system including at least one sensor, at least one display device, and an information processing device that performs the following steps: determining whether a user's action relative to an object placed in space satisfies a first condition based on the detection result of the at least one sensor; and, if it is determined that the user's action satisfies the first condition, displaying a first image in a first area of ​​the space that may be included in the user's field of view due to the action.

[0008] The program disclosed herein causes a processor to determine whether a user's action relative to an object placed in a space satisfies a first condition based on the detection results of at least one sensor, and, if it is determined that the user's action satisfies the first condition, display a first image in a first area of ​​the space that may be included in the user's field of view due to the action. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing the system configuration of an image display system. [Figure 2] FIG. 1 is a block diagram showing the configuration of a projector. [Figure 3] FIG. 1 is a block diagram showing the configuration of an information processing device. [Figure 4] 10 is a flowchart showing the operation of the information processing device. [Figure 5] FIG. 10 is a diagram showing the state of the space in the first embodiment before the first condition is met. [Figure 6] FIG. 10 is a diagram showing the state of the space in the first embodiment after the first condition is met. [Figure 7] FIG. 10 is a diagram showing the state of the space in the second embodiment before the first condition is met. [Figure 8] FIG. 10 is a diagram showing the state of the space in the second embodiment after the first condition is met. [Figure 9] FIG. 10 is a diagram showing the state of the space in the third embodiment before the first condition is met. [Figure 10] FIG. 10 is a diagram showing the state of the space in the third embodiment after the first condition is met. [Figure 11] FIG. 10 is a diagram showing the state of the space in the fourth embodiment before the first condition is met. [Figure 12] FIG. 10 is a diagram showing the state of the space in the fourth embodiment after the first condition is met. DETAILED DESCRIPTION OF THE INVENTION

[0010] [1. System configuration of image display system] Hereinafter, embodiments will be described with reference to the accompanying drawings. FIG. 1 is a diagram showing the system configuration of an image display system 1. As shown in FIG. The image display system 1 is a system in which a plurality of sensors 10, a plurality of projectors 20, and an information processing device 30 are connected via a network 5. While FIG. 1 shows a configuration in which the image display system 1 includes a plurality of sensors 10 and a plurality of projectors 20, the image display system 1 may include only one sensor 10. The image display system 1 may also include only one projector 20. One or more projectors 20 are an example of at least one display device. The display device is not limited to a projector, and may be a direct-view type panel display device, or a combination thereof.

[0011] The network 5 is, for example, a network such as a LAN (Local Area Network) or a WAN (Wide Area Network).

[0012] 1 shows a case where M sensors, ie, sensor 10A, sensor 10B, . . . , sensor 10M, are provided as the plurality of sensors 10. M is an arbitrary natural number. The multiple sensors 10 include, for example, cameras and distance sensors. The multiple sensors 10 may also include a receiver that receives radio waves for short-range wireless communication. The multiple sensors 10 may be configured with only cameras or only distance sensors.

[0013] The camera may be an R (red), G (green), and B (blue) camera capable of capturing images in three wavelength bands: red, green, and blue, or an infrared camera capable of capturing images in dark places or at night using infrared rays. The distance sensor is, for example, a ToF (Time of Flight) sensor that measures the distance to an object using laser light or ultrasonic waves. Hereinafter, when referring to one of the sensors 10, it will be referred to as sensor 10. Also, a stereo camera may be used instead of the distance sensor.

[0014] The multiple projectors 20 include N projectors: projectors 20A, 20B, ..., 20N. N is any natural number, and M and N may be the same value or different values. Hereinafter, any projector included in the multiple projectors 20 will be referred to as projector 20.

[0015] The information processing device 30 is, for example, a computer device of a notebook type or a desktop type, etc. The information processing device 30 may also be configured as a tablet type computer device.

[0016] 1 shows an example in which multiple sensors 10, multiple projectors 20, and an information processing device 30 are connected by wire via a network 5, but some or all of the connections between the multiple sensors 10, multiple projectors 20, and information processing device 30 may be wireless. For example, a Wi-Fi router may be installed to connect the multiple sensors 10, multiple projectors 20, and information processing device 30 to a Wi-Fi network. Alternatively, the multiple sensors 10 and information processing device 30 may be connected by short-range wireless communication such as Bluetooth. Wi-Fi and Bluetooth are registered trademarks.

[0017] [2. Projector configuration] 2 is a block diagram showing the configuration of the projector 20. The projectors 20A, 20B, . . . , 20M have substantially the same configuration. Therefore, the configuration of the projector 20 will be described.

[0018] The projector 20 includes a first communication interface 21, an image processing unit 22, a frame memory 23, an image projection unit 24, and a first control unit 25. Hereinafter, interface will be abbreviated as I / F. The first communication I / F 21, the image processing unit 22, and the image projection unit 24 are connected to the first control unit 25 and operate under the control of the first control unit 25.

[0019] The first communication I / F 21 is connected to the information processing device 30 via the network 5, and performs mutual data communication with the information processing device 30. The first communication I / F 21 is a wired interface having a connection terminal such as a USB (Universal Serial Bus) connector or an Ethernet connector, and an interface circuit. Ethernet is a registered trademark. The first communication I / F 21 may also be a wireless communication interface.

[0020] The image processing unit 22 is connected to a frame memory 23. The frame memory 23 includes a plurality of banks. Each bank has a storage capacity sufficient to store one frame of display image. The frame memory 23 is configured, for example, with an SDRAM (Synchronous Dynamic RAM). The image processing unit 22 expands the display image input from the first communication I / F 21 in the frame memory 23.

[0021] The image processing unit 22 performs image processing on the display image expanded in the frame memory 23. The image processing performed by the image processing unit 22 includes, for example, resolution conversion processing or resizing processing, distortion correction, shape correction processing, digital zoom processing, and adjustment of image color and brightness. The image processing unit 22 executes processing specified by the first control unit 25, and, if necessary, performs processing using parameters input from the first control unit 25. Of course, the image processing unit 22 can also execute a combination of multiple image processing operations described above.

[0022] The image processing unit 22 and the frame memory 23 are configured, for example, by integrated circuits. Examples of integrated circuits include large-scale integration (LSI), application-specific integrated circuits (ASIC), programmable logic devices (PLD), field-programmable gate arrays (FPGA), and system-on-a-chips (SoC). An analog circuit may be included as part of the integrated circuit configuration, or the first control unit 25 may be configured in combination with an integrated circuit.

[0023] The image projection unit 24 includes a light source 241 , a light modulation device 243 , and an optical unit 245 .

[0024] The light source 241 includes a discharge light source lamp such as an ultra-high pressure mercury lamp or a metal halide lamp, or a solid-state light source such as a light-emitting diode or a semiconductor laser.

[0025] The light modulation device 243 is a light modulation element that modulates the light emitted from the light source 241 and includes a transmissive liquid crystal panel in which liquid crystal is sealed between a pair of transparent substrates. The liquid crystal panel is not shown. The liquid crystal panel includes a panel region consisting of a plurality of pixels arranged in a matrix. The light modulation device 243 applies a drive voltage corresponding to the display image input from the image processing unit 22 to each pixel in the panel region, and changes the light transmittance of each pixel to the transmittance corresponding to the display image. The light emitted from the light source 241 is modulated as it passes through the liquid crystal panel, and image light corresponding to the display image is generated.

[0026] The light modulation element included in the light modulation device 243 is not limited to a transmissive liquid crystal panel, but may be, for example, a reflective liquid crystal panel or a DMD (Digital Micromirror Device). In this case, the DMD is an example of a display panel.

[0027] The optical unit 245 includes a projection lens (not shown) and projects the image light modulated by the light modulation device 243 onto a projection target in an enlarged form. Projection targets onto which the projector 20 projects the image light include, for example, a screen, a wall, floor, or ceiling surface in a room, etc. The image displayed on the projection target by the projector 20 may be a moving image or a still image.

[0028] The first control unit 25 is a computer device including a first storage unit 26 and a first processor 27.

[0029] The first storage unit 26 includes a RAM (Random Access Memory) and a ROM (Read Only Memory). The RAM is used for temporary storage of various data, and the ROM stores a control program 265 that controls the operation of the projector 20, various setting information, and the like.

[0030] The first processor 27 is an arithmetic processing device configured with a CPU (Central Processing Unit) and an MPU (Micro Processor Unit). The first processor 27 executes a control program 265 to control each unit of the projector 20. The first processor 27 may be configured with a single processor or multiple processors. The first processor 27 may also be configured with a SoC (System-on-a-Chip) integrated with part or all of the first storage unit 26 or other circuits. The first processor 27 may also be configured with a combination of a CPU that executes programs and a DSP (Digital Signal Processing) that executes predetermined arithmetic processing. Furthermore, all of the functions of the first processor 27 may be implemented in hardware, or may be configured using a programmable device.

[0031] [3. Configuration of information processing device] FIG. 3 is a block diagram showing the configuration of the information processing device 30. The configuration of the information processing device 30 will be described with reference to FIG. The information processing device 30 includes a second communication I / F 31, an operation unit 32, a display unit 33, and a second control unit 35. The second communication I / F 31, the operation unit 32, and the display unit 33 are connected to the second control unit 35.

[0032] The second communication I / F 31 is a wired interface having a connection terminal such as a USB connector or an Ethernet connector, and an interface circuit, and performs data communication between the second communication I / F 31 and the multiple sensors 10 and the multiple projectors 20 via the network 5.

[0033] The operation unit 32 includes input devices such as a mouse and a keyboard, and receives operations from the user. The operation unit 32 outputs an operation signal corresponding to the received operation to the second control unit .

[0034] The display unit 33 includes a display panel such as a liquid crystal panel or an organic EL (Electro Luminescence) panel, and displays an image on the display panel under the control of the second control unit .

[0035] The second control unit 35 includes a second storage unit 36 ​​and a second processor 37.

[0036] The second storage unit 36 ​​includes a RAM and a ROM. The second storage unit 36 ​​also includes an auxiliary storage device such as an SSD (Solid State Drive) or an HDD (Hard Disk Drive). The RAM is used for temporary storage of various data and the like, and the ROM stores a control program 365 such as an OS (Operating System) and application programs that control the operation of the information processing device 30, as well as various setting information. The second storage unit 36 ​​also stores image data that is the source of the image light projected by the projector 20.

[0037] The second processor 37 is an arithmetic processing device configured with a CPU and an MPU. The second processor 37 executes a control program 365 to control each unit of the information processing device 30. The second processor 37 may be configured with a single processor or multiple processors.

[0038] The second control unit 35 determines whether or not the action of the user P with respect to the object 100 placed in the space S satisfies the first condition based on the detection result of at least one sensor 10.

[0039] Space S is a three-dimensional real space. A sensor 10 is installed in space S, and the sensor 10 detects the movements of user P within space S. Also, a projector 20 and walls that partition the space are installed in space S, and an image is displayed on the wall by the installed projector 20.

[0040] The object 100 is a real object placed in the space S, and is an object that attracts the attention of the user P. For example, the object 100 includes a natural object, an object that imitates a natural object, an object related to the image displayed by the projector 20, etc.

[0041] The first condition is a condition indicating that the user P has performed a predetermined action on the object 100. For example, the first condition includes a condition that the user P touches the object 100, the user P avoids the object 100, the user P approaches the object 100, or the user P moves away from the object 100.

[0042] The sensor 10 that detects the action of the user P relative to the object 100 includes, for example, an RGB camera, an infrared camera, a distance sensor, a communication device that performs short-range wireless communication, a touch sensor, a potential sensor, a proximity sensor, and the like.

[0043] For example, the second control unit 35 may analyze the image captured by the camera serving as the sensor 10 to detect actions of the user P, such as the user P touching the object 100, the user P moving away from the object 100, or the user P approaching within a first distance from the object 100.

[0044] Alternatively, a distance sensor may be installed on the object 100 as the sensor 10, and the second control unit 35 may acquire sensor data output by the distance sensor. Based on the sensor data from the distance sensor, the second control unit 35 may detect actions of the user P, such as the user P approaching within a first distance from the object 100, or the user P approaching within the first distance from the object 100 and then moving away from the object 100 by a second distance or more. The first distance and the second distance are preset distances, and the second distance is longer than the first distance.

[0045] Furthermore, a communication device that transmits and receives signals via short-range wireless communication as the sensor 10 may be installed or worn on the object 100 and the user P. The communication device installed on the object 100 transmits and receives signals to and from a communication device installed in a smart device worn by the user P, and outputs the radio wave intensity of the signal received from the smart device and the delay time from when the signal is output until when a response signal is received to the second control unit 35. The second control unit 35 detects the distance between the object 100 and the user P based on the radio wave intensity and delay time output by the communication device installed on the object 100. Furthermore, a distance sensor serving as the sensor 10 may be installed on the object 100, and the second control unit 35 may detect the distance between the object 100 and the user P based on sensor data from the distance sensor. The second control unit 35 may be configured to output a notification to a notification device such as a smart device worn by the user P through at least one of visual, auditory, and tactile means. For example, the second control unit 35 may change the display color of an image displayed on the display of the smart device, or change the output level of sound output by the smart device or the sound itself, based on the distance between the object 100 and the user P. Furthermore, the second control unit 35 may change the vibration intensity of the smart device based on the distance between the object 100 and the user P.

[0046] Furthermore, a touch sensor, potential sensor, or proximity sensor may be installed as the sensor 10 on the object 100. These sensors 10 output sensor data indicating the detection results to the second control unit 35. Based on the sensor data input from the sensors 10 by the second control unit 35, it is determined whether or not the user P has touched the object 100 or whether or not the user P has approached the object 100.

[0047] When the user P performs a predetermined action on the object 100 and the second control unit 35 determines that this action satisfies the first condition, the second control unit 35 displays the first image 200 in a first area that is an area that can be included in the visual field of the user P after the action. The first image 200 may be a moving image or a still image.

[0048] The area that can be included in the field of view refers to the area of ​​a display surface that partitions space S, or a display surface that is placed within space S, that comes into the field of view of user P who is viewing object 100 when user P performs an action such as touching or approaching object 100. For example, assume that object 100 is placed near a wall that constitutes space S. When user P approaches object 100 while looking at object 100 in order to touch or approach object 100, the wall or floor of space S where object 100 is placed nearby comes into user P's field of view. The wall or floor of space S that comes into user P's field of view corresponds to a region that can be included in the field of view. Furthermore, the first region is at least a portion of the wall or floor of space S that comes into user P's field of view.

[0049] Furthermore, the area that can be included in the field of view changes depending on the action taken by user P. For example, assume that an object 100 is present in the user P's direction of travel, and that user P takes a detour route around object 100 to avoid object 100. In this case, the walls that can be included in user P's field of view will differ depending on whether the detour route selected by user P is a detour route that goes around object 100 in a clockwise direction or a detour route that goes around object 100 in a counterclockwise direction.

[0050] When the second control unit 35 determines that the action of the user P satisfies the first condition, it displays the first image 200 in the first area. The first image 200 is, for example, an image or a part of an image that the user P wants to view, and the image may change to attract the user P's attention. For example, an image may be displayed before the user P performs an action that satisfies the first condition, and after the action that satisfies the first condition is detected, the image may be changed to display the first image 200.

[0051] FIG. 4 is a flowchart showing the operation of the second control unit 35. The operation of the second control unit 35 will be described with reference to the flowchart shown in FIG. First, the second control unit 35 acquires sensor data from the multiple sensors 10 (step S1). Next, the second control unit 35 detects the action of the user P relative to the object 100 based on the acquired sensor data (step S2).

[0052] Next, the second control unit 35 determines whether the detected action with respect to the object 100 satisfies a first condition (step S3). For example, the second control unit 35 detects, as an action that satisfies the first condition, an action such as the user P touching the object or the user P approaching the object 100 until the distance between the user P and the object 100 is equal to or less than a predetermined distance. The second control unit 35 may also detect, as an action that satisfies the first condition, an action such as the user P moving away from the object 100 until the distance between the user P and the object 100 is equal to or less than a predetermined first distance. The second control unit 35 may also detect, as an action that satisfies the first condition, an action such as the user P going around the object 100 clockwise or counterclockwise.

[0053] If the second control unit 35 determines that the detected action on the object 100 does not satisfy the first condition (step S3 / NO), the process returns to step S1 and acquires sensor data again. If the second control unit 35 determines that the detected action on the object 100 satisfies the first condition (step S3 / YES), the second control unit 35 displays the first image 200 in the first area (step S4). The first area may be any area that can be included in the field of view of the user P after the action on the object 100. The first image 200 may be a modified image that was displayed before the action of the user P that satisfies the first condition was detected (step S4).

[0054] 4. Example 1 Fig. 5 is a diagram showing the state of the space S1 of the first embodiment before the first condition is met, and Fig. 6 is a diagram showing the state of the space S1 of the first embodiment after the first condition is met. As shown in Figure 5, the space S1 is a space surrounded by three walls. When viewed from the drawing, the left wall is referred to as a left wall 51, the central wall is referred to as a central wall 52, and the right wall is referred to as a right wall 53. A left camera is installed as sensor 10A on the upper part of left wall surface 51, a central camera is installed as sensor 10B on the upper part of central wall surface 52, and a right camera is installed as sensor 10C on the upper part of right wall surface 53. In the explanations of Figures 5 and 6, the left camera will be referred to as "left camera 10A," the central camera will be referred to as "central camera 10B," and the right camera will be referred to as "right camera 10C."

[0055] 5 and 6 show an example in which three projectors 20, projectors 20A, 20B, and 20C, are installed in space S1. 5 and 6 show an example in which projectors 20A, 20B, and 20C are installed on the ceiling surface near left wall surface 51, center wall surface 52, and right wall surface 53, respectively, i.e., installed by ceiling-suspended installation.

[0056] 5 and 6 show an example in which the projectors 20A, 20B, and 20C are installed on the ceiling surface of the space S1, but the projectors 20A, 20B, and 20C may also be installed on the floor surface of the space S1. Also, the projectors 20A, 20B, and 20C may be wall-mounted on the left wall surface 51, the center wall surface 52, and the right wall surface 53, respectively.

[0057] 5 and 6, the projectors 20A, 20B, and 20C, the left camera 10A, the center camera 10B, and the right camera 10C are connected to an information processing device 30 (not shown). The information processing device 30 may be placed behind a decorative item (not shown) installed in the space S1 so as not to be seen by others, or may be placed behind the left wall surface 51, the center wall surface 52, or the right wall surface 53.

[0058] 5 and 6 show a pearl from a pearl oyster as the object 100. Although Fig. 5 and Fig. 6 show the case where the object 100 is a pearl from a pearl oyster, the object 100 may be, for example, a spherical ball or a piece of driftwood.

[0059] FIG. 6 is a diagram showing an image displayed when a user P touches the object 100. As shown in FIG. For example, when the second control unit 35 detects that the user P has touched the object 100 from an image captured by the left camera 10A, the center camera 10B, or the right camera 10C, the second control unit 35 determines that the first condition has been met. When the second control unit 35 determines that the first condition has been met, the second control unit 35 displays the first image 200 in a first area that may be included in the field of view of the user P. Furthermore, the second control unit 35 may determine that the first condition is met when it detects that the distance between the user P and the object 100 has become equal to or shorter than a first distance from an image captured by the left camera 10A, the center camera 10B, or the right camera 10C. When it determines that the first condition is met, the second control unit 35 displays the first image 200 in a first area that can be included in the field of view of the user P.

[0060] FIG. 6 shows an example in which the first area is the central wall surface 52 and the first image 200 is displayed on the central wall surface 52 by the projector 20B. The central wall surface 52 corresponds to at least a portion of the display surface on the opposite side of the object 100 from the user P. FIG. 6 also shows an example in which an underwater creature is displayed as the first image 200 associated with the object 100. Alternatively, a second image may be displayed on the central wall surface 52, which is the first area, before the user P performs an action that satisfies the first condition. When the action that satisfies the first condition is detected, the second image may be changed to display the first image 200. For example, an image of a fish swimming in the sea may be displayed as the second image, and an image of the fish displayed as the second image jumping above the water may be displayed as the first image 200.

[0061] In Example 1, the stimulus that allows user P to discover affordance is the action of touching a pearl from a pearl oyster, which is object 100 placed in space S1. By user P understanding the environment of space S1, user P is more likely to get the feeling that he or she has noticed something and acted on his or her own.

[0062] 5. Example 2 Fig. 7 is a diagram showing the state of the space S2 of the second embodiment before the first condition is met, and Fig. 8 is a diagram showing the state of the space S2 of the second embodiment after the first condition is met. As shown in FIG. 7, the space S2 is also a space surrounded by three walls, namely, a left wall 51, a central wall 52, and a right wall 53, similar to the space S1.

[0063] Similarly to space S1, space S2 also has a left camera 10A and a projector 20A installed on the left wall 51, a central camera 10B and a projector 20B installed on the central wall 52, and a right camera 10C and a projector 20C installed on the right wall 53.

[0064] FIG. 7 shows a bird's wing placed near the center of the space S2 as an example of the object 100. User P, who has moved into space S2, notices that object 100 has fallen near the center of space S2, and approaches object 100 to confirm what object 100 it is.

[0065] FIG. 8 is a diagram showing a case where a user P approaches an object 100 within a first distance. The second control unit 35 analyzes the image captured by any one of the left camera 10A, the center camera 10B, and the right camera 10C, and when it detects that the user P and the object 100 have come closer to each other by a predetermined first distance or less, it determines that the first condition is met and causes at least one of the projectors 20A, 20B, and 20C to display the first image 200. FIG. 8 shows an example in which the projector 20A displays an image of a plurality of flowers as the first image 200 on the floor of the space S, and the projector 20B displays an image of a bird as the first image 200 on the center wall surface 52. Before the user P performs an action that satisfies the first condition, a second image may be displayed on the left wall surface 51 or the center wall surface 52, which is the first area. When the action that satisfies the first condition is detected, the second image may be changed to display the first image 200. For example, an image of a bird perched on a tree may be displayed on the central wall surface 52 as the second image, and an image showing the bird displayed as the second image flying may be displayed as the first image 200.

[0066] In this Example 2, the stimulus that allows user P to discover affordance is the action of approaching a bird's feather, which is object 100 placed in space S2, by user P. In this Example 2 as well, by user P understanding the environment of space S2 by himself, user P is more likely to get the feeling that he has noticed something and acted by himself.

[0067] 6. Example 3 Fig. 9 is a diagram showing the state of the space S3 of the third embodiment before the first condition is met, and Fig. 10 is a diagram showing the state of the space S3 of the third embodiment after the first condition is met. As shown in FIG. 9, the space S3 is also a space surrounded by three walls, namely, a left wall 51, a central wall 52, and a right wall 53, similar to the space S1.

[0068] Similarly to space S1, space S3 also has a left camera 10A and a projector 20A installed on the left wall 51, a central camera 10B and a projector 20B installed on the central wall 52, and a right camera 10C and a projector 20C installed on the right wall 53. Furthermore, space S3 of the third embodiment has a receiver serving as sensor 10D, which receives a beacon signal, embedded in the left wall 51. In FIGS. 9 and 10, the receiver is referred to as receiver 10D. A picture of a fish is also painted on the left wall 51 as object 100.

[0069] Furthermore, user P wears, for example, a wristband 300 on his or her wrist. This wristband 300 is equipped with a beacon transmitter that transmits a beacon signal. The beacon transmitter performs wireless communication in accordance with a communication standard such as BLE (Bluetooth Low Energy) or Wi-Fi.

[0070] User P moves to space S3, finds a picture of a fish as object 100 drawn on left wall 51, and approaches object 100. The second control unit 35 determines the distance between the user P and the left wall surface 51 based on the signal strength of the beacon signal received by the receiver embedded in the left wall surface 51.

[0071] 10 is a diagram showing a case where user P approaches object 100 within a first distance. When second control unit 35 determines that the distance between user P and left wall surface 51 is equal to or less than a preset distance, second control unit 35 determines that the first condition is met and causes one of projectors 20A, 20B, and 20C to display a first image. FIG. 10 shows an example in which projector 20A displays images of multiple fish as first image 200 on the floor of space S.

[0072] In this Example 3, the stimulus that allows the user P to discover affordance is the action of approaching the picture drawn on the left wall surface 51. In this Example 3 as well, by the user P understanding the environment of the space S1 on his own, the user P is more likely to get the feeling that he has noticed and acted on his own.

[0073] 7. Example 4 Fig. 11 is a diagram showing the state of the space S4 of the fourth embodiment before the first condition is met, and Fig. 12 is a diagram showing the state of the space S4 of the fourth embodiment after the first condition is met. As shown in FIG. 11, the space S4 is also a space surrounded by three walls, namely, a left wall 51, a central wall 52, and a right wall 53, similar to the space S1.

[0074] Similarly to space S1, space S4 also shows an example in which a left camera 10A and a projector 20A are installed on a left wall surface 51, a central camera 10B and a projector 20B are installed on a central wall surface 52, and a right camera 10C and a projector 20C are installed on a right wall surface 53. Furthermore, in space S4 of Example 4, an example is shown in which a tree is installed near the center as object 100. The tree may be a real tree planted in a flowerpot or the like, or may be an imitation.

[0075] FIG. 12 is a diagram showing a case where a user P moves around the object 100 while avoiding the object 100. In FIG. For example, when the second control unit 35 performs image analysis on the image captured by the camera 10A and detects that the user P is moving counterclockwise around the object 100 in the viewfinder, the second control unit 35 displays the first image 200 on the left wall surface 51, which is within the range that can be included in the field of view of the user P moving counterclockwise. Fig. 12 shows an example in which a landscape image is displayed as the first image 200. The path of moving counterclockwise around the object 100 corresponds to the first path.

[0076] Furthermore, when the second control unit 35 detects that the user P is moving clockwise around the object 100 in the viewfinder, it displays the first image 200 on the right wall surface 53, which is within the range that can be included in the field of view of the user P moving clockwise. The first image 200 that is displayed when the user P moves clockwise around the object 100 is different from the first image 200 that is displayed when the user P moves counterclockwise around the object 100.

[0077] 8. Other Embodiments The above-described embodiment is a preferred embodiment of the present invention, but the present invention is not limited to the above-described embodiment and various modifications are possible within the scope of the gist of the present invention. For example, in the above-described embodiment, the space S is a real space, but the space S may be a virtual space. In this case, a head-mounted display or the like is attached to the head of the user P, and the first image 200 and the second image are viewed in the virtual space.

[0078] Furthermore, in the examples shown in Figures 5 to 12, the space S is described as a space having three planes, namely, the left wall surface 51, the central wall surface 52, and the right wall surface 53, but it may also be configured to display an image on a single large screen.

[0079] Furthermore, each functional unit of the projector 20 shown in FIG. 2 indicates a functional configuration, and the specific implementation form is not particularly limited. In other words, it is not necessarily necessary to implement hardware corresponding to each functional unit individually, and it is of course possible to configure the projector 20 so that the functions of multiple functional units are realized by a single processor executing a program. Furthermore, some of the functions realized by software in the above embodiments may be realized by hardware, and some of the functions realized by hardware may be realized by software. The same applies to the information processing device 30 shown in FIG. 3.

[0080] Furthermore, the processing units in the flowchart shown in Fig. 4 are divided according to the main processing content in order to make it easier to understand the processing of the second control unit 35. The present invention is not limited by the way in which the processing units are divided or the names of the processing units shown in the flowchart of Fig. 4. In other words, the processing of the second control unit 35 can be divided into more processing units according to the processing content, or one processing unit can be divided so that it includes more processes.

[0081] 9. Summary of the Disclosure A summary of this disclosure is provided below.

[0082] (Appendix 1) An image display method in which an information processing device determines whether a user's action with respect to an object placed in a space satisfies a first condition based on the detection result of at least one sensor, and if it is determined that the user's action satisfies the first condition, displays a first image in a first area of ​​the space that may be included in the user's field of view due to the action.

[0083] As a result, when the user's action with respect to the object satisfies the first condition, the first image is displayed in a first region of space that can be included in the user's field of view due to the user's action. Therefore, since the first image is displayed due to the user's action, the user is more likely to notice the displayed first image. This provides the user with the feeling that they have noticed the image themselves, and can enhance the user's sense of immersion.

[0084] (Appendix 2) An image display method as described in Appendix 1, wherein the first condition is that the distance between the user and the object is less than or equal to a first distance, and the first area is at least a portion of the display surface on the opposite side of the object from the user.

[0085] As a result, when the distance between the user and the object becomes equal to or less than a first value, the first image is displayed on at least a part of the display surface on the opposite side of the object from the user. Therefore, as the user approaches the object, the first image can be displayed in a position that is easily noticeable by the user.

[0086] (Appendix 3) 2. The image display method of claim 1, wherein the at least one sensor includes a sensor that detects that the user has touched an object, the first condition is that the user has touched the object, and the first area is at least a portion of the display surface on the opposite side of the object from the user.

[0087] As a result, when the sensor detects that the user has touched an object, the first image is displayed on at least a part of the display surface on the opposite side of the object from the user. Therefore, when the user touches the object, the first image can be displayed in a position that is easily noticeable by the user.

[0088] (Appendix 4) The image display method of claim 2, wherein the at least one sensor includes a distance sensor, and further includes outputting at least one of a visual, an auditory, and a tactile notification from a notification device carried by the user based on the distance between the user and the object.

[0089] As a result, a notification is output from the notification device carried by the user through at least one of visual, auditory, and tactile means based on the distance between the user and the object. Therefore, the notification is output from the notification device through at least one of visual, auditory, and tactile means in response to the user's actions, allowing the user to recognize the distance to the object.

[0090] (Appendix 5) 2. The image display method according to claim 1, further comprising displaying a second image in the first area, the first image being a modified version of the second image.

[0091] As a result, when the user's action with respect to the object satisfies the first condition, the image displayed in the first region of the space that can be included in the user's field of view changes from the first image to the second image. Therefore, because the first image changes to the second image in response to the user's action, the user can easily notice the image displayed in the first region.

[0092] (Appendix 6) An image display method according to any one of appendices 1 to 5, wherein the object is a natural object, an object simulating a natural object, or an object related to the first image displayed in the space.

[0093] This makes it easier for the user to notice the object, since the object is a natural object, an object that imitates a natural object, or an object related to the first image displayed in space.

[0094] (Appendix 7) An image display method as described in Appendix 1, wherein the first condition is that the user moves around the object along a first path, and the first area is located within a range that can be included in the field of view of the user moving along the first path.

[0095] As a result, when the user moves around the object along the first path, the first image is displayed within a range that can be included in the field of view of the user moving along the first path, making it easier for the user to notice the displayed first image.

[0096] (Appendix 8) An image display system including at least one sensor, at least one display device, and an information processing device that performs the following operations: determining whether a user's action with respect to an object placed in a space satisfies a first condition based on the detection result of the at least one sensor; and, when it is determined that the user's action satisfies the first condition, displaying a first image in a first region of the space that may be included in the user's field of view due to the action.

[0097] As a result, when the user's action with respect to the object satisfies the first condition, the first image is displayed in a first region of space that can be included in the user's field of view due to the user's action. Therefore, since the first image is displayed due to the user's action, the user is more likely to notice the displayed first image. This provides the user with the feeling that they have noticed the image themselves, and can enhance the user's sense of immersion.

[0098] (Appendix 9) A program that causes a processor to determine whether a user's action with respect to an object placed in a space satisfies a first condition based on the detection results of at least one sensor, and when it is determined that the user's action satisfies the first condition, display a first image in a first region of the space that may be included in the user's field of view due to the action.

[0099] As a result, when the user's action with respect to the object satisfies the first condition, the first image is displayed in a first region of space that can be included in the user's field of view due to the user's action. Therefore, since the first image is displayed due to the user's action, the user is more likely to notice the displayed first image. This provides the user with the feeling that they have noticed the image themselves, and can enhance the user's sense of immersion. [Explanation of symbols]

[0100] 1...image display system, 10, 10A, 10B, 10C, 10D...sensor, 20, 20A, 20B, 20C...projector, 21...first communication interface, 22...image processing unit, 23...frame memory, 24...image projection unit, 25...first control unit, 26...first memory unit, 27...first processor, 30...information processing device, 31...second communication I / F, 32...operation unit, 35...second control unit, 36...second memory unit, 37...second processor, 51...left wall, 52...central wall, 53...right wall, 100...object, 200...first image, 241...light source, 243...light modulation device, 245...optical unit, 265...control program, 300...wristband, 365...control program, P...user, S1, S2, S3, S4...space.

Claims

1. The information processing device determining whether a user's action with respect to an object disposed in a space satisfies a first condition based on a detection result of at least one sensor; When it is determined that the user's action satisfies the first condition, displaying a first image in a first region of the space that can be included in the user's field of view due to the action; Implement the image display method.

2. the first condition is that the distance between the user and the object is equal to or shorter than a first distance; The image display method according to claim 1 , wherein the first area is at least a part of the display surface on the opposite side of the object from the user.

3. the at least one sensor includes a sensor that detects when the user touches an object; the first condition is that the user has come into contact with the object; The image display method according to claim 1 , wherein the first area is at least a part of the display surface on the opposite side of the object from the user.

4. the at least one sensor includes a distance sensor; The image display method according to claim 2 , further comprising outputting at least one of a visual, an auditory, and a tactile notification from a notification device carried by the user based on the distance between the user and the object.

5. further comprising displaying a second image in the first area; 2. The image display method according to claim 1, wherein the first image is an image obtained by modifying the second image.

6. The image display method according to claim 1 , wherein the object is a natural object, an object that imitates a natural object, or an object related to the first image displayed in the space.

7. the first condition is that the user moves around the object along a first path; The image display method according to claim 1 , wherein the first area is located within a range that can be included in the field of view of the user traveling along the first path.

8. at least one sensor; at least one display device; determining whether a user's action with respect to an object disposed in a space satisfies a first condition based on a detection result of at least one sensor; When it is determined that the user's action satisfies the first condition, displaying a first image in a first region of the space that can be included in the user's field of view due to the action; an information processing device that executes the above; An image display system comprising:

9. To the processor, determining whether a user's action with respect to an object disposed in a space satisfies a first condition based on a detection result of at least one sensor; When it is determined that the user's action satisfies the first condition, displaying a first image in a first region of the space that can be included in the user's field of view due to the action; A program that executes.

Images