Image display method, image display system, and program

Abstract

To provide an image display method, an image display system, and a program that make it easy for a user to notice a displayed image and provide a feeling that the user himself / herself has noticed the image to enhance a sense of immersion.SOLUTION: Determining whether or not the action of the user P in the space S satisfies a first condition, presenting, when it is determined that the action of the user P does not satisfy the first condition, a visual expression for inducing the action of the user P satisfying the first condition in at least a first region located in a first direction in which the user P is directed in the space S, determining whether or not the action of the user P satisfies a first condition in at least a first region located in a first direction in which the user P is directed in the space S, displaying the first image 200 in a second region of the space S.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image display method, an image display system, and a program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, a system is known that displays an image on a display screen in a video space, thereby giving a user a sense of immersion in the space.

[0003] For example, the video display system disclosed in Patent Document 1 generates an image of a virtual camera at a specific camera position and displays it on a display screen. The video display system detects the position of a real object on the display screen using a sensor, moves the camera position, and makes an object appear in the image based on the position of the real object on the display screen. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-219820 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the video display system of Patent Document 1 makes objects appear in the video based on the position of a real person, regardless of which part of the video displayed on the display screen the person is looking at. When the person is moving around a little, the objects appear less. As a result, it is difficult for the person to notice the objects, making it difficult to achieve a sense of immersion. [Means for solving the problem]

[0006] The image display method disclosed herein includes determining whether a user's behavior in a space satisfies a first condition based on detection results of at least one sensor; if it is determined that the user's behavior does not satisfy the first condition, presenting a visual representation in a first area in the space located at least in a first direction in which the user is facing, which visual representation encourages the user to perform a behavior that satisfies the first condition; after presenting the visual representation, determining whether the user's behavior satisfies the first condition based on detection results of the at least one sensor; and if it is determined that the user's behavior satisfies the first condition, displaying a first image in a second area of ​​the space.

[0007] The image display system of the present disclosure is an image display system including at least one sensor, at least one display device, and an information processing device that performs the following steps: determine whether a user's behavior in a space satisfies a first condition based on the detection result of the at least one sensor; if it is determined that the user's behavior does not satisfy the first condition, present, by the at least one display device, a visual representation that induces the user to perform a behavior that satisfies the first condition in a first area in the space that is located at least in a first direction in which the user is facing; after presenting the visual representation, determine whether the user's behavior satisfies the first condition based on the detection result of the at least one sensor; and if it is determined that the user's behavior satisfies the first condition, display a first image in a second area of ​​the space by the at least one display device.

[0008] The program of the present disclosure causes a computer to determine whether a user's movement in a space satisfies a first condition based on the detection results of at least one sensor; if it is determined that the user's movement does not satisfy the first condition, present a visual representation in a first area in the space that is located at least in a first direction in which the user is facing, which visual representation encourages the user to perform a movement that satisfies the first condition; after presenting the visual representation, determine whether the user's movement satisfies the first condition based on the detection results of the at least one sensor; and if it is determined that the user's movement satisfies the first condition, display a first image in a second area of ​​the space. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing the system configuration of an image display system. [Figure 2] FIG. 1 is a block diagram showing the configuration of a projector. [Figure 3] FIG. 1 is a block diagram showing the configuration of an information processing device. [Figure 4] 10 is a flowchart showing the operation of the information processing device. [Figure 5] FIG. 10 is a diagram showing the state of the space in the first embodiment before the first condition is met. [Figure 6] FIG. 10 is a diagram showing the state of the space in the first embodiment after the first condition is met. [Figure 7] FIG. 10 is a diagram showing the state of space in the modified example of the first embodiment before the first condition is met. [Figure 8] FIG. 10 is a diagram showing the state of space in the modified example of the first embodiment after the first condition is met. [Figure 9] FIG. 10 is a diagram showing the state of the space in the second embodiment before the first condition is met. [Figure 10] FIG. 10 is a diagram showing the state of the space in the second embodiment after the first condition is met. [Figure 11] FIG. 10 is a diagram showing the state of the space in the third embodiment before the first condition is met. [Figure 12] FIG. 10 is a diagram showing the state of the space in the third embodiment after the first condition is met. DETAILED DESCRIPTION OF THE INVENTION

[0010] [1. System configuration of image display system] Hereinafter, embodiments will be described with reference to the accompanying drawings. FIG. 1 is a diagram showing the system configuration of an image display system 1. As shown in FIG. The image display system 1 is a system in which a plurality of sensors 10, a plurality of projectors 20, and an information processing device 30 are connected via a network 5. While FIG. 1 shows a configuration in which the image display system 1 includes a plurality of sensors 10 and a plurality of projectors 20, the image display system 1 may include only one sensor 10. The image display system 1 may also include only one projector 20. One or more projectors 20 are an example of at least one display device. The display device is not limited to a projector, and may be a direct-view type panel display device, or a combination thereof.

[0011] The network 5 is, for example, a network such as a LAN (Local Area Network) or a WAN (Wide Area Network).

[0012] 1 shows a case where M sensors, ie, sensor 10A, sensor 10B, . . . , sensor 1M, are provided as the plurality of sensors 10. M is an arbitrary natural number. The multiple sensors 10 include, for example, cameras and distance sensors. The multiple sensors 10 may also include a communication device that transmits and receives radio waves for short-range wireless communication. The multiple sensors 10 may be configured with only cameras or only distance sensors.

[0013] The camera may be an R (red), G (green), and B (blue) camera capable of capturing images in three wavelength bands: red, green, and blue, or an infrared camera capable of capturing images in dark places or at night using infrared rays. The distance sensor is, for example, a ToF (Time of Flight) sensor that measures the distance to an object using laser light or ultrasonic waves. Hereinafter, when referring to one of the sensors 10, it will be referred to as sensor 10. Also, a stereo camera may be used instead of the distance sensor.

[0014] The multiple projectors 20 include N projectors: projectors 20A, 20B, ..., 20N. N is any natural number, and N and M may be the same value or different values. Hereinafter, any projector included in the multiple projectors 20 will be referred to as projector 20.

[0015] The information processing device 30 is, for example, a computer device of a notebook type or a desktop type, etc. The information processing device 30 may also be configured as a tablet type computer device.

[0016] 1 shows an example in which multiple sensors 10, multiple projectors 20, and an information processing device 30 are connected by wire via a network 5, but some or all of the connections between the multiple sensors 10, multiple projectors 20, and information processing device 30 may be wireless. For example, a Wi-Fi router may be installed to connect the multiple sensors 10, multiple projectors 20, and information processing device 30 to a Wi-Fi network. Alternatively, the multiple sensors 10 and information processing device 30 may be connected by short-range wireless communication such as Bluetooth. Wi-Fi and Bluetooth are registered trademarks.

[0017] [2. Projector configuration] 2 is a block diagram showing the configuration of the projector 20. The projectors 20A, 20B, . . . , 20N have substantially the same configuration. Therefore, the configuration of the projector 20 will be described.

[0018] The projector 20 includes a first communication interface 21, an image processing unit 22, a frame memory 23, an image projection unit 24, and a first control unit 25. Hereinafter, interface will be abbreviated as I / F. The first communication I / F 21, the image processing unit 22, and the image projection unit 24 are connected to the first control unit 25 and operate under the control of the first control unit 25.

[0019] The first communication I / F 21 is connected to the information processing device 30 via the network 5, and performs mutual data communication with the information processing device 30. The first communication I / F 21 is a wired interface having a connection terminal such as a USB (Universal Serial Bus) connector or an Ethernet connector, and an interface circuit. Ethernet is a registered trademark. The first communication I / F 21 may also be a wireless communication interface.

[0020] The image processing unit 22 is connected to a frame memory 23. The frame memory 23 includes a plurality of banks. Each bank has a storage capacity sufficient to store one frame of display image. The frame memory 23 is configured, for example, with an SDRAM (Synchronous Dynamic RAM). The image processing unit 22 expands the display image input from the first communication I / F 21 in the frame memory 23.

[0021] The image processing unit 22 performs image processing on the display image expanded in the frame memory 23. The image processing performed by the image processing unit 22 includes, for example, resolution conversion processing or resizing processing, distortion correction, shape correction processing, digital zoom processing, and adjustment of image color and brightness. The image processing unit 22 executes processing specified by the first control unit 25, and, if necessary, performs processing using parameters input from the first control unit 25. Of course, the image processing unit 22 can also execute a combination of multiple image processing operations described above.

[0022] The image processing unit 22 and the frame memory 23 are configured, for example, by integrated circuits. Examples of integrated circuits include large-scale integration (LSI), application-specific integrated circuits (ASIC), programmable logic devices (PLD), field-programmable gate arrays (FPGA), and system-on-a-chips (SoC). An analog circuit may be included as part of the integrated circuit configuration, or the first control unit 25 may be configured in combination with an integrated circuit.

[0023] The image projection unit 24 includes a light source 241 , a light modulation device 243 , and an optical unit 245 .

[0024] The light source 241 includes a discharge light source lamp such as an ultra-high pressure mercury lamp or a metal halide lamp, or a solid-state light source such as a light-emitting diode or a semiconductor laser.

[0025] The light modulation device 243 is a light modulation element that modulates the light emitted from the light source 241 and includes a transmissive liquid crystal panel in which liquid crystal is sealed between a pair of transparent substrates. The liquid crystal panel is not shown. The liquid crystal panel includes a panel region consisting of a plurality of pixels arranged in a matrix. The light modulation device 243 applies a drive voltage corresponding to the display image input from the image processing unit 22 to each pixel in the panel region, and changes the light transmittance of each pixel to the transmittance corresponding to the display image. The light emitted from the light source 241 is modulated as it passes through the liquid crystal panel, and image light corresponding to the display image is generated.

[0026] The light modulation element included in the light modulation device 243 is not limited to a transmissive liquid crystal panel, but may be, for example, a reflective liquid crystal panel or a DMD (Digital Micromirror Device). In this case, the DMD is an example of a display panel.

[0027] The optical unit 245 includes a projection lens (not shown) and projects the image light modulated by the light modulation device 243 onto a projection target in an enlarged form. Projection targets onto which the projector 20 projects the image light include, for example, a screen, a wall, floor, or ceiling surface in a room, etc. The image displayed on the projection target by the projector 20 may be a moving image or a still image.

[0028] The first control unit 25 is a computer device including a first storage unit 26 and a first processor 27.

[0029] The first storage unit 26 includes a RAM (Random Access Memory) and a ROM (Read Only Memory). The RAM is used for temporary storage of various data, and the ROM stores a control program 265 that controls the operation of the projector 20, various setting information, and the like.

[0030] The first processor 27 is an arithmetic processing device configured with a CPU (Central Processing Unit) and an MPU (Micro Processor Unit). The first processor 27 executes a control program 265 to control each unit of the projector 20. The first processor 27 may be configured with a single processor or multiple processors. The first processor 27 may also be configured with a SoC (System-on-a-Chip) integrated with part or all of the first storage unit 26 or other circuits. The first processor 27 may also be configured with a combination of a CPU that executes programs and a DSP (Digital Signal Processing) that executes predetermined arithmetic processing. Furthermore, all of the functions of the first processor 27 may be implemented in hardware, or may be configured using a programmable device.

[0031] [3. Configuration of information processing device] FIG. 3 is a block diagram showing the configuration of the information processing device 30. The configuration of the information processing device 30 will be described with reference to FIG. The information processing device 30 includes a second communication I / F 31, an operation unit 32, a display unit 33, and a second control unit 35. The second communication I / F 31, the operation unit 32, and the display unit 33 are connected to the second control unit 35.

[0032] The second communication I / F 31 is a wired interface having a connection terminal such as a USB connector or an Ethernet connector and an interface circuit, and performs mutual data communication between multiple sensors 10 and multiple projectors 20 connected via the network 5.

[0033] The operation unit 32 includes input devices such as a mouse and a keyboard, and receives operations from an operator. The operation unit 32 outputs an operation signal corresponding to the received operation to the second control unit .

[0034] The display unit 33 includes a display panel such as a liquid crystal panel or an organic EL (Electro Luminescence) panel, and displays an image on the display panel under the control of the second control unit .

[0035] The second control unit 35 includes a second storage unit 36 ​​and a second processor 37.

[0036] The second storage unit 36 ​​includes a RAM and a ROM. The second storage unit 36 ​​also includes an auxiliary storage device such as an SSD (Solid State Drive) or an HDD (Hard Disk Drive). The RAM is used for temporary storage of various data and the like, and the ROM stores a control program 365 such as an OS (Operating System) and application programs that control the operation of the information processing device 30, as well as various setting information. The second storage unit 36 ​​also stores image data that is the source of the image light projected by the projector 20.

[0037] The second processor 37 is an arithmetic processing device configured with a CPU and an MPU. The second processor 37 executes a control program 365 to control each unit of the information processing device 30. The second processor 37 may be configured with a single processor or multiple processors.

[0038] The second control unit 35 determines whether the movement or state of the user P in the space S satisfies the first condition, based on the sensor data output by at least one sensor 10. The movement or state of the user P may be collectively referred to as the "movement of the user P."

[0039] Space S is a three-dimensional real space. Space S is a space where an image is displayed by a projector 20 installed in space S, and where motion is detected by a sensor 10 installed in space S. Space S has at least one projection surface, and an image is displayed on the projection surface by the projector 20. The projection surface may be a wall surface, a screen, a piece of fabric, or the like. Space S is a so-called image space, and, for example, it is possible to display an image over a range wider than the effective visual field of a human being.

[0040] Satisfying the first condition means that the movement or state of user P detected based on sensor data output by at least one sensor 10 corresponds to a preset movement or state. The preset movement or state includes, for example, whether the line of sight or the direction of the head of user P is facing a specific direction, whether user P is viewing an image displayed by projector 20 installed in space S, etc. The preset movement or state also includes whether user P is located at a preset position in space S.

[0041] For example, the second control unit 35 detects the gaze direction and head direction of the user P by analyzing the image captured by the camera included in at least one sensor 10. For example, the second control unit 35 detects the inner corners of the eyes and irises of the user P captured in the captured image, and detects the gaze direction of the user P based on the positional relationship between the inner corners of the eyes, which are the reference point, and the irises, which are the moving point. The second control unit 35 may also extract the face and head of the user P from the image captured by the camera, and estimate the direction of the face and head using a preset algorithm.

[0042] If the detected gaze or head direction of user P is not directed in a specific direction and does not satisfy the first condition, the second control unit 35 identifies a first direction, which is the direction in which user P is facing, based on the detected gaze or head direction of user P. Furthermore, the second control unit 35 identifies a first region of space S that is viewed by user P facing in the first direction. The first region is a projection surface that partitions space S, or a projection surface disposed within space S. The first region may be the entirety of one surface, or a part of it. Furthermore, the first region may span multiple surfaces.

[0043] When the second control unit 35 identifies the first direction and the first area, it identifies the projector 20 that can display an image in the identified first area. The second control unit 35 uses the identified projector 20, or the identified projector 20 and another projector 20, to present a visual expression that induces an action or state of the user P that satisfies the first condition.

[0044] As a visual expression, the projector 20 displays in the first area, for example, an image that induces the user P to change the direction of their line of sight or head in a direction that satisfies a first condition, or an image that induces the user P to move their own position to a preset position in the space S. The visual expression is preferably an expression that gives a suggestion or a hint through the movement or shape of elements in the image, rather than an explicit expression such as an arrow image or a message that requests the user P to change direction. By finding affordances in the image and performing the action voluntarily, the user P is less likely to sense the image provider's artificiality, and is therefore more likely to feel that they have discovered the first image 200 themselves.

[0045] Furthermore, the projector 20 may reduce the visibility of the image displayed in the first region as a visual presentation. For example, the projector 20 may reduce the visibility of the image by reducing the brightness of the image displayed in the first region to make the image darker, blurring the image, displaying a hazy image, fading the color of the image, or processing the image so that at least a portion of the image fluctuates. Furthermore, the projector 20 may temporarily hide part or all of the image displayed in the first region.

[0046] The user's actions induced by presenting a visual representation include, for example, a shift in the user P's line of sight, a change in the direction of the head or body, and movement of the user P.

[0047] After causing the projector 20 to present the visual representation in the first area, the second control unit 35 again determines whether or not the action or state of the user P satisfies the first condition. If the action or state of user P does not satisfy the first condition, second control unit 35 again identifies the first direction, which is the direction in which user P is facing, based on the line of sight and head direction of user P. Second control unit 35 identifies projector 20 that can display an image in the identified first direction, and uses the identified projector 20 to again present a visual expression that induces the action or state of user P that satisfies the first condition.

[0048] Furthermore, when the second control unit 35 determines that the movement or state of the user P satisfies the first condition, the second control unit 35 causes the first image 200 to be displayed in the second area of ​​the space S. The second area is a projection surface that partitions the space S or a projection surface arranged within the space S. The second area may be a predetermined area. It may also change depending on the time, the number of users P in the space S, etc. The second area may be the entire surface or a part of it. The first area may span multiple surfaces. The second area may be a different area from the first area or an overlapping area.

[0049] The first image 200 may be a moving image or a still image. The first image 200 is, for example, an image or a part thereof that is desired to be viewed by the user P, and the image may be changed to attract the user P's attention. For example, an image may be displayed before the action or state of user P satisfies the first condition, and after the action or state that satisfies the first condition is detected, a further changed first image 200 may be displayed.

[0050] FIG. 4 is a flowchart showing the operation of the second control unit 35. The operation of the second control unit 35 will be described with reference to the flowchart shown in FIG. First, the second control unit 35 acquires sensor data from at least one sensor 10 (step S1). Next, the second control unit 35 detects the gaze direction or head orientation of the user P based on the acquired sensor data (step S2).

[0051] Next, the second control unit 35 determines whether the behavior of the user P satisfies the first condition. Specifically, the second control unit 35 determines whether the detected gaze direction or head orientation of the user P faces the direction of the second area (step S3). If the gaze direction or head orientation of the user P does not face the direction of the second area that satisfies the first condition (step S3 / NO), the second control unit 35 identifies the first direction in which the user P faces based on the detected gaze direction or head orientation of the user P (step S4).

[0052] Next, the second control unit 35 identifies a first area, which is an area of ​​the space S that is visually recognized by the user P facing the identified first direction (step S5). Then, the second control unit 35 identifies a projector 20 that can present a visual representation in this identified first area (step S6). The second control unit 35 causes the identified projector 20 to present the visual representation (step S7). Thereafter, the second control unit 35 returns to step S1 and acquires sensor data from at least one sensor 10 again (step S1).

[0053] Furthermore, when the second control unit 35 determines that the detected gaze direction or head orientation of the user P is directed toward the second area that satisfies the first condition (step S3 / YES), it identifies a projector 20 that can display an image in the second area of ​​the space S toward which the gaze of the user P is directed (step S8). The second control unit 35 causes the identified projector 20 to display the first image 200 (step S9). Alternatively, the first image 200 may be displayed in the second area by changing an image already displayed in the second area.

[0054] 4. Example 1 Fig. 5 is a diagram showing the state of the space S1 of the first embodiment before the first condition is met, and Fig. 6 is a diagram showing the state of the space S1 of the first embodiment after the first condition is met. As shown in Figure 5, the space S1 is a space surrounded by three walls. When viewed from the drawing, the left wall is referred to as a left wall 51, the central wall is referred to as a central wall 52, and the right wall is referred to as a right wall 53. A left camera is installed as sensor 10A on the upper part of left wall surface 51, a central camera is installed as sensor 10B on the upper part of central wall surface 52, and a right camera is installed as sensor 10C on the upper part of right wall surface 53. In the explanations of Figures 5 and 6, the left camera will be referred to as "left camera 10A," the central camera will be referred to as "central camera 10B," and the right camera will be referred to as "right camera 10C."

[0055] 5 and 6 show an example in which three projectors 20, projectors 20A, 20B, and 20C, are installed in space S1. 5 and 6 show an example in which projectors 20A, 20B, and 20C are installed on the ceiling surface near left wall surface 51, center wall surface 52, and right wall surface 53, respectively, i.e., installed by ceiling-suspended installation.

[0056] 5 and 6 show an example in which the projectors 20A, 20B, and 20C are installed on the ceiling surface of the space S1, but the projectors 20A, 20B, and 20C may also be installed on the floor surface of the space S1. Also, the projectors 20A, 20B, and 20C may be wall-mounted on the left wall surface 51, the center wall surface 52, and the right wall surface 53, respectively.

[0057] 5 and 6, the projectors 20A, 20B, and 20C, the left camera 10A, the center camera 10B, and the right camera 10C are connected to an information processing device 30 (not shown). The information processing device 30 may be placed behind a decorative item (not shown) installed in the space S1 so as not to be seen by others, or may be placed behind the left wall surface 51, the center wall surface 52, or the right wall surface 53.

[0058] In the first embodiment, it is assumed that when the line of sight of the user P is directed in the direction of the left wall surface 51, the action or state of the user P is determined to be the action or state that satisfies the first condition. The second control unit 35 detects the line of sight of the user P by analyzing the images captured by the left camera 10A, the center camera 10B, and the right camera 10C. Fig. 5 shows a state in which the user P is facing the direction of the center wall surface 52, and the action or state of the user P does not satisfy the first condition. In other words, the user P is viewing the area including the center wall surface 52 as the first area, and the action or state of the user P does not satisfy the first condition.

[0059] When the second control unit 35 determines that the action or state of the user P does not satisfy the first condition, it identifies the projector 20B capable of displaying an image on the central wall surface 52, which is in the line of sight of the user P. The second control unit 35 causes the identified projector 20B to display an image on the central wall surface 52, thereby presenting a visual representation. The image displayed by the projector 20B on the central wall surface 52 is called an attraction image 300. FIG. 5 shows an example in which an attraction image 300A of a fish swimming from the central wall surface 52 toward the left wall surface 51 is displayed as an image included in the attraction image 300. The attraction image 300A is an example of a first element. The first element will be described later.

[0060] The second control unit 35 causes the projector 20B to display the attraction image 300A, and acquires images captured by the left camera 10A, the center camera 10B, and the right camera 10C. The second control unit 35 performs image analysis on the acquired images to identify the line of sight or the direction in which the user P's head is facing, and determines whether the user P's behavior or state satisfies a first condition. As shown in FIG. 6, when the user P's line of sight is directed toward the left wall surface 51, the second control unit 35 determines that the user P's behavior or state satisfies the first condition. When the second control unit 35 determines that the user P's behavior or state satisfies the first condition, the second control unit 35 displays a first image 200 on the left wall surface 51, which is in the direction in which the user P's line of sight is directed. FIG. 6 shows an example in which an image 200A of multiple fish swimming in the ocean is displayed as an image included in the first image 200. An example of the second element is image 200A of a plurality of fish swimming in the sea, which is included in first image 200. The first element and second element will be described later.

[0061] In Example 1, an attractive image 300A of a fish swimming from the central wall surface 52 to the left wall surface 51 is presented as a visual representation as a stimulus that enables the user P to find affordance. By displaying the attractive image 300A on the central wall surface 52, allowing the user P to view the attractive image 300A and facing the left wall surface 51, the user P is able to understand the environment of the space S1 by himself, and the user P is more likely to get the feeling that he has noticed and acted by himself.

[0062] [4B. Modification of Example 1] Fig. 7 is a diagram showing the state of the space S1 of the modified example of the first embodiment before the first condition is met, and Fig. 8 is a diagram showing the state of the space S1 of the modified example of the first embodiment after the first condition is met. In the modification of the first embodiment, the space S1 is also a space surrounded by three walls, namely, a left wall 51, a central wall 52, and a right wall 53, as shown in FIG.

[0063] Similarly to space S1, space S2 also has a left camera 10A and a projector 20A installed on the left wall 51, a central camera 10B and a projector 20B installed on the central wall 52, and a right camera 10C and a projector 20C installed on the right wall 53.

[0064] In the above-described first embodiment, an image of a fish swimming from the central wall surface 52 toward the left wall surface 51 is displayed as the attracting image 300A to attract the user P to face the left wall surface 51. The attracting image 300A includes one fish. In this modification of the first embodiment, an image of a fish swimming from the central wall surface 52 to the left wall surface 51 is displayed as an attraction image 300B to direct the user P toward the left wall surface 51. In the attraction image 300B, all of the multiple fish displayed in the attraction image 300B may be made to swim from the central wall surface 52 to the left wall surface 51. Alternatively, only one of the multiple fish may be made to swim from the central wall surface 52 to the left wall surface 51. This fish is referred to as a specific fish. The specific fish swimming from the central wall surface 52 to the left wall surface 51 may be distinguished from other fish by, for example, changing its species, size, color, or brightness. In FIG. 7, the specific fish 301A included in the attraction image 300B is indicated by a dashed line, and the other fish 301B, 301C, and 301D are indicated by solid lines. The specific fish 301A and the other fish 301B, 301C, and 301D included in the attracting image 300B correspond to the plurality of first elements.

[0065] In addition, in this modification of the first embodiment, the projector 20A may also display the attraction image 300C on the left wall surface 51, which corresponds to the second area, before the action or state of the user P satisfies the first condition. The attraction image 300C corresponds to the third image.

[0066] FIG. 8 shows an example in which the action or state of the user P becomes the action or state that satisfies the first condition, and the first image 200B is displayed on the left wall surface 51 including the second area. Furthermore, as shown in FIG. 8, the invitation image 300B displayed on the central wall surface 52 before the action or state of the user P satisfies the first condition may be continuously displayed on the central wall surface 52. The user P recognizes the change between the first image 200B and the attraction image 300C displayed on the left wall 51 including the second area as a stimulus that can reveal affordances, while the image displayed on the central wall 52 remains unchanged. When the user P notices the change in the image displayed on the left wall 51, the user P is able to grasp the environment of the space S1 on his own, and is more likely to get the feeling that the user P noticed and acted on his own. The first image 200B corresponds to an image obtained by changing the attraction image 300C, which is the third image.

[0067] The image of a fish included in the first image 200B displayed on the left wall surface 51 shown in FIG. 8 corresponds to the second element. The first element included in the attracting image 300B and the second element included in the first image 200B are related images. For example, if the first element is a sea creature, the second element included in the first image 200B is also highly related to being a sea creature. Furthermore, if the first element is, for example, a land animal or plant, the second element included in the first image 200B is also highly related to being a land animal or plant.

[0068] 5. Example 2 Fig. 9 is a diagram showing the state of the space S2 of the second embodiment before the first condition is met, and Fig. 10 is a diagram showing the state of the space S2 of the second embodiment after the first condition is met. As shown in FIG. 9, the space S2 is also a space surrounded by three walls, namely, a left wall 51, a central wall 52, and a right wall 53, similar to the space S1.

[0069] Similarly to space S1, space S2 also has a left camera 10A and a projector 20A installed on the left wall 51, a central camera 10B and a projector 20B installed on the central wall 52, and a right camera 10C and a projector 20C installed on the right wall 53.

[0070] In this second embodiment, before the user P's behavior or state satisfies the first condition, the projector 20 displays an invitation image 300D on the floor of the space S2. That is, in this second embodiment, the floor of the space S2 becomes the wall of the space S2 that includes the second area. When the user P moves into the space S2, the invitation image 300D is displayed near the center of the space S2, and the user P approaches the invitation image 300D to check the invitation image 300D. The invitation image 300D corresponds to the second image that indicates the second area.

[0071] FIG. 10 is a diagram showing a case where the user P approaches the attraction image 300D and the distance to the attraction image 300D is within the first distance. The second control unit 35 analyzes the image captured by any one of the left camera 10A, the center camera 10B, and the right camera 10C, and when it detects that the user P and the attraction image 300D have come closer to each other than a predetermined distance, determines that the first condition is met and identifies a projector 20 capable of displaying the first image 200C on the floor of the space S2, which is the second area. The second control unit 35 identifies the projector 20 and causes the identified projector 20 to display the first image 200C. FIG. 10 shows an example in which the projector 20A displays the first image 200C in addition to the attraction image 300D on the floor of the space S2. The attraction image 300D and the first image 200C displayed on the floor of the space S2 correspond to changing the second image to the first image.

[0072] In this second embodiment, the user P can also be stimulated to discover affordances by approaching the invitation image 300D displayed in the space S2 on his / her own initiative. Therefore, by the user P perceiving the environment of the space S2 on his / her own initiative, the user P can easily get the feeling that he / she has noticed something and acted on his / her own initiative.

[0073] 6. Example 3 Fig. 11 is a diagram showing the state of the space S3 of the third embodiment before the first condition is met, and Fig. 12 is a diagram showing the state of the space S3 of the third embodiment after the first condition is met. As shown in FIG. 9, the space S3 is also a space surrounded by three walls, namely, a left wall 51, a central wall 52, and a right wall 53, similar to the space S1.

[0074] Similarly to space S1, space S3 also has a left camera 10A and a projector 20A installed on the left wall 51, a central camera 10B and a projector 20B installed on the central wall 52, and a right camera 10C and a projector 20C installed on the right wall 53.

[0075] When the user P moves into the space S3, the second control unit 35 displays an attraction image 300 on at least one of the left wall surface 51, the central wall surface 52, and the right wall surface 53. Fig. 11 shows an example in which an attraction image 300F is displayed on the left wall surface 51 and an attraction image 300E is displayed on the central wall surface 52.

[0076] The second control unit 35 also displays the attraction image 300 on at least one of the left wall surface 51, the center wall surface 52, and the right wall surface 53, and reduces the visibility of the attraction image 300 after a certain period of time has elapsed. For example, the second control unit 35 may reduce the brightness of the space S3 to darken the space S3. The second control unit 35 may also process the attraction images 300E and 300F by blurring the images of the attraction images 300E and 300F, fading the colors of the images of the attraction images 300E and 300F, or fluctuating the images of the attraction images 300E and 300F. The second control unit 35 may also instruct the projectors 20A and 20B to temporarily hide the attraction image 300.

[0077] When a certain time has elapsed since the second control unit 35 reduced the visibility of the attraction images 300E and 300F, the second control unit 35 returns the attraction images 300E and 300F to the state before the visibility was reduced. For example, the second control unit 35 returns the brightness of the space S3 to its original brightness. Alternatively, the second control unit 35 may stop the processing of the attraction images 300E and 300F and cause the projectors 20A and 20B to display the attraction images 300E and 300F before the processing.

[0078] FIG. 12 shows images displayed on the left wall surface 51 and the center wall surface 52 after the visibility of the attraction image 300 has been restored to its original state. The second control unit 35 restores the visibility of the attraction image 300 to the original state, and changes the image displayed on the left wall surface 51, which is the second area, from the attraction image 300F to the first image 200D. User P will notice that the images displayed on left wall 51 and center wall 52 are different before visibility is reduced and after visibility is restored. This provides user P with a stimulus that allows him or her to discover affordances. Therefore, by user P understanding the environment of space S3 on his or her own, user P will be more likely to get the feeling that he or she has noticed something and acted on his or her own.

[0079] 8. Other Embodiments The above-described embodiment is a preferred embodiment of the present invention, but the present invention is not limited to the above-described embodiment and various modifications are possible within the scope of the gist of the present invention. For example, in the above-described embodiment, an example was shown in which a camera or a distance sensor was used as the sensor 10. However, the sensor 10 may be an inertial sensor or a smart device worn by the user P that detects the user P's brain waves, heart rate, pulse wave, etc. The second control unit 35 acquires sensor data measured by the smart device through short-range wireless communication with the smart device. For example, when an inertial sensor included in the smart device is used as the sensor 10, the sensor may detect the acceleration and angular velocity of the user P's movements and detect the user P's gestures. Alternatively, the sensor may detect the user P's brain waves, heart rate, and pulse wave, and detect the user P's state of tension based on the detected biological information.

[0080] In the above-described embodiment, the space S is a real space, but the space S may be a virtual space. In this case, the user P wears a head-mounted display or the like on his / her head, and the user P is allowed to view the attraction image 300 and the first image 200 in the virtual space.

[0081] Furthermore, in the examples shown in Figures 5 to 12, the space S is described as a space having three planes, namely, the left wall surface 51, the central wall surface 52, and the right wall surface 53, but it may also be configured to display an image on a single large screen.

[0082] In addition, in the above-described first to third embodiments, the central wall surface 52, which is the first region, and the left wall surface 51, which is the second region, are adjacent to each other. However, the first region and the second region may be regions located apart from each other. For example, the second region may be the left wall surface 51, and the first region may be the right wall surface 53.

[0083] Furthermore, each functional unit of the projector 20 shown in FIG. 2 indicates a functional configuration, and the specific implementation form is not particularly limited. In other words, it is not necessarily necessary to implement hardware corresponding to each functional unit individually, and it is of course possible to configure the projector 20 so that the functions of multiple functional units are realized by a single processor executing a program. Furthermore, some of the functions realized by software in the above embodiments may be realized by hardware, and some of the functions realized by hardware may be realized by software. The same applies to the information processing device 30 shown in FIG. 3.

[0084] Furthermore, the processing units in the flowchart shown in Fig. 4 are divided according to the main processing content in order to make it easier to understand the processing of the second control unit 35. The present invention is not limited by the way in which the processing units are divided or the names of the processing units shown in the flowchart of Fig. 4. In other words, the processing of the second control unit 35 can be divided into more processing units according to the processing content, or one processing unit can be divided so that it includes more processes.

[0085] Summary of this disclosure A summary of this disclosure is provided below.

[0086] (Appendix 1) 1. An image display method comprising: determining whether a user's behavior in a space satisfies a first condition based on detection results of at least one sensor; if it is determined that the user's behavior does not satisfy the first condition, presenting a visual representation in a first area in the space located at least in a first direction in which the user is facing, the visual representation inviting the user to perform a behavior that satisfies the first condition; after presenting the visual representation, determining whether the user's behavior satisfies the first condition based on the detection results of the at least one sensor; and if it is determined that the user's behavior satisfies the first condition, displaying a first image in a second area in the space.

[0087] As a result, if the user's movement does not satisfy the first condition, a visual representation that induces the user to take a movement that satisfies the first condition is presented in a first area located in the first direction to which the user is facing. By presenting the visual representation, the user's movement can be induced to satisfy the first condition. Then, if it is determined that the user's movement satisfies the first condition, the first image is displayed in a second area of ​​the space. This makes it easier for the user to notice the first image displayed in the second area. This provides the user with the feeling that they have noticed the image themselves, enhancing the user's sense of immersion.

[0088] (Appendix 2) The image display method described in Appendix 1, wherein the first condition is either that the user is facing or looking in the direction of the second area or that the user is viewing an image of the second area, the visual representation includes at least one first element moving from the first area toward the second area, and the first image includes a second element associated with the first element.

[0089] As a result, the first element presented as a visual representation moves from the first region to the second region, allowing the user to change the region of focus from the first region to the second region. Also, the first image including the second element related to the first element is displayed in the second region of focus, allowing the user to recognize the relationship between the visual representation that induces the user's action and the first image.

[0090] (Appendix 3) 2. The image display method of claim 1, wherein the visual representation includes a plurality of the first elements.

[0091] This allows the plurality of first elements presented as visual representations to move from the first region toward the second region, thereby effectively shifting the user's attention from the first region to the second region.

[0092] (Appendix 4) An image display method as described in Appendix 1, wherein the first condition is either that the user is directing their face or gaze in the direction of the second area or that they are viewing an image in the second area, and the visual representation includes making the image displayed in at least the first area difficult to view.

[0093] This makes it difficult for the user to see the image displayed in the first area as a visual representation, which can cause discomfort or a loss of interest to the user, thereby directing the user's attention to other areas of the space.

[0094] (Appendix 5) 5. The image display method according to claim 4, wherein the visual representation includes temporarily hiding an image displayed in the space.

[0095] This temporarily hides the image displayed in the space, allowing the user to search for the image and orient the user to the second area.

[0096] (Appendix 6) An image display method described in any one of Appendix 1 to Appendix 5, further comprising displaying a third image in the second area, wherein the first image is an image obtained by varying the third image.

[0097] As a result, the image displayed in the second area where the user's attention is to be drawn changes from the third image to the first image, thereby making it possible to draw the user's attention to the second area.

[0098] (Appendix 7) An image display method as described in Appendix 1, wherein the first condition is that the user is located in the second area, and presenting the visual representation is displaying a second image showing the second area, and when the user's action satisfies the first condition, the second image is changed to the first image.

[0099] This allows a visual representation to be presented to the user, guiding the user to the second area. Furthermore, when the user's action satisfies the first condition, the second image changes to the first image, guiding the user to the second area and allowing them to view the first image. This provides the user with the feeling that they have moved to the second area themselves, enhancing the user's sense of immersion.

[0100] (Appendix 8) an information processing device that executes the following steps: determine whether a user's behavior in a space satisfies a first condition based on a detection result of the at least one sensor; and, if it is determined that the user's behavior does not satisfy the first condition, present, by the at least one display device, a visual representation that induces the user to perform a behavior that satisfies the first condition in a first region in the space that is located at least in a first direction in which the user is facing; after presenting the visual representation, determine whether the user's behavior satisfies the first condition based on the detection result of the at least one sensor; and, if it is determined that the user's behavior satisfies the first condition, display a first image in a second region of the space by the at least one display device.

[0101] As a result, if the user's movement does not satisfy the first condition, a visual representation that induces the user to take a movement that satisfies the first condition is presented in a first area located in the first direction to which the user is facing. By presenting the visual representation, the user's movement can be induced to satisfy the first condition. Then, if it is determined that the user's movement satisfies the first condition, the first image is displayed in a second area of ​​the space. This makes it easier for the user to notice the first image displayed in the second area. This provides the user with the feeling that they have noticed the image themselves, enhancing the user's sense of immersion.

[0102] (Appendix 9) A program that causes a computer to determine whether a user's movement in a space satisfies a first condition based on detection results of at least one sensor; if it is determined that the user's movement does not satisfy the first condition, presenting a visual representation in a first area in the space that is located at least in a first direction in which the user is facing, which visual representation induces the user to perform a movement that satisfies the first condition; after presenting the visual representation, determining whether the user's movement satisfies the first condition based on the detection results of the at least one sensor; and if it is determined that the user's movement satisfies the first condition, displaying a first image in a second area in the space.

[0103] As a result, if the user's movement does not satisfy the first condition, a visual representation that induces the user to take a movement that satisfies the first condition is presented in a first area located in the first direction to which the user is facing. By presenting the visual representation, the user's movement can be induced to satisfy the first condition. Then, if it is determined that the user's movement satisfies the first condition, the first image is displayed in a second area of ​​the space. This makes it easier for the user to notice the first image displayed in the second area. This provides the user with the feeling that they have noticed the image themselves, enhancing the user's sense of immersion. [Explanation of symbols]

[0104] 1...Image display system, 5...Network, 10...Sensor, 10A...Left camera, 10B...Central camera, 10C...Right camera, 20, 20A, 20B, 20C...Projector, 21...First communication I / F, 22...Image processing unit, 23...Frame memory, 24...Image projection unit, 25...First control unit, 26...First memory unit, 27...First processor, 30...Information processing device, 31...Second communication I / F, 32...Operation unit, 35...Second control unit, 36...Second memory unit, 37 ...Second processor, 51...left wall, 52...center wall, 53...right wall, 200, 200A, 200B, 200C, 200D...first image, 241...light source, 243...light modulator, 245...optical unit, 265...control program, 300, 300A, 300B, 300C, 300D, 300E, 300F...attractant image, 301A...fish, 301B, 301C, 301D...other fish, 365...control program, P...user, S, S1, S2, S3...space.

Claims

1. determining whether a user's movement in the space satisfies a first condition based on a detection result of at least one sensor; When it is determined that the user's action does not satisfy the first condition, presenting a visual representation inviting the user to take an action that satisfies the first condition in a first area located in at least a first direction in the space toward which the user is facing; After presenting the visual representation, determining whether the user's action satisfies the first condition based on detection results of the at least one sensor; When it is determined that the user's action satisfies the first condition, displaying a first image in a second area of ​​the space; An image display method, including:

2. the first condition is either that the user is directing their face or gaze in the direction of the second area, or that the user is viewing an image in the second area; the visual representation includes at least one first element moving from the first region toward the second region; the first image includes a second element associated with the first element; 2. The image display method according to claim 1.

3. the visual representation includes a plurality of the first elements; 3. The image display method according to claim 2.

4. the first condition is either that the user is directing their face or gaze in the direction of the second area, or that the user is viewing an image in the second area; The image display method according to claim 1 , wherein the visual representation includes making the image displayed in at least the first area less visible.

5. The image display method according to claim 4 , wherein the visual representation includes temporarily hiding the image displayed in the space.

6. further comprising displaying a third image in the second area; the first image is an image obtained by changing the third image; The image display method according to any one of claims 1 to 5.

7. the first condition is that the user is located in the second area; presenting the visual representation comprises displaying a second image showing the second region; The image display method according to claim 1 , further comprising changing the second image to the first image when the user's action satisfies the first condition.

8. at least one sensor; at least one display device; determining whether a user's movement in the space satisfies a first condition based on a detection result of the at least one sensor; When it is determined that the user's behavior does not satisfy the first condition, presenting, by at least one display device, a visual representation that induces the user to perform a behavior that satisfies the first condition in a first area in the space that is located at least in a first direction toward which the user is facing; After presenting the visual representation, determining whether the user's action satisfies the first condition based on detection results of the at least one sensor; When it is determined that the user's action satisfies the first condition, displaying a first image in a second region of the space by the at least one display device; an information processing device that executes the above; An image display system comprising:

9. On the computer, determining whether a user's movement in the space satisfies a first condition based on a detection result of at least one sensor; When it is determined that the user's action does not satisfy the first condition, presenting a visual representation inviting the user to take an action that satisfies the first condition in a first area located in at least a first direction in the space toward which the user is facing; After presenting the visual representation, determining whether the user's action satisfies the first condition based on detection results of the at least one sensor; When it is determined that the user's action satisfies the first condition, displaying a first image in a second area of ​​the space; A program that executes.

Images