Generation device, display method, and program

Abstract

It leaves a strong impression on the viewer. [Solution] The generation device 12 includes a generation unit that generates a real-time image showing user U in real time based on a real-time captured image of user U, and generates a past image showing a past user and having a different display manner of user U than the real-time image based on a previously captured image of user U, and a display control unit that causes the display device 14 to display the real-time image and the past image side by side.

Description

[[Technical Field]]

[0001] The present disclosure relates to a generation device, a display method, and a program. [[Background Art]]

[0002] A technique is known that generates an image representing a user in a predetermined display mode from an actual image of the user. For example, Patent Document 1 describes that an avatar image is generated from a moving image of a user who is a subject. Further, Patent Document 2 describes that the appearance of an avatar image is changed based on a change in the posture of the subject. [[Prior Art Documents]] [[Patent Documents]]

[0003] [[Patent Document 1]] Japanese Unexamined Patent Application Publication No. 2016-136324 [[Patent Document 2]] Japanese Unexamined Patent Application Publication No. 2019-197519 [[Summary of the Invention]] [[Problem to be Solved by the Invention]]

[0004] However, when generating an image representing a user in a predetermined display mode from an actual image of the user, there are cases where it is required not only to provide visual information about the user to the viewer of the image, but also to make the viewer have a strong impression.

[0005] An object of the present disclosure is to provide a generation device, a display method, and a program that can make a viewer have a strong impression. [[Means for Solving the Problem]]

[0006] The generation device according to this disclosure includes a generation unit that generates a real-time image showing the user in real time based on a real-time captured image of the user, and generates a past image showing the user in the past, and in which the display manner of the user differs from that of the real-time image, based on a previously captured image of the user, and a display control unit that causes a display device to display the real-time image and the past image side by side.

[0007] The display method relating to this disclosure includes the steps of: generating a real-time image showing the user in real time based on a real-time captured image of the user; generating a past image showing the user in the past, and in which the user's appearance differs from that of the real-time image, based on a previously captured image of the user; and causing a display device to display the real-time image and the past image side by side.

[0008] The program relating to this disclosure causes a computer to perform the following steps: generate a real-time image showing the user in real time based on a real-time image of the user; generate a past image showing the user in the past, and in which the user's appearance differs from that of the real-time image, based on a previously captured image of the user; and cause a display device to display the real-time image and the past image side by side. [Effects of the Invention]

[0009] According to this disclosure, it is possible to create a strong impression on the viewer. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 is a schematic diagram showing an overview of the display system according to the embodiment. [Figure 2] Figure 2 is a schematic block diagram of the control device. [Figure 3] Figure 3 is a schematic block diagram of the generation apparatus. [Figure 4] Figure 4 is a schematic diagram showing an example of an extracted image. [Figure 5] Figure 5 is a schematic diagram showing an example of real-time images and historical images. [Figure 6] Figure 6 is a schematic diagram illustrating the display process of user images. [Figure 7] Figure 7 is a flowchart illustrating the processing flow for displaying user images. [Figure 8] Figure 8 is a schematic diagram illustrating prompt updates. [Modes for carrying out the invention]

[0011] Embodiments of this disclosure will be described in detail below with reference to the drawings. However, the embodiments described below will not limit this disclosure.

[0012] (Display system) Figure 1 is a schematic diagram showing an overview of the display system according to this embodiment. As shown in Figure 1, the display system 1 according to this embodiment includes a management device 10, a generation device 12, a display device 14, and an imaging device 16. The display system 1 uses the imaging device 16 to capture an image of user U, the management device 10 and the generation device 12 to generate a user image P from the captured image of user U, and the display device 14 to display the user image P. The user image P is an image that shows user U in a predetermined display manner. Here, the display manner of user U refers to the appearance of the image showing user U. For example, user image P may be the captured image of user U itself, a silhouette image showing the contour of user U (described later), or a generated image (described later) in which the appearance of user U is modified from the actual image of user U shown in the captured image. Details of the example of user image P in this embodiment will be described later.

[0013] (display device) The display device 14 is a device that displays a user image P. In the present embodiment, the display device 14 is a projection device that projects a user image P. The display device 14 may project the user image P onto a wall surface, a ceiling surface, a floor surface, or the like, or may project the user image P onto a screen. However, the display device 14 is not limited to a projection device, and may be a display that displays the user image P (for example, a liquid crystal type, organic EL type, or inorganic EL type display).

[0014] In the present embodiment, a plurality of display devices 14 are provided, and each display device 14 displays a user image P. In this case, each display device 14 displays the respective user image P side by side. In other words, the display areas of the user image P by each display device 14 are arranged side by side with each other. In the example of FIG. 1, three display devices 14A, 14B, and 14C are provided as the display device 14, and the display areas of the user images PA, PB, and PC by the display devices 14A, 14B, and 14C are arranged side by side. In FIG. 1, the display areas are arranged in the order of user images PC, PA, and PB. In other words, the user image PA is located at the center, but the arrangement order of the display areas is not limited to this and may be arbitrary. Further, the number of display devices 14 is not limited to three, and may be any plural number. Further, there may be one display device 14, and in this case, a plurality of user images P may be displayed by the single display device 14.

[0015] (Imaging Device) The imaging device 16 is a camera that captures an image of a user U. The imaging device 16 captures an image of the user U at every predetermined frame rate. In other words, the imaging device 16 captures a moving image of the user U. The imaging device 16 may be disposed at any position where the user U can be imaged. However, in the present embodiment, the imaging device 16 is disposed at a position where a space facing an area (display surface) on which the user image P is displayed by the display device 14 can be imaged. That is, in this case, it can be said that the imaging device 16 images the space facing the display surface, thereby imaging the user U located in the space.

[0016] The imaging device 16 images the user U and detects the position of the user U relative to the position of the imaging device 16 (or the display device 14). That is, the imaging device 16 may include an optical element such as a lens, an image sensor such as CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor), and a position sensor 17 that measures the position of a subject relative to the imaging device 16 (or the display device 14). The position sensor 17 detects the position of the subject (here, the user U) imaged by the imaging device 16. In the present embodiment, the position sensor 17 may be a distance measuring sensor, and detects the distance between the imaging device 16 (or the display area of the display device 14) and the user U captured in the captured image. When the position sensor 17 is a distance measuring sensor incorporated in the imaging device 16, the position information (distance information) of the subject (here, the user U) detected by the position sensor 17 is a distance image that is an image including distance information of the subject. Note that the position sensor 17 is not limited to being incorporated in the imaging device 16, and may be a separate component from the imaging device 16. The position sensor 17 may be, for example, an optical sensor, and may be a ToF (Time of Flight) sensor, LiDAR (Light Detection And Ranging), or the like.

[0017] (Management Device) Figure 2 is a schematic block diagram of the management device. The management device 10 is a device that acquires an image captured by the imaging device 16 and transmits information for generating a user image P to the generation device 12. The management device 10 is, for example, a computer, and includes a storage unit 20, a communication unit 22, and a control unit 24 as shown in Figure 2. Note that the management device 10 is not particularly limited, and may be configured as a single device, may be configured integrally with another device, or may be configured as a system combining various devices such as an arithmetic device and a data server. Further, although the management device 10 is a separate device from the imaging device 16, it may be a control device of the imaging device 16 incorporated in the imaging device 16.

[0018] The memory unit 20 is a memory that stores various information such as the calculation contents and programs of the control unit 24, and includes at least one of the following: RAM (Random Access Memory), main memory such as ROM (Read Only Memory), and external memory such as HDD (Hard Disk Drive). The program for the control unit 24 stored in the memory unit 20 may be stored on a recording medium that can be read by the management device 10. The communication unit 22 is a communication module that communicates with external devices such as the imaging device 16 and the generation device 12, and includes, for example, an antenna, an RF (Radio Frequency) circuit, a wireless LAN (Local Area Network) card, and other communication processing circuits. The communication unit 22 communicates with external devices via wireless communication, but the communication method may be arbitrary.

[0019] The control unit 24 is an arithmetic unit and includes arithmetic circuits such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The control unit 24 executes processing by reading and executing a program (software) from the storage unit 20.

[0020] The control unit 24 includes an image acquisition unit 30, a position acquisition unit 32, an image extraction unit 34, a prompt setting unit 36, and an information transmission unit 38. The control unit 24 reads a program (software) from the storage unit 20 and executes it to realize the image acquisition unit 30, the position acquisition unit 32, the image extraction unit 34, the prompt setting unit 36, and the information transmission unit 38, and then performs their processing. The control unit 24 may perform these processing using one arithmetic circuit, or it may have multiple arithmetic circuits and perform the processing using those multiple arithmetic circuits. Furthermore, at least a portion of the processing of the image acquisition unit 30, the position acquisition unit 32, the image extraction unit 34, the prompt setting unit 36, and the information transmission unit 38 may be realized by hardware circuits.

[0021] The processing details of the control unit 24 will be described later.

[0022] (Generation device) The generation device 12 is a device that generates a user image P and displays the generated user image P on the display device 14. In this embodiment, multiple generation devices 12 are provided. In this embodiment, a generation device 12 is provided for each display device 14, and in the example of Figure 1, three generation devices 12A, 12B, and 12C are provided. Generation device 12A corresponds to display device 14A and generates a user image PA to be displayed on display device 14A. Generation device 12B corresponds to display device 14B and generates a user image PB to be displayed on display device 14B. Generation device 12C corresponds to display device 14C and generates a user image PC to be displayed on display device 14C. However, the generation device 12 is not limited to being provided for each display device 14, and there may be any number of them, or there may be just one. If a generation device 12 is not provided for each display device, one generation device 12 may generate user images P to be displayed on multiple display devices 14.

[0023] Figure 3 is a schematic block diagram of the generation device. The generation device 12 is, for example, a computer and, as shown in Figure 3, has a storage unit 40, a communication unit 42, and a control unit 44. The generation device 12 may be a standalone device, an integrated device with other devices, or a system combining various devices such as a computing unit and a data server, and is not particularly limited. Furthermore, although the generation device 12 is a separate device from the display device 14, it may also be a control device for the display device 14 that is incorporated into the display device 14.

[0024] The storage unit 40 is a memory that stores various information such as the calculation contents and programs of the control unit 44, and includes at least one of the following: RAM, main memory such as ROM, and external storage device such as HDD. The program for the control unit 44 stored in the storage unit 40 may be stored on a recording medium that the generation device 12 can read. The communication unit 42 is a communication module that communicates with external devices such as the management device 10 and the display device 14, and is an antenna or a communication module, for example. The communication unit 42 communicates with external devices by wireless communication, but the communication method may be arbitrary.

[0025] The control unit 44 is an arithmetic unit and includes arithmetic circuits such as a CPU or GPU. The control unit 44 executes processing by reading and executing a program (software) from the storage unit 40.

[0026] The control unit 44 includes an information acquisition unit 50, an information generation unit 52, and a display control unit 54. The control unit 44 reads a program (software) from the storage unit 40 and executes it to realize the information acquisition unit 50, the generation unit 52, and the display control unit 54, and then performs their respective processes. The control unit 44 may perform these processes using a single arithmetic circuit, or it may have multiple arithmetic circuits, and the processes may be performed using these multiple arithmetic circuits. Furthermore, at least a portion of the processing of the information acquisition unit 50, the generation unit 52, and the display control unit 54 may be realized by hardware circuits.

[0027] The processing details of the control unit 44 will be described later.

[0028] In this embodiment, the management device 10 and the generation device 12 are separate devices, but they may be configured as a single device. That is, the generation device 12 may also incorporate the functions of the management device 10 and perform at least a part of the processing of the management device 10, which will be described later.

[0029] (Display system processing) Next, we will explain the display process of the user image P by the display system 1.

[0030] (Acquisition of captured images) The image acquisition unit 30 of the management device 10 acquires images captured by the imaging device 16. When user U is positioned in the space opposite the display surface of the display device 14, user U will be captured in the image captured by the imaging device 16. Hereafter, the image captured by the imaging device 16 will be referred to as captured image PR. The imaging device 16 captures images at a predetermined frame rate, and the image acquisition unit 30 sequentially acquires the captured image PR captured at each predetermined frame rate. The image acquisition unit 30 may control the imaging device 16 to perform imaging, or the imaging device 16 may perform imaging under the control of a control device built into it, and the image acquisition unit 30 may acquire the captured image via communication.

[0031] (Acquiring user location information) The position acquisition unit 32 of the management device 10 acquires the position information of user U as seen in the captured image PR. The position information of user U refers to the position of user U relative to the position of the imaging device 16 (or the display area of ​​the display device 14) (i.e., the relative position of user U with respect to the display area of ​​the imaging device 16 or the display device 14), and in this embodiment, it is the distance between the imaging device 16 (or the display area of ​​the display device 14) and user U. Hereafter, the position information of user U will be referred to as the distance information of user U as appropriate. That is, the distance information (position information) of user U is information indicating the distance between the imaging device 16 (or the display area of ​​the display device 14) and user U, and the positional relationship between user U and the imaging device 16 (or the display area of ​​the display device 14) can be defined by the distance information of user U. The position acquisition unit 32 acquires the distance information of user U detected by the position sensor 17. The position acquisition unit 32 may control the position sensor 17 to perform position detection, or the position sensor 17 may perform position detection under the control of a control device built into it, and the position acquisition unit 32 may acquire the position detection result via communication.

[0032] In this embodiment, the position acquisition unit 32 acquires distance information for each part of the user U that is captured in the captured image PR (in this embodiment, the distance between each part and the imaging device 16 (or the display area of ​​the display device 14)) as distance information for the user U. Specifically, the position sensor 17 detects distance information for each pixel of the captured image PR, and the position acquisition unit 32 acquires the distance information for each pixel of the subject that has been detected by the position sensor 17.

[0033] The position acquisition unit 32 acquires distance information of the user U that is captured in the captured image PR each time an captured image PR is acquired.

[0034] (Generating extracted images) Figure 4 is a schematic diagram showing an example of an extracted image. As shown in Figure 4, the image extraction unit 34 of the management device 10 generates (extracts) an extracted image PE, which is an image obtained by extracting the actual image of user U that is captured in the captured image PR from the acquired captured image PR. The extracted image PE can be said to be an image that includes only the area in which user U is captured out of the entire area of ​​the captured image PR. That is, the captured image PR is an image in which objects within the field of view of the imaging device 16 are captured, and is an image that includes not only user U but also surrounding objects and background. The image extraction unit 34 extracts the area in which user U is captured (the area surrounded by the outline of user U captured in the captured image PR) from the captured image PR and makes it the extracted image PE. In other words, the periphery of the extracted image PE is the outline of user U, or in other words, the boundary line between user U and things other than user U.

[0035] The method for obtaining the extracted image PE is arbitrary, and the image extraction unit 34 may extract the extracted image PE using any object extraction method. Alternatively, for example, the image extraction unit 34 may extract the extracted image PE based on the captured image PR and the distance information of user U. In this case, for example, the image extraction unit 34 performs image analysis on the captured image PR to extract objects that are captured in the captured image PR. Then, the image extraction unit 34 identifies the area in the captured image PR where user U is captured using the distance information of user U, and extracts the image of the object in the identified area as the extracted image PE.

[0036] The image extraction unit 34 may include in the extracted image PE objects other than the user U that come into contact with the user U (including objects held by the user U) among the objects captured in the captured image PR. In this case, the extracted image PE is an image that includes the user U and the objects that come into contact with the user U. In this case, the periphery of the extracted image PE is extended so that objects other than the user U are part of the user U, becoming the boundary line between the user U (including objects that come into contact with the user U) and objects other than the user U (including objects that come into contact with the user U). In this case, the conversion unit 242 extracts the region in which the user U is captured and the region in which objects that come into contact with the user U are captured as the extracted image PE. The method of extracting the extracted image PE in this case is also arbitrary, but for example, the image extraction unit 34 may extract the user U and the objects that come into contact with the user U as a single object by image analysis of the captured image PR, and use the image of the extracted object as the extracted image PE.

[0037] The image extraction unit 34 extracts an extracted image PE each time an acquired image PR is obtained.

[0038] (Prompt settings) The prompt setting unit 36 ​​of the management device 10 sets prompts for the user image P to be displayed on the display device 14. A prompt is a command that specifies the display manner of user U in the user image P. More specifically, the prompt in this embodiment is a command that specifies the display manner in the generated image described later. For example, prompts can be predetermined objects such as animals, plants, or robots, or predetermined patterns such as waves. That is, for example, if an animal is set as the prompt, the generated image will be an image that displays user U as an animal (an image that imitates user U as an animal).

[0039] The prompt setting unit 36 ​​sets a prompt for each display device 14. In this embodiment, since a generation device 12 is provided for each display device 14, it can also be said that the prompt setting unit 36 ​​sets a prompt for each generation device 12. That is, in the example shown in Figure 1, the prompt setting unit 36 ​​sets a prompt for display device 14A (generation device 12A), a prompt for display device 14B (generation device 12B), and a prompt for display device 14C (generation device 12C).

[0040] The prompt setting unit 36 ​​may set the prompt in any way. In this embodiment, the prompt setting unit 36 ​​sets the prompt automatically and randomly, but is not limited to this; an initial prompt may be set in advance for each display device 14, or a prompt specified by the administrator of the management device 10 may be applied.

[0041] It is preferable that the prompt setting unit 36 ​​sets different prompt content (type of display mode indicated by the prompt) for each display device 14. Furthermore, as will be described later, the prompt setting unit 36 ​​may update the prompt content midway through the process.

[0042] (Sending information) The information transmission unit 38 of the management device 10 transmits the actual image of user U captured by the imaging device 16 to the generation device 12. In this embodiment, the information transmission unit 38 transmits the extracted image PE, in which user U is extracted from the captured image PR, to the generation device 12, but is not limited to this, and may transmit the captured image PR itself.

[0043] The information transmission unit 38 associates the user U's actual image (extracted image PE in this embodiment) with the user U's distance information (the distance between the imaging device 16 or the display device 14 and the user U), and transmits the associated user U's actual image and user U's distance information to the generation device 12. Furthermore, in this embodiment, since distance information for each part of the user U is acquired, the information transmission unit 38 transmits the user U's actual image and the distance information for each part of the user U in association to the generation device 12.

[0044] Furthermore, the information transmission unit 38 transmits to the generation device 12 prompt information set for the display device 14 associated with the generation device 12. In other words, in the example shown in Figure 1, the information transmission unit 38 transmits the prompt for display device 14A to the generation device 12A, the prompt for display device 14B to the generation device 12B, and the prompt for display device 14C to the generation device 12C. When a prompt is updated, the information transmission unit 38 transmits the information of the switched prompt to the generation device 12.

[0045] (Information acquisition) The information acquisition unit 50 of the generation device 12 acquires the actual image of user U (extracted image PE in this embodiment) from the management device 10. More specifically, in this embodiment, the information acquisition unit 50 of the generation device 12 acquires the distance information between the associated actual image of user U and user U. More specifically, in this embodiment, the information acquisition unit 50 of the generation device 12 acquires the distance information between the associated actual image of user U and user U for each part of user U.

[0046] Furthermore, the information acquisition unit 50 of the generation device 12 acquires prompt information set for the display device 14 associated with its generation device 12 from the management device 10. In other words, in the example in Figure 1, the information acquisition unit 50 of generation device 12A acquires prompts for display device 14A, the information acquisition unit 50 of generation device 12B acquires prompts for display device 14B, and the information acquisition unit 50 of generation device 12C acquires prompts for display device 14C.

[0047] (Generating and displaying user images) The generation unit 52 of the generation device 12 generates a user image P representing user U in a predetermined display manner based on the captured image PR (extracted image PE in this embodiment) of user U. The display control unit 54 of the generation device 12 causes the user image P generated by the generation unit 52 to be displayed on the display device 14. The display control unit 54 may transmit the user image P to the display device 14 so that the control unit of the display device 14 displays the received user image P on the display device 14, or the display control unit 54 may control the display device 14A to display the user image P on the display device 14A.

[0048] The user image P will be explained in detail below.

[0049] (Real-time images and historical images) Figure 5 is a schematic diagram showing an example of a real-time image and a past image. The generation unit 52 of the generation device 12 generates at least one of the following as user images P: a real-time image showing a real-time user U and a past image showing a past user U. The display control unit 54 of the generation device 12 then displays the real-time image and the past image side by side at the same time. In this embodiment, some generation devices 12 generate real-time images and other generation devices 12 generate past images, but if there is only one generation device 12, for example, that one generation device 12 may generate both real-time images and past images. Hereafter, we will explain using as an example the case in which generation device 12A generates a real-time image, display device 14A displays that real-time image, generation devices 12B and 12C generate past images, and display devices 14B and 14C display those past images.

[0050] (Real-time image) The generation unit 52 of the generation device 12A generates a real-time image of user U as a user image PA. The generation unit 52 of the generation device 12A generates a real-time image of user U based on a real-time captured image PR of user U. More specifically, in this embodiment, the generation unit 52 of the generation device 12A generates a real-time image of user U based on a real-time extracted image PE of user U. The real-time captured image PR of user U refers to the latest captured image PR captured by the imaging device 16, and the real-time extracted image PE of user U refers to the extracted image PE generated from the real-time captured image PR of user U. However, the real-time captured image PR of user U is not limited to the latest captured image PR, but may be an captured image PR captured in a predetermined number of frames prior to the latest.

[0051] The generation unit 52 of the generation device 12A generates a real-time image of the actual image of the user U captured in the real-time captured image PR (in this embodiment, the real-time extracted image PE) in a predetermined display manner. The display manner of the user U in the real-time image can be set arbitrarily, and may be the same display manner as the captured image PR (i.e., the actual image of the user U), or it may be the display manner of the silhouette image described later, or it may be the display manner of the generated image described later. In this embodiment, the real-time image is displayed by switching between the first image P1 (silhouette image) and the second image P2 (generated image) described later, which will be explained in detail later.

[0052] The generation unit 52 of the generation device 12A updates the real-time image each time a real-time captured image PR is acquired (captured), that is, each time a real-time extracted image PE is acquired in this embodiment. In other words, the generation unit 52 of the generation device 12A sequentially generates a real-time image based on the latest acquired captured image PR (extracted image PE). The display control unit 54 of the generation device 12A causes the display device 14A to display the generated real-time image. The display control unit 54 of the generation device 12A displays the updated real-time image on the display device 14A each time the real-time image is updated. Therefore, the real-time image is an image that reflects the real-time behavior of the user U (an image that displays the real-time behavior of the user U in a predetermined display manner). The behavior of the user U refers to the movement of the user U's body.

[0053] (Past images) The generation unit 52 of the generation device 12B generates past images of user U as user images PB. Past images of user U are images that show user U in the past compared to the real-time image, and can be described as images that are delayed versions of the real-time captured image PR (extracted image PE in this embodiment). The generation unit 52 of the generation device 12B generates past images of user U based on past captured images PR of user U. More specifically, in this embodiment, the generation unit 52 of the generation device 12B generates past images of user U based on past extracted images PE of user U. Past captured images PR of user U refer to captured images PR taken by the imaging device 16 in the past compared to the real-time captured images PR of user U, and past extracted images PE of user U refer to extracted images PE generated from those past captured images PR. The time difference between the timing when past captured images PR of user U were taken and the timing when real-time captured images PR of user U were taken can be any length, but for example it may be about 1 to 60 seconds, or about 5 to 10 seconds.

[0054] The generation unit 52 of the generation device 12B generates an image as a past image, which is an image of a past user U captured image PR (in this embodiment, an extracted image PE of a past user U) in a predetermined display manner. The display manner of user U in the past image is set to be different from the display manner of user U in the real-time image. The display manner of user U in the past image may be any display manner different from the display manner of user U in the real-time image, for example, it may be the same display manner as the captured image PR (i.e., the actual image of user U), or it may be the display manner of the silhouette image described later, or it may be the display manner of the generated image described later. That is, for example, if the real-time image is an actual image, the past image may be a silhouette image or a generated image. Also, for example, if the real-time image is a silhouette image, the past image may be an actual image or a generated image. Also, for example, if the real-time image is a generated image, the past image may be an actual image, a silhouette image, or a generated image with a different display manner from the real-time image (a generated image generated with a different prompt than the real-time image). In this embodiment, past images are displayed by switching between a first image P1 (silhouette image) and a second image P2 (generated image) which has a different display mode than the real-time image. More details will be described later.

[0055] The generation unit 52 of the generation device 12B updates past images each time a captured image PR of user U is acquired (captured). The generation unit 52 of the generation device 12B updates past images in such a way that it maintains a constant time difference between the capture timing of the captured image PR used for generating real-time images and the capture timing of the captured image PR used for generating past images. That is, if the time difference between the real-time image and the past image is set to a first predetermined time, the generation unit 52 of the generation device 12B updates the past image using a captured image PR captured by the first predetermined time before the latest real-time captured image PR. The first predetermined time here can be set arbitrarily, for example, 5 seconds. The display control unit 54 of the generation device 12B causes the display device 14B to display the updated past image. The display control unit 54 of the generation device 12B displays the updated past image on the display device 14B each time the past image is updated. Therefore, the past image is an image that reflects the past behavior of user U (i.e., an image that displays the past behavior of user U in a different display manner than the real-time image).

[0056] Furthermore, if multiple past images are displayed at the same time, it is preferable that these past images reflect the behavior of user U at different past times. In other words, in this case, it is preferable that the generation unit 52 generates multiple past images showing user U at different past times based on the captured image PR of user U taken at different past times, and displays these past images side by side.

[0057] For example, in this embodiment, since the generation devices 12B and 12C generate past images, the generation unit 52 of generation device 12C generates past images based on captured images PR taken at a different past timing than the captured images PR used by the generation unit 52 of generation device 12B. The display control unit 54 of generation device 12C then displays the past images on the display device 14C. That is, for example, if generation device 12B generates past images using captured images PR taken a first predetermined time before (for example, 5 seconds before) the real-time captured images PR, generation device 12C may generate past images using captured images PR taken a second predetermined time before the real-time captured images PR. In this case, the second predetermined time may be any length different from the first predetermined time, but in this embodiment it is set to a longer time than the first predetermined time, for example, 10 seconds.

[0058] The method for generating past images using the generation device 12C and the method for displaying past images (user image PC) using the display device 14C are the same as the method for generating past images using the generation device 12B and the method for displaying past images (user image PB) using the display device 14B, except that they use captured images PR taken at different times in the past. Therefore, a detailed explanation is omitted.

[0059] In the example in Figure 5, the behavior of user U changes in the order of behavior TC, TB, and TA according to the time series, and the captured image PR of user U in behavior TA becomes the real-time captured image PR of user U. Therefore, in this case, the user U shown in the real-time image displayed by display device 14A switches in the order of behavior TC, TB, and TA, and at the timing in Figure 5, display device 14A displays the real-time image reflecting user U in behavior TA as the user image PA. Also at the timing in Figure 5, display device 14B displays a past image reflecting user U in past behavior TB as the user image PB, and display device 14C displays a past image reflecting user U in behavior TC, which is even further back than behavior TB, as the user image PC. In this case, since the display areas of display devices 14A, 14B, and 14C are set side by side, it can be said that the real-time image and the past image are displayed side by side.

[0060] As described above, displaying past images and real-time images side by side can leave a strong impression on the viewer. For example, user U can simultaneously view a real-time image that reflects their current self and a past image that reflects their past self but is displayed in a different way. Consequently, user U can perceive a sense of being themselves but not themselves, or the existence of multiple selves, resulting in a stronger impression than simply viewing their own image.

[0061] (Image 1 and Image 2) The generation unit 52 of the generation device 12 generates a first image P1 that displays user U in a predetermined display manner, and a second image P2 that displays user U in a different display manner than the first image P1. The display control unit 54 of the generation device 12 then switches between displaying the first image P1 and the second image P2 as the user image P. A more detailed explanation follows below.

[0062] (First image) The generation unit 52 generates the first image P1 of user U based on the captured image PR of user U. More specifically, in this embodiment, the generation unit 52 generates the first image P1 of user U based on the extracted image PE of user U.

[0063] The generation unit 52 generates an image as the first image P1, which is an image of user U's captured image PR (in this embodiment, user U's extracted image PE) in a predetermined display manner. In this embodiment, the generation unit 52 generates a silhouette image of user U as the first image P1. A silhouette image is an image that shows the contour of user U, and is an image in which the contour of user U is displayed in a display manner different from that of user U's actual image. In other words, a silhouette image is an image in which the gradation value of each pixel within the region enclosed by user U's contour is different from that of user U's extracted image PE. Therefore, the first image P1 is an image in which the contour reflects the actual image of user U, but the display manner within the contour is different from that of user U's actual image.

[0064] The generation unit 52 generates a first image, which is a silhouette image of user U, by setting the grayscale value of each pixel in the extracted image PE of user U (i.e., within the area of ​​user U's contour in the captured image PR of user U) to a predetermined grayscale value. The grayscale value of each pixel within the area enclosed by user U's contour in the silhouette image may be set arbitrarily, for example, a common grayscale value may be set for each pixel, or different grayscale values ​​may be set for each pixel.

[0065] Furthermore, the first image P1, which is a silhouette image, is not limited to having an outline that exactly matches the outline of the extracted image PE; its outline may differ from that of the extracted image PE. In this case, for example, the first image P1 may be an image with blurred outline lines.

[0066] The generation unit 52 updates the first image P1 each time a captured image PR of user U is acquired (captured), that is, each time an extracted image PE is acquired in this embodiment. Therefore, the first image P1 is an image that reflects the behavior of user U.

[0067] Furthermore, the first image P1 is not limited to a silhouette image of user U, but may be an image that shows user U in any display manner. For example, the first image P1 may be an image that shows user U in the same display manner as the captured image PR (i.e., an actual image of user U), or it may be a generated image as described later.

[0068] (Second image) The generation unit 52 generates a second image P2 of user U based on the captured image PR of user U. More specifically, in this embodiment, the generation unit 52 generates a second image P2 of user U based on the extracted image PE of user U.

[0069] The generation unit 52 generates an image as the second image P2, which is a display of the captured image PR of user U (extracted image PE of user U in this embodiment) in a different display manner than the first image P1. In this embodiment, the generation unit 52 generates the generated image of user U as the second image P2. The generated image is an image in which the appearance of user U has been changed from the actual image of user U shown in the captured image PR (extracted image PE). More precisely, the generated image is an image in which the appearance (display manner) of user U as seen in the captured image PR (extracted image PE) has been changed based on a prompt, and can also be said to be an image that shows user U in the display manner indicated by the prompt. That is, for example, if the prompt is an animal, the generated image will be an image in which the appearance of user U has been changed to that of an animal. Note that the contour of user U shown in the first image P1, which is a silhouette image, matches the contour of the extracted image PE, which is the actual image, but the contour of user U shown in the second image P2, which is the generated image (an image in which the appearance of user U has been changed), does not have to match the contour of the extracted image PE, which is the actual image.

[0070] The generation unit 52 sets the second image P2, which is the image generated by user U, based on the captured image PR (extracted image PE of user U in this embodiment) of user U and the prompt acquired by the information acquisition unit 50. That is, the generation unit 52 generates an image to be shown to user U in the display mode indicated by the prompt, based on the captured image PR (extracted image PE) of user U and the prompt, and sets it as the second image P2. The method for generating the second image P2 based on the captured image PR and the prompt may be arbitrary, but for example, a known image generation AI tool may be used.

[0071] In this embodiment, a prompt is set for each display device 14 (generation device 12), so the generation unit 52 sets the second image P2 based on the prompt set for its own generation device 12. In the example in Figure 1, generation device 12A generates the second image P2 using the prompt for generation device 12A (display device 14A), generation device 12B generates the second image P2 using the prompt for generation device 12B (display device 14B), and generation device 12C generates the second image P2 using the prompt for generation device 12C (display device 14C). Therefore, the second image P2 generated by each generation device 12 will be images with different display modes (appearance of user U) corresponding to their respective prompts.

[0072] The generation unit 52 updates the second image P2 each time a captured image PR of user U is acquired (captured), that is, each time an extracted image PE is acquired in this embodiment. Therefore, the second image P2 is an image that reflects the behavior of user U.

[0073] Furthermore, the second image P2, which is the image generated by user U, is not limited to being generated by an image generation AI tool based on the captured image PR and prompts, but may be generated by any method based on the captured image PR. For example, the second image P2, which is the image generated by user U, may be an avatar that shows user U in a display manner different from the actual image.

[0074] However, the second image P2 is not limited to an image generated by user U, and may be an image that shows user U in any display manner different from the first image P1. For example, if the first image P1 is a silhouette image, the second image P2 may be a real image or a generated image. Also for example, if the first image P1 is a real image, the second image P2 may be a silhouette image or a generated image. Also for example, if the first image P1 is a generated image, the second image P2 may be a real image, a silhouette image, or a generated image in a display manner different from the first image P1 (a generated image generated with a different prompt than the first image P1).

[0075] In this way, the generation unit 52 generates two types of images, a first image P1 and a second image P2, which are two types of images with different display modes, based on the same captured image PR (same extracted image PE). However, the generation unit 52 is not limited to generating only two types of images based on the same captured image PR (same extracted image PE), but may generate three or more types of images with different display modes.

[0076] (Switching between the first and second images) The display control unit 54 causes the display device 14 to switch between displaying the first image P1 and the second image P2, which are generated based on the same captured image PR (same extracted image PE), as user images P. If three or more images are generated based on the same captured image PR, the display control unit 54 may switch between displaying those three or more images on the display device 14.

[0077] In this embodiment, the generation device 12 switches between the first image P1 and the second image P2, which are generated based on the captured image PR, based on the distance information of the user U captured in the captured image PR, that is, based on the positional relationship between the display area of ​​the display device 14 and the user U. For example, the generation device 12 may switch between the first image P1 and the second image P2 depending on the distance between the display area and the user U. That is, for example, the generation device 12 may display the first image P1 as the user image P when the distance between the display area and the user U is greater than or equal to a predetermined threshold, and display the second image P2 as the user image P when the distance between the display area and the user U is less than a predetermined threshold. Also, for example, if there are three or more images to be switched, the images may be switched depending on the distance between the display area and the user U. That is, for example, as the distance between the display area and the user U decreases, the first image (e.g., real image), the first image (e.g., silhouette image), and the second image (e.g., generated image) may be switched in this order.

[0078] The generation device 12 generates the user image P such that at least one of the first image P1 and the second image P2 occupies the entire area of ​​the user image P. In this case, the generation device 12 switches between the first image P1 and the second image P2 by changing the area occupied by the first image P1 and the area occupied by the second image P2 within the entire area of ​​the user image P. That is, for example, the generation device 12 may switch between a user image P in which the entire area (all pixels) is the first image P1, a user image P in which a part of the entire area is the first image P1 and the remainder is the second image P2 (i.e., a user image P in which the entire area is the second image P2), and an image in which the entire area is the second image P2. Note that the user image P in which the first image P1 and the second image P2 are combined may be, for example, an image in which the contour is the second image P2, a part of the area within the contour is the first image P1 and the remaining area is the second image P2.

[0079] In this embodiment, the generation device 12 generates a user image P by changing the area occupied by the first image P1 and the area occupied by the second image P2 based on the distance information of the user U. The generation device 12 generates the user image P such that the area occupied by the first image P1 becomes smaller and the area occupied by the second image P2 becomes larger as the distance between the display area and the user U decreases. In other words, as the distance between the display area and the user U decreases, the area where the first image P1 is displayed in the user image P gradually switches to the area where the second image P2 is displayed.

[0080] In this embodiment, the generation device 12 sets whether each part (pixel) of the user U should be designated as either the first image P1 or the second image P2, based on distance information for each part (pixel) of the user U. In this case, for example, the generation device 12 sets the grayscale value of the pixel corresponding to a part to be the grayscale value of the first image P1 at that pixel for parts whose distance from the display area is greater than or equal to a predetermined threshold, and sets the grayscale value of the pixel corresponding to a part to be the grayscale value of the second image P2 at that pixel for parts whose distance from the display device 14 is less than a predetermined threshold, and generates an image as the user image P. Therefore, for example, if the legs of user U are far from the display area but other parts are close to the display area, the contour will be designated as the second image P2, and an image will be generated as the user image P in which the legs are the first image P1 and the other parts are the second image P2.

[0081] However, the generation device 12 is not limited to setting whether each part should be designated as the first image P1 or the second image P2. For example, if the distance between the user U's reference position (e.g., the position of the head) and the display area is greater than or equal to a predetermined first threshold, the generation device 12 may generate and display a user image P in which the entire area is designated as the first image P1. If the distance between the user U's reference position and the display area is less than a predetermined first threshold but greater than or equal to a second threshold that is shorter than the first threshold, the generation device 12 may generate and display a user image P in which a portion of the entire area is designated as the first image P1 and the remainder as the second image P2. If the distance between the user U's reference position and the display area is less than the second threshold, the generation device 12 may generate and display a user image P in which the entire area is designated as the second image P2. Furthermore, if the distance between the user U's reference position and the display area is less than the first threshold and greater than or equal to the second threshold, as this distance decreases, the area where the first image P1 is displayed will gradually switch to the second image P2, the area occupied by the first image P1 will decrease, and the area occupied by the second image P2 will increase.

[0082] Furthermore, in the above explanation, the switching between the first image P1 and the second image P2 was performed based on the distance information of user U, but it is not limited to this, and the switching between the first image P1 and the second image P2 may be performed by any trigger. For example, the generation device 12 may switch between the first image P1 and the second image P2 according to the passage of time. That is, for example, if the elapsed time since user U appeared in the captured image PR is less than a first predetermined time, the generation device 12 may display the first image P1 as the user image P. Then, if the elapsed time since user U appeared in the captured image PR is at least a first predetermined time and less than a second predetermined time which is longer than the first predetermined time, the generation device 12 may generate and display a user image P in which a portion of the entire area is the first image P1 and the remainder is the second image P2. Then, if the elapsed time since user U appeared in the captured image PR is at least a second predetermined time, the generation device 12 may generate and display a user image P in which the entire area is the second image P2. Furthermore, if the elapsed time is greater than or equal to the first predetermined time and less than or equal to the second predetermined time, the area where the first image P1 is displayed will gradually switch to the second image P2 as time progresses, the area occupied by the first image P1 will decrease, and the area occupied by the second image P2 will increase. Also, when switching images according to the passage of time in this manner, if there are three or more images to be switched, those images may be switched according to the passage of time. That is, for example, the first image (e.g., real image), the first image (e.g., silhouette image), and the second image (e.g., generated image) may be switched in this order as time progresses.

[0083] Furthermore, in the above explanation, the generation device 12 generated both the first image P1 and the second image P2 regardless of whether they were to be used as the user image P. However, the generation device 12 may generate only the image that is to be used as the user image P from among the first image P1 and the second image P2. That is, for example, when only the first image P1 is to be used as the user image P, the generation device 12 may generate only the first image P1 and not the second image P2. Similarly, when only the second image P2 is to be used as the user image P, the generation device 12 may generate only the second image P2 and not the first image P1.

[0084] As described above, by switching between displaying the first image P1 and the second image P2, a strong impression can be made on the viewer. For example, user U will see an image that reflects themselves, but whose display manner changes. Therefore, user U can feel a sense of being themselves but not themselves, or the existence of multiple selves, and will have a stronger impression than simply seeing an image of themselves. Also, for example, if the display device 14 is an interface to a virtual space, the sense of immersion in the virtual space can be enhanced by gradually changing from a real image (e.g., the first image) to a generated image (e.g., the second image).

[0085] (Displaying user images) In this embodiment, the display process for the user image P involves both displaying the real-time image and the past image side by side as described above, and switching between displaying the first image P1 and the second image P2. That is, the generation device 12 generates a real-time image including the first image P1 and the second image P2 that represent the real-time user U, based on the real-time captured image PR of the user U. The generation device 12 then generates a past image including the first image P1 and the second image P2 that represent the past user U, based on the captured image PR of the user U that was captured in the past. The generation device 12 then displays the real-time image and the past image side by side on the display device 14, while switching between the first image P1 and the second image P2. This display process will be described in detail below.

[0086] Figure 6 is a schematic diagram illustrating the display process of user images. As shown in step S1 of Figure 6, if user U is not located within the space SP (the space within the imaging range of the imaging device 16) opposite the display area of ​​the display device 14, user U will not be captured in the captured image PR, and therefore the display device 14 will not display the user image P representing user U.

[0087] As shown in step S2 of Figure 6, when user U enters space SP from outside space SP, user U is captured in the captured image PR, and the display device 14 displays user image P representing user U. In step S2, since the distance between the display area and user U is a certain distance (for example, the distance between the display area and user U is above a predetermined threshold), each display device 14 displays user image P with the entire area as the first image P1 (silhouette image). More specifically, display device 14A displays a real-time image with the entire area as the first image P1 as user image PA, while display devices 14B and 14C display past images with the entire area as the first image P1 as user image PB and PC, respectively. In the example shown in Figure 6, for the sake of explanation, the user image P is displayed on all display devices 14. However, after the user U is positioned within the spatial SP (after the user U is captured in the image PR), the real-time user image PA is displayed first on display device 14A, followed by the past user image PB on display device 14B, and then the even older user image PC on display device 14C.

[0088] As shown in step S3 of Figure 6, if user U approaches the display area within space SP more closely than in step S2, the display device 14 switches a portion of the first image P1 to the second image P2 and displays the user image P. In step S2, since the distance between the display area and user U has approached to a certain extent, each display device 14 displays a user image P in which a portion of the area is the first image P1 (silhouette image) and the remaining area is the second image P2 (generated image). More specifically, in the example in Figure 6, since only the parts of user U other than the legs have approached the display area to a distance less than a predetermined threshold, the user image P is displayed in which the area corresponding to the parts other than the legs becomes the second image P2 and the area corresponding to the legs becomes the first image P1. More specifically, in step S3, the display device 14A displays a real-time image as user image PA, where a portion of the area (in this case, the part other than the legs) is the second image P2 and the remaining area (in this case, the legs) is the first image P1. The display devices 14B and 14C display past images as user image PB and PC, where a portion of the area (in this case, the part other than the legs) is the second image P2 and the remaining area (in this case, the legs) is the first image P1. Since different prompts are set for the display devices 14A, 14B, and 14C, the second image P2 in the real-time image user image PA, the second image P2 in the past image user image PB, and the second image P2 in the past image user image PC all display user U in different ways (appearances).

[0089] As shown in step S4 of Figure 6, if user U approaches the display area within space SP more closely than in step S3, the display device 14 switches the first image P1 to the second image P2 and displays the user image P. In step S4, since the distance between the display area and user U has decreased significantly, each display device 14 displays the user image P with the second image P2 (generated image) covering the entire area. More specifically, in step S4, display device 14A displays a real-time image with the second image P2 covering the entire area as the user image PA, while display devices 14B and 14C display past images with the second image P2 covering the entire area as the user image PB and PC, respectively. In this way, the display device 14 displays the user image P such that the area occupied by the second image P2 increases as user U approaches the display area.

[0090] (Processing flow) Next, we will explain the processing flow for displaying the user image P described above. Figure 7 is a flowchart illustrating the processing flow for displaying the user image.

[0091] As shown in Figure 7, the management device 10 acquires the captured image PR of user U captured by the imaging device 16 (step S10), generates an extracted image PE based on the captured image PR, and acquires the location information (distance information) of user U as seen in the captured image PR (step S12). The management device 10 transmits the extracted image PE and the location information (distance information) of user U to the generation device 12. The management device 10 also sets a prompt and transmits the information of the set prompt to the generation device 12.

[0092] The generation device 12 acquires the extracted image PE and the location information (distance information) of user U from the management device 10 (step S14). The generation device 12 also acquires prompt information from the management device 10. Then, the generation device 12 generates a first image P1, which is a silhouette image of user U, based on the extracted image PE (step S16), and generates a second image P2, which is a generated image of user U, based on the extracted image PE and the prompt (step S18). More specifically, the generation device 12A generates a first image P1 that reflects the real-time behavior of user U based on the real-time extracted image PE, and generates a second image P2 that reflects the real-time behavior of user U based on the real-time extracted image PE and the prompt. On the other hand, the generation devices 12B and 12C generate a first image P1 that reflects the past behavior of user U based on past extracted image PE, and generate a second image P2 that reflects the past behavior of user U based on past extracted image PE and the prompt. Note that the generation order of the first image P1 and the second image P2 is arbitrary.

[0093] The generation device 12 generates a user image P based on the first image P1 and the second image P2, and displays the user image P on the display device 14 (step S20). More specifically, the generation device 12A switches between the real-time first image P1 and the real-time second image P2 to generate and display the real-time user image PA. On the other hand, the generation devices 12B and 12C switch between the past first image P1 and the past second image P2 to generate and display the past user images PB and PC.

[0094] If the process is to be terminated (step S22; Yes), this process is terminated. On the other hand, if the process is not to be terminated (step S22; No), the process returns to step S10, the next captured image PR is acquired, and the subsequent processing is repeated.

[0095] (Prompt update) In the above description, the display device 14 generated the second image P2 (generated image) based on the prompt initially set in the management device 10. However, the management device 10 may update the prompt. The case where the prompt is updated will be described below.

[0096] Figure 8 is a schematic diagram illustrating prompt updates. The prompt setting unit 36 ​​of the management device 10 decides whether to update the prompt based on the captured image PR. The prompt setting unit 36 ​​updates the prompt if the image of user U captured in the captured image PR satisfies predetermined conditions, and does not update the prompt if the predetermined conditions are not met. The predetermined conditions here may be arbitrary, but may be that the image of user U captured in the captured image PR is performing a predetermined action (for example, raising a hand as shown in Figure 8). The determination of whether a predetermined action is being performed may be made by known methods such as gesture detection using the captured image PR.

[0097] The management device 10 updates the prompt when the image of user U captured in the captured image PR meets predetermined conditions and transmits the updated prompt information to the generation device 12. The management device 10 updates the prompts of all display devices 14 when the image of user U captured in the captured image PR meets predetermined conditions. The management device 10 may select the type of new prompt in any way, for example, by random selection. However, in this case, it is preferable that the management device 10 sets the new prompt to be different from the current prompt and different from the prompts of other display devices 14. The management device 10 is not limited to updating the prompts of all display devices 14, but may update the prompts of only some of the display devices 14.

[0098] When the generation device 12 obtains an updated prompt, it uses the updated prompt to generate a generated image (second image P2 in this embodiment) for the subsequent user image P. As a result, the display manner of user U in the generated image changes.

[0099] In this way, by changing the prompt when the image of user U captured in the captured image PR meets predetermined conditions, the display mode of the user image P can be switched according to the user U's actions. In this embodiment, since both real-time images and past images are displayed as user image P, it can also be said that the display mode of the real-time image and past image is switched according to the user U's actions. By switching the prompt in this way, the display mode can be switched interactively, which can leave a stronger impression on the user U. In this example, the display mode of the generated image was switched by switching the prompt, but the method of switching the display mode is not limited to switching the prompt, and any method may be used.

[0100] (Other examples) Next, another example of this embodiment will be described. In the embodiment described above, both real-time images and past images were displayed side by side, and the first image P1 and the second image P2 were switched between and displayed. However, only one of these may be performed.

[0101] For example, if a real-time image and a past image are displayed side by side, it is not necessary to switch between displaying the first image P1 and the second image P2. In other words, as shown in Figure 5, if a user image P, which is a real-time image, and a user image P, which is a past image with a different display format than the real-time image, are displayed side by side, it is not necessary to switch between displaying the first image P1 and the second image P2 as shown in Figure 6.

[0102] For example, the process involves switching between displaying the first image P1 and the second image P2, but it is not necessary to display the real-time image and the past image side by side. In other words, as long as the user image P is displayed by switching between the first image P1 and the second image P2, it is not necessary to display the real-time image and the past image side by side. In this case, for example, the generation device 12 may switch between displaying the first image P1, which is a real image, and the second image P2, which is an avatar image with a different display mode from the real image, within the virtual space.

[0103] (effect) The present disclosure's generation device 12 includes a generation unit 52 that generates a real-time image showing the user U in real time based on the user U's real-time captured image PR, and generates a past image showing the user U in the past, but with a different display mode from the real-time image, based on the user U's captured image PR taken in the past, and a display control unit 54 that causes the display device 14 to display the real-time image and the past image side by side. According to this disclosure, the user can simultaneously view a real-time image that reflects their current self and a past image that reflects their past self but with a different display mode. Therefore, the user U can be made to feel a sense of being themselves but not themselves, or the existence of multiple selves, and a strong impression can be made on the user U.

[0104] The generation unit 52 generates multiple past images showing user U at different past timings based on captured images of user U taken at different past timings, and the display control unit 54 displays the real-time image and the multiple past images side by side. According to this disclosure, it is possible to make a stronger impression on user U.

[0105] The generation unit 52 generates at least one of the real-time image and the past image as a silhouette image showing the contours of the user, or as a generated image in which the appearance of user U in the captured image PR is altered from the actual image. According to this disclosure, by displaying both the silhouette image and the generated image, a stronger impression of user U can be made.

[0106] The generation unit 52 switches the display mode for the user in real-time images and past images according to the user U's actions. According to this disclosure, the display mode can be switched interactively, which can leave a stronger impression on the user U.

[0107] Although embodiments of the present disclosure have been described above, the embodiments are not limited to those described herein. Furthermore, the aforementioned components include those that can be easily conceived by those skilled in the art, those that are substantially the same, and those that fall within the so-called equivalent range. Moreover, the aforementioned components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the gist of the embodiments described above. [Explanation of Symbols]

[0108] 1 Display System 10 Management device 12 Generator 14 Display device 16 Imaging device 52 Generation part 54 Display Control Unit P User Image P1 First image P2 Second image PR captured images

Claims

1. A generation unit generates a real-time image showing the user in real time based on the user's real-time captured image, and generates a past image showing the user in the past, and in which the display of the user differs from that of the real-time image, based on the user's captured image taken in the past. A display control unit that displays the real-time image and the past image side by side on a display device, including, generator.

2. The generation unit generates a plurality of past images showing the user at different past timings based on the captured images of the user taken at different past timings. The display control unit displays the real-time image and a plurality of past images side by side. The generating apparatus according to claim 1.

3. The generation unit generates at least one of the real-time image and the past image as a silhouette image showing the contour of the user, or as a generated image in which the appearance of the user in the captured image is altered from the actual image. The generating apparatus according to claim 1 or claim 2.

4. The generation unit switches the display mode of the user in the real-time image and the past image according to the user's actions. The generating apparatus according to claim 1 or claim 2.

5. The steps include generating a real-time image showing the user in real time based on a real-time image of the user, and generating a past image showing the user in the past, where the user's appearance differs from that of the real-time image, based on a previously captured image of the user. The steps include displaying the real-time image and the past image side by side on a display device, including, Display method.

6. The steps include generating a real-time image showing the user in real time based on a real-time image of the user, and generating a past image showing the user in the past, where the user's appearance differs from that of the real-time image, based on a previously captured image of the user. The steps include displaying the real-time image and the past image side by side on a display device, Make the computer execute it. program.

Images