Video Display System

Abstract

To provide a video display system which displays a video suitable to a viewer, a server for a video display system, and a program for video display.SOLUTION: A video display system 1 comprises: a projector 11 as a display device which displays a video; a wearable sensor 21 as a state sensor which acquires data associated with a state of a viewer who views the video; a state analysis part 42 which specifies the viewer's state associated with the state of the viewer based upon the data that the state sensor acquires; a video determination part 43 which uses information on the viewer's state to determine the video that the display device is to display as a selected video; and an output part 56 which transmits video data related to the selected video to the display device.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a video display system. [Background technology]

[0002] Generally, people watch videos as entertainment. In particular, in hospitals and elderly care facilities, watching videos can play an important role as entertainment, and it is common for televisions, for example, to be installed.

[0003] For example, Patent Document 1 discloses technology related to a monitoring system that can be used in hospitals, elderly care facilities, etc. This monitoring system detects the movements of the person being monitored and projects images that can be viewed by the person. The images for the person may include, for example, information to be conveyed to the person regarding medication or rehabilitation, or live outdoor footage to help the person relax. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-135144 Summary of the Invention [Problem to be solved by the invention]

[0005] An object of the present invention is to provide a video display system that displays video suitable for the viewer watching the video. [Means for solving the problem]

[0006] According to one aspect of the present invention, a video display system includes a display device configured to display video, a status sensor configured to acquire data regarding the status of a viewer watching the video, a status analysis unit configured to identify a viewer status regarding the status of the viewer based on the data acquired by the status sensor, a video determination unit configured to determine the video to be displayed by the display device as a selected video using the viewer status information, and a video output unit configured to transmit video data related to the selected video to the display device. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide a video display system that displays video suitable for the viewer watching the video. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram schematically illustrating an example of the configuration of a video display system according to an embodiment. [Figure 2] FIG. 2 is a diagram schematically illustrating an example of an image including a skylight. [Figure 3] FIG. 3 is a flowchart showing an outline of an example of the physical and mental state analysis process for identifying the state of the viewer, which is carried out by the state analysis unit. [Figure 4] FIG. 4 is a flowchart showing an outline of an example of a video selection process performed by the video determination unit to determine a selected video. [Figure 5] FIG. 5 is a flowchart showing an outline of an example of a video information update process that updates the evaluation of a video performed by the video evaluation unit. [Figure 6] FIG. 6 is a flowchart showing an outline of an example of window adjustment processing performed by the video creation unit to create a video including a window frame. DETAILED DESCRIPTION OF THE INVENTION

[0009] An embodiment will be described with reference to the drawings. This embodiment relates to a video display system that displays various images over a relatively long period of time. This video display system can be used, for example, but not limited to, in homes, hospitals, elderly care facilities, and the like. One possible use scenario is, but is not limited to, for elderly people who spend a lot of time in bed. This video display system is configured to allow users to enjoy watching the images in a comfortable environment.

[0010] [System Configuration] FIG. 1 is a diagram schematically illustrating an example of the configuration of a video display system 1 according to this embodiment. The video display system 1 of this embodiment includes a projector 11 as a display device for displaying images. Other display devices, such as various display panels, may be used instead of the projector 11. In this embodiment, as an example, it is assumed that a viewer 91 viewing an image is lying on a bed 92. FIG. 1 illustrates a state in which the projector 11 projects an image onto a projection area 94 on a ceiling 93. However, this is not a limitation, and the image may be displayed anywhere. Furthermore, the video display system 1 may include a speaker 12 for outputting sound in addition to the projector 11 for displaying images.

[0011] The video display system 1 is configured to display an image according to the state of the viewer 91. The video display system 1 includes a wearable sensor 21 worn by the viewer 91 to acquire information related to the state of the viewer 91. The wearable sensor 21 may be, for example, a wristwatch-type sensor, and may be worn around the wrist of the viewer 91, but is not limited to this. The wearable sensor 21 functions as a state sensor configured to acquire data related to the state of the viewer 91. The wearable sensor 21 may include at least one of various sensors, such as a six-axis acceleration sensor, a heart rate sensor, a blood pressure sensor, a blood oxygen saturation sensor, a respiration sensor, a skin moisture sensor, and a temperature sensor. Physiological indicators of the viewer 91 can be acquired by such sensors.

[0012] The video display system 1 may also include a camera 22 for capturing images of the viewer 91, a microphone 23 for capturing the voice of the viewer 91, an illuminance sensor 24 for capturing the illuminance of the room, and the like.

[0013] The image display system 1 includes a control device 40 that controls projection by the projector 11. The control device 40 is a computer that performs various types of information processing. The control device 40 includes a CPU, various types of memory, storage, input devices, output devices, interfaces, etc., and is composed of integrated circuits, wiring, terminals, buttons, sensors, lamps, display elements, etc. The control device 40 can operate according to a program using hardware and software.

[0014] The video display system 1 may also include a server 71 and a family terminal 76 connected to the control device 40 via a communication network 70 such as the Internet. The family terminal 76 may be an information terminal such as a smartphone or personal computer owned by a family member or, in some cases, a friend of the viewer 91. The server 71 may be provided with a database (DB) 72 that includes the video to be projected by the projector 11 and various information.

[0015] [Control device configuration] The control device 40 has functions as a sensor communication unit 41, an input unit 46, an output unit 56, and a communication unit 58 that transmit and receive data to and from the outside of the control device 40.

[0016] The sensor communication unit 41 communicates with the wearable sensor 21 and acquires data relating to the state of the spectator 91 acquired by the wearable sensor 21. The wearable sensor 21 and the control device 40 communicate, for example, wirelessly. The sensor communication unit 41 is configured to be able to acquire various data about the spectator 91 acquired by the various sensors of the wearable sensor 21, including physiological indicators or medical data such as the movement, heart rate, blood pressure, oxygen saturation, respiration, sweating, and body temperature of the spectator 91.

[0017] The input unit 46 receives data from devices connected externally to the control device 40, such as the camera 22, the microphone 23, and the illuminance sensor 24. For example, the input unit 46 acquires video of the viewer 91 from the camera 22, acquires audio of the viewer 91 speaking from the microphone 23, and acquires data related to the illuminance of the room from the illuminance sensor 24. The video and audio of the viewer 91 can be used to determine the state of the viewer 91, make a video call with an external information terminal such as the family terminal 76, operate the video display system 1, and so on.

[0018] The output unit 56 transmits data and the like to devices connected to the outside of the control device 40, such as the projector 11 and the speaker 12. For example, the output unit 56 transmits data of an image to be projected to the projector 11, and transmits a signal related to audio to be output to the speaker 12.

[0019] The communication unit 58 communicates with a server 71 and a family terminal 76 external to the control device 40 via a communication network 70 .

[0020] The control device 40 further has the functions of a state analysis unit 42, an image determination unit 43, and an image recording unit 45.

[0021] The state analysis unit 42 is configured to identify a viewer state related to the state of the viewer 91 based on data related to the state of the viewer 91 acquired via the sensor communication unit 41. The state analysis unit 42 identifies, for example, a physical state, a psychological state, etc. as the viewer state. The state analysis unit 42 identifies, for example, the physical state, whether the viewer 91 is in an awake state, a sleeping state, an awake period which is a period in transition from a sleeping state to an awake state, or a sleep onset period which is a period in transition from an awake state to a sleep state, based on a combination of the various acquired data, etc. Furthermore, the state analysis unit 42 identifies, for example, the psychological state, whether the viewer 91 is in a comfortable state or an uncomfortable state, based on a combination of the various acquired data, etc. The comfortable state includes, for example, a state in which the viewer 91 feels joy or is relaxed, etc. The state analysis unit 42 may identify the state of the viewer not only using information obtained by the wearable sensor 21 acquired from the sensor communication unit 41, but also using information such as video of the viewer 91 captured by the camera 22 and audio uttered by the viewer 91 acquired by the microphone 23 acquired via the input unit 46.

[0022] The video recording unit 45 records original data of various images projected by the projector 11. Considering that the images recorded in the video recording unit 45 will be projected onto the ceiling 93, the images may include, for example, blue skies and flowing clouds, forest trees, sunrises, sunsets, starry skies, etc. The images recorded in the video recording unit 45 may also be other landscapes, such as natural or cityscapes. The images recorded in the video recording unit 45 may also be memorable photos, videos, family photos, video messages from family and friends, etc. of the viewer 91. In addition to these images, the video recording unit 45 may also record audio data related to the images, such as the sounds of birds singing, waterfalls, and family voices output from the speaker 12.

[0023] The control device 40 may acquire video and the like from the database 72 of the server 71 and record it in the video recording unit 45. The control device 40 may also acquire video and the like from the family terminal 76 and record it in the video recording unit 45.

[0024] Alternatively, the original data itself, such as the image projected by the projector 11, may be recorded in, for example, a database 72 of the server 71, and only information relating to the image recorded in the database 72 may be recorded in the image recording unit 45. In this case, the control device 40 reads the image recorded in the database 72 or the like based on the information recorded in the image recording unit 45, and uses this for various processes. Alternatively, all information relating to the image may be recorded in the server 71 or the like.

[0025] The image determination unit 43 is configured to determine the image to be projected by the projector 11 as the selected image, using the information related to the viewer state identified by the state analysis unit 42. The image determination unit 43 selects the optimal image that reflects the state of the viewer 91 from the various images recorded in the image recording unit 45.

[0026] For example, when the viewer 91 is in an awake state, an image that encourages wakefulness may be selected. For example, when the viewer 91 is in an awake state, an image of a sunrise, an image of a forest illuminated by the morning sun, and the chirping of birds may be selected. Furthermore, the projection of an image may begin when the viewer 91 wakes up. For example, when the viewer 91 is in a sleep-onset state, an image that encourages sleep may be selected. For example, when the viewer 91 is in a sleep-onset state, an image of a starry sky may be selected. Furthermore, the projection of an image may end when the viewer 91 falls asleep. For example, when the viewer 91 is in an awake state, an image that the viewer 91 likes or a video message from a family member may be selected, or an image in response to an instruction from the viewer 91 may be selected. Furthermore, when the viewer 91 is in an awake state and communication with the family terminal 76 is established via the communication network 70, execution of a video call using the projector 11, speaker 12, camera 22, microphone 23, etc. may be selected. When the viewer 91 is asleep, no video may be selected so that the projection of the video is stopped. In determining the selected video, the video determination unit 43 may use information on the date and time obtained from the clock unit 47 and information on the illuminance of the room obtained from the illuminance sensor 24.

[0027] Furthermore, the video determination unit 43 may provide feedback on the viewer 91's reaction to the selected video. That is, the video determination unit 43 may also function as the video evaluation unit 44. In this case, for example, the video recording unit 45 may record the viewer 91's evaluation of the video, etc., in association with the video, etc. The video evaluation unit 44 acquires data on the viewer 91's psychological state, such as pleasant or unpleasant, from the state analysis unit 42. The video evaluation unit 44 updates the evaluations recorded in the video recording unit 45 so as to increase the evaluation of the video displayed when it is determined that the viewer 91 is in a pleasant state and of videos related to the video, including the video. On the other hand, the video evaluation unit 44 updates the evaluations recorded in the video recording unit 45 so as to decrease the evaluation of the video displayed when it is determined that the viewer 91 is in an unpleasant state and of videos related to the video, including the video.

[0028] The video determination unit 43 may determine the selected video using, for example, the evaluation of each video, etc., updated as described above. In this way, a video, etc., that matches the preferences, etc., of the viewer 91 can be appropriately selected. It is preferable that the evaluation of each video, etc., is set for each condition, such as the time, the state of the viewer 91, etc. In this way, a more appropriate video, etc., can be selected.

[0029] The more pleasant the psychological state of the viewer 91, the higher the evaluation, and therefore the more likely such a video is to be selected by the video determination unit 43. For example, a scene that relaxes the viewer 91 may be repeatedly selected and displayed. Also, a video of a person that makes the viewer 91 happy may be repeatedly selected and displayed.

[0030] Furthermore, the video evaluation unit 44 may transmit information regarding the evaluation of the video, etc. to the server 71 via the communication unit 58. The server 71 may acquire information regarding the evaluation of the video, etc. from a large number of control devices 40 used by different viewers 91, tally this information, and determine the evaluation of the video, etc. recorded in the database 72. In this way, videos, etc. that are popular with many people become clear, and better videos, etc. can be provided to each user.

[0031] Furthermore, if the video evaluation unit 44 can modify the video itself recorded in the video recording unit 45 or the like, it may modify the video so that it receives a higher evaluation from the viewer 91. For example, the video evaluation unit 44 may modify a video that does not contain a feature that the viewer 91 likes so that the feature is included. For example, the video may be modified so that a feature that the viewer 91 dislikes is removed. For example, the color tone of the video may be adjusted, or motifs such as scenery, buildings, people, and animals included in the video may be added, changed, or deleted. The modified video may also be transmitted to the server 71 or the like.

[0032] The control device 40 further functions as an image creation unit 51. The image creation unit 51 acquires the selected image determined by the image determination unit 43 from the image recording unit 45, etc., and creates an output image to be output to the projector 11 via the output unit 56 based on the selected image. The output image may be the image recorded in the image recording unit 45 as is, or may be an image that has been edited in various ways. The created output image is transmitted to the projector 11 via the output unit 56, and is projected onto the ceiling 93 by the projector 11.

[0033] [Video Production Team Composition] The image projected onto the ceiling 93 using the projector 11 may be the image recorded in the image recording unit 45 as described above, or may be processed in various ways. For example, a composite image combining multiple images may be projected. For example, memorable photos and videos, or video messages from family members may be placed against a background of scenery or a starry sky. The image creation unit 51 creates such an image based on information from the image determination unit 43 and the image recorded in the image recording unit 45.

[0034] The image projected onto the ceiling 93 may be, for example, an image simulating a skylight, that is, an image in which there is a skylight on the ceiling 93 and the outside scenery or the like can be seen through the skylight. For this reason, the image creation unit 51 may have a function to create an image including a skylight based on the image recorded in the image recording unit 45.

[0035] An example of an image 110 including a skylight is shown in FIG. 2. This image 110 has a configuration in which a display image 120 recorded in the image recording unit 45 is surrounded by an image 130 of a window frame. The display image 120 is an image selected by the image determination unit 43. In the example shown in FIG. 2, the display image 120 is an image of the sky including clouds. The image 130 of the window frame is an image of an interior wall that corresponds to the thickness of the ceiling 93. In FIG. 2, an image of a stone interior wall is shown.

[0036] 1, consider a case where the position of face 91a of viewer 91, i.e., the position of the eyes, is biased to the left side of the figure with respect to the center of projection area 94 onto which the image is projected by projector 11. In this case, image 110 including the skylight is adjusted so that it corresponds to the skylight being viewed obliquely from the position of face 91a of viewer 91. That is, as shown in FIG. 2, the inner walls in image 130 of the window frame are adjusted so that right inner wall 131 is wider and left inner wall 132 is narrower.

[0037] Also, for example, assume that the displayed image 120 is an image of sunlight shining from the left side, as shown in Fig. 2. In this case, as shown in Fig. 2, the image 130 of the window frame is adjusted so that the right inner wall 131 is bright and the left inner wall 132 is dark.

[0038] In order to create such an image, the image creation unit 51 has functions as an image acquisition unit 52, a position information acquisition unit 53, a window frame image creation unit 54, an output image creation unit 55, and the like.

[0039] The video acquisition unit 52 is configured to acquire the display video determined by the video determination unit 43 from the video recording unit 45 or the like. The position information acquisition unit 53 is configured to acquire information relating to the position of the viewer 91. Any method may be used to acquire the position of the viewer 91. As one example, the position information acquisition unit 53 may acquire an image of the viewer 91 from the camera 22, and identify the position of the viewer 91, in particular the position of the face 91a and the position of the eyes of the viewer 91, based on the image.

[0040] The window frame image creation unit 54 is configured to create the window frame image 130 using information related to the position of the viewer 91. At this time, as described above, the window frame image 130 becomes an image of an interior wall that appears appropriately as a window frame to the viewer 91 according to the position of the viewer 91. Furthermore, the brightness or color of the window frame image 130 is adjusted according to the amount of light shining on the display image 120. For example, when light shines from left to right on the display image 120, the window frame image 130 becomes bright on the right side and dark on the left side, and when light shines from right to left, the window frame image 130 becomes bright on the left side and dark on the right side. For example, if the display image 120 is a daytime image, the window frame image 130 will appear whitish, and if the display image 120 is an image of a sunrise or sunset, a portion of the window frame image 130 will appear reddish accordingly. If the display image 120 is a starry sky, the window frame image 130 will be in accordance with the brightness of the room obtained from the illuminance sensor 24, and if the room is dark, the window frame image 130 will also be dark.

[0041] The window frame image creation unit 54 may create the window frame image 130 in any manner. Various patterns may be prepared in advance for part or all of the window frame image 130, and the window frame image 130 may be prepared by selecting from among these patterns. That is, the window frame image creation unit 54 may be configured to select one piece of data from multiple patterns prepared in advance for at least part of the window frame image 130, based on information related to the position of the viewer 91. Alternatively, multiple patterns with different colors may be prepared, but only one pattern of shape as viewed from the center may be prepared, and the window frame image 130 may be created by performing coordinate transformation processing on these patterns. That is, the window frame image creation unit 54 may be configured to perform coordinate transformation processing on the prepared data for at least part of the window frame image 130, based on information related to the position of the viewer 91.

[0042] The frequency of acquiring the position of the viewer 91 and changing the shape of the window frame image 130 may be any. In the least frequent mode, since the posture of the viewer 91 lying on the bed 92 generally does not change much, the position information acquisition unit 53 determines the shape of the window frame image 130 only once when the bed 92 and projector 11 are installed. In other words, the shape of the window frame image 130 is determined assuming that the position of the viewer 91 is fixed with respect to the position of the bed 92, and the shape of the window frame image 130 does not change until the position of the bed 92 or the projector 11 is changed. In this case, the computational load on the image creation unit 51 can be kept low. In the most frequent mode, the image acquisition unit 52 may acquire and identify the position of the viewer 91 in real time based on, for example, an image acquired by the camera 22. In this case, the computational load on the image creation unit 51 is high, but a natural-looking window frame image 130 is always obtained even if the viewer 91 moves. Similarly, this frequency can be appropriately determined taking into consideration the required accuracy and the calculation load, for example, at intervals of several minutes or when a large change in the posture of the viewer 91 is detected.

[0043] Output image creation unit 55 is configured to create a final output image, such as image 110 including a skylight with window frame image 130 surrounding display image 120. The output image created in this manner is transmitted via output unit 56 to projector 11, which then projects it onto ceiling 93.

[0044] [System Operation] The operation of the video display system 1 will be described with reference to a flowchart showing an overview thereof.

[0045] <Physical and psychological state analysis processing> FIG. 3 is a diagram showing an outline of an example of the physical and psychological state analysis process performed by the state analysis unit 42 to identify the state of the viewer.

[0046] In step S101, the state analysis unit 42 determines whether the viewer 91 is sleeping based on a combination of various data related to the state of the viewer 91 acquired by the wearable sensor 21. If it is determined that the viewer 91 is sleeping, the process proceeds to step S102. In step S102, the state analysis unit 42 determines that the physical state of the viewer 91 is sleeping. Thereafter, the process returns to step S101. That is, while the viewer 91 is sleeping, it is repeatedly determined whether the viewer 91 is sleeping.

[0047] If it is determined in step S101 that the viewer 91 is not asleep, the process proceeds to step S103. In step S103, the state analysis unit 42 determines whether the viewer 91 is about to wake up from a sleeping state, based on a combination of various data related to the state of the viewer 91 acquired by the wearable sensor 21. If it is determined that the viewer 91 is about to wake up, the process proceeds to step S104. In step S104, the state analysis unit 42 determines that the physical state of the viewer 91 is in the wakefulness stage. Thereafter, the process proceeds to step S108.

[0048] If it is determined in step S103 that the viewer 91 is not about to wake up, the process proceeds to step S105. In step S105, the state analysis unit 42 determines whether the viewer 91 is about to fall asleep from an awake state, based on a combination of various data related to the state of the viewer 91 acquired by the wearable sensor 21. If it is determined that the viewer 91 is about to fall asleep, the process proceeds to step S106. In step S106, the state analysis unit 42 determines that the physical state of the viewer 91 is in the sleep onset stage. Thereafter, the process proceeds to step S108.

[0049] If it is determined in step S105 that the viewer 91 is not about to fall asleep, the process proceeds to step S107. In step S107, the state analysis unit 42 determines that the physical state of the viewer 91 is an awake state. Thereafter, the process proceeds to step S108.

[0050] Thus, when the physical state is the awake stage, the sleep stage, or the awake state, the process proceeds to step S108. In step S108, the state analysis unit 42 determines whether the viewer 91 is showing a pleasant reaction based on a combination of various data related to the state of the viewer 91 acquired by the wearable sensor 21. If it is determined that the viewer 91 is showing a pleasant reaction, the process proceeds to step S109. In step S109, the state analysis unit 42 determines that the psychological state of the viewer 91 is a pleasant state. Thereafter, the process returns to step S101.

[0051] If it is determined in step S108 that spectator 91 is not showing a pleasant reaction, the process proceeds to step S110. In step S110, state analysis unit 42 determines whether spectator 91 is showing an unpleasant reaction, based on a combination of various data related to the state of spectator 91 acquired by wearable sensor 21. If it is determined that spectator 91 is showing an unpleasant reaction, the process proceeds to step S111. In step S111, state analysis unit 42 determines that spectator 91's psychological state is an unpleasant state. Thereafter, the process returns to step S101. If it is determined in step S110 that spectator 91 is not showing an unpleasant reaction, the psychological state is not determined, and the process returns to step S101.

[0052] That is, when the physical state is in the awake stage, the sleep stage, or the awake state, it is repeatedly determined whether the psychological state is a comfortable state, an uncomfortable state, or neither. In this manner, the state analysis unit 42 continuously determines the physical state and the psychological state of the viewer 91. For example, in this manner, the state analysis unit 42 identifies various states of the viewer 91. Information related to these states is transmitted to the video determination unit 43.

[0053] <Video Selection Processing> FIG. 4 is a diagram showing an outline of an example of the video selection process performed by the video determination unit 43 to determine the selected video.

[0054] In step S201, the image determination unit 43 determines whether the physical state of the viewer 91 identified by the state analysis unit 42 is in the wakefulness period. If it is determined to be in the wakefulness period, the process proceeds to step S202. In step S202, the image determination unit 43 selects an image suitable for the wakefulness period according to a predetermined algorithm. Thereafter, the process returns to step S201.

[0055] If it is determined in step S201 that the viewer 91 is not in the wakefulness period, the process proceeds to step S203. In step S203, the image determination unit 43 determines whether the physical state of the viewer 91 identified by the state analysis unit 42 is in the sleep onset period. If it is determined that the physical state is in the sleep onset period, the process proceeds to step S204. In step S204, the image determination unit 43 selects an image suitable for the sleep onset period according to a predetermined algorithm. Thereafter, the process returns to step S201.

[0056] If it is determined in step S203 that the user is not in the sleep onset period, the process proceeds to step S205. In step S205, the video selection unit 43 determines whether or not there is an incoming call from another terminal, such as the family terminal 76. If it is determined that there is an incoming call, the process proceeds to step S206. In step S206, the video selection unit 43 selects communication, such as a video call, with the other terminal that has received the incoming call. Thereafter, the process returns to step S201.

[0057] If it is determined in step S205 that there is no incoming call, the process proceeds to step S207. In step S207, the image determination unit 43 determines whether or not a video has been designated by, for example, the viewer 91. This is not limited to designations by the viewer 91, but also applies to designations from, for example, the family terminal 76. If a designation has been made, the process proceeds to step S208. In step S208, the image determination unit 43 selects the designated video. The designation of a video is not limited to designating an individual video. For example, the viewer 91 or the like may designate keywords such as forest, sea, city, sky, etc., and the image determination unit 43 may then select videos related to the keywords one after another. Thereafter, the process returns to step S201.

[0058] If it is determined in step S207 that no designation has been made, the process proceeds to step S209. In step S209, the image determination unit 43 determines whether the physical state of the viewer 91 identified by the state analysis unit 42 is an awake state. If it is determined to be an awake state, the process proceeds to step S210. In step S210, the image determination unit 43 selects an image based on information such as the current time, the viewer 91's rating of each image, and the history of images selected by the viewer 91. For example, an image corresponding to the current time may be selected, such as a morning scene in the morning, a daytime scene in the afternoon, a sunset image in the evening, and a starry sky at night. These may also be selected according to the season. During such selection, or regardless of the date and time, an image highly rated by the viewer 91 or an image similar to an image with a high rating may be selected, or an image frequently selected by the viewer 91 or an image similar to an image with a high frequency may be selected. In addition, a plurality of images may be selected to create a video that combines a plurality of images, such as inserting memorable photos or video messages from family into a scenic image. After that, the process returns to step S201.

[0059] If it is determined in step S209 that the viewer 91 is not in an awake state, the process returns to step S201. That is, if the physical state of the viewer 91 identified by the state analysis unit 42 is in a sleeping state, no video is selected, and the process returns to step S201.

[0060] For example, in the above manner, the video determination unit 43 selects a video and determines the selected video. The video selected will be one that is in accordance with the situation at the time, the state of the viewer 91, and the like. The selected video will also be one that is highly rated by the viewer 91, i.e., one that gives the viewer 91 a pleasant feeling. Information about the selected video is transmitted to the video creation unit 51.

[0061] <Video information update processing> FIG. 5 is a diagram showing an outline of an example of a video information update process performed by the video evaluation unit 44, which updates the evaluation of the video.

[0062] In step S301, the image evaluation unit 44 determines whether the psychological state of the viewer 91 identified by the state analysis unit 42 is a comfortable state. If it is determined to be a comfortable state, the process proceeds to step S302. In step S302, the image evaluation unit 44 adds an evaluation score recorded in the image recording unit 45 for the image currently being projected by the projector 11. That is, an image to which the viewer 91 responds with a comfortable reaction receives a higher evaluation score. The evaluation score may be added not only to the image currently being displayed, but also to images similar to the image currently being displayed. The process then proceeds to step S305.

[0063] If it is determined in step S301 that the viewer 91 is not in a comfortable state, the process proceeds to step S303. In step S303, the image evaluation unit 44 determines whether the psychological state of the viewer 91 identified by the state analysis unit 42 is in an uncomfortable state. If it is determined that the psychological state is in an uncomfortable state, the process proceeds to step S304. In step S304, the image evaluation unit 44 deducts the evaluation points recorded in the image recording unit 45 for the image currently being projected by the projector 11. That is, the evaluation points of images to which the viewer 91 has an uncomfortable reaction are lowered. The evaluation points may be deducted not only for the image currently being displayed, but also for images similar to the image currently being displayed. Thereafter, the process proceeds to step S305.

[0064] In step S305, the video evaluation unit 44 determines whether or not the video can be modified in relation to the video that has been scored or deducted. If it is determined that the modification is not possible, the process proceeds to step S307. If it is determined that the modification is possible, the process proceeds to step S306. In step S306, the video evaluation unit 44 performs processing related to the modification of the video. For example, when the evaluation of the video is increased, the video or other videos are modified to make better use of the parts related to the high evaluation. For example, when the evaluation of the video is decreased, the video or other videos are modified to remove or improve the parts related to the low evaluation. The modified video is recorded in the video recording unit 45. Then, the process proceeds to step S307.

[0065] In step S307, the video evaluation unit 44 transmits the evaluation score for the video with the updated evaluation score to the server 71. It also transmits the video that has been modified based on the evaluation to the server 71. Thereafter, the process returns to step S301.

[0066] If it is determined in step S303 that the state is not uncomfortable, the process proceeds to step S308. In step S308, the video evaluation unit 44 determines whether or not information related to the video has been received from the server 71. If information related to the video has been received, the process proceeds to step S309. In step S309, the video evaluation unit 44 updates the record in the video recording unit 45 based on the information received from the server 71. For example, when new video or the like is acquired, the video or the like is recorded in the video recording unit 45. For example, when information related to the evaluation score of the video recorded in the video recording unit 45 is acquired, the evaluation score is updated. Thereafter, the process returns to step S301.

[0067] If it is determined in step S308 that no information has been received from the server 71, the process proceeds to step S310. In step S310, the video evaluation unit 44 determines whether or not information related to the video has been received from the family terminal 76. If information related to the video has been received, the process proceeds to step S311. In step S311, the video evaluation unit 44 updates the record in the video recording unit 45 based on the information received from the family terminal 76. For example, when new video, a video message, or the like is acquired, the video, or the like is recorded in the video recording unit 45. Thereafter, the process returns to step S301. If it is determined in step S310 that no information has been received from the family terminal 76, the process returns to step S301.

[0068] For example, as described above, information on the video etc. recorded in the video recording unit 45 is updated. By updating the video etc. recorded in the video recording unit 45, the viewer 91 can view new videos one after another. Furthermore, by updating the evaluation of the video etc. by each viewer 91, a video more suitable for each viewer 91 is selected in the video selection process described above. It is preferable that this evaluation is set according to the state of the viewer 91, the time of day, etc. In this way, the video selected by the video determination unit 43 reflects the psychological state of the viewer 91, such as pleasant or unpleasant, identified by the state analysis unit 42. In other words, the video determination unit 43 determines the selected video using the viewer's state, such as pleasant or unpleasant. Furthermore, by updating the video etc. based on the evaluation of the video etc. by each viewer 91, a video that suits each viewer 91 is created.

[0069] Furthermore, by aggregating the ratings from each viewer 91 in the server 71, the server 71 is able to identify videos and the like that have a good reputation among a large number of users, and this information is sent to each viewer 91. The server 71 aggregates data in various ways to obtain information tailored to each viewer 91, such as by aggregating data for each viewer 91, or by the viewer's gender, age, or other type. The server 71 also aggregates data for each viewer's 91 state, time of day, weather, and other conditions, and creates information to enable the most appropriate video to be selected depending on the situation.

[0070] Window adjustment processing FIG. 6 is a diagram showing an outline of an example of window adjustment processing performed by the image creating unit 51 to create an image including a window frame.

[0071] In step S401, the image creation unit 51 determines whether it is time to adjust the window frame position of the image displayed by the projector 11 so that the position of the window frame corresponds to the position of the viewer 91. The timing to adjust the window frame position may be frequent enough to be considered real time, may be when it is detected that the viewer 91 has made a significant change in posture, may be once a day, or may be when the image display system 1 is installed. If it is determined that it is time to adjust the window frame position, the process proceeds to step S402.

[0072] In step S402, the image creation unit 51 acquires information about the position of the viewer 91, for example, based on the image captured by the camera 22, the detection results of other sensors, input by the person installing the device, etc. In step S403, the image creation unit 51 calculates a window frame position that will appear to be an appropriate window frame from the viewer 91, based on the information about the position of the viewer 91. Thereafter, the processing proceeds to step S404.

[0073] If it is determined in step S401 that it is not the time to adjust the window frame position, the process proceeds to step S404. In step S404, the image creation unit 51 acquires the display image selected by the image determination unit 43 from the image recording unit 45, etc. In step S405, the image creation unit 51 acquires the date and time from the clock unit 47, the indoor illuminance acquired by the illuminance sensor 24, and other information.

[0074] In step S406, the image creation unit 51 creates a window frame image. At this time, the shape of the window frame is determined based on the window frame position calculated in step S403. The image creation unit 51 also analyzes the amount of light entering the displayed image and analyzes how the window frame is illuminated accordingly. At this time, the current date and time may be taken into consideration. The image creation unit 51 also analyzes how the window frame is illuminated from the inside of the room depending on the illuminance in the room, etc. The image creation unit 51 combines these results to create the window frame image.

[0075] In step S407, image creation unit 51 combines the display image acquired in step S404 with the window frame image created in step S406 to create a composite image in which the display image is fitted into the window frame. In step S408, image creation unit 51 outputs the created composite image to output unit 56 as an output image. The output image is transmitted to projector 11 via output unit 56, and the image is projected onto ceiling 93 by projector 11.

[0076] Through the above processing, a window frame image adjusted to a position corresponding to the position of viewer 91, in which the display image is included within the window frame image corresponding to the display image, etc., is projected onto ceiling 93. As a result, an image is projected that makes it appear as if there is a real skylight on ceiling 93 when viewed from viewer 91, and viewer 91 can enjoy the image as if he or she were looking outside through a real skylight.

[0077] [About the video display system] According to the video display system 1 of this embodiment, appropriate images and the like are displayed at the right time, allowing the viewer 91 to wake up or fall asleep comfortably, for example. Furthermore, images and the like that match the viewer 91's preferences are displayed preferentially, allowing the viewer 91 to spend time comfortably in bed 92, etc. In particular, people hospitalized in hospitals and elderly people residing in nursing homes often spend a lot of time in bed 92, and these people can also spend their time comfortably. People often feel a sense of claustrophobia in private rooms in hospitals and facilities, but when an image of a large skylight on the ceiling 93 is displayed, the viewer 91 can experience a sense of openness. Furthermore, by being able to easily make video calls with others, the world in which the viewer 91 lives is opened to the outside world.

[0078] [Variations] In the above embodiment, a wristwatch-type wearable sensor 21 is used as a status sensor for acquiring data related to the status of spectator 91, but this is not limiting. Devices equipped with various types of sensors may be used as wearable sensor 21. The number of devices equipped with sensors is not limited to one, and may be multiple. Various sensors may be used as a status sensor instead of or together with wearable sensor 21.

[0079] The status sensor may be, for example, a bed-mounted sensor provided on the bed 92. The bed-mounted sensor is installed on the bed 92 and may be, for example, a thin sensor placed under a mattress or a sheet, or a sensor attached to the bottom surface of the bed 92.

[0080] The bed-mounted sensor may include, for example, any of a vibration sensor, an acoustic sensor, an acceleration sensor, a piezoelectric sensor, or the like configured to detect minute vibrations and sounds related to the viewer 91. For example, the respiratory rate and heart rate may be determined by detecting minute vibrations associated with breathing and heartbeats using a sensor. The bed-mounted sensor may acquire various information related to the viewer 91, including physiological indicators or medical data such as heart rate, respiratory rate, presence or absence of body movement, and values ​​indicating whether the viewer is in bed or out of bed.

[0081] Such a bed-mounted sensor can be used without causing discomfort to the viewer 91 because the sensor does not come into contact with the viewer 91, unlike the wearable sensor 21. This is particularly effective when the viewer 91 is bedridden in bed 92.

[0082] The status sensor may also be a non-contact vital sensor that is installed on the ceiling 93 or the like and acquires the status of the viewer 91. The non-contact vital sensor irradiates the human body with electromagnetic waves, such as weak microwaves, acquires the waves reflected from the surface of the human body, and identifies the movement of the human body surface by utilizing the Doppler effect, for example. For example, the respiratory rate can be determined from the movement of the chest or abdomen. Furthermore, the heart rate can be determined from the venous flow near the body surface, for example. In this way, the non-contact vital sensor can acquire various information about the viewer 91, including physiological indicators or medical data such as the heart rate, respiratory rate, the presence or absence of body movement, and values ​​indicating whether the viewer is in bed or out of bed.

[0083] Furthermore, to acquire the state of the viewer 91, an image of the viewer 91 acquired by the camera 22 or the voice of the viewer 91 acquired by the microphone 23 may be used. For example, the facial expression of the viewer 91 may be analyzed based on the image acquired by the camera 22 and used, or the emotion of the viewer 91 may be analyzed based on the voice of the viewer 91 acquired by the microphone 23 and used. In other words, the camera 22, the microphone 23, or the like may be used as a state sensor.

[0084] The various sensors described above can be used as status sensors. These sensors may be used alone or in combination. The status of the viewer 91 can be determined based on any one or a combination of data obtained by these sensors.

[0085] In the above embodiment, an example was shown in which an image simulating a skylight was displayed on the ceiling 93, but this is not limiting. An image may be displayed on a side wall instead of the ceiling 93. Furthermore, the image displayed on the side wall may also include a window frame or the like to resemble a side window.

[0086] Furthermore, the area in which the image is displayed is not limited to one, and may be multiple. These multiple images may be linked. For example, an image simulating a side window and an image simulating a ceiling window may be connected according to their positions. When multiple images are linked, image acquisition unit 52 is configured to acquire display images including linked first and second display images, window frame image creation unit 54 is configured to create a first window frame image for the first display image and a second window frame image for the second display image, and output image creation unit 55 is configured to create a first output image in which the first window frame image surrounds the first display image and a second output image in which the second window frame image surrounds the second display image.

[0087] The control device 40 may be installed anywhere. For example, a control device 40 may be installed in a room where a viewer 91 is present, for each viewer 91, i.e., for each projector 11. The control device 40 may also be installed in a facility where the viewer 91 is present, with one control device 40 controlling the operation of various devices in multiple rooms. The control device 40 may also be installed in a remote location and control the operation of various devices in multiple facilities, etc., via various communication networks. The functions of the control device 40 may also be realized by devices that are distributed. In this case, the entire set of distributed devices can be said to be the control device 40.

[0088] The display device may include a small planetarium for projecting a starry sky in addition to the projector 11 or a display panel.

[0089] Furthermore, data obtained by wearable sensor 21 included in video display system 1 may be used for health management, etc. Images obtained by camera 22 may be used to detect abnormalities such as the viewer 91 falling off bed 92 or wandering around. Access rights and the like are managed for this information as well, and privacy and the like are taken into consideration.

[0090] The present invention has been described above by showing preferred embodiments, but it goes without saying that the present invention is not limited to the above-described embodiments, and various modifications can be made within the scope of the present invention. [Explanation of symbols]

[0091] 1. Video display system 11 Projector 12 speakers 21 Wearable Sensors 22 Camera 23. Mike 24 Illuminance sensor 40 Control device 41 Sensor communication unit 42 Condition analysis unit 43 Video Decision Department 44 Video Evaluation Department 45 Video Recording Section 46 Input section 47 Clock Section 51 Video Production Department 52 Video acquisition unit 53 Location information acquisition unit 54 Window Frame Video Creation Department 55 Output video creation section 56 Output section 58 Communications Department 70 Communication Network 71 servers 72 databases 76 Family Devices 91 viewers 92 beds 93 Ceiling 94 Projection area

Claims

1. a display device configured to display the video; a state sensor configured to acquire data relating to a state of a viewer viewing the video; a state analysis unit configured to identify a state of the viewer related to a state of the viewer based on data acquired by the state sensor; an image determination unit configured to determine an image to be displayed by the display device as a selected image using the information on the viewer state; a video output unit configured to transmit video data relating to the selected video to the display device; Equipped with The condition sensor includes a wearable sensor. Video display system.

2. a display device configured to display the video; a state sensor configured to acquire data relating to a state of a viewer viewing the video; a state analysis unit configured to identify a state of the viewer related to a state of the viewer based on data acquired by the state sensor; an image determination unit configured to determine an image to be displayed by the display device as a selected image using the information on the viewer state; a video output unit configured to transmit video data relating to the selected video to the display device; Equipped with The state analysis unit is configured to identify at least whether the viewer is in a comfortable state or an uncomfortable state. Video display system.

3. The video display system of claim 2, wherein the video determination unit has a video evaluation unit configured to increase the evaluation of a video related to the video displayed on the display device when the comfortable state is identified, or to decrease the evaluation of a video related to the video displayed on the display device when the uncomfortable state is identified.

4. The video display system according to claim 3 , wherein the video determination unit is configured to determine the selected video based on the evaluation by the video evaluation unit.

5. The image display system of claim 2, wherein the image determination unit is configured to create an altered image based on the image that was displayed on the display device when the comfortable state was identified, or to create an altered image based on the image that was displayed on the display device when the uncomfortable state was identified.

6. A video display system as described in any one of claims 2 to 5, further comprising a communication unit configured to transmit information regarding the image displayed on the display device when the comfortable state was identified and the comfortable state, or information regarding the image displayed on the display device when the uncomfortable state was identified and the uncomfortable state.

7. a display device configured to display the video; a state sensor configured to acquire data relating to a state of a viewer viewing the video; a state analysis unit configured to identify a state of the viewer related to a state of the viewer based on data acquired by the state sensor; an image determination unit configured to determine an image to be displayed by the display device as a selected image using the information on the viewer state; a video output unit configured to transmit video data relating to the selected video to the display device; Equipped with The state analysis unit is configured to identify at least whether the viewer is in a wakefulness period, a sleep-onset period, or a wakefulness state. Video display system.

8. The video display system of claim 7, wherein the video determination unit is configured to select a video that encourages the viewer to fall asleep when the viewer is in a sleep phase, or to select a video that encourages the viewer to wake up when the viewer is in a wake phase.

9. The image display system according to claim 2 , wherein the state sensor includes a wearable sensor.

10. 7. The video display system according to claim 1, wherein the state analysis unit is configured to identify at least whether the viewer is in a wakefulness stage, a sleep onset stage, or an awakening state.

11. The image display system according to claim 1 , wherein the state sensor includes a bed-mounted sensor.

12. 12. The image display system according to claim 1, wherein the state sensor is configured to acquire a physiological index of the viewer.

13. 13. The image display system of claim 1, wherein the display device includes a projector configured to project the image.

14. A server for a video display system configured to tally up video evaluations based on information acquired from the video display system according to claim 6.

15. acquiring data relating to a state of a viewer of an image displayed by a display device; determining a viewer state related to a state of the viewer based on the state data; determining an image to be displayed by the display device as a selected image based on the viewer state; transmitting video data relating to the selected video to the display device; A program for displaying images that causes a computer to execute the following: The data regarding the state is data acquired by a state sensor including a wearable sensor configured to acquire data regarding the state of the viewer. program.

16. acquiring data relating to a state of a viewer of an image displayed by a display device; determining a viewer state related to a state of the viewer based on the state data; determining an image to be displayed by the display device as a selected image based on the viewer state; transmitting video data relating to the selected video to the display device; A program for displaying images that causes a computer to execute the following: Identifying the viewer state includes at least identifying that the viewer is in a comfortable state or an uncomfortable state. program.

17. acquiring data relating to a state of a viewer of an image displayed by a display device; determining a viewer state related to a state of the viewer based on the state data; determining an image to be displayed by the display device as a selected image based on the viewer state; transmitting video data relating to the selected video to the display device; A program for displaying images that causes a computer to execute the following: Identifying the viewer state includes identifying at least the viewer as being in a wakefulness stage, a sleep-onset stage, or an awakening state. program.

Images