Display method, display system, and recording medium

Abstract

Display method, display system, and recording medium. In a display surface viewed by multiple users at the same time, when control is performed to reduce eye fatigue, a decrease in display convenience is also suppressed. The display method includes: displaying an image on a display surface; and based on the movement of the eyelids of a first user whose face is facing the display surface, making the brightness or hue of a first part of the image corresponding to the first user different from the brightness or hue of a second part of the image different from the first part.

Description

Technical Field

[0001] The invention relates to a display method, a display system and a recording medium. Background Art

[0002] In the past, a technology for reducing eye fatigue of a user who views a display screen has been developed. For example, the head-mounted display device disclosed in the following patent document 1 includes a display unit, an optical unit, a detection unit, a timer, and a control unit. The display unit displays an image. The optical unit allows the user to visually detect the image displayed on the display unit. The detection unit detects whether the head-mounted display device is worn by the user. The timer measures the time when the detection unit detects that the head-mounted display device is worn by the user and an image is displayed on the display unit. The control unit changes the visually detected image in accordance with the passage of time measured by the timer to suppress the user's eye fatigue.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2003-076353

[0004] In the technology described in Patent Document 1, the user's eye fatigue is reduced by reducing the overall brightness of the display. If this method is applied to a large screen that is used by multiple users at the same time, for example, the visibility of the large screen is reduced when viewed from a distance, and thus the convenience may be reduced. Summary of the invention

[0005] A display method according to one embodiment of the present invention includes: displaying an image on a display surface; and based on the movement of the eyelids of a first user whose face is toward the display surface, making the brightness or hue of a first part of the image corresponding to the first user different from the brightness or hue of a second part of the image different from the first part.

[0006] A display system according to one embodiment of the present invention comprises: a display device that displays an image on a display surface; and a processing device that outputs a signal representing the image to the display device based on the movement of the eyelids of a first user whose face is toward the display surface, so that the brightness or hue of a first part of the image corresponding to the first user is different from the brightness or hue of a second part of the image that is different from the first part.

[0007] A display program of one embodiment of the present invention causes a processing device to execute: causing a display surface to display an image; and based on the movement of the eyelids of a first user whose face is toward the display surface, causing the brightness or hue of a first part of the image corresponding to the first user to be different from the brightness or hue of a second part of the image that is different from the first part. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1It is a schematic diagram showing a usage example of the display system 1 according to the embodiment.

[0009] Figure 2A It is a schematic diagram showing an example of the projected image PC.

[0010] Figure 2B It is a schematic diagram showing an example of the projected image PC.

[0011] Figure 3 It is a diagram schematically showing the configuration of the display system 1 .

[0012] Figure 4 It is a flowchart showing the processing procedure performed by the processing device 103.

[0013] Figure 5 1 is a diagram schematically showing a situation in which one target user UT1 is viewing the projection image PC.

[0014] Figure 6 1 is a diagram schematically showing a situation where a plurality of target users UT2 to UT5 are located near the projected image PC.

[0015] Fig. 7A 1 is a diagram schematically showing a situation in which one target user UT1 is viewing the projection image PC.

[0016] Figure 7B 1 is a diagram schematically showing a situation in which one target user UT1 is viewing the projection image PC.

[0017] Figure 8 1 is a diagram schematically showing a situation where a plurality of target users UT2 to UT4 are located near the projected image PC.

[0018] Description of symbols

[0019] 1: display system; 10: projector; 12: light curtain generating device; 14A: first shooting device; 14B: second shooting device; 100: input device; 101: communication device; 102: storage device; 103: processing device; 103A: touch detection unit; 103B: display image control unit; 103C: user detection unit; 103D: fatigue detection unit; 103E: image processing control unit; 105: projection device; 121: laser light source unit; 122: emission control device; FP: projection area; L: light curtain; P1: first part; P2: second part; PC: projected image; U: user; UT: target user. DETAILED DESCRIPTION

[0020] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. In addition, in the accompanying drawings, the size or scale of each part is appropriately different from the actual one, and there are also parts schematically shown for easy understanding. In addition, in the following description, as long as the description of the present invention is not particularly limited, the scope of the present invention is not limited to these modes.

[0021] [Implementation Method]

[0022] [1. Overview of display system]

[0023] Figure 1 1 is a schematic diagram showing an example of use of the display system 1 according to the embodiment. Figure 1 As shown, the display system 1 is provided for displaying images in a space S where a plurality of people gather, such as a station or a shopping mall. In the present embodiment, the display system 1 projects a projection image PC onto at least a portion of a wall surface W by means of a projector 10. The area of ​​the wall surface W onto which the projection image PC is projected by the projector 10 is referred to as a projection area FP. Alternatively, the entire surface of the wall surface W may be the projection area FP. The projection area FP is an example of a display surface. In addition, the projection image PC is an example of an image. Furthermore, the display system 1 may be, for example, a system for displaying images on a display provided on the wall surface W. In this case, the surface of the display is equivalent to the display surface.

[0024] In the present embodiment, the projection area FP of the display system 1 is of a size that allows multiple users U to simultaneously view the projection image PC. In this case, one projection image PC may be projected onto the entire projection area FP, and multiple users U may simultaneously view the projection image PC, or multiple projection images PC may be projected onto the projection area FP, and each user U may view a different projection image PC. In the latter case, the user U may also be a group of multiple users U acting together. In the present embodiment, it is assumed that, as in the former case, one projection image PC is projected onto the projection area FP, and each user U may view a portion of a different projection image PC.

[0025] In addition, the display system 1 uses the light curtain generating device 12 to Figure 3 The light curtain L shown is projected near the projection area FP. By detecting the reflected light of the light curtain L, the position on the projection image PC touched by the user U is detected. In the display system 1, the content of the projection image PC changes dynamically based on the touch operation of the user U on the projection area FP. That is, in the display system 1, interactive image display is performed.

[0026] Figure 2A as well as Figure 2B It is a schematic diagram showing an example of the projected image PC. Figure 2A and Figure 2B A part area FP-A of the projection area FP is shown. Figure 2A as well as Figure 2B The example assumes that the display system 1 is set in a shopping mall. In the initial state, the image PC-A displayed in the area FP-A includes a plurality of icons IC1 to IC3. For example, the icon IC1 corresponds to a keyword such as "eat". Similarly, the icon IC2 corresponds to a keyword such as "buy", and the icon IC3 corresponds to a keyword such as "play". When the user U touches the area where any of the icons IC1 to IC3 are projected, a list LS of stores in the shopping mall corresponding to the keyword of the icon IC is displayed.

[0027] For example, when the user U touches the area where the icon IC1 corresponding to "eat" is projected, Figure 2B As shown in FIG. 1 , an image PC-B including a list LS of restaurants in a shopping mall is displayed. That is, the list LS of restaurants becomes a part of the projection image PC that changes in conjunction with the touch of the user U.

[0028] The restaurant list LS includes summary information SI of the restaurants, including the restaurant name, the type of food provided, the approximate location information of the floor, etc. When the user U touches the area where the summary information SI of any restaurant is projected, detailed information such as the location information, business hours information, menu information, etc. of the restaurant is displayed.

[0029] Furthermore, the display system 1 may be a system that does not perform interactive image display and displays a predetermined image regardless of whether or not there is an operation by the user U.

[0030] [2. Display system structure]

[0031] Figure 3 1 is a block diagram showing the structure of the display system 1. The display system 1 includes a projector 10, a light curtain generating device 12, a first imaging device 14A, and a second imaging device 14B. Figure 1 As shown, the display system 1 has a relatively large projection area FP. Therefore, at least one of the projector 10, the light curtain generating device 12, the first imaging device 14A, and the second imaging device 14B may be provided in plurality. Figure 1 A plurality of device groups including the projector 10, the light curtain generating device 12, the first imaging device 14A, and the second imaging device 14B are arranged in the width direction of the wall surface W.

[0032] The first camera 14A has a CCD (Charge Coupled Device) image sensor that receives light in the infrared region and the visible light region as a camera element. The camera element is not limited to the CCD image sensor, and may be, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor that receives light in the infrared region and the visible light region. In the first camera 14A, an infrared filter is installed to receive reflected light from the light curtain L based on an object such as the finger F of the user U. The first camera 14A generates first captured image data corresponding to the first captured image obtained by capturing the projection area FP of the wall surface W. In the case where a portion of the light curtain L is reflected by an object, the first captured image data includes the reflected light of the light curtain L based on the object. The first camera 14A may, for example, be integrally formed with the projector 10. The first camera 14A outputs the first captured image data to the projector 10.

[0033] The second camera 14B has a CCD image sensor that receives light in the visible light region as a camera element. The camera element is not limited to the CCD image sensor, and may be, for example, a CMOS image sensor that receives light in the visible light region. The second camera 14B outputs second captured image data corresponding to the second captured image by capturing an area opposite to the projection area FP. As an example, the second camera 14B is set on the wall W with the normal direction of the wall W as the capturing direction. This is to capture the face of the user U whose face is facing the projection area FP in the second captured image. The second camera 14B outputs the second captured image data to the projector 10.

[0034] like Figure 1 As shown, the light curtain generating device 12 is arranged above the projection area FP in the wall surface W. The light curtain generating device 12 may also be arranged on a part of the ceiling C instead of the wall surface W. The light curtain generating device 12 emits a planar light, namely a light curtain L, along the surface of the wall surface W. The light curtain L is not limited to planar light, but may also be light of other shapes. The light curtain L is light in the infrared band. In the present embodiment, the light curtain L is unfolded along the wall surface W, covering at least the projection area FP in the wall surface W. The light curtain L is emitted in a manner separated from the surface of the wall surface W by a distance of several millimeters in the normal direction. In order to improve the detection accuracy of the touch position TP, it is preferable to reduce the distance between the light curtain L and the surface of the wall surface W. On the other hand, if they are too close, false detection may occur depending on the surface state of the wall surface W, etc. Therefore, the distance between the light curtain L and the surface of the wall surface W is appropriately adjusted.

[0035] The light curtain L is used to detect a touch position TP on the projected image PC indicated by the user U. The user U touches the projected image PC with a finger F of the user U in many cases, but the projected image PC may be touched with a stick or a pen, for example.

[0036] Specifically, the light curtain generating device 12 includes a laser light source unit 121 and an emission control device 122 that controls the laser light source unit 121 .

[0037] The laser light source unit 121 emits a light curtain L. As described above, the light curtain L is light in the infrared band, and the laser light source unit 121 emits laser light in the infrared band. The laser light source unit 121 is, for example, a unit in which optical system components such as a collimating lens and a Powell lens are mounted on one or more LDs (Laser Diodes). The light curtain generating device 12 is not limited to using the laser light source unit 121, and may be, for example, a unit that emits the light curtain L using an LED (Light Emitting Diode).

[0038] The emission control device 122 controls the emission angle of the light curtain L relative to the wall surface W and the emission position of the light curtain L on the wall surface W. The emission control device 122 includes an actuator that adjusts the direction and position of the laser light source unit 121 and a control device that controls the actuator.

[0039] like Figure 1 As shown, the projector 10 is installed on a ceiling C connected to a wall surface W. The projector 10 may be installed not on the ceiling C but on a part of the wall surface W. The display surface on which the projector 10 projects an image is not limited to the wall surface W, and may be, for example, a screen.

[0040] The projector 10 includes an input device 100 , a communication device 101 , a storage device 102 , a processing device 103 , and a projection device 105 .

[0041] The input device 100 includes, for example, various operation buttons, operation keys, or a touch panel for receiving input from an administrator of the display system 1. The input device 100 may also be a remote controller that transmits information input from the administrator to the projector 10 in a wireless or wired manner. In this case, the projector 10 has a receiving device that receives information transmitted from the remote controller. The remote controller has various operation buttons, operation keys, or a touch panel for receiving input from the administrator.

[0042] The communication device 101 is an interface for communicating with other devices. In the present embodiment, the communication device 101 communicates with an image server not shown in the figure. The communication in the communication device 101 can be wireless communication or wired communication. In the case of wireless communication, the communication device 101 has a communication circuit and an antenna, etc., for example, wireless communication is performed via Bluetooth or Wi-Fi. Bluetooth and Wi-Fi are registered trademarks. In addition, in the case of wired communication, the communication device 101 has a communication circuit and a connector for connecting communication wiring, etc.

[0043] The storage device 102 is a computer-readable recording medium. The storage device 102 includes, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), and an EEPROM (Electrically Erasable Programmable Read Only Memory). The storage device 102 stores various information and programs processed by the processing device 103. The programs include a display program.

[0044] The projection device 105 projects a projection image PC onto a projection area FP of the wall surface W. The projection device 105 is an example of a display device, and includes an image processing circuit, a frame memory, a liquid crystal light valve, a light valve driving circuit, a light source, a projection optical system, and the like. When the image processing circuit receives image data from an image supply device such as the processing device 103 or an image server (not shown), the image processing circuit expands the image data in the frame memory and performs necessary image processing. The image processing includes, for example, a process of converting the resolution of the image data into the resolution of the liquid crystal light valve, a geometric correction process of eliminating trapezoidal distortion, and the like. The image data after image processing is converted into an image signal, and the image data is projected as an image by driving the liquid crystal light valve, etc. based on the image signal. In addition, in addition to the liquid crystal method using the above-mentioned liquid crystal light valve, the projection device 105 may also be a DLP (Digital Lighting Processing: Digital Light Processing, registered trademark) method. In addition, when the display device is a display, the display device includes a light source and a display panel, etc.

[0045] The processing device 103 may also be composed of a single or multiple processors. For example, the processing device 103 may be composed of a single or multiple CPUs (Central Processing Units). Part or all of the functions of the processing device 103 may also be composed of circuits such as DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), FPGA (Field Programmable Gate Array), etc. The processing device 103 performs various processes in parallel or sequentially.

[0046] The processing device 103 implements the touch detection unit 103A, the display image control unit 103B, the user detection unit 103C, the fatigue detection unit 103D, and the image processing control unit 103E by reading and executing the display program from the storage device 102. All or part of the touch detection unit 103A, the display image control unit 103B, the user detection unit 103C, the fatigue detection unit 103D, and the image processing control unit 103E may be constituted by hardware such as FPGA.

[0047] The touch detection unit 103A detects that the projection area FP is touched. When the reflected light of the light curtain L is captured in the first captured image data captured by the first imaging device 14A, the touch detection unit 103A detects that the projection area FP is touched.

[0048] In addition, the touch detection unit 103A detects the touch position on the projection area FP. When the projection area FP is touched, the touch detection unit 103A detects the touch position TP on the projection area FP based on the first calibration data and the first captured image data. The first calibration data is data that corresponds the coordinates on the image expanded in the frame memory of the projection device 105 to the coordinates on the first captured image data. The coordinates on the frame memory correspond to the coordinates on the projection area FP. By using the first calibration data, the touch detection unit 103A can convert the touch position on the first captured image data into the touch position TP on the projection area FP.

[0049] The display image control unit 103B controls the image to be projected as the projection image PC according to the touch position TP on the projection region FP. That is, the display image control unit 103B changes a part of the projection image PC in conjunction with the touch on the projection region FP.

[0050] For example, the display image control unit 103B sends the coordinates of the touch position TP on the projection area FP to the image server. The image server determines the image to be displayed around the touch position TP, i.e., the next display image, based on the coordinates of the touch position TP on the projection area FP and the image data of the projection image PC currently displayed in the projection area FP, and sends the image data of the next display image to the projector 10. Figure 2A as well as Figure 2B For example, let the projection image PC currently projected on the projection area FP be Figure 2A The image PC-A has the coordinates of the touch position TP included in the display range of the icon IC1 corresponding to "eat". Figure 2B The image data of the image PC-B shown is transmitted as the next display image. The display image control unit 103B projects the image data transmitted as the next display image onto the projection area FP as the projection image PC.

[0051] The user detection unit 103C detects the user U viewing the projection area FP based on the second captured image captured by the second camera 14B. The user detection unit 103C determines, for example, an area having feature points corresponding to a person from the second captured image, and determines the position of the user U on the second captured image. In addition, the user detection unit 103C detects the line of sight of each user U, thereby detecting the user U whose line of sight is directed toward the projection area FP. Hereinafter, the user U viewing the projection area FP detected by the user detection unit 103C is referred to as a "target user UT". The target user UT is an example of the first user.

[0052] The fatigue detection unit 103D detects the movement of the eyelids of the target user UT based on the second captured image. In addition, the fatigue detection unit 103D determines whether the movement of the eyelids of the target user UT shown in the second captured image is in the first state. The first state is, for example, the eyelid movement in a case where there is a high possibility that the target user UT is in an eye fatigue state. Usually, when the user U watches the displayed image for a long time, the eyes become tired. When the eyes are tired, the user U performs characteristic movements of the eyelids such as blinking more and squinting compared to a case where the eyes are not tired. In addition, in this specification, "eye fatigue" and "eye fatigue" have the same meaning.

[0053] Therefore, in this embodiment, the first state is a state in which the number of blinks per unit time of the target user UT increases by a predetermined value or more compared to the initial state. The initial state can be, for example, the number of blinks per unit time after the target user UT is detected by the user detection unit 103C.

[0054] In addition, a second state is defined as a state different from the first state. The second state is, for example, a state other than the first state. In the present embodiment, the second state includes a case where the number of blinks per unit time of the target user UT increases by less than a specified value compared to the initial state, a case where the number of blinks per unit time of the target user UT does not change compared to the initial state, and a case where the number of blinks per unit time of the target user UT decreases compared to the initial state. In addition, the first state and the second state are not limited to the number of blinks, and may be defined, for example, by the ratio of the area of ​​the eye in the second captured image to the facial area of ​​the target user UT.

[0055] The image processing control unit 103E instructs the image processing circuit to perform image processing on a part of the projection image PC. The image processing control unit 103E makes the brightness or hue of the first part P1 corresponding to the target user UT in the projection image PC different from the brightness or hue of the second part P2 different from the first part P1 in the projection image PC based on the movement of the eyelids of the target user UT whose face is facing the projection area FP.

[0056] The first part P1 will be described in detail later, but the first part P1 includes, for example, a part of the projection image PC facing the target user UT. In addition, the second part P2 is a part of the projection image PC other than the first part P1. In other words, the second part P2 is a part having the same brightness or color tone as the brightness or color tone of the first part P1 before the movement of the eyelid of the target user UT becomes the first state.

[0057] In the present embodiment, brightness refers to, for example, the brightness of the projected image PC. As an example, making brightness different refers to making brightness different. In addition, making brightness different may, for example, make the ratio of the maximum brightness to the minimum brightness of the projected image PC, i.e., the contrast, different, or may make the γ value representing the relationship between the size of the input image signal and the brightness of the image different. In addition, in the present embodiment, hue refers to, for example, at least one of brightness and saturation. For example, the brightness of the appearance of the first part P1 may be changed by overlapping a color filter on the first part P1.

[0058] In the present embodiment, when the eyelid movement of the target user UT is in the first state, the image processing control unit 103E makes the brightness of the first portion P1 darker than the brightness of the first portion P1 when the eyelid movement of the user U is in the second state different from the first state. As described above, the first state is a state in which the number of blinks per unit time of the target user UT increases by a predetermined value or more compared to the initial state, and is a state in which it is estimated that the eyes of the target user UT are tired.

[0059] Generally, the projection image PC is displayed brighter than the surrounding environment, so if the user continues to watch the projection image PC for a long time, the eyes may become tired. Therefore, in the display system 1, when it is predicted that the eyes of the target user UT who is watching the projection image PC are tired, the brightness of the first part P1 including the part that the target user UT is watching in the projection image PC is dimmed. As a result, the stimulation to the eyes of the target user UT is reduced, and the eye fatigue of the target user UT can be reduced.

[0060] The image processing control unit 103E may also change the brightness of the first part P1 according to the number of blinks per unit time in the first state. For example, the first state is a state in which the number of blinks per unit time of the target user UT increases by 20% or more compared to the initial state. In this case, when the number of blinks increases by 20% or more and is less than 30% compared to the initial state, the brightness of the first part P1 is set to -5%. In addition, when the number of blinks increases by 30% or more and is less than 40% compared to the initial state, the brightness of the first part P1 is set to -10%. That is, the greater the increase in the number of blinks, the lower the brightness of the first part P1. Thus, by adjusting the brightness of the first part P1 according to the degree of eye fatigue of the target user UT, it is possible to achieve both the reduction of eye fatigue and the easy viewing of the projected image PC. In addition, the brightness of the first part P1 may also be changed to about -30%, for example.

[0061] In addition, when the movement of the eyelid of the target user UT is in the first state, the image processing control unit 103E may change the hue of the first part P1 instead of the brightness of the first part P1, or may change the hue of the first part P1 together with the brightness. Specifically, for example, at least one of the brightness and saturation of the first part P1 may be reduced. In this case, the stimulation to the eyes of the target user UT is also reduced, and the eye fatigue of the target user UT can be reduced.

[0062] In addition, if the brightness or hue of the first part P1 suddenly changes while the target user UT is continuously viewing the first part P1, the target user UT may feel uncomfortable or suspect that the display system 1 has a malfunction. Therefore, the processing device 103 may also notify the target user UT that the brightness or hue is changed when the image processing control unit 103E makes the brightness or hue of the first part P1 different. The notification method is arbitrary, for example, a message or an icon image indicating that the brightness or hue is adjusted may be displayed on the first part P1 or at a position adjacent to the outer edge of the first part P1. The presence or absence of the notification may be set by the administrator of the display system 1 via the input device 100 of the projector 10, for example.

[0063] In addition, after changing the brightness or hue of the first part P1, the image processing control unit 103E may return the brightness or hue of the first part P1 to the state before the change when a prescribed condition is satisfied. The prescribed condition is, for example, a condition that is satisfied when there is a high possibility that the target user UT stops viewing the first part P1. Specifically, the so-called prescribed condition may include, for example, no touch operation from the target user UT for more than a prescribed time, the state in which the face of the target user UT is not facing the projection image PC for more than a prescribed time, and the target user UT is separated from the projection area FP by more than a prescribed distance. When at least one of these conditions is satisfied, there is a high possibility that the target user UT stops viewing the first part P1. Therefore, the image processing control unit 103E returns the brightness or hue of the first part P1 to the state before the change.

[0064] That is, the image processing control unit 103E may also return the brightness of the first part P1 to the brightness of the first part P1 when the movement of the eyelid of the target user UT is in the second state, when at least any of the following conditions 1 to 3 occurs. Condition 1 is a case where the projection area FP is not touched again within a second specified distance from the touched position within a first specified time after the projection area FP is detected to be touched. Condition 2 is a case where the target user UT does not turn his face toward the projection area FP for more than the second specified time. Condition 3 is a case where the target user UT is separated from the projection area FP by more than a third specified distance. In addition, within the second specified distance may also be, for example, within the range of the projection image PC changed by the last touch operation.

[0065] [3. Operation of display system 1]

[0066] Figure 41 is a flowchart showing the processing steps performed by the processing device 103. The processing device 103 functions as the display image control unit 103B, thereby projecting the projection image PC to the projection area FP (step S12). The processing device 103 functions as the user detection unit 103C, thereby detecting the user U viewing the projection area FP, that is, the target user UT (step S14). The processing device 103 stands by during the period when there is no target user UT (step S14: No). When there is a target user UT (step S14: Yes), the processing device 103 functions as the touch detection unit 103A, thereby determining whether the projection area FP is touched (step S16). When the projection area FP is touched (step S16: Yes), the processing device 103 functions as the display image control unit 103B, thereby changing a part of the projection image PC based on the touch position TP (step S18). In addition, when the projection area FP is not touched (step S16: No), the processing device 103 does not change the projection image PC, and transfers the processing to step S20.

[0067] The processing device 103 functions as the fatigue detection unit 103D, thereby detecting the movement of the eyelid of the target user UT (step S20). The processing device 103 functions as the fatigue detection unit 103D, thereby determining whether the movement of the eyelid of the target user UT is in the first state (step S22). When the movement of the eyelid of the target user UT is in the first state (step S22: Yes), the processing device 103 functions as the image processing control unit 103E, thereby dimming the brightness of the first part P1 of the projected image PC (step S24). As described above, the first part P1 of the projected image PC is the part that the target user UT is viewing, for example, the part whose display changes due to the touch of the target user UT. In addition, when the movement of the eyelid of the target user UT is not in the first state (step S22: No), the processing device 103 returns the processing to step S16.

[0068] After dimming the brightness of the first part P1 of the projected image PC in step S24, the processing device 103 functions as the user detection unit 103C, thereby determining whether the target user UT is viewing the first part P1 (step S26). If the target user UT is viewing the first part P1 (step S26: Yes), the processing device 103 returns the process to step S16. In this case, the state in which the brightness of the first part P1 is darker continues. In addition, after returning the process to step S16, if the target user UT further touches the projection area FP and changes the projected image PC, the processing device 103 regards the portion changed by the touch as the new first part P1 and continues the state in which the brightness is dimmed.

[0069] When the target user UT no longer views the first part P1 (step S26 : No), the processing device 103 functions as the image processing control unit 103E to restore the brightness of the first part P1 darkened in step 24 (step S28 ), and returns the process to step S12 .

[0070] [4. Example of setting of Part 1 P1]

[0071] As described above, in the display system 1, when it is predicted that the eyes of the target user UT who is viewing the projected image PC are tired, the brightness of the first part P1 corresponding to the target user UT is dimmed to reduce the eyestrain of the target user UT. On the other hand, in the present embodiment, the range of the projection area FP is large, and it is possible that a plurality of users U view the projected image PC at the same time. Therefore, for example, if the entire projection image PC is dimmed based on the eyelid movement of one target user UT, the projection image PC viewed by other target users UT whose eyes are not tired suddenly darkens, so they may feel uncomfortable or have difficulty viewing the image. Therefore, in the case of a target user UT with tired eyes, which part of the projection image PC is to be dimmed becomes a problem. That is, which area the first part P1 is set to becomes a problem.

[0072] [Setting example 1: Based on an image that changes when touched]

[0073] The first part P1 preferably includes, for example, a part that the target user UT with tired eyes is viewing. As described above, the projected image PC changes according to the touch of the user U. The target user UT is likely to view the part that changes due to his / her touch. Therefore, in setting example 1, the first part P1 is set to the part of the projected image PC that changes due to the touch of the target user UT.

[0074] Figure 5 1 is a diagram schematically showing a situation in which one target user UT1 is viewing the projection image PC. Figure 5 This is a diagram of the target user UT1 as seen from behind. Figure 5 In the projection area FP, Figure 2B Assume that the target user UT1 comes from Figure 2A As a result of touching the icon IC1 corresponding to "eat" among the plurality of icons IC shown, a list LS of restaurants is displayed as a part of the projection image PC. One target user UT1 is viewing the projection image PC, and the list LS of restaurants is located opposite to the target user UT1.

[0075] While the target user UT1 is viewing the list LS of restaurants, if the movement of the eyelids of the target user UT1 becomes the first state, the processing device 103 makes the brightness of the display portion of the list LS of restaurants lower than that of other portions of the projected image PC. As a result, the brightness of the display portion of the list LS of restaurants becomes darker. That is, the list LS of restaurants becomes the first portion P1. In addition, the portion other than the list LS of restaurants in the projected image PC is the second portion P2, and the original brightness is maintained.

[0076] Figure 6 1 is a diagram schematically showing a situation where a plurality of target users UT2 to UT5 are located near the projected image PC. Figure 6 This is a diagram of the target users UT2 to UT5 as seen from behind. Figure 6 In the projection area FP, it is also displayed Figure 2B Assume that the target user UT2 comes from Figure 2A As a result of touching the icon IC1 corresponding to "eat" among the multiple icons IC shown, the list LS of restaurants is displayed as a part of the projected image PC. Since multiple target users UT are viewing the projected image PC, only the target user UT2 is completely opposite to the list LS of restaurants, the target users UT3 and UT4 are only opposite to a part of the list LS of restaurants, and the target user UT5 is not opposite to the list LS of restaurants.

[0077] In this case, it becomes a question as to who among the target users UT2 to UT5 is viewing the list LS of restaurants. Usually, the user U acting in the same group as the target user UT2 is viewing the list LS of restaurants displayed by the touch of the target user UT2. A group is, for example, a plurality of users U acting together as friends, family, acquaintances, etc. It is considered that users U in the same group are located closer to each other than other people outside the group. In addition, even if it is in the same group as the target user UT2, it is difficult for a user U far away from the list LS of restaurants to see the list LS of restaurants, and it is highly likely that the user U is viewing other areas of the projected image PC.

[0078] Therefore, the processing device 103 infers that the target user UT that is a distance Dx from the list of restaurants LS is the target user UT that is viewing the list of restaurants LS. Furthermore, when the eyelid movement of at least one of the target users UT becomes the first state, the processing device 103 makes the brightness of the display portion of the list of restaurants LS darker than that of the other projected image PC portions.

[0079] That is, in the projector 10, the fatigue detection unit 103D detects the movement of the eyelids of each of the target users UT2 to UT4 when the multiple target users UT2 to UT4 are within the distance Dx from the list of restaurants. In addition, the image processing control unit 103E makes the brightness of the list of restaurants LS darker when the eyelid movement of any one of the multiple target users UT2 to UT4 is in the first state than the brightness of the list of restaurants LS when the eyelid movement of all the target users UT2 to UT4 is in the second state. For example, the target user UT2 is an example of the first user, and the target user UT3 is an example of the second user. The distance Dx is an example of the first distance.

[0080] exist Figure 6 In the example, the processing device 103 takes a distance Dx from the width end of the list of restaurants LS, and infers the target user UT that is located within the distance Dx from the list of restaurants LS as the target user UT viewing the list of restaurants LS. The target user UT that is located within the distance Dx from the list of restaurants LS is, for example, the target user UT whose eyes are located within the distance Dx from the list of restaurants LS. In this case, Figure 6 In the example of , the target users UT2 to UT4 are estimated as the target user UT viewing the list LS of dining stores, and the target user UT5 is not estimated as the target user UT viewing the list LS of dining stores.

[0081] When the eyelid movement of at least one of the target users UT2 to UT4 becomes the first state, the processing device 103 makes the brightness of the display portion of the list of restaurants LS darker than other portions. That is, for the target users UT2 to UT4, the list of restaurants LS becomes the first portion P1. On the other hand, when the eyelid movement of the target user UT5 becomes the first state, the processing device 103 does not make the brightness of the display portion of the list of restaurants LS darker. That is, for the target user UT5, the list of restaurants LS does not become the first portion P1.

[0082] In addition, as described above, when a group views the list LS of restaurants, in addition to the method of dimming the brightness of the list LS of restaurants if even one target user UT in the group has tired eyes, for example, the brightness of the list LS of restaurants may be dimmed only when the eyes of the target user UT performing a touch operation are tired. In this case, the brightness of the displayed image of the projection image PC can be changed to match the eye state of the target user UT who actively operates the screen. In particular, when a group views the projection image PC, it is possible to avoid dimming the brightness of the projection image PC in a short period of time. In addition, even when one target user UT is viewing the projection image PC, by identifying the user U who has performed a touch operation, it is possible to more reliably determine the portion that the target user UT is viewing.

[0083] In this case, the user detection unit 103C identifies the user U who has touched the touch position TP based on the second calibration data, the second captured image captured by the second camera 14B, and the touch position TP on the projection area FP detected by the touch detection unit 103A. The second calibration data is data that matches the coordinates on the projection area FP with the coordinates on the second captured image. By matching the coordinates on the projection area FP with the coordinates on the second captured image data, the user U who has touched the touch position TP on the projection area FP can be identified. Specifically, the user detection unit 103C can also detect the movement of the hand of the user U reflected in the second captured image, for example, to identify the user U who has touched the touch position TP.

[0084] That is, the image processing control unit 103E may also make the brightness of the first part P1 darker than the brightness of the first part P1 when the target user UT touches the projection area FP and the movement of the eyelid of the target user UT is in the first state, than when the target user UT touches the projection area FP and the movement of the eyelid of the target user UT is in the second state.

[0085] [Setting Example 2: Using the location of the target user UT as a reference]

[0086] As described above, the first part P1 preferably includes the part that the target user UT with tired eyes is viewing. Generally, the position that the target user UT is viewing is likely to be near the target user UT. Therefore, in setting example 2, the first part P1 is set to be the part of the projected image PC within a predetermined distance from the target user UT.

[0087] Fig. 7A as well as Figure 7B 1 is a diagram schematically showing a situation in which one target user UT1 is viewing the projection image PC. Fig. 7A is a diagram showing the target user UT1 as viewed from behind. Figure 7Bis a diagram of the target user UT1 viewed from the side. Figure 7B 3 , the projection image PC is shown in a three-dimensional manner. However, this is for the purpose of identifying the first portion P1 in the drawing. The projection image PC is actually projected onto the wall surface W.

[0088] For example, with the middle point of both eyes of the target user UT1 as the reference point, the area defined by the sphere of radius R is set as the specific area SP1. The radius R can be set to about 1 m, for example. When the movement of the eyelid of the target user UT1 becomes the first state, the processing device 103 sets the part of the projection image PC overlapping with the specific area SP1 of the target user UT1 as the first part P1. The first part P1 is indicated by a shadow. In this case, the closer the target user UT1 is to the projection area FP, the wider the first part P1 is. In addition, the first part P1 is not set as a part where the target user UT1 is more than the radius R away from the projection area FP.

[0089] This allows the first part P1 to be set regardless of whether the projection image PC changes due to a touch operation. Also, since the first part P1 is set based on the distance from the target user UT regardless of the display content of the projection image PC, the processing load on the processing device 103 can be reduced.

[0090] Figure 8 1 is a diagram schematically showing a situation where a plurality of target users UT2 to UT4 are located near the projected image PC. Figure 8 This is a diagram of the target users UT2 to UT4 as seen from behind. Figure 8 , for example, the specific areas SP2 to SP4 defined by a sphere of radius R are set with the center of both eyes of each target user UT2 to UT4 as a reference point. When the movement of the eyelid of any one of the target users UT2 to UT4 becomes the first state, the processing device 103 sets the portion of the projected image PC overlapping with the specific area SP of the target user UT as the first portion P1. For example, when the movement of the eyelid of the target user UT4 becomes the first state, the portion of the projected image PC overlapping with the specific area SP4 is set as the first portion P1, and the brightness is dimmed.

[0091] Here, it is assumed that a plurality of target users UT are located close to each other and the specific areas SP overlap. Figure 8In the projection image, there is an overlapping region SW where the specific region SP2 of the target user UT2 overlaps with the specific region SP3 of the target user UT3. In this case, when the movement of the eyelid of only one of the target users UT2 and UT3 becomes the first state, the processing device 103 sets the portion of the specific region SP of the target user UT where the movement of the eyelid becomes the first state, excluding the overlapping region SW, as the first portion P1. For example, when the movement of the eyelid of the target user UT2 becomes the first state and the movement of the eyelid of the target user UT3 does not become the first state, the portion of the projection image PC that overlaps with the specific region SP2 and is not included in the overlapping region SW is set as the first portion P1, and the brightness is dimmed.

[0092] As a result, the brightness of most of the portion being viewed by the target user UT2 becomes darker, while the brightness of the portion being viewed by the target user UT3 does not change. That is, when only some of the target users UT have tired eyes, the visual field of the tired target user UT is made darker, while the visual fields of other target users UT can be prevented from becoming darker, thereby improving the visibility of the projected image PC.

[0093] [5. Conclusion]

[0094] As described above, the display system 1 makes the brightness of the first part P1, which is a part corresponding to the target user UT, in the projected image PC different from that of other parts based on the movement of the eyelids of the target user UT. That is, the display system 1 changes only the brightness of the first part P1 corresponding to the target user UT in the projected image PC, which is likely to be viewed by a plurality of users U, based on the movement of the eyelids of the target user UT. Thus, when the brightness of the projected image PC is changed, the influence on users other than the target user UT can be suppressed, so that even when a plurality of users view the projected image PC, the convenience is not easily reduced.

[0095] In addition, when the movement of the eyelid of the target user UT is in the first state, the display system 1 makes the brightness of the first part P1 darker than the brightness of the first part P1 when the movement of the eyelid of the target user UT is in the second state. Thus, for example, when the first state is a state indicating eye fatigue of the target user UT, the brightness of the part viewed by the target user UT can be darkened, thereby reducing the eye fatigue of the target user UT.

[0096] In the display system 1, a portion of the projected image that changes in conjunction with a touch on the projection area FP may be set as the first portion P1. This allows the portion that the target user UT is viewing to be accurately determined, and the effect on other users U when dimming the brightness of the first portion P1 can be suppressed.

[0097] Furthermore, in the display system 1, when a plurality of target users UT are located within a predetermined distance from a portion of the projected image PC, the brightness of a portion of the projected image PC may be dimmed when the eyelid movement of at least one of the plurality of target users UT becomes the first state. This can improve the convenience of viewing the same projected image PC as a group.

[0098] In addition, in the display system 1, when the movement of the eyelid of the target user UT performing the touch operation becomes the first state, the brightness of a part of the projected image PC that changes due to the touch operation may be dimmed. Thus, only the state of the eyelid of the target user UT performing the touch operation is reflected in the brightness of the projected image PC, and the brightness of the projected image PC can be stabilized, and convenience is not easily reduced even when multiple users view the projected image PC.

[0099] Furthermore, in the display system 1, when the target user UT no longer views the first part P1, the brightness of the first part P1 can be restored. Thus, in the display system 1 used by a plurality of users U, when the users U are replaced, the brightness of the projected image PC can be restored to the initial state, and even when a plurality of users view the projected image PC, the convenience is not easily reduced.

[0100] In addition, in the display system 1, when the brightness of the first part P1 is made different, the target user UT may be notified of this fact. Thus, when the brightness of a part of the projected image PC becomes darker, the possibility that the target user UT suspects that the display system 1 is faulty or the like is reduced, and the convenience of using the display system 1 can be improved.

Claims

1. A display method, wherein: The display method includes: displaying an image on a display surface; Based on the movement of the eyelids of the first user whose face is facing the display surface, the brightness or hue of a first portion of the image corresponding to the first user is made different from the brightness or hue of a second portion of the image different from the first portion; detecting a situation where the display surface is touched; as well as In response to the touching, a part of the image is changed. The first part includes the part, Making the brightness or hue of the first part different from the brightness or hue of the second part includes: when the movement of the eyelid of the first user is in a first state, making the brightness of the first part darker than the brightness of the first part when the movement of the eyelid of the first user is in a second state different from the first state, In any of the following situations: within a first time from detection of the display surface being touched, if the display surface is not touched again within a second distance from the touched position, if the first user does not turn his face toward the display surface for more than a second time, and if the first user is more than a third distance away from the display surface, the brightness of the first part is returned to the brightness of the first part when the movement of the eyelid of the first user is in the second state.

2. The display method according to claim 1, wherein: The display method further includes: when the first user and the second user are within a first distance from the portion, detecting movement of the eyelids of the first user and movement of the eyelids of the second user, Making the brightness or hue of the first part different from the brightness or hue of the second part includes: when the movement of the eyelid of the first user is the first state, making the brightness of the first part darker than the brightness of the first part when the movement of the eyelid of the first user and the movement of the eyelid of the second user are the second state.

3. The display method according to claim 1, wherein: Making the brightness or hue of the first part different from the brightness or hue of the second part includes: when the first user touches the display surface and the movement of the eyelid of the first user is the first state, making the brightness of the first part darker than the brightness of the first part when the first user touches the display surface and the movement of the eyelid of the first user is the second state.

4. The display method according to any one of claims 1 to 3, wherein: The display method further includes, when the brightness or the color tone of the first portion is made different, notifying the first user that the brightness or the color tone of the first portion is made different.

5. A display method, wherein: The display method includes: displaying an image on a display surface; Based on the movement of the eyelids of the first user whose face is facing the display surface, the brightness or hue of a first portion of the image corresponding to the first user is made different from the brightness or hue of a second portion of the image different from the first portion; detecting a situation where the display surface is touched; In response to the touching, causing a portion of the image to change; as well as When the first user and the second user are within a first distance from the portion, detecting movement of the eyelids of the first user and movement of the eyelids of the second user, Making the brightness or hue of the first part different from the brightness or hue of the second part includes: when the movement of the eyelid of the first user is in a first state, making the brightness of the first part darker than the brightness of the first part when the movement of the eyelid of the first user and the movement of the eyelid of the second user are in a second state different from the first state, The first part includes the first part.

6. A display system, wherein: The display system has: A display device that displays an image on a display surface; as well as a processing device that outputs, to the display device, a signal representing the image in which the brightness or hue of a first portion of the image corresponding to the first user is made different from the brightness or hue of a second portion of the image different from the first portion, based on the movement of the eyelids of the first user whose face is facing the display surface; The processing device detects that the display surface is touched, The processing device changes a part of the image in response to the touching condition. The processing device returns the brightness of the first portion to the brightness of the first portion when the movement of the eyelid of the first user is in the second state, if any of the following occurs: within a first time from detecting that the display surface is touched, the display surface is not touched again within a second distance from the touched position; the first user does not turn the face toward the display surface for more than a second time; and the first user is more than a third distance away from the display surface. The first part includes the part, The image that makes the brightness or hue of the first part different from the brightness or hue of the second part includes the following image: the brightness of the first part when the movement of the eyelid of the first user is in the first state is darker than the brightness of the first part when the movement of the eyelid of the first user is in the second state different from the first state.

7. The display system according to claim 6, wherein: The display system further includes a camera that captures an area facing the display surface and outputs a captured image. The processing device detects movement of the eyelid of the first user based on the captured image.

8. A recording medium storing a display program, wherein: The display program causes the processing device to execute: causing the display surface to display an image; Based on the movement of the eyelids of the first user whose face is facing the display surface, the brightness or hue of a first portion of the image corresponding to the first user is made different from the brightness or hue of a second portion of the image different from the first portion; detecting a situation where the display surface is touched; In response to the touching, causing a portion of the image to change; as well as In the event that, within a first time from detection of the display surface being touched, no further touching is performed within a second distance from the touched position of the display surface, the first user does not turn his face toward the display surface for more than a second time, or the first user is more than a third distance away from the display surface, the brightness of the first portion is returned to the brightness of the first portion when the movement of the eyelid of the first user is in the second state, The first part includes the part, Making the brightness or hue of the first part different from the brightness or hue of the second part includes: when the movement of the eyelid of the first user is the first state, making the brightness of the first part darker than the brightness of the first part when the movement of the eyelid of the first user is the second state different from the first state.

Images