Display method and display system

Abstract

To improve convenience for a user when a projector variable in a projection direction is used.SOLUTION: A display method comprises: acquiring first information for identifying a user on the basis of an output from a first sensor; acquiring second information on a position of the user on the basis of an output from a second sensor; determining a first direction on the basis of the second information; determining a type of an image on the basis of a combination of the first information and the second information; and projecting image light indicating an image of the type in the first direction by a projector.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a display method and a display system. [Background technology]

[0002] Patent Document 1 discloses a technique for adjusting the projection position and projection direction of image content based on environmental information indicating the usage environment of the projector and user information relating to the user who uses the projector. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2020 / 031740 Summary of the Invention [Problem to be solved by the invention]

[0004] For example, a method for determining the projection position and details of image content may affect user convenience, but this method is not described in Patent Document 1. Therefore, with the technology in Patent Document 1, it is difficult to improve user convenience when using a projector with a variable projection direction. [Means for solving the problem]

[0005] A display method of one embodiment of the present invention includes obtaining first information identifying a user based on output of a first sensor, obtaining second information regarding a position of the user based on output of a second sensor, determining a first direction based on the second information, determining a type of image based on a combination of the first information and the second information, and projecting image light representing the type of image in the first direction by a projector.

[0006] A display system according to one embodiment of the present invention comprises a first sensor, a second sensor, an optical device for projecting image light, a drive device for directing the direction in which the image light is projected in a predetermined direction, and a processor, wherein the processor performs the following operations: obtaining first information identifying a user based on the output of the first sensor; obtaining second information regarding the position of the user based on the output of the second sensor; determining a first direction based on the second information; determining a type of image based on a combination of the first information and the second information; and projecting image light representing the type of image in the first direction by controlling the optical device and the drive device. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a perspective view of a display system according to a first embodiment. [Diagram 2] FIG. 1 is a side view of a display system according to a first embodiment. [Diagram 3] 1 is a block diagram showing a functional configuration of a display system according to a first embodiment. [Figure 4] 4 is a flowchart showing a process executed by a processor according to the first embodiment. [Diagram 5] FIG. 2 is a plan view showing an example of installation of the display system. [Figure 6] FIG. 13 is a diagram showing a state in which the projection direction is oriented in a first direction. [Figure 7] FIG. 4 is a diagram showing an example of a first content selection table. [Figure 8] FIG. 2 is a diagram showing a state in which image light is projected in a first direction. [Figure 9] FIG. 11 is a block diagram showing the functional configuration of a display system according to a second embodiment. [Figure 10] 11 is a flowchart showing a process executed by a processor according to the second embodiment. [Figure 11] FIG. 2 is a plan view showing an example of installation of the display system. [Figure 12]FIG. 13 is a diagram showing a state in which the projection direction is oriented in a first direction. [Figure 13] FIG. 11 is a diagram showing an example of a second content selection table. [Figure 14] FIG. 2 is a diagram showing a state in which image light is projected in a first direction. [Figure 15] FIG. 13 is a block diagram showing the functional configuration of a display system according to a third embodiment. [Figure 16] 13 is a flowchart showing a process executed by a processor according to a third embodiment. [Figure 17] FIG. 2 is a plan view showing an example of installation of the display system. [Figure 18] FIG. 11 is a diagram illustrating an example of a direction setting table. [Figure 19] FIG. 13 is a diagram showing a state in which the projection direction is oriented in a first direction. [Figure 20] FIG. 2 is a diagram showing a state in which image light is projected in a first direction. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. In the drawings, the dimensions of the components may be shown on different scales in order to make the components easier to see.

[0009] [First embodiment] First, a first embodiment of the present disclosure will be described. Fig. 1 is a perspective view showing a schematic external appearance of a display system 1 according to the present embodiment. Fig. 2 is a side view of the display system 1 shown in Fig. 1. As shown in Figs. 1 and 2, the display system 1 includes a projector 10, a base 20, and a driving device 30.

[0010] The projector 10 projects the image light L onto a projection surface (not shown) to display an image on the projection surface. The projection surface may be a dedicated projector screen, or may be a wall surface. As an example, the projector 10 is a hexahedron having six flat surfaces. A light emitting unit 12 that emits the image light L generated inside the projector 10 is provided on a front surface 11 of the projector 10. In the following description, the direction in which the image light L is projected from the projector 10 is referred to as a "projection direction Dp". For example, the projection direction Dp is a direction perpendicular to the front surface 11 and away from the projector 10.

[0011] The base 20 is a support member for installing the projector 10 at a predetermined location. The base 20 supports the projector 10 and the drive unit 30. As an example, the base 20 has a disk shape. The drive unit 30 is disposed on an upper surface 21 of the base 20, and the base 20 is connected to a lower surface 13 of the projector 10 via the drive unit 30.

[0012] The driving device 30 directs the direction in which the image light L is projected, i.e., the projection direction Dp, to a predetermined direction. Specifically, as shown in Fig. 2, the driving device 30 directs the projection direction Dp to a predetermined direction by rotating the projector 10 around at least one of the yaw axis Y and the pitch axis X. The driving device 30 has a first driving device 31 and a second driving device 32.

[0013] The first driving device 31 rotates the projector 10 around the yaw axis Y. As an example, the first driving device 31 has a cylindrical shape with a diameter smaller than that of the base 20. The first driving device 31 is disposed on the upper surface 21 of the base 20 with its central axis coinciding with the central axis of the base 20. The central axis of the first driving device 31 is the yaw axis Y. As the first driving device 31 rotates around the yaw axis Y, the projector 10 also rotates around the yaw axis Y.

[0014] The second driving device 32 rotates the projector 10 around the pitch axis X. As an example, the second driving device 32 has a semi-cylindrical shape. Of the surfaces of the second driving device 32, a surface extending in the radial direction of the second driving device 32 is connected to the lower surface 13 of the projector 10. The second driving device 32 is disposed on the upper end surface of the first driving device 31 with its central axis perpendicular to the yaw axis Y. The central axis of the second driving device 32 is the pitch axis X. As the second driving device 32 rotates around the pitch axis X, the projector 10 also rotates around the pitch axis X.

[0015] The operation of the driving device 30 configured as described above is controlled by a processor 47, which will be described later. That is, the rotational operation of the first driving device 31 about the yaw axis Y and the rotational operation of the second driving device 32 about the pitch axis X are each controlled by the processor 47. More specifically, the yaw angle θ, which is the rotational angle of the first driving device 31 about the yaw axis Y, and the pitch angle α, which is the rotational angle of the second driving device 32 about the pitch axis X, are each controlled by the processor 47, so that the projection direction Dp is directed in a predetermined direction. In this way, in the display system 1 of this embodiment, the projection direction Dp of the projector 10 is variable.

[0016] Fig. 3 is a block diagram showing a schematic functional configuration of the display system 1. As shown in Fig. 3, the display system 1 includes the above-mentioned projector 10 and a driving device 30 as functional components. Furthermore, the projector 10 includes an optical device 41, an input device 42, a communication device 43, a microphone array 44, a speaker 45, a memory 46, and a processor 47.

[0017] The optical device 41 generates image light L representing a color image under the control of the processor 47, and projects the generated image light L in a projection direction Dp. The optical device 41 has a first image generating panel 41a, a second image generating panel 41b, a third image generating panel 41c, a dichroic prism 41d, and a projection optical system 41e.

[0018] The first image generating panel 41a generates red image light LR representing a red image and outputs it to the dichroic prism 41d. The first image generating panel 41a has a plurality of pixels arranged in a matrix, and each of the plurality of pixels outputs red light. The processor 47 controls the amount of red light emitted for each pixel, so that the first image generating panel 41a outputs the red image light LR.

[0019] The second image generating panel 41b generates green image light LG that represents a green image and outputs it to the dichroic prism 41d. The second image generating panel 41b has a plurality of pixels arranged in a matrix, and each of the plurality of pixels outputs green light. The processor 47 controls the amount of green light output for each pixel, so that the green image light LG is output from the second image generating panel 41b.

[0020] The third image generating panel 41c generates blue image light LB representing a blue image and outputs it to the dichroic prism 41d. The third image generating panel 41c has a plurality of pixels arranged in a matrix, and each of the plurality of pixels outputs blue light. The processor 47 controls the amount of blue light output for each pixel, so that the blue image light LB is output from the third image generating panel 41c.

[0021] For example, each of the image generating panels 41a, 41b, and 41c is a self-luminous electro-optical device such as an OLED (Organic Light Emitting Diode) panel or a μLED (Micro Light Emitting Diode) panel. Each of the image generating panels 41a, 41b, and 41c may be a non-self-luminous electro-optical device such as a liquid crystal panel or a DMD (Digital Micromirror Device). When each of the image generating panels 41a, 41b, and 41c is a non-self-luminous electro-optical device, light from a light source (not shown) such as an LED is separated into red light, green light, and blue light. The red light is incident on the first image generating panel 41a. The green light is incident on the second image generating panel 41b. The blue light is incident on the third image generating panel 41c. Also, a single-plate image generating panel may be used to emit light of each color in a time-division manner.

[0022] The dichroic prism 41d generates image light L representing a color image by combining the red image light LR, the green image light LG, and the blue image light LB, and outputs the image light L to the projection optical system 41e. The projection optical system 41e is composed of a plurality of optical elements such as lenses, and enlarges and projects the image light L output from the dichroic prism 41d in the projection direction Dp. Although not shown, the projection optical system 41e is provided with a mechanism that can adjust optical parameters such as the lens shift amount, the lens focus amount, and the lens zoom amount. The optical parameters of the projection optical system 41e are adjusted by controlling these mechanisms by the processor 47.

[0023] The input device 42 is a device that accepts input operations by a user to the projector 10. As an example, the input device 42 includes an operation unit 42a and a light receiving unit 42b. The operation unit 42a is composed of a plurality of operation keys provided on the projector 10. For example, the operation keys include a power key, a menu call key, direction keys, a confirmation key, and a volume adjustment key. The operation keys may be hardware keys or software keys displayed on a touch panel provided on the projector 10. The operation unit 42a outputs an electric signal generated by the user operating each operation key to the processor 47 as an operation signal.

[0024] The light receiving unit 42b includes a photoelectric conversion circuit that receives infrared light transmitted from a remote controller (not shown) of the projector 10 and converts it into an electric signal. The light receiving unit 42b outputs an electric signal obtained by photoelectric conversion of the infrared light to the processor 47 as a remote operation signal. The remote controller is provided with a plurality of operation keys, similar to the operation unit 42a. The remote controller converts an electric signal generated by a user operating each operation key provided on the remote controller into infrared light and transmits it to the projector 10. In other words, the remote operation signal output from the light receiving unit 42b is substantially the same as an electric signal generated by a user operating each operation key on the remote controller. Note that, when the remote controller transmits a radio signal in accordance with a short-range wireless communication standard such as Bluetooth (registered trademark), a receiving device for receiving the radio signal may be provided instead of the light receiving unit 42b.

[0025] The communication device 43 accesses the Internet via a wireless LAN (Local Area Network) that supports wireless communication standards such as Wi-Fi (registered trademark) in accordance with instructions from the processor 47, and communicates with a video content distribution server (not shown), which is an Internet server that provides a video content distribution service. The communication device 43 outputs a video signal received from the video content distribution server to the processor 47.

[0026] The microphone array 44 has a plurality of microphones arranged at a predetermined interval. Each microphone converts the user's voice into an electrical signal. The microphone array 44 outputs the electrical signals obtained from the plurality of microphones as audio signals to the processor 47. The speaker 45 is controlled by the processor 47 to output audio at a predetermined volume.

[0027] The memory 46 includes a non-volatile memory that stores programs and various setting data required for the processor 47 to execute various processes, and a volatile memory that is used as a temporary storage destination for data when the processor 47 executes various processes. For example, the non-volatile memory is an EEPROM (Electrically Erasable Programmable Read-Only Memory) or a flash memory. For example, the volatile memory is a RAM (Random Access Memory).

[0028] The processor 47 is an arithmetic processing device that controls the operations of the projector 10 and the driving device 30 according to a program previously stored in the memory 46. As an example, the processor 47 is configured with one or more central processing units (CPUs). Some or all of the functions of the processor 47 may be configured with circuits such as a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), and a field programmable gate array (FPGA). The processor 47 executes various types of processing in parallel or sequentially.

[0029] The processor 47 controls the drive device 30, the optical device 41, and the speaker 45 based on an operation signal input from the operation unit 42a, a remote operation signal input from the light receiving unit 42b, an audio signal input from the microphone array 44, and a video signal input from the communication device 43. The processes executed by the processor 47 will be described in detail later.

[0030] Next, the operation of the display system 1 configured as above will be described. Fig. 4 is a flowchart showing a first display process executed by the processor 47 in the first embodiment. When an audio signal is input from the microphone array 44 while the processor 47 is operating in an audio standby mode waiting for input of an audio signal, the processor 47 reads a program from the memory 46 and executes it to execute the first display process shown in Fig. 4. The display method of the first embodiment is realized by the processor 47 executing the first display process.

[0031] Fig. 5 is a plan view showing an example of installation of the display system 1. In the example shown in Fig. 5, the display system 1 is installed in the center of a room 200 provided in the home of a user 100. The room 200 has a north wall surface 210 located on the north (N), a south wall surface 220 located on the south (S), an east wall surface 230 located on the east (E), and a west wall surface 240 located on the west (W). In the following description, it is assumed that the projection direction Dp of the projector 10 faces south, the user 100 is located on the west side of the display system 1, and the processor 47 is operating in the voice standby mode, as shown in Fig. 5.

[0032] When user 100 speaks in the situation shown in Fig. 5, an audio signal is output from microphone array 44 to processor 47. When an audio signal is input from microphone array 44 during an operation in the audio standby mode, processor 47 starts the first display process shown in Fig. 4.

[0033] 4, when the processor 47 starts the first display process, it first performs user authentication based on the audio signal input from the microphone array 44 and user information pre-stored in the memory 46 (step S1). Step S1 includes steps S1a, S1b, S1c, and S1d.

[0034] The user information is information about a plurality of users registered as legitimate users of the display system 1. Hereinafter, a user registered as a legitimate user of the display system 1 is referred to as a "registered user." As an example, the user information includes a user ID of the registered user and registered voiceprint data linked to the user ID. The user ID is identification information of the registered user. For example, the user ID may be the name of the registered user or an identification number individually assigned to the registered user. The registered voiceprint data is voiceprint data of the registered user obtained in advance by analyzing the voice of the registered user.

[0035] Specifically, in step S1, the processor 47 first acquires voiceprint data of the user 100 by performing frequency analysis on at least one of the audio signals output from the multiple microphones included in the microphone array 44 (step S1a).

[0036] Next, processor 47 determines whether or not the user information contains registered voiceprint data that matches the voiceprint data of user 100 (step S1b). If the user information contains registered voiceprint data that matches the voiceprint data of user 100 (step S1b: Yes), processor 47 acquires, from among the user IDs contained in the user information, the user ID linked to the registered voiceprint data that matches the voiceprint data of user 100, as the user ID of user 100 (step S1c).

[0037] On the other hand, if the user information does not contain registered voiceprint data matching the voiceprint data of user 100 (step S1b: No), processor 47 controls speaker 45 to output a voice notifying user 100 that user authentication has failed (step S1d). After executing step S1d, processor 47 ends the first display process and returns to the voice standby mode.

[0038] As described above, in step S1, the processor 47 executes obtaining first information identifying the user 100 based on the output of the first sensor. That is, the display method of the first embodiment includes obtaining first information identifying the user 100 based on the output of the first sensor. In the first embodiment, the user ID of the user 100 corresponds to the first information, and the microphone array 44 corresponds to the first sensor.

[0039] Furthermore, in the display method of the first embodiment, the microphone array 44, which is the first sensor, has at least one microphone, and acquiring the first information includes acquiring voiceprint data of the user 100 based on an output of the at least one microphone, and acquiring identification information indicating the user 100 as the first information based on the voiceprint data. In the first embodiment, the user ID of the user 100 corresponds to the identification information indicating the user 100. Note that since voice-based user authentication technology is a commonly known technology, a detailed description of step S1 will be omitted in this embodiment.

[0040] Next, processor 47 determines sound source direction Ds based on the audio signal input from microphone array 44 (step S2). Sound source direction Ds is the direction in which a sound source is located with respect to microphone array 44. Note that since user 100 is the sound source, sound source direction Ds can be rephrased as the direction in which user 100 is located with respect to microphone array 44.

[0041] When the distance from each microphone included in the microphone array 44 to the sound source is different, a time difference occurs between the time when the sound wave arrives from the sound source to each microphone. Hereinafter, the time difference occurring between the time when the sound wave arrives from the sound source to each microphone is referred to as a "sound wave arrival time difference". A technique for determining or calculating a sound source direction Ds based on such a sound wave arrival time difference is generally known as a sound source localization technique. As an example, in the first embodiment, the sound source direction Ds is determined by using this sound source localization technique.

[0042] That is, in step S2, the processor 47 calculates the sound wave arrival time difference based on the audio signals output from the multiple microphones included in the microphone array 44, and determines the sound source direction Ds based on the calculated sound source arrival time difference. As described above, since the sound source localization technology is a generally known technology, detailed description of step S2 is omitted. For example, as shown in FIG. 5, when the user 100 is located on the west side of the display system 1, the processor 47 determines "west" as the sound source direction Ds. Note that, when determining various directions as the orientation, an orientation sensor (not shown) may be used, and the processor 47 may refer to information indicating the correspondence between the direction and the orientation based on the projector 10, which is stored in advance in the memory 46.

[0043] As described above, in step S2, the processor 47 executes acquiring second information on the position of the user 100 based on the output of the second sensor. That is, the display method of the first embodiment includes acquiring second information on the position of the user 100 based on the output of the second sensor. The second information includes information indicating a second direction in which the user 100 is located relative to the second sensor. In the first embodiment, the microphone array 44 corresponds to the second sensor, the sound source direction Ds, i.e., the direction in which the user 100 is located relative to the microphone array 44, corresponds to the second direction, and the information indicating the second direction corresponds to the second information. In this way, the first sensor and the second sensor may be the same sensor, or the first sensor and the second sensor may be different sensors.

[0044] In addition, in the display method of the first embodiment, the microphone array 44, which is the second sensor, has a plurality of microphones, and acquiring the second information includes calculating the time difference between the time when a sound wave arrives from the sound source to each of the plurality of microphones, i.e., the sound wave arrival time difference, based on the output of the plurality of microphones, and determining the sound source direction Ds, which is the second direction, based on the time difference.

[0045] As shown in Fig. 4, after executing the above step S2, the processor 47 determines a first direction D1 based on the sound source direction Ds (step S3). Specifically, in step S3, the processor 47 determines the opposite direction of the sound source direction Ds as the first direction D1. For example, as shown in Fig. 5, when the processor 47 determines "west" as the sound source direction Ds, it determines "east", which is the opposite direction of the sound source direction Ds, as the first direction D1.

[0046] As described above, in step S3, the processor 47 executes determining the first direction D1 based on the information indicating the sound source direction Ds, which is the second information. That is, the display method of the first embodiment includes determining the first direction D1 based on the second information. Determining the first direction D1 includes determining the opposite direction of the sound source direction Ds, which is the second direction, as the first direction D1.

[0047] 4, after executing step S3, the processor 47 controls the driving device 30 so that the projection direction Dp of the projector 10 is oriented to the first direction D1 (step S4). More specifically, in step S4, the processor 47 controls the yaw angle θ of the first driving device 31 and the pitch angle α of the second driving device 32 so that the projection direction Dp of the projector 10 is oriented to the first direction D1.

[0048] Fig. 6 is a diagram showing a state in which the projection direction Dp of the projector 10 is directed to the first direction D1. For example, as described with reference to Fig. 5, when the processor 47 determines "east" as the first direction D1 in a state in which the projection direction Dp of the projector 10 faces south, the processor 47 controls the driving device 30 so that the projection direction Dp of the projector 10 faces from south to east. As a result, as shown in Fig. 6, the projector 10 rotates 90 degrees counterclockwise around the yaw axis Y, so that the projection direction Dp of the projector 10 faces east.

[0049] 4, after executing step S4, the processor 47 determines the projection content based on a combination of the user ID of the user 100 and the sound source direction Ds (step S5). The projection content is video content that is provided to the user 100 by being projected as image light L from the projector 10. The video content is, for example, a video work or a broadcast program that belongs to each video genre, such as a movie, a TV drama, an animation, weather, politics, economics, and sports.

[0050] Specifically, in step S5, the processor 47 determines the projection content based on a combination of the user ID of the user 100 and the sound source direction Ds, and a first content selection table pre-stored in the memory 46. Fig. 7 is a diagram showing an example of the first content selection table. As shown in Fig. 7, the first content selection table is data indicating the correspondence between the combination of the user ID and the sound source direction Ds, and the video genre.

[0051] For example, in the first content selection table, the combination of a user ID of "user A" and a sound source direction Ds of "west" is associated with "movie" as the video genre. In addition, in the first content selection table, the combination of a user ID of "user B" and a sound source direction Ds of "east" is associated with "weather" as the video genre. In addition, in the first content selection table, the combination of a user ID of "user C" and a sound source direction Ds of "south" is associated with "animation" as the video genre.

[0052] In step S5, processor 47 first selects a video genre corresponding to the combination of the user ID of user 100 and sound source direction Ds from among the video genres included in the first content selection table. For example, if "User A" is acquired as the user ID of user 100 in step S1 and "West" is determined as the sound source direction Ds in step S2, processor 47 selects "Movie" from among the video genres included in the first content selection table.

[0053] For example, after selecting a video genre, the processor 47 acquires a distribution list of video contents belonging to the selected video genre from the video content distribution server by communicating with the video content distribution server via the communication device 43. The processor 47 generates a distribution list image including the distribution list and a message requesting the user 100 to perform an operation to select video content from the distribution list. The processor 47 controls the optical device 41 so that image light L representing the distribution list image is projected.

[0054] Fig. 8 is a diagram showing a state in which image light L is projected in a first direction D1. For example, as shown in Fig. 8, when the projection direction Dp of the projector 10 is directed east, which is an example of the first direction D1, and the processor 47 controls the optical device 41 to project the image light L representing the distribution list image, the image light L is projected eastward from the projector 10. As a result, the distribution list image is displayed on the east side of the wall surfaces of the room 200, that is, on the east wall surface 230 located in front of the user 100.

[0055] For example, user 100 uses a remote controller to select video content from the distribution list while viewing the distribution list image displayed on east wall 230. When processor 47 determines that an operation to select video content from the distribution list has been received based on a remote operation signal input from light receiving unit 42b while the distribution list image is being displayed, processor 47 determines the video content selected by user 100 as the projection content.

[0056] Alternatively, the processor 47 may determine the projection content by executing the following process. For example, when video signals of multiple video contents belonging to each video genre are stored in advance in the memory 46, the processor 47 creates a storage list of video contents belonging to a selected video genre. The processor 47 generates a storage list image including the storage list and a message requesting the user 100 to perform an operation to select video content from the storage list. The processor 47 controls the optical device 41 so that image light L representing the storage list image is projected.

[0057] For example, while viewing the displayed saved list image, user 100 uses the remote controller to select video content from the saved list. When processor 47 determines that an operation to select video content from the saved list has been received based on a remote operation signal input from light receiving unit 42b while the saved list image is being displayed, processor 47 determines the video content selected by user 100 as the projection content.

[0058] As described above, in step S5, the processor 47 executes determining the type of image based on a combination of the first information and the second information. That is, the display method of the first embodiment includes determining the type of image based on a combination of the first information and the second information. In the first embodiment, the user ID of the user 100 corresponds to the first information, the information indicating the sound source direction Ds corresponds to the second information, and the projection content corresponds to the type of image. That is, the type of image is video content such as a video work or a broadcast program classified by video genre.

[0059] 4, after executing step S5, the processor 47 controls the optical device 41 to project image light L representing an image of the projection content (step S6). Specifically, in step S6, the processor 47 receives a video signal of the video content determined as the projection content from the video content distribution server via the communication device 43, or reads it from the memory 46. Then, the processor 47 controls the optical device 41 to project image light L representing an image based on image data included in the video signal, and controls the speaker 45 to output sound based on audio data included in the video signal.

[0060] 8, when the processor 47 controls the optical device 41 to project image light L representing an image of the projection content in a state where the projection direction Dp of the projector 10 is directed east, which is an example of the first direction D1, the image light L is projected eastward from the projector 10. As a result, an image of the video content determined as the projection content is displayed on an east-side wall surface 230 located in front of the user 100, among the walls of the room 200.

[0061] As described above, in steps S4 and S6, the processor 47 executes projecting the image light L representing the image of the projection content, which is the determined type of image, in the first direction D1 by controlling the optical device 41 and the driving device 30. That is, the display method of the first embodiment includes projecting the image light L representing the determined type of image in the first direction D1 by the projector 10.

[0062] (Effects of the first embodiment) As described above, the display method of the first embodiment includes obtaining a user ID, which is first information for identifying user 100, based on the output of microphone array 44, which is a first sensor; obtaining information indicating a sound source direction Ds, which is second information regarding the position of user 100, based on the output of microphone array 44, which is a second sensor; determining a first direction D1 based on the second information; determining projection content, which is a type of image, based on a combination of the first information and the second information; and projecting image light L representing an image of the projection content, which is the determined type of image, in the first direction D1 by projector 10. The video content desired by the user 100 may differ depending on the personal attributes of the user 100, such as the age, sex, occupation, and hobbies, as well as the location of the user 100. In the display method of the first embodiment, the projection content is determined based on first information for identifying the user 100 and second information related to the location of the user 100, and image light L representing an image of the projection content is projected in a first direction D1 determined based on the second information. This increases the likelihood that an image of video content suited to the personal attributes and position of the user 100 will be displayed on a projection surface that is easily visible to the user 100. Therefore, according to the display method of the first embodiment, it is possible to improve the convenience for the user 100 when using the projector 10 with a variable projection direction Dp.

[0063] In the display method of the first embodiment, the second information includes information indicating a sound source direction Ds, which is a second direction in which the user 100 is located relative to the microphone array 44, which is the second sensor, and determining the first direction D1 includes determining the opposite direction of the sound source direction Ds, which is the second direction, as the first direction D1. In this way, by determining the opposite direction of the second direction in which user 100 is positioned relative to the second sensor as the first direction D1, an image of the projected content can be displayed on a projection surface located in front of user 100, i.e., a projection surface that is easily visible to user 100, thereby improving convenience for user 100.

[0064] In the display method of the first embodiment, determining the projection content, which is a type of image, includes determining the projection content, which is a type of image, based on a combination of a user ID, which is a first information, and a sound source direction Ds, which is a second direction. In this way, by determining the projection content based on the combination of the first information, the user ID, and the second direction, the sound source direction Ds, an image of video content that is suitable for the combination of the personal attributes of user 100 and the second direction in which user 100 is located can be displayed on a projection surface located in front of user 100, thereby improving convenience for user 100.

[0065] In the display method of the first embodiment, the microphone array 44, which is the second sensor, has a plurality of microphones, and acquiring the second information includes calculating the time difference between the time when the sound wave reaches each of the plurality of microphones from the sound source based on the output of the plurality of microphones, and determining the sound source direction Ds, which is the second direction, based on the time difference. In this way, by calculating the time difference between the time when the sound wave reaches each of the multiple microphones from the sound source based on the output of the multiple microphones, the sound source direction Ds, which is the direction in which the sound source is located relative to the second sensor, can be accurately obtained as the second direction in which the user 100 is located.

[0066] In the display method of the first embodiment, the microphone array 44, which is a first sensor, has at least one microphone, and acquiring the first information includes acquiring voiceprint data of the user 100 based on the output of the at least one microphone, and acquiring a user ID, which is identification information indicating the user 100, as the first information based on the voiceprint data. In this way, by acquiring voiceprint data, which is biometric data unique to user 100, based on the output of at least one microphone, it is possible to accurately acquire the user ID of user 100 as the first information. Also, as in the first embodiment, the first sensor and the second sensor are the same sensor, i.e., microphone array 44, so that both the first information and the second information can be acquired based on the output of this one sensor.

[0067] The display system 1 of the first embodiment includes a microphone array 44 as a first sensor, a microphone array 44 as a second sensor, an optical device 41 that projects image light L, a driving device 30 that directs the direction in which the image light L is projected to a predetermined direction, and a processor 47. The processor 47 executes the following operations: acquiring a user ID that is first information for identifying a user 100 based on the output of the first sensor; acquiring information indicating a sound source direction Ds that is second information related to the position of the user 100 based on the output of the second sensor; determining a first direction D1 based on the second information; determining projection content that is a type of image based on a combination of the first information and the second information; and projecting image light L representing an image of the projection content that is the determined type of image in the first direction D1 by controlling the optical device 41 and the driving device 30. In the display system 1 of the first embodiment, the projection content is determined based on first information identifying the user 100 and second information regarding the position of the user 100, and image light L representing an image of the projection content is projected in a first direction D1 determined based on the second information. This increases the likelihood that an image of video content suited to the personal attributes and position of the user 100 will be displayed on a projection surface that is easily visible to the user 100. Therefore, according to the display system 1 of the first embodiment, it is possible to improve the convenience for the user 100 when using the projector 10 with a variable projection direction Dp.

[0068] Second Embodiment A second embodiment of the present disclosure will be described below. In each embodiment exemplified below, components common to the first embodiment will be denoted by the same reference numerals as those used in the first embodiment, and detailed descriptions thereof will be omitted as appropriate. Fig. 9 is a block diagram showing a schematic functional configuration of a display system 2 according to the second embodiment. As shown in Fig. 9, the display system 2 includes a projector 10A and a driving device 30 as functional components. The projector 10A according to the second embodiment is identical to the projector 10 according to the first embodiment in that it includes an optical device 41, an input device 42, a communication device 43, a speaker 45, a memory 46, and a processor 47.

[0069] Projector 10A differs from projector 10 in that projector 10A includes a microphone 48 as a first sensor instead of microphone array 44. Projector 10A also differs from projector 10 in that projector 10A includes a first camera 49 as a second sensor. Below, the differences between the second embodiment and the first embodiment will be described in detail.

[0070] The microphone 48 converts the voice of the user 100 into an electrical signal and outputs the electrical signal to the processor 47 as an audio signal. The first camera 49 is a digital camera that captures an image in one direction. The first camera 49 is attached to the projector 10A so as to capture an image in the projection direction Dp. In other words, the first camera 49 is attached to the projector 10A so that its capture direction coincides with the projection direction Dp. The first camera 49 outputs captured image data indicating the captured image to the processor 47.

[0071] Next, the operation of the display system 2 configured as above will be described. Fig. 10 is a flowchart showing the second display process executed by the processor 47 in the second embodiment. When an audio signal is input from the microphone 48 during an operation in the voice standby mode, the processor 47 reads a program from the memory 46 and executes it to execute the second display process shown in Fig. 10. The display method of the second embodiment is realized by the processor 47 executing the second display process.

[0072] Fig. 11 is a plan view showing an example of installation of the display system 2. In the example shown in Fig. 11, the display system 2 is installed in the center of a room 200 in the home of the user 100, similar to the example shown in Fig. 5. In the following description, it is assumed that the projection direction Dp of the projector 10A faces east, the user 100 is located on the east side of the display system 2, and the processor 47 is operating in the voice standby mode, as shown in Fig. 11. In this case, the first camera 49 captures the east, which is the same direction as the projection direction Dp.

[0073] 11, when user 100 speaks, an audio signal is output from microphone 48 to processor 47. When an audio signal is input from microphone 48 during an operation in the audio standby mode, processor 47 starts the second display process shown in FIG.

[0074] 10, when the second display process is started, the processor 47 first performs user authentication based on a voice signal input from the microphone 48 and user information pre-stored in the memory 46 (step S11). Step S11 includes steps S11a, S11b, S11c, and S11d. The process of step S11 in the second display process is substantially the same as the process of step S1 in the first display process, so the process of step S11 will be briefly described below.

[0075] In step S11, processor 47 first acquires voiceprint data of user 100 by performing frequency analysis on the audio signal output from microphone 48 (step S11a). Processor 47 determines whether or not registered voiceprint data matching the voiceprint data of user 100 is present in the user information (step S11b). If registered voiceprint data matching the voiceprint data of user 100 is present in the user information (step S11b: Yes), processor 47 acquires, from among the user IDs included in the user information, the user ID linked to the registered voiceprint data matching the voiceprint data of user 100 as the user ID of user 100 (step S11c).

[0076] On the other hand, if the user information does not contain registered voiceprint data matching the voiceprint data of user 100 (step S11b: No), processor 47 controls speaker 45 to output a voice notifying user 100 that user authentication has failed (step S11d). After executing step S11d, processor 47 ends the second display process and returns to the voice standby mode.

[0077] As described above, in step S11, the processor 47 executes acquiring first information for identifying the user 100 based on the output of the first sensor. That is, the display method of the second embodiment includes acquiring first information for identifying the user 100 based on the output of the first sensor. In the second embodiment, the user ID corresponds to the first information, and the microphone 48 corresponds to the first sensor.

[0078] In addition, in the display method of the second embodiment, the first sensor has at least one microphone 48, and acquiring the first information includes acquiring voiceprint data of the user 100 based on the output of the at least one microphone 48, and acquiring a user ID, which is identification information indicating the user 100, as the first information based on the voiceprint data.

[0079] Next, the processor 47 determines a user direction Du based on the captured image data input from the first camera 49 (step S12). The user direction Du is the direction in which the user 100 is located relative to the first camera 49. For example, as shown in FIG. 11, when the user 100 is located on the east side of the display system 2 and the first camera 49 captures an image of the east, which is the projection direction Dp, the captured image data indicating the captured image in which the user 100 appears is output from the first camera 49 to the processor 47.

[0080] In the second embodiment, the user information includes face identification information linked to the user ID, in addition to the user ID and registered voiceprint data of the registered user. The face identification information is information that identifies the face of the registered user. In step S12, the processor 47 acquires face identification information linked to the user ID of the user 100 from the face identification information included in the user information. Then, the processor 47 performs image analysis of the captured image based on the captured image data and the face identification information, thereby determining, from among the image areas included in the captured image, an image area corresponding to the face of the user 100 as the user image area.

[0081] In the second embodiment, a coordinate conversion formula for converting coordinates in the image coordinate system of the captured image into coordinates in a coordinate system representing the movable range of the projection direction Dp is stored in advance in the memory 46. The coordinate system representing the movable range of the projection direction Dp is a coordinate system defined by a yaw angle θ and a pitch angle α. In step S12, the processor 47 converts the center coordinates of the user image area in the image coordinate system into coordinates defined by the yaw angle θ and the pitch angle α based on this coordinate conversion formula. The coordinates obtained by such coordinate conversion represent the direction in which the user 100 is located relative to the first camera 49. That is, the processor 47 acquires the direction corresponding to the coordinates obtained by the coordinate conversion as the user direction Du. For example, the processor 47 acquires the orientation corresponding to the coordinates as the user direction Du by referring to a table indicating the correspondence between the orientation stored in advance in the memory 46 and the coordinate system representing the movable range of the projection direction Dp. As shown in FIG. 11, when the user 100 is located on the east side of the display system 2, the processor 47 acquires "east" corresponding to the coordinates as the user direction Du. The processor 47 may acquire, as the user direction Du, the coordinates themselves in the coordinate system that represents the movable range of the projection direction Dp.

[0082] As described above, in step S12, the processor 47 executes acquiring second information on the position of the user 100 based on the output of the second sensor. That is, the display method of the second embodiment includes acquiring second information on the position of the user 100 based on the output of the second sensor. The second information includes information indicating a second direction in which the user 100 is located relative to the second sensor. In the second embodiment, the first camera 49 corresponds to the second sensor, the user direction Du, i.e., the direction in which the user 100 is located relative to the first camera 49, corresponds to the second direction, and the information indicating the second direction corresponds to the second information.

[0083] Moreover, in the display method of the second embodiment, acquiring the second information includes acquiring a first image including the user 100 based on the output of the first camera 49, and determining a user direction Du, which is the second direction, based on the first image. As described above, among the images captured by the first camera 49, the image in which the user 100 appears corresponds to the first image.

[0084] If the user image area does not exist in the captured image obtained at the start of step S12, it is considered that the user 100 does not exist in the projection direction Dp, which is the capturing direction of the first camera 49. Therefore, if the user image area does not exist in the captured image obtained at the start of step S12, the processor 47 acquires captured image data from the first camera 49 at a predetermined time interval while controlling the drive device 30 so that the projector 10A rotates once around the yaw axis Y. Then, each time the processor 47 acquires the captured image data, it determines whether or not the user image area exists in the captured image. If it is determined that the user image area exists in the captured image, the processor 47 stops controlling the drive device 30 and executes the above-mentioned coordinate conversion to determine the user direction Du.

[0085] As shown in Fig. 10, after executing the above step S12, the processor 47 determines a first direction D1 based on the user direction Du (step S13). Specifically, in step S13, the processor 47 determines the opposite direction of the user direction Du as the first direction D1. For example, as shown in Fig. 11, when the processor 47 determines "east" as the user direction Du, it determines "west", which is the opposite direction of the user direction Du, as the first direction D1.

[0086] As described above, in step S13, the processor 47 executes determining the first direction D1 based on the information indicating the user direction Du, which is the second information. That is, the display method of the second embodiment includes determining the first direction D1 based on the information indicating the user direction Du, which is the second information. Determining the first direction D1 includes determining the opposite direction of the user direction Du, which is the second direction, as the first direction D1.

[0087] 10, after executing step S13 described above, the processor 47 controls the drive device 30 so that the projection direction Dp of the projector 10A is oriented to the first direction D1 (step S14). More specifically, in step S14, the processor 47 controls the yaw angle θ of the first drive device 31 and the pitch angle α of the second drive device 32 so that the projection direction Dp of the projector 10A is oriented to the first direction D1. As a result, the shooting direction of the first camera 49 is also oriented to the first direction D1.

[0088] Fig. 12 is a diagram showing a state in which the projection direction Dp of the projector 10A is directed to the first direction D1. For example, as described with reference to Fig. 11, when the processor 47 determines "west" as the first direction D1 in a state in which the projection direction Dp of the projector 10A faces east, the processor 47 controls the driving device 30 so that the projection direction Dp of the projector 10A faces from east to west. As a result, as shown in Fig. 12, the projector 10A rotates 180 degrees counterclockwise around the yaw axis Y, so that the projection direction Dp of the projector 10A faces west. In this case, the shooting direction of the first camera 49 is also directed west.

[0089] 10, after executing step S14, processor 47 determines a viewing location based on the captured image data input from first camera 49 (step S15). The viewing location is the location where user 100 is located.

[0090] For example, as shown in FIG. 12, when the projection direction Dp is directed toward the first direction D1, i.e., the "west," and the first camera 49 captures the first direction D1, captured image data showing the captured image that captures the area of ​​the room 200 that is located in the first direction D1, i.e., the western area including the western wall surface 240, is output from the first camera 49 to the processor 47.

[0091] In step S15, the processor 47 determines one of the multiple viewing location candidates as the viewing location based on the captured image data. For example, the multiple viewing location candidates include rooms such as a living room, a bedroom, and a washroom. The memory 46 stores in advance image data obtained by capturing an image of the inside of the room that is the viewing location candidate using the first camera 49 as reference image data.

[0092] Specifically, in step S15, processor 47 calculates the similarity between the captured image indicated by the captured image data and the reference image indicated by the reference image data by image processing such as pattern matching. As described above, the reference image is an image obtained by capturing an image of the inside of a room that is a candidate viewing location using first camera 49. Then, processor 47 determines, as the viewing location, the candidate viewing location from which the reference image having the highest similarity to the captured image is obtained, among the multiple candidate viewing locations.

[0093] As described above, in the display method of the second embodiment, the second information on the position of the user 100 includes, in addition to information indicating the user direction Du, which is the second direction in which the user 100 is located, information indicating the viewing location, which is the location where the user 100 is located. Also, in the display method of the second embodiment, acquiring the second information includes acquiring a second image including an area located in the first direction D1 based on the output of the first camera 49, and determining the viewing location based on the second image. As described above, of the captured images obtained from the first camera 49, the captured image obtained by capturing the first direction D1 corresponds to the second image.

[0094] As shown in Fig. 10, after executing step S15, processor 47 determines the projection content based on the combination of user ID of user 100 and viewing location (step S16). Specifically, in step S16, processor 47 determines the projection content based on the combination of user ID of user 100 and viewing location and a second content selection table pre-stored in memory 46. Fig. 13 is a diagram showing an example of the second content selection table. As shown in Fig. 13, the second content selection table is data indicating the correspondence between the combination of user ID and viewing location and the video genre.

[0095] For example, in the second content selection table, the combination of a user ID of "User A" and a viewing location of "Living room" is associated with "Movie" as the video genre. Also, in the second content selection table, the combination of a user ID of "User B" and a viewing location of "Bedroom" is associated with "Cosmetics" as the video genre. Also, in the second content selection table, the combination of a user ID of "User C" and a viewing location of "Bathroom" is associated with "Animation" as the video genre.

[0096] In step S16, processor 47 first selects a video genre corresponding to the combination of user ID of user 100 and viewing location from the video genres included in the second content selection table. For example, if "user A" is acquired as the user ID of user 100 in step S11 and "living room" is determined as the viewing location in step S15, processor 47 selects "movie" from the video genres included in the second content selection table. After selecting the video genre, processor 47 determines the projection content by performing the same process as the process described in the first embodiment.

[0097] As described above, in step S16, processor 47 executes determining the projection content, which is the type of image, based on a combination of the user ID, which is the first information, and the information indicating the viewing location, which is the second information. That is, the display method of the second embodiment includes determining the projection content, which is the type of image, based on a combination of the user ID, which is the first information, and the information indicating the viewing location, which is the second information. In addition, the second information includes information indicating the viewing location, which is the location where the user is located, and determining the projection content, which is the type of image, includes determining the projection content, which is the type of image, based on a combination of the user ID, which is the first information, and the viewing location.

[0098] 10, after executing step S16, the processor 47 controls the optical device 41 to project image light L representing an image of the projection content (step S17). The process of step S17 in the second display process is the same as the process of step S6 in the first display process, and therefore a description of step S17 will be omitted.

[0099] Fig. 14 is a diagram showing a state in which image light L is projected in a first direction D1. For example, as shown in Fig. 14, when the projection direction Dp of the projector 10A is directed to the west, which is an example of the first direction D1, and the processor 47 controls the optical device 41 to project the image light L representing the projection content, the image light L is projected from the projector 10A toward the west. As a result, an image of the video content determined as the projection content is displayed on a western wall surface 240 located in front of the user 100, among the walls of the room 200.

[0100] As described above, in steps S14 and S17, the processor 47 executes projecting the image light L representing the image of the projection content, which is the determined type of image, in the first direction D1 by controlling the optical device 41 and the drive device 30. That is, the display method of the second embodiment includes projecting the image light L representing the image of the projection content, which is the determined type of image, in the first direction D1 by the projector 10A.

[0101] (Effects of the second embodiment) As described above, the display method of the second embodiment includes acquiring a user ID, which is first information for identifying a user 100, based on the output of the microphone 48, which is a first sensor; acquiring second information regarding the position of the user 100, which is information indicating a user direction Du and a viewing location, based on the output of the first camera 49, which is a second sensor; determining a first direction D1 based on the second information, which is the information indicating the user direction Du; determining projection content, which is a type of image, based on a combination of the first information and the second information, which is the information indicating the viewing location; and projecting image light L representing an image of the projection content, which is the determined type of image, in the first direction D1 by the projector 10A. As in the first embodiment, in the display method of the second embodiment, projection content is determined based on first information identifying user 100 and second information regarding the position of user 100, and image light L representing an image of the projection content is projected in a first direction D1 determined based on the second information. This increases the likelihood that an image of video content suited to the personal attributes and position of the user 100 will be displayed on a projection surface that is easily visible to the user 100. Therefore, according to the display method of the second embodiment, it is possible to improve the convenience for the user 100 when using the projector 10A with a variable projection direction Dp.

[0102] In the display method of the second embodiment, the second information includes information indicating a user direction Du, which is a second direction in which the user 100 is located relative to the first camera 49, which is the second sensor, and determining the first direction D1 includes determining the opposite direction of the second direction, the user direction Du, as the first direction D1. In this way, by determining the opposite direction of the second direction in which user 100 is positioned relative to the second sensor as the first direction D1, an image of the projected content can be displayed on a projection surface located in front of user 100, i.e., a projection surface that is easily visible to user 100, thereby improving convenience for user 100.

[0103] In the display method of the second embodiment, the second information further includes information indicating a viewing location, which is a location where the user 100 is located, and determining the projection content, which is a type of image, includes determining the projection content, which is a type of image, based on a combination of the first information, which is a user ID, and the viewing location. In the first embodiment, the projection content is determined based on a combination of the user ID, which is the first information, and the sound source direction Ds. Since the sound source direction Ds is the direction in which the user 100 is located relative to the second sensor, it is difficult to determine the location in which the user 100 is located, i.e., the room or the like that is the viewing location, from the sound source direction Ds. Therefore, as described above, by determining the projection content based on a combination of the first information, the user ID, and the viewing location, it is possible to display on the projection surface an image of video content that is suitable for the personal attributes and viewing location of the user 100, thereby improving the convenience for the user 100 compared to the first embodiment.

[0104] In the display method of the second embodiment, the second sensor is a first camera 49 that captures an image in one direction, and acquiring the second information includes acquiring a first image including a user 100 based on the output of the first camera 49, determining a user direction Du, which is a second direction, based on the first image, acquiring a second image including an area located in the first direction D1 based on the output of the first camera 49, and determining a viewing location based on the second image. In this way, by determining both the user direction Du and the viewing location based on the first and second images obtained from the first camera 49, second information regarding the position of the user 100 can be accurately obtained using a single camera.

[0105] In the display method of the second embodiment, the first sensor has at least one microphone 48, and acquiring the first information includes acquiring voiceprint data of the user 100 based on the output of the at least one microphone 48, and acquiring a user ID, which is identification information indicating the user 100, as the first information based on the voiceprint data. In this way, by acquiring voiceprint data, which is biometric data unique to user 100, based on the output of at least one microphone 48, it is possible to accurately obtain the user ID of user 100 as the first information.

[0106] The display system 2 of the second embodiment includes a microphone 48 as a first sensor, a first camera 49 as a second sensor, an optical device 41 that projects image light L, a driving device 30 that directs the direction in which the image light L is projected to a predetermined direction, and a processor 47. The processor 47 executes the following operations: acquiring a user ID that is first information for identifying a user 100 based on the output of the first sensor; acquiring information indicating a user direction Du and a viewing location that is second information regarding the position of the user 100 based on the output of the second sensor; determining a first direction D1 based on the information indicating the user direction Du that is the second information; determining a projection content that is a type of image based on a combination of the first information and the information indicating the viewing location that is the second information; and projecting image light L that represents an image of the projection content that is the determined type of image in the first direction D1 by controlling the optical device 41 and the driving device 30. In the display system 2 of the second embodiment, the projection content is determined based on first information identifying the user 100 and second information regarding the position of the user 100, which is information indicating the viewing location, and image light L representing an image of the projection content is projected in a first direction D1 determined based on the second information, which is information indicating the user direction Du. This increases the likelihood that an image of video content suited to the personal attributes and position of the user 100 will be displayed on a projection surface that is easily visible to the user 100. Therefore, according to the display system 2 of the second embodiment, it is possible to improve the convenience for the user 100 when using the projector 10A with a variable projection direction Dp.

[0107] Third Embodiment The third embodiment of the present disclosure will be described below. Fig. 15 is a block diagram showing a schematic functional configuration of a display system 3 according to the third embodiment. As shown in Fig. 15, the display system 3 includes a projector 10B and a drive device 30 as functional components. The projector 10B of the third embodiment is identical to the projector 10A of the second embodiment in that it includes an optical device 41, an input device 42, a communication device 43, a speaker 45, a memory 46, a processor 47, and a microphone 48.

[0108] Projector 10B differs from projector 10A in that projector 10B includes second camera 50 as a second sensor instead of first camera 49. The differences between the third embodiment and the second embodiment will be described in detail below.

[0109] The second camera 50 is a digital camera that captures images in a 360-degree direction centered on the second camera 50. For example, the second camera 50 is attached to the top surface of the housing of the projector 10B. The second camera 50 outputs captured image data indicating the captured image to the processor 47.

[0110] Next, the operation of the display system 3 configured as above will be described. Fig. 16 is a flowchart showing a third display process executed by the processor 47 in the third embodiment. When an audio signal is input from the microphone 48 during an operation in the voice standby mode, the processor 47 reads a program from the memory 46 and executes it to execute the third display process shown in Fig. 16. The display method of the third embodiment is realized by the processor 47 executing the third display process.

[0111] Fig. 17 is a plan view showing an example of installation of the display system 3. In the example shown in Fig. 17, the display system 3 is installed in the center of a room 200 in the home of the user 100, similar to the example shown in Fig. 5. In the following description, it is assumed that the processor 47 is operating in a voice standby mode with the projection direction Dp of the projector 10B facing east and the user 100 located on the east side of the display system 3, as shown in Fig. 17. The second camera 50 captures images in a 360-degree direction with the second camera 50 as the center.

[0112] 17, when user 100 speaks, an audio signal is output from microphone 48 to processor 47. When an audio signal is input from microphone 48 during an operation in the audio standby mode, processor 47 starts the third display process shown in FIG.

[0113] 16, when the third display process is started, the processor 47 first performs user authentication based on an audio signal input from the microphone 48 and user information pre-stored in the memory 46 (step S21). Step S21 includes steps S21a, S21b, S21c, and S21d. The process of step S21 in the third display process is the same as the process of step S11 in the second display process, so the process of step S21 will be briefly described below.

[0114] In step S21, processor 47 first acquires voiceprint data of user 100 by performing frequency analysis on the audio signal output from microphone 48 (step S21a). Processor 47 determines whether or not registered voiceprint data matching the voiceprint data of user 100 is present in the user information (step S21b). If registered voiceprint data matching the voiceprint data of user 100 is present in the user information (step S21b: Yes), processor 47 acquires, from among the user IDs included in the user information, the user ID linked to the registered voiceprint data matching the voiceprint data of user 100 as the user ID of user 100 (step S21c).

[0115] On the other hand, if there is no registered voiceprint data in the user information that matches the voiceprint data of user 100 (step S21b: No), processor 47 controls speaker 45 to output a voice notifying user 100 that user authentication has failed (step S21d). After executing step S21d, processor 47 ends the third display process and returns to the voice standby mode.

[0116] As described above, in step S21, the processor 47 executes acquiring first information for identifying the user 100 based on the output of the first sensor. That is, the display method of the third embodiment includes acquiring first information for identifying the user 100 based on the output of the first sensor. In the third embodiment, the user ID corresponds to the first information, and the microphone 48 corresponds to the first sensor.

[0117] Next, the processor 47 determines the viewing location based on the captured image data input from the second camera 50 (step S22). As in the second embodiment, the viewing location is the location where the user 100 is located.

[0118] For example, as shown in FIG. 17, when the display system 3 is installed in the center of the room 200 and the second camera 50 captures an image in a 360-degree direction centered on the second camera 50, captured image data indicating a captured image that captures an area within the room 200 that is included in a 360-degree range centered on the second camera 50 is output from the second camera 50 to the processor 47.

[0119] In step S22, the processor 47 determines one of the multiple viewing location candidates as the viewing location based on the captured image data. For example, the multiple viewing location candidates include rooms such as a living room, a bedroom, and a bathroom. Image data obtained by capturing an image of the inside of the room that is the viewing location candidate using the second camera 50 is stored in advance in the memory 46 as reference image data.

[0120] Specifically, in step S22, processor 47 calculates the similarity between the captured image indicated by the captured image data and the reference image indicated by the reference image data by image processing such as pattern matching. As described above, the reference image is an image obtained by capturing an image of the inside of a room that is a candidate viewing location using second camera 50. Then, processor 47 determines, as the viewing location, the candidate viewing location from which a captured image having the highest similarity to the reference image has been obtained, among the multiple candidate viewing locations.

[0121] As described above, in step S22, the processor 47 executes obtaining second information regarding the position of the user 100 based on the output of the second sensor. That is, the display method of the third embodiment includes obtaining second information regarding the position of the user 100 based on the output of the second sensor. The second information includes information indicating a viewing location where the user 100 is located. In the third embodiment, the second camera 50 capturing images in a 360-degree direction corresponds to the second sensor.

[0122] In the display method of the third embodiment, acquiring the second information includes acquiring a third image including an area included in a range of 360 degrees centered on the second camera 50 based on the output of the second camera 50, and determining a viewing location based on the third image. As described above, among the images captured by the second camera 50, an image that includes an area included in a range of 360 degrees centered on the second camera 50 corresponds to the third image.

[0123] As shown in Fig. 16, after executing the above step S22, the processor 47 determines a first direction D1 based on a combination of the user ID of the user 100 and the viewing location (step S23). Specifically, in step S23, the processor 47 determines the first direction D1 based on the combination of the user ID of the user 100 and the viewing location and a direction setting table pre-stored in the memory 46. Fig. 18 is a diagram showing an example of the direction setting table. As shown in Fig. 18, the direction setting table is data indicating the correspondence between the combination of the user ID and the viewing location and the first direction candidates.

[0124] For example, in the direction setting table, the combination of a user ID of "user A" and a viewing location of "living room" is associated with "west" as the first direction candidate. In addition, in the direction setting table, the combination of a user ID of "user B" and a viewing location of "bedroom" is associated with "east" as the first direction candidate. In addition, in the direction setting table, the combination of a user ID of "user C" and a viewing location of "bedroom" is associated with "ceiling" as the first direction candidate.

[0125] In step S23, processor 47 determines, as first direction D1, a first direction candidate that corresponds to a combination of the user ID of user 100 and the viewing location, from among the first direction candidates included in the direction setting table. For example, as shown in Fig. 17, when "user A" is acquired as the user ID of user 100 in step S21 and "living room" is determined as the viewing location in step S22, processor 47 determines "west" from among the first direction candidates included in the direction setting table, as first direction D1.

[0126] As described above, in step S23, the processor 47 executes determining the first direction D1 based on the information indicating the viewing location, which is the second information. That is, the display method of the third embodiment includes determining the first direction D1 based on the information indicating the viewing location, which is the second information. Determining the first direction D1 includes determining the first direction D1 based on a combination of the user ID and the viewing location, which is the first information.

[0127] 16, after executing step S23, the processor 47 controls the driving device 30 so that the projection direction Dp of the projector 10B is oriented to the first direction D1 (step S24). More specifically, in step S24, the processor 47 controls the yaw angle θ of the first driving device 31 and the pitch angle α of the second driving device 32 so that the projection direction Dp of the projector 10B is oriented to the first direction D1.

[0128] Fig. 19 is a diagram showing a state in which the projection direction Dp of the projector 10B is directed to the first direction D1. For example, as described with reference to Fig. 17, when the processor 47 determines "west" as the first direction D1 in a state in which the projection direction Dp of the projector 10B faces east, the processor 47 controls the drive device 30 so that the projection direction Dp of the projector 10B faces from east to west. As a result, as shown in Fig. 19, the projector 10B rotates 180 degrees counterclockwise around the yaw axis Y, so that the projection direction Dp of the projector 10B faces west.

[0129] 16, after executing step S24 above, processor 47 determines the projection content based on the combination of the user ID of user 100 and the viewing location (step S25). Specifically, in step S25, processor 47 determines the projection content based on the combination of the user ID of user 100 and the viewing location, and the second content selection table shown in FIG. 13, which is pre-stored in memory 46.

[0130] In step S25, processor 47 first selects a video genre corresponding to the combination of user ID of user 100 and viewing location from the video genres included in the second content selection table. For example, if "user A" is acquired as the user ID of user 100 in step S21 and "living room" is determined as the viewing location in step S22, processor 47 selects "movie" from the video genres included in the second content selection table. After selecting the video genre, processor 47 determines the projection content by performing the same process as the process described in the first embodiment.

[0131] As described above, in step S25, processor 47 executes determining the projection content, which is the type of image, based on a combination of the user ID, which is the first information, and the information indicating the viewing location, which is the second information. That is, the display method of the third embodiment includes determining the projection content, which is the type of image, based on a combination of the user ID, which is the first information, and the information indicating the viewing location, which is the second information. Also, determining the projection content, which is the type of image, includes determining the projection content, which is the type of image, based on a combination of the user ID, which is the first information, and the viewing location.

[0132] 16, after executing step S25, the processor 47 controls the optical device 41 to project image light L representing an image of the projection content (step S26). The process of step S26 in the third display process is the same as the process of step S6 in the first display process, and therefore a description of step S26 will be omitted.

[0133] Fig. 20 is a diagram showing a state in which image light L is projected in a first direction D1. For example, as shown in Fig. 20, when the projection direction Dp of projector 10B is directed to the west, which is an example of the first direction D1, and processor 47 controls optical device 41 to project image light L representing the projection content, image light L is projected from projector 10B toward the west. As a result, the video content determined as the projection content is displayed on western wall surface 240, which is located in front of user 100, among the walls of room 200.

[0134] As described above, in steps S23 and S26, the processor 47 controls the optical device 41 and the drive device 30 to project the image light L representing the image of the projection content, which is the determined type of image, in the first direction D1. That is, the display method of the third embodiment includes projecting the image light L representing the image of the projection content, which is the determined type of image, in the first direction D1 by the projector 10B.

[0135] (Effects of the third embodiment) As described above, the display method of the third embodiment includes obtaining a user ID, which is first information for identifying user 100, based on the output of microphone 48, which is a first sensor; obtaining information indicating a viewing location, which is second information regarding the position of user 100, based on the output of second camera 50, which is a second sensor; determining a first direction D1 based on the second information, which is the information indicating the viewing location; determining projection content, which is a type of image, based on a combination of the first information and the second information; and projecting image light L representing an image of the projection content, which is the determined type of image, in the first direction D1 by projector 10B. As in the first and second embodiments, in the display method of the third embodiment, projection content is determined based on first information identifying user 100 and second information related to the position of user 100, and image light L representing an image of the projection content is projected in a first direction D1 determined based on the second information. This increases the likelihood that an image of video content suited to the personal attributes and position of the user 100 will be displayed on a projection surface that is easily visible to the user 100. Therefore, according to the display method of the third embodiment, it is possible to improve the convenience for the user 100 when using the projector 10B with a variable projection direction Dp.

[0136] In the display method of the third embodiment, the second information includes information indicating the viewing location, which is the location where the user 100 is located, and determining the first direction D1 includes determining the first direction D1 based on a combination of the first information, which is the user ID, and the viewing location. In this way, by determining the first direction D1 based on a combination of the first information, the user ID, and the viewing location, the image of the projection content can be displayed on a projection surface that is easily visible to the user 100, depending on the personal attributes and viewing location of the user 100, thereby improving convenience for the user 100.

[0137] In the display method of the third embodiment, determining the projection content, which is the type of image, includes determining the projection content, which is the type of image, based on a combination of the user ID, which is the first information, and the viewing location. In the first embodiment, the projection content is determined based on a combination of the user ID, which is the first information, and the sound source direction Ds. Since the sound source direction Ds is the direction in which the user 100 is located relative to the second sensor, it is difficult to determine the location in which the user 100 is located, i.e., the room or the like that is the viewing location, from the sound source direction Ds. Therefore, as described above, by determining the projection content based on a combination of the first information, the user ID, and the viewing location, it is possible to display on the projection surface an image of video content that is suitable for the personal attributes and viewing location of the user 100, thereby improving the convenience for the user 100 compared to the first embodiment.

[0138] In the display method of the third embodiment, the second sensor is a second camera 50 that captures images in a 360-degree direction, and acquiring the second information includes acquiring a third image including an area included in a 360-degree range centered on the second camera 50 based on the output of the second camera, and determining a viewing location based on the third image. In this way, by determining the viewing location based on a third image obtained from the second camera 50 capturing images in a 360-degree direction, second information regarding the position of the user 100 can be accurately obtained using a single camera.

[0139] In the display method of the third embodiment, the first sensor has at least one microphone 48, and acquiring the first information includes acquiring voiceprint data of the user 100 based on the output of the at least one microphone 48, and acquiring a user ID, which is identification information indicating the user 100, as the first information based on the voiceprint data. In this way, by acquiring voiceprint data, which is biometric data unique to user 100, based on the output of at least one microphone 48, it is possible to accurately obtain the user ID of user 100 as the first information.

[0140] The display system 3 of the third embodiment includes a microphone 48 which is a first sensor, a second camera 50 which is a second sensor, an optical device 41 which projects image light L, a driving device 30 which directs the direction in which the image light L is projected to a predetermined direction, and a processor 47. The processor 47 executes the following operations: acquiring a user ID which is first information for identifying a user 100 based on the output of the first sensor; acquiring information indicating a viewing location which is second information related to the position of the user 100 based on the output of the second sensor; determining a first direction D1 based on the second information; determining projection content which is a type of image based on a combination of the first information and the second information; and projecting image light L which represents an image of the projection content which is the determined type of image in the first direction D1 by controlling the optical device 41 and the driving device 30. In the display system 3 of the third embodiment, the projection content is determined based on first information that identifies the user 100 and second information regarding the position of the user 100, which is information indicating the viewing location, and image light L representing an image of the projection content is projected in a first direction D1 determined based on the second information, which is information indicating the viewing location. This increases the likelihood that an image of video content suited to the personal attributes and position of the user 100 will be displayed on a projection surface that is easily visible to the user 100. Therefore, according to the display system 3 of the third embodiment, it is possible to improve the convenience for the user 100 when using the projector 10B with a variable projection direction Dp.

[0141] Although the embodiments of the present disclosure have been described above, the technical scope of the present disclosure is not limited to the above embodiments, and various modifications can be made without departing from the spirit of the present disclosure.

[0142] For example, in the above first and second embodiments, a form has been exemplified in which the sound source direction Ds and the user direction Du, which are the second direction in which the user 100 is located, and the first direction D1 are expressed by four directions, east, west, north, south, etc. The present disclosure is not limited to this, and each direction may be expressed by coordinates defined by the yaw angle θ and the pitch angle α.

[0143] For example, in the above-described first embodiment, a form has been exemplified in which the projection direction Dp, which is the direction in which the image light L is projected from the projector 10, is directed to the first direction D1 by changing the orientation of the projector 10 using the drive device 30 externally attached to the projector 10. The present disclosure is not limited to this, and a drive device that directs the direction in which the image light is projected from the optical device to a predetermined direction may be provided inside the housing of the projector. This drive device may be a device that mechanically changes the orientation of the optical device, or may be a device that changes the traveling direction of the image light using an optical element such as a mirror. In this way, a projector including a first sensor, a second sensor, an optical device, a drive device, and a processor can be said to be a display system of the present disclosure.

[0144] Summary of the Disclosure The following is a summary of this disclosure.

[0145] (Supplementary Note 1) A display method including: acquiring first information identifying a user based on an output of a first sensor; acquiring second information regarding a position of the user based on the output of a second sensor; determining a first direction based on the second information; determining a type of image based on a combination of the first information and the second information; and projecting image light representing the type of image in the first direction by a projector. The type of image desired by a user may differ depending on the user's location as well as personal attributes such as the user's age, sex, occupation, and hobbies. In the display method of Supplementary Note 1, the type of image is determined based on first information for identifying the user and second information related to the user's location, and image light representing the determined type of image is projected in a first direction determined based on the second information. This increases the likelihood that an image of a type suited to the user's personal attributes and position will be displayed on a projection surface that is easy for the user to view. Therefore, according to the display method of Supplementary Note 1, it is possible to improve user convenience when using a projector with a variable projection direction.

[0146] (Appendix 2) The display method described in Appendix 1, wherein the second information includes information indicating a second direction in which the user is located relative to the second sensor, and determining the first direction includes determining a direction opposite to the second direction as the first direction. In this way, by determining the opposite direction of the second direction in which the user is located relative to the second sensor as the first direction, the determined type of image can be displayed on a projection surface located in front of the user, i.e., a projection surface that is easily visible to the user, thereby improving user convenience.

[0147] (Supplementary Note 3) The display method according to Supplementary Note 2, wherein determining the type includes determining the type based on a combination of the first information and the second direction. In this way, by determining the type of image based on a combination of the first information and the second direction, an image type suitable for the user's personal attributes and the second direction in which the user is located can be displayed on a projection surface located in front of the user, thereby improving convenience for the user.

[0148] (Appendix 4) The display method described in Appendix 2 or 3, wherein the second sensor has a plurality of microphones, and acquiring the second information includes calculating a time difference between the time when a sound wave reaches each of the plurality of microphones from a sound source based on outputs of the plurality of microphones, and determining the second direction based on the time difference. In this way, by calculating the time difference between the time it takes for a sound wave to reach each of the multiple microphones from the sound source based on the output of the multiple microphones, the direction in which the sound source is located relative to the second sensor can be accurately obtained as the second direction in which the user is located.

[0149] (Appendix 5) The display method described in Appendix 2, wherein the second information further includes information indicating a location where the user is located, and determining the type includes determining the type based on a combination of the first information and the location. In the display method of Supplementary Note 3, the type of image is determined based on a combination of the first information and the second direction. Since the second direction is the direction in which the user is located relative to the second sensor, it is difficult to determine the location of the user from the second direction. Therefore, as in the display method of Appendix 5, by determining the type of image based on a combination of the first information and the location where the user is located, it is possible to display on the projection surface an image type that is suitable for the user's personal attributes and location, thereby improving user convenience compared to the display method of Appendix 3.

[0150] (Appendix 6) The display method described in Appendix 5, wherein the second sensor is a first camera that captures an image in one direction, and acquiring the second information includes acquiring a first image including the user based on an output of the first camera, determining the second direction based on the first image, acquiring a second image including an area located in the first direction based on the output of the first camera, and determining the location based on the second image. In this way, by determining both the second direction in which the user is located and the location in which the user is located based on the first image and the second image obtained from the first camera, second information regarding the user's location can be accurately obtained using a single camera.

[0151] (Appendix 7) The display method described in Appendix 1, wherein the second information includes information indicating a location where the user is located, and determining the first direction includes determining the first direction based on a combination of the first information and the location. In this way, by determining the first direction based on a combination of the first information and the location where the user is located, the determined type of image can be displayed on a projection surface that is easily visible to the user, depending on the user's personal attributes and location, thereby improving user convenience.

[0152] (Supplementary Note 8) The display method according to Supplementary Note 7, wherein determining the type includes determining the type based on a combination of the first information and the location. In the display method of Supplementary Note 3, the type of image is determined based on a combination of the first information and the second direction. Since the second direction is the direction in which the user is located relative to the second sensor, it is difficult to determine the location of the user from the second direction. Therefore, as in the display method of Appendix 8, by determining the type of image based on a combination of the first information and the location where the user is located, it is possible to display on the projection surface an image type that is suitable for the user's personal attributes and location, thereby improving user convenience compared to the display method of Appendix 3.

[0153] (Appendix 9) The display method described in Appendix 7 or 8, wherein the second sensor is a second camera that captures images in a 360-degree direction, and acquiring the second information includes acquiring a third image including an area included in the 360-degree range centered on the second camera based on an output of the second camera, and determining the location based on the third image. In this way, by determining where the user is located based on a third image obtained from a second camera capturing images in a 360-degree direction, second information about the user's location can be accurately obtained using a single camera.

[0154] (Appendix 10) The display method described in any one of Appendices 1 to 9, wherein the first sensor has at least one microphone, and acquiring the first information includes acquiring voiceprint data of the user based on output of the at least one microphone, and acquiring identification information indicating the user as the first information based on the voiceprint data. In this way, by acquiring voiceprint data, which is biometric data unique to a user, based on the output of at least one microphone, it is possible to accurately obtain identification information indicative of the user as the first information.

[0155] (Supplementary Note 11) A display system comprising a first sensor, a second sensor, an optical device that projects image light, a drive device that directs the direction in which the image light is projected in a predetermined direction, and a processor, wherein the processor executes the following: acquiring first information identifying a user based on an output of the first sensor; acquiring second information regarding a position of the user based on an output of the second sensor; determining a first direction based on the second information; determining a type of image based on a combination of the first information and the second information; and projecting image light representing the type of image in the first direction by controlling the optical device and the drive device. In the display system of Appendix 11, a type of image is determined based on first information identifying a user and second information regarding the user's position, and image light representing the determined type of image is projected in a first direction determined based on the second information. This increases the likelihood that an image suitable for the user's personal attributes and position will be displayed on a projection surface that is easy for the user to view. Therefore, the display system of Appendix 11 can improve user convenience when using a projector with a variable projection direction. [Explanation of symbols]

[0156] 1, 2, 3...Display system, 10, 10A, 10B...Projector, 20...Base, 30...Driver, 41...Optical device, 44...Microphone array (first sensor, second sensor), 47...Processor, 48...Microphone (first sensor), 49...First camera (second sensor), 50...Second camera (second sensor), 100...User, L...Image light

Claims

1. Obtaining first information identifying a user based on an output of a first sensor; obtaining second information about a location of the user based on an output of a second sensor; and determining a first direction based on the second information; and determining a type of image based on a combination of the first information and the second information; projecting image light representing the type of image in the first direction by a projector; Including, how to display.

2. the second information includes information indicative of a second direction in which the user is located relative to the second sensor; determining the first direction includes determining a direction opposite to the second direction as the first direction. The display method according to claim 1 .

3. determining the type includes determining the type based on a combination of the first information and the second direction. The display method according to claim 2.

4. the second sensor includes a plurality of microphones; Obtaining the second information includes: Calculating a time difference between the time when a sound wave arrives at each of the plurality of microphones from a sound source based on outputs of the plurality of microphones; determining the second direction based on the time difference; and Including, The display method according to claim 2 or 3.

5. the second information further includes information indicating a location where the user is located; determining the type includes determining the type based on a combination of the first information and the location. The display method according to claim 2.

6. the second sensor is a first camera that captures an image in one direction; Obtaining the second information includes: acquiring a first image including the user based on an output of the first camera; determining the second direction based on the first image; and acquiring a second image including an area located in the first direction based on an output of the first camera; determining the location based on the second image; and Including, The display method according to claim 5.

7. the second information includes information indicating a location where the user is located; determining the first direction includes determining the first direction based on a combination of the first information and the location. The display method according to claim 1 .

8. determining the type includes determining the type based on a combination of the first information and the location. The display method according to claim 7.

9. the second sensor is a second camera that captures images in a 360-degree direction; Obtaining the second information includes: acquiring a third image including an area included in the 360-degree range centered on the second camera based on an output of the second camera; determining the location based on the third image; and Including, The display method according to claim 7 or 8.

10. the first sensor includes at least one microphone; Obtaining the first information includes: obtaining voiceprint data of the user based on an output of the at least one microphone; acquiring, as the first information, identification information indicative of the user based on the voiceprint data; Including, The display method according to claim 1 .

11. A first sensor; A second sensor; An optical device that projects image light; a driving device for directing the projection direction of the image light in a predetermined direction; A processor; Equipped with The processor, obtaining first information identifying a user based on an output of the first sensor; obtaining second information regarding a location of the user based on an output of the second sensor; and determining a first direction based on the second information; and determining a type of image based on a combination of the first information and the second information; projecting image light representing the type of image in the first direction by controlling the optical device and the driving device; Execute Display system.

Images