Display method and display system

Abstract

To improve convenience for a user when a projector variable in a projection direction is used.SOLUTION: A display method comprises: acquiring first information for identifying a user on the basis of an output from a first sensor; acquiring second information on a position of the user on the basis of an output from a second sensor; determining a first direction on the basis of the second information; determining a type of an image on the basis of a combination of the first information and the second information; and projecting image light indicating an image of the type in the first direction by a projector.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a display method and a display system.

Background Art

[0002] Patent Document 1 discloses a technique for adjusting the projection position and projection orientation of image content based on environment information indicating the usage environment of a projector and user information regarding the user who uses the projector.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] For example, the method of determining the projection position and content of image content can affect the convenience of the user, but that method is not described in Patent Document 1. Therefore, with the technology of Patent Document 1, it is difficult to improve the convenience of the user when using a projector whose projection direction is variable.

Means for Solving the Problems

[0005] A display method according to one aspect of the present invention includes: acquiring first information for identifying a user based on an output of a first sensor; acquiring second information regarding the position of the user based on an output of a second sensor; determining a first direction based on the second information; determining a type of an image based on a combination of the first information and the second information; and projecting image light representing the type of image in the first direction by a projector.

[0006] A display system in one aspect of the present invention comprises a first sensor, a second sensor, an optical device for projecting image light, a drive device for directing the direction in which the image light is projected in a predetermined direction, and a processor, wherein the processor performs the following: acquiring first information for identifying a user based on the output of the first sensor; acquiring second information for the user's position based on the output of the second sensor; determining a first direction based on the second information; determining a type of image based on a combination of the first and second information; and projecting image light representing the type of image in the first direction by controlling the optical device and the drive device. [Brief explanation of the drawing]

[0007] [Figure 1] This is a perspective view of the display system of the first embodiment. [Figure 2] This is a side view of the display system according to the first embodiment. [Figure 3] This block diagram shows the functional configuration of the display system according to the first embodiment. [Figure 4] This flowchart shows the process performed by the processor of the first embodiment. [Figure 5] This is a plan view showing an example of a display system installation. [Figure 6] This diagram shows how the projection direction is directed towards the first direction. [Figure 7] This figure shows an example of the first content selection table. [Figure 8] This diagram shows how image light is projected in the first direction. [Figure 9] This block diagram shows the functional configuration of the display system according to the second embodiment. [Figure 10] This flowchart shows the process performed by the processor of the second embodiment. [Figure 11] This is a plan view showing an example of a display system installation. [Figure 12]It is a diagram showing a state where the projection direction is directed to the first direction. [Figure 13] It is a diagram showing an example of a second content selection table. [Figure 14] It is a diagram showing a state where image light is projected in the first direction. [Figure 15] It is a block diagram showing the functional configuration of the display system of the third embodiment. [Figure 16] It is a flowchart showing the processing executed by the processor of the third embodiment. [Figure 17] It is a plan view showing an installation example of the display system. [Figure 18] It is a diagram showing an example of a direction setting table. [Figure 19] It is a diagram showing a state where the projection direction is directed to the first direction. [Figure 20] It is a diagram showing a state where image light is projected in the first direction.

Mode for Carrying Out the Invention

[0008] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. In each of the following drawings, in order to make each component easy to see, the scale of the dimensions may be shown differently depending on the component.

[0009] 〔First Embodiment〕 First, the first embodiment of the present disclosure will be described. FIG. 1 is a perspective view schematically showing the appearance of the display system 1 of the present embodiment. FIG. 2 is a side view of the display system 1 shown in FIG. 1. As shown in FIGS. 1 and 2, the display system 1 includes a projector 10, a pedestal 20, and a drive device 30.

[0010] The projector 10 projects the image light L onto a projection surface (not shown), thereby displaying an image on the projection surface. The projection surface may be a dedicated projector screen or a wall surface, etc. As an example, the projector 10 is a hexahedron having six planes. On the front surface 11 of the projector 10, a light emitting unit 12 that emits the image light L generated inside the projector 10 is provided. In the following description, the direction in which the image light L is projected from the projector 10 is referred to as the "projection direction Dp". For example, the projection direction Dp is a direction orthogonal to the front surface 11 and away from the projector 10.

[0011] The pedestal 20 is a support member for installing the projector 10 at a predetermined location. The pedestal 20 supports the projector 10 and the drive device 30. As an example, the pedestal 20 has a disk shape. The drive device 30 is disposed on the upper surface 21 of the pedestal 20, and the pedestal 20 is connected to the lower surface 13 of the projector 10 via the drive device 30.

[0012] The drive device 30 directs the direction in which the image light L is projected, that is, the projection direction Dp, in a predetermined direction. Specifically, as shown in FIG. 2, the drive device 30 rotates the projector 10 around at least one of the yaw axis Y and the pitch axis X to direct the projection direction Dp in a predetermined direction. The drive device 30 includes a first drive device 31 and a second drive device 32.

[0013] The first drive device 31 rotates the projector 10 around the yaw axis Y. As an example, the first drive device 31 has a cylindrical shape with a smaller diameter than the pedestal 20. The first drive device 31 is disposed on the upper surface 21 of the pedestal 20 with its central axis coinciding with the central axis of the pedestal 20. The central axis of the first drive device 31 is the yaw axis Y. When the first drive device 31 rotates around the yaw axis Y, the projector 10 also rotates around the yaw axis Y.

[0014] The second drive unit 32 rotates the projector 10 around the pitch axis X. For example, the second drive unit 32 has a semi-cylindrical shape. Of the surfaces of the second drive unit 32, the surface extending radially is connected to the lower surface 13 of the projector 10. The second drive unit 32 is positioned on the upper end surface of the first drive unit 31 with its central axis perpendicular to the yaw axis Y. The central axis of the second drive unit 32 is the pitch axis X. As the second drive unit 32 rotates around the pitch axis X, the projector 10 also rotates around the pitch axis X.

[0015] The operation of the drive unit 30 configured as described above is controlled by the processor 47, which will be described later. Specifically, the rotational movement of the first drive unit 31 around the yaw axis Y and the rotational movement of the second drive unit 32 around the pitch axis X are controlled by the processor 47. More specifically, the yaw angle θ, which is the rotation angle of the first drive unit 31 around the yaw axis Y, and the pitch angle α, which is the rotation angle of the second drive unit 32 around the pitch axis X, are controlled by the processor 47, thereby directing the projection direction Dp in a predetermined direction. Thus, in the display system 1 of this embodiment, the projection direction Dp of the projector 10 is variable.

[0016] Figure 3 is a schematic block diagram showing the functional configuration of the display system 1. As shown in Figure 3, the display system 1 includes the projector 10 and the drive unit 30 as functional components. Furthermore, the projector 10 includes an optical device 41, an input device 42, a communication device 43, a microphone array 44, a speaker 45, a memory 46, and a processor 47.

[0017] The optical device 41, controlled by the processor 47, generates image light L representing a color image and projects the generated image light L in the projection direction Dp. The optical device 41 includes a first image generation panel 41a, a second image generation panel 41b, a third image generation panel 41c, a dichroic prism 41d, and a projection optical system 41e.

[0018] The first image generation panel 41a generates red image light LR representing a red image and emits it to the dichroic prism 41d. The first image generation panel 41a has multiple pixels arranged in a matrix, and each of the multiple pixels emits red light. The amount of red light emitted from each pixel is controlled by the processor 47, so that red image light LR is emitted from the first image generation panel 41a.

[0019] The second image generation panel 41b generates green image light LG representing a green image and emits it to the dichroic prism 41d. The second image generation panel 41b has multiple pixels arranged in a matrix, and each of the multiple pixels emits green light. The amount of green light emitted from each pixel is controlled by the processor 47, so that the green image light LG is emitted from the second image generation panel 41b.

[0020] The third image generation panel 41c generates blue image light LB representing a blue image and emits it to the dichroic prism 41d. The third image generation panel 41c has multiple pixels arranged in a matrix, and each of the multiple pixels emits blue light. The blue image light LB is emitted from the third image generation panel 41c by controlling the amount of blue light emitted from each pixel by the processor 47.

[0021] For example, each image generation panel 41a, 41b, and 41c is a self-emissive electro-optical device such as an OLED (Organic Light Emitting Diode) panel or a μLED (Micro Light Emitting Diode) panel. Alternatively, each image generation panel 41a, 41b, and 41c may be a liquid crystal panel or a non-self-emissive electro-optical device such as a DMD (Digital Micromirror Device). If each image generation panel 41a, 41b, and 41c is a non-self-emissive electro-optical device, light from a light source (not shown), such as an LED, is separated into red light, green light, and blue light, respectively. Red light is incident on the first image generation panel 41a. Green light is incident on the second image generation panel 41b. Blue light is incident on the third image generation panel 41c. Alternatively, a single-panel image generation panel may be used to emit light of each color in a time-division manner.

[0022] The dichroic prism 41d generates image light L representing a color image by combining red image light LR, green image light LG, and blue image light LB, and emits it to the projection optical system 41e. The projection optical system 41e is composed of multiple optical elements such as lenses, and magnifies and projects the image light L emitted from the dichroic prism 41d toward the projection direction Dp. Although not shown in the diagram, the projection optical system 41e is equipped with a mechanism that can adjust optical parameters such as lens shift amount, lens focus amount, and lens zoom amount. These mechanisms are controlled by the processor 47, thereby adjusting the optical parameters of the projection optical system 41e.

[0023] The input device 42 is a device that receives user input operations to the projector 10. For example, the input device 42 includes an operation unit 42a and a light receiving unit 42b. The operation unit 42a consists of a plurality of operation keys provided on the projector 10. For example, the operation keys include a power key, a menu call key, directional keys, a select key, and a volume adjustment key. The operation keys may be hardware keys or software keys displayed on a touch panel provided on the projector 10. The operation unit 42a outputs the electrical signals generated when each operation key is operated by the user as operation signals to the processor 47.

[0024] The light-receiving unit 42b includes a photoelectric conversion circuit that receives infrared light transmitted from the projector 10's remote controller (not shown) and converts it into an electrical signal. The light-receiving unit 42b outputs the electrical signal obtained by the photoelectric conversion of infrared light to the processor 47 as a remote operation signal. The remote controller is provided with multiple operation keys, similar to the operation unit 42a. The remote controller converts the electrical signals generated when each operation key on the remote controller is operated by the user into infrared light and transmits it to the projector 10. In other words, the remote operation signal output from the light-receiving unit 42b is substantially the same as the electrical signals generated when each operation key on the remote controller is operated by the user. If the remote controller transmits radio signals according to a short-range wireless communication standard such as Bluetooth®, a receiving device for receiving radio signals may be provided instead of the light-receiving unit 42b.

[0025] The communication device 43, following instructions from the processor 47, accesses the internet via a wireless LAN (Local Area Network) that supports wireless communication standards such as Wi-Fi (registered trademark), and communicates with a video content distribution server (not shown), which is an internet server that provides video content distribution services. The communication device 43 outputs the video signal received from the video content distribution server to the processor 47.

[0026] The microphone array 44 has multiple microphones arranged at predetermined intervals. Each microphone converts the user's voice into an electrical signal. The microphone array 44 outputs the electrical signals obtained from the multiple microphones as an audio signal to the processor 47. The speaker 45, controlled by the processor 47, outputs audio at a predetermined volume.

[0027] Memory 46 includes non-volatile memory that stores programs and various setting data necessary for the processor 47 to execute various processes, and volatile memory used as a temporary storage location for data when the processor 47 executes various processes. For example, non-volatile memory may be EEPROM (Electrically Erasable Programmable Read-Only Memory) or flash memory. Volatile memory may be RAM (Random Access Memory).

[0028] The processor 47 is an arithmetic processing unit that controls the operation of the projector 10 and the drive unit 30 according to a program pre-stored in the memory 46. For example, the processor 47 is composed of one or more CPUs (Central Processing Units). Some or all of the functions of the processor 47 may be composed of circuits such as a DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), and FPGA (Field Programmable Gate Array). The processor 47 executes various processes in parallel or sequentially.

[0029] The processor 47 controls the drive unit 30, the optical device 41, and the speaker 45 based on the operation signal input from the operation unit 42a, the remote operation signal input from the light receiving unit 42b, the audio signal input from the microphone array 44, and the video signal input from the communication device 43. Details of the processing performed by the processor 47 will be described later.

[0030] Next, the operation of the display system 1 configured as described above will be explained. Figure 4 is a flowchart showing the first display process executed by the processor 47 in the first embodiment. When the processor 47 is operating in audio standby mode, waiting for an audio signal input, and an audio signal is input from the microphone array 44, it reads a program from memory 46 and executes it, thereby executing the first display process shown in Figure 4. The display method of the first embodiment is realized when the processor 47 executes the first display process.

[0031] Figure 5 is a plan view showing an example of the installation of the display system 1. In the example shown in Figure 5, the display system 1 is installed in the center of a room 200 located in the home of user 100. Room 200 has a north wall 210 located to the north (N), a south wall 220 located to the south (S), an east wall 230 located to the east (E), and a west wall 240 located to the west (W). In the following description, it is assumed that the projection direction Dp of the projector 10 is facing south, as shown in Figure 5, and that user 100 is located to the west of the display system 1, and that the processor 47 is operating in voice standby mode.

[0032] In the situation shown in Figure 5, when user 100 speaks, an audio signal is output from the microphone array 44 to the processor 47. When the processor 47 receives an audio signal from the microphone array 44 while operating in audio standby mode, it starts the first display processing shown in Figure 4.

[0033] As shown in Figure 4, when the processor 47 starts the first display process, it first performs user authentication based on the audio signal input from the microphone array 44 and user information pre-stored in the memory 46 (step S1). Step S1 includes steps S1a, S1b, S1c, and S1d.

[0034] User information refers to information about multiple users registered as legitimate users of Display System 1. Hereafter, users registered as legitimate users of Display System 1 will be referred to as "registered users." As an example, user information includes the user ID of a registered user and the registered voiceprint data associated with the user ID. The user ID is the identification information of a registered user. For example, the user ID may be the name of the registered user or an identification number individually assigned to the registered user. The registered voiceprint data is the voiceprint data of the registered user obtained by analyzing the registered user's voice in advance.

[0035] Specifically, in step S1, the processor 47 first acquires the user 100's voiceprint data by performing frequency analysis on at least one audio signal from among the audio signals output from multiple microphones included in the microphone array 44 (step S1a).

[0036] Next, the processor 47 determines whether registered voiceprint data matching user 100's voiceprint data exists in the user information (step S1b). If registered voiceprint data matching user 100's voiceprint data exists in the user information (step S1b: Yes), the processor 47 obtains the user ID from among the user IDs included in the user information that is associated with the registered voiceprint data matching user 100's voiceprint data as user ID for user 100 (step S1c).

[0037] On the other hand, if no registered voiceprint data matching the voiceprint data of user 100 exists in the user information (step S1b: No), the processor 47 controls the speaker 45 to output a sound to inform user 100 that user authentication has failed (step S1d). After executing step S1d, the processor 47 terminates the first display process and returns to the voice waiting mode.

[0038] As described above, in step S1, the processor 47 performs the action of acquiring first information that identifies user 100 based on the output of the first sensor. That is, the display method of the first embodiment includes acquiring first information that identifies user 100 based on the output of the first sensor. In the first embodiment, the user ID of user 100 corresponds to the first information, and the microphone array 44 corresponds to the first sensor.

[0039] Furthermore, in the display method of the first embodiment, the microphone array 44, which is the first sensor, has at least one microphone, and acquiring the first information includes acquiring the voiceprint data of user 100 based on the output of at least one microphone, and acquiring identification information indicating user 100 as the first information based on the voiceprint data. In the first embodiment, the user ID of user 100 corresponds to the identification information indicating user 100. Note that since voice-based user authentication technology is a generally known technology, a detailed explanation of step S1 is omitted in this embodiment.

[0040] Next, the processor 47 determines the sound source direction Ds based on the audio signal input from the microphone array 44 (step S2). The sound source direction Ds is the direction in which the sound source is located relative to the microphone array 44. Since the user 100 is the sound source, the sound source direction Ds can be rephrased as the direction in which the user 100 is located relative to the microphone array 44.

[0041] If the distance from each microphone in the microphone array 44 to the sound source is different, a time difference occurs between the time it takes for sound waves to reach each microphone from the sound source. Hereinafter, the time difference that occurs between the time it takes for sound waves to reach each microphone from the sound source will be referred to as the "sound wave arrival time difference". Techniques for determining or calculating the sound source direction Ds based on such sound wave arrival time differences are generally known as sound source localization techniques. As an example, in the first embodiment, the sound source direction Ds is determined by utilizing this sound source localization technique.

[0042] In other words, in step S2, the processor 47 calculates the sound wave arrival time difference based on the audio signals output from multiple microphones included in the microphone array 44, and the calculated sound wave The sound source direction Ds is determined based on the difference in arrival time. As described above, sound source localization technology is a generally known technology, so a detailed explanation of step S2 is omitted. For example, as shown in Figure 5, if the user 100 is located to the west of the display system 1, the processor 47 determines "west" as the sound source direction Ds. When determining various directions as bearings, an azimuth sensor (not shown) may be used, and the processor 47 may refer to information stored in memory 46 beforehand that shows the correspondence between direction and bearing relative to the projector 10.

[0043] As described above, in step S2, the processor 47 performs the action of acquiring second information regarding the user 100's position based on the output of the second sensor. That is, the display method of the first embodiment includes acquiring second information regarding the user 100's position based on the output of the second sensor. The second information includes information indicating the second direction in which the user 100 is located relative to the second sensor. In the first embodiment, the microphone array 44 corresponds to the second sensor, the sound source direction Ds, i.e., the direction in which the user 100 is located relative to the microphone array 44, corresponds to the second direction, and the information indicating the second direction corresponds to the second information. Thus, the first sensor and the second sensor may be the same sensor, or they may be different sensors.

[0044] Furthermore, in the display method of the first embodiment, the microphone array 44, which is the second sensor, has a plurality of microphones, and acquiring the second information includes calculating the time difference that occurs between the time when sound waves arrive at each of the plurality of microphones from the sound source, i.e., the sound wave arrival time difference, based on the output of the plurality of microphones, and determining the sound source direction Ds, which is the second direction, based on the time difference.

[0045] As shown in Figure 4, after performing step S2 above, the processor 47 determines a first direction D1 based on the sound source direction Ds (step S3). Specifically, in step S3, the processor 47 determines the direction opposite to the sound source direction Ds as the first direction D1. For example, as shown in Figure 5, if the processor 47 determines "west" as the sound source direction Ds, it determines "east," which is the opposite direction of the sound source direction Ds, as the first direction D1.

[0046] As described above, in step S3, the processor 47 performs the action of determining the first direction D1 based on the second information, which is information indicating the sound source direction Ds. That is, the display method of the first embodiment includes determining the first direction D1 based on the second information. Determining the first direction D1 includes determining the direction opposite to the second direction, which is the sound source direction Ds, as the first direction D1.

[0047] As shown in Figure 4, after performing step S3 above, the processor 47 controls the drive unit 30 so that the projection direction Dp of the projector 10 is directed in the first direction D1 (step S4). More specifically, in step S4, the processor 47 controls the yaw angle θ of the first drive unit 31 and the pitch angle α of the second drive unit 32 so that the projection direction Dp of the projector 10 is directed in the first direction D1.

[0048] Figure 6 shows how the projection direction Dp of the projector 10 is directed towards the first direction D1. For example, as explained with reference to Figure 5, if the processor 47 determines "east" as the first direction D1 while the projection direction Dp of the projector 10 is facing south, it controls the drive unit 30 so that the projection direction Dp of the projector 10 is directed from south to east. As a result, as shown in Figure 6, the projection direction Dp of the projector 10 is directed towards east by rotating the projector 10 90 degrees counterclockwise around the yaw axis Y.

[0049] As shown in Figure 4, after performing step S4 above, the processor 47 determines the projection content based on the combination of the user ID of user 100 and the sound source direction Ds (step S5). The projection content is the video content provided to user 100 by being projected as image light L from projector 10. The video content is, for example, a video work or broadcast program belonging to a video genre such as movies, TV dramas, animations, weather, politics, economics, and sports.

[0050] Specifically, in step S5, the processor 47 determines the projection content based on the combination of user ID and sound source direction Ds of user 100 and a first content selection table pre-stored in memory 46. Figure 7 shows an example of the first content selection table. As shown in Figure 7, the first content selection table is data that shows the correspondence between the combination of user ID and sound source direction Ds and the video genre.

[0051] For example, in the first content selection table, the combination of user ID "User A" and sound source direction Ds "West" is associated with the video genre "Movie". Also in the first content selection table, the combination of user ID "User B" and sound source direction Ds "East" is associated with the video genre "Weather". Furthermore, in the first content selection table, the combination of user ID "User C" and sound source direction Ds "South" is associated with the video genre "Animation".

[0052] In step S5, the processor 47 first selects a video genre from the video genres included in the first content selection table that corresponds to the combination of user ID and sound source direction Ds for user 100. For example, if "User A" is obtained as user ID for user 100 in step S1, and "West" is determined as sound source direction Ds in step S2, the processor 47 selects "Movie" from the video genres included in the first content selection table.

[0053] For example, after selecting a video genre, the processor 47 communicates with the video content distribution server via the communication device 43 to obtain a distribution list of video content belonging to the selected video genre from the video content distribution server. The processor 47 generates a distribution list image that includes the distribution list and a message requesting the user 100 to select video content from the distribution list. The processor 47 controls the optical device 41 so that an image light L representing the distribution list image is projected.

[0054] Figure 8 shows how the image light L is projected in a first direction D1. For example, as shown in Figure 8, when the projection direction Dp of the projector 10 is directed east, which is an example of the first direction D1, the processor 47 controls the optical device 41 so that the image light L representing the distribution list image is projected, and the image light L is projected from the projector 10 toward the east. As a result, the distribution list image is displayed on the east wall 230 of the room 200, which is located in the east, i.e., in front of the user 100.

[0055] For example, user 100 uses a remote controller to select video content from the distribution list while viewing the distribution list image displayed on the east wall 230. During the period when the distribution list image is displayed, the processor 47 determines, based on the remote operation signal input from the light receiving unit 42b, that it has received an operation to select video content from the distribution list, and then determines that the video content selected by user 100 will be projected as the content.

[0056] Alternatively, the processor 47 may determine the projected content by performing the following process. For example, if video signals of multiple video contents belonging to each video genre are pre-stored in memory 46, the processor 47 creates a saved list of video contents belonging to the selected video genre. The processor 47 generates a saved list image that includes the saved list and a message requesting the user 100 to perform an operation to select video content from the saved list. The processor 47 controls the optical device 41 so that an image light L representing the saved list image is projected.

[0057] For example, user 100 uses a remote controller to select video content from the saved list while viewing the displayed saved list image. If the processor 47 determines that it has received an operation to select video content from the saved list based on the remote operation signal input from the light receiving unit 42b during the period when the saved list image is displayed, it determines that the video content selected by user 100 will be projected as the content.

[0058] As described above, in step S5, the processor 47 performs the action of determining the type of image based on the combination of the first information and the second information. That is, the display method of the first embodiment includes determining the type of image based on the combination of the first information and the second information. In the first embodiment, the user ID of user 100 corresponds to the first information, the information indicating the sound source direction Ds corresponds to the second information, and the projected content corresponds to the type of image. That is, the type of image is video content such as video works or broadcast programs classified by video genre.

[0059] As shown in Figure 4, after executing step S5 above, the processor 47 controls the optical device 41 so that an image light L representing the image of the projection content is projected (step S6). Specifically, in step S6, the processor 47 receives the video signal of the video content determined as the projection content from the video content distribution server via the communication device 43, or reads it from the memory 46. The processor 47 then controls the optical device 41 so that an image light L representing an image based on the image data contained in the video signal is projected, and controls the speaker 45 so that sound based on the audio data contained in the video signal is output.

[0060] For example, as shown in Figure 8, when the projection direction Dp of the projector 10 is directed eastward, which is an example of the first direction D1, the processor 47 controls the optical device 41 so that image light L representing the image of the projection content is projected, and the image light L is projected eastward from the projector 10. As a result, the image of the video content determined as the projection content is displayed on the east wall 230 of the room 200, which is located in front of the user 100.

[0061] As described above, in steps S4 and S6, the processor 47 controls the optical device 41 and the drive device 30 to project an image light L representing the image of the projection content, which is an image of the determined type, in a first direction D1. That is, the display method of the first embodiment includes projecting an image light L representing the determined type of image in a first direction D1 by the projector 10.

[0062] (Effects of the first embodiment) As described above, the display method of the first embodiment includes: acquiring a user ID, which is first information that identifies the user 100, based on the output of a microphone array 44, which is a first sensor; acquiring information indicating the sound source direction Ds, which is second information that relates to the location of the user 100, based on the output of a microphone array 44, which is a second sensor; determining a first direction D1 based on the second information; determining a projection content, which is the type of image, based on a combination of the first information and the second information; and projecting an image light L representing the image of the projection content, which is the determined type of image, in the first direction D1 using a projector 10. The video content desired by user 100 may vary depending not only on user 100's personal attributes such as age, gender, occupation, and hobbies, but also on user 100's location. In the display method of the first embodiment, the projected content is determined based on first information identifying user 100 and second information regarding user 100's location, and an image light L representing the image of the projected content is projected in a first direction D1 determined based on the second information. This increases the likelihood that images of video content suitable for the user 100's personal attributes and location will be displayed on a projection surface that is easily visible to the user 100. Therefore, according to the display method of the first embodiment, the convenience of the user 100 can be improved when a projector 10 with a variable projection direction Dp is used.

[0063] In the display method of the first embodiment, the second information includes information indicating the sound source direction Ds, which is the second direction in which the user 100 is positioned relative to the microphone array 44, which is the second sensor, and determining the first direction D1 includes determining the direction opposite to the sound source direction Ds, which is the second direction, as the first direction D1. In this way, by determining the direction opposite to the second direction in which user 100 is positioned relative to the second sensor as the first direction D1, the image of the projected content can be displayed on the projection surface located in front of user 100, that is, on the projection surface that is easily visible to user 100, thereby improving user convenience.

[0064] In the display method of the first embodiment, determining the projected content, which is the type of image, includes determining the projected content, which is the type of image, based on a combination of a first piece of information, a user ID, and a second direction, a sound source direction Ds. In this way, by determining the projection content based on the combination of the first piece of information, the user ID, and the second piece of direction, the sound source direction Ds, it is possible to display an image of video content suitable for the combination of the user 100's personal attributes and the second direction in which the user 100 is located, on the projection surface located in front of the user 100, thereby improving the user 100's convenience.

[0065] In the display method of the first embodiment, the microphone array 44, which is a second sensor, has a plurality of microphones, and acquiring the second information includes calculating the time difference that occurs between the time when sound waves arrive at each of the plurality of microphones from the sound source, based on the output of the plurality of microphones, and determining the sound source direction Ds, which is the second direction, based on the time difference. In this way, by calculating the time difference between the time when sound waves arrive at each of the multiple microphones from the sound source, based on the output of the multiple microphones, the sound source direction Ds, which is the direction in which the sound source is located relative to the second sensor, can be accurately obtained as the second direction in which the user 100 is located.

[0066] In the display method of the first embodiment, the microphone array 44, which is the first sensor, has at least one microphone, and acquiring first information includes acquiring the voiceprint data of user 100 based on the output of at least one microphone, and acquiring a user ID, which is identification information indicating user 100, as first information based on the voiceprint data. In this way, by acquiring voiceprint data, which is biometric data unique to user 100, based on the output of at least one microphone, the user ID of user 100 can be accurately obtained as first information. Furthermore, as in the first embodiment, since the first sensor and the second sensor are the same sensor, i.e., the microphone array 44, both the first information and the second information can be acquired based on the output of this single sensor.

[0067] The display system 1 of the first embodiment includes a microphone array 44 which is a first sensor, a microphone array 44 which is a second sensor, an optical device 41 which projects image light L, a drive device 30 which directs the direction in which the image light L is projected to a predetermined direction, and a processor 47. The processor 47 performs the following: acquires a user ID which is first information that identifies the user 100 based on the output of the first sensor; acquires information indicating the sound source direction Ds which is second information that relates to the position of the user 100 based on the output of the second sensor; determines the first direction D1 based on the second information; determines the projection content which is the type of image based on the combination of the first information and the second information; and projects image light L representing the image of the projection content which is the determined type of image in the first direction D1 by controlling the optical device 41 and the drive device 30. In the display system 1 of the first embodiment, the projection content is determined based on first information that identifies the user 100 and second information that relates to the location of the user 100, and an image light L representing the image of the projection content is projected in a first direction D1 determined based on the second information. This increases the likelihood that images of video content suitable for the user 100's personal attributes and location will be displayed on a projection surface that is easily visible to the user 100. Therefore, according to the display system 1 of the first embodiment, the convenience of the user 100 can be improved when a projector 10 with a variable projection direction Dp is used.

[0068] [Second Embodiment] The following describes a second embodiment of this disclosure. In each of the embodiments described below, components common to the first embodiment are denoted by the same reference numerals used in the first embodiment, and detailed descriptions are omitted as appropriate. Figure 9 is a schematic block diagram showing the functional configuration of the display system 2 of the second embodiment. As shown in Figure 9, the display system 2 includes a projector 10A and a drive unit 30 as functional components. The projector 10A of the second embodiment is identical to the projector 10 of the first embodiment in that it includes an optical device 41, an input device 42, a communication device 43, a speaker 45, a memory 46, and a processor 47.

[0069] Projector 10A differs from Projector 10 in that it includes a microphone 48 as the first sensor instead of the microphone array 44. Furthermore, Projector 10A differs from Projector 10 in that it includes a first camera 49 as the second sensor. The differences between the second embodiment and the first embodiment will be described in detail below.

[0070] The microphone 48 converts the user 100's voice into an electrical signal and outputs the electrical signal as an audio signal to the processor 47. The first camera 49 is a digital camera that captures images in one direction. The first camera 49 is mounted on the projector 10A so as to capture images in the projection direction Dp. In other words, the first camera 49 is mounted on the projector 10A so as to have its shooting direction coincide with the projection direction Dp. The first camera 49 outputs captured image data showing the captured image to the processor 47.

[0071] Next, we will explain the operation of the display system 2 configured as described above. Figure 10 is a flowchart showing the second display process executed by the processor 47 in the second embodiment. When the processor 47 receives an audio signal from the microphone 48 while operating in audio standby mode, it reads a program from the memory 46 and executes it, thereby executing the second display process shown in Figure 10. The display method of the second embodiment is realized when the processor 47 executes the second display process.

[0072] Figure 11 is a plan view showing an example of the installation of the display system 2. In the example shown in Figure 11, the display system 2 is installed in the center of a room 200 in the home of user 100, similar to the example shown in Figure 5. In the following description, we assume that the projection direction Dp of the projector 10A is facing east, and that user 100 is positioned to the east of the display system 2, and that the processor 47 is operating in audio standby mode. In this case, the first camera 49 photographs the east, which is the same direction as the projection direction Dp.

[0073] In the situation shown in Figure 11, when user 100 speaks, an audio signal is output from microphone 48 to processor 47. When processor 47 receives an audio signal from microphone 48 while operating in audio standby mode, it starts the second display process shown in Figure 10.

[0074] As shown in Figure 10, when the processor 47 starts the second display process, it first performs user authentication based on the audio signal input from the microphone 48 and the user information pre-stored in the memory 46 (step S11). Step S11 includes steps S11a, S11b, S11c, and S11d. Since the processing of step S11 in the second display process is substantially the same as the processing of step S1 in the first display process, the processing of step S11 will be briefly described below.

[0075] In step S11, the processor 47 first obtains the voiceprint data of user 100 by performing frequency analysis on the audio signal output from the microphone 48 (step S11a). The processor 47 determines whether or not registered voiceprint data matching the voiceprint data of user 100 exists in the user information (step S11b). If registered voiceprint data matching the voiceprint data of user 100 exists in the user information (step S11b: Yes), the processor 47 obtains the user ID from the user information that is associated with the registered voiceprint data matching the voiceprint data of user 100 as the user ID of user 100 (step S11c).

[0076] On the other hand, if no registered voiceprint data matching the voiceprint data of user 100 exists in the user information (step S11b: No), the processor 47 controls the speaker 45 to output a sound to inform user 100 that user authentication has failed (step S11d). After executing step S11d, the processor 47 terminates the second display process and returns to the voice waiting mode.

[0077] As described above, in step S11, the processor 47 performs the action of obtaining first information that identifies the user 100 based on the output of the first sensor. That is, the display method of the second embodiment includes obtaining first information that identifies the user 100 based on the output of the first sensor. In the second embodiment, the user ID corresponds to the first information and the microphone 48 corresponds to the first sensor.

[0078] Furthermore, in the display method of the second embodiment, the first sensor has at least one microphone 48, and acquiring first information includes acquiring voiceprint data of user 100 based on the output of at least one microphone 48, and acquiring a user ID, which is identification information indicating user 100, as first information based on the voiceprint data.

[0079] Next, the processor 47 determines the user direction Du based on the captured image data input from the first camera 49 (step S12). The user direction Du is the direction in which the user 100 is located relative to the first camera 49. For example, as shown in Figure 11, when the user 100 is located on the east side of the display system 2 and the first camera 49 captures an image of the east, which is the projection direction Dp, captured image data showing the captured image with the user 100 in it is output from the first camera 49 to the processor 47.

[0080] In the second embodiment, the user information includes the registered user's user ID and registered voiceprint data, as well as face recognition information linked to the user ID. The face recognition information is information that identifies the face of the registered user. In step S12, the processor 47 obtains the face recognition information linked to the user ID of user 100 from the face recognition information included in the user information. Then, the processor 47 performs image analysis of the captured image data based on the captured image data and the face recognition information to determine the image region corresponding to the face of user 100 from the image regions included in the captured image as the user image region.

[0081] In the second embodiment, a coordinate transformation formula is pre-stored in memory 46 to transform the coordinates of the captured image in the image coordinate system to coordinates in a coordinate system representing the movable range of the projection direction Dp. The coordinate system representing the movable range of the projection direction Dp is a coordinate system defined by the yaw angle θ and the pitch angle α. In step S12, the processor 47 transforms the center coordinates of the user image area in the image coordinate system into coordinates defined by the yaw angle θ and the pitch angle α based on this coordinate transformation formula. The coordinates obtained by such a coordinate transformation represent the direction in which the user 100 is located relative to the first camera 49. That is, the processor 47 acquires the direction corresponding to the coordinates obtained by the coordinate transformation as the user direction Du. For example, the processor 47 refers to a table pre-stored in memory 46 that shows the correspondence between the direction and the coordinate system representing the movable range of the projection direction Dp, and acquires the direction corresponding to the coordinate as the user direction Du. As shown in Figure 11, if the user 100 is located on the east side of the display system 2, the processor 47 acquires "east" corresponding to the coordinate as the user direction Du. The processor 47 may also obtain the coordinates themselves in the coordinate system representing the range of motion of the projection direction Dp as the user direction Du.

[0082] As described above, in step S12, the processor 47 performs the action of acquiring second information regarding the user 100's position based on the output of the second sensor. That is, the display method of the second embodiment includes acquiring second information regarding the user 100's position based on the output of the second sensor. The second information includes information indicating the second direction in which the user 100 is located relative to the second sensor. In the second embodiment, the first camera 49 corresponds to the second sensor, the user direction Du, i.e., the direction in which the user 100 is located relative to the first camera 49, corresponds to the second direction, and the information indicating the second direction corresponds to the second information.

[0083] Furthermore, in the display method of the second embodiment, acquiring the second information includes acquiring a first image including the user 100 based on the output of the first camera 49, and determining the user direction Du, which is the second direction, based on the first image. As described above, among the captured images obtained from the first camera 49, the captured image in which the user 100 is captured corresponds to the first image.

[0084] If the captured image obtained at the start of step S12 does not contain a user image area, it is assumed that the user 100 is not located in the projection direction Dp, which is the shooting direction of the first camera 49. Therefore, if the captured image obtained at the start of step S12 does not contain a user image area, the processor 47 controls the drive unit 30 so that the projector 10A rotates 1 full turn around the yaw axis Y, and acquires captured image data from the first camera 49 at predetermined time intervals. Each time the processor 47 acquires captured image data, it determines whether or not a user image area exists in the captured image. If the processor 47 determines that a user image area exists in the captured image, it stops controlling the drive unit 30 and determines the user direction Du by performing the above coordinate transformation.

[0085] As shown in Figure 10, after performing step S12 above, the processor 47 determines a first direction D1 based on the user direction Du (step S13). Specifically, in step S13, the processor 47 determines the opposite direction of the user direction Du as the first direction D1. For example, as shown in Figure 11, if the processor 47 determines "east" as the user direction Du, it determines "west," which is the opposite direction of the user direction Du, as the first direction D1.

[0086] As described above, in step S13, the processor 47 performs the action of determining the first direction D1 based on the second information, which is information indicating the user direction Du. That is, the display method of the second embodiment includes determining the first direction D1 based on the second information, which is information indicating the user direction Du. Determining the first direction D1 includes determining the direction opposite to the second direction, which is the user direction Du, as the first direction D1.

[0087] As shown in Figure 10, after performing step S13 above, the processor 47 controls the drive unit 30 so that the projection direction Dp of the projector 10A is directed in the first direction D1 (step S14). More specifically, in step S14, the processor 47 controls the yaw angle θ of the first drive unit 31 and the pitch angle α of the second drive unit 32 so that the projection direction Dp of the projector 10A is directed in the first direction D1. As a result, the shooting direction of the first camera 49 is also directed in the first direction D1.

[0088] Figure 12 shows how the projection direction Dp of the projector 10A is directed towards the first direction D1. For example, as explained with reference to Figure 11, if the processor 47 determines "west" as the first direction D1 while the projection direction Dp of the projector 10A is facing east, it controls the drive unit 30 so that the projection direction Dp of the projector 10A is directed from east to west. As a result, as shown in Figure 12, the projection direction Dp of the projector 10A is directed west by rotating the projector 10A 180 degrees counterclockwise around the yaw axis Y. In this case, the shooting direction of the first camera 49 is also directed west.

[0089] As shown in Figure 10, after performing step S14 above, the processor 47 determines the viewing location based on the captured image data input from the first camera 49 (step S15). The viewing location is the location where the user 100 is located.

[0090] For example, as shown in Figure 12, when the projection direction Dp is directed towards the first direction D1, which is "west", and the first camera 49 photographs the first direction D1, the first camera 49 outputs image data to the processor 47 showing the captured image that includes the area within the room 200 located in the first direction D1, that is, the western area including the west wall surface 240.

[0091] In step S15, the processor 47 determines one of several viewing location candidates as the viewing location based on the captured image data. For example, the multiple viewing location candidates include rooms such as a living room, bedroom, and bathroom. The memory 46 has pre-stored reference image data obtained by capturing the interior of the room that is a viewing location candidate using the first camera 49.

[0092] Specifically, in step S15, the processor 47 calculates the similarity between the captured image shown by the captured image data and the reference image shown by the reference image data using image processing such as pattern matching. The reference image is, as described above, an image obtained by capturing the interior of a room that is a candidate viewing location using the first camera 49. Then, the processor 47 determines the candidate viewing location from among the multiple candidate viewing locations that has obtained the reference image with the highest similarity to the captured image as the viewing location.

[0093] As described above, in the display method of the second embodiment, the second information relating to the position of the user 100 further includes information indicating the user direction Du, which is the second direction in which the user 100 is located, as well as information indicating the viewing location, which is the place where the user 100 is located. Furthermore, in the display method of the second embodiment, acquiring the second information includes acquiring a second image including a region located in the first direction D1 based on the output of the first camera 49, and determining the viewing location based on the second image. As described above, among the captured images obtained from the first camera 49, the captured image obtained by capturing the first direction D1 corresponds to the second image.

[0094] As shown in Figure 10, after executing step S15 described above, the processor 47 determines the projected content based on the combination of user ID and viewing location of user 100 (step S16). Specifically, in step S16, the processor 47 determines the projected content based on the combination of user ID and viewing location of user 100 and a second content selection table pre-stored in memory 46. Figure 13 shows an example of the second content selection table. As shown in Figure 13, the second content selection table is data that shows the correspondence between the combination of user ID and viewing location and the video genre.

[0095] For example, in the second content selection table, the combination of user ID "User A" and viewing location "Living Room" is associated with the video genre "Movie." Also in the second content selection table, the combination of user ID "User B" and viewing location "Bedroom" is associated with the video genre "Cosmetics." Furthermore, in the second content selection table, the combination of user ID "User C" and viewing location "Bathroom" is associated with the video genre "Animation."

[0096] In step S16, the processor 47 first selects a video genre from the video genres included in the second content selection table that corresponds to the combination of user ID and viewing location of user 100. For example, if "User A" is obtained as the user ID of user 100 in step S11, and "Living Room" is determined as the viewing location in step S15, the processor 47 selects "Movie" from the video genres included in the second content selection table. After selecting the video genre, the processor 47 determines the projected content by performing the same processing as described in the first embodiment.

[0097] As described above, in step S16, the processor 47 determines the projected content, which is the type of image, based on a combination of first information, the user ID, and second information, information indicating the viewing location. That is, the display method of the second embodiment includes determining the projected content, which is the type of image, based on a combination of first information, the user ID, and second information, information indicating the viewing location. The second information includes information indicating the viewing location, which is the location where the user is located, and determining the projected content, which is the type of image, includes determining the projected content, which is the type of image, based on a combination of first information, the user ID, and the viewing location.

[0098] As shown in Figure 10, after executing step S16 above, the processor 47 controls the optical device 41 so that an image light L representing the image of the projected content is projected (step S17). The process in step S17 in the second display process is the same as the process in step S6 in the first display process, so the explanation of step S17 is omitted.

[0099] Figure 14 shows how the image light L is projected in a first direction D1. For example, as shown in Figure 14, when the projection direction Dp of the projector 10A is directed west, which is an example of the first direction D1, the processor 47 controls the optical device 41 so that the image light L representing the projection content is projected, and the image light L is projected westward from the projector 10A. As a result, the image of the video content determined as the projection content is displayed on the west wall 240 of the room 200, which is located in front of the user 100.

[0100] As described above, in steps S14 and S17, the processor 47 controls the optical device 41 and the drive device 30 to project an image light L representing the image of the projection content, which is an image of the determined type, in the first direction D1. That is, the display method of the second embodiment includes projecting an image light L representing the image of the projection content, which is an image of the determined type, in the first direction D1 using the projector 10A.

[0101] (Effects of the second embodiment) As described above, the display method of the second embodiment includes: acquiring a user ID, which is first information that identifies the user 100, based on the output of a microphone 48, which is a first sensor; acquiring second information that relates to the location of the user 100, namely the user direction Du and information indicating the viewing location, based on the output of a first camera 49, which is a second sensor; determining a first direction D1 based on the information indicating the user direction Du, which is the second information; determining a projected content, which is the type of image, based on a combination of the first information and the information indicating the viewing location, which is the second information; and projecting an image light L representing the image of the projected content, which is the determined type of image, in the first direction D1 using a projector 10A. Similar to the first embodiment, in the display method of the second embodiment, the projected content is determined based on first information identifying the user 100 and second information relating to the location of the user 100, and an image light L representing the image of the projected content is projected in a first direction D1 determined based on the second information. This increases the likelihood that images of video content suitable for the user 100's personal attributes and location will be displayed on a projection surface that is easily visible to the user 100. Therefore, according to the display method of the second embodiment, the convenience of the user 100 can be improved when a projector 10A with a variable projection direction Dp is used.

[0102] In the display method of the second embodiment, the second information includes information indicating the user direction Du, which is the second direction in which the user 100 is located relative to the first camera 49, which is the second sensor, and determining the first direction D1 includes determining the direction opposite to the second direction, the user direction Du, as the first direction D1. In this way, by determining the direction opposite to the second direction in which user 100 is positioned relative to the second sensor as the first direction D1, the image of the projected content can be displayed on the projection surface located in front of user 100, that is, on the projection surface that is easily visible to user 100, thereby improving user convenience.

[0103] In the display method of the second embodiment, the second information further includes information indicating the viewing location, which is the location where user 100 is located, and determining the projected content, which is the type of image, includes determining the projected content, which is the type of image, based on the combination of the first information, which is the user ID and the viewing location. In the first embodiment, the projected content is determined based on a combination of first information, namely the user ID and the sound source direction Ds. Since the sound source direction Ds is merely the direction in which the user 100 is located relative to the second sensor, it is difficult to determine the location where the user 100 is located, i.e., the room or other viewing location, from the sound source direction Ds. Therefore, as described above, by determining the projected content based on the combination of the first piece of information, the user ID, and the viewing location, it is possible to display an image of video content suitable for the user 100's personal attributes and viewing location on the projection surface, thus improving the user 100's convenience compared to the first embodiment.

[0104] In the display method of the second embodiment, the second sensor is a first camera 49 that captures images in one direction, and acquiring second information includes acquiring a first image including the user 100 based on the output of the first camera 49, determining a second direction, which is the user direction Du, based on the first image, acquiring a second image including a region located in the first direction D1 based on the output of the first camera 49, and determining a viewing location based on the second image. In this way, by determining both the user direction Du and the viewing location based on the first and second images obtained from the first camera 49, it is possible to accurately obtain second information about the user 100's location using a single camera.

[0105] In the display method of the second embodiment, the first sensor has at least one microphone 48, and acquiring first information includes acquiring voiceprint data of user 100 based on the output of at least one microphone 48, and acquiring a user ID, which is identification information indicating user 100, as first information based on the voiceprint data. In this way, by acquiring voiceprint data, which is biometric data unique to user 100, based on the output of at least one microphone 48, the user ID of user 100 can be accurately obtained as the first piece of information.

[0106] The display system 2 of the second embodiment includes a microphone 48 which is a first sensor, a first camera 49 which is a second sensor, an optical device 41 which projects image light L, a drive device 30 which directs the direction from which the image light L is projected to a predetermined direction, and a processor 47. The processor 47 performs the following: acquire a user ID which is first information that identifies the user 100 based on the output of the first sensor; acquire second information which is the user direction Du and information indicating the viewing location which is second information about the location of the user 100 based on the output of the second sensor; determine a first direction D1 based on the information indicating the user direction Du which is second information; determine the projection content which is the type of image based on the combination of the first information and the information indicating the viewing location which is second information; and project image light L representing the image of the projection content which is the determined type of image into the first direction D1 by controlling the optical device 41 and the drive device 30. In the display system 2 of the second embodiment, the projected content is determined based on first information that identifies the user 100 and second information that indicates the viewing location, which is the location of the user 100. An image light L representing the image of the projected content is projected in a first direction D1 determined based on second information that indicates the user direction Du. This increases the likelihood that images of video content suitable for the user 100's personal attributes and location will be displayed on a projection surface that is easily visible to the user 100. Therefore, according to the display system 2 of the second embodiment, the convenience of the user 100 can be improved when a projector 10A with a variable projection direction Dp is used.

[0107] [Third Embodiment] A third embodiment of this disclosure will be described below. Figure 15 is a schematic block diagram showing the functional configuration of the display system 3 of the third embodiment. As shown in Figure 15, the display system 3 includes a projector 10B and a drive unit 30 as functional components. The projector 10B of the third embodiment is identical to the projector 10A of the second embodiment in that it includes an optical device 41, an input device 42, a communication device 43, a speaker 45, a memory 46, a processor 47, and a microphone 48.

[0108] Projector 10B differs from Projector 10A in that it is equipped with a second camera 50 as a second sensor instead of the first camera 49. The differences between the third embodiment and the second embodiment will be described in detail below.

[0109] The second camera 50 is a digital camera that captures images in a 360-degree direction centered on the second camera 50. For example, the second camera 50 is mounted on the top surface of the housing of the projector 10B. The second camera 50 outputs captured image data, which represents the captured image, to the processor 47.

[0110] Next, we will explain the operation of the display system 3 configured as described above. Figure 16 is a flowchart showing the third display process executed by the processor 47 in the third embodiment. When the processor 47 receives an audio signal from the microphone 48 while operating in audio standby mode, it reads a program from memory 46 and executes it, thereby executing the third display process shown in Figure 16. The display method of the third embodiment is realized when the processor 47 executes the third display process.

[0111] Figure 17 is a plan view showing an example of the installation of the display system 3. In the example shown in Figure 17, the display system 3 is installed in the center of a room 200 in the home of user 100, similar to the example shown in Figure 5. In the following description, it is assumed that the projection direction Dp of the projector 10B is facing east, and that user 100 is positioned to the east of the display system 3, and that the processor 47 is operating in voice standby mode. The second camera 50 captures images in a 360-degree direction centered on the second camera 50.

[0112] In the situation shown in Figure 17, when user 100 speaks, an audio signal is output from microphone 48 to processor 47. When processor 47 receives an audio signal from microphone 48 while operating in audio standby mode, it starts the third display process shown in Figure 16.

[0113] As shown in Figure 16, when the processor 47 starts the third display process, it first performs user authentication based on the audio signal input from the microphone 48 and the user information pre-stored in the memory 46 (step S21). Step S21 includes steps S21a, S21b, S21c, and S21d. Since the processing of step S21 in the third display process is the same as the processing of step S11 in the second display process, the processing of step S21 will be briefly described below.

[0114] In step S21, the processor 47 first obtains the voiceprint data of user 100 by performing frequency analysis on the audio signal output from the microphone 48 (step S21a). The processor 47 determines whether or not registered voiceprint data matching the voiceprint data of user 100 exists in the user information (step S21b). If registered voiceprint data matching the voiceprint data of user 100 exists in the user information (step S21b: Yes), the processor 47 obtains the user ID from the user information that is associated with the registered voiceprint data matching the voiceprint data of user 100 as the user ID of user 100 (step S21c).

[0115] On the other hand, if the processor 47 does not have registered voiceprint data in the user information that matches the voiceprint data of user 100 (step S21b: No), it controls the speaker 45 to output a sound to inform user 100 that user authentication has failed (step S21d). After executing step S21d, the processor 47 terminates the third display process and returns to the voice waiting mode.

[0116] As described above, in step S21, the processor 47 performs the action of acquiring first information that identifies the user 100 based on the output of the first sensor. That is, the display method of the third embodiment includes acquiring first information that identifies the user 100 based on the output of the first sensor. In the third embodiment, the user ID corresponds to the first information, and the microphone 48 corresponds to the first sensor.

[0117] Next, the processor 47 determines the viewing location based on the captured image data input from the second camera 50 (step S22). Similar to the second embodiment, the viewing location is the location where the user 100 is located.

[0118] For example, as shown in Figure 17, with the display system 3 installed in the center of the room 200, when the second camera 50 captures images in a 360-degree direction centered on the second camera 50, the second camera 50 outputs captured image data to the processor 47 that shows the captured image of the area within the room 200 that is included in the 360-degree range centered on the second camera 50.

[0119] In step S22, the processor 47 determines one of several viewing location candidates as the viewing location based on the captured image data. For example, the multiple viewing location candidates include rooms such as a living room, bedroom, and bathroom. The memory 46 has pre-stored reference image data obtained by capturing the interior of the room that is a viewing location candidate using the second camera 50.

[0120] Specifically, in step S22, the processor 47 calculates the similarity between the captured image shown by the captured image data and the reference image shown by the reference image data using image processing such as pattern matching. The reference image is, as described above, an image obtained by taking a picture of the interior of a room that is a candidate viewing location using the second camera 50. Then, the processor 47 determines the candidate viewing location from among the multiple candidate viewing locations that has obtained the captured image with the highest similarity to the reference image as the viewing location.

[0121] As described above, in step S22, the processor 47 performs the action of acquiring second information regarding the location of the user 100 based on the output of the second sensor. That is, the display method of the third embodiment includes acquiring second information regarding the location of the user 100 based on the output of the second sensor. The second information includes information indicating the viewing location, which is the location where the user 100 is located. In the third embodiment, a second camera 50 that captures a 360-degree view corresponds to the second sensor.

[0122] Furthermore, in the display method of the third embodiment, acquiring the second information includes acquiring a third image, which includes an area within a 360-degree range centered on the second camera 50, based on the output of the second camera 50, and determining the viewing location based on the third image. As described above, among the captured images obtained from the second camera 50, the captured image that includes an area within a 360-degree range centered on the second camera 50 corresponds to the third image.

[0123] As shown in Figure 16, after executing step S22 above, the processor 47 determines a first direction D1 based on the combination of user ID and viewing location of user 100 (step S23). Specifically, in step S23, the processor 47 determines the first direction D1 based on the combination of user ID and viewing location of user 100 and a direction setting table pre-stored in memory 46. Figure 18 is a diagram showing an example of a direction setting table. As shown in Figure 18, the direction setting table is data that shows the correspondence between the combination of user ID and viewing location and the first direction candidate.

[0124] For example, in the direction setting table, "West" is associated as the first direction candidate for the combination of user ID "User A" and viewing location "Living Room". Also in the direction setting table, "East" is associated as the first direction candidate for the combination of user ID "User B" and viewing location "Bedroom". Furthermore, in the direction setting table, "Ceiling" is associated as the first direction candidate for the combination of user ID "User C" and viewing location "Bedroom".

[0125] In step S23, the processor 47 determines the first direction candidate from among the first direction candidates included in the direction setting table that corresponds to the combination of user ID of user 100 and viewing location as the first direction D1. For example, as shown in Figure 17, if "User A" is obtained as the user ID of user 100 in step S21 and "Living Room" is determined as the viewing location in step S22, the processor 47 determines "West" as the first direction D1 from among the first direction candidates included in the direction setting table.

[0126] As described above, in step S23, the processor 47 performs the action of determining a first direction D1 based on the second information, which is information indicating the viewing location. That is, the display method of the third embodiment includes determining a first direction D1 based on the second information, which is information indicating the viewing location. Determining a first direction D1 includes determining a first direction D1 based on a combination of the first information, which is a user ID and a viewing location.

[0127] As shown in Figure 16, after performing step S23 above, the processor 47 controls the drive unit 30 so that the projection direction Dp of the projector 10B is directed in the first direction D1 (step S24). More specifically, in step S24, the processor 47 controls the yaw angle θ of the first drive unit 31 and the pitch angle α of the second drive unit 32 so that the projection direction Dp of the projector 10B is directed in the first direction D1.

[0128] Figure 19 shows how the projection direction Dp of the projector 10B is directed towards the first direction D1. For example, as explained with reference to Figure 17, if the processor 47 determines "west" as the first direction D1 while the projection direction Dp of the projector 10B is facing east, it controls the drive unit 30 so that the projection direction Dp of the projector 10B is directed from east to west. As a result, as shown in Figure 19, the projection direction Dp of the projector 10B is directed towards west by rotating the projector 10B 180 degrees counterclockwise around the yaw axis Y.

[0129] As shown in Figure 16, after executing step S24 above, the processor 47 determines the projected content based on the combination of user ID and viewing location of user 100 (step S25). Specifically, in step S25, the processor 47 determines the projected content based on the combination of user ID and viewing location of user 100 and the second content selection table shown in Figure 13, which is pre-stored in memory 46.

[0130] In step S25, the processor 47 first selects a video genre from the video genres included in the second content selection table that corresponds to the combination of user ID and viewing location of user 100. For example, if "User A" is obtained as user ID for user 100 in step S21, and "Living Room" is determined as the viewing location in step S22, the processor 47 selects "Movie" from the video genres included in the second content selection table. After selecting the video genre, the processor 47 determines the projected content by performing the same processing as described in the first embodiment.

[0131] As described above, in step S25, the processor 47 determines the projected content, which is the type of image, based on a combination of first information, the user ID, and second information, the viewing location. That is, the display method of the third embodiment includes determining the projected content, which is the type of image, based on a combination of first information, the user ID, and second information, the viewing location. Furthermore, determining the projected content, which is the type of image, includes determining the projected content, which is the type of image, based on a combination of first information, the user ID, and the viewing location.

[0132] As shown in Figure 16, after executing step S25 described above, the processor 47 controls the optical device 41 so that an image light L representing the image of the projected content is projected (step S26). The process in step S26 in the third display process is the same as the process in step S6 in the first display process, so the explanation of step S26 is omitted.

[0133] Figure 20 shows how the image light L is projected in a first direction D1. For example, as shown in Figure 20, when the projection direction Dp of the projector 10B is directed west, which is an example of the first direction D1, the processor 47 controls the optical device 41 so that the image light L representing the projection content is projected, and the image light L is projected westward from the projector 10B. As a result, the video content determined to be the projection content is displayed on the west wall 240 of the room 200, which is located in front of the user 100.

[0134] As described above, in steps S23 and S26, the processor 47 controls the optical device 41 and the drive device 30 to project an image light L representing the image of the projection content, which is of the determined type of image, in the first direction D1. That is, the display method of the third embodiment includes projecting an image light L representing the image of the projection content, which is of the determined type of image, in the first direction D1 using the projector 10B.

[0135] (Effects of the third embodiment) As described above, the display method of the third embodiment includes: acquiring a user ID, which is first information that identifies the user 100, based on the output of a microphone 48, which is a first sensor; acquiring information indicating the viewing location, which is second information regarding the location of the user 100, based on the output of a second camera 50, which is a second sensor; determining a first direction D1 based on the information indicating the viewing location, which is the second information; determining the projection content, which is the type of image, based on a combination of the first information and the second information; and projecting an image light L representing the image of the projection content, which is the determined type of image, in the first direction D1 using a projector 10B. Similar to the first and second embodiments, in the display method of the third embodiment, the projection content is determined based on first information identifying the user 100 and second information relating to the location of the user 100, and an image light L representing the image of the projection content is projected in a first direction D1 determined based on the second information. This increases the likelihood that images of video content suitable for the user 100's personal attributes and location will be displayed on a projection surface that is easily visible to the user 100. Therefore, according to the display method of the third embodiment, the convenience of the user 100 can be improved when a projector 10B with a variable projection direction Dp is used.

[0136] In the display method of the third embodiment, the second information includes information indicating the viewing location, which is the location where user 100 is located, and determining the first direction D1 includes determining the first direction D1 based on a combination of the first information, which is the user ID and the viewing location. In this way, by determining the first direction D1 based on the combination of the first piece of information, the user ID, and the viewing location, the image of the projected content can be displayed on a projection surface that is easily visible to user 100, according to user 100's personal attributes and viewing location, thereby improving user convenience.

[0137] In the display method of the third embodiment, determining the projected content, which is the type of image, includes determining the projected content, which is the type of image, based on a combination of the first information, which is the user ID and the viewing location. In the first embodiment, the projected content is determined based on a combination of first information, namely the user ID and the sound source direction Ds. Since the sound source direction Ds is merely the direction in which the user 100 is located relative to the second sensor, it is difficult to determine the location where the user 100 is located, i.e., the room or other viewing location, from the sound source direction Ds. Therefore, as described above, by determining the projected content based on the combination of the first piece of information, the user ID, and the viewing location, it is possible to display an image of video content suitable for the user 100's personal attributes and viewing location on the projection surface, thus improving the user 100's convenience compared to the first embodiment.

[0138] In the display method of the third embodiment, the second sensor is a second camera 50 that captures images in a 360-degree direction, and acquiring the second information includes acquiring a third image that includes an area within a 360-degree range centered on the second camera 50, based on the output of the second camera, and determining the viewing location based on the third image. In this way, by determining the viewing location based on a third image obtained from a second camera 50 that captures images in a 360-degree direction, it is possible to accurately obtain second information about the user 100's location using a single camera.

[0139] In the display method of the third embodiment, the first sensor has at least one microphone 48, and acquiring first information includes acquiring voiceprint data of user 100 based on the output of at least one microphone 48, and acquiring a user ID, which is identification information indicating user 100, as first information based on the voiceprint data. In this way, by acquiring voiceprint data, which is biometric data unique to user 100, based on the output of at least one microphone 48, the user ID of user 100 can be accurately obtained as the first piece of information.

[0140] The display system 3 of the third embodiment includes a microphone 48 which is a first sensor, a second camera 50 which is a second sensor, an optical device 41 which projects image light L, a drive device 30 which directs the direction from which the image light L is projected to a predetermined direction, and a processor 47. The processor 47 performs the following: acquire a user ID which is first information that identifies the user 100 based on the output of the first sensor; acquire information indicating the viewing location which is second information that relates to the location of the user 100 based on the output of the second sensor; determine a first direction D1 based on the second information; determine the projection content which is the type of image based on the combination of the first information and the second information; and project image light L representing the image of the projection content which is the determined type of image in the first direction D1 by controlling the optical device 41 and the drive device 30. In the display system 3 of the third embodiment, the projected content is determined based on first information that identifies the user 100 and second information that indicates the viewing location, which is the location of the user 100. An image light L representing the image of the projected content is projected in a first direction D1 determined based on the second information that indicates the viewing location. This increases the likelihood that images of video content suitable for the user 100's personal attributes and location will be displayed on a projection surface that is easily visible to the user 100. Therefore, according to the display system 3 of the third embodiment, the convenience of the user 100 can be improved when a projector 10B with a variable projection direction Dp is used.

[0141] While embodiments of this disclosure have been described above, the technical scope of this disclosure is not limited to the embodiments described above, and various modifications can be made without departing from the spirit of this disclosure.

[0142] For example, in the first and second embodiments described above, the sound source direction Ds and user direction Du, which are the second directions in which the user 100 is located, and the first direction D1 are exemplified as being represented by the four directions of east, west, north, and south. The disclosure is not limited thereto, and each direction may be represented by coordinates defined by the yaw angle θ and the pitch angle α.

[0143] For example, in the first embodiment described above, a drive device 30 attached externally to the projector 10 is used to change the orientation of the projector 10, thereby aligning the projection direction Dp, which is the direction in which image light L is projected from the projector 10, to a first direction D1. The disclosure is not limited thereto, and a drive device that directs the direction in which image light is projected from the optical device to a predetermined direction may be provided inside the housing of the projector. This drive device may be a device that mechanically changes the orientation of the optical device, or it may be a device that changes the direction of propagation of image light using an optical element such as a mirror. Thus, a projector comprising a first sensor, a second sensor, an optical device, a drive device, and a processor can be said to be a display system of the disclosure.

[0144] [Summary of this disclosure] A summary of this disclosure is provided below.

[0145] (Note 1) A display method comprising: acquiring first information for identifying a user based on the output of a first sensor; acquiring second information for the user's location based on the output of a second sensor; determining a first direction based on the second information; determining an image type based on a combination of the first information and the second information; and projecting image light representing the type of image in the first direction using a projector. The type of image a user desires may vary depending not only on the user's personal attributes such as age, gender, occupation, and hobbies, but also on the user's location. In the display method described in Appendix 1, the type of image is determined based on first information that identifies the user and second information about the user's location, and image light representing the determined type of image is projected in a first direction determined based on the second information. This increases the likelihood that images appropriate to the user's personal attributes and location will be displayed on a projection surface that is easily visible to the user. Therefore, according to the display method described in Appendix 1, user convenience can be improved when using a projector with a variable projection direction.

[0146] (Note 2) The display method according to Note 1, wherein the second information includes information indicating the second direction in which the user is located relative to the second sensor, and determining the first direction includes determining the opposite direction of the second direction as the first direction. In this way, by determining the direction opposite to the second direction in which the user is positioned relative to the second sensor as the first direction, the determined type of image can be displayed on the projection surface located in front of the user, i.e., the projection surface that is easily visible to the user, thus improving user convenience.

[0147] (Note 3) The display method described in Note 2, wherein determining the type includes determining the type based on a combination of the first information and the second direction. In this way, by determining the type of image based on the combination of the first piece of information and the second direction, it is possible to display an image suitable for the user's personal attributes and the second direction in which the user is located on the projection surface facing the user, thereby improving user convenience.

[0148] (Note 4) The display method according to Note 2 or 3, wherein the second sensor has a plurality of microphones, and acquiring the second information includes calculating the time difference between the times when sound waves travel from the sound source to each of the plurality of microphones based on the output of the plurality of microphones, and determining the second direction based on the time difference. In this way, by calculating the time difference that occurs when sound waves travel from the sound source to each of the multiple microphones, based on the output of the multiple microphones, the direction in which the sound source is located relative to the second sensor can be accurately obtained as the second direction in which the user is located.

[0149] (Note 5) The display method according to Note 2, wherein the second information further includes information indicating the location where the user is located, and determining the type includes determining the type based on a combination of the first information and the location. In the display method described in Appendix 3, the type of image is determined based on a combination of the first piece of information and the second direction. Since the second direction is simply the direction in which the user is positioned relative to the second sensor, it is difficult to determine the user's location from the second direction alone. Therefore, as shown in the display method in Appendix 5, by determining the type of image based on the combination of the first information and the user's location, it is possible to display an image suitable for the user's personal attributes and location on the projection surface, thus improving user convenience compared to the display method in Appendix 3.

[0150] (Note 6) The display method according to Note 5, wherein the second sensor is a first camera that captures images in one direction, and acquiring the second information includes acquiring a first image including the user based on the output of the first camera, determining the second direction based on the first image, acquiring a second image including an area located in the first direction based on the output of the first camera, and determining the location based on the second image. In this way, by determining both the second direction in which the user is located and the location where the user is located based on the first and second images obtained from the first camera, it is possible to accurately obtain second information about the user's location using a single camera.

[0151] (Note 7) The display method according to Note 1, wherein the second information includes information indicating the location of the user, and determining the first direction includes determining the first direction based on a combination of the first information and the location. In this way, by determining a first direction based on a combination of first information and the user's location, it is possible to display a determined type of image on a projection surface that is easily visible to the user, according to the user's personal attributes and location, thereby improving user convenience.

[0152] (Note 8) The method of display described in Note 7, wherein determining the type includes determining the type based on a combination of the first information and the location. In the display method described in Appendix 3, the type of image is determined based on a combination of the first piece of information and the second direction. Since the second direction is simply the direction in which the user is positioned relative to the second sensor, it is difficult to determine the user's location from the second direction alone. Therefore, as shown in the display method in Appendix 8, by determining the type of image based on the combination of the first information and the location where the user is located, it is possible to display an image suitable for the user's personal attributes and location on the projection surface, thus improving user convenience compared to the display method in Appendix 3.

[0153] (Note 9) The display method according to Note 7 or 8, wherein the second sensor is a second camera that captures images in a 360-degree direction, and acquiring the second information includes acquiring a third image based on the output of the second camera, which includes an area within the 360-degree range centered on the second camera, and determining the location based on the third image. In this way, by determining the user's location based on a third image obtained from a second camera that captures a 360-degree view, it is possible to accurately obtain second information about the user's location using only one camera.

[0154] (Note 10) The display method according to any one of Notes 1 to 9, wherein the first sensor has at least one microphone, and acquiring the first information includes acquiring the user's voiceprint data based on the output of the at least one microphone, and acquiring identification information indicating the user as the first information based on the voiceprint data. In this way, by acquiring voiceprint data, which is user-specific biometric data, based on the output of at least one microphone, it is possible to accurately obtain user identification information as the first piece of information.

[0155] (Note 11) A display system comprising: a first sensor; a second sensor; an optical device for projecting image light; a drive device for directing the direction in which the image light is projected in a predetermined direction; and a processor, wherein the processor performs the following actions: acquiring first information for identifying a user based on the output of the first sensor; acquiring second information for the user's position based on the output of the second sensor; determining a first direction based on the second information; determining a type of image based on a combination of the first information and the second information; and projecting image light representing the type of image in the first direction by controlling the optical device and the drive device. In the display system described in Appendix 11, the type of image is determined based on first information that identifies the user and second information that relates to the user's location, and image light representing the determined type of image is projected in a first direction determined based on the second information. This increases the likelihood that images appropriate to the user's personal attributes and location will be displayed on a projection surface that is easily visible to the user. Therefore, according to the display system described in Appendix 11, user convenience can be improved when using a projector with a variable projection direction. [Explanation of Symbols]

[0156] 1, 2, 3…Display system, 10, 10A, 10B…Projector, 20…Base, 30…Driver, 41…Optical device, 44…Microphone array (first sensor, second sensor), 47…Processor, 48…Microphone (first sensor), 49…First camera (second sensor), 50…Second camera (second sensor), 100…User, L…Image light

Claims

1. Obtaining first information to identify the user based on the output of a first sensor, The second information regarding the user's location is obtained based on the output of the second sensor, Based on the second piece of information mentioned above, the first direction is determined, Based on the combination of the first information and the second information, the type of image is determined, Projecting image light representing the aforementioned type of image in the first direction using a projector, Display methods, including those mentioned above.

2. The second information includes information indicating the second direction in which the user is positioned relative to the second sensor. Determining the first direction includes determining the direction opposite to the second direction as the first direction. The display method according to claim 1.

3. Determining the type includes determining the type based on a combination of the first information and the second direction. The display method according to claim 2.

4. The second sensor has multiple microphones, To obtain the information described in the second paragraph, The time difference that occurs between the time when sound waves travel from the sound source to each of the multiple microphones is calculated based on the output of the multiple microphones, Based on the aforementioned time difference, the second direction is determined, including, The display method according to claim 2 or 3.

5. The second information further includes information indicating the location of the user, Determining the type includes determining the type based on the combination of the first information and the location. The display method according to claim 2.

6. The second sensor is a first camera that captures images in one direction. To obtain the information described in the second paragraph, The first image, including the user, is acquired based on the output of the first camera. Based on the first image, the second direction is determined, A second image including the region located in the first direction is acquired based on the output of the first camera, Based on the second image, the location is determined, including, The display method according to claim 5.

7. The second information includes information indicating the location of the user, Determining the first direction includes determining the first direction based on a combination of the first information and the location. The display method according to claim 1.

8. Determining the type includes determining the type based on the combination of the first information and the location. The display method according to claim 7.

9. The second sensor is a second camera that captures images in a 360-degree direction. To obtain the information described in the second paragraph, A third image is acquired based on the output of the second camera, which includes an area within the 360-degree range centered on the second camera. Based on the third image described above, the location is determined, including, The display method according to claim 7 or 8.

10. The first sensor has at least one microphone, Obtaining the aforementioned first information is The user's voiceprint data is acquired based on the output of at least one microphone. Based on the voiceprint data, identification information indicating the user is obtained as the first information, including, The display method according to claim 1.

11. The first sensor and The second sensor, An optical device that projects image light, A drive device that directs the direction from which the image light is projected in a predetermined direction, Processor and Equipped with, The aforementioned processor, First information for identifying the user is obtained based on the output of the first sensor, The second information relating to the user's location is obtained based on the output of the second sensor, Based on the second piece of information mentioned above, the first direction is determined, Based on the combination of the first information and the second information, the type of image is determined, By controlling the optical device and the drive device, an image light representing the type of image is projected in the first direction, Execute Display system.

Images