Projection system and control method for projection system

Abstract

To allow sharing of a state where an image such as a projected image from a projection device and handwritten characters are superimposed with each other between a plurality of devices.SOLUTION: A projection device comprises: a projection unit that projects image light PL on a projection object OB; a photographing unit that photographs a range including the projection object OB to create a photographed image; a communication unit; and a control unit that causes the communication unit to transmit the photographed image to an information processing apparatus, and causes the projection unit to project a received image received by the communication unit from the information processing apparatus. The projection unit includes an image light forming unit that forms the image light PL, and a projection optical system that projects the image light formed by the image light forming unit toward the projection object. The photographing unit includes an image pickup device that receives light through the projection optical system. The control unit causes the communication unit to receive the received image including an image drawn by the information processing apparatus.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention is a projection system , and projection system This relates to a control method. [Background technology]

[0002] Conventionally, there are known techniques for drawing characters or the like by superimposing them on an image projected by a projector. For example, the device disclosed in Patent Document 1 allows a user to write characters by hand and superimpose them on an image projected on a screen. This device records the handwritten character data on a recording medium in association with the projected video data, and can then play back the video data with the handwritten characters superimposed on it. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-161748 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventionally, it has been difficult to share an image, such as an image projected by a projector, with handwritten characters or the like, overlaid on the image by a plurality of devices. [Means for solving the problem]

[0005] One aspect of the present disclosure is a projection device comprising: a projection unit that projects image light onto a projection target; a capturing unit that captures an image of an area including the projection target and generates a captured image; a communication unit; and a control unit that transmits the captured image to an information processing device via the communication unit and causes the projection unit to project a received image received from the information processing device via the communication unit, wherein the projection unit comprises an image light forming unit that forms image light and a projection optical system that projects the image light formed by the image light forming unit towards the projection target, the capturing unit comprises an image sensor that receives light through the projection optical system, and the control unit receives the received image, including an image drawn by the information processing device, via the communication unit.

[0006] Another aspect of the present disclosure is a projection device comprising a projection unit that projects image light onto a projection target, a photographing unit that photographs an area including the projection target and generates a photographed image, a communication unit that communicates with an information processing device, and a control unit, wherein the control unit transmits the photographed image to the information processing device via the communication unit, receives a drawing image from the information processing device, and controls the projection unit to project the drawing image onto the projection target and not to project the photographed image.

[0007] Another aspect of the present disclosure is a control method for a projection device comprising a projection unit that projects image light onto a projection target, a capture unit that captures an area including the projection target to generate a captured image, and a communication unit that communicates with an information processing device, wherein the communication unit transmits the captured image generated by the capture unit to the information processing device as a first image, receives a second image in the information processing device that includes a first drawing image drawn on the first image, projects the first drawing image onto the projection target by the projection unit, captures an area including the projection target onto which the first drawing image is projected to generate a captured image by the capture unit, transmits the captured image to the information processing device as a third image by the communication unit, receives a fourth image in the information processing device that includes a second drawing image drawn on the third image, and projects the first drawing image and the second drawing image onto the projection target by the projection unit. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a projection system according to a first embodiment. [Figure 2] FIG. 2 is a block diagram of each device constituting the projection system. [Figure 3] FIG. 1 is a diagram showing an example of the configuration of a main part of a projection device. [Figure 4] FIG. 4 is a sequence diagram showing the operation of the projection system. [Figure 5] Schematic diagram of the operation of the projection system. [Figure 6] FIG. 10 is a diagram showing another example of the configuration of the main parts of the projection device according to the second embodiment. [Figure 7] FIG. 10 is a diagram showing an example of the configuration of a main part of a projection device according to a third embodiment. [Figure 8] 10 is a timing chart showing the operation of the projection device according to the third embodiment. [Figure 9] FIG. 10 is a diagram showing an example of the configuration of a main part of a projection device according to a fourth embodiment. [Figure 10] FIG. 10 is an external view of a portable projection device according to a fifth embodiment. [Figure 11] FIG. 10 is a cross-sectional view of a portable projection device according to a fifth embodiment. [Figure 12] FIG. 10 is a diagram showing another example of the configuration of a projection system. DETAILED DESCRIPTION OF THE INVENTION

[0009] [1. First embodiment] [1-1. Projection system configuration] 1 is a diagram showing a schematic configuration of a projection system 1000 according to a first embodiment. The projection system 1000 includes a projection device 1 and a display device 6. The projection device 1 and the display device 6 are connected to each other via a communication network 3 so as to be able to communicate data with each other. The display device 6 corresponds to an example of an information processing device.

[0010] There is no limitation on the installation locations of the projection device 1 and the display device 6. For example, the use location S1 of the projection device 1 and the use location S2 of the display device 6 may be located far from each other or close to each other.

[0011] The communication network 3 is a network that enables data communication between devices. The communication network 3 may be, for example, a local network such as a LAN (Local Area Network) or a wide area network. The communication network 3 may also be, for example, an open network such as the Internet. The communication network 3 may include communication lines such as dedicated lines, public lines, or cellular communication lines, and communication devices such as routers and gateway devices. The projection device 1 and the communication network 3 may be connected by wire via a communication cable, or may be connected wirelessly by a wireless communication path. Similarly, the display device 6 and the communication network 3 may be connected by wire via a communication cable, or may be connected wirelessly by a wireless communication path. The communication cable may be, for example, a LAN cable or a USB cable conforming to the USB (Universal Serial Bus) communication standard. The wireless communication path may be, for example, Wi-Fi or Bluetooth. Wi-Fi is a registered trademark. Bluetooth is a registered trademark.

[0012] The projection device 1 projects image light PL toward the projection target OB, forming a projection image PP on the projection target OB. Projecting the image light PL by the projection device 1 corresponds to displaying the projection image PP on the projection target OB. In the following description, the image includes video and still images. The projection device 1 also captures an area including the projection target OB and generates a captured image.

[0013] The projection device 1 shown in FIG. 1 has a fixed part 101 that contacts an installation part 110 such as a desk. The fixed part 101 is fixed to the installation part 110 by clamps, screws, bolts, or adhesive. In the example of FIG. 1, the fixed part 101 is fixed by sandwiching an edge of the plate-shaped installation part 110. The surface of the installation part 110 is flat, and this flat surface is used as the projection target OB of the projection device 1. The fixed part 101 corresponds to an example of a base part.

[0014] The projection device 1 has an optical unit 10 including a projection unit 11 that projects image light PL and a photographing unit 15, which will be described later. With a fixed unit 101 fixed to an installation unit 110, the projection unit 11 projects the image light PL onto the upper surface of the installation unit 110. The photographing unit 15 photographs an area including a projection target OB onto which the image light PL is projected. The optical unit 10 is housed in an optical unit case 103. The optical unit case 103 is positioned above the projection target OB and projects the image light PL downward. The optical unit case 103 corresponds to an example of a projection unit case.

[0015] The optical unit case 103 is connected to the fixed part 101 by an arm 104. The arm 104 is a rod-shaped member that supports the optical unit case 103. The arm 104 has one or more hinges 105. By bending the arm 104 at the hinges 105, the position of the optical unit case 103 relative to the projection target OB base 101 can be adjusted.

[0016] The fixed part 101 is provided with an operation panel 35 having switches and the like used to operate the projection device 1.

[0017] The projection device 1 transmits a captured image of an area including the projection target OB by the imaging unit 15 to the display device 6 via the communication network 3. The projection device 1 also receives a drawing image DP from the display device 6, which will be described later, and projects the received drawing image DP onto the projection target OB.

[0018] The display device 6 is a device having a display 61. The display device 6 may be any device having the display 61, an input function, and a communication function. For example, the display device 6 is configured as a tablet computer, a laptop computer, or a smartphone. The display device 6 described in this embodiment has a function of detecting an operation of a pointer 65 on the display 61 and drawing a drawn image DP based on the operation by the pointer 65. The pointer 65 is, for example, a pen-type device as shown in FIG. 1. The display device 6 may be configured so that the user's finger can be used as the pointer 65.

[0019] 1 is an example of the configuration of the projection device 1. The projection device 1 may be installed, for example, in an orientation to project the image light PL horizontally or in an orientation to project the image light PL upward. Another example of a usage scenario for the projection device 1 is distance education. For example, a student uses the projection device 1 at a usage location S1, and a teacher uses the display device 6 at a usage location S2. For example, as shown in FIG. 1 , the projection device 1 captures an image of a student writing a figure C with a pencil on a notebook spread out on the projection target OB, and transmits the captured image to the teacher's display device 6. The teacher corrects the figure C written by the student and displayed on the display device 6, and uses a pointer 65 to draw a drawn image DP as a result of the correction. The drawn image DP is transmitted to the projection device 1. The projection device 1 receives the drawn image DP written by the teacher and projects it on the projection target OB. However, the projection device 1 does not project the image transmitted by the projection device 1 to the display device 6, i.e., the image showing the figure C written by the student with a pencil on the projection target OB. The student can easily view and share the drawn image DP written by the teacher as a result of the correction, corresponding to the figure C written by the student with a pencil.

[0020] [1-2. Configuration of Projection Device and Display Device] FIG. 2 is a block diagram of each device that constitutes the projection system 1000. The projection device 1 includes a projection unit 11 that projects image light PL and a drive circuit 14 that drives the projection unit 11. The projection unit 11 includes an image light formation unit 12 and a projection optical system 13.

[0021] The image light formation unit 12 generates the image light PL. The image light formation unit 12 includes a light emitter that emits predetermined color light. The predetermined color light is, for example, red light, blue light, and green light. The light emitter may be, for example, an LED (Light Emitting Diode) element or an OLED (Organic LED) element. The configuration of the image light formation unit 12 will be described later.

[0022] The image light forming unit 12 may include, for example, a light source having a lamp or a solid-state light source, and a light modulation device that modulates the light emitted by the light source. Examples of the lamp include a halogen lamp, a xenon lamp, and an ultra-high pressure mercury lamp. Examples of the solid-state light source include an LED and a laser light source. Examples of the light modulation device include a transmissive liquid crystal panel, a reflective liquid crystal panel, and a digital micromirror device (DMD).

[0023] The projection optical system 13 includes optical elements that guide the image light PL emitted by the image light formation unit 12 toward the projection target OB. The optical elements include a single lens or a lens group including multiple lenses. The optical elements may also include a prism and a dichroic mirror.

[0024] The drive circuit 14 is connected to an image processing unit 43, which will be described later. The drive circuit 14 generates the image light PL by driving the image light formation unit 12 based on an image signal input from the image processing unit 43. For example, the drive circuit 14 causes the image light formation unit 12 to form an image on a frame-by-frame basis.

[0025] The projection device 1 includes a photographing unit 15. The photographing unit 15 is a digital camera having an image sensor 151, as will be described later. The photographing unit 15 photographs images under the control of a PJ control unit 20, as will be described later, and outputs the generated photographed image to the PJ control unit 20. The photographing range of the photographing unit 15 includes the direction in which the projection unit 11 projects the image light PL. For example, the photographing range of the photographing unit 15 includes the projection target OB. The image sensor 151 is, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0026] The projection unit 11 and the photographing unit 15 are arranged facing a viewing surface such as a desk. Specifically, the projection unit 11 and the photographing unit 15 are housed in an optical unit case 103 and are positioned above the projection target OB. The projection unit 11 and the photographing unit 15 are collectively referred to as the optical unit 10. The configuration of the optical unit 10 will be described later with reference to FIG. 3.

[0027] The projection device 1 includes a projection device control unit 20, an operation unit 31, a remote control receiver 32, an input interface 33, a connection unit 41, a projection device communication unit 42, and an image processing unit 43. In the following description and drawings, the projection device may be abbreviated as PJ. For example, the projection device control unit 20 will be referred to as the PJ control unit 20, and the projection device communication unit 42 will be referred to as the PJ communication unit 42. The PJ control unit 20, the input interface 33, the connection unit 41, the PJ communication unit 42, and the image processing unit 43 are connected to each other via a bus 39 so that they can communicate data with each other.

[0028] The operation unit 31 detects user operations on the projection device 1. The operation unit 31 detects operations on buttons and switches provided on the operation panel 35, generates operation signals corresponding to the operations on the operation panel 35, and outputs them to the input interface 33. The input interface 33 includes a circuit that outputs the operation signals input from the operation unit 31 to the PJ control unit 20.

[0029] The remote control light receiving unit 32 has a light receiving element that receives infrared light, and receives an infrared signal transmitted from the remote control 2. When a switch (not shown) provided on the remote control 2 is operated, the remote control 2 transmits an infrared signal indicating the operation. The remote control light receiving unit 32 decodes the received infrared signal to generate an operation signal. The remote control light receiving unit 32 outputs the generated operation signal to the input interface 33. The input interface 33 has a circuit that outputs the operation signal input from the remote control light receiving unit 32 to the PJ control unit 20.

[0030] There are no limitations on the specific manner in which signals are transmitted and received between the remote control 2 and the remote control light receiving unit 32. The configuration in which the remote control 2 transmits an infrared signal to the remote control light receiving unit 32 is one example. For example, the remote control 2 and the remote control light receiving unit 32 may transmit and receive signals by performing short-range wireless communication such as Bluetooth.

[0031] The connection unit 41 is an interface device that receives images from an external device, and is connected to, for example, a player that plays back an optical disc type recording medium or a personal computer.

[0032] The PJ communication unit 42 is connected to the communication network 3 and transmits and receives images to and from the display device 6 via the communication network 3. The PJ communication unit 42 is, for example, a communication device including a connector for connecting a communication cable and a communication circuit for inputting and outputting signals via the communication cable. The PJ communication unit 42 may also be a wireless communication device. In this case, the PJ communication unit 42 includes, for example, an antenna, an RF (Radio Frequency) circuit, a baseband circuit, etc. The PJ communication unit 42 corresponds to an example of a communication unit.

[0033] The image processing unit 43 selects an image source under the control of the PJ control unit 20. Sources available to the projection device 1 are, for example, images received by the connection unit 41 and images received by the PJ communication unit .

[0034] The image processing unit 43 performs image processing on the image of the selected source under the control of the PJ control unit 20. The image processing performed by the image processing unit 43 includes, for example, resolution conversion processing, geometric correction processing, digital zoom processing, and image correction processing for adjusting the color tone and brightness of the image.

[0035] The image processing unit 43 generates an image signal based on the image after image processing and outputs it to the drive circuit 14. A frame memory (not shown) may be connected to the image processing unit 43. In this case, the image processing unit 43 loads the image acquired from the source into the frame memory. The image processing unit 43 performs image processing on the image loaded into the frame memory.

[0036] The image processing unit 43 can be configured, for example, by an integrated circuit. The integrated circuit is configured, for example, by an LSI (Large Scale Integration). More specifically, the image processing unit 43 is configured by an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or the like. The PLD includes, for example, an FPGA (Field-Programmable Gate Array). Furthermore, an analog circuit may be included as part of the configuration of the integrated circuit, or the integrated circuit may be a combination of a processor and an integrated circuit. The combination of a processor and an integrated circuit is called a microcontroller (MCU), a SoC (System-on-a-Chip), a system LSI, a chipset, or the like.

[0037] The PJ control unit 20 includes a processor 21 and a memory 25. The memory 25 is a storage device that stores programs and data executed by the processor 21 in a nonvolatile manner. The memory 25 is configured from a magnetic storage device, a semiconductor storage element such as a flash ROM (Read Only Memory), or other types of nonvolatile storage device. The memory 25 may include RAM (Random Access Memory) that configures the work area of ​​the processor 21. The memory 25 stores data processed by the processor 21 and a control program 26 executed by the processor 21. The PJ control unit 20 corresponds to an example of a control unit.

[0038] The processor 21 is configured with a CPU (Central Processing Unit), an MPU (Micro-processing unit), or the like. The processor 21 may be configured with a single processor, or multiple processors functioning as the processor 21. The processor 21 may be configured as an SoC integrated with part or all of the memory 25 and / or other circuits. As described above, the processor 21 may be configured by combining a CPU that executes programs and a DSP (Digital Signal Processor) that executes predetermined arithmetic processing. All of the functions of the processor 21 may be implemented in hardware, or may be configured using a programmable device. The processor 21 may also have the functions of the image processing unit 43. In other words, the functions of the image processing unit 43 may be executed by the processor 21.

[0039] The processor 21 controls each part of the projection device 1 by executing a control program 26 stored in the memory 25 .

[0040] The processor 21 causes the image processing unit 43 to select a source and acquire an image of the selected source by the image processing unit 43. The processor 21 controls the drive circuit 14 to cause the projection unit 11 to project the image light PL based on the image signal output by the image processing unit 43 and display the image.

[0041] The memory 25 includes a received image memory 27. The received image memory 27 is a logical or virtual storage area that is provided using a part of the storage area of ​​the memory 25.

[0042] The PJ control unit 20 stores images received from the display device 6 via the PJ communication unit 42 in the received image memory 27. Every time the PJ control unit 20 receives an image from the display device 6, it combines the received image with the image stored in the received image memory 27 to update the image stored in the received image memory 27. The PJ control unit 20 causes the projection unit 11 to project the image stored in the received image memory 27.

[0043] The processor 21 controls the image capturing unit 15 to capture an image of an area including the projection target OB. The processor 21 transmits the image captured by the image capturing unit 15 to the display device 6 via the PJ communication unit .

[0044] The projection device 1 is connected to, for example, an external DC power supply and has a power supply circuit that supplies power from the external DC power supply to each component, including the optical unit 10 and the PJ control unit 20.

[0045] The display device 6 includes a display 61, a touch sensor 62, a display communication unit 63, and a display control unit 70. In the following description and drawings, the display may be abbreviated as DP. For example, the display communication unit 63 will be referred to as the DP communication unit 63, and the display control unit 70 will be referred to as the DP control unit 70. The display device 6 includes a battery (not shown) and operates on power supplied from the battery.

[0046] The display 61 displays an image under the control of the DP control unit 70. The display 61 includes, for example, a liquid crystal display panel, an organic EL display panel, or another display panel.

[0047] The touch sensor 62 detects operations on the display panel of the display 61. The touch sensor 62 detects contact operations or pressure operations on the display 61 and outputs a signal indicating the operation position to the DP control unit 70. The touch sensor 62 is configured, for example, as a pressure-sensitive sensor, a resistive sensor, or a capacitance sensor. The touch sensor 62 may also be configured to detect operations by performing wireless communication with the indicator 65. The touch sensor 62 may be configured to detect operations at one position on the display panel of the display 61, or may be configured to be able to simultaneously detect operations at multiple positions on the display panel. The touch sensor 62 corresponds to an example of an operation unit.

[0048] The DP communication unit 63 is connected to the communication network 3 and transmits and receives data to and from the projection device 1 via the communication network 3. The DP communication unit 63 is, for example, a communication device including a connector for connecting a communication cable and a communication circuit for inputting and outputting signals via the communication cable. The DP communication unit 63 may also be a wireless communication device. In this case, the DP communication unit 63 includes, for example, an antenna, an RF circuit, a baseband circuit, etc.

[0049] The DP control unit 70 includes a processor 71 and a memory 75. The memory 75 is a storage device that stores programs and data executed by the processor 71 in a nonvolatile manner. The memory 75 is configured from a magnetic storage device, a semiconductor storage element such as a flash ROM, or other types of nonvolatile storage device. The memory 75 may include RAM that configures the work area of ​​the processor 71. The memory 75 stores data processed by the processor 71 and a control program 76 executed by the processor 71.

[0050] The processor 71 is configured with a CPU, an MPU, or the like. The processor 71 may be configured with a single processor, or multiple processors may function as the processor 71. The processor 71 may be configured as an SoC integrated with part or all of the memory 75 and / or other circuits. As described above, the processor 71 may also be configured with a combination of a CPU that executes programs and a DSP that executes predetermined arithmetic processing. All of the functions of the processor 71 may be implemented in hardware, or may be configured using a programmable device.

[0051] The processor 71 controls each part of the display device 6 by executing a control program 76 stored in the memory 75 .

[0052] The processor 71 causes the DP communication unit 63 to receive the captured image transmitted by the projection device 1. The processor 71 causes the display 61 to display the captured image received by the DP communication unit 63.

[0053] The processor 71 receives an operation detected by the touch sensor 62. The processor 71 generates a drawing image based on the received operation. The processor 71 generates a transmission image including the drawing image, and transmits the transmission image to the projection device 1 via the DP communication unit 63.

[0054] [1-3. Optical unit configuration] FIG. 3 is a diagram showing an example of the configuration of the main parts of the projection device 1. As shown in FIG. 3, the projection optical system 13 includes a dichroic prism 131 and a lens 132. The lens 132 is an optical component that focuses the image light PL onto the projection target OB.

[0055] The optical axis of the image light PL projected by the lens 132 toward the projection target OB is indicated by the symbol AX. The optical axis AX is the central axis of the image light PL irradiated from the lens 132 onto the projection target OB, and is a virtual axis that passes through the optical center of the lens 132 and follows the direction in which the image light PL is irradiated from the lens 132. The lens 132 may be a lens group made up of multiple lenses.

[0056] The image light forming unit 12 includes a light emitting element 121. The light emitting element 121 has a configuration in which light emitting bodies are arranged in a row on a light emitting surface 122. The light emitting bodies arranged on the light emitting surface 122 include a light emitting body that emits red light, a light emitting body that emits blue light, and a light emitting body that emits green light. By arranging these light emitting bodies in a matrix, the light emitting element 121 emits image light PL that forms an image from the light emitting surface 122.

[0057] The light-emitting surface 122 faces the dichroic prism 131. The image light PL emitted by the light-emitting surface 122 enters the dichroic prism 131 along the optical axis AX, passes through the dichroic prism 131, and enters the lens 132. The lens 132 irradiates the image light PL that has passed through the dichroic prism 131 onto the projection target OB. The optical axis of the image light PL emitted by the light-emitting surface 122 is defined as the projection optical axis PAX. The projection optical axis PAX is the central axis of the image light PL emitted by the light-emitting element 121, and is a virtual axis that passes through the center of the area on the light-emitting surface 122 where the light emitters are arranged and is perpendicular to the light-emitting surface 122. In the configuration of FIG. 3, the projection optical axis PAX coincides with the optical axis AX. In other words, the light-emitting element 121 is arranged on the optical axis AX of the projection optical system 13.

[0058] The photographing unit 15 includes an image sensor 151. The image sensor 151 is disposed opposite the dichroic prism 131. In the image sensor 151, image sensors are arranged side by side on an image capturing surface 152 facing the dichroic prism 131. The photographing unit 15 has photosensors arranged on the image capturing surface 152, and each photosensor receives light incident from the dichroic prism 131 to capture an image. The image sensor 151 faces a surface of the dichroic prism 131 that is different from the surface of the light emitting element 121. In detail, the light emitting element 121 is disposed so as to be aligned with the dichroic prism 131 in a direction along the optical axis AX. In contrast, the image sensor 151 faces the dichroic prism 131 at an angle of approximately 90 degrees with respect to the optical axis AX.

[0059] As described above, the light-emitting element 121 is disposed opposite the first surface 131a of the dichroic prism 131, and the imaging surface 152 is disposed opposite the second surface 131b of the dichroic prism 131. The first surface 131a is on the optical axis AX and intersects with the optical axis AX perpendicularly. The second surface 131b is generally parallel to the optical axis AX. In other words, the first surface 131a forms an angle of 90 degrees with the second surface 131b.

[0060] The dichroic prism 131 is an example of a separating optical member that separates light emitted by the light-emitting element 121 from light incident on the dichroic prism 131 from the projection target OB. The dichroic prism 131 has, for example, a semi-reflective surface 131d inside. The dichroic prism 131 transmits the light emitted by the light-emitting element 121 and causes the light to enter the lens 132 along the optical axis AX. The dichroic prism 131 also reflects the light incident on a third surface 131c of the dichroic prism 131 from the lens 132 toward the image sensor 151 by the semi-reflective surface 131d.

[0061] The semi-reflective surface 131d of the dichroic prism 131 includes, for example, a polarization separation film. In this case, the dichroic prism 131 transmits, for example, P-polarized light and reflects S-polarized light. As a result, the dichroic prism 131 separates the light emitted by the light-emitting element 121 and the light incident on the dichroic prism 131 from the projection target OB using the semi-reflective surface 131d.

[0062] The semi-reflective surface 131d of the dichroic prism 131 may include, for example, a wavelength separation film. The wavelength separation film may also be called a wavelength selection filter. In this case, the semi-reflective surface 131d transmits, for example, light in a specific wavelength range and reflects light in other wavelength ranges. As a result, the dichroic prism 131 separates the light emitted by the light-emitting element 121 and the light incident on the dichroic prism 131 from the projection target OB using the semi-reflective surface 131d.

[0063] The semi-reflective surface 131d of the dichroic prism 131 may include, for example, a semi-transmissive film. In this case, the dichroic prism 131 transmits light incident on the first surface 131a and reflects light incident on the third surface 131c at the semi-reflective surface 131d.

[0064] The optical axis of light reflected by the dichroic prism 131 toward the image sensor 151 is indicated by the symbol IAX. The photographing optical axis IAX is the central axis of light traveling from the dichroic prism 131 toward the image sensor 151, and is the axis of light received by the image sensor 151 at its imaging surface 152. The photographing optical axis IAX is a virtual axis perpendicular to the imaging surface 152. In other words, the image sensor 151 is positioned so that the center of the imaging surface 152 coincides with the photographing optical axis IAX.

[0065] The imaging optical axis IAX coincides with the optical axis AX between the projection target OB and the semi-reflective surface 131d, and is bent by approximately 90 degrees at the semi-reflective surface 131d. In other words, the projection optical axis PAX and the imaging optical axis IAX coincide in an area closer to the projection target OB than the semi-reflective surface 131d. In this way, the optical unit 10 is configured such that the projection unit 11 and the imaging unit 15 are optically arranged on the same axis. Therefore, the projection unit 11 and the imaging unit 15 perform projection and imaging along the same axis.

[0066] 3 is a schematic diagram of the main configuration of the projection device 1, and the optical unit 10 may include components not shown in FIG. 3. For example, the projection optical system 13 may include optical elements other than the dichroic prism 131 and the lens 132. Specifically, the projection optical system 13 may include a light guide element between the dichroic prism 131 and the lens 132. Furthermore, a polarization separation element and a polarization conversion element may be provided between the light emitting element 121 and the dichroic prism 131, so that the image light PL incident on the dichroic prism 131 is converted into P-polarized light.

[0067] [1-4. Projection System Operation] Fig. 4 is a sequence diagram showing the operation of the projection system 1000. Fig. 5 is a schematic diagram showing the operation of the projection system 1000. The operation of the projection system 1000 will be described with reference to these figures.

[0068] In FIG. 4, the PJ control unit 20 of the projection device 1 executes the processes of steps SA11 to SA16, and the DP control unit 70 of the display device 6 executes the processes of steps SB11 to SB17.

[0069] In step SA11, the projection device 1 captures an image of an area including the projection target OB and generates a captured image. In step SA12, the projection device 1 transmits the captured image to the display device 6.

[0070] In step SB11, the display device 6 receives the captured image transmitted by the projection device 1. The display device 6 displays the received captured image on the display 61 in step SB12.

[0071] Here, when the display device 6 detects an operation by the indicator 65, it accepts this operation in step SB13. In step SB14, the display device 6 generates a drawn image based on the operation accepted in step SB13. For example, the drawn image is a trajectory of a curve or a straight line along which the indicator 65 has moved.

[0072] In step SB15, the display device 6 generates a composite image by superimposing the drawn image generated in step SB14 on the photographed image displayed in step SB12, and causes the display 61 to display the composite image.

[0073] In step SB16, the display device 6 transmits the drawn image generated in step SB14 to the projection device 1, and then proceeds to step SB17.

[0074] In step SA13, the projection device 1 receives the image transmitted by the display device 6. The image transmitted by the display device 6 and received by the projection device 1 includes a drawn image generated by the display device 6.

[0075] In step SA14, the projection device 1 updates the image in the received image memory 27 based on the received image. Specifically, the projection device 1 generates a composite image by combining the image received in step SA13 with the image stored in the received image memory 27. The projection device 1 updates the image in the received image memory 27 by storing the composite image in the received image memory 27.

[0076] The projection device 1A proceeds to step SA15, where it causes the projection unit 11 to project the image in the updated received image memory 27 onto the projection target OB, and then proceeds to step SA16. The image that the projection device 1A projects onto the projection target OB includes a composite image.

[0077] In step SA16, the projection device 1 determines whether to end the operation. If the projection device 1 determines to end the operation (step SA16; YES), such as when an operation instructing to end the operation is detected by the input interface 33, the projection device 1 ends this process. If the projection device 1 determines not to end the operation (step SA16; NO), the projection device 1 returns to step SA11.

[0078] After transmitting the image to the projection device 1, the display device 6 determines whether to end the operation in step SB17. If the display device 6 determines to end the operation (step SB17; YES), such as when an operation instructing to end the operation is detected by the touch sensor 62, the display device 6 ends this process. If the display device 6 determines not to end the operation (step SB17; NO), the process returns to step SB11.

[0079] FIG. 5 shows the operating states ST1, ST2, ST3, and ST4 of the projection system 1000. The following shows an example of a state in which a sheet OB1 and an object OB2 are included as the projection target OB of the projection device 1. The sheet OB1 is a flat sheet such as paper, and a user of the projection device 1 can write letters and figures on the sheet OB1 with a writing implement such as a pencil.

[0080] An example of a usage form of the projection system 1000 is distance education. For example, a student uses the projection device 1 at a usage location S1, and a teacher uses the display device 6 at a usage location S2. The student spreads out a notebook or teaching material on the projection target OB, and the projection device 1 projects a projection image PP onto the notebook or teaching material. In other words, the sheet OB1 is the notebook or teaching material. When the student writes letters or figures on the notebook or teaching material with a writing implement, the projection device 1 captures the letters or figures and sends the generated captured image to the display device 6. The teacher understands the student's answers, etc., from the letters and figures in the captured image displayed on the display 61 and corrects them using the pointer 65. When the teacher inputs using the pointer 65, the display device 6 generates a drawn image in response to the input. The display device 6 transmits the drawn image based on the teacher's input to the projection device 1. The projection device 1 then projects the received drawn image onto the projection target OB. This allows the students using the projection device 1 to clearly see the drawing image corrected by the teacher on the display device 6 on the notebook or teaching materials of the alumni who are the projection target.

[0081] In state ST1, the projection device 1 transmits a captured image generated by capturing images of the sheet OB1 and the object OB2 as a first image P1 to the display device 6. The first image P1 includes an object image OP2, which is a captured image of the object OB2. The entire first image P1 also includes the object image OP1, which is a captured image of the sheet OB1.

[0082] The display device 6 receives the first image P1 and displays the first image P1 on the display 61. On the display 61, an object image OP1 is displayed.

[0083] State ST2 is a state in which an operation is performed on display device 6 using indicator 65. Display device 6 generates a first drawn image DP1 based on the operation of indicator 65. Display device 6 displays first drawn image DP1 on display 61 and transmits a second image P2 including first drawn image DP1 to projection device 1. The second image P2 does not include object images OP1 and OP2.

[0084] The projection device 1 receives the second image P2. The projection device 1 stores the received first image P2 in the received image memory 27. If an image is already stored in the received image memory 27, the projection device 1 updates the image stored in the received image memory 27.

[0085] State ST3 is a state in which a figure is written on sheet OB1 with a writing implement in the projection device 1. The projection device 1 generates a captured image by capturing an image of the projection target OB, and transmits the captured image to the display device 6 as a third image P3.

[0086] The third image P3 includes an object image OP1 which is an image of the sheet OB1, an object image OP2 which is an image of the object OB2, an object image OP3, and an object image OP4. The object image OP3 is an image obtained by capturing the first drawn image DP1 projected onto the projection target OB by the projection unit 11, and captured by the capturing unit 15. The object image OP4 is a captured image of a drawn figure written on the sheet OB1 with a writing implement.

[0087] The display device 6 receives the third image P3 and displays the third image P3 on the display 61. The display 61 displays the object images OP1, OP2, OP3, and OP4 included in the third image P3. When displaying the third image P3, the display device 6 switches the image previously displayed on the display 61 to the third captured image P3. As a result, the display of the first drawn image DP1 displayed in state ST2 is stopped.

[0088] State ST4 is a state in which an operation is performed on display device 6 by pointer 65. Display device 6 generates a second drawn image DP2 based on the operation of pointer 65. Display device 6 transmits a fourth image P4 including second drawn image DP2 to projection device 1.

[0089] The projection device 1 receives the fourth image P4 and updates the image stored in the received image memory 27 based on the fourth image P4. After receiving the second image P2, the received image memory 27 stores the first drawn image DP1 included in the second image P2. After receiving the fourth image P4, the projection device 1 adds the second drawn image DP2 included in the second transmitted image P4 to the image stored in the received image memory 27. As a result, the image stored in the received image memory 27 is updated to an image in which the first drawn image DP1 and the second drawn image DP2 are superimposed. The projection device 1 projects the image stored in the received image memory 27 onto the projection target OB. As a result, the first drawn image DP1 and the second drawn image DP2 are projected onto the projection target OB.

[0090] The first image P1 is an example of an image captured and transmitted by the projection device 1, and an example of an image received by the display device 6. The second image P2 is an example of an image transmitted by the display device 6, and an example of an image received and projected by the projection device 1. The third image P3 is an example of an image captured and transmitted by the projection device 1, and an example of an image received by the display device 6. The fourth image P4 is an example of an image transmitted by the display device 6, and an example of an image received and projected by the projection device 1. The first drawn image DP1 and the second drawn image DP2 are examples of drawn images generated on the display device 6 based on the operation of the pointer 65.

[0091] The characteristics of the operation shown in Figure 5 include that when the display device 6 receives the third image P3, it displays the third image P3 on the display 61 and stops displaying the first drawn image DP1, and the projection device 1 updates the image in the received image memory 27.

[0092] For example, in state ST3, if the display device 6 displays the third image P3 on the display 61 while still displaying the first drawn image DP1 on the display 61, the object image OP3 and the first drawn image DP1 will be displayed superimposed at the same position. The object image OP3 is an image obtained by projecting the first drawn image DP1 by the projection device 1 and then photographing it, and therefore has the same shape as the first drawn image DP1. However, it is highly likely that the positions, shapes, and sizes of the object image OP3 and the first drawn image DP1 will not completely match. Possible reasons for this include the influence of ambient light around the projection device 1 and the difference in resolution between the image projected by the projection unit 11 and the image photographed by the photographing unit 15.

[0093] If the object image OP3 and the first drawn image DP1 do not match, the difference between the object image OP3 and the first drawn image DP1 may cause the display on the display 61 to be distorted. For example, the lines constituting the object image OP3 and the lines constituting the first drawn image DP1 may be displayed as double lines. Furthermore, for example, the lines constituting the object image OP3 and the lines constituting the first drawn image DP1 may be displayed as interference fringes. Such displays result in a decrease in display quality and are a factor in reducing the visibility of the object image OP3 and the first drawn image DP1.

[0094] 3, when the projection optical axis PAX of the projection unit 11 and the imaging optical axis IAX of the imaging unit 15 are the same axis, i.e., coaxial, the difference in position and size between the object image OP3 and the first drawn image DP1 is very small, which makes it easy for the lines constituting the object image OP3 and the lines constituting the first drawn image DP1 to be displayed like interference fringes.

[0095] In this embodiment, the display device 6 ends the display of the first drawn image DP1 when displaying the third image P3 on the display 61. Specifically, after receiving the third captured image P3 including the object image OP3 corresponding to the first drawn image DP1, the display device 6 does not display the first drawn image DP1 on the display 61. The same applies to the second drawn image DP2. This prevents the first drawn image DP1 and the object image OP3 from being displayed overlapping each other. This prevents a decrease in the display quality of the image displayed on the display 61.

[0096] Then, the projection device 1 stores the first drawn image DP1 included in the second image P2 and the second drawn image DP2 included in the fourth image P4 in the received image memory 27. This allows the projection device 1 to project the first drawn image DP1 and the second drawn image DP2, which are multiple drawn images generated at different times. This has the advantage that the drawn images drawn by the user of the display device 6 will not be lost.

[0097] [1-5. Operation of the embodiment] As described above, the projection device 1 described in the first embodiment includes a projection unit 11 that projects image light PL onto the projection target OB, and a capture unit 15 that captures an area including the projection target OB and generates a captured image. The projection device 1 also includes a PJ communication unit 42 and a PJ control unit 20 that transmits the captured image to the display device 6 via the PJ communication unit 42 and causes the projection unit 11 to project a received image received from the display device 6 via the PJ communication unit 42. The projection unit 11 includes an image light formation unit 12 that forms image light PL, and a projection optical system 13 that projects the image light PL formed by the image light formation unit 12 toward the projection target OB. The capture unit 15 includes an image sensor 151 that receives light through the projection optical system 13. The PJ control unit 20 receives a received image, including an image drawn by the display device 6, via the PJ communication unit 42.

[0098] As a result, the projection device 1 projects the image received from the display device 6 onto the projection target OB using the projection unit 11, captures an image of the projection target OB, and transmits the image to the display device 6. As a result, a user of the projection device 1 can share the state of the projection target OB and information held by the display device 6 with a user of the display device 6.

[0099] The image capturing unit 15 captures an image by receiving light collected by the lens 132 of the projection optical system 13. Therefore, the lens 132 that projects the image light PL can be used for image capturing by the image capturing unit 15. This allows the projection device 1 to have a compact configuration. Furthermore, the projection optical axis PAX along which the projection unit 11 projects the image light PL can be positioned close to the image capturing optical axis IAX along which the image capturing unit 15 captures the image. This reduces distortion when the image capturing unit 15 captures the projection image PP formed on the projection target OB by the image light PL. This allows the display device 6 and the projection device 1 to share the state of the projection target OB in a high-quality image. For example, as described above, when the projection system 1000 is used for distance education, a teacher using the display device 6 can check the state of the projection target OB of the projection device 1 used by a student using an image with minimal distortion, thereby facilitating distance education.

[0100] In the projection device 1, the projection optical system 13 includes a separating optical element that separates the light incident from the projection target OB from the image light PL and guides the separated light to the imaging element. The separating optical element is, for example, a dichroic prism 131. This configuration allows the projection unit 11 and the imaging unit 15 to use a common lens 132 for projection and imaging. By using the separating optical element, the imaging unit 15 is less susceptible to the influence of the image light PL from the projection unit 11, and therefore a high-quality captured image can be obtained.

[0101] The dichroic prism 131, which is a separating optical member, has, for example, a polarization separating film that separates light by polarization. In this case, it is possible to efficiently separate the image light PL emitted by the projection unit 11 from the external light to be received by the imaging unit 15.

[0102] The dichroic prism 131, which is a separating optical element, may have, for example, a wavelength separation film that separates light of different wavelengths. In this case, the image light PL and the external light to be received by the image capturing unit 15 can be separated by selecting them by wavelength. In this case, the image capturing unit 15 may be configured to detect light in a wavelength range outside the visible range, such as infrared light or ultraviolet light, using the image sensor 151, and the projection device 1 may generate a monochrome captured image. With this configuration, the wavelength separation film more reliably separates the external light captured by the image capturing unit 15 from the image light PL, thereby efficiently suppressing the influence of the image light PL on the image capturing by the image capturing unit 15. Furthermore, the dichroic prism 131, which is a separating optical member, may have a semi-transparent film.

[0103] The projection device 1 includes a fixed unit 101 and an optical unit case 103 connected to the fixed unit 101, and the projection unit 11 and the image capture unit 15 are housed in the optical unit case 103. This allows the projection device 1 to be made more compact by taking advantage of the configuration in which the projection unit 11 and the image capture unit 15 share a common lens 132. This makes it possible to install the projection device 1 in a limited space, such as on a desk. This is suitable, for example, when using the projection device 1 for distance learning.

[0104] The projection device 1 includes a received image memory 27 that stores an image included in a received image. When the projection device 1 receives a received image from the display device 6, it updates the image stored in the received image memory 27 based on the received image, and projects the updated image stored in the received image memory using the projection unit 11. With this configuration, when the projection device 1 projects the received image received from the display device 6 onto the projection target OB, it is possible to avoid a situation in which the actual sheet OB1 or object OB2 placed on the projection target OB overlaps with the captured image of the sheet OB1 or object OB2. This prevents a decrease in the visibility of the sheet OB1, object OB2, and projected image PP on the projection target OB, allowing the projection device 1 to be used in good condition.

[0105] The projection device 1 also includes a projection unit 11 that projects image light PL onto the projection target OB, an imaging unit 15 that captures an image of an area including the projection target OB and generates a captured image, a PJ communication unit 42 that communicates with the display device 6, and a PJ control unit 20. The PJ control unit 20 transmits the captured image to the display device 6 via the PJ communication unit 42, receives a drawn image from the display device 6, and controls the projection unit 11 to project the drawn image onto the projection target OB and not to project the captured image. As a result, the projection device 1 projects an image received from the display device 6 onto the projection target OB using the projection unit 11 and transmits a captured image of the projection target OB to the display device 6. This allows a user of the projection device 1 to share the state of the projection target OB and information held by the display device 6 with a user of the display device 6. Here, the projection device 1 does not capture the captured image using the projection unit 11. This prevents an image or object formed by handwriting or the like on the projection target OB from overlapping with the image light PL containing an image obtained by capturing this image or object using the capture unit 15 on the projection target OB. This prevents a situation in which the projection image projected onto the projection target OB overlaps with an actual image or object, impairing visibility. This allows a user of the projection device 1 to share images with the display device 6 in a good environment.

[0106] The control method of the projection device 1 involves transmitting the captured image generated by the image capturing unit 15 as a first image to the display device 6 via the PJ communication unit 42, and receiving a second image including a first drawn image drawn on the first image at the display device 6. The projection unit 11 projects the first drawn image onto the projection target OB, the image capturing unit 15 captures an area including the projection target OB onto which the first drawn image is projected to generate a captured image, and the PJ communication unit 42 transmits the captured image to the display device 6 as a third image. The display device 6 also receives a fourth image including a second drawn image drawn on the third image, and the projection unit 11 projects the first drawn image and the second drawn image onto the projection target OB. As a result, the projection device 1 projects the image received from the display device 6 onto the projection target OB using the projection unit 11, and transmits a captured image of the projection target OB to the display device 6. As a result, a user of the projection device 1 can share the state of the projection target OB and information held by the display device 6 with a user of the display device 6.

[0107] [2. Second Embodiment] 6 is a diagram showing an example of the configuration of the main parts of a projection device 1A according to the second embodiment. In the description of the projection device 1A, components common to those of the projection device 1 described in the first embodiment are given the same reference numerals, and descriptions thereof will be omitted. Like the projection device 1, the projection device 1A is connected to a display device 6 via a communication network 3 to form a projection system 1000.

[0108] 6 instead of the optical unit 10 included in the projection device 1. The configuration of the projection device 1A is the same as that of the projection device 1 except for the optical unit 10A.

[0109] The optical unit 10A has a projection unit 11A and an imaging unit 15. The projection unit 11A has an image light formation unit 12 and a projection optical system 13A. The projection optical system 13A has a dichroic mirror 133 instead of the dichroic prism 131 in the configuration of the projection optical system 13 of the optical unit 10. The other components, such as the lens 132, the light emitting element 121, and the image sensor 151, can be configured in common with the optical unit 10. The lens 132 in FIG. 6 may be the same as the lens 132 illustrated in FIG. 3, or may have a different configuration.

[0110] The optical unit 10A includes a dichroic mirror 133 instead of the dichroic prism 131 included in the optical unit 10. The dichroic mirror 133 is an example of a separating optical member that separates the light emitted by the light emitting element 121 from the light incident on the dichroic mirror 133 from the projection target OB. Like the semi-reflective surface 131d of the dichroic prism 131, the dichroic mirror 133 transmits the light emitted by the light emitting element 121, while reflecting the light incident on the dichroic mirror 133 from the lens 132 toward the image sensor 151.

[0111] Dichroic mirror 133 may include, for example, a polarization separation film. In this case, dichroic mirror 133 transmits, for example, P-polarized light and reflects S-polarized light. Dichroic mirror 133 may include, for example, a wavelength separation film. In this case, semi-reflective surface 131d transmits, for example, light in a specific wavelength range and reflects light in other wavelength ranges. Dichroic mirror 133 may include, for example, a semi-transmitting film.

[0112] The optical axis of the light reflected by the dichroic mirror 133 toward the image sensor 151 is the imaging optical axis IAX, similar to the configuration example in Fig. 3. The imaging optical axis IAX is a virtual axis perpendicular to the imaging surface 152.

[0113] The optical axis of light reflected by the dichroic prism 131 toward the image sensor 151 is indicated by the symbol IAX. The photographing optical axis IAX is the central axis of light traveling from the dichroic prism 131 toward the image sensor 151, and is the axis of light received by the image sensor 151 at its imaging surface 152. The photographing optical axis IAX is a virtual axis perpendicular to the imaging surface 152. In other words, the image sensor 151 is positioned so that the center of the imaging surface 152 coincides with the photographing optical axis IAX.

[0114] As described above, the optical unit 10A has the dichroic mirror 133 as a separating optical member. Similar to the dichroic prism 131, the dichroic mirror 133 separates the image light PL emitted by the projection unit 11 from the external light detected by the image capturing unit 15. Therefore, by including the optical unit 10A, the projection device 1A achieves the same effects as the projection device 1.

[0115] The capturing unit 15 captures an image by receiving light collected by the lens 132 of the projection optical system 13. Therefore, the lens 132 that projects the image light PL can be used for capturing an image by the capturing unit 15. This allows the projection device 1A to have a compact configuration. Furthermore, the projection optical axis PAX along which the projection unit 11A projects the image light PL and the capturing optical axis IAX along which the capturing unit 15 captures an image can be positioned close to each other. This reduces distortion when the projection image PP formed on the projection target OB by the image light PL is captured by the capturing unit 15. This allows the state of the projection target OB to be shared between the display device 6 and the projection device 1A through a high-quality image.

[0116] The dichroic mirror 133, which is a separating optical element, has, for example, a polarization separation film that separates light by polarization. In this case, the image light PL emitted by the projection unit 11 can be efficiently separated from the external light to be received by the capture unit 15. The dichroic mirror 133, which is a separating optical element, may have, for example, a wavelength separation film that separates light of different wavelengths. In this case, the image light PL can be selectively separated from the external light to be received by the capture unit 15 based on wavelength. In this case, the capture unit 15 may be configured to detect light in a wavelength range outside the visible range, such as infrared light or ultraviolet light, using the image sensor 151, and the projection device 1A may generate a monochrome captured image. With this configuration, the wavelength separation film more reliably separates the external light captured by the capture unit 15 from the image light PL, thereby efficiently suppressing the influence of the image light PL on the capture by the capture unit 15. The dichroic mirror 133, which is a separating optical element, may also have a semi-transparent film.

[0117] Similarly to the projection device 1, the projection device 1A can be configured to include a fixed part 101 and an optical unit case 103 connected to the fixed part 101, with the projection part 11A and the image capturing part 15 housed in the optical unit case 103. This allows the projection device 1A to be configured more compactly by taking advantage of the configuration in which the projection part 11A and the image capturing part 15 use a common lens 132.

[0118] 3. Third Embodiment 7 is a diagram showing an example of the configuration of the main parts of a projection device 1B according to the third embodiment. In the description of the projection device 1B, components common to the projection device 1 described in the first embodiment are given the same reference numerals, and description thereof will be omitted. Like the projection device 1, the projection device 1B is connected to a display device 6 via a communication network 3 to form a projection system 1000.

[0119] 7 instead of the optical unit 10 included in the projection device 1. The configuration of the projection device 1B is the same as that of the projection device 1 except for the optical unit 10B.

[0120] The optical unit 10B has a projection section 11B and an imaging section 15B. The projection section 11B has an image light formation section 12A and a projection optical system 13A. The projection optical system 13B has a lens 135. The lens 135 is an optical component that projects the image light PL formed by the image light formation section 12B onto the projection target OB, forming an image on the projection target OB. The lens 135 may be the same as the lens 132 in FIG. 3 or may have a different configuration. The lens 135 may be a lens group made up of a plurality of lenses.

[0121] The image light forming unit 12B includes a transmissive light emitting element 123. The transmissive light emitting element 123 has a configuration in which light emitters are arranged side by side on a light-transmitting substrate 123a. The light emitters included in the transmissive light emitting element 123 include a light emitter that emits red light, a light emitter that emits blue light, and a light emitter that emits green light. These light emitters are arranged in a matrix, so that the light emitting element 123 forms the image light PL. At least a portion of the substrate 123a, where no light emitters are arranged, transmits light.

[0122] In the transmissive light-emitting element 123, a light-emitting surface 123b on which a light emitter is arranged is disposed opposite the lens 135. The image light PL emitted from the light-emitting surface 123b of the transmissive light-emitting element 123 enters the lens 135 along the optical axis AX, and the lens 135 irradiates the image light PL onto the projection target OB. The central axis of the image light PL is defined as the optical axis AX. The optical axis AX is a virtual axis that passes through the center of the light-emitting surface 123b and is perpendicular to the light-emitting surface 123b. The optical axis AX also passes through the optical center of the lens 135.

[0123] The photographing unit 15B includes a transmitted light imaging element 153. The transmitted light imaging element 153 corresponds to an example of an imaging element. The transmitted light imaging element 153 is arranged along the optical axis AX, overlapping the transmissive light emitting element 123. External light that has passed through the lens 135 is incident on the transmissive light emitting element 123 along the optical axis AX. A portion of the external light that has entered the transmissive light emitting element 123 passes through the transmissive light emitting element 123 and enters the transmitted light imaging element 153. The transmitted light imaging element 153 performs photographing by receiving transmitted light that has passed through the transmissive light emitting element 123 with a photosensor. The photosensors of the transmitted light imaging element 153 are arranged side by side on a plane perpendicular to the optical axis AX.

[0124] In this way, the optical unit 10B has a configuration in which the transmissive light emitting element 123 and the transmitted light imaging element 153 are arranged to overlap on the optical axis AX. In other words, the image light forming section 12B and the photographing section 15B are arranged coaxially.

[0125] The lens 135 has both the function of irradiating the image light PL emitted by the image light formation unit 12B toward the projection target OB and the function of collecting external light and guiding it to the transmitted-light imaging element 153. In other words, the transmissive light emitting element 123 and the transmitted-light imaging element 153 use the common lens 135 to project and capture the image light PL.

[0126] FIG. 8 is a timing chart showing the operation of the projection device 1B. 2, the projection device 1B drives an optical unit 10B by a drive circuit 14. The projection device 1B also controls an image capturing unit 15B by a PJ control unit 20.

[0127] 8, (A) shows the operation of the transmitted light imaging element 153. In detail, the operation of the PJ control unit 20 to read out the detection value of the photosensor from the transmitted light imaging element 153 is shown. 8(B) shows the operation of the red light emitter included in the transmissive light emitting element 123, (C) shows the operation of the blue light emitter included in the transmissive light emitting element 123, and (D) shows the operation of the green light emitter included in the transmissive light emitting element 123. The operation of the drive circuit 14 to cause the red light emitter, blue light emitter, and green light emitter to emit light is shown.

[0128] As shown in FIG. 8 , the drive circuit 14, under the control of the PJ control unit 20, causes the light emitter of the transmissive light emitting element 123 to emit light at time t1 and stops the light emission at time t2. Therefore, the light emitter of the transmissive light emitting element 123 emits light during a period T1 between times t1 and t2. Period T1 indicates the period during which the drive circuit 14 drives the light emitter of the transmissive light emitting element 123, but does not mean that the light emitter of the transmissive light emitting element 123 is constantly lit during period T1. For example, if the light emitter of the transmissive light emitting element 123 is a solid-state light source such as the LED or OLED described above, the drive circuit 14 controls the brightness of the light emitter of the transmissive light emitting element 123 using PWM (Pulse Wave Modulation) control. In this case, the light emitter of the transmissive light emitting element 123 repeatedly turns on and off during period T1, but this repetition is not shown in FIG. 8 . The same applies to period T3, which will be described later.

[0129] The PJ control unit 20 reads out the detection value of the transmitted light imaging element 153 during period T2, between times t2 and t3. The drive circuit 14 does not cause the transmissive light emitting element 123 to emit light until time t3. Thereafter, under the control of the PJ control unit 20, the drive circuit 14 causes the transmissive light emitting element 123 to emit light during period T3, between times t3 and t4. During period T3, the PJ control unit 20 does not read out the transmitted light imaging element 153. The drive circuit 14 causes the transmissive light emitting element 123 to emit light until time t4, at which time the light emission is stopped. The PJ control unit 20 reads out the detection value of the transmitted light imaging element 153 during period T4, between times t4 and t5. The PJ control unit 20 repeatedly performs the operations shown in FIG. 8.

[0130] 8 is a diagram schematically illustrating the operation of the transmissive light emitting element 123 and the transmitted light imaging element 153, and the lengths of the periods T1, T2, T3, and T4 are not limited to the example in Fig. 8. For example, the periods T1 and T3 do not have to be the same length, and the same applies to the periods T2 and T4.

[0131] 8, the PJ control unit 20 controls the drive circuit 14 so that the period during which the transmissive light emitting element 123 is caused to emit light does not overlap with the period during which readout from the transmitted-light imaging element 153. In other words, while the PJ control unit 20 is reading out from the transmitted-light imaging element 153, the transmissive light emitting element 123 does not emit light. This allows the transmitted-light imaging element 153 to receive ambient light without being affected by the image light PL emitted by the transmissive light emitting element 123. Therefore, in a configuration in which the transmissive light emitting element 123 and the transmitted-light imaging element 153 are arranged overlapping on the optical axis AX, ambient light can be appropriately captured by the transmitted-light imaging element 153.

[0132] As described above, the projection device 1B of the third embodiment includes an optical unit 10B. The optical unit 10B includes a transmissive light-emitting element 123 and a transmitted-light imaging element 153 that are arranged to overlap in the direction of the optical axis AX. The transmissive light-emitting element 123 and the transmitted-light imaging element 153 are arranged so that the light-emitting surfaces 123b of the transmissive light-emitting element 123 and the image light formation unit 12B intersect with the optical axis of the lens 135 of the projection optical system 13, and so that the light-receiving surface of the transmitted-light imaging element 153 intersects with the optical axis of the projection optical system 13.

[0133] The image capturing unit 15B captures an image by receiving light collected by the lens 135 of the projection optical system 13B. Therefore, the lens 135 that projects the image light PL can be used for image capturing by the image capturing unit 15B. This allows the projection device 1B to have a compact configuration. Furthermore, the optical axis AX along which the projection unit 11B projects the image light PL is the same as the optical axis AX along which the image capturing unit 15B captures the image. This reduces distortion when the projection image PP formed on the projection target OB by the image light PL is captured by the image capturing unit 15B. Furthermore, by arranging the transmissive light emitting element 123 and the transmitted-light imaging element 153 overlapping each other, the projection device 1B can achieve a configuration in which the image light PL is projected and captured coaxially without using the dichroic prism 131 or the dichroic mirror 133. Therefore, the state of the projection target OB can be shared between the display device 6 and the projection device 1B through a high-quality image.

[0134] Like the projection devices 1 and 1A, the projection device 1B can be configured to include a fixed part 101 and an optical unit case 103 connected to the fixed part 101, with the optical unit 10B housed in the optical unit case 103.

[0135] 8, the PJ control unit 20 stops light emission from the transmissive light emitting element 123 while the photographing unit 15B is photographing using the transmitted light imaging element 153, and stops photographing by the photographing unit 15B while the transmissive light emitting element 123 is emitting light. This allows the transmitted light imaging element 153 to perform photographing without being affected by the light emitted by the transmissive light emitting element 123. Therefore, in a configuration in which the transmissive light emitting element 123 and the transmitted light imaging element 153 are close to each other, a high-quality photographed image can be obtained by the transmitted light imaging element 153.

[0136] In the third embodiment, the transmitted-light imaging element 153 may be configured to include a wavelength-selective filter and receive light transmitted through the wavelength-selective filter. For example, the transmitted-light imaging element 153 may be configured to include a wavelength-selective filter on the surface facing the transmissive light-emitting element 123. This wavelength-selective filter may be a filter that attenuates the wavelength ranges of light emitted by the blue, red, and green light-emitting elements of the transmissive light-emitting element 123. In this case, the transmitted-light imaging element 153 can capture images without being affected by the light emitted by the transmissive light-emitting element 123. Therefore, even when the transmissive light-emitting element 123 and the transmitted-light imaging element 153 are located close to each other, a high-quality captured image can be obtained by the transmitted-light imaging element 153. When the wavelength-selective filter is provided in the transmitted-light imaging element 153, the PJ control unit 20 may omit the control shown in FIG. 8 . In other words, readout from the transmitted-light imaging element 153 may be performed while the transmissive light-emitting element 123 is emitting light.

[0137] [4. Fourth Embodiment] 9 is a diagram showing an example of the configuration of the main parts of a projection device 1C according to the fourth embodiment. In the description of the projection device 1C, components common to the projection device 1 described in the first embodiment are given the same reference numerals, and description thereof will be omitted. Like the projection device 1, the projection device 1C is connected to a display device 6 via a communication network 3 to form a projection system 1000.

[0138] 9 instead of the optical unit 10 included in the projection device 1. The configuration of the projection device 1C is the same as that of the projection device 1 except for the optical unit 10C.

[0139] The optical unit 10C has a projection optical system 13C and a light receiving / emitting element 16 that serves as both an image light forming unit and an image capturing unit. The projection optical system 13C includes a lens 136. The lens 136 is an optical component that projects image light PL formed by the light receiving / emitting element 16 onto a projection target OB, forming an image on the projection target OB. The lens 136 may be the same as the lens 132 in FIG. 3 or the lens 135 in FIG. 7, or may have a different configuration. The lens 136 may be a lens group composed of multiple lenses.

[0140] The light emitting / receiving element 16 is disposed on the optical axis AX of the lens 136. The light emitting / receiving element 16 has a light emitting body and a photosensor on a light emitting / receiving surface 16a facing the lens 136. That is, as shown enlarged in circle A in FIG. 9, a blue light emitting body 161, a red light emitting body 162, a green light emitting body 163, and a photosensor 165 are disposed on the light emitting / receiving surface 16a. The light emitting / receiving surface 16a corresponds to an example of a light emitting surface. The blue light emitting body 161, the red light emitting body 162, and the green light emitting body 163 are light emitting bodies.

[0141] The blue light-emitting element 161, the red light-emitting element 162, and the green light-emitting element 163 are configured, for example, by an LED or an OLED. The blue light-emitting element 161 is an element that emits light in a blue wavelength region, the red light-emitting element 162 is an element that emits light in a red wavelength region, and the green light-emitting element 163 is an element that emits light in a green wavelength region. In the example of FIG. 9, one pixel region 160 is configured by two blue light-emitting elements 161, one red light-emitting element 162, and one green light-emitting element 163. The pixel region 160 forms one pixel included in an image formed by the light-emitting and receiving element 16. In the example of FIG. 9, the pixel region 160 forms the color of one pixel by two blue light-emitting elements 161, one red light-emitting element 162, and one green light-emitting element 163.

[0142] The light receiving and emitting surface 16a includes one photosensor 165 per pixel region 160. The photosensor 165 is an element made of a CMOS or CCD, and receives light that is incident on the light receiving and emitting surface 16a. The imaging unit 15 captures an image by receiving light with the photosensor 165.

[0143] The light emitting / receiving elements 16 function as an image light forming section that forms image light PL and as an imaging section that captures images. The optical axis of the image light PL emitted by the light emitting / receiving elements 16 is optical axis AX, and the light emitting / receiving elements 16 captures images using light that is incident along the optical axis AX. Similar to the optical unit 10B shown in Fig. 7, the projection section and imaging section of the optical unit 10C are optically arranged coaxially, and the projection optical axis AX and the imaging optical axis AX are common to each other.

[0144] The projection device 1C executes the light emission and image capture of the light receiving and emitting elements 16 in the same manner as the operation described in FIG. 8 of the third embodiment. Specifically, the PJ control unit 20 of the projection device 1C controls the light emission of the blue light emitter 161, the red light emitter 162, and the green light emitter 163 in the same manner as the light emission timing of the transmissive light emitting element 123. The PJ control unit 20 also executes readout of the photosensor 165 at the same timing as the transmitted light imaging element 153. Therefore, readout of the photosensor 165 is not performed while the blue light emitter 161, the red light emitter 162, and the green light emitter 163 are emitting light, and the blue light emitter 161, the red light emitter 162, and the green light emitter 163 do not emit light while readout of the photosensor 165 is being performed. Therefore, the photosensor 165 can perform image capture without being affected by the light emissions of the blue light emitter 161, the red light emitter 162, and the green light emitter 163.

[0145] As described above, the projection device 1C of the fourth embodiment includes an optical unit 10C including an optical element 16 that functions as both a projection unit and an image capture unit. The optical element 10C's image light generating unit includes a blue light emitter 161, a red light emitter 162, and a green light emitter 163, which are light emitters arranged side by side on the light-emitting surface. The optical element 16 also functions as an image capture element, and includes multiple photosensors 165 arranged in a matrix on the light-receiving surface. The light-emitting and light-receiving surfaces of the optical element 16 are the same optical element 16a. This configuration allows the optical element 16, which functions as both a projection unit and an image capture unit, to project and capture image light PL coaxially without using a dichroic prism 131 or a dichroic mirror 133. This allows the display device 6 and the projection device 1B to share the state of the projection target OB through a high-quality image, thereby achieving further miniaturization of the projection device 1C.

[0146] The projection device 1C stops the emission of light from the blue light emitter 161, the red light emitter 162, and the green light emitter 163 while capturing an image using the photosensor 165 of the light emitting / receiving element 16. Furthermore, capturing an image using the photosensor 165 is stopped while the blue light emitter 161, the red light emitter 162, and the green light emitter 163 are emitting light. This allows capturing an image using the photosensor 165 without being affected by the light emitted by the blue light emitter 161, the red light emitter 162, and the green light emitter 163. Therefore, a high-quality captured image can be obtained in a configuration in which the light emitting / receiving element 16 functions as both a light emitting element and an imaging element.

[0147] [5. Fifth Embodiment] 10 is an external view of a portable projection device 5 according to the fifth embodiment. FIG. 11 is a cross-sectional view of the portable projection device 5.

[0148] The portable projection device 5 is a portable device that can be held by the user. The portable projection device 5 has a cylindrical main body case 51, and the components described below are housed inside the main body case 51. A grip portion 53 is provided on one end of the main body case 51. The grip portion 53 is a portion of the main body case 51 with a narrower diameter. Because the grip portion 53 has a narrower diameter, the user can easily hold the portable projection device 5 in their hand. The main body case 51 corresponds to an example of an exterior.

[0149] An opening 52 is provided at the tip of the main body case 51. The portable projection device 5 projects image light PL from the opening 52. This allows the image light PL to be projected onto a projection target located in the direction in which the opening 52 is pointed, thereby forming a projection image.

[0150] 11, main body case 51 houses optical unit 10D, drive unit 55, and battery 56. Optical unit 10D has, for example, image light formation unit 12B and imaging unit 15B described with reference to FIG. 7. Optical unit 10D also has lens 137 as projection optical system 13D. Lens 137 is an optical component that guides image light PL emitted by image light formation unit 12B in a direction to project it from opening 52.

[0151] The functional configuration of the projection device 1D is the same as that of the projection device 1 shown in Fig. 2. The drive unit 55 includes at least the PJ control unit 20 and the drive circuit 14, and may also include an operation unit 31, a remote control light receiving unit 32, an input interface 33, a connection unit 41, a PJ communication unit 42, and an image processing unit 43. The drive unit 55 may be configured to omit the remote control light receiving unit 32 and the connection unit 41. The PJ communication unit 42 may also be configured to perform wireless communication. The battery 56 includes a rechargeable secondary battery, and supplies power to the driving unit 55. The battery 56 corresponds to an example of a power source.

[0152] The portable projection device 5 can project image light PL from the opening 52 by emitting light from the transmissive light emitting element 123, thereby forming a projection image. Furthermore, the portable projection device 5 uses the transmitted light imaging element 153 to capture an image of external light collected by the lens 137. This allows projection and capture to be performed along a common optical axis AX.

[0153] In this way, the portable projection device 5 has a portable main body case 51, and the projection unit 11B, the image capturing unit 15B, the PJ communication unit 42, the PJ control unit 20, and the battery 56 are housed in the main body case 51. This allows the user to carry the portable projection device 5 around and project and capture a projection image using the portable projection device 5. When the portable projection device 5 is used in the projection system 1000 instead of the projection device 1, the status of the projection target OB and the projected image can be shared between the portable projection device 5 and the display device 6 without being restricted by the installation location of the portable projection device 5.

[0154] 6. Other Embodiments The above-described embodiments are preferred embodiments of the present invention, but the present invention is not limited to these and various modifications are possible within the scope of the present invention. For example, in the above embodiments, the projection devices 1, 1A, 1B, 1C, 1D, and portable projection device 5 are configured to project a full-color image using image light PL including red light, blue light, and green light. The present invention is not limited to this, and may be configured to project a monochrome image using single-color image light PL, for example. Furthermore, the image sensor 151, transmitted-light image sensor 153, and light-emitting / receiving element 16 may be configured to capture a color image or a monochrome image.

[0155] 1 has been described as being fixed to the installation unit 110 by the fixing unit 101, it may also be configured not to require the installation unit 110. Fig. 12 shows another example configuration of the projection system 1000 of Fig. 1. The projection device 1E shown in FIG. 12 has a base 106 that contacts an installation surface such as a desk. In the example of FIG. 12, the base 106 is flat, and the upper surface of the base 106 is used as the imaging stand 102. The imaging stand 102 is preferably flat, but may have a curved or uneven surface. The imaging stand 102 includes a projection target OB onto which the projection unit 11 projects the image light PL. The base 106 corresponds to an example of a pedestal. An optical unit case 103 that houses the optical unit 10 is connected to the base 106 by an arm 104. The arm 104 is erected on the base 106. An operation panel 35 having switches and the like used to operate the projection device 1E is also provided on the base 106. The projection device 1E uses the imaging unit 15 to capture an image of an area including the imaging stand 102. The projection device 1E transmits the captured image of the imaging stand 102 to the display device 6 via the communication network 3.

[0156] The configurations of the projection device 1 and the display device 6 shown in Fig. 2 show functional configurations, and the specific implementation form is not particularly limited. In other words, it is not necessary to implement hardware corresponding to each functional unit individually, and it is of course possible to implement a configuration in which a single processor executes a program to realize the functions of multiple functional units. Furthermore, some of the functions realized by software in each of the above embodiments may be realized by hardware, and vice versa.

[0157] 4 are divided according to the main processing content to facilitate understanding of the operation of each device in projection system 1000, and the present invention is not limited by the way the processing units are divided or the names of the processing units. The processing performed by each device can be divided into more processing units depending on the processing content, or one processing unit can be divided so that it includes even more processing.

[0158] The control program 26 may be recorded on a recording medium that is readable by the projection device 1. The recording medium may be a magnetic or optical recording medium or a semiconductor memory device. Specific examples include portable or fixed recording media such as flexible disks, optical disc recording media, magneto-optical disc recording media, and semiconductor storage devices. Alternatively, these programs may be stored on a server device or the like, and downloaded from the server device as needed. The same applies to the control program 76. [Explanation of symbols]

[0159] 1, 1A, 1B, 1C, 1D, 1E...Projection device, 3...Communication network, 5...Portable projection device, 6...Display device (information processing device), 10, 10A, 10B, 10C, 10D...Optical unit, 11, 11A, 11B...Projection unit, 12, 12A, 12B...Image light formation unit, 12A, 12B...Image light formation unit, 13, 13A, 13B, 13C, 13D...Projection optical system, 14...Driver circuit, 15, 15B...Photographing unit, 16...light-emitting / receiving element, 16a...light-emitting / receiving surface (light-emitting surface), 20...projection device control unit, 21...processor, 25...memory, 26...control program, 27...received image memory, 35...operation panel, 41...connection unit, 42...projection device communication unit, 43...image processing unit, 51...main body case (exterior), 52...opening, 53...holding unit, 55...drive unit, 56...battery (power source), 61...display, 62...touch sensor , 63...display communication unit, 65...pointer, 70...display control unit, 101...fixing unit, 102...shooting stand, 103...optical unit case (projection unit case), 104...arm, 105...hinge, 106...base (pedestal unit), 121...light-emitting element, 122...light-emitting surface, 123...transmissive light-emitting element, 123a...substrate, 123b...light-emitting surface, 131...dichroic prism (separating optical member), 131a...first surface, 131b...second surface, 131c...third surface, 131d...reflecting surface, 132, 135, 136, 137...lens, 133...dichroic mirror (separating optical component), 151...imaging element, 152...imaging surface, 153...transmitted light imaging element, 160...pixel area, 161...blue light emitter (light emitter), 162...red light emitter (light emitter), 163...green light emitter (light emitter), 165...photosensor, 1000...projection system, OB...projection target, PL...image light.

Claims

1. a display device having a display, a pointer, a display communication unit, and a display control unit; a projection device including a projection unit that projects image light onto a projection target, a photographing unit that photographs an area including the projection target and generates a photographed image, a communication unit, and a control unit that transmits the photographed image to the display communication unit by the communication unit and causes the projection unit to project a received image received from the display communication unit by the communication unit; The projection unit is an image light forming unit that forms image light; a projection optical system that projects the image light formed by the image light forming unit toward the projection target, The imaging unit is an image pickup element that receives light through the projection optical system; the optical axis of the projection optical system and the optical axis of the photographing unit are the same axis, the communication unit transmits a first image captured by the imaging unit to the display communication unit; The display control unit displaying the first image received by the display communication unit on the display; When the indicator is operated, a first drawing image based on the operation of the indicator is displayed on the display, and a second image including the first drawing image is transmitted to the communication unit; the control unit receives the second image including the first drawn image via the communication unit; the control unit includes a received image memory that stores the first drawn image included in the received image; when the control unit receives the first drawn image from the display communication unit of the display device, updates the image stored in the received image memory based on the received first drawn image, and projects the updated image stored in the received image memory by the projection unit; the photographing unit photographs a third image including the image stored in the received image memory after updating and projected by the projection unit; the communication unit transmits a third image captured by the imaging unit to the display communication unit; When the display communication unit receives the third image, the display control unit stops displaying the first image on the display and displays the third image on the display.

2. The projection system according to claim 1 , wherein the image capturing section captures images by receiving light collected by a lens included in the projection optical system.

3. The projection system according to claim 2 , wherein the projection optical system comprises a separating optical member that separates the light incident from the projection target from the image light and guides the separated light to the image sensor.

4. The projection system of claim 3 , wherein the separating optical element comprises a dichroic prism.

5. The projection system of claim 3 , wherein the separating optical element comprises a dichroic mirror.

6. The projection system according to claim 3 , wherein the separating optical member has a polarization separating film that separates light by polarization.

7. The projection system according to claim 3 , wherein the separating optical member has a wavelength separating film that separates light beams having different wavelengths.

8. The projection system according to claim 3 , wherein the separated optical member has a semi-transparent film.

9. 3. The projection system according to claim 1, wherein the image light forming unit and the image capturing element are arranged overlapping in the direction of the optical axis of the projection optical system so that the light emitting surface of the image light forming unit intersects with the optical axis of the projection optical system and the light receiving surface of the image capturing element intersects with the optical axis of the projection optical system.

10. the image light forming unit includes light emitters arranged on a light emitting surface, the imaging element includes a plurality of photosensors arranged in a matrix on a light receiving surface; 3. The projection system according to claim 1, wherein the light-emitting surface and the light-receiving surface are the same surface.

11. 11. The projection system according to claim 9, wherein light emission is stopped while the image capturing section is capturing an image using the image sensor, and image capturing by the image capturing section is stopped while light emission is being emitted.

12. The projection system according to claim 9 , wherein the image sensor includes a wavelength selection filter and receives light that has passed through the wavelength selection filter.

13. a base and a projection unit case connected to the base, The projection system according to claim 1 , wherein the projection unit and the image capture unit are housed in the projection unit case.

14. It has a portable exterior, The projection system according to claim 1 , wherein the projection unit, the image capturing unit, the communication unit, the control unit, and a power supply are housed in the exterior.

15. A control method for a projection system including a display device having a display, a pointer, a display communication unit, and a display control unit; a projection device having a projection unit that projects image light onto a projection target; an image capture unit that captures an area including the projection target to generate a captured image and has an optical axis that is the same as that of the projection unit; and a communication unit that communicates with the display communication unit, transmitting the captured image generated by the imaging unit to the display communication unit as a first image by the communication unit; displaying the first image received by the display communication unit by the display control unit; When the indicator is operated, the display control unit transmits a second image including a first drawn image drawn on the first image displayed on the display by the operation of the indicator to the communication unit, projecting the first drawing image included in the second image received by the communication unit onto the projection target by the projection unit; The imaging unit captures an image of an area including the projection target onto which the first rendering image is projected, to generate a third image; transmitting the third image to the display communication unit by the communication unit; the display control unit stops displaying the first image on the display, and displays the third image received by the display communication unit; When the indicator is operated, the display control unit transmits a fourth image including a second drawn image drawn on the third image displayed on the display by the operation of the indicator to the communication unit, projecting the first drawn image and the second drawn image included in the fourth image received by the communication unit onto the projection target by the projection unit; A method for controlling a projection system.

Images