Projection method and projection device

Abstract

To clarify which object a projection device recognizes as an obstacle, and allow a user to determine if appropriate control is performed.SOLUTION: A projection method includes: detecting a projection target; specifying a first area of the projection target where an obstacle is not present and a second area of the projection target where an obstacle is present, on the basis of a result of detecting the projection target; projecting a first projection image on a part or all of the first area; and projecting a second projection image different from the first projection image and indicating that the obstacle is present in the projection target.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a projection method and a projection device. [Background technology]

[0002] 2. Description of the Related Art Conventionally, when a projector projects a projection image onto a projection surface, there is known a technique for changing the projection position if an obstacle is present on the projection surface.

[0003] For example, Patent Document 1 discloses a technology in which, when an obstacle is present on the projection surface, a projectable area where the obstacle does not exist is determined based on sensing information obtained by sensing the projection target area, and a projection area, which is the area onto which the projection will actually be performed, is selected from the projectable area. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2004-48694 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the technology disclosed in Patent Document 1, if it is unclear which object the projector has recognized as an obstacle, the user does not know the basis for determining the projectable area, making it difficult to determine whether appropriate control is being performed. [Means for solving the problem]

[0006] A projection method according to one aspect of the present invention includes detecting a projection object; identifying a first area of ​​the projection object where no obstacle exists and a second area of ​​the projection object where the obstacle exists based on the result of detecting the projection object; projecting a first projection image onto part or all of the first area; and projecting a second projection image that is different from the first projection image and indicates that the obstacle exists on the projection object.

[0007] Furthermore, a projection device according to one aspect of the present invention detects a projection target; based on the result of detecting the projection target, identifies a first area of ​​the projection target where no obstacles exist and a second area of ​​the projection target where the obstacles exist; projects a first projection image onto part or all of the first area; and projects a second projection image that differs from the first projection image and indicates that the obstacle exists on the projection target. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a projection device 1A. [Figure 2] FIG. 2 is a block diagram showing an example of the configuration of a control device 10A. [Figure 3] FIG. 4 is an explanatory diagram of the detection operation of a detection unit 122. [Figure 4] FIG. 4 is an explanatory diagram of the calculation operation of a calculation unit 124. [Figure 5] FIG. 10 is an explanatory diagram of the correction operation of the correction unit 126A. [Figure 6] 10A and 10B are diagrams showing examples of a projected image PI and an instruction image II in a panel coordinate system. [Figure 7] 1A and 1B are diagrams showing examples of a projection image PI (display image DI) and an instruction image II displayed on a projection target PO. [Figure 8] 1A and 1B are diagrams showing examples of a projection image PI (display image DI) and an instruction image II displayed on a projection target PO. [Figure 9] 1A and 1B are diagrams showing examples of a projection image PI (display image DI) and an instruction image II displayed on a projection target PO. [Figure 10]1A and 1B are diagrams showing examples of a projection image PI (display image DI) and an instruction image II displayed on a projection target PO. [Figure 11] FIG. 2 is a block diagram showing an example of the configuration of a projector 30. [Figure 12] 10 is a flowchart showing a first operation of the control device 10A. [Figure 13] 10 is a flowchart showing a first operation of the control device 10A. [Figure 14] 10A and 10B are diagrams showing examples of a projection image PI (display image DI) and an instruction image II displayed on a projection target PO during a first operation. [Figure 15] 10 is a flowchart showing a second operation of the control device 10A. [Figure 16] 10 is a flowchart showing a second operation of the control device 10A. [Figure 17] 10A and 10B are diagrams showing examples of a projection image PI (display image DI) and an instruction image II displayed on the projection target PO during a second operation. [Figure 18] FIG. 2 is a block diagram showing an example of the configuration of a control device 10B. [Figure 19] FIG. 10 is a functional block diagram of a determination unit 130B. [Figure 20] FIG. 2 is a block diagram showing an example of the configuration of a control device 10C. [Figure 21] FIG. 2 is a block diagram showing an example of the configuration of a control device 10D. [Figure 22] FIG. 10 is a functional block diagram of a determination unit 130D. [Figure 23] 10 is a flowchart showing a first operation of a control device 10D. [Figure 24] 10 is a flowchart showing a first operation of a control device 10D. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, in each drawing, the dimensions and scale of each part are appropriately different from those of the actual parts. Furthermore, since the embodiments described below are preferred specific examples of the present invention, various technically preferable limitations are applied, but the scope of the present invention is not limited to these embodiments unless otherwise specified in the following description to the effect that the present invention is limited.

[0010] 1: First embodiment 1-1: Configuration of the first embodiment Hereinafter, a projection method and a projection device according to the first embodiment will be described with reference to FIGS.

[0011] 1-1-1: Overall configuration of the projection device 1 is a diagram showing an example configuration of a projection device 1A according to a first embodiment of the present disclosure. As shown in Fig. 1, the projection device 1A includes a control device 10A, an imaging device 20 that captures an image of a projection target PO, and a projector 30 that projects a projection image PI onto the projection target PO.

[0012] 1, the imaging device 20 is connected to the control device 10A via a communication line L1 such as a USB (Universal Serial Bus) cable, and the projector 30 is connected to the control device 10A via a communication line L2 such as a USB cable.

[0013] The imaging device 20 captures an image of a projection target PO. In the present disclosure, the projection target PO is an object having a three-dimensional shape onto which a projection image PI is projected. The projection target PO is, for example, a wall or a screen onto which the projection image PI is projected. The imaging device 20 captures various images under the control of the control device 10A. As will be described later, if the control device 10A is a PC, a tablet terminal, or a smartphone, the imaging device 20 may be a camera provided in these devices. However, the imaging device 20 is not limited to this and may be an external camera such as a web camera. Furthermore, as will be described later, the imaging device 20 may be a stereo camera.

[0014] The control device 10A communicates with the imaging device 20 via the communication line L1, thereby acquiring a captured image GI of the projection target PO from the imaging device 20. Note that the communication between the control device 10A and the imaging device 20 may be wireless communication. The control device 10A may be, for example, a personal computer or a tablet terminal.

[0015] The projector 30 projects a projection image PI onto a projection target PO. The projector 30 projects various projection images under the control of the control device 10A. The control device 10A communicates with the projector 30 via the communication line L2, causing the projector 30 to project the projection image PI onto the projection target PO. Note that the communication between the control device 10A and the projector 30 may be wireless communication.

[0016] 1, for convenience of explanation, the control device 10A, the image capture device 20, and the projector 30 are depicted as separate entities. However, in the projection device 1A, two or more of these components may be incorporated into a single housing.

[0017] 1-1-2: Overall configuration of the control device 2 is a block diagram showing an example configuration of the control device 10A. The control device 10A is typically a PC (Personal Computer), but is not limited to this and may be, for example, a tablet terminal or a smartphone. The control device 10A includes a processing device 120A, a storage device 140A, a display device 150, an input device 160, and a communication device 170. The elements of the control device 10A are connected to each other by one or more buses for communicating information.

[0018] The processing device 120A is a processor that controls the entire control device 10A, and is composed of, for example, one or more chips. The processing device 120A is composed of, for example, a central processing unit (CPU) that includes an interface with peripheral devices, an arithmetic unit, and registers. Note that some or all of the functions of the processing device 120A may be realized by hardware such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The processing device 120A executes various processes in parallel or sequentially.

[0019] The storage device 140A is a recording medium that can be read and written by the processing device 120A, and stores a plurality of programs including a control program PR1A executed by the processing device 120A. The storage device 140A also stores image information indicating an original image RI of the projection image PI. The image information indicating the original image RI is an example of "first image information." The storage device 140A may be configured with at least one of, for example, a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), a RAM (Random Access Memory), etc. The storage device 140A may also be called a register, a cache, a main memory, a primary storage device, or the like.

[0020] The display device 150 is a device that displays images and text information. The display device 150 may be a display device separate from the other components of the control device 10A.

[0021] The input device 160 is a device that accepts operations from a user of the projection device 1A. For example, the input device 160 includes a keyboard, a touchpad, a touch panel, or a pointing device such as a mouse. Here, if the input device 160 includes a touch panel, it may also serve as the display device 150.

[0022] The communication device 170 is hardware serving as a transmitting / receiving device for communicating with other devices. The communication device 170 is also called, for example, a network device, a network controller, a network card, or a communication module. The communication device 170 may include a connector for wired connection and an interface circuit corresponding to the connector. The communication device 170 may also include a wireless communication interface. Examples of the connector and interface circuit for wired connection include those compliant with wired LAN (Local Area Network), IEEE 1394, and USB (Universal Serial Bus). Examples of the wireless communication interface include those compliant with wireless LAN, Bluetooth (registered trademark), etc.

[0023] The processing device 120A reads out and executes the control program PR1A from the storage device 140A, thereby functioning as a calibration unit 121, a detection unit 122, an identification unit 123, a calculation unit 124, an acquisition unit 125, a correction unit 126A, a first projection control unit 127, a generation unit 128A, a second projection control unit 129, and a determination unit 130A. Note that the control program PR1A may be transmitted from another device, such as a server that manages the control device 10A, via a communication network.

[0024] The calibration unit 121 performs a calibration process to detect a correspondence between the camera coordinate system of the image capture device 20 and the panel coordinate system of the projector 30. As an example, the calibration unit 121 causes the projector 30 to project a pattern image onto the projection target PO. The pattern image may be, for example, a chessboard pattern, a pattern of multiple circles, a Gray code pattern, or a sine wave pattern. Next, the calibration unit 121 causes the image capture device 20 to capture the pattern image projected onto the projection target PO. The calibration unit 121 detects a correspondence between the coordinate values ​​in the panel coordinate system of a first feature point in the pattern image projected from the projector 30 and the coordinate values ​​in the camera coordinate system of a second feature point in the captured image GI corresponding to the first feature point. The calibration unit 121 detects multiple correspondences that correspond one-to-one to multiple pairs of first feature points and second feature points. Furthermore, the calibration unit 121 detects a correspondence between the camera coordinate system and the panel coordinate system based on the multiple correspondences.

[0025] The calibration unit 121 may use other methods to detect the correspondence between the camera coordinate system of the image capture device 20 and the panel coordinate system of the projector 30. For example, the calibration unit 121 may detect the above correspondence by using a portion of a normal image having a characteristic luminance distribution, instead of a pattern image. Alternatively, the calibration unit 121 may detect the correspondence between all pixels in the camera coordinate system and all pixels in the panel coordinate system by a structured light method used in 3D scanning.

[0026] The detection unit 122 detects the projection target PO. Fig. 3 is an explanatory diagram of the detection operation of the detection unit 122. For example, the detection unit 122 causes the projector 30 to project white light onto a wall WL. Then, the detection unit 122 causes the imaging device 20 to capture an image of the wall or screen. Fig. 3A shows an image GI captured by the imaging device 20. The detection unit 122 acquires the captured image GI to detect the wall as the projection target PO.

[0027] In FIG. 2, the identification unit 123 detects a first obstacle region DA1 in the captured image GI, which indicates the region of an obstacle DO present in the projection region RM of the projection target PO. The obstacle DO is, for example, a protrusion, a depression, or a stain on the wall WL. As an example, in FIG. 3A, the detection unit 122 detects a non-white region in the captured image GI as the first obstacle region DA1. The first obstacle region DA1 corresponds to the region in the projection target PO where the obstacle DO exists. Furthermore, the identification unit 123 detects the region of the projection region RM other than the first obstacle region DA1 as the first obstacle-free region PA1. The first obstacle-free region PA1 corresponds to the region in the projection target PO where no obstacle DO exists. An area of ​​the projection target PO where no obstacle DO exists is an example of a “first area FA.” An area of ​​the projection target PO where an obstacle DO exists is an example of a “second area SA.” In other words, the identification unit 123 identifies the first area FA and the second area SA of the projection target PO. 3B shows an example of a first obstacle area DA1 and a first obstacle-free area PA1 detected by the detection unit 122. In FIG. 3B, the first obstacle area DA1 is shown in black, while the first obstacle-free area PA1 is shown in white.

[0028] The identification unit 123 may detect the first obstacle area DA1 using other methods. For example, as described above, if the image capture device 20 is a stereo camera, the identification unit 123 may calculate the three-dimensional shape of the projection target PO based on the images GI captured by the stereo camera, and detect the first obstacle area DA1 based on the calculated three-dimensional shape. Alternatively, the identification unit 123 may similarly calculate the three-dimensional shape of the projection area RM using a TOF (Time of Flight) sensor (not shown), and detect the first obstacle area DA1 based on the calculated three-dimensional shape.

[0029] 2, the calculation unit 124 applies the correspondence between the camera coordinate system and the panel coordinate system detected by the calibration unit 121 to the first obstacle area DA1 and first obstacle-free area PA1 in the camera coordinate system detected by the identification unit 123. As a result, the calculation unit 124 calculates second obstacle area information indicating the second obstacle area DA2, which is an obstacle area in the panel coordinate system, and second obstacle-free area information indicating the second obstacle-free area PA2, which is an obstacle-free area in the panel coordinate system.

[0030] Fig. 4 is an explanatory diagram of the calculation operation of the calculation unit 124. Fig. 4A shows the same first obstacle area DA1 and first obstacle-free area PA1 as Fig. 3B. Fig. 4B shows the above-mentioned second obstacle area DA2 and second obstacle-free area PA2. The calculation unit 124 calculates the second obstacle area DA2 and second obstacle-free area PA2 on the panel coordinate system shown in Fig. 4B by applying a correspondence relationship between the camera coordinate system and the panel coordinate system to the first obstacle area DA1 and first obstacle-free area PA1 on the camera coordinate system shown in Fig. 4A and performing projective transformation.

[0031] Alternatively, if the calibration unit 121 detects a correspondence between all pixels in the camera coordinate system and all pixels in the panel coordinate system using the structured light method used in 3D scanning, the calculation unit 124 may calculate the second obstacle area DA2 and the second obstacle-free area PA2 by applying the correspondence to all pixels in the first obstacle area DA1 and the first obstacle-free area PA1.

[0032] In FIG. 2, the acquisition unit 125 acquires first image information indicating an original image RI of a projection image PI to be projected onto the projection target PO. The projection image PI is a projection image that is primarily projected onto the projection target PO. The projection image PI is an image different from an instruction image II, which will be described later, that indicates the presence of an obstacle DO on the projection target PO. The projection image PI is an example of a "first projection image." The instruction image II, which will be described later, is an example of a "second projection image." The acquiring unit 125 may acquire the first image information stored in the storage device 140 A. Alternatively, the acquiring unit 125 may acquire the first image information input from an external device.

[0033] Furthermore, if the optical axis of the projector 30 is positioned obliquely with respect to the projection target PO, the projection image PI is projected onto the projection target PO, and as a result, the display image DI displayed on the projection target PO will not be similar in shape to the original image RI indicated by the first image information and will be distorted. In order to make the original image RI and the display image DI similar in shape, it is necessary to generate the projection image PI by performing keystone correction on the original image RI using geometric transformation. The acquisition unit 125 acquires keystone correction information required for the keystone correction.

[0034] The correction unit 126A generates a projection image PI by performing keystone correction on the original image RI indicated by the first image information acquired by the acquisition unit 125 in the panel coordinate system, using the keystone correction information also acquired by the acquisition unit 125. Then, the correction unit 126A corrects the projection image PI by adjusting the shape of the projection image PI so that the projection image PI fits within the second obstacle-free area PA2 while avoiding the second obstacle area DA2 in the panel coordinate system.

[0035] FIG. 5 is an explanatory diagram of the correction operation of the correction unit 126A. FIG. 5A shows the second obstacle area DA2 and second obstacle-free area PA2, which are the same as those in FIG. 4B. As shown in FIG. 5B, the correction unit 126A detects, within the second obstacle-free area PA2, a maximum area MM that has the aspect ratio of the display screen of the projection target PO and a shape similar to the shape obtained by trapezoidal correction of the original image RI. As an example, the correction unit 126A detects the second obstacle-free area PA2 from the upper left to the lower right of the second obstacle-free area PA2, and determines the largest area that satisfies the desired aspect ratio as the maximum area MM. The maximum area MM may be a part or the entire second obstacle-free area PA2.

[0036] Thereafter, as shown in FIG. 5C, the correction unit 126A adjusts the shape of the projection image PI to match the size and shape of the maximum area MM.

[0037] 2, the first projection control unit 127 causes the projector 30 to project the projection image PI corrected by the correction unit 126A onto a first area FA of the projection target PO. Note that the first projection control unit 127 may cause the projector 30 to project only the projection image PI after correction by the correction unit 126A onto the projection target PO. Alternatively, the first projection control unit 127 may cause the projector 30 to continuously project the projection image PI onto the projection target PO while the correction by the correction unit 126A is in progress.

[0038] As described above, in the panel coordinate system, the maximum area MM may be a part or the whole of the second obstacle-free area PA2. Therefore, the first projection control unit 127 may project the projection image PI onto a part of the first area FA of the projection target PO where no obstacle DO exists, or may project the projection image PI onto the whole of the first area FA.

[0039] The generation unit 128A generates an indication image II indicating that an obstacle DO is present on the projection target PO.

[0040] FIG. 6 is a diagram showing an example of a projection image PI and an instruction image II in a panel coordinate system. FIG. 6A is an example of the same projection image PI as FIG. 5C. FIG. 6B is an example of an instruction image II. As shown in FIG. 6B, the instruction image II may be, for example, a line image LN that surrounds part or all of the second obstacle area DA2 in the panel coordinate system, as shown in FIG. 4B. In the example shown in FIG. 4B, the line image LN is displayed as a dotted line. As a result, as will be described later, part or all of the second area SA is surrounded by the line image LN on the projection target PO.

[0041] 6C shows both the projection image PI and the instruction image II in the panel coordinate system. As shown in FIG. 6C, both the projection image PI and the instruction image II are formed on a liquid crystal panel (to be described later) provided in the optical device 210 of the projector 30.

[0042] 2, the second projection control unit 129 causes the projector 30 to project the instruction image II onto the projection target PO. As an example, the second projection control unit 129 causes the projector 30 to project the instruction image II onto a second area SA on the projection target PO.

[0043] The second projection control unit 129 may project the instruction image II while the correction unit 126A is correcting the projection image PI, and may terminate the projection of the instruction image II after the correction is complete. In this case, the second projection control unit 129 may change the projection mode of the instruction image II while the correction unit 126A is correcting the projection image PI. For example, at least one of the shape, hue, lightness, saturation, and luminance of the instruction image II may change during the correction. Alternatively, the instruction image II may blink.

[0044] Alternatively, the second projection control unit 129 may not project the instruction image II while the correction unit 126A is correcting the projection image PI, and may project the instruction image II after the correction unit 126A has completed correcting the projection image PI.

[0045] The second projection control unit 129 may also change the projection mode of the instruction image II over time. For example, at least one of the shape, hue, brightness, saturation, and luminance of the instruction image II may change over time. Alternatively, the instruction image II may blink.

[0046] In the above description, for convenience of explanation, it has been assumed that the first projection control unit 127 projects the projection image PI shown in Fig. 6A, and the second projection control unit 129 projects the instruction image II shown in Fig. 6B. However, a single projection control unit may cause the projector 30 to project both the projection image PI and the instruction image II shown in Fig. 6C onto the projection target PO.

[0047] In addition, in this disclosure, the projection image PI as the first projection image and the instruction image II as the second projection image simply refer to the images themselves. The "projection image" in this disclosure does not simply refer to white light projected from the projector 30 onto the projection target PO.

[0048] 7 to 10 are diagrams showing examples of a projection image PI (display image DI) and an instruction image II displayed on a projection target PO.

[0049] For example, as shown in FIG. 7, the instruction image II may be a line image LN that surrounds the second area SA.

[0050] 8, the instruction image II may be a line image LN surrounding the second area SA and a visual effect EF surrounding the second area SA. The visual effect EF may be, for example, a flashing image or a blinking image.

[0051] Alternatively, as shown in FIG. 9, the instruction image II may be a color image PT that fills the second area SA with a single color.

[0052] Alternatively, as shown in FIG. 10, the instruction image II may be the letters LT indicating that an obstacle DO is present.

[0053] Alternatively, the instruction image II may be a combination of various instruction images II exemplified in FIGS.

[0054] 2, the determination unit 130A determines whether or not the operation of correcting the projection image PI has been completed. Details of the operation of the determination unit 130A, particularly cooperation with the operations of other components, will be described later in the description of the flowcharts in FIGS. 12 and 13 and 15 and 16.

[0055] 1-2: Projector configuration 11 is a block diagram showing an example of the configuration of the projector 30. The projector 30 includes an optical device 310, a processing device 320, a storage device 330, and a communication device 340. The elements of the projector 30 are connected to one another by one or more buses for communicating information. Furthermore, the elements of the projector 30 are configured by one or more devices, and some elements of the projector 30 may be omitted.

[0056] The optical device 310 is a device that projects an image represented by an image signal acquired by the acquisition unit 221 (described later) onto a projection target PO such as a screen or a wall. The optical device 310 projects various images under the control of the processing device 320. The optical device 310 includes, for example, a light source, a liquid crystal panel, and a projection lens, and modulates light from the light source using the liquid crystal panel and projects the modulated light onto a screen, a wall, or the like via the projection lens.

[0057] The processing device 320 is a processor that controls the entire projector 30, and is configured, for example, by one or more chips. The processing device 320 is configured, for example, by a central processing unit (CPU) that includes an interface with peripheral devices, an arithmetic unit, a register, etc. Note that some or all of the functions of the processing device 320 may be realized by hardware such as a DSP, ASIC, PLD, and FPGA. The processing device 320 executes various processes in parallel or sequentially.

[0058] The storage device 330 is a recording medium readable by the processing device 320, and stores a plurality of programs including the control program PR3 executed by the processing device 320. The storage device 330 may be configured with at least one of, for example, a ROM, an EPROM, an EEPROM, and a RAM. The storage device 330 may also be called a register, a cache, a main memory, a primary storage device, or the like.

[0059] The communication device 340 is hardware serving as a transmitting / receiving device for communicating with other devices. The communication device 340 is also called, for example, a network device, a network controller, a network card, or a communication module. The communication device 340 may include a connector for wired connection and an interface circuit corresponding to the connector. The communication device 340 may also include a wireless communication interface. Examples of the connector and interface circuit for wired connection include those compliant with wired LAN, IEEE1394, and USB. Examples of the wireless communication interface include those compliant with wireless LAN, Bluetooth (registered trademark), etc.

[0060] The processing device 320 reads and executes the control program PR3 from the storage device 330, thereby functioning as an acquisition unit 321 and a projection control unit 322. The control program PR3 may be transmitted from another device, such as a server that manages the projector 30, via a communication network.

[0061] The acquisition unit 321 acquires image signals corresponding to the projection image PI and the instruction image II, and a control signal for controlling the projector 30, from the control device 10A.

[0062] The projection control unit 322 causes the optical device 310 to project the projection image PI and the instruction image II corresponding to the image signal acquired by the acquisition unit 321 onto the object based on the control signal acquired by the acquisition unit 321.

[0063] 1-3: Operation of the control device 10A 1-3-1: First movement 12 and 13 are flowcharts showing a first operation of the control device 10A.

[0064] In step S1, the processing device 120A functions as the calibration unit 121. The processing device 120A detects the correspondence between the camera coordinate system in the image capture device 20 and the panel coordinate system in the projector 30 by executing a calibration process.

[0065] In step S2, the processing device 120A functions as the detection unit 122. The processing device 120A detects the projection object PO.

[0066] In step S3, the processing device 120A functions as the identification unit 123. The processing device 120A detects, from within the captured image GI, a first obstacle area DA1 that indicates the area of ​​the obstacle DO that exists within the projection target PO, within the projection area RM.

[0067] In step S4, the processing device 120A functions as the identification unit 123. The processing device 120A detects, in the captured image GI, the area of ​​the projection area RM other than the first obstacle area DA1 as a first obstacle-free area PA1. As a result of steps S3 and S4, the processing device 120A identifies a first area FA and a second area SA on the projection object PO.

[0068] In step S5, the processing device 120A functions as the calculation unit 124. The processing device 120A applies the correspondence relationship detected in step S1 to the first obstacle area DA1 on the camera coordinate system detected in step S3 and the first obstacle-free area PA1 detected in step S4. As a result, the processing device 120A calculates second obstacle area information indicating the second obstacle area DA2 on the panel coordinate system and second obstacle-free area information indicating the second obstacle-free area PA2.

[0069] In step S6, the processing device 120A functions as the acquisition unit 125. The processing device 120A acquires first image information indicating an original image RI of the projection image PI. The processing device 120A also acquires keystone correction information required for keystone correction.

[0070] In step S7, the processing device 120A functions as the correction unit 126A. The processing device 120A generates a projection image PI by performing keystone correction on the original image RI represented by the first image information acquired in step S6 in the panel coordinate system using the keystone correction information also acquired in step S6. The processing device 120A also adjusts the shape of the projection image PI in the panel coordinate system so that the projection image PI fits within the second obstacle-free area PA2 while avoiding the second obstacle area DA2. Note that these corrections are automatic corrections that do not involve any operation by the user of the projection device 1A.

[0071] In step S8, the processing device 120A functions as the determination unit 130A. The processing device 120A determines whether or not the automatic correction of the projection image PI has been completed. If the determination result is positive ("YES" in step S8), the processing device 120A executes the process of step S9. If the determination result is negative ("NO" in step S8), the processing device 120A executes the process of step S7.

[0072] In step S9, the processing device 120A functions as the first projection control unit 127. The processing device 120A causes the projector 30 to project the corrected projection image PI onto the first area FA of the projection target PO.

[0073] In step S10, the processing device 120A functions as the generating unit 128A. The processing device 120A generates an indication image II indicating that an obstacle DO exists on the projection target PO.

[0074] In step S11, the processing device 120A functions as the second projection control unit 129. The processing device 120A causes the projector 30 to project the instruction image II generated in step S10 onto the projection target PO.

[0075] In step S12, the processing device 120A functions as the correction unit 126A. The processing device 120A further corrects the projection image PI based on an operation by the user of the projection device 1A on the input device 160. This correction is a manual correction that accompanies an operation by the user of the projection device 1A.

[0076] In step S13, the processing device 120A functions as a determination unit 130A. The processing device 120A determines whether or not manual correction of the projection image PI has been completed. If the determination result is positive ("YES" in step S13), the processing device 120A executes the process of step S14. If the determination result is negative ("NO" in step S13), the processing device 120A executes the process of step S12.

[0077] In step S14, the processing device 120A functions as the second projection control unit 129. The processing device 120A causes the projector 30 to end the projection of the instruction image II.

[0078] 14 is a diagram showing an example of a projection image PI (display image DI) and an instruction image II displayed on the projection target PO during the first operation. In FIG. 14, adjustment icons CI, which are displayed during manual correction, are displayed at the four corners of the projection image PI (display image DI). In FIG. 14, the position where the original image RI was located is indicated by a dotted line. Furthermore, the processing device 120A may function as a display control unit (not shown) to cause the display device 150 to display the projection image PI (display image DI), instruction image II, and adjustment icon CI similar to those shown in FIG. 14. In addition, upon completion of manual correction, the projection of the instruction image II also ends.

[0079] 1-3-2: Second movement 15 and 16 are flowcharts showing a second operation of the control device 10A.

[0080] Steps S21 to S26 are the same as steps S1 to S6 in the first operation, and therefore their explanation will be omitted.

[0081] In step S27, the processing device 120A functions as the first projection control unit 127. The processing device 120A causes the projector 30 to project the projection image PI onto the first area FA of the projection target PO.

[0082] In step S28, the processing device 120A functions as a generating unit 128A. The processing device 120A generates an indication image II indicating that an obstacle DO exists on the projection target PO.

[0083] In step S29, the processing device 120A functions as the second projection control unit 129. The processing device 120A causes the projector 30 to project the instruction image II generated in step S onto the projection target PO.

[0084] In step S30, the processing device 120A functions as the correction unit 126A. The processing device 120A generates a projection image PI by performing keystone correction on the original image RI represented by the first image information acquired in step S6 in the panel coordinate system using the keystone correction information also acquired in step S6. The processing device 120A also adjusts the shape of the projection image PI in the panel coordinate system so that the projection image PI fits within the second obstacle-free area PA2 while avoiding the second obstacle area DA2. Note that these corrections are automatic corrections that do not involve any operation by the user of the projection device 1A.

[0085] In step S31, the processing device 120A functions as the determination unit 130A. The processing device 120A determines whether or not the automatic correction of the projection image PI has been completed. If the determination result is positive ("YES" in step S31), the processing device 120A executes the process of step S32. If the determination result is negative ("NO" in step S31), the processing device 120A executes the process of step S30.

[0086] In step S32, the processing device 120A functions as the second projection control unit 129. The processing device 120A causes the projector 30 to end the projection of the instruction image II.

[0087] Fig. 17 is a diagram showing an example of a projection image PI (display image DI) and an instruction image II displayed on the projection target PO during the second operation. As shown in Fig. 17, during the second operation, the projection image PI corrected from the original image RI is displayed in animation as the correction progresses. While the animation showing the correction process is displayed, the instruction image II is displayed. When the correction is completed, the projection of the instruction image II ends.

[0088] 2: Second embodiment 2-1: Configuration of the second embodiment The projection method and projection device according to the second embodiment will be described below with reference to Figures 18 and 19. For the sake of simplicity, the following description will focus on the differences between the projection method and projection device according to the second embodiment and the projection method and projection device according to the first embodiment. Furthermore, among the components provided in the projection device according to the second embodiment, the same components as those provided in the projection device according to the first embodiment will be denoted by the same reference numerals, and a description of their functions may be omitted.

[0089] 2-1-1: Overall configuration of the projection device Projection device 1B according to the second embodiment of the present disclosure differs from projection device 1A according to the first embodiment in that it includes control device 10B instead of control device 10A. In other respects, the overall configuration of projection device 1B is the same as the overall configuration of projection device 1A, and therefore is not shown in the drawings.

[0090] 2-1-2: Overall configuration of the control device 18 is a block diagram showing an example of the configuration of the control device 10B. The control device 10B differs from the control device 10A in that it includes a processing device 120B instead of the processing device 120A and a storage device 140B instead of the storage device 140A.

[0091] The storage device 140B differs from the storage device 140A in that it includes a control program PR1B instead of the control program PR1A.

[0092] The processing device 120B reads and executes the control program PR1B from the storage device 140B, thereby functioning as a calibration unit 121, a detection unit 122, an identification unit 123, a calculation unit 124, an acquisition unit 125, a correction unit 126A, a first projection control unit 127, a generation unit 128B, a second projection control unit 129, and a determination unit 130B. Note that the control program PR1B may be transmitted from another device, such as a server that manages the control device 10B, via a communication network.

[0093] 19 is a functional block diagram of the determination unit 130B. The determination unit 130B includes a first determination unit 130B[1] and a second determination unit 130B[2].

[0094] The first determination section 130B[1] determines whether or not the operation of correcting the projection image PI has ended, similarly to the determination section 130A according to the first embodiment.

[0095] The second determination unit 130B[2] determines whether the size of the second area SA is larger than the size of the third area TA, which is the area of ​​the projection image PI (display image DI) on the projection target PO. More specifically, the second determination unit 130B[2] determines whether the number of pixels of the second obstacle area DA2 is larger than the number of pixels of the projection image PI in the panel coordinate system. Based on the determination result, the second determination unit 130B[2] determines whether the size of the second area SA is larger than the size of the third area TA on the projection target PO. The third area TA is the area of ​​the first area FA onto which the projection image PI is projected.

[0096] In FIG. 18, the generation unit 128B generates an indication image II indicating that an obstacle DO is present on the projection target PO, similar to the generation unit 128A.

[0097] Furthermore, the generation unit 128B changes the type of the instruction image II based on the determination result of the second determination unit 130B[2]. That is, the type of the instruction image II differs depending on the size relationship between the second area SA and the third area TA.

[0098] For example, if the size of the second area SA is larger than the size of the third area TA, the instruction image II will be a line image LN surrounding the second area SA. On the other hand, if the size of the second area SA is equal to or smaller than the size of the third area TA, the instruction image II will be a color image PT in which the second area SA is filled with one color.

[0099] 2-2: Operation of the control device 2-2-1: First operation The first operation of the control device 10B is similar to the first operation shown in FIGS. 12 and 13 except for the following points, and therefore will not be illustrated.

[0100] In step S10 of the first operation of the control device 10B, the processing device 120B functions as a second determination unit 130B[2]. The processing device 120B determines whether the size of the second area SA is larger than the size of the third area TA. The processing device 120B also functions as a generation unit 128B. The processing device 120B generates an indication image II indicating that an obstacle DO is present on the projection target PO. At this time, the processing device 120B changes the type of the indication image II based on the above determination result.

[0101] 2-2-2: Second operation The second operation of the control device 10B is similar to the second operation shown in FIGS. 15 and 16 except for the following points, and therefore will not be illustrated.

[0102] In step S28 of the second operation of the control device 10B, the processing device 120B functions as a second determination unit 130B[2]. The processing device 120B determines whether the size of the second area SA is larger than the size of the third area TA. The processing device 120B also functions as a generation unit 128B. The processing device 120B generates an indication image II indicating that an obstacle DO is present on the projection target PO. At this time, the processing device 120B changes the type of the indication image II based on the above determination result.

[0103] 3: Third embodiment 3-1: Configuration of the third embodiment The projection method and projection device according to the third embodiment will be described below with reference to Fig. 20. For the sake of simplicity, only the differences between the projection method and projection device according to the third embodiment and those according to the first embodiment will be described. Furthermore, among the components of the projection device according to the third embodiment, the same components as those of the projection device according to the first embodiment will be designated by the same reference numerals, and a description of their functions may be omitted.

[0104] 3-1-1: Overall configuration of the projection device Projection device 1C according to the third embodiment of the present disclosure differs from projection device 1A according to the first embodiment in that it includes control device 10C instead of control device 10A. In other respects, the overall configuration of projection device 1C is the same as the overall configuration of projection device 1A, and therefore is not shown in the drawings.

[0105] 3-1-2: Overall configuration of the control device 20 is a block diagram showing an example of the configuration of the control device 10C. The control device 10C differs from the control device 10A in that it includes a processing device 120C instead of the processing device 120A and a storage device 140C instead of the storage device 140A.

[0106] The storage device 140C differs from the storage device 140A in that it includes a control program PR1C instead of the control program PR1A, and also includes a learning model LM.

[0107] The learning model LM is a learning model used by the obstacle detection unit 131, which will be described later, to detect the type of obstacle DO. The learning model LM is generated by learning training data in the learning phase. The training data used to generate the learning model LM has a plurality of pairs of captured images GI of obstacles DO and the types of the obstacles DO. When the obstacle detection unit 131 inputs the captured image GI of the projection target PO to the learning model LM, the type of obstacle DO present at the projection target PO is output from the learning model LM.

[0108] The processing device 120C reads out and executes the control program PR1C from the storage device 140A, thereby functioning as a calibration unit 121, a detection unit 122, an identification unit 123, a calculation unit 124, an acquisition unit 125, a correction unit 126A, a first projection control unit 127, a generation unit 128C, a second projection control unit 129, a determination unit 130A, and an obstacle detection unit 131. Note that the control program PR1C may be transmitted from another device, such as a server that manages the control device 10C, via a communication network.

[0109] The obstacle detection unit 131 detects the type of obstacle DO present at the projection target PO based on the captured image GI of the projection target PO. Specifically, the obstacle detection unit 131 inputs the captured image GI of the projection target PO to a learning model LM, and acquires the type of obstacle DO output from the learning model LM.

[0110] The obstacle detection unit 131 may also detect the color of an obstacle DO present on the projection target PO based on a captured image GI of the projection target PO.

[0111] Similar to the generation unit 128A, the generation unit 128C generates an indication image II indicating that an obstacle DO is present on the projection target PO.

[0112] Furthermore, the generation unit 128C changes the type of the instruction image II based on the type of the obstacle DO detected by the obstacle detection unit 131. That is, the type of the instruction image II differs depending on the type of the obstacle DO.

[0113] Furthermore, when the obstacle detection unit 131 detects the color of the obstacle DO, the generation unit 128C changes the type of the instruction image II based on the color of the obstacle DO detected by the obstacle detection unit 131. That is, the type of the instruction image II differs depending on the color of the obstacle DO.

[0114] 3-2: Operation of the control device 3-2-1: First action The first operation of the control device 10C is similar to the first operation shown in FIGS. 12 and 13 except for the following points, and therefore will not be illustrated.

[0115] In step S10 of the first operation of the control device 10C, the processing device 120C functions as an obstacle detection unit 131. The processing device 120C detects the type of obstacle DO present in the projection target PO. The processing device 120C also functions as a generation unit 128C. The processing device 120C generates an indication image II indicating that an obstacle DO is present on the projection target PO. At this time, the processing device 120C changes the type of the indication image II based on the type of the obstacle DO.

[0116] Alternatively, in step S10 of the first operation of the control device 10C, the processing device 120C may function as the obstacle detection unit 131 to detect the color of the obstacle DO present on the projection target PO. In this case, the processing device 120C changes the type of the instruction image II based on the color of the obstacle DO.

[0117] 3-2-2: Second movement The second operation of the control device 10C is similar to the second operation shown in FIGS. 15 and 16 except for the following points, and therefore will not be illustrated.

[0118] In step S28 of the second operation of the control device 10C, the processing device 120C functions as the obstacle detection unit 131. The processing device 120C detects the type of obstacle DO present in the projection target PO. The processing device 120C also functions as a generation unit 128C. The processing device 120C generates an indication image II indicating that an obstacle DO is present on the projection target PO. At this time, the processing device 120C changes the type of the indication image II based on the type of the obstacle DO.

[0119] Alternatively, in step S28 of the second operation of the control device 10C, the processing device 120C may function as the obstacle detection unit 131 to detect the color of the obstacle DO present on the projection target PO. In this case, the processing device 120C changes the type of the instruction image II based on the color of the obstacle DO.

[0120] 4: Fourth embodiment 4-1: Configuration of the fourth embodiment 21 to 24, a projection method and projection device according to the fourth embodiment will be described. For the sake of simplicity, only the differences between the projection method and projection device according to the fourth embodiment and those according to the first embodiment will be described. Furthermore, among the components of the projection device according to the fourth embodiment, the same components as those of the projection device according to the first embodiment will be designated by the same reference numerals, and a description of their functions may be omitted.

[0121] 4-1-1: Overall configuration of the projection device Projection device 1D according to the fourth embodiment of the present disclosure differs from projection device 1A according to the first embodiment in that it includes control device 10D instead of control device 10A. In other respects, the overall configuration of projection device 1D is the same as the overall configuration of projection device 1A, and therefore is not shown in the drawings.

[0122] 4-1-2: Overall configuration of the control device 21 is a block diagram showing an example of the configuration of the control device 10D. The control device 10D differs from the control device 10A in that it includes a processing device 120D instead of the processing device 120A and a storage device 140D instead of the storage device 140A.

[0123] The storage device 140D differs from the storage device 140D in that it includes a control program PR1D instead of the control program PR1A.

[0124] The processing device 120D reads out and executes the control program PR1D from the storage device 140D, thereby functioning as a calibration unit 121, a detection unit 122, an identification unit 123, a calculation unit 124, an acquisition unit 125, a correction unit 126D, a first projection control unit 127, a generation unit 128A, a second projection control unit 129, a determination unit 130D, and a motion detection unit 132. Note that the control program PR1D may be transmitted from another device, such as a server that manages the control device 10D, via a communication network.

[0125] The motion detection unit 132 detects a motion of the user based on a captured image GM of the projection target PO and the user of the projection device 1D located near the projection target, captured by the imaging device 20. More specifically, the motion detection unit 132 detects a motion made by the user using a pointer. The pointer is, for example, the user's hand, index finger, or a tool held by the user. Note that the above "nearby" means within a range in which the user fits within the captured image GM in which the projection target PO is captured.

[0126] 22 is a functional block diagram of the determination unit 130D. The determination unit 130D includes a first determination unit 130D[1], a second determination unit 130D[2], and a third determination unit 130D[3].

[0127] The first determination section 130D[1] determines whether or not the operation of correcting the projection image PI has ended, similarly to the determination section 130A according to the first embodiment.

[0128] The second determination unit 130D[2] determines whether or not the pointing image II is indicated by the above-mentioned pointing object. This determination is an example of a "first determination."

[0129] The third determination unit 130D[3] determines whether the indicator is performing a predetermined action. This determination is an example of a "second determination." The predetermined action is an example of a "first action."

[0130] Similar to correction unit 126A, correction unit 126D generates projection image PI in the panel coordinate system by performing keystone correction on original image RI indicated by the first image information acquired by acquisition unit 125, using keystone correction information also acquired by acquisition unit 125. Then, correction unit 126A adjusts the shape of projection image PI in the panel coordinate system so that projection image PI fits within second obstacle-free area PA2 while avoiding second obstacle area DA2.

[0131] The correction unit 126D further changes the size of the projection image PI so that the projection image PI is projected onto the first area FA and the second area SA when both the first judgment result and the second judgment result are positive.

[0132] For example, when a projection image PI (display image DI) and an instruction image II as illustrated in Figures 7 to 10 are displayed on a projection target PO, if a user of the projection device 1D points their index finger at the instruction image II in front of the projection target PO and then waves the index finger, the projection image PI (display image DI) expands to cover part of the first area FA and the second area SA.

[0133] 4-2: Operation of the control device 4-2-1: First action 23 and 24 are flowcharts showing a first operation of the control device 10D.

[0134] Steps S41 to S50 are the same as steps S21 to S30 in the second operation according to the first embodiment, and therefore a description thereof will be omitted.

[0135] In step S51, the processing device 120D functions as a first determination unit 130D[1]. The processing device 120D determines whether or not the automatic correction of the projection image PI has been completed. If the determination result is positive ("YES" in step S51), the processing device 120D executes the process of step S52. If the determination result is negative ("NO" in step S51), the processing device 120D executes the process of step S50.

[0136] In step S52, the processing device 120D functions as the motion detection unit 132. The processing device 120D detects a motion of the user based on an image GM of the projection target PO and the user of the projection device 1D located near the projection target, captured by the imaging device 20. More specifically, the processing device 120D detects a motion made by the user using a pointer.

[0137] In step S53, the processing device 120D functions as a second determination unit 130D[2]. The processing device 120D determines whether or not the pointing image II is indicated by the pointer. If the determination result is positive ("YES" in step S53), the processing device 120D executes the process of step S54. If the determination result is negative ("NO" in step S53), the processing device 120D executes the process of step S56.

[0138] In step S54, the processing device 120D functions as a third determination unit 130D[3]. The processing device 120D determines whether or not the indicator is performing a predetermined action. If the determination result is positive ("YES" in step S54), the processing device 120D executes the process of step S55. If the determination result is negative ("NO" in step S54), the processing device 120D executes the process of step S56.

[0139] In step S55, the processing device 120D functions as the correction unit 126D. The processing device 120D changes the size of the projection image PI so that the projection image PI is projected onto the first area FA and the second area SA.

[0140] In step S56, the processing device 120D functions as the second projection control unit 129. The processing device 120D causes the projector 30 to end the projection of the instruction image II.

[0141] 5: Variation The above-described embodiments can be modified in various ways. Specific modifications that can be applied to the above-described embodiments are exemplified below. Two or more embodiments arbitrarily selected from the following examples can be combined as appropriate within the scope of not mutually contradictory.

[0142] 5-1: Variation 1 At least some of the operations performed by the control devices 10A to 10D may be executed by at least one of the processing device 320 included in the projector 30 and the processing device included in the image capturing device 20.

[0143] 6: Summary of this disclosure A summary of this disclosure is provided below.

[0144] (Appendix 1) A projection method comprising: detecting a projection object; identifying a first area of ​​the projection object where no obstacle exists and a second area of ​​the projection object where the obstacle exists based on the result of detecting the projection object; projecting a first projection image onto at least a part of the first area; and projecting a second projection image that is different from the first projection image and indicates that the obstacle exists on the projection object.

[0145] This makes it clear which object the projection device 1 has recognized as the obstacle DO, allowing the user to determine whether appropriate control is being performed. More specifically, the projection device 1 projects a projection image PI onto a first area FA where no obstacle DO exists, and projects an indication image II indicating the presence of an obstacle DO separately from the projection image PI. By viewing the indication image II, the user can understand which specific object the projection device 1 has recognized as the obstacle DO.

[0146] (Supplementary Note 2) The projection method according to Supplementary Note 1, wherein the second projection image is projected onto the second area.

[0147] As a result, the projection device 1 projects the projection image PI onto the first area FA where no obstacle DO exists, and projects the instruction image II indicating the presence of the obstacle DO onto the second area SA where the obstacle DO exists. By viewing the instruction image II, the user can understand which specific object the projection device 1 has recognized as an obstacle.

[0148] (Supplementary Note 3) The projection method according to Supplementary Note 1 or Supplementary Note 2, wherein the second projection image is a line image that surrounds at least a part of the second region.

[0149] This allows the user to easily determine the size of the second area SA. Furthermore, since the instruction image II is merely a line image LN, the degree to which the instruction image II affects the user's visual recognition of the projection image PI can be reduced.

[0150] (Appendix 4) A projection method described in any one of Appendices 1 to 3, further comprising determining whether the size of the second area is larger than the size of a third area of ​​the first area onto which the first projection image is projected, and wherein the type of the second projection image differs depending on the size relationship between the second area and the third area.

[0151] This allows the projection device 1 to change the type of the instruction image II depending on the size of the second area SA. As a result, the projection device 1 can control the degree of influence of the instruction image II on the user's recognition of the projection image PI depending on the relative size between the second area SA and the third area TA.

[0152] (Appendix 5) The projection method described in Appendix 4, characterized in that when the size of the second area is larger than the size of the third area, the second projection image is a line image surrounding the second area, and when the size of the second area is equal to or smaller than the size of the third area, the second projection image is a color image in which the second area is filled with one color.

[0153] As a result, when the size of the second area SA is larger than the size of the third area TA, the line image LN is projected as the instruction image II, and the projection device 1 can prevent the instruction image II from being more conspicuous than necessary.

[0154] (Appendix 6) A projection method described in any one of Appendices 1 to 5, further comprising detecting the color of the obstacle, wherein the color of the second projection image varies depending on the color of the obstacle.

[0155] As a result, the color of the instruction image II differs depending on the color of the obstacle DO, making it easier for the user to recognize the presence of the obstacle DO.

[0156] (Appendix 7) A projection method described in any one of Appendices 1 to 6, further comprising detecting the type of the obstacle, wherein the type of the second projection image differs depending on the type of the obstacle.

[0157] As a result, the type of the instruction image II varies depending on the type of the obstacle DO, making it easier for the user to recognize the presence of the obstacle DO.

[0158] (Appendix 8) A projection method described in any one of Appendices 1 to 7, further comprising adjusting the shape of the first projection image, wherein the second projection image is projected while the shape of the first projection image is being adjusted, and is not projected after the adjustment of the shape of the first projection image has been completed.

[0159] As a result, the instruction image II is not projected after the adjustment of the outer shape of the projection image PI is completed, which is the time when viewing of the projection image PI begins, thereby reducing the degree to which the instruction image II affects viewing of the projection image PI.

[0160] (Supplementary Note 9) The projection method according to any one of Supplementary Notes 1 to 8, wherein the projection mode of the second projection image is changed while the shape of the first projection image is being adjusted.

[0161] As a result, the instruction image II not only allows the user to grasp the area of ​​the projection target PO where the obstacle DO exists, but also functions as a means for the user to grasp that the projection image PI is undergoing geometric correction, thereby further improving the ease of use of the projection device 1 for the user.

[0162] (Supplementary Note 10) The projection method according to Supplementary Note 1, wherein the projection mode of the second projection image changes over time.

[0163] This makes it possible to increase the likelihood that the user will perceive the instruction image II by changing the projection mode of the instruction image II over time, which allows the user to more easily grasp which area of ​​the projection target PO is recognized as an area where an obstacle exists.

[0164] (Appendix 11) The projection method described in Appendix 1, further comprising: making a first determination as to whether the second projection image has been pointed to by an indicator; making a second determination as to whether the indicator is performing a first action; and, if both the results of the first determination and the second determination are positive, changing the size of the first projection image so that the first projection image is projected onto the first area and the second area.

[0165] This allows the projection device 1 to change the size of the projection image PI on the projection target PO in accordance with the movement of the pointer. In other words, if the user determines that the second area SA should also be included in the projection area of ​​the projection image PI, the user can specify the size of the projection image PI on the projection target PO by the movement of the pointer.

[0166] (Supplementary Note 12) A projection device including a processor that controls the following: detecting a projection target; identifying a first area of ​​the projection target where no obstacle exists and a second area of ​​the projection target where the obstacle exists based on the result of detecting the projection target; projecting a first projection image onto part or all of the first area; and projecting a second projection image that differs from the first projection image and indicates that the obstacle exists on the projection target.

[0167] This makes it clear which object the projection device 1 has recognized as the obstacle DO, allowing the user to determine whether appropriate control is being performed. More specifically, the projection device 1 projects a projection image PI onto a first area FA where no obstacle DO exists, and projects an indication image II indicating the presence of an obstacle DO separately from the projection image PI. By viewing the indication image II, the user can understand which specific object the projection device 1 has recognized as the obstacle DO. [Explanation of symbols]

[0168] 1, 1A, 1B, 1C, 1D: projection device, 10A, 10B, 10C, 10D: control device, 20: imaging device, 30: projector, 120A, 120B, 120C, 120D: processing device, 121: calibration unit, 122: detection unit, 123: identification unit, 124: calculation unit, 125: acquisition unit, 126A, 126D: correction unit, 127: first projection control unit, 128A, 128B, 12 8C: Generation unit, 129: Second projection control unit, 130A, 130B: Determination unit, 130B[1]: First determination unit, 130B[2]: Second determination unit, 130D: Determination unit, 130D[1]: First determination unit, 130D[2]: Second determination unit, 130D[3]: Third determination unit, 131: Obstacle detection unit, 132: Motion detection unit, 140A, 140B, 140C, 140D: Storage device, 150: Display device, 16 0: Input device, 170: Communication device, 210: Optical device, 221: Acquisition unit, 310: Optical device, 320: Processing device, 321: Acquisition unit, 322: Projection control unit, 330: Storage device, 340: Communication device, CI: Adjustment icon, DA1: First obstacle area, DA2: Second obstacle area, DI: Display image, DO: Obstacle, EF: Visual effect, FA: First area, GI: Captured image, GM: Captured video, II: Instruction image, L1, L2: Communication line, LM: Learning model, LN: Line image, LT: Character, MM: Maximum area, PA1: First obstacle-free area, PA2: Second obstacle-free area, PI: Projected image, PO: Projection target, PR1A, PR1B, PR1C, PR1D: Control program, PR3: Control program, PT: Color image, RI: Original image, RM: Projection area, SA: Second area, TA: Third area, WL: Wall

Claims

1. Detecting a projection target; Identifying a first area of ​​the projection target where no obstacle exists and a second area of ​​the projection target where the obstacle exists based on a result of detecting the projection target; projecting a first projection image onto at least a portion of the first area; projecting a second projection image different from the first projection image, the second projection image indicating the presence of the obstacle on the projection target; A projection method comprising:

2. The second projection image is projected onto the second area. The projection method according to claim 1 .

3. The second projection image is a line image surrounding at least a part of the second area. The projection method according to claim 1 .

4. The method further includes determining whether a size of the second area is larger than a size of a third area of ​​the first area onto which the first projection image is projected; The type of the second projection image differs depending on the size relationship between the second area and the third area. The projection method according to claim 1 .

5. When the size of the second region is larger than the size of the third region, the second projection image is a line image surrounding the second region, When the size of the second region is equal to or smaller than the size of the third region, the second projection image is a color image in which the second region is filled with one color. The projection method according to claim 4 .

6. further comprising detecting a color of the obstacle; The color of the second projection image varies depending on the color of the obstacle. The projection method according to claim 1 .

7. further comprising detecting a type of the obstacle; The type of the second projection image varies depending on the type of the obstacle. The projection method according to claim 1 .

8. adjusting a shape of the first projected image; the second projection image is projected while the shape of the first projection image is being adjusted, and is not projected after the adjustment of the shape of the first projection image is completed. The projection method according to claim 1 .

9. The projection mode of the second projection image is changed while the shape of the first projection image is being adjusted. The projection method according to claim 1 .

10. The projection mode of the second projection image changes over time. The projection method according to claim 1 .

11. making a first determination as to whether the second projection image is pointed to by a pointer; making a second determination as to whether the indicator is performing a first action; If both the first determination result and the second determination result are positive, changing the size of the first projection image so that the first projection image is projected onto the first area and the second area. The projection method according to claim 1 .

12. Detecting a projection target; Identifying a first area of ​​the projection target where no obstacle exists and a second area of ​​the projection target where the obstacle exists based on a result of detecting the projection target; projecting a first projection image onto a part or all of the first area; projecting a second projection image different from the first projection image, the second projection image indicating the presence of the obstacle on the projection target; The processor that controls A projection device comprising:

Images