Projection device and its control method

Abstract

To provide a technology that can keep a relation between a target image and an object image to be a user's intended relation when a layout of the target image is changed.SOLUTION: A projection device of the present invention comprises: projection means that projects a projection image having a target image arranged onto a projection surface; change means that changes a layout of the target image in the projection image; detection means that detects a user operation with respect to the projection surface; and overlap means that overlaps an object image responding to the user operation detected by the detection means on the projection image. When the layout of the target image is changed by the change means, the overlap means is configured to overlap the object image on the projection image so that a layout of the object image in the projection image is changed in accordance with the change in the layout of the target image.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a projection device and a control method thereof.

Background Art

[0002] There is a projection device (projector) called an interactive projector that generates an object image according to a user operation on an image (projection image) including a target image projected on a projection surface, and overlays and displays the object image on the projection image. The object image is, for example, an image including characters and figures drawn by a user operating on the projection surface with a pen, finger, or the like. This enables the user to obtain a feeling of drawing characters and figures on the image projected on the projection surface.

[0003] In addition, the projection device can change the layout (position, size, shape, etc.) of the target image in the projection image. For example, it is possible to change the layout from a state where the entire projection image is occupied by the target image to a state where the target image is displayed in the left half region of the projection image.

[0004] Patent Document 1 discloses a technique (projection device) for controlling so that the character size of text data added to a projection image does not change when the size of the projection image is optically changed.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] When the layout of the target image in the projected image is changed, the relationship (such as the relationship of position, size, shape, etc.) between the object image and the target image may shift. Specifically, assume that the user makes a writing (addition of an object image) to a target part of the target image for a projected image in which the entire projected image is occupied by the target image. In such a case, when the layout of the target image is changed, the position of the object image will shift from the position of the target part of the target image.

[0007] The projection device described in Patent Document 1 performs control so as not to change the character size of the text data to be overlaid when the size of the projected image changes. When such a technique is used, even if the layout (size) of the target image in the projected image changes, the size of the object image does not change. Therefore, as described above, the relationship between the target image and the object image after the layout change will shift.

[0008] An object of the present invention is to provide a technique capable of maintaining the relationship between the target image and the object image in accordance with the user's intention when the layout of the target image is changed.

Means for Solving the Problems

[0009] A first aspect of the present invention includes an acquisition unit that acquires a first input image and a second input image, a first projection process that projects the first input image, a second projection process that projects the first input image and the second input image arranged in parallel, a switching unit that switches the process from the first projection process to the second projection process, and a generation unit that generates an object image according to a user operation on a projection surface on which an image is projected by the projection unit. The projection unit superimposes and projects the object image generated by the generation unit on the first input image during execution of the first projection process, and when switched from the first projection process to the second projection process by the switching unit, projects the object image at a position and size corresponding to the relative position and relative size of the first input image and the object image in the first projection process. In the second projection process, in response to the layout of the first input image being changed, the object image is projected at a position and size corresponding to the changed layout. A projection device characterized by the above.

[0011] The second 2 aspect of the present invention includes an acquisition step of acquiring a first input image and a second input image, a first projection step of projecting the first input image, a second projection step of projecting the first input image and the second input image arranged in parallel, a switching step of switching the process from the first projection step to the second projection step, and a generation step of generating an object image according to a user operation on a projection surface on which an image is projected by the projection step. In the projection step, the object image generated in the generation step is superimposed and projected on the first input image during execution of the first projection step, and when switched from the first projection step to the second projection step by the switching step, the object image is projected at a position and size corresponding to the relative position and relative size of the first input image and the object image in the first projection step. In the second projection process, in response to the layout of the first input image being changed, the object image is projected at a position and size corresponding to the changed layout. A projection method characterized by the above. 。

Advantages of the Invention

[0012] According to the present invention, when the layout of the target image is changed, the relationship between the target image and the object image can be maintained in the relationship intended by the user.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Modes for Carrying Out the Invention

[0014] Hereinafter, embodiments of the present invention will be described.

[0015] <Configuration of the Projection System> FIG. 1 is a schematic diagram showing a configuration example of the projection system according to the present embodiment. As shown in FIG. 1, the projection system according to the present embodiment includes a projector 100 (projection device), a screen board 500 (projection surface), and an operation device 600 (indicating device). The projector 100 can project a projection image 300 onto the screen board 500. The projection image 300 has a screen area 400 where the screen is arranged. As shown in FIG. 1, the projection image 300 may also have a non-screen area 700 where the screen is not arranged. On the screen, there is a target image (projection target An image of an elephant) is placed. The projector 100 can change the layout (position, size, shape, etc.) of the screen area 400, the target image, etc. in the projection image 300. The user 200 can perform user operations on the screen board 500 using the operation device 600. The projector 100 projects an object image corresponding to the user operation on the screen board 500 onto the screen board 500 by superimposing it on the projection image 300. For example, the projector 100 projects an object image so that the user 200 can obtain a feeling of drawing characters and figures on the screen board 500.

[0016] In FIG. 1, the projector 100 is installed above the screen board 500, but the positional relationship between the projector 100 and the screen board 500 is not particularly limited. In this embodiment, an example in which the target image is arranged over the entire screen is described, but the target image may be arranged on a part of the screen. In this embodiment, an example in which the image (input image) input to the projector 100 is used as the target image is described, but an image stored in the projector 100, an image generated by the projector 100, etc. may be used as the target image. The target image may be a still image or a moving image. The target image is, for example, a photographed image, a CG (computer graphic) image, an image for presentation, an image for meeting, an image imitating a whiteboard, etc.

[0017] <Configuration of the projector> FIG. 2 is a block diagram showing a configuration example of the projector 100.

[0018] The control unit 101 controls the entire projector 100. For example, the control unit 101 is a microcomputer or the like. The ROM 102 is a non-volatile memory, and program codes, data, etc. for the operation of the control unit 101 are stored in advance in the ROM 102. Data necessary for the operation of the projector 100 may be newly stored in the ROM 102, or the data stored in the ROM 102 may be updated. The RAM 103 is a volatile memory and is used as a work memory for the operation of the control unit 101.

[0019] The detection unit 104 detects a user operation on the screen board 500 and outputs the detection result to the processing unit 107 (the object unit 111 described later). In the present embodiment, the operation device 600 can emit invisible light (infrared light, ultraviolet light, etc.) having a predetermined wavelength. The user 200 performs a user operation of irradiating the screen board 500 with invisible light from the operation device 600 as a user operation on the screen board 500. The detection unit 104 detects the invisible light emitted from the operation device 600 and reflected by the screen board 500. Then, the detection unit 104 notifies the processing unit 107 (the object unit 111 described later) of the coordinates of the position where the invisible light is detected (the position specified by the user operation on the screen board 500). Here, the coordinate system when the detection unit 104 detects the invisible light (for example, the coordinate system of the real space) may be different from the coordinate system when the processing unit 107 performs image processing (for example, the coordinate system of the image (image data)). In that case, the detection unit 104 performs coordinate conversion in order to output the coordinates of the position where the invisible light is detected in the coordinate system when the processing unit 107 performs image processing.

[0020] Note that the user operations on the screen board 500 are not limited to the above-described user operations. The method for detecting user operations on the screen board 500 is also not limited to the above-described method. For example, the operation device 600 may emit visible light having a predetermined wavelength, and the detection unit 104 may detect visible light having a predetermined wavelength. The user operation on the screen board 500 may be a user operation of bringing a finger, a pen, the operation device 600, etc. into contact with the screen board 500, or making the distance between the finger, the pen, the operation device 600, etc. and the screen board 500 equal to or less than a threshold value. Then, the detection unit 104 may detect a user operation on the screen board 500 based on an image obtained by imaging the screen board 500. This is acceptable. For example, the detection unit 104 may detect the operation device 600 from an image obtained by imaging the screen board 500 and output the coordinates of the position where the operation device 600 is detected. The imaging unit for imaging the screen board 500 may be provided in the projector 100, or may be provided in a device different from the projector 100 (for example, an imaging device).

[0021] The operation unit 105 receives user operations on the projector 100. For example, the operation unit 105 is a switch, a button, etc. provided on the housing of the projector 100. The operation unit 105 may be a receiving unit that receives a signal emitted from a remote controller or the like of the projector 100. When a user operation on the projector 100 is performed, the operation unit 105 outputs a control signal corresponding to the performed user operation to the control unit 101.

[0022] The connection unit 106 connects the projector 100 to an external device such as a personal computer, a tablet terminal, a smartphone, or a digital camera. The connection unit 106 may be a communication unit for wireless connection with the external device, or may be a connection terminal for wired connection with the external device. The connection unit 106 acquires image data (input image; target image) from the external device connected to the projector 100, and outputs the acquired image data to the processing unit 107 (layout unit 110 described later). The connection unit 106 can select at least any one of a plurality of external devices connected to the projector 100 according to an instruction from the control unit 101, and acquire image data from the selected external device. The connection unit 106 can also select and acquire at least any one of a plurality of image data stored in the external device according to an instruction from the control unit 101. Here, the data format of the image data output from the external device to the connection unit 106 may be different from the data format processable by the processing unit 107. In that case, the connection unit 106 performs format conversion in order to output the acquired image data in a data format processable by the processing unit 107. The image data is acquired and processed, for example, at a predetermined frame rate. One frame of image data (image signal) includes a gradation signal indicating the gradation value of each pixel and a synchronization signal indicating the timing of the corresponding frame.

[0023] The processing unit 107 performs various image processes according to an instruction from the control unit 101, and generates a projection image 300 (image data) on which the input image is arranged. Then, the processing unit 107 outputs the projection image 300 (image data) to the projection unit 108. As image processes, the processing unit 107 can execute gradation conversion processes (gain process, offset process, gamma process, etc.), inter-frame thinning processes, frame interpolation processes, resolution conversion processes, image composition processes, etc. As image processes, the processing unit 107 can also execute geometric correction processes (keystone correction process; curved surface correction process), panel correction processes, layout processes, superimposition processes, object generation processes, etc.

[0024] The projection unit 108 projects the projection image 300 onto the screen board 500 based on the image data output from the processing unit 107. For example, the projection unit 108 includes a projection control unit and a projection processing unit. The projection control unit controls the projection processing unit with a control signal based on the image data output from the processing unit 107 so that the projection of the projection image 300 is performed. The projection processing unit performs projection based on the control signal output from the projection control unit. Specifically, the projection processing unit includes a liquid crystal panel, a lens, and a light source. The projection processing unit may include one liquid crystal panel or a plurality of liquid crystal panels (for example, three liquid crystal panels). The transmittance (transmittance distribution) of the liquid crystal panel is controlled based on the control signal output from the projection control unit. The light emitted from the light source passes through the liquid crystal panel, and thus an optical image corresponding to the projection image 300 is emitted from the liquid crystal panel to the screen board 500. That is, the projection image 300 is projected onto the screen board 500. Further, the projection control unit can adjust the luminance of the projection image 300 by adjusting parameters such as the light amount of the light source with adjustment parameters (such as the dimming rate). The adjustment parameters may or may not be set (determined) by the control unit 101 according to a user operation on the projector 100 (operation unit 105). For example, the adjustment parameters may be automatically set by the control unit 101 according to the usage environment of the projector 100 (such as the brightness around the projector 100).

[0025] The internal bus 109 communicably connects the above-described plurality of functional units to each other. The internal bus 109 functions as a transmission path for the plurality of functional units to transmit and receive various data, various control information, etc. to and from each other.

[0026] <Configuration of the operation device> FIG. 2 schematically shows a configuration example of the operation device 600. The operation device 600 is a pen-type operation device and is used to indicate an arbitrary position on the screen board 500. The ON / OFF button 601 is a button for switching ON / OFF of the output of invisible light from the operation device 600. The light emitting unit 602 emits invisible light in response to a user operation on the ON / OFF button 601. For example, when the ON / OFF button 601 is pressed while the light emitting unit 602 is not emitting invisible light, the light emitting unit 602 emits invisible light. Then, when the ON / OFF button 601 is pressed while the light emitting unit 602 is emitting invisible light, the light emitting unit 602 stops the output of invisible light. The light emitting unit 602 may emit invisible light only while the ON / OFF button 601 is pressed, and the light emitting unit 602 may stop the output of invisible light in response to the release of the pressing of the ON / OFF button 601. Note that, as described above, the user operation on the screen board 500 is not limited to the user operation using the operation device 600. For example, the user operation on the screen board 500 may be a user operation of pointing at an arbitrary position on the screen board 500 with a finger.

[0027] <Configuration of the processing unit> FIG. 3 is a block diagram showing a configuration example of the processing unit 107. As shown in FIG. 3, the processing unit 107 includes a layout unit 110 that performs layout processing, an object unit 111 that performs object generation processing, and an overlay unit 112 that performs overlay processing. Note that the processing unit 107 may further include a functional unit that performs other image processing.

[0028] Based on the image data (input image; target image) output from the connection unit 106, the layout unit 110 generates a projection image 300 (image data) on which the input image is arranged. Then, the layout unit 110 outputs the projection image 300 (image data) to the superimposing unit 112. Further, the layout unit 110 can change the layout of the input image in the projection image 300. For example, the layout of the input image may or may not be changed according to a user operation on the projector 100 (operation unit 105). For example, the layout of the input image may be automatically changed according to changes in the type, number, etc. of external devices connected to the projector 100.

[0029] Figs. 6(A) to 6(F) are schematic diagrams showing an example of the layout of the input image. In Fig. 6(A), the input image is arranged in a part of the projection image 300. Therefore, a part of the projection image 300 becomes the screen area 400 of the input image, and the other part of the projection image 300 becomes the non-screen area 700. Fig. 6(B) shows a state where the input image is enlarged from the state of Fig. 6(A). In Fig. 6(B), the input image is arranged over the entire projection image 300. Therefore, the entire projection image 300 becomes the screen area 400, and there is no non-screen area 700. Figs. 6(C) and 6(D) show states where the aspect ratio of the input image is changed from the state of Fig. 6(A). Specifically, Fig. 6(C) shows a state where the input image is enlarged in the vertical direction (longitudinal direction) from the state of Fig. 6(A), and Fig. 6(D) shows a state where the input image is enlarged in the horizontal direction (lateral direction) from the state of Fig. 6(A). In the layout process, not only enlargement of the input image but also reduction of the input image can be executed. Fig. 6(E) shows a state where the input image is reduced in the horizontal direction from the state of Fig. 6(A). Figs. 6(A) to 6(E) all show examples of single-screen projection (single-screen display) in which one screen on which the input image (target image) is arranged is projected. Fig. 6(F) is An example of two-screen projection (two-screen display; multi-screen projection) that projects two screens on which an input image (target image) is placed is shown. In multi-screen projection, a plurality of screens, a plurality of images, etc. are projected side by side. In FIG. 6(F), two screen areas 401 and 402 are arranged in the projection image 300. When switching from the single-screen projection in FIG. 6(A) to the two-screen projection in FIG. 6(F), the position and size of the screen area 400 in FIG. 6(A) are changed, and the screen area 400 in FIG. 6(A) becomes the screen area 401 or the screen area 402 in FIG. 6(F). In the layout process, in both single-screen projection and multi-screen projection, the position, size, shape (aspect ratio), etc. of the screen area 400 of the input image can be changed. In the layout process, in multi-screen projection, the layout of a plurality of screens can be changed.

[0030] Note that in FIG. 6(F), there is a non-screen area 700, but the screen areas 401 and 402 may be arranged so that the non-screen area 700 does not exist. The number of screens in multi-screen projection may be more than two. The layout of a plurality of screens in the layout process may or may not be changed individually. For example, in response to a change in the layout of any one of the plurality of screens, the layout of other screens may be automatically changed.

[0031] The object unit 111 generates an object image (image data) in response to a user operation on the screen board 500 and outputs the object image (image data) to the superimposing unit 112. In the present embodiment, the object unit 111 generates an object image so that the object image is projected at the position specified by the user operation (user operation on the screen board 500) on the screen board 500. Specifically, the object unit 111 generates an object image by performing drawing on the coordinates notified from the detection unit 104. When there is a movement of the position specified by the user operation on the screen board 500, drawing is performed so as to trace the locus of the specified position. By projecting such an object image, the user 200 can obtain a feeling of drawing characters and figures on the screen board 500.

[0032] The superimposing unit 112 synthesizes the image data output from the object unit 111 with the image data output from the layout unit 110 so that the object image is superimposed on the projection image 300 on which the input image (target image) is arranged. As a result, the projection image 300 on which the input image is arranged and the object image is not arranged is updated to the projection image 300 on which the input image and the object image are arranged. Then, the superimposing unit 112 outputs the updated projection image 300 (image data obtained by synthesis) to the projection unit 108.

[0033] FIGS. 7(A) and 7(B) are schematic diagrams showing an example of the superimposing process of the superimposing unit 112. FIG. 7(A) shows a state where the object image 800 is superimposed on the projection image 300 (FIGS. 1 and 6(A)) for single-screen projection. In FIG. 7(A), a projection image 300 in which the object image 800 is arranged at the center of the screen area 400 (input image) is obtained by the superimposing process. FIG. 7(B) shows a state where the object image 800 is superimposed on the projection image 300 (FIG. 6(F)) for two-screen projection. In FIG. 7(B), a projection image 300 in which the object image 800 is arranged at the center of the screen area 401 is obtained by the superimposing process.

[0034] <Processing Flow of Projector> FIG. 4 is a flowchart showing an example of the processing flow of the projector 100. When AC power is supplied to the projector 100 via the power cable, power is supplied to the control unit 101 of the projector 100, and the control unit 101 starts up and enters the standby state. Then, when the control unit 101 detects that there is a startup instruction from the user, the processing flow in FIG. 4 starts. For example, when the control unit 101 receives a signal output from the power switch in response to the pressing of the power switch (operation unit 105) provided on the projector 100, the processing flow in FIG. 4 starts. When the power button provided on the remote controller of the projector 100 is pressed The control unit 101 may receive a signal output from the remote controller via the receiving unit (operation unit 105), thereby starting the processing flow of FIG. 4. The control unit 101 may receive a control command output from an external device (control device) connected to the projector 100 via the connection unit 106, thereby starting the processing flow of FIG. 4.

[0035] In step S100, the control unit 101 performs startup processing for each functional unit of the projector 100. Specifically, the control unit 101 controls to supply power to each functional unit and performs settings so that each functional unit becomes operable. Then, the control unit 101 controls so that the light source of the projection unit 108 is lit and the cooling fan that cools the inside of the projector 100 is driven. Thereby, the projection of the input image (specifically, the projection image 300 on which the input image is arranged and the object image 800 is not arranged) acquired by the connection unit 106 is started.

[0036] In step S101, the control unit 101 makes an inquiry to the detection unit 104 and determines whether a user operation (drawing operation) on the screen board 500 has been detected. If a drawing operation is detected, the process proceeds to step S102, and if a drawing operation is not detected, the process proceeds to step S103. When a drawing operation is detected, the object image 800 is generated by the object unit 111.

[0037] In step S102, the control unit 101 instructs the superimposing unit 112 to perform superimposing processing. Thereby, the superimposing unit 112 performs superimposing processing, and the projection image 300 on which the input image is arranged and the object image 800 is not arranged is updated to the projection image 300 on which the input image and the object image 800 are arranged.

[0038] In step S103, the control unit 101 makes an inquiry to the operation unit 105 to determine whether a user operation (layout change operation) for changing the layout of the input image in the projection image 300 is detected. If a layout change operation is detected, the process proceeds to step S104; if no layout change operation is detected, the process proceeds to step S108.

[0039] In step S104, the control unit 101 makes an inquiry to the superimposing unit 112 to determine whether superimposing processing is being performed. If superimposing processing is being performed, the process proceeds to step S105; if superimposing processing is not being performed, the process proceeds to step S106.

[0040] In each of step S105 and step S106, the control unit 101 instructs the layout unit 110 to change the layout of the input image in the projection image 300. As a result, the layout unit 110 performs layout processing (layout change processing), and the layout of the input image is changed. Here, assume that the user presses the two-screen projection button (operation unit 105) in a state where single-screen projection in FIG. 1,6(A) is being performed. When the control unit 101 receives a signal output from the operation unit 105 in response to the pressing of the two-screen projection button, it instructs the layout unit 110 to switch to two-screen projection. Then, the layout unit 110 changes the layout of the input image and the like so that two-screen projection in FIG. 5 (FIG. 6(F)) is performed.

[0041] Note that the layout of the input image (target image) may be changed by a process different from the switching from single-screen projection to multi-screen projection. For example, the layout of the input image may be changed by switching from multi-screen projection to single-screen projection. That is, the layout of the input image may be changed by switching from one of single-screen projection and multi-screen projection to the other. The position, size, and aspect ratio of the input image in the projection image 300 The layout of the input image may be changed by at least any change, a change in the layout of a plurality of screens in multi-screen projection, or the like.

[0042] Here, the relationship (such as the relationship of position, size, shape, etc.) between the input image and the object image 800 in the projection image 300 before the layout of the input image is changed is the relationship intended by the user. And when the layout of the input image is changed without changing the layout of the object image 800, the relationship between the input image and the object image 800 in the projection image 300 is also changed. That is, when the layout of the input image is changed, the relationship between the input image and the object image 800 deviates from the relationship intended by the user. Therefore, in this embodiment, the process of step S107 is performed.

[0043] In step S107, the control unit 101 instructs the object unit 111 to change the layout of the object image 800 in the projection image 300 according to the change in the layout of the input image in the projection image 300. Thereby, the object unit 111 updates the object image 800 so that the layout of the object image 800 is changed according to the change in the layout of the input image (object update process). Note that the control unit 101 may instruct the superimposing unit 112 to change the layout of the object image 800. And the superimposing unit 112 may process the object image 800 so that the layout of the object image 800 is changed and superimpose it on the projection image 300.

[0044] In this embodiment, the layout of the object image 800 is changed so that the change in the layout of the object image 800 matches the change in the layout of the target image. Here, consider the case where the projection is switched from the single-screen projection in Fig. 7(A), where the object image 800 is placed at the center of the screen area 400 (input image), to the two-screen projection. Assume that the screen area 400 is moved to the left and reduced to become the screen area 401. If the process of step S107 is not performed, the layout of the object image 800 remains unchanged from the layout in Fig. 7(A) and, as shown in Fig. 7(C), is projected at a large size at a position shifted from the center of the screen area 401 (input image). In this embodiment, by the process of step S107, the object image 800 is moved to the left by the same amount of movement as the screen area 400 and reduced at the same reduction rate as the screen area 400. As a result, as shown in Fig. 7(B), the object image 800 can be projected at the center of the screen area 401 (input image) without changing the relationship (relationships such as position, size, shape, etc.) between the input image and the object image 800 from Fig. 7(A). That is, the relationship between the input image and the object image 800 can be maintained as the relationship intended by the user.

[0045] In addition, when the aspect ratio of the input image is changed, the aspect ratio of the object image 800 is also changed in the same way as the change in the aspect ratio of the input image. For example, when the input image is enlarged in the vertical direction, the object image 800 is enlarged at the same enlargement rate as the enlargement rate of the input image (the enlargement rate in the vertical direction). When the input image is reduced in the horizontal direction, the object image 800 is reduced at the same reduction rate as the reduction rate of the input image (the enlargement rate in the horizontal direction).

[0046] Note that the method for changing the layout of the object image 800 is not limited to the above method. For example, the change in the layout of the object image 800 does not have to match the change in the layout of the target image. The layout of the object image 800 may be changed so that the difference between the change in the layout of the target image and the change in the layout of the object image 800 is equal to or less than a threshold value. The threshold value only needs to be such that the change in the layout of the object image 800 is similar to the change in the layout of the target image.

[0047] In step S108, the control unit 101 makes an inquiry to the operation unit 105 and determines whether a user operation (end operation) for ending the processing flow of FIG. 4 has been detected. If an end operation is detected, the control unit 101 performs end processing on each functional unit of the projector 100. Specifically, the control unit 101 controls to stop the power supply to each functional unit of the projector 100. As a result, the control unit 101 returns to the standby state, and the processing flow of FIG. 4 ends. If an end operation is not detected, the processing returns to step S101.

[0048] As described above, according to the present embodiment, when the layout of the target image in the projection image is changed, the layout of the object image in the projection image is changed according to the change in the layout of the target image. Thereby, when the layout of the target image is changed, the relationship between the target image and the object image can be maintained in the relationship intended by the user.

[0049] Note that each functional unit in this embodiment (Fig. 2) may be individual hardware, or may not be. The functions of two or more functional units may be realized by common hardware. Each of the multiple functions of one functional unit may be realized by individual hardware. Two or more functions of one functional unit may be realized by common hardware. Also, each functional unit may be realized by hardware, or may not be. For example, the device may have a processor and a memory storing a control program. And the functions of at least some of the functional units of the device may be realized by the processor reading out and executing the control program from the memory.

[0050] Note that this embodiment (including the above-described modified examples) is merely an example, and configurations obtained by appropriately modifying or changing the configuration of this embodiment within the scope of the gist of the present invention are also included in the present invention.

[0051] <Other Embodiments> The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiment to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Description of Reference Numerals

[0052] 100: Projector 101: Control Unit 104: Detection Unit 108: Projection Unit 110: Layout Unit 111: Object Unit 112: Superimposition Unit

Claims

1. An acquisition means for acquiring a first input image and a second input image; a projection means for performing a first projection process of projecting the first input image and a second projection process of arranging the first input image and the second input image in parallel and projecting them; A switching means for switching a process from the first projection process to the second projection process; a generating means for generating an object image in response to a user's operation on a projection surface onto which an image is projected by the projection means; having The projection means includes: projecting the object image generated by the generating means during execution of the first projection process onto the first input image in a superimposed manner; when the switching means switches from the first projection process to the second projection process, projecting the object image at a position and size according to a relative position and a relative size between the first input image and the object image in the first projection process; In the second projection process, in response to a change in the layout of the first input image, the object image is projected at a position and size corresponding to the changed layout.

13. A projection device comprising:

2. A detection means for detecting a position of a device operated by a user; A communication means for communicating with a device operated by the user; and The projection means projects the object image generated by the generation means on the basis of the position of the device and the operation of the user, superimposing the object image on the image being projected.

2. The projection device according to claim 1.

3. The projection means projects the object image at a position and a size according to a relative position and a relative size between the first input image and the object image in the second projection process even when the projection means switches from the second projection process to the first projection process.

3. The projection device according to claim 1, wherein the object image is projected in a superimposed manner on the first input image.

4. 4. The projection device according to claim 1, wherein the layout is a position, a size, or a shape.

5. A projection device as described in any one of claims 1 to 4, characterized in that the projection means projects an object image based on the position of the user's operation.

6. 6. The projection device according to claim 5, wherein the projection means projects a character or a figure as the object image.

7. 5. The projection device according to claim 1, wherein the projection unit projects an object image based on a trajectory of a position of the user's operation.

8. 8. The projection device according to claim 1, wherein the switching unit switches from the first projection process to the second projection process in response to a switching instruction received from a remote controller.

9. 9. The projection device according to claim 1, wherein the user's operation is an operation using a pen-type device.

10. 9. The projection device according to claim 1, wherein the user's operation is an operation using a finger of the user.

11. 11. The projection device according to claim 1, wherein the user's operation is detected by detecting infrared light.

12. When the switching means switches from the first projection process to the second projection process, the projection means projects the first input image and the second input image in parallel, and then superimposes and projects the object image at a position and size according to the relative positions and relative sizes of the first input image and the object image in the first projection process.

12. A projection device according to claim 1, wherein the projection device is a projection lens.

13. acquiring a first input image and a second input image; a projection step of performing a first projection process of projecting the first input image and a second projection process of arranging the first input image and the second input image in parallel and projecting them; a switching step of switching a process from the first projection process to the second projection process; a generating step of generating an object image in response to a user's operation on a projection surface onto which the image is projected by the projection step; having In the projection step, projecting the object image generated in the generating step onto the first input image while executing the first projection process; When the first projection process is switched to the second projection process by the switching step, the object image is projected at a position and a size according to a relative position and a relative size between the first input image and the object image in the first projection process; In the second projection process, the layout of the first input image is changed. In response, the object image is projected at a position and size corresponding to the changed layout. A projection method comprising:

Images