CONTROL DEVICE, CONTROL METHOD, PROJECTION SYSTEM, AND CONTROL PROGRAM

Abstract

To provide a control device, a control method, a projection system, and a control program that can easily adjust projection of a projection apparatus to a wide projection range.SOLUTION: A projection system 100 includes a projection apparatus 10 that projects a first image 71 including a plurality of marker images, an imaging apparatus 90 that captures at least a part of the first image 71, and a computer 50. The computer 50 performs control of projecting second images 81-84 including the plurality of marker images from the projection apparatus 10 on the basis of an imaging result of at least the part of the first image 71 by the imaging apparatus 90.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present invention relates to a control device, a control method, a projection system, and a control program. [Background technology]

[0002] Patent Document 1 describes a method of dividing calibration projection images projected from multiple projectors into multiple regions and capturing the images with a camera, and calculating various correction coefficients for each projector to perform alignment, scale adjustment, distortion correction, brightness correction of overlapping regions, etc.

[0003] Patent Document 2 describes a method of capturing an image of a projected area by a projector by dividing the area into multiple areas so that the areas partially overlap, and estimating a projection transformation matrix for joining adjacent partial images together based on the captured image.

[0004] Patent Document 3 describes a method of stack projection in which test patterns are simultaneously projected and imaged while still overlapping, and patterns with different wavelength ranges such as R, G, and B or patterns with different polarization characteristics are projected for each projector, and the overlapped patterns are separated afterwards. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2015-026992 A [Patent Document 2] Re-tabled publication 2016 / 204068 [Patent Document 3] JP 2012-047849 A Summary of the Invention

[0006] One embodiment of the technique disclosed herein provides a control device, a control method, a projection system, and a control program that can easily adjust the projection of a projection device over a wide projection range. [Means for solving the problem]

[0007] A control device of one embodiment of the present invention is a control device for a projection system including one or more projection devices that project a first image including multiple marker images and an imaging device that captures at least a portion of the first image, and includes a processor that controls the projection of a second image including multiple marker images from the projection device based on the imaging result of at least a portion of the first image by the imaging device.

[0008] A control method of one embodiment of the present invention is a control method by a control device of a projection system including one or more projection devices that project a first image including a plurality of marker images and an imaging device that captures at least a portion of the first image, wherein the control device is equipped with a processor, and the processor controls the projection of a second image including a plurality of marker images from the projection device based on the imaging result of at least a portion of the first image by the imaging device.

[0009] A projection system according to one embodiment of the present invention comprises one or more projection devices that project a first image including a plurality of marker images, an imaging device that captures at least a portion of the first image, and a control device, wherein the control device comprises a processor that controls the projection of a second image including a plurality of marker images from the projection device based on the imaging result of at least a portion of the first image by the imaging device.

[0010] A control program of one embodiment of the present invention is intended to cause a processor of a control device of a projection system including one or more projection devices that project a first image including multiple marker images, and an imaging device that captures at least a portion of the first image, to execute processing to control the projection of a second image including multiple marker images from the projection device based on the imaging result of at least a portion of the first image by the imaging device. Effect of the Invention

[0011] According to the present invention, it is possible to provide a control device, a control method, a projection system, and a control program that can easily adjust the projection of a projection device over a wide projection range. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 illustrates an example of a projection system 100 according to an embodiment. [Diagram 2] FIG. 1 is a diagram illustrating an example of a projection device 10. [Diagram 3] 2 is a schematic diagram showing an example of the internal configuration of a projection unit 1. FIG. [Figure 4] 1 is a schematic diagram showing the external configuration of a projection device 10. FIG. [Diagram 5] 5 is a schematic cross-sectional view of an optical unit 106 of the projection device 10 shown in FIG. [Figure 6] FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer 50. [Figure 7] 2 is a diagram showing an example of the projection of a first image by a projection device 10 and an example of an image capturing range of an image capturing device 90. FIG. [Figure 8] 1 is a diagram (part 1) showing an example of projection of a second image by the projection device 10. FIG. [Figure 9] FIG. 2 shows an example of projection of a second image by the projection device 10. [Figure 10] FIG. 11 is a diagram (part 3) showing an example of projection of a second image by the projection device 10. [Figure 11] FIG. 4 shows an example of projection of a second image by the projection device 10. [Figure 12] 13 is a flowchart showing an example of processing by a computer 50. [Figure 13] 13 is a diagram showing another example of the first image 71 projected by the projection device 10. FIG. [Figure 14] 13 is a diagram showing another example of the second image 81 projected by the projection device 10. FIG. [Figure 15] FIG. 13 is a diagram showing an example of a state before adjustment of stack projection by a plurality of projection devices. [Figure 16] 11 is a flowchart showing an example of processing based on imaging conditions by a computer 50. [Figure 17] FIG. 1 is a diagram (part 1) showing an example of individual projection by projection devices 10 and 10A. [Figure 18] FIG. 2 is a diagram (part 2) showing an example of individual projection by the projection devices 10 and 10A. [Figure 19] FIG. 1 is a diagram showing an example of simultaneous projection by projection devices 10 and 10A. [Figure 20] FIG. 13 is a diagram showing an example of stack projection by overlapping projection ranges 11 and 11A. [Figure 21] FIG. 13 is a diagram illustrating an example of a state before adjustment of blending projection by a plurality of projection devices. [Figure 22] FIG. 13 is a diagram showing an example of blending projection by superimposing projection ranges 11 and 11A. [Diagram 23] 13 is a flowchart showing another example of processing based on imaging conditions by the computer 50. [Figure 24] 13 is a diagram showing an example of a second image when the resolution of the image captured by the imaging device 90 is low. FIG. [Diagram 25] 13 is a flowchart showing yet another example of processing based on imaging conditions by the computer 50. [Figure 26] 13 is a diagram showing an example of a marker image in the second image when the resolution of the image captured by the imaging device 90 is low. FIG. [Figure 27] 13 is a schematic diagram showing another external configuration of the projection device 10. FIG. [Figure 28] 28 is a schematic cross-sectional view of an optical unit 106 of the projection device 10 shown in FIG. 27. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an example of an embodiment of the present invention will be described with reference to the drawings.

[0014] (Embodiment) <Projection system 100 according to the embodiment> Fig. 1 is a diagram showing an example of a projection system 100 according to an embodiment. As shown in Fig. 1, the projection system 100 includes a projection device 10, a computer 50, and an imaging device 90. The computer 50 is an example of a control device in the present invention.

[0015] 1, the computer 50 is connected to the projection device 10 via a communication cable 8 and is capable of communicating with the projection device 10. The computer 50 is connected to the imaging device 90 via a communication cable 9 and is capable of communicating with the imaging device 90.

[0016] The projection device 10 is a projection device capable of projecting onto a projection object 6. The imaging device 90 is an imaging device capable of capturing an image projected onto the projection object 6 by the projection device 10.

[0017] The projection object 6 is an object such as a screen having a projection surface on which a projected image is displayed by the projection device 10. In the example shown in Fig. 1, the projection surface of the projection object 6 is a rectangular plane. The top, bottom, left and right of the projection object 6 in Fig. 1 are assumed to be the top, bottom, left and right of the actual projection object 6.

[0018] A projection range 11, shown by a dashed line, is a region of the projection object 6 onto which projection light is irradiated by the projection device 10. In the example shown in Fig. 1, the projection range 11 is rectangular. The projection range 11 is a part or the entirety of a projectable range onto which projection can be performed by the projection device 10.

[0019] <Projection device 10> Fig. 2 is a diagram showing an example of the projection device 10. As shown in Fig. 2, the projection device 10 includes a projection unit 1, a control unit 4, an operation reception unit 2, and a communication unit 5. The projection unit 1 is configured by, for example, a liquid crystal projector or a projector using LCOS (Liquid Crystal On Silicon). In the following, the projection unit 1 will be described as being a liquid crystal projector.

[0020] The control unit 4 controls the projection by the projection device 10. The control unit 4 is a device including a control unit configured with various processors, a communication interface (not shown) for communicating with each unit, and a storage medium 4a such as a hard disk, a solid state drive (SSD) or a read only memory (ROM), and controls the projection unit 1. The various processors of the control unit 4 include a CPU (Central Processing Unit), which is a general-purpose processor that executes a program and performs various processes, a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture such as an FPGA (Field Programmable Gate Array), or a dedicated electric circuit, which is a processor having a circuit configuration designed specifically for performing specific processes such as an ASIC (Application Specific Integrated Circuit).

[0021] More specifically, the structure of these various processors is an electric circuit that combines circuit elements such as semiconductor elements. The control unit of the control unit 4 may be composed of one of the various processors, or may be composed of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs or a combination of a CPU and an FPGA).

[0022] The operation reception unit 2 detects instructions from the user (user instructions) by receiving various operations from the user. The operation reception unit 2 may be a button, a key, a joystick, or the like provided on the control unit 4, or may be a receiving unit or the like that receives a signal from a remote controller that remotely controls the control unit 4.

[0023] The communication unit 5 is a communication interface capable of communicating with the computer 50. The communication unit 5 may be a wired communication interface for performing wired communication as shown in Fig. 1, or a wireless communication interface for performing wireless communication.

[0024] The projection unit 1, the control unit 4, and the operation reception unit 2 may be realized, for example, by a single device (see, for example, FIGS. 4 and 5). Alternatively, the projection unit 1, the control unit 4, and the operation reception unit 2 may be separate devices that communicate with each other to cooperate with each other.

[0025] <Internal configuration of projection unit 1> Fig. 3 is a schematic diagram showing an example of the internal configuration of the projection unit 1. As shown in Fig. 3, the projection unit 1 includes a light source 21, a light modulation unit 22, a projection optical system 23, and a control circuit 24. The light source 21 includes a light emitting element such as a laser or an LED (Light Emitting Diode), and emits, for example, white light.

[0026] The light modulation unit 22 is composed of three liquid crystal panels (light modulation elements) that modulate the three color lights emitted from the light source 21 and separated into red, blue, and green by a color separation mechanism not shown based on image information to emit each color image, and a dichroic prism that mixes the color images emitted from the three liquid crystal panels and emits them in the same direction. Red, blue, and green filters may be mounted on each of the three liquid crystal panels, and the white light emitted from the light source 21 may be modulated by each liquid crystal panel to emit each color image.

[0027] The projection optical system 23 receives light from the light source 21 and the light modulation unit 22 and includes at least one lens, and is configured, for example, by a relay optical system. The light that passes through the projection optical system 23 is projected onto the projection object 6.

[0028] An area of ​​the projection object 6 that is irradiated with light that transmits through the entire range of the light modulation unit 22 becomes a projectable range where projection by the projection unit 1 is possible. Within this projectable range, an area that is irradiated with light that actually transmits from the light modulation unit 22 becomes a projection range 11. For example, by controlling the size, position, and shape of the area of ​​the light modulation unit 22 through which light transmits, the size, position, and shape of the projection range 11 change within the projectable range.

[0029] The control circuit 24 controls the light source 21, the light modulation unit 22, and the projection optical system 23 based on the display data input from the control unit 4, thereby projecting an image based on the display data onto the projection object 6. The display data input to the control circuit 24 is composed of three data: red display data, blue display data, and green display data.

[0030] Moreover, the control circuit 24 enlarges or reduces the projection range 11 (see FIG. 1 ) of the projection unit 1 by changing the projection optical system 23 based on a command input from the control unit 4. The control unit 4 may also move the projection range 11 of the projection unit 1 by changing the projection optical system 23 based on an operation from a user accepted by the operation accepting unit 2.

[0031] The projection device 10 also includes a shift mechanism that mechanically or optically moves the projection range 11 while maintaining the image circle of the projection optical system 23. The image circle of the projection optical system 23 is an area where the projection light incident on the projection optical system 23 passes through the projection optical system 23 appropriately in terms of light amount drop, color separation, peripheral curvature, etc.

[0032] The shift mechanism is realized by at least one of an optical system shift mechanism that performs an optical system shift and an electronic shift mechanism that performs an electronic shift.

[0033] The optical system shift mechanism is, for example, a mechanism for moving the projection optical system 23 in a direction perpendicular to the optical axis (see, for example, FIGS. 5 and 28), or a mechanism for moving the light modulation unit 22 in a direction perpendicular to the optical axis instead of moving the projection optical system 23. The optical system shift mechanism may also be a mechanism for combining the movement of the projection optical system 23 and the movement of the light modulation unit 22.

[0034] The electronic shift mechanism is a mechanism for shifting the pseudo projection range 11 by changing the range through which light is transmitted in the light modulation section 22 .

[0035] Furthermore, the projection device 10 may include a projection direction changing mechanism that moves the projection range 11 together with the image circle of the projection optical system 23. The projection direction changing mechanism is a mechanism that changes the projection direction of the projection unit 1 by changing the orientation of the projection unit 1 through mechanical rotation (see, for example, FIG. 28).

[0036] <Mechanical configuration of the projection device 10> Fig. 4 is a schematic diagram showing the external configuration of the projection device 10. Fig. 5 is a schematic cross-sectional view of the optical unit 106 of the projection device 10 shown in Fig. 4. Fig. 5 shows a cross section taken along a plane along the optical path of light emitted from the main body 101 shown in Fig. 4.

[0037] 4, the projection device 10 includes a main body 101 and an optical unit 106 provided to protrude from the main body 101. In the configuration shown in Fig. 4, the operation reception unit 2, the control unit 4, the light source 21, the light modulation unit 22, and the control circuit 24 in the projection unit 1, and the communication unit 5 are provided in the main body 101. The projection optical system 23 in the projection unit 1 is provided in the optical unit 106.

[0038] The optical unit 106 includes a first member 102 supported by the main body 101. The optical unit 106 may be configured to be detachable from the main body 101 (in other words, configured to be replaceable).

[0039] As shown in FIG. 5, the main body 101 has a housing 15 in which an opening 15a for passing light is formed at a portion connected to the optical unit .

[0040] As shown in Fig. 4, a light source 21 and a light modulation unit 12 including a light modulation section 22 (see Fig. 3) that generates an image by spatially modulating the light emitted from the light source 21 based on input image data are provided inside the housing 15 of the main body 101. The light emitted from the light source 21 is incident on the light modulation section 22 of the light modulation unit 12, spatially modulated by the light modulation section 22, and emitted.

[0041] As shown in FIG. 5, the image formed by the light spatially modulated by the light modulation unit 12 passes through the opening 15a of the housing 15 and enters the optical unit 106, and is projected onto the projection object 6, making the image G1 visible to the observer.

[0042] As shown in FIG. 5, the optical unit 106 includes a first member 102 having a hollow portion 2A connected to the inside of the main body 101, a first optical system 121 arranged in the hollow portion 2A, a lens 34, and a first shift mechanism 105.

[0043] The first member 102 is a member having a cross-sectional outer shape of, for example, a rectangle, and the openings 2a and 2b are formed on planes parallel to each other. The first member 102 is supported by the main body 101 with the opening 2a disposed at a position facing the opening 15a of the main body 101. Light emitted from the optical modulation section 22 of the optical modulation unit 12 of the main body 101 passes through the openings 15a and 2a and enters the hollow section 2A of the first member 102.

[0044] The incident direction of light entering hollow portion 2A from main body portion 101 is referred to as direction X1, the opposite direction to direction X1 is referred to as direction X2, and directions X1 and X2 are collectively referred to as direction X. In addition, in Fig. 5, the direction from the front of the paper to the back and the opposite direction thereto are referred to as direction Z. Of direction Z, the direction from the front of the paper to the back is referred to as direction Z1, and the direction from the back of the paper to the front is referred to as direction Z2.

[0045] Moreover, a direction perpendicular to directions X and Z is referred to as direction Y, and within direction Y, the upward direction in Fig. 5 is referred to as direction Y1, and the downward direction in Fig. 5 is referred to as direction Y2. In the example of Fig. 5, the projection device 10 is disposed so that direction Y2 is the vertical direction.

[0046] The projection optical system 23 shown in Fig. 3 is configured with a first optical system 121 and a lens 34 in the example of Fig. 5. Fig. 5 shows an optical axis K of the projection optical system 23. The first optical system 121 and the lens 34 are arranged along the optical axis K in this order from the light modulation unit 22 side.

[0047] The first optical system 121 includes at least one lens, and guides light traveling in a direction X1 that is incident on the first member 102 from the main body 101 to the lens 34.

[0048] The lens 34 is disposed at an end portion on the X1 direction side of the first member 102 so as to cover the opening 2b formed at this end portion. The lens 34 projects the light incident from the first optical system 121 onto the projection object 6.

[0049] The first shift mechanism 105 is a mechanism for moving the optical axis K of the projection optical system (in other words, the optical unit 106) in a direction perpendicular to the optical axis K (direction Y in FIG. 5). Specifically, the first shift mechanism 105 is configured to be able to change the position of the first member 102 in direction Y relative to the main body 101. The first shift mechanism 105 may be one that moves the first member 102 manually, or one that moves the first member 102 electrically.

[0050] Fig. 5 shows a state in which the first member 102 is moved to the maximum extent in the direction Y1 by the first shift mechanism 105. When the first member 102 is moved in the direction Y2 by the first shift mechanism 105 from the state shown in Fig. 5, the relative position between the center of the image formed by the light modulation unit 22 (in other words, the center of the display surface) and the optical axis K changes, and the image G1 projected onto the projection object 6 can be shifted (translated) in the direction Y2.

[0051] The first shift mechanism 105 may be a mechanism that moves the light modulation unit 22 in the direction Y, instead of moving the optical unit 106 in the direction Y. Even in this case, the image G1 projected onto the projection object 6 can be moved in the direction Y.

[0052] <Hardware configuration of computer 50> Fig. 6 is a diagram showing an example of a hardware configuration of the computer 50. As shown in Fig. 5, the computer 50 shown in Fig. 1 includes a processor 51, a memory 52, a communication interface 53, and a user interface 54. The processor 51, the memory 52, the communication interface 53, and the user interface 54 are connected by, for example, a bus 59.

[0053] The processor 51 is a circuit that performs signal processing, and is, for example, a CPU that controls the entire computer 50. The processor 51 may be realized by other digital circuits such as an FPGA or a DSP (Digital Signal Processor). The processor 51 may also be realized by combining a plurality of digital circuits.

[0054] The memory 52 includes, for example, a main memory and an auxiliary memory. The main memory is, for example, a random access memory (RAM). The main memory is used as a work area for the processor 51.

[0055] The auxiliary memory is a non-volatile memory such as a magnetic disk, an optical disk, a flash memory, etc. The auxiliary memory stores various programs that operate the computer 50. The programs stored in the auxiliary memory are loaded into the main memory and executed by the processor 51.

[0056] The auxiliary memory may also include a portable memory that is removable from the computer 50. The portable memory may be a memory card such as a Universal Serial Bus (USB) flash drive or a Secure Digital (SD) memory card, or an external hard disk drive.

[0057] The communication interface 53 is a communication interface that communicates with the outside of the computer 50 (for example, the projection device 10 or the imaging device 90). The communication interface 53 is controlled by the processor 51. The communication interface 53 may be a wired communication interface that performs wired communication, may be a wireless communication interface that performs wireless communication, or may include both a wired communication interface and a wireless communication interface.

[0058] The user interface 54 includes, for example, an input device that accepts operation input from the user, and an output device that outputs information to the user. The input device can be realized by, for example, a pointing device (for example, a mouse), a key (for example, a keyboard), or a remote control. The output device can be realized by, for example, a display or a speaker. The input device and the output device may also be realized by a touch panel or the like. The user interface 54 is controlled by the processor 51.

[0059] <Projection of the First Image by the Projection Device 10 and the Capturing Range of the Imaging Device 90> Fig. 7 is a diagram showing an example of the projection of a first image by the projection device 10 and an example of an image capturing range of the image capturing device 90. As shown in Fig. 7, the computer 50 controls the projection device 10 to project a first image 71 into a projection range 11. The first image 71 is an image in which 91 rectangular marker images are arranged in a 7 x 13 matrix.

[0060] The imageable range 72 is a range in the projection object 6 that can be imaged by the imaging device 90. As shown in Fig. 7, there are cases where the imageable range 72 is narrower than the projection range 11, and the entire first image 71 cannot be imaged by the imaging device 90. This is due to, for example, limitations on the angle of view of the imaging device 90, limitations on the distance the imaging device 90 can be away from the projection object 6, and the like.

[0061] For example, the computer 50 performs control to output a message urging the user of the imaging device 90 to capture the first image 71 so as to include as many marker images as possible among the marker images. Note that the user of the imaging device 90 may be the same as or different from the user of the computer 50.

[0062] 7, it is assumed that the imaging device 90 is unable to capture the entire first image 71, and captures an area including 28 marker images (4×7) in the center of the first image 71 within the imageable range 72. The imaging device 90 transmits the captured image obtained by the capture to the computer 50.

[0063] The computer 50 calculates the imageable range 72 based on the captured image transmitted from the imaging device 90. Specifically, the calculation of the imageable range 72 is a calculation of the relative size of the imageable range 72 with respect to the first image 71 projected onto the projection object 6. For example, the computer 50 calculates the number of marker images (28 in the example of FIG. 7) included in the captured image from the imaging device 90 out of the marker images included in the first image 71 as the size of the imageable range 72. Then, the computer 50 generates a second image including a plurality of marker images included in the imageable range 72 based on the calculation result of the imageable range 72.

[0064] For example, the computer 50 determines the number of marker images included in the captured image from the imaging device 90 by image recognition. In the example of Fig. 7, the captured image from the imaging device 90 includes 4 x 7, or 28, marker images. Therefore, the computer 50 generates a second image including 4 x 7, or 28, marker images.

[0065] <Projection of the second image by the projection device 10> 8 to 11 are diagrams showing an example of projection of a second image by the projection device 10. After calculating the imageable range 72 as described in Fig. 7, the computer 50 performs control so as to project a second image 81 from the projection device 10 to the upper left of the projection range 11, for example, as shown in Fig. 8.

[0066] The second image 81 is obtained by cutting out the 28 marker images in a 4×7 matrix from the upper left corner of the first image 71. That is, the second image 81 is an image in which 28 rectangular marker images are arranged in a 4×7 matrix. The size and spacing of the marker images included in the second image 81 are the same as those of the marker images included in the first image 71. Therefore, the user of the imaging device 90 can capture the 28 marker images included in the second image 81 by placing them in the imaging possible range 72 using the projection device 10. The imaging device 90 transmits the captured image of the second image 81 obtained by imaging to the computer 50.

[0067] In the state shown in Fig. 8, the computer 50 may perform control to prompt the user of the imaging device 90 to capture an image including all marker images in the second image 81. In the example of Fig. 8, the computer 50 controls the projection device 10 to project a message "Please capture an image including the upper left marker group" at a position in the projection range 11 different from the second image 81.

[0068] Next, the computer 50 controls the projection device 10 to project the second image 82 to the upper right of the projection range 11, for example, as shown in FIG. 9. The second image 82 is obtained by cutting out the 28 marker images of 4×7 in the upper right corner of the first image 71. That is, the second image 82 is an image in which 28 rectangular marker images are arranged in a 4×7 matrix, similar to the second image 81. Therefore, the user of the imaging device 90 can capture the 28 marker images included in the second image 82 by placing them in the imaging possible range 72 using the projection device 10. The imaging device 90 transmits the captured image of the second image 82 obtained by imaging to the computer 50.

[0069] In the state shown in Fig. 9, the computer 50 may perform control to prompt the user of the imaging device 90 to capture an image including all marker images of the second image 82. In the example of Fig. 9, the computer 50 controls the projection device 10 to project a message "Please capture an image including the marker group on the upper right" at a position in the projection range 11 different from the second image 82.

[0070] Next, the computer 50 controls the projection device 10 to project the second image 83 to the lower left of the projection range 11, for example, as shown in FIG. 10. The second image 83 is obtained by cutting out the 28 marker images of 4×7 at the lower left from the first image 71. That is, the second image 83 is an image in which 28 rectangular marker images are arranged in a 4×7 matrix, similar to the second images 81 and 82. Therefore, the user of the imaging device 90 can capture the 28 marker images included in the second image 83 by placing them in the imaging possible range 72 with the projection device 10. The imaging device 90 transmits the captured image of the second image 83 obtained by imaging to the computer 50.

[0071] 10, the computer 50 may perform control to prompt the user of the imaging device 90 to capture an image including all marker images of the second image 83. In the example of Fig. 10, the computer 50 controls the projection device 10 to project a message "Please capture an image including the marker group at the lower left" at a position in the projection range 11 different from the second image 83.

[0072] Next, the computer 50 controls the projection device 10 to project the second image 84 to the lower right of the projection range 11 as shown in FIG. 11. The second image 84 is obtained by cutting out the 28 marker images of 4×7 at the lower right from the first image 71. That is, the second image 84 is an image in which 28 rectangular marker images are arranged in a 4×7 matrix, similar to the second images 81 to 83. Therefore, the user of the imaging device 90 can capture the 28 marker images included in the second image 84 by placing them in the imaging possible range 72 with the projection device 10. The imaging device 90 transmits the captured image of the second image 84 obtained by imaging to the computer 50.

[0073] 11, the computer 50 may perform control to prompt the user of the imaging device 90 to capture an image including all marker images of the second image 84. In the example of FIG. 11, the computer 50 controls the projection device 10 to project a message saying "Please capture an image including the marker group at the lower right" at a position in the projection range 11 different from the second image 84.

[0074] 8 to 11, the computer 50 can obtain information equivalent to the entire captured image of the first image 71 by receiving the captured images of the second images 81 to 84 from the imaging device 90. The computer 50 performs control to adjust the projection by the projection device 10 based on the received captured images of the second images 81 to 84.

[0075] For example, the computer 50 generates an image equivalent to the entire captured image of the first image 71 by synthesizing the received captured images of the second images 81 to 84. Then, the computer 50 detects distortion of the projected image in the projection range 11 based on distortion of the shape and arrangement of the marker images included in the generated image, and performs distortion correction of the projected image based on the detection result.

[0076] In this way, the computer 50 repeatedly controls the projection device 10 to project a first image 71 including a plurality of marker images, thereby causing the imaging device 90 to capture at least a portion of the first image 71, and based on the imaging results, to project a second image (e.g., second images 81-84) including a plurality of marker images.

[0077] Specifically, the computer 50 calculates the imageable range 72 of the imaging device 90 based on at least a part of the captured image of the first image 71, and controls the projection device 10 to project the second images 81-84 based on the calculated imageable range 72. At this time, the computer 50 projects the second images 81-84 from the projection device 10 while changing the projection positions of the second images 81-84. Then, the computer 50 controls the adjustment of the projection of the projection device 10 based on the image capturing results of the second images 81-84 by the imaging device 90.

[0078] As a result, even if the imaging device 90 cannot capture the entire first image 71 projected in the projection range 11, the second images 81-84 that can be captured by the imaging device 90 can be projected and captured by the imaging device 90, and the projection of the projection device 10 can be adjusted based on the captured image results. Therefore, the projection of the projection device 10 for the wide projection range 11 can be easily adjusted.

[0079] Furthermore, the projection device 10 may perform control to change the second image projected from the projection device 10 when repeatedly performing control to project the second image from the projection device 10. For example, the projection device 10 may make the marker image included in the second image 81 rectangular, the marker image included in the second image 82 circular, the marker image included in the second image 83 triangular, and the marker image included in the second image 84 an X mark. In this way, when the computer 50 receives the captured images of the second images 81 to 84 from the projection device 10, the computer 50 can reliably determine which of the second images 81 to 84 the received captured image is, by determining the shape of the marker image included in the received captured image.

[0080] Although the process of sequentially projecting and capturing the second images 81-84 has been described, the process is not limited to this. For example, the computer 50 may project only the second image 81 of the second images 81-84 from the projection device 10. In this case, the computer 50 estimates the distortion of the entire projected image in the projection range 11 based on the distortion of the shape and arrangement of the marker images included in the second image 81, and performs distortion correction of the projected image based on the estimation result.

[0081] Further, although the second images 81-84 are cut-out portions of the first image 71 in the above description, the second images 81-84 are not limited to this. For example, in the example shown in Fig. 7 to Fig. 11, the second images 81-84 may be images in which 28 circular or triangular marker images are arranged in a 4 x 7 matrix. Furthermore, the second images 81-84 may be images that include marker images of different shapes or colors.

[0082] <Processing by computer 50> 12 is a flowchart showing an example of processing by the computer 50. The computer 50 executes the processing shown in FIG.

[0083] First, the computer 50 transmits a control signal to the projection device 10 to cause the projection device 10 to project the first image 71 (step S1201). For example, the computer 50 causes the projection device 10 to project the first image 71 shown in FIG.

[0084] Next, the computer 50 performs control to prompt the user of the imaging device 90 to capture the first image 71 projected in step S1201 by the imaging device 90 (step S1202). For example, this control is performed by controlling the projection device 10 to project a message into the projection range 11.

[0085] Next, the computer 50 receives the captured image of the first image 71 obtained by the imaging prompted by step S1202 from the imaging device 90 (step S1203). The transmission of the captured image by the imaging device 90 may be automatically performed by the imaging device 90 when imaging is performed by the imaging device 90, or may be performed by a user operation after imaging by the imaging device 90.

[0086] Next, the computer 50 determines whether or not all the marker images of the first image 71 are included in the captured image received in step S1203 (step S1204). If all the marker images are included in step S1204 (step S1204: Yes), the computer 50 performs control to adjust the projection by the projection device 10 based on the captured image of the first image 71 received in step S1203 (step S1205), and ends the series of processes.

[0087] In step S1204, if at least any marker image is not included (step S1204: No), the computer 50 generates a plurality of second images (step S1206). For example, the computer 50 calculates the imageable range 72 based on the captured image of the first image 71 received in step S1203, and generates second images (e.g., second images 81 to 84) that can cover the projection range 11 based on the calculated imageable range 72.

[0088] Next, the computer 50 causes the projection device 10 to project the unprojected second image among the second images generated in step S1206 (step S1207). Next, the computer 50 performs control to prompt the user of the imaging device 90 to capture the second image projected in step S1207 by the imaging device 90 (step S1208).

[0089] Next, the computer 50 receives from the imaging device 90 a captured image of the second image obtained by the imaging prompted in step S1208 (step S1209). Next, the computer 50 determines whether or not all of the second images generated in step S1206 have been projected in step S1207 (step S1210).

[0090] In step S1210, if at least any of the second images has not been projected (step S1210: No), the computer 50 returns to step S1207. If all of the second images have been projected (step S1210: Yes), the computer 50 performs control to adjust the projection by the projection device 10 based on the captured images of the multiple second images received in step S1209 (step S1211), and ends the series of processes.

[0091] <Another example of the first image 71 projected by the projection device 10> Fig. 13 is a diagram showing another example of the first image 71 projected by the projection device 10. As shown in Fig. 13, the first image 71 may be an image in which different numbers or the like are arranged at different positions as marker images.

[0092] Furthermore, computer 50 stores a correspondence table in which each number included in first image 71 as a marker image is associated with a position of the number in projection range 11. That is, the marker image of first image 71 shown in FIG. 13 is associated with the position of the marker image in projection range 11.

[0093] As a result, even if the captured image received from the imaging device 90 contains only a portion of the marker image of the first image 71, the computer 50 can determine at which position in the projection range 11 that portion of the marker image belongs.

[0094] For example, when the captured image of first image 71 received from imaging device 90 contains a predetermined ratio or more of the marker images of first image 71, computer 50 determines the positions in projection range 11 of the marker images contained in the captured image of first image 71 based on the above correspondence table, detects distortion of the projected image in the entire projection range 11 based on the determination result, and performs distortion correction of the projected image based on the detection result. In this case, computer 50 does not need to control projection device 10 to project second images (e.g., second images 81-84).

[0095] <Another example of the second image 81 projected by the projection device 10> Fig. 14 is a diagram showing another example of the second image 81 projected by the projection device 10. As shown in Fig. 14, the second image 81 may be an image in which different numbers or the like are arranged at different positions as marker images. In the example of Fig. 14, the second image 81 is a portion of the first image 71 shown in Fig. 13 in which 28 marker images of 4 × 7 are cut out from the upper left.

[0096] Furthermore, computer 50 stores a correspondence table in which each number included in second image 81 as a marker image is associated with a position in projection range 11 where the number is located. This correspondence table may be the same as the correspondence table described in Fig. 13. That is, the marker image in second image 81 shown in Fig. 14 is associated with the position of the marker image in projection range 11.

[0097] As a result, even if the captured image received from the imaging device 90 includes only a portion of the second image 81, the computer 50 can determine at which position in the projection range 11 the marker image of that portion is located.

[0098] Although the second image 81 has been described, the second images 82 to 84 may also be images in which different numbers or the like are arranged at different positions as marker images.

[0099] For example, when the captured image of the second image 81 received from the imaging device 90 includes only a part of the marker image of the second image 81, the computer 50 determines the positions of the marker images included in the captured image of the second image 81 in the projection range 11 based on the above-mentioned correspondence table, and synthesizes the captured image of the second image 81 with the captured images of the second images 82 to 84 based on the determination result. As a result, although a part of the area of ​​the second image 81 is missing, an image equivalent to the entire captured image of the first image 71 can be generated. Then, the computer 50 detects the distortion of the projected image in the projection range 11 based on the distortion of the shape and arrangement of the marker images included in the generated image, and performs distortion correction of the projected image based on the detection result.

[0100] 13 and 14, the marker images associated with positions in the projection range 11 have been described as being numbers that differ for each position, but the marker images associated with positions in the projection range 11 are not limited to this, and may be images of alphabets or symbols that differ for each position, for example. The marker images associated with positions in the projection range 11 may be images of different colors for each position. Furthermore, although an example has been described in which all marker images in an image are different, only some of the marker images in an image may be different.

[0101] Furthermore, the marker images associated with positions in the projection range 11 may be QR (Quick Response) codes (registered trademark) or ArUco markers that differ for each position. In this case, the marker images themselves can include information indicating the positions of the marker images in the projection range 11. Therefore, even if the computer 50 does not store the above-mentioned correspondence table, it can determine the positions of the marker images in the projection range 11 by reading information on the QR codes or ArUco markers included as marker images in the captured image.

[0102] <An example of stack projection using multiple projection devices before adjustment> Fig. 15 is a diagram showing an example of a state before adjustment of stack projection by a plurality of projection devices. In Fig. 15, the same parts as those shown in Fig. 1 are given the same reference numerals and the description thereof will be omitted. As shown in Fig. 15, the projection system 100 may further include a projection device 10A.

[0103] The projection device 10A has the same configuration as the projection device 10, and performs projection onto the projection object 6 together with the projection device 10. The computer 50 is capable of communicating with the projection device 10A. In the example shown in Fig. 1, the computer 50 is connected to the projection device 10A via a communication cable 8A, and is capable of communicating with the projection device 10A.

[0104] A projection range 11A indicated by a two-dot chain line is a region of the projection object 6 onto which projection light is irradiated by the projection device 10A. In the example shown in Fig. 15, the projection range 11A is rectangular. The projection range 11A is a part or the entirety of a projectable range onto which projection can be performed by the projection device 10A.

[0105] In this example, we will explain the case of performing stack projection, which aims to improve the dynamic range and gradation expression by superimposing the entire projection range 11 of projection device 10 and the entire projection range 11A of projection device 10A and projecting the same image from projection devices 10 and 10A.

[0106] <Processing Based on Imaging Conditions by Computer 50> Fig. 16 is a flowchart showing an example of processing based on imaging conditions by the computer 50. In the example shown in Fig. 15, the computer 50 may execute, for example, the processing shown in Fig. 16.

[0107] First, computer 50 acquires the imaging conditions of imaging device 90 (step S161). The imaging conditions of imaging device 90 include whether imaging by imaging device 90 is fixed imaging or handheld imaging. Fixed imaging is imaging in a state where imaging device 90 is fixed to a non-shakeable object such as a tripod or a pedestal. Handheld imaging is imaging in a state where imaging device 90 is held by the user's hand and shaking occurs.

[0108] The computer 50 acquires the imaging condition, whether fixed imaging or handheld imaging, by, for example, accepting a user operation on the computer 50. Alternatively, the computer 50 may acquire the imaging condition, whether fixed imaging or handheld imaging, based on the state of shake of the imaging device 90 based on a gyro sensor or the like in the imaging device 90 by communicating with the imaging device 90. For example, the computer 50 determines that the imaging is fixed imaging if the shake of the imaging device 90 detected by the gyro sensor or the like is less than a threshold value, and determines that the imaging is handheld imaging if the shake of the imaging device 90 detected by the gyro sensor or the like is equal to or greater than a threshold value.

[0109] Alternatively, computer 50 may acquire the imaging condition, whether fixed imaging or handheld imaging, based on the on / off state of the image stabilization function of imaging device 90 by communicating with imaging device 90. For example, computer 50 determines fixed imaging when the image stabilization function of imaging device 90 is off, and determines handheld imaging when the image stabilization function of imaging device 90 is on.

[0110] Next, computer 50 determines whether or not the imaging by imaging device 90 is handheld imaging based on the imaging conditions acquired in step S161 (step S162). If the imaging is not handheld imaging (step S162: No), computer 50 individually controls the projection adjustment of projection devices 10 and 10A (step S163) and ends the series of processes. The individual projection adjustment control in step S163 will be described with reference to Figs. 17 and 18.

[0111] In step S162, if the imaging is handheld imaging (step S162: Yes), computer 50 simultaneously performs projection adjustment control of projection devices 10, 10A (step S164), and ends the series of processes. The projection adjustment control in step S164 will be described with reference to FIG.

[0112] <Individual Projection of Projection Devices 10 and 10A> 17 and 18 are diagrams showing an example of individual projections of the projection devices 10 and 10A. When the imaging by the imaging device 90 is fixed imaging, in step S163 of Fig. 16, the computer 50 adjusts the projection of the projection device 10 by, for example, first executing the process shown in Fig. 12 for the projection device 10. Next, the computer 50 adjusts the projection of the projection device 10A by executing the process shown in Fig. 12 for the projection device 10A.

[0113] Fig. 17 shows a state in which a first image 71 is projected by the projection device 10 in step S1201 of the processing of Fig. 12 executed for the projection device 10. Although not shown, when step S1207 is executed in the processing of Fig. 12 executed for the projection device 10, second images 81 to 84 are projected from the projection device 10 in the same manner as Figs. 8 to 11.

[0114] Fig. 18 shows a state in which a first image 71A is projected by the projection device 10A in step S1201 of the process of Fig. 12 executed for the projection device 10A. The first image 71A shown in Fig. 18 is an image similar to the first image 71 shown in Fig. 17. However, the marker image included in the first image 71A is a marker image that can be distinguished by image recognition from the marker image included in the first image 71. Marker images that can be distinguished by image recognition are marker images that have, for example, different shapes, different colors, different line types (solid lines and dotted lines, etc.), or different spatial frequencies (arrangement intervals).

[0115] 17 and 18, the marker image included in the first image 71 is a white rectangular image, whereas the marker image included in the first image 71A is a black rectangular image. Although not shown, when step S1207 is executed in the process of FIG. 12 executed for the projection device 10A, the second image is projected from the projection device 10A in the same manner as in FIGS. 8 to 11. This second image is similar to the second images 81 to 84, but includes a black rectangular marker image instead of a white rectangular marker image.

[0116] 17 and 18, when the imaging by the imaging device 90 is fixed imaging, the computer 50 individually controls the projection adjustment of the projection devices 10 and 10A. In this case, the computer 50 projects the first images 71 and 71A from the projection devices 10 and 10A at different timings. However, since the imaging by the imaging device 90 is fixed imaging, the position and orientation of the imaging device 90 are fixed even if the projection and imaging of the first images 71 and 71A are different timings. Therefore, the computer 50 can determine the relative positional relationship between the projection range 11 and the projection range 11A based on the captured images of the first images 71 and 71A.

[0117] Furthermore, since the first images 71 and 71A are projected and captured at different times, the computer 50 can receive each of the captured images of the first images 71 and 71A from the imaging device 90. Therefore, the computer 50 does not need to perform a process of distinguishing and extracting each of the first images 71 and 71A from one captured image, and can reliably determine the relative positional relationship between the projection ranges 11 and 11A by simple processing.

[0118] <Simultaneous Projection by Projection Devices 10 and 10A> Fig. 19 is a diagram showing an example of simultaneous projection by the projection devices 10 and 10A. When the imaging by the imaging device 90 is handheld imaging, in step S164 of Fig. 16, the computer 50 simultaneously adjusts the projection of the projection devices 10 and 10A by, for example, collectively executing the processes shown in Fig. 12 for the projection devices 10 and 10A.

[0119] Fig. 19 shows a state in which first images 71 and 71A are simultaneously projected by the projection devices 10 and 10A in step S1201 of the process of Fig. 12 executed for the projection devices 10 and 10A. When step S1205 is executed in the process of Fig. 12 executed for the projection devices 10 and 10A, the computer 50 determines the relative positional relationship between the projection range 11 and the projection range 11A by distinguishing and extracting the first image 71 from the projection device 10 and the first image 71A from the projection device 10A from the captured image obtained in step S1203.

[0120] Although not shown in the figure, when step S1207 is executed in the processing of Figure 12 performed for projection devices 10 and 10A, first, the second image 81 from projection device 10 shown in Figure 8 and the second image from projection device 10A (for example, a portion of the 4 x 7 28 marker images cut out from the upper left corner of the first image 71A) are projected simultaneously.

[0121] Next, the second image 82 from the projection device 10 shown in Fig. 9 and the second image from the projection device 10A (for example, a portion of the 4 x 7 28 marker images at the top right corner cut out from the first image 71A) are simultaneously projected. Next, the second image 83 from the projection device 10 shown in Fig. 10 and the second image from the projection device 10A (for example, a portion of the 4 x 7 28 marker images at the bottom left cut out from the first image 71A) are simultaneously projected. Next, the second image 84 from the projection device 10 shown in Fig. 11 and the second image from the projection device 10A (for example, a portion of the 4 x 7 28 marker images at the bottom right cut out from the first image 71A) are simultaneously projected.

[0122] Then, in step S1211 of the processing of FIG. 12 executed for projection devices 10 and 10A, computer 50 determines the relative positional relationship between projection range 11 and projection range 11A by distinguishing and extracting the second image from projection device 10 and the second image from projection device 10A from each of the captured images obtained in step S1209.

[0123] 19, when the imaging device 90 captures images by hand, the computer 50 simultaneously performs projections of the projection devices 10 and 10A. As a result, even if the position or orientation of the imaging device 90 changes due to hand-held imaging, the image of the projection device 10 and the image of the projection device 10A can be extracted separately from the captured image including the image of the projection device 10 and the image of the projection device 10A, thereby determining the relative positional relationship between the projection range 11 and the projection range 11A. In addition, since the projection and capture of the first images 71 and 71A are simultaneous, the number of times the user of the imaging device 90 captures images can be reduced.

[0124] <Stack projection by overlapping projection ranges 11 and 11A> Fig. 20 is a diagram showing an example of stack projection by overlapping the projection ranges 11 and 11 A. As described in Fig. 17, Fig. 18 or Fig. 19, the computer 50 determines the relative positional relationship between the projection ranges 11 and 11 A.

[0125] Then, based on the result of the determination, the computer 50 adjusts the relative projection positions of the projection device 10 and the projection device 10A so that the entire projection range 11 and the entire projection range 11A are superimposed on each other, as shown in Fig. 20. This adjustment can be performed, for example, by controlling the shift mechanism (optical system shift mechanism or electronic shift mechanism) of at least one of the projection devices 10 and 10A.

[0126] For example, the computer 50 uses the projection range 11 of the projection device 10 as a reference and adjusts the projection range 11A by controlling the shift mechanism of the projection device 10A, thereby enabling stack projection by overlapping the entire projection range 11 with the entire projection range 11A.

[0127] For example, when an electronic shift mechanism is used, the computer 50 calculates transformation parameters for correcting the projection range 11A so that the projection range 11A coincides with the projection range 11. The transformation parameters include, for example, a projective transformation (homography) matrix. Then, the computer 50 can make the projection range 11A coincide with the projection range 11 by correcting the input image of the projection device 10A using the calculated transformation parameters and projecting it from the projection device 10A.

[0128] In this way, when performing stack projection in which the projection ranges of the projection devices 10, 10A (multiple projection devices) are overlapped, the computer 50 adjusts the overlap between the projection range 11 and the projection range 11A by adjusting the projection range 11A of the projection device 10A based on the projection range 11 of the projection device 10 as a reference.

[0129] Regarding projection by the projection devices 10, 10A, we have described a case in which projection adjustment control is performed in stack projection in which the entire projection range 11 of the projection device 10 and the entire projection range 11A of the projection device 10A are superimposed to project the same image from the projection devices 10, 10A. However, the form in which projection adjustment control is performed for projection by the projection devices 10, 10A is not limited to this.

[0130] For example, a blending projection may be performed to enlarge the projection screen by overlapping an end of the projection range 11 of the projection device 10 with an end of the projection range 11A of the projection device 10A and projecting divided images of a large image from the projection devices 10 and 10A, respectively.

[0131] <Before adjustment of blended projection using multiple projection devices> Fig. 21 is a diagram showing an example of a state before adjustment of blending projection by a plurality of projection devices. In Fig. 21, the same parts as those shown in Fig. 15 are given the same reference numerals and the description thereof is omitted. In the example of Fig. 21, in order to perform blending projection, the positions and orientations of the projection devices 10 and 10A are adjusted so that only the ends of the projection ranges 11 and 11A overlap.

[0132] In this case, similarly to the case of stack projection, the computer 50 can perform projection adjustment control when performing alignment and distortion correction of the projection ranges 11, 11A. For example, the computer 50 executes the process shown in Fig. 16 in the state shown in Fig. 21. In this case, the first image and the second image projected from the projection devices 10, 10A are similar to those in the case of stack projection.

[0133] However, in adjusting the projection ranges 11, 11A in this case, the computer 50 adjusts the relative positions of the projection ranges 11, 11A so that a specific area of ​​the projection range 11 (for example, an area of ​​a certain width on the right edge) and a specific area of ​​the projection range 11A (for example, an area of ​​a certain width on the left edge) are superimposed. The specific area of ​​the projection range 11 and the specific area of ​​the projection range 11A are the same size.

[0134] Furthermore, the computer 50 performs blending processing such as halving the brightness of the projected images from the projection devices 10 and 10A for the overlapping portion of the projection ranges 11 and 11A, respectively. This makes it possible to reduce the sense of incongruity caused by only the overlapping portion of the projection ranges 11 and 11A being displayed brightly.

[0135] <Blending projection by overlapping projection ranges 11 and 11A> 22 is a diagram showing an example of blending projection by superimposing the projection ranges 11, 11 A. The computer 50 determines the relative positional relationship between the projection ranges 11, 11 A based on the imaging data from the imaging device 90 in the process shown in FIG.

[0136] Then, based on the result of the determination, the computer 50 adjusts the relative projection positions of the projection device 10 and the projection device 10A so that the specific area of ​​the projection range 11 and the specific area of ​​the projection range 11A are superimposed, as shown in Fig. 22. This adjustment can be performed, for example, by controlling the shift mechanism (optical system shift mechanism or electronic shift mechanism) of at least one of the projection devices 10 and 10A.

[0137] For example, the computer 50 controls the shift mechanisms of the projection devices 10 and 10A to adjust the projection ranges 11 and 11A, thereby allowing a specific area of ​​the projection range 11 to overlap with a specific area of ​​the projection range 11A, thereby enabling blended projection.

[0138] For example, when an electronic shift mechanism is used, the computer 50 calculates transformation parameters for correcting the projection ranges 11, 11A so that the specific area of ​​the projection range 11 coincides with the specific area of ​​the projection range 11A. The transformation parameters include, for example, a projective transformation (homography) matrix. The computer 50 then corrects the input image of the projection device 10, 10A using the calculated transformation parameters and projects the image from the projection device 10, 10A, thereby making it possible to match the specific area of ​​the projection range 11 with the specific area of ​​the projection range 11A.

[0139] In this way, when performing blending projection in which parts of the projection ranges of the projection devices 10, 10A (multiple projection devices) are overlapped, the computer 50 adjusts the overlap between a specific area of ​​the projection range 11 and a specific area of ​​the projection range 11A by adjusting each of the projection ranges 11, 11A. As a result, by performing blending processing such as brightness adjustment on the specific area, the blending processing is applied only to the overlapping part of the projection range 11 and the projection range 11A, and the visual discomfort can be reduced.

[0140] <Another Example of Processing Based on Imaging Conditions by the Computer 50> Fig. 23 is a flowchart showing another example of processing based on image capturing conditions by the computer 50. The computer 50 may execute the processing shown in Fig. 23. In this example, the projection device that performs projection is, for example, the projection device 10 only.

[0141] First, the computer 50 acquires the imaging conditions of the imaging device 90 (step S231). The imaging conditions of the imaging device 90 include the resolution of the imaging by the imaging device 90. The resolution is the fineness of the imaging, and is determined by the number of pixels of the imaging sensor of the imaging device 90, the angle of view of the imaging by the imaging device 90, etc.

[0142] The computer 50 acquires the imaging resolution, for example, by receiving it through a user operation on the computer 50. Alternatively, the computer 50 may acquire the imaging resolution by communicating with the imaging device 90 and receiving information such as the number of pixels and the angle of view from the imaging device 90.

[0143] Next, the computer 50 sets the size of the second image based on the imaging resolution included in the imaging conditions acquired in step S231 (step S232). For example, the computer 50 sets the size of the second image to "normal" when the imaging resolution is equal to or greater than a threshold, and sets the size of the second image to "small" when the imaging resolution is less than the threshold.

[0144] Next, the computer 50 performs projection adjustment control of the projection device 10 (step S233), and ends the series of processes. In step S233, the computer 50 adjusts the projection of the projection device 10 by, for example, executing the process shown in Fig. 12 for the projection device 10. When executing step S1206 in the process shown in Fig. 12, the computer 50 generates the second image based on the size of the second image set in step S232.

[0145] For example, when the size of the second image is set to "normal" in step S232, computer 50 generates second images 81 to 84 shown in Figures 8 to 11 in step S1206. On the other hand, when the size of the second image is set to "small" in step S232, computer 50 generates second images smaller than second images 81 to 84 shown in Figures 8 to 11 in step S1206. Since this second image is smaller than second images 81 to 84 shown in Figures 8 to 11, computer 50 generates a greater number of second images than second images 81 to 84 shown in Figures 8 to 11 so as to cover projection range 11.

[0146] <Second image when the resolution of the image captured by the imaging device 90 is low> Fig. 24 is a diagram showing an example of the second image when the resolution of the image captured by the imaging device 90 is low. When the size of the second image is set to "small" in step S232 in Fig. 23, the computer 50 causes the projection device 10 to project, for example, the second image 81 shown in Fig. 24 in the first step S1206 of the process in Fig. 12.

[0147] The second image 81 shown in Fig. 24 is a smaller image containing fewer marker images than the second image 81 shown in Fig. 8. This allows the user of the imaging device 90 to get closer to the projection object 6 to capture the second image 81. Therefore, even if the imaging resolution of the imaging device 90 is low, it is possible to obtain a captured image from which the marker images contained in the second image 81 can be extracted with high accuracy.

[0148] FIG. 24 describes the second image 81 projected in step S1206 for the first time in the processing of FIG. 12. However, in step S1206 for the second or subsequent times in the processing of FIG. 12, computer 50 controls projection device 10 to project a second image similar to second image 81 shown in FIG. 24 while changing the projection position.

[0149] 23 and 24, the computer 50 may control the projection device 10 to project a second image having a size according to the resolution of the image captured by the imaging device 90. This makes it possible to obtain an imaged image from which a marker image included in the second image can be extracted with high accuracy even if the resolution of the image captured by the imaging device 90 is low, and allows the projection of the projection device 10 to be adjusted with high accuracy.

[0150] Although the above describes the control for projecting a second image having a size corresponding to the imaging resolution from projection device 10 when projection is performed only by projection device 10, similarly, when projection is performed by projection devices 10, 10A, control for projecting a second image having a size corresponding to the imaging resolution from projection device 10, 10A may be performed.

[0151] <Still another example of processing based on imaging conditions by the computer 50> Fig. 25 is a flowchart showing yet another example of processing based on imaging conditions by the computer 50. The computer 50 may execute the processing shown in Fig. 25. In this example, the projection device that performs projection is, for example, the projection device 10 only.

[0152] First, the computer 50 acquires the imaging conditions of the imaging device 90 (step S251), similar to step S231 in Fig. 23. Next, the computer 50 sets the size of the marker image included in the second image based on the imaging resolution included in the imaging conditions acquired in step S251 (step S252). For example, if the imaging resolution is equal to or greater than a threshold, the computer 50 sets the size of the marker image included in the second image to "normal," and if the imaging resolution is less than the threshold, the computer 50 sets the size of the marker image included in the second image to "large."

[0153] Next, the computer 50 performs projection adjustment control of the projection device 10 (step S253), and ends a series of processes. In step S253, the computer 50 adjusts the projection of the projection device 10 by, for example, executing the process shown in Fig. 12 for the projection device 10. When executing step S1206 in the process shown in Fig. 12, the computer 50 generates a second image based on the size of the marker image set in step S252.

[0154] For example, when the size of the marker image is set to "normal" in step S232, computer 50 generates, in step S1206, second images 81 to 84 shown in Figures 8 to 11. On the other hand, when the size of the marker image is set to "large" in step S232, computer 50 generates, in step S1206, second images including marker images larger than second images 81 to 84 shown in Figures 8 to 11.

[0155] <Marker image of the second image when the resolution of the image captured by the imaging device 90 is low> Fig. 26 is a diagram showing an example of a marker image of the second image when the resolution of the image captured by the imaging device 90 is low. When the size of the second image is set to "large" in step S252 of Fig. 25, the computer 50 causes the projection device 10 to project, for example, the second image 81 shown in Fig. 26 in the first step S1206 of the process of Fig. 12.

[0156] The marker image of the second image 81 shown in Fig. 26 is a larger image compared to the marker image of the second image 81 shown in Fig. 8. As a result, even if the resolution of the image captured by the imaging device 90 is low, it is possible to obtain a captured image of the second image 81 including a marker image that can be extracted with higher accuracy because it is larger.

[0157] FIG. 26 describes the second image 81 projected in step S1206 for the first time in the processing of FIG. 12. However, in step S1206 for the second or subsequent times in the processing of FIG. 12, computer 50 controls projection device 10 to project a second image similar to second image 81 shown in FIG. 26 while changing the projection position.

[0158] 25 and 26, the computer 50 may control the projection device 10 to project a second image including a marker image having a size according to the resolution of the image captured by the image capture device 90. This makes it possible to obtain a captured image including a marker image that can be extracted with higher accuracy even if the resolution of the image captured by the image capture device 90 is low, and allows the projection of the projection device 10 to be adjusted with higher accuracy.

[0159] Although the above has been described regarding the control of projecting a second image including a marker image of a size corresponding to the imaging resolution from the projection device 10 when projection is performed only by the projection device 10, similarly, when projection is performed by the projection device 10, 10A, control may be performed to project a second image including a marker image of a size corresponding to the imaging resolution from the projection device 10, 10A.

[0160] <Variation 1> 4 and 5, the projection device 10 is configured such that the optical axis K is not bent. However, the optical unit 106 may be provided with a reflecting member so that the optical axis K is bent one or more times.

[0161] Fig. 27 is a schematic diagram showing another external configuration of the projection device 10. Fig. 28 is a schematic cross-sectional view of the optical unit 106 of the projection device 10 shown in Fig. 27. In Fig. 27 and Fig. 28, the same parts as those shown in Fig. 4 and Fig. 5 are denoted by the same reference numerals and the description thereof will be omitted.

[0162] 27, the optical unit 106 includes a first member 102 supported by a main body 101, and a second member 103 supported by the first member 102. The first member 102 and the second member 103 may be integrated into one member.

[0163] As shown in FIG. 28, the optical unit 106 includes, in addition to a first member 102, a second member 103 having a hollow portion 3A connected to the hollow portion 2A of the first member 102, a first optical system 121 and a reflecting member 122 arranged in the hollow portion 2A, a second optical system 31, a reflecting member 32, a third optical system 33, and a lens 34 arranged in the hollow portion 3A, a first shift mechanism 105, and a projection direction change mechanism 104.

[0164] In the example of Fig. 27 and Fig. 28, the openings 2a and 2b of the first member 102 are formed on mutually perpendicular surfaces. The projection optical system 23 shown in Fig. 27 and Fig. 28 is composed of a reflecting member 122, a second optical system 31, a reflecting member 32, and a third optical system 33 in addition to the first optical system 121 and the lens 34 shown in Fig. 4 and Fig. 5. With such a projection optical system 23, as shown in Fig. 28, the optical axis K is folded back by bending twice. The first optical system 121, the reflecting member 122, the second optical system 31, the reflecting member 32, the third optical system 33, and the lens 34 are arranged along the optical axis K in this order from the light modulation unit 22 side.

[0165] The first optical system 121 guides light traveling in a direction X1 incident on the first member 102 from the main body 101 to the reflecting member 122. The reflecting member 122 reflects the light incident from the first optical system 121 in a direction Y1. The reflecting member 122 is formed of, for example, a mirror. In the first member 102, an opening 2b is formed on the optical path of the light reflected by the reflecting member 122, and this reflected light passes through the opening 2b and travels to the hollow portion 3A of the second member 103.

[0166] The second member 103 is a member having a substantially L-shaped cross-sectional outer shape, and has an opening 3a formed at a position facing the opening 2b of the first member 102. Light from the main body 101 that has passed through the opening 2b of the first member 102 passes through this opening 3a and enters the hollow portion 3A of the second member 103. Note that the cross-sectional outer shapes of the first member 102 and the second member 103 are arbitrary and are not limited to those described above.

[0167] The second optical system 31 includes at least one lens, and guides the light incident from the first member 102 to the reflecting member 32. The reflecting member 32 reflects the light incident from the second optical system 31 in the direction X2 and guides the light to the third optical system 33. The reflecting member 32 is formed of, for example, a mirror. The third optical system 33 includes at least one lens, and guides the light reflected by the reflecting member 32 to the lens 34.

[0168] The lens 34 is disposed at the end portion on the X2 side of the second member 103 so as to cover the opening 3c formed at this end portion. The lens 34 projects the light incident from the third optical system 33 onto the projection object 6.

[0169] Fig. 28 shows a state in which the first member 102 is moved to the maximum extent in the direction Y1 by the first shift mechanism 105. When the first member 102 is moved in the direction Y2 by the first shift mechanism 105 from the state shown in Fig. 28, the relative position between the center of the image formed by the light modulation unit 22 and the optical axis K changes, and the image G1 projected onto the projection object 6 can be shifted in the direction Y1.

[0170] The projection direction change mechanism 104 is a rotation mechanism that rotatably connects the second member 103 to the first member 102. The projection direction change mechanism 104 configures the second member 103 to be rotatable around a rotation axis (specifically, optical axis K) extending in the direction Y. Note that the projection direction change mechanism 104 is not limited to the arrangement position shown in FIG. 28 as long as it can rotate the optical system. Furthermore, the number of rotation mechanisms is not limited to one, and multiple mechanisms may be provided.

[0171] <Variation 2> Although the computer 50 has been described as an example of the control device of the present invention, the control device of the present invention is not limited to this. For example, the control device of the present invention may be the projection device 10 (or the projection device 10A). In this case, each control by the computer 50 described above is performed by the projection device 10. The projection device 10 may communicate with the imaging device 90 via the computer 50, or may communicate with the imaging device 90 without the computer 50. When the projection device 10 communicates with the imaging device 90 without the computer 50, the projection system 100 may be configured to omit the computer 50.

[0172] Alternatively, the control device of the present invention may be an imaging device 90. In this case, each control by the computer 50 described above is performed by the imaging device 90. The imaging device 90 may communicate with the projection devices 10, 10A via the computer 50, or may communicate with the projection devices 10, 10A without the computer 50. When the imaging device 90 communicates with the projection devices 10, 10A without the computer 50, the projection system 100 may be configured without the computer 50.

[0173] <Modification 3> Although the projection devices 10 and 10A have been described as an example of a plurality of projection devices, the plurality of projection devices may be three or more projection devices.

[0174] This specification describes at least the following:

[0175] (1) A control device for a projection system including one or more projection devices that project a first image including a plurality of marker images, and an imaging device that captures at least a portion of the first image, A processor is provided. The processor controls the projection device to project a second image including a plurality of marker images based on an imaging result of at least a part of the first image by the imaging device. Control device.

[0176] (2) The control device according to (1), The processor performs control to adjust the projection of the projection device based on the imaging result of the second image by the imaging device. Control device.

[0177] (3) The control device according to (1) or (2), One or more of the marker images are images associated with a position of the marker image in the projection range of the projection device. Control device.

[0178] (4) The control device according to any one of (1) to (3), Further, a correspondence table is provided that associates the marker image with a position of the marker image within a projection range of the projection device, the processor determines a position of the marker image within a projection range of the projection device based on the correspondence table; Control device.

[0179] (5) The control device according to any one of (1) to (4), The processor is calculating an image capturing range of the imaging device based on the imaging result of at least a part of the first image, and performing the control of projecting the second image based on the image capturing range from the projection device; Control device.

[0180] (6) The control device according to (5), The processor is repeating the control of generating the second image, which is different from the first image, based on the image capture range and projecting the second image from the projection device; Control device.

[0181] (7) The control device according to (6), The processor performs control to change a projection position of the second image when repeatedly performing the control to project the second image from the projection device. Control device.

[0182] (8) The control device according to (6) or (7), When the processor repeatedly performs the control of projecting the second image from the projection device, the processor performs control of changing the second image projected from the projection device. Control device.

[0183] (9) The control device according to any one of (1) to (8), The processor acquires an imaging condition of the imaging device, and performs the control of projecting the second image from the projection device based on the acquired imaging condition. Control device.

[0184] (10) The control device according to (9), The projection device includes a plurality of projection devices, The imaging conditions include whether imaging by the imaging device is fixed imaging or handheld imaging, When the imaging is the fixed imaging, the processor performs the control to project at least one of the first image and the second image from the multiple projection devices at different timings. Control device.

[0185] (11) The control device according to (10), When the imaging is the handheld imaging, the processor performs the control to simultaneously project at least one of the first image and the second image from the multiple projection devices, the distinguishable images being the first image and the second image. Control device.

[0186] (12) The control device according to any one of (9) to (11), The imaging conditions include a resolution of the imaging, The processor performs the control to project the second image having a size according to the resolution from the projection device. Control device.

[0187] (13) The control device according to any one of (9) to (11), The imaging conditions include a resolution of the imaging, The processor performs the control to project the second image including a marker image having a size according to the resolution from the projection device. Control device.

[0188] (14) The control device according to any one of (1) to (13), The projection device includes a plurality of projection devices in which at least a portion of the projection ranges are overlapped with each other, The processor performs control to adjust the superimposition. Control device.

[0189] (15) The control device according to (14), the processor performs the control of adjusting the superimposition by adjusting a projection range of a second projection device that is included in the plurality of projection devices and is different from the first projection device, based on a projection range of a first projection device that is included in the plurality of projection devices. Control device.

[0190] (16) The control device according to (15), the processor performs the control of adjusting the superposition by adjusting the projection range of the second projection device based on the projection range of the first projection device when the projection ranges of the multiple projection devices are superimposed. Control device.

[0191] (17) The control device according to (14), the processor performs the control of adjusting the superimposition by adjusting the projected images of the plurality of projectors; Control device.

[0192] (18) The control device according to (17), the processor performs the control of adjusting the overlap by adjusting the projection ranges of the plurality of projection devices when the projection ranges of the plurality of projection devices are to be partially overlapped. Control device.

[0193] (19) A control method by a control device of a projection system including one or more projection devices that project a first image including a plurality of marker images, and an imaging device that captures at least a part of the first image, the method comprising: The control device includes a processor; a processor controls the projection device to project a second image including a plurality of marker images based on an imaging result of at least a part of the first image by the imaging device; Control methods.

[0194] (20) The control method according to (19), The processor performs control to adjust the projection of the projection device based on the imaging result of the second image by the imaging device. Control methods.

[0195] (twenty one) The control method according to (19) or (20), One or more of the marker images are images associated with a position of the marker image in the projection range of the projection device. Control methods.

[0196] (twenty two) The control method according to any one of (19) to (21), Further, a correspondence table that associates the marker image with a position of the marker image within a projection range of the projection device, the processor determines a position of the marker image within a projection range of the projection device based on the correspondence table; Control methods.

[0197] (twenty three) The control method according to any one of (19) to (22), The processor is calculating an image capturing range of the imaging device based on the imaging result of at least a part of the first image, and performing the control of projecting the second image based on the image capturing range from the projection device; Control methods.

[0198] (twenty four) The control method according to (23), The processor is repeating the control of generating the second image, which is different from the first image, based on the image capture range and projecting the second image from the projection device; Control methods.

[0199] (twenty five) The control method according to (24), The processor performs control to change a projection position of the second image when repeatedly performing the control to project the second image from the projection device. Control methods.

[0200] (26) The control method according to (24) or (25), When the processor repeatedly performs the control of projecting the second image from the projection device, the processor performs control of changing the second image projected from the projection device. Control methods.

[0201] (27) The control method according to any one of (19) to (26), The processor acquires an imaging condition of the imaging device, and performs the control of projecting the second image from the projection device based on the acquired imaging condition. Control methods.

[0202] (28) The control method according to (27), The projection device includes a plurality of projection devices, The imaging conditions include whether imaging by the imaging device is fixed imaging or handheld imaging, When the imaging is the fixed imaging, the processor performs the control to project at least one of the first image and the second image from the multiple projection devices at different timings. Control methods.

[0203] (29) (28) The control method according to claim 1, When the imaging is the handheld imaging, the processor performs the control to simultaneously project at least one of the first image and the second image from the multiple projection devices, the distinguishable images being the first image and the second image. Control methods.

[0204] (30) The control method according to any one of (27) to (29), The imaging conditions include a resolution of the imaging, The processor performs the control to project the second image having a size according to the resolution from the projection device. Control methods.

[0205] (31) The control method according to any one of (27) to (29), The imaging conditions include a resolution of the imaging, The processor performs the control to project the second image including a marker image having a size according to the resolution from the projection device. Control methods.

[0206] (32) The control method according to any one of (19) to (31), The projection device includes a plurality of projection devices in which at least a portion of the projection ranges are overlapped with each other, The processor performs control to adjust the superimposition. Control methods.

[0207] (33) The control method according to (32), the processor performs the control of adjusting the superimposition by adjusting a projection range of a second projection device that is included in the plurality of projection devices and is different from the first projection device, based on a projection range of a first projection device that is included in the plurality of projection devices. Control methods.

[0208] (34) The control method according to (33), the processor performs the control of adjusting the superposition by adjusting the projection range of the second projection device based on the projection range of the first projection device when the projection ranges of the multiple projection devices are superimposed. Control methods.

[0209] (35) The control method according to (32), the processor performs the control of adjusting the superimposition by adjusting the projected images of the plurality of projectors; Control methods.

[0210] (36) The control method according to (35), the processor performs the control of adjusting the overlap by adjusting the projection ranges of the plurality of projection devices when the projection ranges of the plurality of projection devices are to be partially overlapped. Control methods.

[0211] (37) one or more projection devices for projecting a first image including a plurality of marker images; an imaging device that captures at least a portion of the first image; a control device; and a projection system comprising: The control device includes a processor. The processor controls the projection device to project a second image including a plurality of marker images based on an imaging result of at least a part of the first image by the imaging device. Projection system.

[0212] (38) A processor of a control device of a projection system including one or more projection devices that project a first image including a plurality of marker images and an imaging device that captures at least a portion of the first image, controlling the projection device to project a second image including a plurality of marker images based on an imaging result of at least a part of the first image by the imaging device; A control program for executing processing. [Explanation of symbols]

[0213] 1 Projection section 2 Operation reception section 2A,3A Hollow part 2a,2b,3a,3c,15a opening 4. Control section 4a Storage medium 5. Communications Department 6. Projection Object 8,8A,9 Communication cable 10,10A projection device 11,11A Projection range 12 Optical Modulation Unit 15. Cabinet 21 Light source 22 Optical modulation section 23 Projection optical system 24 Control circuit 31 Second optical system 32,122 Reflective material 33 Third optical system 34 Lens 50 Computer 51 Processors 52 Memory 53 Communication Interface 54 User Interface 59 Bus 71,71A 1st image 72 Imaging range 81~84 2nd image 90 Imaging Device 100 Projection System 101 Main body 102 First member 103 Second member 104 Projection direction change mechanism 105 First shift mechanism 106 Optical Unit 121 1st optical system G1 Image

Claims

1. A control device for a projection system including one or more projection devices that project a first image including a plurality of marker images, and an imaging device that captures at least a portion of the first image, A processor is provided. The processor calculates a relative size of an image capturing range of the imaging device with respect to the first image projected onto a projection object based on an imaging result of at least a part of the first image by the imaging device, and controls the projection device to project a second image based on the image capturing range, the second image including a plurality of marker images. Control device.

2. The control device according to claim 1 , The processor performs control to adjust the projection of the projection device based on the image capturing result of the second image by the image capturing device. Control device.

3. The control device according to claim 1 or 2, One or more of the marker images are images associated with a position of the marker image in a projection range of the projection device. Control device.

4. The control device according to any one of claims 1 to 3, Further, a correspondence table is provided that associates the marker image with a position of the marker image within a projection range of the projection device, the processor determines a position of the marker image in a projection range of the projection device based on the correspondence table; Control device.

5. A control device according to any one of claims 1 to 4, The processor, and repeating the control of generating the second image, which is different from the first image, based on the image capture range and projecting the second image from the projection device. Control device.

6. The control device according to claim 5, the processor performs control to change a projection position of the second image when repeatedly performing the control to project the second image from the projection device; Control device.

7. A control device according to claim 5 or 6, the processor performs control to change the second image projected from the projection device when repeatedly performing the control to project the second image from the projection device; Control device.

8. A control device according to any one of claims 1 to 7, The processor acquires an imaging condition of the imaging device, and performs the control to project the second image from the projection device based on the acquired imaging condition. Control device.

9. The control device according to claim 8, The projection device includes a plurality of projection devices; the imaging conditions include whether imaging by the imaging device is fixed imaging or handheld imaging, When the imaging is the fixed imaging, the processor performs the control to project at least one of the first image and the second image from the plurality of projection devices at different timings. Control device.

10. The control device according to claim 9, When the imaging is the handheld imaging, the processor performs the control to simultaneously project at least one of the first image and the second image from the multiple projection devices, the first image and the second image being distinguishable from each other. Control device.

11. A control device according to any one of claims 8 to 10, The imaging conditions include a resolution of the imaging, The processor performs the control to project the second image having a size according to the resolution from the projection device. Control device.

12. A control device according to any one of claims 8 to 10, The imaging conditions include a resolution of the imaging, The processor performs the control to project the second image including the marker image having a size according to the resolution from the projection device. Control device.

13. A control device according to any one of claims 1 to 12, the projection device includes a plurality of projection devices in which at least a portion of the projection ranges are overlapped; The processor performs control to adjust the superposition. Control device.

14. The control device according to claim 13, the processor performs the control of adjusting the superimposition by adjusting a projection range of a second projection device that is included in the plurality of projection devices and is different from the first projection device, based on a projection range of a first projection device that is included in the plurality of projection devices. Control device.

15. The control device according to claim 14, the processor performs the control of adjusting the superposition by adjusting the projection range of the second projection device based on the projection range of the first projection device when the projection ranges of the multiple projection devices are superimposed. Control device.

16. The control device according to claim 13, The processor performs the control to adjust the superimposition by adjusting the projected images of the plurality of projectors. Control device.

17. The control device according to claim 16, the processor performs the control of adjusting the overlap by adjusting the projection ranges of the plurality of projection devices when parts of the projection ranges of the plurality of projection devices are to be overlapped. Control device.

18. A control method by a control device of a projection system including one or more projection devices that project a first image including a plurality of marker images, and an imaging device that captures at least a portion of the first image, comprising: The controller comprises a processor; A processor calculates a relative size of an image capture range of the imaging device with respect to the first image projected onto a projection object based on an imaging result of at least a part of the first image by the imaging device, and controls the projection device to project a second image based on the image capture range, the second image including a plurality of marker images. Control methods.

19. The control method of claim 18, comprising: The processor performs control to adjust the projection of the projection device based on the image capturing result of the second image by the image capturing device. Control methods.

20. The control method according to claim 18 or 19, One or more of the marker images are images associated with a position of the marker image in a projection range of the projection device. Control methods.

21. A control method according to any one of claims 18 to 20, comprising: Further, a correspondence table that associates the marker image with a position of the marker image in a projection range of the projection device, the processor determines a position of the marker image in a projection range of the projection device based on the correspondence table; Control methods.

22. A control method according to any one of claims 18 to 21, comprising: The processor, and repeating the control of generating the second image, which is different from the first image, based on the image capture range and projecting the second image from the projection device. Control methods.

23. The control method of claim 22, comprising: the processor performs control to change a projection position of the second image when repeatedly performing the control to project the second image from the projection device; Control methods.

24. The control method according to claim 22 or 23, the processor performs control to change the second image projected from the projection device when repeatedly performing the control to project the second image from the projection device; Control methods.

25. A control method according to any one of claims 18 to 24, comprising: The processor acquires an imaging condition of the imaging device, and performs the control to project the second image from the projection device based on the acquired imaging condition. Control methods.

26. The control method of claim 25, The projection device includes a plurality of projection devices; the imaging conditions include whether imaging by the imaging device is fixed imaging or handheld imaging, When the imaging is the fixed imaging, the processor performs the control to project at least one of the first image and the second image from the plurality of projection devices at different timings. Control methods.

27. The control method of claim 26, When the imaging is the handheld imaging, the processor performs the control to simultaneously project at least one of the first image and the second image from the multiple projection devices, the first image and the second image being distinguishable from each other. Control methods.

28. A control method according to any one of claims 25 to 27, comprising: The imaging conditions include a resolution of the imaging, The processor performs the control to project the second image having a size according to the resolution from the projection device. Control methods.

29. A control method according to any one of claims 25 to 27, comprising: The imaging conditions include a resolution of the imaging, The processor performs the control to project the second image including the marker image having a size according to the resolution from the projection device. Control methods.

30. A control method according to any one of claims 18 to 29, comprising: the projection device includes a plurality of projection devices in which at least a portion of the projection ranges are overlapped; The processor performs control to adjust the superposition. Control methods.

31. The control method according to claim 30, the processor performs the control of adjusting the superimposition by adjusting a projection range of a second projection device that is included in the plurality of projection devices and is different from the first projection device, based on a projection range of a first projection device that is included in the plurality of projection devices. Control methods.

32. The control method according to claim 31, the processor performs the control of adjusting the superposition by adjusting the projection range of the second projection device based on the projection range of the first projection device when the projection ranges of the multiple projection devices are superimposed. Control methods.

33. The control method of claim 30, The processor performs the control to adjust the superimposition by adjusting the projected images of the plurality of projectors. Control methods.

34. The control method according to claim 33, comprising: the processor performs the control of adjusting the overlap by adjusting the projection ranges of the plurality of projection devices when parts of the projection ranges of the plurality of projection devices are to be overlapped. Control methods.

35. One or more projection devices that project a first image including a plurality of marker images; an imaging device that captures at least a portion of the first image; a control device; and a projection system comprising: The control device includes a processor. The processor calculates a relative size of an image capturing range of the imaging device with respect to the first image projected onto a projection object based on an imaging result of at least a part of the first image by the imaging device, and controls the projection device to project a second image based on the image capturing range, the second image including a plurality of marker images. Projection system.

36. A processor of a control device for a projection system including one or more projection devices that project a first image including a plurality of marker images, and an imaging device that captures at least a portion of the first image, Calculating a relative size of an image capture range of the imaging device with respect to the first image projected onto a projection object based on an imaging result of at least a part of the first image by the imaging device, and controlling the projection device to project a second image based on the image capture range, the second image including a plurality of marker images. A control program for executing processing.

Images