Installation Information Acquisition Method, Correction Method, Program, and Installation Information Acquisition System

Abstract

A projector in an installation information acquisition method is installed at an installation position in a real space, the projection direction thereof being variable, and projects a projection image based on a virtual image to a projection position. The virtual image is an image displayed at a display position in a virtual space that corresponds to the projection position in the real space, the image being viewed from a virtual installation position in the virtual space that corresponds to the installation position. The installation information acquisition method includes a first acquisition process, a projection process, a second acquisition process, and a third acquisition process. In the first acquisition process, position information of three or more first adjustment points in the virtual space is acquired. In the projection process, an index image is projected to the real space by the projector. In the second acquisition process, angle information relative to the reference direction of the projection direction of the projector in a state in which the position of the index image conforms to each of three or more second adjustment points. The three or more second adjustment points correspond to the three or more first adjustment points, respectively. In the third acquisition process, the installation information of the projector is acquired on the basis of the position information and the angle information.

Description

Installation information acquisition method, correction method, program, and installation information acquisition system

[0001] The present disclosure relates to an installation information acquisition method, a correction method, a program, and an installation information acquisition system for acquiring installation information related to the installation state of a projector that projects an image in an image projection system.

[0002] Japanese Patent Application Laid-Open No. 2006-124493 discloses a projector that includes an image projection unit that projects an image onto a screen and a correction processing unit. The correction processing unit deforms the image to compensate for trapezoidal distortion and barrel distortion in the projected image.

[0003] Furthermore, Patent Document 2 discloses an image display system that projects an image using a projector means within a projection mapping display area. In the image display system described in Patent Document 1, the projector means projects one image within an image display area and projects the image onto the image display area in response to the movement of the image display area. That is, in Patent Document 2, the image display area can be moved, rotated, or swiveled in any direction, and the projector (projector means) can also be moved or displaced in any direction. When the image display area moves, the image projection system (image display system) detects the movement of the image display area and moves the projector to follow the movement of the image display area, thereby changing the projection position where the image is projected in real space.

[0004] JP 2019-176356 A JP 2016-81497 A

[0005] Since the relative positional relationship between the projector and the screen can also cause distortion in the projected image, it has been necessary to install the projector in a manner that ensures a predetermined relative positional relationship between the projector and the screen.

[0006] An object of the present disclosure is to provide an installation information acquisition method, a correction method, a program, and an installation information acquisition system that can increase the degree of freedom in the installation state of a projector.

[0007] An installation information acquisition method according to one aspect of the present disclosure is a projector installation information acquisition method. The projector is installed at an installation position in a real space, and its projection direction is changeable relative to a reference direction. The projector projects a projection image based on a virtual image at the projection position in the real space. The virtual image is an image of an image displayed at a display position in a virtual space corresponding to the projection position in the real space, as viewed from a virtual installation position in the virtual space corresponding to the installation position. The installation information acquisition method includes a first acquisition process, a projection process, a second acquisition process, and a third acquisition process. The first acquisition process acquires position information regarding positions of three or more first adjustment points in the virtual space. The projection process causes the projector to project an index image into the real space. The second acquisition process acquires angle information regarding an angle of the projection direction of the projector relative to the reference direction when the position of the index image coincides with each of the three or more second adjustment points. The three or more second adjustment points respectively correspond to the three or more first adjustment points in the virtual space. In the third acquisition process, installation information relating to an installation state of the projector is acquired based on the position information and the angle information.

[0008] A correction method according to one aspect of the present disclosure corrects the projection image using the installation information acquired by the installation information acquisition method, the correction method including a correction process of correcting the virtual installation position based on the installation information and generating the projection image based on the virtual image viewed from the corrected virtual installation position.

[0009] According to an aspect of the present disclosure, a correction method is provided for correcting a projection direction in which the projected image is projected by the projector, using the angle information acquired by the installation information acquisition method, the correction method including a correction process for interpolating the projection direction at an arbitrary display position other than the three or more second adjustment points, using the angle information acquired for each of the three or more second adjustment points in the second acquisition process.

[0010] A program according to one aspect of the present disclosure is a program for causing one or more processors to execute the installation information acquisition method.

[0011] A program according to one aspect of the present disclosure is a program for causing one or more processors to execute the correction method.

[0012] An installation information acquisition system according to one aspect of the present disclosure is an installation information acquisition system for a projector. The projector is installed at an installation position in a real space, and its projection direction is changeable relative to a reference direction. The projector projects a projection image based on a virtual image at the projection position in the real space. The projection image is an image based on a virtual image projected at a display position in a virtual space corresponding to the real space, as viewed from a virtual installation position in the virtual space corresponding to the installation position. The installation information acquisition system includes a first acquisition unit, a projection processing unit, a second acquisition unit, and a third acquisition unit. The first acquisition unit acquires position information regarding the positions of three or more first adjustment points in the virtual space. The projection processing unit causes the projector to project an index image into the real space. The second acquisition unit acquires angle information regarding the angle of the projection direction of the projector relative to the reference direction when the position of the index image coincides with each of the three or more second adjustment points. The three or more second adjustment points respectively correspond to the three or more first adjustment points in the virtual space. The third acquisition unit acquires the installation information relating to an installation state of the projector based on the position information and the angle information.

[0013] According to the present disclosure, it is possible to increase the degree of freedom in the installation state of the projector.

[0014] FIG. 1 is a schematic configuration diagram of an image projection system according to an embodiment. FIG. 2 is a schematic block diagram of the image projection system. FIG. 3A is an explanatory diagram of a virtual space formed in a generation mode of a control system of the image projection system. FIG. 3B is an explanatory diagram of a state in which a first object is displayed in the virtual space in the generation mode of the control system of the image projection system. FIG. 3C is an explanatory diagram of a state in which a virtual camera is placed in the virtual space in the generation mode of the control system of the image projection system. FIG. 4 is an explanatory diagram showing how the control system of the image projection system generates image content and causes a projector to project it. FIG. 5A is an explanatory diagram of a second image rendered from a first image by the image projection system. FIG. 5B is an explanatory diagram of a second object projected into real space by the projector of the image projection system. FIG. 6A is an explanatory diagram of a second image rendered from a first image by the image projection system. FIG. 6B is an explanatory diagram of a second object projected into real space by the projector of the image projection system. FIG. 7 is a schematic explanatory diagram illustrating a method for adjusting the size of a rendering area of ​​the second image by the image projection system. Fig. 8 is an explanatory diagram illustrating how the image projection system of the same renders a second image from a first image displayed in a virtual space. Fig. 9 is an explanatory diagram illustrating how the image projection system of the same renders a second image from a first image displayed in a virtual space. Fig. 10 is an explanatory diagram illustrating how the image projection system of the same renders a second image from a first image displayed in a virtual space. Fig. 11 is an explanatory diagram illustrating how the image projection system of the same renders a second image from a first image displayed in a virtual space. Fig. 12 is an explanatory diagram illustrating how the image projection system of the same renders a second image from a first image displayed in a virtual space. Fig. 13 is a flowchart showing an example of an installation information acquisition method by an installation information acquisition system of the same. Fig. 14 is a schematic explanatory diagram illustrating an installation information acquisition method by the installation information acquisition system of the same. Fig. 15 is a schematic explanatory diagram illustrating an installation information acquisition method by the installation information acquisition system of the same.FIG. 16 is a schematic diagram illustrating an installation information acquisition method by the installation information acquisition system of the image projection system of the same. FIG. 17 is a schematic diagram illustrating an installation information acquisition method by the installation information acquisition system of the image projection system of the same. FIG. 18 is a flowchart illustrating an image adjustment method by the image projection system of the same. FIG. 19 is an explanatory diagram of a first adjusted image rendered from a first image by the image projection system of the same. FIG. 20 is an explanatory diagram of a second object based on the first adjusted image projected into real space by the image projection system of the same. FIG. 21 is an explanatory diagram of a second adjusted image rendered from the first image by the image projection system of the same. FIG. 22 is an explanatory diagram of a second object based on the second adjusted image projected into real space by the image projection system of the same. FIG. 23 is a schematic diagram illustrating an installation information acquisition method of the image projection system according to Modification 2. FIG. 24 is a schematic diagram illustrating an installation information acquisition method of the image projection system according to Modification 3. FIG. 25 is a schematic diagram illustrating an installation information acquisition method of the image projection system of the same. Fig. 26 is a schematic explanatory diagram illustrating a method for acquiring installation information for an image projection system according to Modification 4. Fig. 27 is a schematic explanatory diagram illustrating a method for acquiring installation information for the image projection system according to Modification 4. Fig. 28 is a schematic explanatory diagram illustrating a method for acquiring installation information for the image projection system according to Modification 4. Fig. 29 is a flowchart illustrating a method for adjusting an image using an image projection system according to Modification 5. Fig. 30 is an explanatory diagram of a first adjusted image projected into real space by the image projection system according to Modification 5. Fig. 31 is an explanatory diagram of a second object based on a second adjusted image projected into real space by the image projection system according to Modification 5. Fig. 32 is a schematic block diagram of an image projection system according to Modification 6. Fig. 33 is a flowchart illustrating a method for adjusting an image using an image projection system according to Modification 6.

[0015] (Embodiment) (1) Overview An overview of an installation information acquisition method and an image adjustment method for an image projection system 100 according to this embodiment will be described below with reference to Figures 1 to 7. As shown in Figure 2, the image projection system 100 includes a control system 10 and a projector 30 controlled by the control system 10.

[0016] The projector 30 is installed at an installation position P4 in the real space A1 and is capable of changing at least the projection direction. The projector 30 projects a third image Im3 (projected image) including a second object Ob2 at a projection position P1 on the projection surface A11, such as a wall, floor, or ceiling, by irradiating light toward the projection surface A11 in the real space A1. As shown in FIG. 4 , the projector 30 projects a second object Ob2 corresponding to the first object Ob1 based on the second image Im2 at a projection position P1 in the real space A1 corresponding to the display position of the first object Ob1 in the virtual space A2. The image content D1 is data representing a second image Im2 (virtual image) including the first object Ob1 (see FIG. 3C ) displayed at a display position P3 in the virtual space A2. Here, the second image Im2 is an image obtained by rendering the first image Im1 displayed in the virtual space A2 in a predetermined rendering area Ra1 with a viewpoint of a virtual installation position P2 in the virtual space A2 corresponding to the installation position P4. The first image Im1 is an image displayed in the virtual space A2 corresponding to the real space A1, and includes the first object Ob1.

[0017] The image content D1 in the present disclosure refers to the content of information (data) forming an image created based on the second image Im2, and the content of information (data) forming the third image Im3 projected by the projector 30. In other words, by inputting the image content D1 to the projector 30, the projector 30 can project the third image Im3 using the image content D1. In other words, the content of the third image Im3 is determined by the image content D1. The third image Im3 may be processed by applying, for example, trimming, brightness adjustment, or contrast adjustment to the image content D1.

[0018] The control system 10 mainly comprises a computer system having one or more processors and one or more memories. In other words, the installation information acquisition method and image adjustment method according to this embodiment are used on a computer system (control system 10). In other words, the installation information acquisition method and image adjustment method can also be embodied as a program. The program according to this embodiment is a program for causing one or more processors to execute the installation information acquisition method and image adjustment method according to this embodiment.

[0019] (1.1) Installation Information Acquisition Method The installation information acquisition method in the image projection system 100 is a method of acquiring installation information regarding the installation state of the projector 30 that projects image content D1 at a projection position P1 in the real space A1, as shown in FIG.

[0020] When the projector 30 projects the third image Im3, the size, shape, and orientation of the third image Im3 change depending on the installation position and installation orientation of the projector 30 relative to the projection position P1. Therefore, even when the third image Im3 is projected onto the front or left / right wall, floor, or ceiling, for example, to project the second object Ob2 in the third image Im3 with a constant size, shape, and orientation, the size, shape, and orientation of the first object Ob1 in the image content D1 must be determined in accordance with the relative positional relationship between the projector 30 and the projection position P1. As an example, if the projection distance from the projector 30 to the projection position P1 increases while the throw ratio is constant, the third image Im3 becomes larger. In this case, to maintain a constant size of the second object Ob2 in the third image Im3, it is necessary to reduce the size of the first object Ob1 in the image content D1 or to reduce the size of the third image Im3 itself by changing the throw ratio using a zoom lens. Furthermore, when the projection position P1 moves, for example, from the front wall to the left or right wall, the projection direction changes and the orientation of the second object Ob2 rotates. Even in this case, in order to keep the size, shape, and orientation of the second object Ob2 in the third image Im3 constant, it is necessary to increase the size of the first object Ob1 and rotate its orientation.

[0021] As described above, when generating image content D1 to be projected by projector 30, the installation state of projector 30 must be taken into consideration, which requires more effort than generating image content to be displayed on a non-projection display. Here, the installation information regarding the installation state of projector 30 includes information regarding at least one of the installation position of projector 30 relative to projection position P1 in real space A1 and the installation direction of projector 30. In particular, when projection position P1 changes from time to time in real space A1, the relative positional relationship between projector 30 and projection position P1 changes, and therefore it requires a great deal of effort to generate image content D1 that takes into consideration the positional relationship between the two, i.e., the installation state of projector 30.

[0022] Therefore, in this embodiment, the image content D1 to be projected by the projector 30 is generated by the following content generation method. In this content generation method, a virtual space A2 (see FIGS. 3A to 3C ) corresponding to the real space A1 is generated on a computer. In the virtual space A2, a virtual installation position P2 corresponding to the installation position of the projector 30 in the real space A1 and a display position P3 corresponding to the projection position P1 in the real space A1 are reproduced. Then, in the content generation method, a second image Im2 (virtual image) including the first object Ob1 as viewed from the virtual installation position P2 in a state where the first object Ob1 is projected (displayed) at the display position P3 in the virtual space A2 is generated as the image content D1. In other words, in a state where the first object Ob1 is displayed at the display position P3 in the virtual space A2, the second image Im2 including the first object Ob1 as viewed from the virtual installation position P2 is generated as the image content D1. In the virtual space A2, the first object Ob1 can be virtually displayed in any size, shape, and orientation at any display position P3. When a projector 30 installed at a real installation position P4 corresponding to the virtual installation position P2 projects a third image Im3 based on the image content D1 at the projection position P1, the projected second object Ob2 substantially coincides with the first object Ob1 virtually displayed at the display position P3. Therefore, if the projector 30 is installed in the specified installation state, the desired second object Ob2 can be projected at the projection position P1 by projecting the image content D1 created by the content generation method described above.

[0023] However, when the above-described image content D1 is projected by the projector 30, if the installation position and installation direction relative to the projection position P1 are significantly different from the installation position and installation direction assumed when the image content D1 was created, the second object Ob2 may not be correctly projected at the projection position P1. Here, "incorrectly projecting" the second object Ob2 refers to a state in which at least one of the size, shape, and orientation of the second object Ob2 projected at the projection position P1 does not match the size, shape, and orientation of the first object Ob1 displayed at the display position P3 in the virtual space A2. In particular, if the projection position P1 of the projector 30 in the real space A1 moves from one of the front or left / right wall surfaces, the floor, or the ceiling surface to another surface, the image may be deformed or otherwise incorrectly projected when projected across the surface.

[0024] Therefore, in the installation information acquisition method using the installation information acquisition system of this embodiment, installation information regarding the installation state of the projector 30 is acquired. The projector 30 is installed at an installation position P4 in the real space A1, and the projection direction is changeable relative to a reference direction. The projector 30 projects a third image Im3 based on the second image Im2 at a projection position P1 in the real space A1. The second image Im2 is an image projected at a display position P3 in a virtual space A2 corresponding to the real space A1, as viewed from a virtual installation position P2 in the virtual space A2 corresponding to the installation position P4. This installation information acquisition method includes a first acquisition process, a projection process, a second acquisition process, and a third acquisition process. In the first acquisition process, position information regarding the positions of three or more first adjustment points AP1 (see FIG. 14 ) in the virtual space A2 is acquired. In the projection process, an index image is projected into the real space A1 by the projector 30. In the second acquisition process, angle information is acquired regarding the angle of the projection direction of projector 30 with respect to the reference direction when the position of the index image coincides with each of three or more second adjustment points AP2 (see FIG. 15 ). The three or more second adjustment points AP2 correspond to three or more first adjustment points AP1 in virtual space A2, respectively. In the third acquisition process, installation information regarding the installation state of projector 30 is acquired based on the above position information and the above angle information.

[0025] Here, the installation information regarding the installation state of the projector 30 includes information regarding at least one of the installation position and installation direction of the projector 30 in the real space A1. The reference direction is a direction that serves as a reference when the projector 30, whose projection direction can be changed, projects an image. For example, if the projector 30 is capable of moving its projection direction horizontally and vertically (panning and tilting), the reference direction is the projection direction in which the pan angle and tilt angle are both zero. Here, when the pan angle and tilt angle are both zero, an image is projected directly in front of the projector 30, for example. In the following description, the state in which the projector 30 is installed in the installation position and installation direction assumed when creating the image content D1 may be referred to as the reference state. The three or more first adjustment points AP1 are points that are set in the virtual space A2 to estimate the installation state of the projector 30 in the real space A1. The three or more second adjustment points AP2 are points in the real space A1 that correspond to the three or more first adjustment points AP1 set in the virtual space A2, respectively.

[0026] In this installation information acquisition method, the second acquisition process acquires angle information when the position of the index image, i.e., the projection direction of the projector 30, coincides with each of three or more second adjustment points AP2. Here, the positions of the three or more second adjustment points AP2 correspond to the positions of three or more first adjustment points AP1 in the virtual space A2. Therefore, the third acquisition process acquires installation information of the projector 30 based on the position information and angle information of the three or more first adjustment points AP1, and it is possible to determine whether the installation state of the projector 30 deviates from the installation state at the virtual installation position P2. As a result, even if the projector 30 is installed in an installation state different from the installation state assumed when generating the virtual image, the projected image can be displayed more accurately by performing processing such as correcting the projection direction of the projector 30 or the image data of the virtual image based on the installation information acquired in the third acquisition process. Therefore, there is no need to strictly install the projector 30 in the real space A1 in accordance with the installation state assumed when generating the virtual image, which has the advantage of increasing the freedom of the installation state of the projector 30 in the real space A1.

[0027] The installation information acquisition method according to this embodiment is executed by an installation information acquisition system 15 as shown in FIG. 2 , for example. In other words, the installation information acquisition system 15 is one aspect for realizing the above-described installation information acquisition method. The installation information acquisition system 15 according to this embodiment acquires installation information related to the installation state of the projector 30. The projector 30 is installed at an installation position P4 in the real space A1, and its projection direction is changeable relative to a reference direction. The projector 30 projects a projection image based on a virtual image at a projection position P1 in the real space A1. The virtual image is an image projected at a display position P3 in a virtual space A2 corresponding to the real space A1, as viewed from a virtual installation position P2 in the virtual space A2 corresponding to the installation position P4. The installation information acquisition system 15 includes a first acquisition unit 151, a projection processing unit 154, a second acquisition unit 152, and a third acquisition unit 153. The first acquisition unit 151 acquires position information related to the positions of three or more first adjustment points AP1 in the virtual space A2. The projection processing unit 154 causes the projector 30 to project an index image into the real space A1. The second acquisition unit 152 acquires angle information regarding the angle of the projection direction of the projector 30 with respect to the reference direction when the position of the index image coincides with each of the three or more second adjustment points AP2. The three or more second adjustment points AP2 correspond to the three or more first adjustment points AP1 in the virtual space A2, respectively. The third acquisition unit 153 acquires installation information regarding the installation state of the projector 30 based on the above position information and the above angle information.

[0028] In the following embodiment, an example will be described in which the number of first adjustment points AP1 from which the first acquisition unit 151 acquires position information and the number of second adjustment points AP2 corresponding to the first adjustment points AP1 are three, but the number of first adjustment points AP1 and second adjustment points AP2 is not limited to three and may be four or more.

[0029] (1.2) Image Adjustment Method The image adjustment method in the image projection system according to this embodiment is a method of adjusting the size of the image projected by the projector 30 at the projection position P1 in the real space A1.

[0030] The image projection system 100 can project, for example, a second object Ob2 corresponding to one or more first objects Ob1 from the second image Im2 into the real space A1. When projecting only the first object Ob1 from the second image Im2 into the real space A1 as the second object Ob2, the control system 10 accepts the setting of a rendering area set as the second image Im2 from the first image Im1, creates image content D1 in which the pixel values ​​(brightness values) of the portion of the rendering area other than the second object Ob2, i.e., the portion corresponding to the background, are set to 0 (black), and causes the projector 30 to project a third image Im3 based on this image content D1 into the real space A1. Here, the size of the second object Ob2 projected into the real space A1 is determined based on the size of the first object Ob1 relative to the size of the rendering area in which the second image Im2 is rendered, with the virtual installation position P2 as the viewpoint. In other words, when the size of the first object Ob1 in the virtual space A2 is constant, the size of the second object Ob2 is determined according to the field of view (FOV) when the second image Im2 is captured by the virtual camera V30 installed at the virtual installation position P2. When displaying the second object Ob2 and a portion corresponding to the background in the third image Im3, the control system 10 receives the setting of a rendering area set as the second image Im2 from the first image Im1 and appropriately sets the pixel values ​​of the portion of the rendering area other than the second object Ob2, i.e., the portion corresponding to the background, to create the image content D1. The setting of the background may be determined by selecting background content in addition to setting the pixel values ​​of the background. Here, the background is determined in the first image Im1, for example, as an image that does not move over time. On the other hand, the first object Ob1 is determined in the first image Im1, for example, as an image that moves over time. In addition, by controlling the control system 10 using the input device 50, the user can determine one or more first objects to be projected into the real space A1, and can project a second object Ob2 corresponding to the first object Ob1 determined by the user into the real space A1.

[0031] Here, since the rendering area set as the second image Im2 in the virtual space A2 does not match the area onto which the image is projected at the angle of view of the projector 30 set in the real space A1, the position of the projected image, particularly the position of the object in the image, may deviate from the desired projection position.

[0032] For example, as shown in FIG. 5B , when projecting a third image Im3A using the projector 30, the third image Im3A includes a second object Ob22 (e.g., a building) that is to be displayed as a background in a fixed position in the real space A1 and a second object Ob23 (e.g., a person) that is to be moved from left to right in the real space A1, the control system 10 first generates a first image Im1 (see FIG. 5A ) in the virtual space A2, in which a first object Ob12 corresponding to the second object Ob22 is in a fixed position and a first object Ob13 corresponding to the second object Ob23 is at a starting point on the left side. The control system 10 then generates a second image Im2A by rendering the first image Im1 in a predetermined rendering area using the virtual installation position P2 as the viewpoint, and projects a third image Im3A based on the second image Im2A into the real space A1 using the projector 30. The control system 10 generates the second image Im2A so that the moving first object Ob13 is located at the image center CT1. Thereafter, the control system 10 creates a first image Im1 (moving image) in the virtual space A2 in which the first object Ob12 exists in a fixed position and the first object Ob13 gradually moves to the right, and renders the second image Im2A (moving image) so that the first object Ob13 is located at the image center CT1.

[0033] The control system 10 then causes the projector 30 to project a third image Im3A (moving image) based on the second image Im2A into the real space A1. In the real space A1, a moving image is projected in which the second object Ob22 is present in a fixed position and the second object Ob23 moves from left to right. The control system 10 changes the projection position of the third image Im3A by changing the projection direction of the projector 30 in accordance with the movement of the first object Ob13 in the virtual space A2, i.e., the movement of the image center CT1 of the second image Im2A. This causes the second object Ob23 projected into the real space A1 to move. Note that the second image Im2A is formed so that the first object Ob13 is present at the image center CT1. Therefore, when the first object Ob13 moves to the right, the first object Ob12, which is present in a fixed position, moves to the left relative to the first object Ob13 in the second image Im2A. Therefore, even if the projection direction of the projector 30 is moved to the right in accordance with the movement of the second object Ob23 corresponding to the first object Ob13, the second object Ob22 will be projected at a fixed position in the real space A1. In particular, even if the image center CT1 of the second image Im2A (i.e., the image center of the third image Im3A projected into the real space A1) moves in accordance with the movement of the first object Ob13 in the first image Im1, the second object Ob22 appears to not move in the real space A1.

[0034] Here, even if the image size of the third image Im3A projected into the real space A1 by the projector 30 does not match the size of the rendering region Ra1 of the second image Im2A created in the virtual space A2, the moving first object Ob13 is present at the image center CT1 of the second image Im2A, so its display position in the virtual space A2 and its projection position in the real space A1 are the same. On the other hand, the first object Ob12, which is present in a fixed position, is displayed at a position different from the image center CT1 of the second image Im2A. Therefore, if the image size of the rendering region Ra1 of the second image Im2A does not match the image size of the third image Im3A, the second object Ob22 corresponding to the first object Ob12 will be projected into a position in the real space A1 that is different from the projection position corresponding to its display position in the virtual space A2. Therefore, if the projection direction of the projector 30 is changed in accordance with the movement of the first object Ob13 (second object Ob23), the second object Ob22, which is supposed to be projected at a fixed position, may appear to be moving. Here, the fact that the size of the rendering region Ra1 of the second image Im2A and the image size of the third image Im3A do not match means that their relative sizes do not match. In other words, this means that the size obtained by converting the size of the rendering region Ra1 of the second image Im2A at the scale of the virtual space A2 to the scale of the real space A1 does not match the image size of the third image Im3A projected into the real space A1.

[0035] 5B shows an example of a third image Im3A projected onto the real space A1. In the example of FIG. 5B, the image size of the third image Im3A is larger than the size of the rendering area Ra1 of the second image Im2A (the size converted to the scale of the real space A1). Therefore, the second object Ob2, which should be projected at a fixed position, is projected at a position (shown by a solid line) different from the projection position (shown by a dotted line) corresponding to the display position in the virtual space A2. Furthermore, when the second image Im2B is rendered so that the first object Ob13 moves from left to right as shown in FIG. 6A, a problem occurs in that the second object Ob22, which should be projected at a fixed position in the real space A1, is projected to move from right to left as shown in FIG. 6B.

[0036] Therefore, in the image adjustment method of this embodiment, a first image generation step, a first adjusted image generation step, a first adjusted image projection step, a second adjusted image generation step, a second adjusted image projection step, and an adjustment step are performed to reduce the positional deviation between the projection position of the image projected into the real space A1 by the projector 30 and the position corresponding to the display position in the virtual space A2. Here, the image adjustment method of this embodiment will be described with reference to FIGS. 19 to 22. For convenience of explanation, FIGS. 19 to 22 assume that a diamond-shaped first object Ob5 is displayed in the first image Im1 and a diamond-shaped second object Ob6 corresponding to the first object Ob1 is projected into the real space A1; however, the shapes and sizes of the first object Ob1 and the second object Ob6 can be changed as appropriate.

[0037] In the first image generation step, a first image Im1 is generated in which the first object Ob5 is displayed at a second position P6 (a position shown in FIG. 19) in the virtual space A2 corresponding to a first position P5 in the real space A1 (the position of the second object Ob6 shown by a dotted line in FIG. 20). In the first adjusted image generation step, a first adjusted image Im21 (see FIG. 19) in which the first object Ob5 is positioned in a first direction (for example, the right direction, which is one of the pan directions) with respect to the image center CT1 is rendered as a second image Im2 from the first image Im1.

[0038] In the first adjusted image projection step, as shown in FIG. 20 , the projector 30 projects a second object Ob6 (specifically, a third image Im31 including the second object Ob6) based on the first adjusted image Im21 onto a first projection position PT1 in the real space A1. Note that because the size of the rendering region Ra1 of the first adjusted image Im21 does not match the image size of the third image Im31, the second object Ob6 is projected at a position shifted from the first position P5 corresponding to the second position P6 in the virtual space A2. In the examples of FIGS. 19 and 20 , the image size of the third image Im31 projected into the real space A1 is larger than the size of the rendering region Ra1 of the first adjusted image Im21 (the size when converted to the scale of the real space A1), so the second object Ob6 is projected to the right of the first position P5. That is, the distance from the image center CT2 of the third image Im31 to the actual projection position of the second object Ob6 is longer than the distance from the image center CT2 to the first position P5.

[0039] Note that when the image size of the third image Im31 projected into the real space A1 is smaller than the size of the rendering area Ra1 of the first adjustment image Im21 (the size when converted into the scale of the real space A1), the second object Ob6 is projected to the left of the first position P5 corresponding to the display position in the virtual space A2. In other words, the distance from the image center CT2 of the third image Im31 to the actual projection position of the second object Ob6 is shorter than the distance from the image center CT2 to the first position P5.

[0040] Next, in the second adjusted image generation step, a second adjusted image Im22 (see Figure 21) is rendered from the first image Im1 as the second image Im2, in which the first object Ob5 is positioned in a second direction (e.g., the left direction) opposite to the first direction relative to the image center CT1.

[0041] In the second adjusted image projection step, as shown in FIG. 22 , the projector 30 projects a second object Ob6 based on the second adjusted image Im22 (specifically, a third image Im32 including the second object Ob6) onto a second projection position PT2 (position indicated by a solid line in FIG. 22 ) in the real space A1. In the examples of FIGS. 20 and 22 , the image size of the third image Im32 is larger than the size of the rendering region Ra1 of the second adjusted image Im22 (the size when converted to the scale of the real space A1). Therefore, the second object Ob6 is projected to the left of the first position P5. Note that if the image size of the third image Im32 is smaller than the size of the rendering region Ra1 of the second adjusted image Im22 (the size when converted to the scale of the real space A1), the second object Ob6 is projected to the right of the first position P5.

[0042] In the adjustment step, the projection size of the second object Ob6 in the real space A1 is adjusted so as to reduce the positional deviation between the first projection position PT1 and the second projection position PT2. Note that the method for adjusting the projection size of the second object Ob6 in the real space A1 includes at least one of adjusting the zoom of the projector 30 and adjusting the size of the rendering area Ra1.

[0043] 19 to 22, the image size of the third image Im31 is larger than the size of the rendering region Ra1 of the first adjusted image Im21 (the size when converted to the scale of the real space A1), so the zoom adjustment of the projector 30 can be performed to reduce the projection size of the second object Ob6 projected by the projector 30. In Fig. 22, if the zoom adjustment is performed to reduce the image size (angle of view) of the third image Im32 to the size shown by the dotted line, the positional deviation between the first projection position PT1 and the second projection position PT2 can be reduced.

[0044] Here, in the case of a projector 30 that does not have zoom adjustment capability, the adjustment step may involve adjusting the size of the rendering region Ra1 of the second image Im2 that is rendered from the first image Im1, thereby adjusting the projection size of the second object Ob6, and thereby adjusting the projection position of the second object Ob6. In the case of a projector 30 that has zoom adjustment capability, the adjustment step may involve adjusting the zoom of the projector 30 to adjust the projection size of the second object Ob6, or may involve adjusting the size of the rendering region Ra1 that renders the second image Im2 to adjust the projection size of the second object Ob6, or may involve both adjusting the zoom of the projector 30 and adjusting the size of the rendering region Ra1 of the second image Im2. As described above, by adjusting the projection size of the second object Ob6 in the real space A1 in the adjustment step, the projection position of the second object Ob6 can be adjusted, thereby reducing the positional misalignment between the first projection position PT1 and the second projection position PT2.

[0045] Therefore, when the third image Im3 projected by the projector 30 includes a second object Ob22 (see Figure 5B) that is displayed at a fixed position in the real space A1, there is an advantage that the position of the second object Ob22 is less likely to change even if the projection direction of the projector 30 is changed.

[0046] 5A to 6B , the image sizes of the third images Im3A and Im3B are larger than the image sizes of the second images Im2A and Im2B. Therefore, as shown in FIG. 6B , by performing a zoom adjustment to reduce the image size (angle of view) of the third image Im3B to the size indicated by the dotted line, the misalignment between the first projection position PT1 and the second projection position PT2 can be reduced. Here, the projection size of the second object Ob6 in the real space A1 can be adjusted not only by zoom adjustment using the projector 30, but also by adjusting the size of the rendering region Ra1 of the second image Im2. For example, as shown in FIG. 7 , by increasing the size of the rendering region when rendering the second image Im2B to the size indicated by the dotted line, the projection size of the second object Ob6 in the real space A1 can be reduced, thereby reducing the misalignment between the first projection position PT1 and the second projection position PT2.

[0047] Moreover, the image adjustment method according to this embodiment is executed, for example, by a control system 10 as shown in Fig. 2. In other words, the control system 10 is one aspect for realizing the image adjustment method described above.

[0048] The control system 10 according to this embodiment controls the projector 30 of the image projection system 100. The projector 30 is installed at an installation position P4 in the real space A1, and is capable of changing at least the projection direction. In the image projection system 100, the projector 30 projects a second object Ob6 corresponding to the first object Ob5 based on a second image Im2 to a projection position in the real space A1 corresponding to the display position of the first object Ob5 in the virtual space A2. The second image Im2 is an image obtained by rendering the first image Im1 including the first object Ob5 in a predetermined rendering area Ra1, with a viewpoint being a virtual installation position P2 in the virtual space A2 corresponding to the installation position P4.

[0049] (2) Premise In the present disclosure, "image" includes a moving image (moving image) and a still image (still image). Furthermore, "moving image" includes an image composed of a plurality of still images obtained by stop-motion photography or the like. Furthermore, "image" includes a monochrome image and a color image. In this embodiment, as an example, the "image" is an image that changes over time (i.e., a moving image) and is a full-color image. In other words, in this embodiment, as an example, the third image Im3 projected by the projector 30 is a full-color moving image, and the image content D1, which is the content of the information forming the third image Im3, is also full-color moving image data.

[0050] In the present disclosure, "real space" refers to an actual space, i.e., a space that actually exists, and includes the interior space (indoor space) of an existing facility (building) and outdoor space. Facilities here include, for example, non-residential facilities such as commercial facilities, theme parks, office buildings, schools, welfare facilities, hospitals, and factories, as well as facilities such as apartment buildings and detached houses. Non-residential facilities also include theaters, movie theaters, public halls, amusement parks, complexes, restaurants, department stores, hotels, inns, kindergartens, libraries, museums, art galleries, underground shopping malls, stations, and airports. Furthermore, in the present disclosure, "facility" includes not only buildings (structures) but also outdoor facilities such as baseball stadiums, parking lots, sports fields, and parks. For example, in the interior space of a facility consisting of a structure, the surface of the facility's structure, such as a wall, floor, or ceiling (such as a wall, floor, or ceiling), can serve as the projection surface A11 for projecting the third image Im3. Furthermore, the surfaces of equipment (including fixtures, furniture, equipment, and devices) installed inside the facility, such as doors, partitions, shelves, desks, chairs, home appliances, whiteboards, screens, etc., can serve as the projection surface A11 for projecting the third image Im3. Outdoors, the surfaces of objects such as the exterior walls, roofs, or pillars of a facility consisting of a building, the ground, rocks, or trees, can serve as the projection surface A11 for projecting the third image Im3. In this embodiment, as an example, the real space A1 is an indoor space (indoor space) of a room in a non-residential facility such as a commercial facility, and a wall surface, floor surface, ceiling surface, or the like of this room will be described as the projection surface A11.

[0051] In the present disclosure, the term "virtual space" refers to a non-existent virtual space; it does not exist in real space (real space A1) and does not have any physical entity. Therefore, the virtual space A2 is, for example, composed of data processable by one or more processors, and is visually recognized by a user via a user interface such as a display device as a virtual space reproduced by a computer system. Here, the virtual space A2 is a space equivalent to the real space A1 onto which the third image Im3 is projected. In other words, the virtual space A2 is a space that simulates the real space A1 in which the projector 30 is installed, and is a virtual space in which, for example, structures such as walls, floors, and ceilings are laid out in the same manner as the real space A1. Therefore, the virtual space A2 includes a virtual projection surface A21 (see FIGS. 3A to 3C ) that corresponds to the projection surface A11 onto which the third image Im3 is projected in the real space A1. However, the virtual space A2 only needs to imitate the real space A1 in terms of at least the layout of the surface of the structure including the projection surface A11, and does not need to imitate the real space A1 in terms of the interior and back side of the structure, the surface condition of the structure, the layout of equipment such as lighting fixtures, etc. In this embodiment, as an example, the virtual space A2 is a three-dimensional space represented by an XYZ Cartesian coordinate system having three mutually orthogonal axes, namely, the X axis, the Y axis, and the Z axis.

[0052] Furthermore, the term "position" in the present disclosure may have a certain size and shape, or may be a "point" with no size. Therefore, for example, the projection position P1 may be defined as an area with a certain size and shape in the real space A1, or as a "point" on coordinates with no size. In the present embodiment, as an example, the projection position P1 is assumed to be an area with a certain size (area) on the projection surface A11. Similarly, the display position P3 (see FIG. 1 ) corresponding to the projection position P1 is assumed to be an area with a certain size (area) in the virtual space A2. On the other hand, the virtual installation position P2 (see FIGS. 1 and 3C ) is assumed to be a "point" with no size. Furthermore, the installation position P4 (see FIG. 1 ), which will be described later, represents the position where the projector 30 is installed in the real space A1, and is assumed to be a "point" with no size. In other words, the installation position P4 is the position of a point (e.g., the center point of the mirror unit 32, which will be described later) defined relative to the projector 30. Furthermore, the virtual installation position P2 and the display position P3 in the virtual space A2 are merely defined as existing virtually within the virtual space A2, and do not actually exist in the real space (real space A1) and do not have any substance.

[0053] In the present disclosure, an "object" is composed of data processable by one or more processors, does not actually exist in real space (actual space), and has no physical substance. The first objects Ob1, Ob5, Ob11-Ob13 and the second objects Ob2, Ob6, Ob22, and Ob23 are virtual models representing some kind of object (including a living thing), a design, a symbol, a number, a letter, or the like, and may be either two-dimensional or three-dimensional models. In this embodiment, the first object Ob1 and the second object Ob2 shown in FIGS. 1 and 4 are assumed to be two-dimensional models imitating a "butterfly." Furthermore, the first object Ob12 shown in FIGS. 5A, 6A, and 7 and the second object Ob22 shown in FIGS. 5B and 6B are assumed to be two-dimensional models imitating a "house" set as a background. It is also assumed that the first object Ob13 shown in FIGS. 5A, 6A, and 7, and the second object Ob23 shown in FIGS. 5B and 6B are two-dimensional models that resemble a person.

[0054] (3) Image Projection System (3.1) Overall Configuration The control system 10 includes an installation information acquisition system 15 that acquires installation information of the projector 30, and generates image data to be projected by the projector 30. The control system 10, together with the projector 30 that projects the generated image content D1, constitutes the image projection system 100. That is, the image projection system 100 includes the control system 10 that includes the installation information acquisition system 15, and the projector 30. The image projection system 100 can perform all of the processes necessary to project the third image Im3 at the projection position P1 in the real space A1, from the generation of the image content D1 to the projection of the generated image content D1.

[0055] As described above, the control system 10 is primarily configured as a computer system having one or more processors and one or more memories. In this embodiment, as an example, the control system 10 is realized by a single information terminal 1 made up of a personal computer. Dedicated application software (program) is installed in the information terminal 1 (see FIG. 1 ), and by starting this application software, the information terminal 1 functions as the control system 10, and the installation information acquisition method and the image adjustment method are realized.

[0056] The control system 10 is also connected to a display device 40 and an input device 50. The control system 10 displays information on the display device 40 to the user, and receives user operations via the input device 50. For example, in the installation information acquisition method according to this embodiment, it is possible to perform operations such as inputting position information of three first adjustment points AP1 on a graphical user interface (GUI) of a computer system, and in the image adjustment method, it is possible to perform operations such as instructing the execution of a first adjustment image projection step and a second adjustment image projection step on the GUI.

[0057] Furthermore, in the image projection system 100, the control system 10 is configured to be able to communicate with the projector 30. In this disclosure, "communicative" means being able to exchange information directly or indirectly via a network or a repeater, using an appropriate communication method such as wired or wireless communication. That is, the control system 10 and the projector 30 can exchange information with each other. In this embodiment, the control system 10 can transmit to the projector 30 an index image, a first adjusted image Im21, a second adjusted image Im22, and control information D2 for projecting these images. This allows the image projection system 100 to transmit image data for the index image, the first adjusted image Im21, and the second adjusted image Im22 to the projector 30 and cause the projector 30 to project these images. Furthermore, the control system 10 can transmit image content D1, control information D2, and the like to the projector 30. This allows the image projection system 100 to transmit the image content D1 generated by the control system 10 to the projector 30 and cause the projector 30 to project the image content D1.

[0058] In the present embodiment, as an example, the control system 10 and the projector 30 are wired together via a video cable 101, a control cable 102, and a communication cable 103. The video cable 101 conforms to a communication standard such as HDMI (registered trademark), for example, and is used to transmit image content D1 from the control system 10 to the projector 30. The control cable 102 conforms to a communication standard such as LAN (Local Area Network), for example, and is used to transmit control information D2 from the control system 10 to the projector 30. The communication cable 103 conforms to a communication standard such as LAN, for example, and is used to exchange various types of information between the control system 10 and the projector 30. The control cable 102 and the communication cable 103 may be a single LAN cable, or may be used separately to control the mirror unit 32 (described later) and the main body of the projector 30, respectively.

[0059] The projector 30 projects the image content D1 generated by the control system 10 onto a projection position P1 in the real space A1. Here, the projector 30 is installed at an installation position P4 in the real space A1. In the present embodiment, the installation position P4 is a single point determined with respect to the projector 30 as described above, and is, for example, the position of the center point of the mirror unit 32 described below. That is, the projector 30 is installed in the real space A1 so that the center point of the mirror unit 32 is located at the installation position P4. Based on the image content D1, the projector 30 irradiates light toward a projection surface A11, such as a wall, floor, or ceiling, present in the real space A1, thereby projecting a third image Im3 including a second object Ob2 onto the projection position P1 on the projection surface A11.

[0060] In this embodiment, the projector 30 projects, in real time, the image content D1 transmitted (distributed) from the control system 10. In other words, the control system 10 functions as a video playback device that plays back the image content D1 and outputs (transmits) a video signal. That is, in the image projection system 100, the control system 10 generates and plays back the image content D1, and the projector 30 projects the image content D1.

[0061] The display device 40 is realized by, for example, a liquid crystal display or an organic EL (Electro Luminescence) display, etc. The display device 40 receives a video signal from the control system 10 and displays a "screen" such as an input screen, thereby presenting the installation information acquisition method and image adjustment method to the user.

[0062] The input device 50 may be, for example, a keyboard, a pointing device such as a mouse, a mechanical switch, a gesture sensor, or a voice input device. The input device 50 receives user operations (including voice operations) in the installation information acquisition method and the image adjustment method, and outputs an operation signal corresponding to the user operation to the control system 10.

[0063] At least one of the display device 40 and the input device 50 may be included as a component of the control system 10. In other words, the control system 10 may include at least one of the display device 40 and the input device 50. Furthermore, for example, the display device 40 and the input device 50 may be realized by a touch panel display, in which case the display device 40 and the input device 50 are integrated.

[0064] (3.2) Projector Next, the configuration of the projector 30 will be described in more detail.

[0065] In this embodiment, the projector 30 is a floor-standing type that is placed on the floor (including the ground) for use. Therefore, the position (installation position P4) and installation direction of the projector 30 in the horizontal plane can be set arbitrarily depending on the position where the projector 30 is placed and the orientation of the projector 30 on the floor. Note that, although the projector 30 is placed on the floor in this embodiment, the projector 30 is not limited to being a floor-standing type, and may be fixed to an arm provided on a wall, floor, or ceiling, or may be suspended from the ceiling by a wire or the like, for example.

[0066] Furthermore, the projector 30 has, for example, an adjuster function on the legs that support the main body, and by adjusting the height of the main body from the floor, the vertical height can also be adjusted. This makes it possible to arbitrarily set the installation position P4, which is the position of the center point of the surface (reflection surface) of the mirror section 32 of the projector 30.

[0067] As an example, the projector 30 is installed at a position that is a suitable distance from any wall surface in a real space A1 that is an indoor space of a facility, as shown in Fig. 1. The projector 30 projects a third image Im3 at a projection position P1 on the projection surface A11 using the image content D1 by irradiating light onto a projection surface A11 such as a wall surface, floor surface, or ceiling surface that exists around the projector 30.

[0068] Here, the projector 30 is a movable projection system in which the projection position P1 is not fixed but is variable within the real space A1. In other words, the projection position P1 of the third image Im3 is movable within the real space A1. Here, the projector 30 moves the projection position P1 by changing the direction of light irradiation (i.e., the projection direction of the image). As an example, the projector 30 installed in front of the projection surface A11 changes the direction of light irradiation from the left edge to the right edge of the projection surface A11, thereby moving the projection position P1 to the right on the projection surface A11.

[0069] In this embodiment, the projector 30 is a moving-mirror projection system that moves a mirror unit 32 located on the optical path of the emitted light to change the direction of light irradiation and thereby move the projection position P1 of the third image Im3. That is, as shown in FIGS. 1 and 2 , the projector 30 includes a projection unit 31, a mirror unit 32, and a drive unit 33. The projection unit 31 uses image content D1 to emit light for projecting an image (third image Im3) into real space A1. The mirror unit 32 reflects the light emitted from the projection unit 31. The drive unit 33 drives the mirror unit 32 to change the orientation of the mirror unit 32, thereby changing the projection direction of the image and moving the projection position P1 of the image. In short, in this embodiment, the drive unit 33 causes the mirror unit 32 to oscillate, thereby changing the reflection direction of the light emitted from the projection unit 31 at the mirror unit 32, and changing the irradiation direction of the light from the projector 30 (the light reflected by the mirror unit 32), i.e., the projection direction of the image.

[0070] More specifically, as shown in FIG. 1 , the projection unit 31 includes, as an example, a cylindrical optical unit 311 that protrudes upward from the top surface of the projector 30. The optical unit 311 includes a lens system including a plurality of lens elements. When the projector 30 receives control information D2 including a control instruction input from the control system 10, the projector 30 adjusts the image size of the third image Im3 projected into the real space A1 (i.e., the projection size of the second object Ob2) by changing the zoom of the lens system in accordance with the control instruction. Note that in this embodiment, the user can also manually adjust the zoom of the lens system to adjust the image size of the third image Im3 projected into the real space A1 (i.e., the projection size of the second object Ob2). The projection unit 31 emits light upward from an opening on the top surface of the optical unit 311 along the optical axis of the optical unit 311. The light emitted from the projection unit 31 is light for projecting a third image Im3 including the second object Ob2, and when this light is irradiated onto the projection surface A11, an image of the third image Im3 including the second object Ob2 is formed on the projection surface A11, and the third image Im3 including the second object Ob2 is projected. When image content D1 is input to the projection unit 31, the projection unit 31 emits an image corresponding to the image content D1 as light. In this embodiment, the optical axis of the light emitted from the projection unit 31 is along the vertical direction and intersects with the center of the surface (reflective surface) of the mirror unit 32.

[0071] The mirror unit 32 is, for example, a plane mirror whose surface (reflecting surface) is polygonal (hexagonal) and flat, and is held so as to be able to swing by the drive unit 33. As described above, light from the projection unit 31 is incident from below onto the center point of the surface of the mirror unit 32, so the mirror unit 32 is basically held in a position with its surface facing diagonally downward. As a result, light emitted upward from the projection unit 31 is reflected laterally by the surface of the mirror unit 32 and irradiated toward a projection surface A11 such as a wall, floor, or ceiling surface around the projector 30.

[0072] The drive unit 33 holds the mirror unit 32 so that it can oscillate, and drives the mirror unit 32 to change the orientation of the mirror unit 32. The drive unit 33 is realized, for example, by an actuator including a motor or the like. When the drive unit 33 drives the mirror unit 32, the orientation of the mirror unit 32 changes and the projection position P1 moves within the real space A1. The drive unit 33 is controlled by control information D2 from the control system 10. In other words, the control information D2 transmitted from the control system 10 to the projector 30 includes information used to control the drive unit 33, and includes information for moving the projection position P1 at which the image is projected in the real space A1.

[0073] In this embodiment, the driver 33 is capable of two types of operations: a "panning operation" that rotates the mirror unit 32 around a vertical axis passing through the center point of the surface of the mirror unit 32, and a "tilting operation" that rotates the mirror unit 32 around a horizontal axis passing through the center point of the surface of the mirror unit 32. With the panning operation, the direction of the normal to the center point of the surface of the mirror unit 32 changes along the horizontal direction (also referred to as the panning direction), thereby changing the azimuth angle of the mirror unit 32. With the tilting operation, the direction of the normal to the center point of the surface of the mirror unit 32 changes along the vertical direction (also referred to as the tilting direction), thereby changing the elevation / depression angle of the mirror unit 32. In this embodiment, the driver 33 combines the panning operation and the tilting operation to freely change the orientation of the mirror unit 32 around the center point of the surface of the mirror unit 32. Therefore, the projection position P1 where the light reflected by the mirror unit 32 is irradiated (i.e., where the image is projected) can be moved in two dimensions, up, down, left, and right, on the projection surface A11, which is, for example, a wall surface.

[0074] (3.3) Control System Next, a more detailed configuration of the control system 10 including the installation information acquisition system 15 according to this embodiment will be described.

[0075] 2 , the control system 10 includes a first image generation unit 11, a second image generation unit 12, a projection control unit 13, an adjustment unit 14, an installation information acquisition system 15, a correction processing unit 16, a generation unit 18, a control information generation unit 19, an input unit 20, an output unit 21, a communication unit 22, and a data storage unit 23. The installation information acquisition system 15 includes a first acquisition unit 151, a second acquisition unit 152, a third acquisition unit 153, and a projection processing unit 154.

[0076] In this embodiment, as described above, the control system 10 is mainly configured as a computer system having one or more processors and one or more memories. Specifically, the functions of the control system 10 other than the communication unit 22 and the data storage unit 23 are realized by the one or more processors executing programs.

[0077] The first image generation unit 11 generates a first image Im1 in which a first object Ob5 is displayed at a second position P6 (see Figure 19) in the virtual space A2 corresponding to a first position P5 (see Figure 20) in the real space A1.

[0078] The second image generation unit 12 renders the second image Im2 from the first image Im1.

[0079] The projection control unit 13 causes the projector 30 to project a second object Ob6 corresponding to the first object Ob5 based on the second image Im2.

[0080] The second image generation unit 12 renders, from the first image Im1 to the second image Im2, a first adjusted image Im21 (see FIG. 19 ) in which the first object Ob5 is located in a first direction (e.g., the rightward direction) with respect to the image center CT1. The second image generation unit 12 also renders, from the first image Im1 to the second image Im2, a second adjusted image Im22 (see FIG. 21 ) in which the first object Ob5 is located in a second direction (e.g., the leftward direction) opposite to the first direction with respect to the image center CT1.

[0081] The projection control unit 13 performs a first projection process and a second projection process. In the first projection process, the projector 30 projects a second object Ob6 (see FIG. 20 ) based on a first adjusted image Im21 onto a first projection position PT1 in the real space A1. In the second projection process, the projector 30 projects a second object Ob6 (see FIG. 22 ) based on a second adjusted image Im22 onto a second projection position PT2 in the real space A1.

[0082] The adjustment unit 14 adjusts the projection size of the second object Ob6 in the real space A1 so as to reduce the positional deviation between the first projection position PT1 and the second projection position PT2.

[0083] In the following embodiments, the control system 10 is connected to an input device 50 that accepts adjustment instructions input by, for example, a user, and the adjustment unit 14 performs the above adjustment steps in accordance with the adjustment instructions accepted by the input device 50. Note that the adjustment unit 14 may cause the projector 30 to perform zoom adjustment by outputting a control instruction to the projector 30 to perform zoom adjustment based on the adjustment instructions accepted by the input device 50. Furthermore, the adjustment unit 14 may cause the user to manually adjust the zoom of the projector 30 by, for example, displaying the details of the zoom adjustment on the display device 40 based on the adjustment instructions accepted by the input device 50. Furthermore, the control system 10 may be configured to execute some or all of the steps of the image adjustment method based on the user's operation instructions accepted by the input device 50, or may be configured to automatically execute some or all of the steps of the image adjustment method.

[0084] As described above, the first acquisition unit 151 acquires position information related to the positions of the three first adjustment points AP1 set in the virtual space A2. In this embodiment, the first acquisition unit 151 acquires the position information from the input unit 20.

[0085] As described above, the projection processing unit 154 causes the projector 30 to project the index image into the real space A1.

[0086] As described above, the second acquisition unit 152 acquires angle information regarding the angle of the projection direction of the projector 30 with respect to the reference direction when the position of the index image coincides with each of the three second adjustment points AP2 in the real space A1. The three second adjustment points AP2 in the real space A1 correspond to the three first adjustment points AP1 in the virtual space A2, respectively. For example, if the installation state of the projector 30 is deviated from the reference state, when the projector 30 projects an index image at the second adjustment point AP2, the position of the actually projected index image may deviate from the position of the second adjustment point AP2. Here, when a user of the image projection system 100 moves the mirror unit 32 of the projector 30 directly by hand or moves the mirror unit 32 via the input device 50 to display the index image at the position of the second adjustment point AP2, the projection direction of the projector 30 is moved by the angle necessary to coincide the index image with the position of the second adjustment point AP2. The drive unit 33 of the projector 30 can detect the angle of the projection direction of the projector 30 with respect to a reference direction using, for example, a sensor provided on an actuator such as a motor, and output angle information related to this angle. Here, since the relationship between the control amount of the projection direction (pan angle and tilt angle) and the angle is known in advance, the second acquisition unit 152 acquires angle information from the control amount. Note that the second acquisition unit 152 may acquire angle information related to the angle of the projection direction of the projector 30 with respect to the reference direction when the position of the index image coincides with each second adjustment point AP2 by communicating with the projector 30.

[0087] As described above, the third acquisition unit 153 acquires installation information of the projector 30 based on position information about the positions of the three first adjustment points AP1 in the virtual space A2 and angle information when projecting index images onto the three second adjustment points AP2, respectively. The three second adjustment points AP2 are set at positions in the real space A1 that correspond to the three first adjustment points AP1, respectively. Therefore, based on the position information and angle information of the second adjustment points AP2 calculated from the position information of the first adjustment points AP1, the third acquisition unit 153 can acquire installation information about the installation position and installation direction of the projector 30 in the real space A1 (i.e., installation information about the installation state).

[0088] The correction processing unit 16 performs a correction process to correct the projection direction in which the projector 30 projects an image, based on the angle information acquired by the second acquisition unit 152. For example, the correction processing unit 16 corrects the projection direction (pan angle and tilt angle) in which the projector 30 projects an image, based on angle information related to the projection direction when the image is projected onto the second adjustment point AP2 in the real space A1, thereby projecting the image in a desired direction. Note that it is not essential for the correction processing unit 16 to perform the correction process to correct the projection direction in the control system 10 of this embodiment. The correction process performed by the correction processing unit 16 will be described in "(5.4) Modification 4."

[0089] The generation unit 18 executes a generation process for generating image content D1. In the generation process, a predetermined rendering area Ra1 is rendered from a first image Im1 in which a first object Ob1 is displayed (placed) at a display position P3 in a virtual space A2, thereby generating a second image Im2 (image content D1). Specifically, the generation unit 18 generates a virtual space A2 simulating the real space A1, and generates image content D1 (second image Im2) as an image captured at the display position P3 from a virtual camera V30 (see FIGS. 1 and 3C ) installed at a virtual installation position P2, with the first object Ob1 virtually displayed at the display position P3 in the virtual space A2. In other words, the rendering area Ra1 used when rendering the second image Im2 corresponds to the area covered by the angle of view of the virtual camera V30 when capturing the second image Im2. Note that the virtual display of the first object Ob1 at the display position P3 in the virtual space A2 is executed by the generation unit 18.

[0090] The control information generator 19 executes a control information generation process to generate control information D2. In the control information generation process, control information D2 for the projector 30 is generated to move the projection position P1 in the real space A1 in synchronization with the movement of the display position P3. The control information D2 generated by the control information generator 19 is output from the output unit 21.

[0091] As an example, in this embodiment, the generation unit 18 virtually installs a virtual camera V30 (see FIG. 3C ) at a virtual installation position P2 in the virtual space A2. That is, the installation information acquisition method and image adjustment method of this embodiment further include a placement process for placing the virtual camera V30 at the virtual installation position P2 on a display screen representing the virtual space A2. The term "virtual camera" as used in this disclosure is a concept for identifying the rendering area Ra1, a virtual device that simulates the functions of a camera, and is merely defined as virtually existing in the virtual space A2; it does not actually exist in real space (real space A1) and does not have a physical entity. In this embodiment, the virtual camera V30 simulates the functions of a video camera capable of capturing full-color video.

[0092] In this embodiment, the virtual installation position P2 represents the position where the virtual camera V30 is installed in the virtual space A2, and, like the installation position P4, is a "point" with no size. In other words, the virtual installation position P2 is a position in the virtual space A2 that corresponds to the installation position P4 in the real space A1. Here, the virtual installation position P2 is a single point position determined with respect to the virtual camera V30, and as an example, is the position of the center point of a virtual mirror section V32, which will be described later. In other words, the virtual camera V30 is virtually installed in the virtual space A2 so as to be located at the virtual installation position P2, which is the center point of the virtual mirror section V32.

[0093] Like a typical camera, the virtual camera V30 has the function of capturing an image of a subject present within the field of view of the virtual camera V30 and outputting the image captured within the field of view as an image (image signal). Therefore, as shown in FIG. 3C , when a first object Ob1 is virtually displayed at display position P3 in virtual space A2 and the virtual camera V30 captures the display position P3, a second image Im2 is output from the virtual camera V30, where the second image Im2 is rendered within a predetermined rendering area Ra1 with the virtual installation position P2 as the viewpoint. In other words, the output second image Im2 includes the first object Ob1, and the rendering area Ra1 coincides with the area seen by the angle of view of the virtual camera V30. Therefore, the generation unit 18 defines the virtual camera V30 and generates the image content D1 by using the second image Im2 captured by the virtual camera V30 as the image content D1. In other words, the generation unit 18 generates the second image Im2 by rendering the first image Im1, which is displayed (placed) at the display position P3, in a predetermined rendering area Ra1 with the virtual installation position P2 as the viewpoint.

[0094] The input unit 20 accepts input of various information and signals from outside the control system 10. The input unit 20 accepts input of at least an operation signal from the input device 50. This allows the control system 10 to indirectly accept user operations (including voice operations, etc.) in the installation information acquisition method and the image adjustment method via the input device 50. The input unit 20 may accept input of information and signals from outside via the communication unit 22.

[0095] The output unit 21 outputs at least the image content D1 generated by the generation unit 18. The output unit 21 also outputs the control information D2 generated by the control information generation unit 19. Furthermore, the output unit 21 outputs a video signal for displaying, for example, a screen for executing the installation information acquisition method and the image adjustment method on the display device 40. Here, the output manner of the output unit 21 includes, for example, display, communication (transmission), sound, printing (printout), and recording (writing) to a non-temporary recording medium. As an example in the present embodiment, the output unit 21 outputs at least the image content D1 and the control information D2 by recording (writing) them to the data storage unit 23 and by communication (transmission) from the communication unit 22 to the projector 30.

[0096] The communication unit 22 has a function of communicating with the projector 30. In this embodiment, the communication unit 22 is connected to the projector 30 by wire via a video cable 101, a control cable 102, and a communication cable 103.

[0097] The data storage unit 23 stores image content D1, control information D2, etc. The data storage unit 23 also stores position information of the first adjustment point AP1 acquired by the first acquisition unit 151, angle information acquired by the second acquisition unit 152, and installation information acquired by the third acquisition unit 153. Furthermore, the data storage unit 23 stores information necessary for calculations in the first image generation unit 11, the second image generation unit 12, the projection control unit 13, the first acquisition unit 151, the second acquisition unit 152, the third acquisition unit 153, the generation unit 18, and the control information generation unit 19, etc. The data storage unit 23 includes a rewritable non-volatile memory such as an EEPROM (Electrically Erasable Programmable Read-Only Memory).

[0098] The operation of each part of the control system 10 will be explained in detail in the section "(4.1) Generation Mode."

[0099] The control system 10 has a function of generating image content D1 based on the second image Im2, a function of acquiring installation information, a function as a video playback device that plays back the image content D1 and outputs (transmits) a video signal, and a function of adjusting the projection size of the second object projected into the real space A1. However, it is assumed that the control system 10 generates the image content D1, acquires the installation information, plays back the image content D1 (outputs the video signal), and adjusts the projection size of the second object in separate situations rather than simultaneously.

[0100] Therefore, the control system 10 has at least four operating modes: a generation mode, an installation information acquisition mode, an adjustment mode, and a projection mode. The generation mode is an operating mode for generating image content D1. When operating in the generation mode, the control system 10 generates image content D1 and stores (stores) the generated image content D1 in the data storage unit 23. The installation information acquisition mode is an operating mode for acquiring installation information related to the installation state of the projector 30. When operating in the installation information acquisition mode, the installation information acquisition system 15 causes the projector 30 to project an index image into the real space A1. Then, the installation information acquisition system 15 acquires installation information of the projector 30 based on angle information when aligning the index image with the position of the second adjustment point AP2 and position information of the first adjustment point AP1 (i.e., the second adjustment point AP2). The adjustment mode is an operation mode for performing image adjustment so as to reduce the difference between the size of the rendering area Ra1 of the second image Im2 in the virtual space A2 (the size when converted to the scale of the real space A1) and the image size of the third image Im3 in the real space A1. The projection mode is an operation mode for projecting the image content D1, and when operating in the projection mode, the control system 10 plays the image content D1 stored in the data storage unit 23 and outputs (transmits) a video signal to the projector 30. Switching between the operation modes (generation mode, installation information acquisition mode, adjustment mode, projection mode) may be performed manually by a user's operation on the input device 50, or automatically, for example.

[0101] (4) Description of Operation (4.1) Generation Mode The operation of the control system 10 according to this embodiment in the generation mode will be described below with reference to Figures 3A to 4. Figures 3A to 3C are explanatory diagrams that visualize the virtual space A2 created by the control system 10 to generate image content D1. Figure 4 is an explanatory diagram conceptually showing the processing flow in which image content D1 is generated by a virtual camera V30 in the virtual space A2 and the generated image content D1 is projected by a projector 30 in the real space A1.

[0102] When generating the image content D1, the generation unit 18 first forms a virtual space A2 corresponding to the real space A1, as shown in FIG. 3A . The generation unit 18 forms the virtual space A2 based on spatial information specifying the layout of structures constituting the space, such as walls, floors, and ceilings, and items arranged in the real space A1 (e.g., a clock 200, home appliances such as a television, and decorative items such as paintings). This spatial information may be stored in advance in the data storage unit 23, or may be acquired from an external server device, etc. The virtual space A2 is a non-existent virtual space that is composed of data processable by one or more processors and is reproduced by a computer system. In the virtual space A2 thus formed, for example, the surfaces of structures such as walls, floors, and ceilings and items arranged in the real space A1 become a virtual projection surface A21 corresponding to the projection surface A11 of the real space A1.

[0103] In addition, the virtual space A2 does not require the same length (dimension) scale as the real space A1, so for example, in the case of a wall, the aspect ratio needs to be the same in the real space A1 and the virtual space A2, and the height dimension may be different in the real space A1 and the virtual space A2.

[0104] Next, the control system 10 identifies a display position P3 as shown in Fig. 3B based on display position information about a display position P3 corresponding to the projection position P1 in the real space A1. The display position information is information used to identify the display position P3 in the virtual space A2, and is information that identifies a position within a virtual projection plane A21 made up of the surfaces of structures such as walls, floors, and ceilings in the virtual space A2. The display position information may be stored in advance in the data storage unit 23, for example, or may be input by the user by the input unit 20, or may be obtained from an external server device or the like.

[0105] Next, the generation unit 18 acquires information about the first object Ob1 from, for example, the data storage unit 23, etc., to display the first object Ob1 as shown in FIG. 3B . Specifically, the generation unit 18 acquires data for displaying the first object Ob1 on the virtual projection surface A21 on three-dimensional computer graphics software from the data storage unit 23, etc. Then, the generation unit 18 places the first object Ob1 at a display position P3 in the virtual space A2. The first object Ob1 is virtually displayed by placing the first object Ob1 at the display position P3 set on the virtual projection surface A21 in the virtual space A2.

[0106] Next, based on the installation position P4 (design value) where the projector 30 is installed in the real space A1, the generation unit 18 places a virtual camera V30 as shown in Figure 3C at a virtual installation position P2 corresponding to the installation position P4.

[0107] Here, the virtual camera V30 is a virtual device simulating the projector 30. That is, in accordance with the movable projector 30, the virtual camera V30 is a movable camera system whose field of view is not fixed but variable within the virtual space A2. That is, the virtual camera V30 moves its field of view by moving a virtual mirror unit V32 located on the optical path of the incident light, thereby changing the incident direction of the light. That is, as shown in FIG. 3C , the virtual camera V30 has a virtual shooting unit V31 and a virtual mirror unit V32. The virtual shooting unit V31 captures an object present within its field of view and outputs the image reflected within the field of view as an image (image signal). The virtual mirror unit V32 reflects light arriving from the surroundings toward the virtual shooting unit V31. The virtual camera V30 moves the field of view of the virtual camera V30 by driving the virtual mirror unit V32 to change the orientation of the virtual mirror unit V32. In short, in this embodiment, the virtual camera V30 changes the direction of light entering the virtual shooting section V31 at the virtual mirror section V32 by oscillating the virtual mirror section V32, thereby changing the direction of light entering the virtual camera V30.

[0108] Furthermore, the virtual camera V30 also mimics the projector 30 in terms of the direction of light incidence of the virtual shooting section V31, the shape of the virtual mirror section V32, and the operation of the virtual mirror section V32. For example, the optical axis of light incident on the virtual shooting section V31 is vertical and intersects with the center or center of rotation of the surface (reflective surface) of the virtual mirror section V32. Furthermore, the orientation of the virtual mirror section V32 can be freely changed around the center point of the surface of the virtual mirror section V32 by combining panning and tilting, for example. In FIG. 3C , the movement (rotation) direction of the virtual mirror section V32 during panning is indicated by arrow M1, and the movement (rotation) direction of the virtual mirror section V32 during tilting is indicated by arrow M2. Furthermore, the viewing angle (field angle) of the virtual camera V30 is the same as the field angle of the projector 30. At this time, since the virtual camera V30 is installed at the virtual installation position P2 corresponding to the installation position of the projector 30, it becomes easier to accurately reproduce the real space A1 based on the image captured by the virtual camera V30.

[0109] Then, as shown in Fig. 4, the generation unit 18 virtually displays the first object Ob1 at display position P3 in the virtual space A2, and then uses the virtual camera V30 in the virtual space A2 to capture an image of the display position P3. That is, in the virtual space A2, an image of the first object Ob1 displayed at display position P3 captured by the virtual camera V30 at the virtual installation position P2 is generated by the virtual camera V30. As a result, the control system 10 can generate, as image content D1, an image including the first object Ob1 viewed from the virtual installation position P2, as shown in Fig. 4, using the virtual camera V30 defined by the generation unit 18.

[0110] However, the virtual camera V30 virtually captures an image of a subject present within the field of view of the virtual camera V30 and outputs the image captured within the field of view as an image (image signal). Therefore, strictly speaking, the image output from the virtual camera V30 is not an actually captured image, but a rendered image generated (rendered) through computation. In any case, the image output from the virtual camera V30 at this time is an image including the first object Ob1 as seen from the virtual installation position P2 when the first object Ob1 is displayed at the display position P3 in the virtual space A2. Therefore, the generation unit 18 generates the image content D1 by using the image virtually captured by the virtual camera V30 as the image content D1.

[0111] 8 to 12 show an example of a screen on which the display device 40 displays, as a reference image Im2r, a second image Im2 that is rendered from the first image Im1 and is input to the projector 30 as image content D1 when the generation unit 18 generates a first image Im1 that includes a first object Ob11. The user can check the image content D1 that is input to the projector 30 by visually checking the reference image Im2r on the display device 40.

[0112] 8 shows a case where a circular first object Ob11 is displayed on a virtual projection plane A22 in front of a virtual camera V30 in a virtual space A2. In this case, the shooting range of the virtual camera V30 is rectangular, and therefore the first object Ob11 included in the rendered second image Im2 is also circular.

[0113] 9 illustrates a case in which, in virtual space A2, a circular first object Ob11 is displayed on a virtual projection plane A23 that intersects with a virtual projection plane A22 in front of a virtual camera V30, and a rendering area Ra1 is set to span the virtual projection planes A22 and A23. When viewed from the front of the virtual projection plane A23, the first object Ob11 appears circular, but when viewed from the virtual camera V30, the first object Ob11 appears distorted into an ellipse, and therefore the first object Ob11 included in the rendered second image Im2 also has an elliptical distorted shape.

[0114] 10, a circular first object Ob11 is displayed on the virtual projection plane A23, similar to Fig. 9, but the first object Ob11 is displayed at a position closer to the virtual camera V30 than in Fig. 9. In this case, the first object Ob11 included in the rendered second image Im2 is displayed larger than in the example of Fig. 9 and is closer to a position directly in front of the virtual camera V30, so the shape of the first object Ob11 becomes closer to a circle.

[0115] 11 shows that in virtual space A2, a first object Ob11 is displayed on a virtual projection plane A22 in front of virtual camera V30, but because the first object Ob11 is displayed at the edge of virtual projection plane A22, the rendering area Ra1 also includes blank areas. Note that, because the parts of the second image Im2 other than the first object Ob11 are displayed in black, the circular first object Ob11 is correctly displayed in the rendered second image Im2.

[0116] In this way, the second image Im2 is formed as an image rendered in a predetermined rendering area Ra1 with the virtual installation position P2 of the virtual camera V30 as the viewpoint, and therefore, by projecting a second object based on the second image Im2 into the real space A1 using the projector 30, the second object corresponding to the first object Ob11 is correctly displayed in the real space A1.

[0117] Note that, when adjusting the size of the rendering area Ra1 when rendering the second image Im2 in the adjustment step, the projection size of the second object Ob2 projected onto the real space A1 based on the second image Im2 can be adjusted. For example, FIG. 12 illustrates a case in which the rendering area Ra1 when rendering the first object Ob11 displayed on the virtual projection plane A22 in front of the virtual camera V30 is larger than that illustrated in FIG. 9 . In this case, the size of the first object Ob11 in the rendered second image Im2 is smaller than that illustrated in FIG. 9 , so the projection size of the second object corresponding to the first object Ob11 can be reduced. Conversely, when the rendering area Ra1 is smaller than that illustrated in FIG. 9 , the size of the first object Ob11 in the rendered second image Im2 is larger than that illustrated in FIG. 9 , so the projection size of the second object corresponding to the first object Ob11 can be increased.

[0118] In this embodiment, since the image content D1 is (full-color) video data, the generation unit 18 continuously generates the image content D1 by continuously capturing images of the first object Ob1 with the virtual camera V30 during the generation process. That is, the generation unit 18 generates the image content D1 consisting of video data by capturing images of the first object Ob1 displayed at the display position P3, as if the virtual camera V30 in the virtual space A2 were capturing a video. Therefore, if the first object Ob1 moves, i.e., if the first object Ob1 is an animation that changes shape over time, the image content D1 reflects the movement of the first object Ob1. For example, if the first object Ob1, a butterfly, moves as if flapping its wings, as illustrated in FIG. 4, the image content D1 becomes video data of the butterfly flapping its wings.

[0119] The image content D1 generated in this manner is then transmitted to the projector 30, as shown in FIG. 4, so that the projector 30 can project it into the real space A1. The third image Im3 projected at this time is displayed in the real space A1 at a projection position P1, which corresponds to the display position P3 in the virtual space A2, as shown in FIG. 4. Furthermore, when the projector 30 projects the image content D1 at the projection position P1, the projected second object Ob2 substantially coincides with the first object Ob1 virtually displayed at the display position P3. Therefore, in the real space A1, the first object Ob1 virtually displayed in the virtual space A2 is projected as the third image Im3, as if it were a copy of the virtual space A2. Details of the process of actually projecting the image content D1 by the projector 30 will be described in the section "(4.4) Projection Mode."

[0120] In this embodiment, the image content D1 is an image of the first object Ob1 in the virtual space A2, excluding the background of the first object Ob1. That is, the second image Im2 virtually captured by the virtual camera V30 may include the virtual projection plane A21, which is the background, but the generation unit 18 generates the image content D1 by extracting only the first object Ob1 from the second image Im2 captured by the virtual camera V30. That is, in the image content D1 shown in FIG. 4, the periphery of the first object Ob1 is transparent (a state in which no image data is present).

[0121] As a result, the image content D1 generated by the generation unit 18 becomes image data of only the first object Ob1, excluding the background. Specifically, when the first object Ob1 is displayed at the display position P3 in the virtual space A2, the area inside the display position P3 other than the first object Ob1, i.e., the area corresponding to the background, is displayed in black, with a pixel value (brightness value) of 0. Since the projector 30 does not emit light to the black image, the result is that an image of only the first object Ob1, excluding the background, is generated as the image content D1.

[0122] In this embodiment, as described above, the projector 30 that projects the generated image content D1 is a movable projection system in which the projection position P1 in the real space A1 is variable, and the projection position P1 is movable within the real space A1. Therefore, the display position P3 corresponding to the projection position P1 is also movable within the virtual space A2, similar to the projection position P1. In other words, when the projection position P1 is moved by the projector 30 that projects the image content D1, the display position P3 is also moved when generating the image content D1. In this case, the first object Ob1 displayed at the display position P3 also moves within the virtual space A2. The generation unit 18 can continuously generate the image content D1 by tracking the first object Ob1 with the virtual camera V30 and continuously capturing images of the first object Ob1 with the virtual camera V30 during the generation process.

[0123] Here, the control system 10 executes a control information generation process in which the control information generator 19 generates control information D2. That is, in the control information generation process, control information D2 for the projector 30 is generated to move the projection position P1 in the real space A1 in synchronization with the movement of the display position P3. In other words, in the control information generation process, the control information D2 is generated in synchronization with the movement of the first object Ob1 in the virtual space A2.

[0124] In the present embodiment, the control system 10 generates, as control information D2, information used to control the virtual camera V30 when the virtual camera V30 follows the moving first object Ob1. Specifically, in the control information generation process, information used by the generation unit 18 to oscillate the virtual mirror part V32 is generated as the control information D2. In short, the parameters used to control the virtual mirror part V32 of the virtual camera V30 in the virtual space A2 are identical to the control information D2, which is the parameters used to control the mirror part 32 of the projector 30 in the real space A1.

[0125] After generating the control information D2, the control system 10 outputs the control information D2 together with the image content D1. That is, the control system 10 outputs the image content D1 and the control information D2 by recording (writing) them in the data storage unit 23 using the output unit 21. As a result, the image content D1 generated in the generation process is recorded (saved) as needed in the data storage unit 23 together with the control information D2 generated in the control information generation process. After outputting the image content D1 (and the control information D2), the control system 10 ends operation in the generation mode when, for example, the input unit 20 receives an operation signal for an end operation from the input device 50.

[0126] (4.2) Installation Information Acquisition Mode Next, the operation of the installation information acquisition system 15 according to this embodiment in the installation information acquisition mode, that is, the installation information acquisition method according to this embodiment will be described with reference to FIGS.

[0127] In the image projection system 100, the projector 30 projects the image content D1 generated by the control system 10 at a projection position P1 in the real space A1. Therefore, as a preparation, it is necessary to install the projector 30 at an installation position P4 that was assumed when the virtual image was generated. Here, the installation position P4 is a position corresponding to the virtual installation position P2 in the virtual space A2 where the virtual camera V30 is installed, and the user installs the projector 30 at this installation position P4. The user installs the projector 30 so that the reference direction of the projection direction in which the projector 30 projects an image (the installation direction of the projector 30) faces a predetermined direction. The projector 30 installed at the installation position P4 is wired to the control system 10 via the video cable 101, the control cable 102, and the communication cable 103, as described above.

[0128] Here, if the actual installation state of the projector 30 in the real space A1 deviates from the installation state assumed when the virtual image was generated, the second object Ob2 projected by the projector 30 onto the projection surface A11 may not be displayed correctly. Therefore, the installation information acquisition system 15 executes an installation information acquisition method to acquire installation information related to the installation state of the projector 30 installed in the real space A1. Then, the control system 10 uses the installation information acquired by the installation information acquisition system 15 to, for example, correct the projection direction of the projector 30, thereby enabling the second object Ob2 to be projected more accurately onto the projection surface A11. Here, the installation information related to the installation state of the projector 30 includes at least one of the installation position of the projector 30 relative to the projection position P1 in the real space A1 and the installation direction of the projector 30.

[0129] When the installation information acquisition system 15 starts operating in the installation information acquisition mode, the first acquisition unit 151 performs a first acquisition process to acquire position information regarding the positions of three first adjustment points AP1 (see FIG. 14 ) in the virtual space A2 (step S1). In the installation information acquisition mode, the installation information acquisition system 15 displays an input screen on the display device 40 for the user to input the first adjustment points AP1. The input screen displays a virtual space A2 formed corresponding to the real space A1. When the user performs an operation to specify the three first adjustment points AP1 using the input device 50, the first acquisition unit 151 acquires position information regarding the positions of the three first adjustment points AP1 in the virtual space A2 based on an operation signal received by the input unit 20 from the input device 50. The position information of the three first adjustment points AP1 is acquired, for example, as coordinate information in an XYZ Cartesian coordinate system representing the virtual space A2. In the following description, the three first adjustment points AP1 will be referred to as first adjustment points AP11, AP12, and AP13. In the example of FIG. 14 , there are four discontinuous surfaces (virtual projection surfaces A211 to A214) within the virtual space A2. Here, "two surfaces being discontinuous" can include a state in which two adjacent surfaces are non-parallel or a state in which there is a step between the two surfaces. In the illustrated example, the first adjustment point AP11 is set at the upper end of the boundary line between the adjacent virtual projection surfaces A211 and A212. The first adjustment point AP12 is set at the lower end of the boundary line between the adjacent virtual projection surfaces A212 and A213. The first adjustment point AP13 is set at the upper end of the boundary line between the adjacent virtual projection surfaces A213 and A214.

[0130] When the position information of the three first adjustment points AP1 is acquired, the projection processing unit 154 outputs image data of an index image (e.g., a circular index) from the communication unit 22 to the projector 30, and performs a projection process in which the index image is projected by the projector 30 (step S2). Note that the index image is not limited to a circular index, and its shape can be changed as appropriate.

[0131] Here, the user adjusts the orientation (pan and tilt) of the mirror unit 32 of the projector 30 to project the index image onto a second adjustment point AP2 (AP21 to AP23) in real space that corresponds to the first adjustment point AP1 (AP11 to AP13) (see FIG. 15 ). First, the user adjusts the orientation of the mirror unit 32 to project the index image onto the second adjustment point AP21. In this state, when the user uses the input device 50 to input that the index image has been aligned with the second adjustment point AP21 that corresponds to the first adjustment point AP11, the second acquisition unit 152 acquires angle information of the projection direction with respect to the reference direction from the projector 30 via the communication unit 22. For example, the second acquisition unit 152 acquires, as angle information, a pan angle PA1 and a tilt angle TL1 of the projection direction with respect to the reference direction when the index image is projected onto the second adjustment point AP21.

[0132] Next, when the user adjusts the orientation of the mirror unit 32 to project the index image onto the second adjustment point AP22 and uses the input device 50 to input that the index image is aligned with the second adjustment point AP22, the second acquisition unit 152 acquires angle information in this state. The second acquisition unit 152 acquires, as angle information, a pan angle PA2 and a tilt angle TL2 of the projection direction with respect to the reference direction in the state in which the index image is projected onto the second adjustment point AP22.

[0133] Thereafter, when the user adjusts the orientation of the mirror unit 32 to project the index image onto the second adjustment point AP23 and uses the input device 50 to input that the index image is aligned with the second adjustment point AP23, the second acquisition unit 152 acquires angle information in this state. The second acquisition unit 152 acquires, as angle information, a pan angle PA3 and a tilt angle TL3 of the projection direction with respect to the reference direction in the state in which the index image is projected onto the second adjustment point AP23.

[0134] As described above, the second acquisition unit 152 acquires angle information in a state where the position of the index image coincides with each of three or more (three in this embodiment) second adjustment points AP2 in the real space A1 (step S3). After the second acquisition process in step S3 is completed, the third acquisition unit 153 performs a third acquisition process to acquire installation information related to the installation state of the projector 30 (step S4).

[0135] In the third acquisition process, installation information of the projector 30 is acquired based on position information regarding the positions of the three first adjustment points AP1 in the virtual space A2 and angle information when projecting index images onto each of the three second adjustment points AP2.

[0136] For example, based on the angle information obtained in the second acquisition process, the third acquisition unit 153 calculates an angle θ12 formed between a projection direction DR1 in a state where the index image is projected onto the second adjustment point AP21 and a projection direction DR2 in a state where the index image is projected onto the second adjustment point AP22. Furthermore, based on the angle information obtained in the second acquisition process, the third acquisition unit 153 calculates an angle θ23 formed between a projection direction DR2 in a state where the index image is projected onto the second adjustment point AP22 and a projection direction DR3 in a state where the index image is projected onto the second adjustment point AP23. Furthermore, based on the angle information obtained in the second acquisition process, the third acquisition unit 153 calculates an angle θ31 formed between a projection direction DR3 in a state where the index image is projected onto the second adjustment point AP23 and a projection direction DR1 in a state where the index image is projected onto the second adjustment point AP21.

[0137] Once the angles θ12, θ21, and θ31 are determined, the third acquisition unit 153 generates a straight line L1 (see FIG. 16 ) of a vector (a1, a2, a3) that passes through the second adjustment point AP21 in the XYZ orthogonal coordinate system. The vector (a1, a2, a3) is, for example, the vector (0, 0, -1). Next, the third acquisition unit 153 generates a straight line L2 that passes through the second adjustment point AP22 and has an intersection angle with the straight line L1 of angle θ12. The third acquisition unit 153 also generates a straight line L3 that passes through the intersection point CP1 of the straight lines L1 and L2, and has an intersection angle with the straight line L1 of angle θ31 and an intersection angle with the straight line L2 of angle θ23. Here, two lines are obtained, one of which has an intersection angle θ31 with line L1 and another of which has an intersection angle θ23 with line L2, and the third acquisition unit 153 generates, as line L3, the line that matches the positional relationship of the three second adjustment points AP2. Here, the intersection point CP1 of the three lines L1, L2, and L3 corresponds to the installation position of projector 30.

[0138] The third acquisition unit 153 calculates the distance D31 between the line L3 and the second adjustment point AP23 by rotating the intersection point CP1 by an angle ΔΦ around the line L10 connecting the second adjustment points AP21 and AP22 while maintaining the relationship between the three lines L1, L2, and L3. Furthermore, as shown in Fig. 17 , the third acquisition unit 153 calculates the distance D31 between the line L3 and the second adjustment point AP23 by rotating the line L1 by an angle ΔΨ around the second adjustment point AP21 within a plane that includes the two lines L1 and L2 and the vector (a1, a2, a3) while maintaining the relationship between the three lines L1, L2, and L3. Then, the third acquisition unit 153 calculates the angle ΔΦ and the angle ΔΨ when the distance D31 is minimum.

[0139] After determining the angles ΔΦ and ΔΨ when the distance D31 is minimum, the third acquisition unit 153 determines the coordinates of the intersection CP1, i.e., the installation position P4 of the projector 30, based on the position coordinates of the three second adjustment points AP21, AP22, and AP23 and the angles ΔΦ and ΔΨ. Furthermore, after determining the coordinates of the installation position of the projector 30, the third acquisition unit 153 determines the installation direction of the projector 30 based on the projection direction to the three second adjustment points AP21, AP22, and AP23. The third acquisition unit 153 places the virtual camera V30 at a virtual installation position P2 corresponding to the installation position P4 in the virtual space A2. The third acquisition unit 153 sets the orientation of the mirror unit 32 to a pan angle PA1 and a tilt angle TL1 to generate a straight line L1, and sets the orientation of the mirror unit 32 to a pan angle PA2 and a tilt angle TL2 to generate a straight line L2. The third acquisition unit 153 then determines the installation orientation of the projector 30 by rotating the virtual camera V30 in a direction in which the line L1 passes through the first adjustment point AP11 and the line L2 passes through the first adjustment point AP12. In this way, the third acquisition unit 153 can acquire installation information related to the installation state of the projector 30, and stores the acquired installation information in the data storage unit 23.

[0140] In the control system 10 of this embodiment, if there is an error between the actual installation state of the projector 30 and the reference state, the correction processing unit 16 performs a correction process to correct the projection direction of the image by the projector 30 (step S5). In the correction process, based on the installation information acquired by the third acquisition unit 153, a correction value for correcting the projection direction (pan angle and tilt angle) in which the image is projected by the projector 30 is calculated and output to the data storage unit 23. When the control system 10 projects an image by the projector 30, the correction value calculated in the correction process is taken into account to control the orientation of the mirror unit 32, thereby correcting the projection direction of the image in consideration of the installation state of the projector 30. Thus, the control system 10 can cause the projector 30 to project the second object Ob2 of the correct size, shape, and orientation.

[0141] (4.3) Adjustment Mode Next, the operation in the adjustment mode of the control system 10 that performs the image adjustment method in the image projection system 100 will be described with reference to Figures 18 to 22. Note that, for convenience of explanation, in Figures 19 to 22, the first object Ob5 displayed in the first image Im1 and the second object Ob2 projected into the real space A1 are each depicted as diamond-shaped objects, but the shapes, sizes, etc. of the first object Ob5 and the second object Ob6 can be changed as appropriate.

[0142] The control system 10 starts operation in the adjustment mode, for example, when the input device 50 receives an operation by the user to start the adjustment mode.

[0143] First, the first image generation unit 11 generates a first image Im1 (see FIG. 19 ) in which a first object Ob5 is displayed at a second position P6 corresponding to a first position P5 (see FIG. 20 ) in the real space A1 (step S1: first image generation step). The first position P5 is, for example, a position in front of the projector 30, and is the position where the object is projected when light is emitted in a projection direction where the pan angle and tilt angle are both 0 degrees. Note that the first position P5 can be changed as appropriate. If there is no wall that serves as the projection surface A11 at the first position P5 or if the user wants to change the first position P5, the user can input coordinates representing the first position P5 using the input device 50, and the control system 10 sets the position corresponding to the input coordinates as the first position P5.

[0144] After the first image Im1 including the first object Ob5 is generated, the input device 50 receives a user operation instructing execution of the first projection process (step S2). The control system 10 then generates a first adjusted image Im21 (see FIG. 19 ) and starts projecting it into real space. The second image generator 12 renders the first adjusted image Im21 from the first image Im1 as a second image Im2, in which the first object Ob5 is positioned in a first direction (e.g., rightward) relative to the image center CT1 (step S3: first adjusted image generation step). The projection controller 13 outputs the image content D1 and control information D2 of the first adjusted image Im21 to the projector 30, causing the projector 30 to project the image. The projector 30 changes the projection direction to match the image center CT1 of the first adjusted image Im21, and projects a third image Im31 (see FIG. 20 ) based on the first adjusted image Im21. That is, the projection control unit 13 projects the second object Ob6 corresponding to the first object Ob5 onto the first projection position PT1 in the real space A1 using the projector 30 based on the first adjustment image Im21 (step S4: first adjustment image projection step). Here, it is assumed that the angle of view of the virtual camera V30 (rendering area) is not set correctly because the angle of view of the projector 30 is unknown. In other words, due to reasons such as the installation position of the projector 30 differing from the design conditions, the image size of the third image Im31 is larger than the size (when converted to the scale of the real space A1) of the rendering area Ra1 of the first adjustment image Im21, which is the second image Im2. Therefore, the second object Ob6 is displayed to the right of the first position P5, which corresponds to the second position P6.

[0145] Next, when the input device 50 receives a user operation instructing execution of the second projection process (step S5), the control system 10 starts a process of generating a second adjusted image Im22 (see FIG. 21 ) and projecting it into real space. The second image generation unit 12 renders the second adjusted image Im22 from the first image Im1 as the second image Im2, in which the first object Ob5 is positioned in a second direction (e.g., leftward) opposite to the first direction with respect to the image center CT1 (step S6: second adjusted image generation step). The projection control unit 13 outputs the image content D1 and control information D2 of the second adjusted image Im22 to the projector 30, causing the projector 30 to project the image. The projector 30 changes the projection direction to match the image center CT1 of the second adjusted image Im22, and projects a third image Im32 (see FIG. 22 ) based on the second adjusted image Im22. That is, the projection control unit 13 projects a second object Ob6 corresponding to the first object Ob5 onto a second projection position PT2 in the real space A1 based on the second adjusted image Im22 (step S7: second adjusted image projection step). Here, because the image size of the third image Im32 is larger than the size (when converted to the scale of the real space A1) of the rendering region Ra1 of the second adjusted image Im22, which is the second image Im2, the second object Ob6 is displayed to the left of the first position P5 corresponding to the second position P6. Therefore, the second object Ob6 moves leftward from the first projection position PT1 to the second projection position PT2.

[0146] If the second object Ob6 moves from right to left in the second adjusted image projection step (step S8: Yes), the user operates the input device 50 to input an adjustment instruction to reduce the projection size of the second object Ob6 (projected image) in the real space A1. At this time, the adjustment unit 14 performs a zoom adjustment to reduce the image size of the third image Im3 in the projector 30, or performs a process to increase the rendering area when rendering the second image Im2 (step S9: adjustment step), and then returns to step S2. If the rendering area when rendering the second image Im2 is increased, the image size of the projected image (third image Im3) of the projector 30 does not change, but the size of the object relative to the rendering area is reduced, resulting in an effect equivalent to reducing the image size, and thus reducing the projection size of the second object Ob6. In other words, in the adjustment step, if the second projection position PT2 is shifted in the second direction (leftward) relative to the first projection position PT1, at least one of a zoom adjustment to reduce the size of the third image Im3 projected by the projector 30 and a size adjustment to increase the rendering area is performed.

[0147] Furthermore, if the second object Ob6 moves from left to right in the second adjusted image projection step (step S8: No, step S10: Yes), the user operates the input device 50 to input an adjustment instruction to enlarge the projection size of the second object Ob6 (projected image) in the real space A1. At this time, the adjustment unit 14 performs a zoom adjustment to increase the image size of the third image Im3 in the projector 30, or performs a process to reduce the rendering area when rendering the second image Im2 (step S11: adjustment step), and then returns to step S2. Here, if the rendering area when rendering the second image Im2 is reduced, the image size of the projected image (third image Im3) of the projector 30 does not change, but the size of the object relative to the rendering area increases, resulting in an effect equivalent to enlarging the image size, and thus increasing the projection size of the second object Ob6. In other words, in the adjustment step, if the second projection position PT2 is shifted in the first direction (rightward) relative to the first projection position PT1, at least one of a zoom adjustment to enlarge the third image Im3 projected by the projector 30 and a size adjustment to reduce the rendering area is performed.

[0148] Furthermore, if the position of the second object Ob6 does not change in the second adjusted image projection step (step S8: No, step S10: No), the user determines that the adjustment step is complete and operates the input device 50 to input an instruction to end the adjustment step. At this time, the control system 10 ends the adjustment step based on the end instruction received by the input device 50. Note that "the position of the second object Ob6 does not change in the second adjusted image projection step" may also include a case where the change in the position of the second object Ob6 falls within a predetermined error range. If the position of the second object Ob6 does not change, the size of the rendering region Ra1 of the second image Im2 in the virtual space A2 (the size when converted to the scale of the real space A1) and the image size of the third image Im3 in the real space A1 become substantially identical. This reduces the deviation between the projection position of the image projected into the real space A1 by the projector 30 and the position in the real space A1 corresponding to the display position in the virtual space A2. In addition, the size of the rendering area Ra1 of the second image Im2 (the size when converted to the scale of the real space A1) being approximately the same as the image size of the third image Im3 can include a case where the image sizes of both are the same, and a case where the difference between the image sizes of both is within a specified error range.

[0149] As described above, the control system 10 of this embodiment repeats the processes of steps S2 to S11 until the position of the second object Ob6 no longer changes in the second adjusted image projection step.

[0150] The first adjustment image Im21 and the second adjustment image Im22 can be generated by placing a diamond-shaped object in the virtual space A2 and virtually changing the shooting direction of the virtual camera V30 according to the procedure described in the generation mode without changing the position of the diamond-shaped object.

[0151] (4.4) Projection Mode Next, the operation of the control system 10 according to this embodiment in the projection mode will be described.

[0152] In the projection mode, the control system 10 causes the projector 30 to project the image content D1 generated in the generation mode onto a projection position P1 in the real space A1.

[0153] The control system 10 reproduces the image content D1 stored in the data storage unit 23, outputs (transmits) the video signal to the projector 30, and causes the projector 30 to project the image at a projection position P1 in the real space A1. Since the image content D1 is an image of the first object Ob1 in the virtual space A2 excluding the background, the projector 30 can project only the second object Ob2.

[0154] In the projection mode, the control system 10 also transmits control information D2 to the projector 30 along with the image content D1. The control system 10 corrects the control information D2 based on the correction value calculated by the correction processing unit 16 and transmits the corrected control information D2 to the projector 30. The projector 30 controls the orientation of the mirror unit 32 based on the corrected control information D2, and can more accurately project an image at the desired projection position P1 even if the actual installation state of the projector 30 deviates from the reference state. Here, the image content D1 and the control information D2 are input to the projector 30 in a synchronized state. Therefore, in the projector 30, while projecting the image content D1, the drive unit 33 can drive the mirror unit 32 in accordance with the control information D2. This allows the projector 30 to move the projection position P1 in the real space A1 and reproduce the movement of the first object Ob1 when the image content D1 was generated in the virtual space A2.

[0155] Here, the third image Im3 projected by the projector 30 is displayed in the real space A1 at a projection position P1 that corresponds to the display position P3 in the virtual space A2. Moreover, when the projector 30 projects the image content D1 at the projection position P1, the projected second object Ob2 substantially coincides with the first object Ob1 virtually displayed at the display position P3 in the virtual space A2. Furthermore, for the image content D1 generated while moving the display position P3 in the virtual space A2, the projection position P1 can be moved in the real space A1 in the same manner.

[0156] (5) Modifications The above embodiment is merely one of various embodiments of the present disclosure. The above embodiment can be modified in various ways depending on the design, etc., as long as the object of the present disclosure can be achieved. The figures described in this disclosure are schematic diagrams, and the ratios of the size and thickness of each component in each figure do not necessarily reflect the actual dimensional ratios. Furthermore, functions similar to those of the control system 10 according to the above embodiment may be embodied in an installation information acquisition method, an image adjustment method, a computer program, or a non-transitory recording medium on which a computer program is recorded, etc.

[0157] Modifications of the above embodiment are listed below. The modifications described below can be applied in appropriate combinations.

[0158] (5.1) Modification 1 In the installation information acquisition mode of the installation information acquisition system 15 of the above embodiment, in the projection process of step S2, the projection processing unit 154 may project an index image at each of three or more initial projection positions using the projector 30. The three or more initial projection positions correspond to three or more first adjustment points AP1 in the real space A1.

[0159] Here, if the actual installation state of projector 30 deviates from the reference state, the index image projected by projector 30 at the initial projection position will be projected at a position that is shifted from the actual second adjustment point AP2. At this time, when the user moves mirror unit 32 of projector 30 via input device 50 to align the index image with second adjustment point AP2, second acquisition unit 152 acquires angle information in a state in which the position of the index image is aligned with second adjustment point AP2. In other words, in the second acquisition process, angle information is acquired based on the adjustment angle obtained by adjusting the projection direction of projector 30 to align the position of the index image projected at the initial projection position with the position of second adjustment point AP2. Note that the mirror unit 32 may be moved directly by hand without using input device 50.

[0160] In this way, in the projection process, if the projection processing unit 154 controls the projector 30 to project the index image at the initial projection position, the index image will be projected at a position close to the second adjustment point AP2 even if the actual installation state of the projector 30 deviates from the reference state. Therefore, the user need only move the mirror unit 32 a small amount in order to align the index image with the position of the second adjustment point AP2, and the effort required by the user to align the index image with the position of the second adjustment point AP2 can be reduced.

[0161] (5.2) Modification 2 In the installation information acquisition mode of the installation information acquisition system 15 of the above embodiment, if the installation position P4 of the projector 30 and the three second adjustment points AP2 are located at the vertices of a regular tetrahedron, the three second adjustment points AP21, AP22, and AP23 are also solutions for the intersection point CP1. Furthermore, if the installation position P4 of the projector 30 and the three second adjustment points AP2 are slightly off from the vertices of the regular tetrahedron, a point close to the second adjustment point AP2 may be the solution for the intersection point CP1.

[0162] In such a case, points whose distance to the second adjustment point AP2 is less than a predetermined value may be excluded from the solution of the intersection point CP1, but there is also a possibility that the projector 30 will be placed near the second adjustment point AP2.

[0163] Therefore, in the installation information acquisition system 15 of the second modification, four first adjustment points AP1 are set in the virtual space A2. Then, the installation information acquisition system 15 sets four second adjustment points AP2 (AP21 to AP24) corresponding to the four first adjustment points AP1 in the real space A1 (see FIG. 23 ), and executes the installation information acquisition method.

[0164] Specifically, when the installation information acquisition system 15 starts operating in the installation information acquisition mode, the first acquisition unit 151 performs a first acquisition process to acquire position information related to the positions of the four first adjustment points AP1 in the virtual space A2. In the installation information acquisition mode, the installation information acquisition system 15 displays an input screen on the display device 40 for the user to input the first adjustment points AP1. When the user performs an operation to specify the four first adjustment points AP1 using the input device 50, the first acquisition unit 151 acquires position information related to the positions of the four first adjustment points AP1 in the virtual space A2, based on an operation signal received by the input unit 20 from the input device 50.

[0165] When the position information of the four first adjustment points AP1 is acquired, the projection processing unit 154 outputs image data of an index image (e.g., a circular index) from the communication unit 22 to the projector 30, and performs a projection process in which the projector 30 projects the index image.

[0166] Here, the user projects the index image onto the second adjustment point AP2 by adjusting the orientation (pan and tilt) of the mirror unit 32 of the projector 30. The user projects the index image onto the second adjustment points AP21 to AP24 in order by adjusting the orientation of the mirror unit 32. Then, the second acquisition unit 152 acquires angle information of the projection direction with respect to the reference direction while the index image is projected onto each of the second adjustment points AP21 to AP24.

[0167] When the angle information is acquired by the second acquisition unit 152, the third acquisition unit 153 acquires installation information of the projector 30 based on position information regarding the positions of the four first adjustment points AP1 and angle information when projecting index images onto each of the four second adjustment points AP2.

[0168] Specifically, the third acquisition unit 153 obtains projection directions DR1 to DR4 when projecting index images onto the four second adjustment points AP21 to AP24 from the projector 30. The third acquisition unit 153 calculates the angles that each of the projection directions DR1 to DR4 forms with the other three projection directions.

[0169] The third acquisition unit 153 generates a straight line L1 that passes through the second adjustment point AP21 and a straight line L2 that passes through the second adjustment point AP22 and intersects with the straight line L1 at an angle formed by the projection directions DR1 and DR2. The third acquisition unit 153 also generates a straight line L3 that passes through the intersection point CP1 of the straight lines L1 and L2, intersects with the straight line L1 at an angle formed by the projection directions DR1 and DR3, and intersects with the straight line L2 at an angle formed by the projection directions DR2 and DR3. The third acquisition unit 153 also generates a straight line L4 that passes through the intersection point CP1 of the straight lines L1 and L2, intersects with the straight line L1 at an angle formed by the projection directions DR1 and DR4, intersects with the straight line L2 at an angle formed by the projection directions DR2 and DR4, and intersects with the straight line L3 at an angle formed by the projection directions DR3 and DR4.

[0170] The third acquisition unit 153 rotates the intersection point CP1 around the line L10 connecting the second adjustment points AP21 and AP22 while maintaining the relationship between the four lines L1 to L4, and obtains a first candidate coordinate of the intersection point CP1 when the distance between the line L3 and the second adjustment point AP23 is minimum. Furthermore, the third acquisition unit 153 rotates the intersection point CP1 around the line L10 connecting the second adjustment points AP21 and AP22 while maintaining the relationship between the four lines L1 to L4, and obtains a second candidate coordinate of the intersection point CP1 when the distance between the line L4 and the second adjustment point AP24 is minimum. The third acquisition unit 153 then obtains the point where the first candidate coordinate and the second candidate coordinate coincide as the intersection point CP1, i.e., the installation position of the projector 30.

[0171] In this way, the installation information acquisition system 15 of variant example 2 can uniquely determine the intersection point CP1, i.e., the installation position of the projector 30, by setting four second adjustment points AP2 and executing the installation information acquisition method.

[0172] In addition, if the installation information acquisition system 15 executes the installation information acquisition method and multiple candidates for the intersection CP1, i.e., the installation location of the projector 30, are obtained, the user may select the installation location of the projector 30 from the multiple candidates.

[0173] Note that the installation information acquisition system 15 may obtain installation information related to the installation state of the projector 30 by setting five or more second adjustment points AP2 and executing the installation information acquisition method, and the numbers of the first adjustment points AP1 and the second adjustment points AP2 can be changed as appropriate as long as they are three or more. As described above, in the installation information acquisition system 15 of Modification 2, the numbers of the first adjustment points AP1 and the second adjustment points AP2 are each four or more. By setting the numbers of the first adjustment points AP1 and the second adjustment points AP2 to four or more, it is possible to obtain installation information related to the installation state of the projector 30 more accurately.

[0174] (5.3) Modification 3 If there is a difference between the virtual projection surface A21 in the virtual space A2 and the projection surface A11 in the real space A1, there is a possibility that the actual installation state of the projector 30 cannot be correctly acquired if the number of first adjustment points AP1 and second adjustment points AP2 is three.

[0175] As shown in FIG. 24 , a first adjustment point AP13 is set at the boundary between virtual projection planes A213 and A214 in virtual space A2. In contrast, in real space A1, the plane corresponding to virtual projection plane A213 is wider than its size in the virtual space. In other words, a projection plane A115 is added between projection planes A113 and A114. For convenience of explanation, FIG. 24 illustrates a virtual boundary line between projection planes A113 and A115. However, since projection planes A113 and A115 are continuous and the boundary is not visible, a second adjustment point AP23A corresponding to first adjustment point AP13, which should be at the boundary between projection planes A113 and A115, is set at the boundary between projection planes A114 and A115. As a result, the virtual installation position of the virtual projector in virtual space A2 will not match the installation position of projector 30 in real space A1, which may result in projector 30 not projecting the second object Ob2 at the correct position.

[0176] Therefore, the installation information acquisition system 15 of variant example 3 performs installation information acquisition processing by setting four second adjustment points AP2 (AP21, AP22, AP23A, AP24) and executing the installation information acquisition method, as shown in Figure 25.

[0177] The installation information acquisition system 15 calculates the installation position of the projector 30 based on the position information and angle information of the four second adjustment points AP2 in the same manner as in the above embodiment. Here, the third acquisition unit 153 calculates the installation position of the projector 30 using the method described in "(5.2) Modification 2." The third acquisition unit 153 obtains the position of the intersection point CP1 by rotating the intersection point around a line connecting two points selected from the four second adjustment points AP2. However, the third acquisition unit 153 obtains the position of the intersection point CP1 while changing the two points selected from the four second adjustment points AP2, and then obtains the midpoint between them to obtain the installation position of the projector 30. This reduces the influence of differences between the virtual projection surface A21 constructed in the virtual space A2 and the projection surface A11 in the real space A1, thereby suppressing changes in the position, size, orientation, and the like of the image projected by the image projection system 100.

[0178] (5.4) Variation 4 If there is a large difference between the virtual projection surface A21 in the virtual space A2 and the projection surface A11 in the real space A1, even if the number of first adjustment points AP1 and second adjustment points AP2 is increased to four as in Variation 3, there is a possibility that the image will not be projected correctly by the projector 30.

[0179] 26 , the virtual projection plane A211 in the virtual space A2 corresponds to the projection plane A111 and the projection plane A116 in the real space A1, and the virtual projection plane A213 in the virtual space A2 corresponds to the projection plane A113 and the projection plane A115 in the real space A1. Therefore, the first adjustment point AP11 in the virtual space A2 corresponds to a second adjustment point AP21A that is different from the actual second adjustment point AP21. Furthermore, the first adjustment point AP14 in the virtual space A2 corresponds to a second adjustment point AP24A that is different from the actual second adjustment point AP24. Furthermore, the first adjustment point AP13 in the virtual space A2 corresponds to a second adjustment point AP23A that is different from the actual second adjustment point AP23.

[0180] In this way, if the position of the index image is adjusted to the second adjustment points AP21A, AP23A, and AP24A, which are different from the actual second adjustment points AP21, AP23, and AP24, respectively, relative to the first adjustment points AP11, AP13, and AP14 in the virtual space A2, and angle information is obtained, the installation information of the projector 30 will differ from the reference state.

[0181] In this case, three of the four second adjustment points AP2 are shifted from the correct position, so even if the number of second adjustment points AP2 is increased, the installation information of projector 30 cannot be obtained correctly, and therefore there is a possibility that the image cannot be projected correctly by projector 30.

[0182] Therefore, in the control system 10 of the fourth modification, the third acquisition unit 153 acquires angle information (pan angle and tilt angle) of the projection direction of the projector 30 with respect to the reference direction in a state in which an index image is projected onto each of the second adjustment points AP21A, AP22, AP23A, and AP24A. After acquiring the angle information for each of the second adjustment points AP21A, AP22, AP23A, and AP24A, the third acquisition unit 153 stores the acquired angle information in the data storage unit 23. Here, Table 1 shows an example of the control amount (pan angle and tilt angle) of the projection direction set for the first adjustment point AP1 in the virtual space A2 and the angle information (pan angle and tilt angle) acquired at the second adjustment point AP2 in the real space A1 corresponding to the first adjustment point AP1.

[0183]

[0184] When the control system 10 projects an image into the real space A1 using the projector 30, the correction processing unit 16 corrects the projection direction in which the image is projected based on the angle information. That is, the correction processing unit 16 performs a correction process to correct the projection direction in which the projector 30 projects the projected image, using the angle information acquired by the installation information acquisition method.

[0185] When an image is projected onto a position in real space A1 corresponding to first adjustment point AP1 in virtual space A2, i.e., onto one of second adjustment points AP21A, AP22, AP23A, or AP24A for which angle information has been acquired, correction processing unit 16 corrects the control amount of the projection direction set for first adjustment point AP1 using the angle information acquired for the corresponding second adjustment point AP2. For example, when an image is projected onto a position in real space A1 corresponding to first adjustment point AP11 in virtual space A2, control system 10 controls the projection direction of projector 30 based on the angle information of second adjustment point AP21A corresponding to first adjustment point AP11. Specifically, control system 10 controls the projection direction of projector 30 by setting mirror unit 32 to pan angle PA11 and tilt angle TL11.

[0186] On the other hand, when an image is projected at a projection position P1 in the real space A1 that corresponds to a display position P3 other than the first adjustment point AP1 in the virtual space A2, the correction processing unit 16 determines the projection direction of the image by the projector 30 based on angle information at multiple second adjustment points AP2 around the projection position P1. For example, when the control system 10 projects an image at a projection position P1 (see FIG. 27 ) surrounded by four second adjustment points AP21A, AP22, AP23A, and AP24A, the correction processing unit 16 determines the projection direction of the image based on the angle information at the second adjustment points AP21A, AP22, AP23A, and AP24A. Specifically, the control system 10 determines the pan angle PAx and tilt angle TLx when projecting an image at the projection position P1 based on the following equations 1 and 2. Here, the minimum pan angle at the four second adjustment points AP21A, AP22, AP23A, and AP24A is defined as PAmin, the maximum pan angle as PAmax, the minimum tilt angle as TLmin, and the maximum tilt angle as TLmax. The X coordinate of projection position P1 is defined as PX1, and the Y coordinate as TY1. The X coordinate on the CG screen of first adjustment point AP1 where the pan angle is minimum is defined as PXmin, and the X coordinate on the CG screen of first adjustment point AP1 where the pan angle is maximum is defined as PXmax. The Y coordinate on the CG screen of first adjustment point AP1 where the tilt angle is minimum is defined as TYmin, and the Y coordinate on the CG screen of first adjustment point AP1 where the tilt angle is maximum is defined as TYmax.

[0187]

[0188]

[0189] In this way, the pan angle PAx and tilt angle TLx when an image is projected at the projection position P1 are determined by linear interpolation using the pan angles and tilt angles at the multiple second adjustment points AP2 surrounding the projection position P1. Therefore, the projection direction at any projection position P1 can be set based on the angle information at the second adjustment points AP2 in the real space A1. In other words, the correction process performed by the correction processing unit 16 includes a correction process that uses the angle information determined for each of the three or more second adjustment points AP2 in the second acquisition process to determine the projection direction at any display position P3 other than the three or more second adjustment points AP2 by interpolation. As a result, even if the virtual projection surface A21 in the virtual space A2 and the projection surface A11 in the real space A1 are significantly different from each other, the projection direction when an image is projected at the projection position is determined, for example, by linear interpolation based on the angle information at the multiple second adjustment points AP2. Therefore, even if the virtual projection surface A21 in the virtual space A2 and the projection surface A11 in the real space A1 are significantly different from each other, the size, shape, and orientation of the image projected at the projection position P1 can be prevented from changing from the size, shape, and orientation at the time of generating the image content D1.

[0190] In order to determine the projection direction (pan angle and tilt angle) when an image is projected at an arbitrary projection position P1 in the real space A1 by linear interpolation as described above, it is preferable to set a plurality of second adjustment points AP21 to AP30 on each of a plurality of projection surfaces A11 as shown in Fig. 28. Since the correction processing unit 16 corrects the projection direction when an image is projected at an arbitrary projection position P1 by linear interpolation using angle information at a plurality of second adjustment points AP2 around the projection position P1, the projector 30 can project the image more accurately.

[0191] The correction method performed by the correction processing unit 16 can also be realized as a program. This program is a program for causing one or more processors to execute the correction method performed by the correction processing unit 16.

[0192] The interpolation method performed by the correction processing unit 16 is not limited to linear interpolation, and may be a non-linear function.

[0193] (5.5) Modification 5 The control system 10 of modification 5 will be described with reference to FIGS. 29 to 31.

[0194] The control system 10 of the fifth modified example differs from that of the above embodiment in the operation in the adjustment mode. The configuration of the control system 10 is the same as that of the above embodiment, so the same components are given the same reference numerals and the description thereof will be omitted.

[0195] The control system 10 starts operation in the adjustment mode, for example, when the input device 50 receives an operation by the user to start the adjustment mode.

[0196] First, the first image generation unit 11 generates a first image Im1 (see Figure 19) in which a first object Ob5 is displayed at a second position P6 corresponding to a first position P5 in the real space A1 (step S21: first image generation step).

[0197] After the first image Im1 including the first object Ob5 is generated, the input device 50 receives a user operation instructing execution of the first projection process (step S22). The control system 10 then generates a first adjusted image Im21 (see FIG. 19 ) and starts projecting it into real space. The second image generator 12 renders the first adjusted image Im21 from the first image Im1 as a second image Im2, in which the first object Ob5 is positioned in a first direction (e.g., rightward) relative to the image center CT1 (step S23: first adjusted image generation step). The projection controller 13 outputs the image content D1 and control information D2 of the first adjusted image Im21 to the projector 30, causing the projector 30 to project the second object Ob6. The projector 30 changes the projection direction to match the image center CT1 of the first adjusted image Im21 and projects a third image Im31 (see FIG. 19 ) including the second object Ob6 based on the first adjusted image Im21, thereby projecting the second object Ob6 at a first projection position PT1 in the real space A1 (step S24: first adjusted image projection step). Here, because the installation position of the projector 30 is different from the design conditions, the image size of the third image Im31 is larger than the size (when converted to the scale of the real space A1) of the rendering region Ra1 of the first adjusted image Im21, which is the second image Im2. Therefore, the second object Ob6 is displayed to the right of the first position P5 corresponding to the second position P6. The user marks the first projection position PT1 where the second object Ob6 is projected in the real space A1, for example, by attaching a marker (step S25).

[0198] Next, when the input device 50 receives a user operation instructing execution of the second projection process (step S26), the control system 10 starts a process of generating a second adjusted image Im22 (see FIG. 21 ) and projecting it into real space. The second image generation unit 12 renders the second adjusted image Im22 from the first image Im1 as a second image Im2, in which the first object Ob5 is positioned in a second direction (e.g., leftward) relative to the image center CT1 (step S27: second adjusted image generation step). The projection control unit 13 outputs the image content D1 and control information D2 of the second adjusted image Im22 to the projector 30, causing the projector 30 to project the second object Ob6. The projector 30 changes the projection direction to match the image center CT1 of the second adjusted image Im22 and projects a third image Im32 (see FIG. 31 ) including the second object Ob6 based on the second adjusted image Im22, thereby projecting the second object Ob6 at a second projection position PT2 in the real space A1 (step S28: second adjusted image projection step). Here, the image size of the third image Im32 is larger than the size (when converted to the scale of the real space A1) of the rendering region Ra1 of the second adjusted image Im22, which is the second image Im2. Therefore, the second object Ob6 is displayed to the left of the first position P5 corresponding to the second position P6. Therefore, the second object Ob6 moves leftward from the first projection position PT1 to the second projection position PT2. The user marks the second projection position PT2 where the second object Ob6 is projected in the real space A1, for example, by attaching a marker (step S29).

[0199] Here, if the second projection position PT2 is to the left (second direction) of the first projection position PT1 (step S30: Yes), the user operates the input device 50 to input an adjustment instruction to reduce the projection size of the second object Ob6 (projected image) in the real space A1. At this time, the adjustment unit 14 performs a zoom adjustment to reduce the image size of the third image Im3 in the projector 30, or performs a process to enlarge the rendering area when rendering the second image Im2 (step S31: adjustment step), and then returns to step S22.

[0200] Furthermore, if the second projection position PT2 is to the right (first direction) of the first projection position PT1 (step S30: No, step S32: Yes), the user operates the input device 50 to input an adjustment instruction to enlarge the projection size of the second object Ob6 (projected image) in the real space A1. At this time, the adjustment unit 14 performs a zoom adjustment to increase the image size of the third image Im3 in the projector 30, or performs a process to reduce the rendering area when rendering the second image Im2 (step S33: adjustment step), and then returns to step S22.

[0201] Furthermore, if the second projection position PT2 and the first projection position PT1 are identical (step S30: No, step S32: No), the user determines that the adjustment step is complete and operates the input device 50 to input an instruction to end the adjustment step. At this time, the control system 10 ends the adjustment step based on the end instruction received by the input device 50. Note that the second projection position PT2 and the first projection position PT1 being identical may also include a case where the displacement of the second projection position PT2 relative to the first projection position PT1 falls within a predetermined error range. By adjusting the image size of the third image Im3 so that the second projection position PT2 and the first projection position PT1 are identical, it is possible to reduce the deviation between the projection position of the image projected onto the real space A1 by the projector 30 and the position in the real space A1 corresponding to the display position in the virtual space A2.

[0202] As described above, the control system 10 of this modified example repeats the processes of steps S21 to S33 until the second projection position PT2 and the first projection position PT1 become the same.

[0203] In this modification, as shown in FIG. 21 , the first distance Ln1 between the image center CT1 in the first adjustment image Im21 and the first object Ob5 is equal to the second distance Ln2 between the image center CT1 in the second adjustment image Im22 and the first object Ob5. In this case, the intermediate position between the first projection position PT1 and the second projection position PT2 is the second position P6 to which the second object Ob6 is projected. Therefore, the user marks the first projection position PT1 and the second projection position PT2 in the real space A1, and then marks the intermediate position between the first projection position PT1 and the second projection position PT2 (first position P5). Then, in the adjustment step, at least one of the zoom adjustment of the projector 30 and the image size adjustment of the second image Im2 may be performed so that the second object Ob6 is projected to the intermediate position (marked position) between the first projection position PT1 and the second projection position PT2. In this case, the adjustment steps can be completed in a short time without repeating the processes of steps S22 to S33 multiple times.

[0204] (5.6) Modification 6 The control system 10 of modification 6 will be described with reference to FIGS. 32 and 33. FIG.

[0205] The control system 10 of Modification 6 differs from the above-described embodiment in that it further includes a camera 60 that captures an image of the second object Ob6 projected by the projector 30. The camera 60 has a fixed imaging direction, and image data of the image captured by the camera 60 is output to the control system 10. Note that the configuration other than the camera 60 is the same as in the above-described embodiment or Modification 5, and therefore the same components are denoted by the same reference numerals and their description will be omitted.

[0206] The operation of the control system 10 in the adjustment mode of the sixth modification will be described below.

[0207] The control system 10 starts operation in the adjustment mode, for example, when the input device 50 receives an operation by the user to start the adjustment mode.

[0208] First, the first image generation unit 11 generates a first image Im1 (see Figure 19) in which a first object Ob5 is displayed at a second position P6 corresponding to a first position P5 in the real space A1 (step S41: first image generation step).

[0209] Once the first image Im1 including the first object Ob5 is generated, the control system 10 starts a process of generating a first adjusted image Im21 (see FIG. 19 ) and projecting it into real space. The second image generation unit 12 generates the first adjusted image Im21 as the second image Im2, in which the first object Ob5 is positioned in a first direction (e.g., rightward) relative to the image center CT1 (step S42: first adjusted image generation step). The projection control unit 13 outputs the image content D1 and control information D2 of the first adjusted image Im21 to the projector 30, causing the projector 30 to project the image. The projector 30 changes the projection direction to match the image center CT1 of the first adjusted image Im21, and projects a third image Im31 (see FIG. 30 ) based on the first adjusted image Im21, thereby projecting the second object Ob6 at the first projection position PT1 in the real space A1 (step S43: first adjusted image projection step). Here, due to reasons such as the installation position of the projector 30 being different from the design conditions, the image size of the third image Im31 is larger than the size (when converted to the scale of the real space A1) of the rendering area Ra1 of the first adjusted image Im21, which is the second image Im2, and therefore the second object Ob6 is displayed to the right of the first position P5 corresponding to the second position P6.

[0210] At this time, the adjustment unit 14 performs image processing on the image captured by the camera 60 to determine the position of the second object Ob6 in the image. The adjustment unit 14 then calculates the coordinates of a first projection position PT1 in the real space A1 at which the second object Ob6 is projected, based on the position of the second object Ob6 in the image, the imaging direction of the camera 60, and the like (step S44: first coordinate acquisition step). In other words, in the first coordinate acquisition step, the first coordinates of the first projection position PT1 are acquired from an image of the second object Ob6 projected at the first projection position PT1 captured by the camera 60, which is installed in the real space A1 and has a fixed imaging direction. Here, the first coordinates of the first projection position PT1 are, for example, coordinates on the image captured by the camera 60, but may also be coordinates within the real space A1. Furthermore, in the first coordinate acquisition step, the adjustment unit 14 is only required to determine at least the X coordinate in the pan direction of the first projection position PT1 at which the second object Ob6 is projected. When determining the X coordinate in the pan direction in an image captured by the camera 60 as the first coordinate, the position of the left edge of the image captured by the camera 60 may be set as the origin of the X coordinate.

[0211] Next, the control system 10 starts a process of generating a second adjusted image Im22 (see FIG. 21 ) and projecting it into real space. The second image generation unit 12 generates, as the second image Im2, the second adjusted image Im22 in which the first object Ob5 is positioned in a second direction (e.g., leftward) relative to the image center CT1 (step S45: second adjusted image generation step). The projection control unit 13 outputs the image content D1 and control information D2 of the second adjusted image Im22 to the projector 30 and causes the projector 30 to project the image. The projector 30 changes the projection direction to match the image center CT1 of the second adjusted image Im22 and projects a third image Im32 (see FIG. 22 ) based on the second adjusted image Im22, thereby projecting the second object Ob6 at a second projection position PT2 in the real space A1 (step S46: second adjusted image projection step). Here, because the image size of the third image Im32 is larger than the size (when converted into the scale of the real space A1) of the rendering area Ra1 of the second adjusted image Im22, which is the second image Im2, the second object Ob6 is displayed to the left of the first position P5, which corresponds to the second position P6. Therefore, the second object Ob6 moves leftward from the first projection position PT1 to the second projection position PT2.

[0212] At this time, the adjustment unit 14 calculates the coordinates of the second projection position PT2 in the real space A1 onto which the second object Ob6 is projected, based on the image captured by the camera 60 (step S47: second coordinate acquisition step). In other words, in the second coordinate acquisition step, the second coordinates of the second projection position PT2 are acquired from the image captured by the camera 60 of the second object Ob6 projected onto the second projection position PT2. Here, the second coordinates of the second projection position PT2 are, for example, coordinates on the image captured by the camera 60, but may also be coordinates within the real space A1. Note that, in the second coordinate acquisition step, the adjustment unit 14 is only required to determine at least the X coordinate in the pan direction of the second projection position PT2 onto which the second object Ob6 is projected.

[0213] Thereafter, the adjustment unit 14 calculates the movement amount dX (=X2-X1) of the second object Ob6 by subtracting the X coordinate X1 of the first projection position PT1 from the X coordinate X2 of the second projection position PT2 (step S48). In other words, in the adjustment step, the projection size of the second object Ob6 is adjusted (for example, by adjusting at least one of the zoom of the projector 30 and the size of the rendering area) so as to reduce the difference between the first coordinate and the second coordinate in the directions along the first and second directions (pan directions).

[0214] Here, if the sign of the movement amount dX is negative (step S49: Yes), the adjustment unit 14 performs a zoom adjustment to reduce the image size of the third image Im3 in the projector 30, or performs a process to enlarge the rendering area of ​​the second image Im2 (step S50: adjustment step), and then returns to step S42.

[0215] Also, if the sign of the movement amount dX is positive (step S49: No, step S51: Yes), the adjustment unit 14 performs a zoom adjustment on the projector 30 to increase the image size of the third image Im3, or performs a process to reduce the rendering area of ​​the second image Im2 (step S52: adjustment step), and then returns to step S42.

[0216] Furthermore, if the value of the movement amount dX becomes zero (step S49: No, step S51: No), the control system 10 ends the adjustment step and displays the end of the adjustment step, for example, on the display device 40. Note that the value of the movement amount dX being zero may also include the case where the value of the movement amount dX falls within a predetermined error range.

[0217] As described above, the control system 10 of this modified example repeats the processes of steps S42 to S52 until the value of the movement amount dX becomes zero.

[0218] 21 , in this modification, the first distance Ln1 between the image center CT1 in the first adjustment image Im21 and the first object Ob5 is equal to the second distance Ln2 between the image center CT1 in the second adjustment image Im22 and the first object Ob5. In this case, the intermediate position between the first projection position PT1 and the second projection position PT2 is the second position P6 onto which the second object Ob6 is projected. Therefore, the adjustment unit 14 can calculate the X coordinate of the intermediate position (first position P5) between the first projection position PT1 and the second projection position PT2 from the X coordinate of the first projection position PT1 and the X coordinate of the second projection position PT2. In this case, in the adjustment step, the projection size of the second object Ob6 in the real space A1 is adjusted (at least one of the zoom adjustment of the projector 30 and the size adjustment of the rendering area) based on the X coordinate of the intermediate position so that the second object Ob6 is projected at an intermediate position between the first projection position PT1 and the second projection position PT2 in the direction along the first and second directions (pan direction). This allows the adjustment step to be completed in a short time without repeating the processes of steps S42 to S52 multiple times.

[0219] (5.7) Other Modifications The control system 10 in the present disclosure includes a computer system. The computer system is primarily composed of a processor and memory as hardware. The processor executes a program stored in the memory of the computer system to realize the functions of the installation information acquisition system 15 and the control system 10 in the present disclosure. The program may be pre-stored in the memory of the computer system, provided via a telecommunications line, or provided by being stored on a non-transitory recording medium such as a memory card, optical disk, or hard disk drive that is readable by the computer system. The processor of the computer system is composed of one or more electronic circuits including a semiconductor integrated circuit (IC) or a large-scale integrated circuit (LSI). The integrated circuits, such as ICs and LSIs, are referred to by different names depending on the degree of integration, and include integrated circuits called system LSIs, very large-scale integration (VLSI), or ultra-large-scale integration (ULSI). Furthermore, a field-programmable gate array (FPGA), which is programmed after the LSI is manufactured, or a logic device capable of reconfiguring the connections within the LSI or the circuit partitions within the LSI, can also be employed as a processor. Multiple electronic circuits may be integrated into a single chip or distributed across multiple chips. Multiple chips may be integrated into a single device or distributed across multiple devices. The computer system referred to here includes a microcontroller having one or more processors and one or more memories. Therefore, the microcontroller is also composed of one or more electronic circuits, including a semiconductor integrated circuit or a large-scale integrated circuit.

[0220] Similarly, it is not essential for the control system 10 that at least some of the functions of the control system 10 be concentrated in a single housing, and the components of the control system 10 may be distributed across multiple housings. For example, at least some of the functions of the control system 10 may be provided in separate housings. Furthermore, at least some of the functions of the control system 10 may be realized by the cloud (cloud computing), the edge (edge ​​computing), or a combination thereof.

[0221] Conversely, in the above embodiment, at least some of the functions distributed among multiple devices may be integrated into a single housing. For example, the functions distributed between the control system 10 and the projector 30 may be integrated into a single housing.

[0222] In the above embodiment, the user specifies three or more first adjustment points AP1 in the virtual space A2, but the installation information acquisition system 15 may automatically set three or more first adjustment points AP1, or the user may select three or more first adjustment points AP1 from a plurality of candidate points presented by the installation information acquisition system 15. Here, it is preferable that the installation information acquisition system 15 determines a combination of three or more first adjustment points AP1 so that the installation position of the projector 30 is uniquely determined.

[0223] In the above embodiment, the real space A1 includes a plurality of discontinuous surfaces (projection surfaces A11), and the first adjustment point AP1 and the second adjustment point AP2 may include points on at least two of the plurality of discontinuous surfaces. In other words, the first adjustment point AP1 and the second adjustment point AP2 are preferably set on projection surfaces that are discontinuous with each other. When the projection position of an image moves from one projection surface to another that is discontinuous with respect to the projection surface, image distortion or rotation is likely to occur. However, if the first adjustment point AP1 and the second adjustment point AP2 are set on projection surfaces that are discontinuous with each other, the installation state of the projector 30 can be more accurately acquired, and the projector 30 can project the image more accurately. Furthermore, the first adjustment point AP1 and the second adjustment point AP2 may be set on the outline of an object placed in the real space A1. In other words, the first adjustment point AP1 and the second adjustment point AP2 may include points on the outline of an object present in the real space A1. This has the advantage of making it easier for the user to align the index image with the second adjustment point AP2. Examples of this type of article include household electrical appliances such as a clock 200, decorative items such as paintings, and whiteboards.

[0224] In the above embodiment, the correction processing unit 16 may correct the image data of the virtual image viewed from the virtual installation position P2 by reflecting the actual installation state of the projector 30 in the installation state when generating the first data. That is, the correction processing unit 16 may execute a correction method for correcting the projected image using the installation information acquired by the installation information acquisition method. This correction method corrects the virtual installation position P2 based on the installation information. The correction method includes a correction process for placing a virtual camera V30 at the corrected virtual installation position P2 and generating a projected image based on a virtual image viewed by the virtual camera V30 from the corrected virtual installation position P2. By correcting the virtual installation position P2 to match the actual installation state of the projector 30 and correcting the projected image based on the virtual image viewed from the virtual installation position P2, the projector 30 can project a more accurate image into the real space A1. Note that this correction method can also be embodied as a program. This program is a program for causing one or more processors to execute the correction method performed by the correction processing unit 16.

[0225] In the above embodiment, the control system 10 does not include the projector 30, the display device 40, and the input device 50 as components, but at least one of the projector 30, the display device 40, and the input device 50 may be included as components of the control system 10.

[0226] In the above embodiment, the first direction and the second direction were directions along the pan direction of the projector 30 (left-right direction), but the first direction and the second direction may also be directions along the tilt direction of the projector 30 (up-down direction).

[0227] In this case, the second image generation unit 12 generates, as the second image, a first adjusted image Im21 in which the first object Ob5 is located in a first direction (e.g., upward) along the tilt direction with respect to the image center CT1. The projection control unit 13 causes the projector 30 to project the second object Ob6 to a first projection position PT1 in the real space A1 based on this first adjusted image Im21. The second image generation unit 12 also generates, as the second image, a second adjusted image Im22 in which the first object Ob5 is located in a second direction (e.g., downward) opposite to the first direction with respect to the image center CT1. The projection control unit 13 causes the projector 30 to project the second object Ob6 to a second projection position PT2 in the real space A1 based on this second adjusted image Im22. At this time, the adjustment unit 14 performs at least one of adjusting the zoom of the projector 30 and adjusting the size of the rendering area Ra1 used when rendering the second image Im2 so as to reduce the positional deviation between the first projection position PT1 and the second projection position PT2. This reduces the positional deviation between the first projection position PT1 and the second projection position PT2, and can reduce the deviation between the projection position of the image projected onto the real space A1 by the projector 30 and the position in the real space A1 that corresponds to the display position in the virtual space A2.

[0228] Furthermore, the first direction and the second direction are not limited to the pan direction or the tilt direction, but may be directions that intersect with the pan direction and the tilt direction, respectively.

[0229] Furthermore, the projector 30 is not limited to a floor-standing type, but may be attached to, for example, a ceiling or a wall, etc. Furthermore, the height of the projector 30 being adjustable is not an essential component of the installation information acquisition method and the image adjustment method, and the height of the projector 30 from the floor surface may be fixed.

[0230] Furthermore, the projector 30 is not limited to a moving mirror type projection system, and may be configured such that the projection unit 31 moves (including rotates), or the entire projector 30 moves (including rotates). In either case, the direction of light irradiation from the projector 30 (i.e., the direction of image projection) changes, and the projection position P1 can be moved.

[0231] Furthermore, the projector 30 is not limited to a movable projection system in which the projection position P1 in the real space A1 is variable, but may be a fixed projection system in which the projection position P1 is fixed. Even in this case, the installation information acquisition method, correction method, program, and installation information acquisition system 15 according to the present embodiment are useful. If the projector 30 is a fixed projection system, the virtual camera V30 may also be a fixed camera system.

[0232] Furthermore, it is not essential that the control system 10 record (write) the generated image content D1 in the data storage unit 23. For example, the control system 10 may transmit the generated image content D1 to the projector 30 in real time.

[0233] In the above embodiment, the virtual camera V30 has a virtual shooting unit V31 and a virtual mirror unit V32, and the virtual mirror unit V32 changes the direction of light incident on the virtual shooting unit V31, thereby changing the direction of light incident on the virtual camera V30. However, the virtual shooting unit V31 may be positioned and oriented in a mirror image position relative to the virtual mirror. In this case, the position and orientation of the virtual shooting unit V31 are changed in accordance with the position and orientation of the mirror image, which change depending on the projection direction. In this case, the virtual mirror unit V32 may be removed, or only the peripheral edge may be left, with the inside of the edge being a light-transmitting object.

[0234] (Summary) As described above, the installation information acquisition method of the first aspect is an installation information acquisition method for a projector (30). The projector (30) is installed at an installation position (P4) in a real space (A1), and the projection direction is changeable with respect to a reference direction. The projector (30) projects an image to be projected at a display position (P3) in a virtual space (A2) corresponding to the real space (A1), based on a virtual image as viewed from a virtual installation position (P2) in the virtual space (A2) corresponding to the installation position (P4), at a projection position (P1) in the real space (A1). The installation information acquisition method includes a first acquisition process, a projection process, a second acquisition process, and a third acquisition process. The first acquisition process acquires position information regarding the positions of three or more first adjustment points (AP1) in the virtual space (A2). The projection process causes the projector (30) to project an index image into the real space (A1). In the second acquisition process, angle information is acquired regarding the angle of the projection direction of the projector (30) relative to the reference direction when the position of the index image coincides with each of the three or more second adjustment points (AP2). In the third acquisition process, installation information regarding the installation state of the projector (30) is acquired based on the position information and angle information.

[0235] According to this aspect, since the installation information regarding the installation state of the projector (30) can be acquired in the third acquisition process, it is not necessary to install the projector (30) strictly according to the installation state assumed when generating the virtual image, thereby increasing the degree of freedom in the installation state of the projector (30).

[0236] The installation information acquisition method of the second aspect further includes a placement process in the first aspect. In the placement process, a virtual camera (V30) is placed at a virtual installation position (P2) on a display screen representing a virtual space (A2). The virtual image is an image projected at a display position (P3) as viewed from the virtual camera (V30).

[0237] According to this aspect, an image viewed from a virtual camera (V30) virtually placed at a virtual installation position (P2) can be used as a virtual image.

[0238] The installation information acquisition method of the third aspect is the first or second aspect, in which, in the projection processing, a projector (30) projects index images at three or more initial projection positions corresponding to three or more first adjustment points (AP1) in a real space (A1). In the second acquisition processing, angle information is acquired based on an adjustment angle obtained by adjusting the projection direction of the projector (30) to align the positions of the index images projected at the initial projection positions with the positions of the second adjustment points (AP2).

[0239] According to this aspect, if the index image is projected at the initial projection position, the index image is projected at a position close to the second adjustment point (AP2), which reduces the effort required for the user to adjust the projection direction of the projector (30) in order to align the index image with the position of the second adjustment point (AP2).

[0240] In the installation information acquisition method of the fourth aspect, in any one of the first to third aspects, the number of first adjustment points (AP1) and the number of second adjustment points (AP2) are four or more.

[0241] According to this aspect, the installation state of the projector (30) can be determined more accurately.

[0242] In the installation information acquisition method of the fifth aspect, in any of the first to fourth aspects, the three or more second adjustment points (AP2) include points on the contour of the item (200) existing in the real space (A1).

[0243] This embodiment has the advantage that it is easy to align the index image with the position of the second adjustment point (AP2).

[0244] In the installation information acquisition method of a sixth aspect, in any one of the first to fifth aspects, the real space (A1) has a plurality of discontinuous surfaces (A11) that are discontinuous with one another, and the three or more second adjustment points (AP2) include points on at least two of the plurality of discontinuous surfaces (A11).

[0245] According to this aspect, the installation state of the projector (30) can be determined more accurately, and the projector (30) can project an image more accurately.

[0246] In a seventh aspect of the installation information acquisition method according to any one of the first to sixth aspects, the projector (30) includes a projection unit (31), a mirror unit (32), and a drive unit (33). The projection unit (31) emits light for projecting an image into the real space (A1). The mirror unit (32) reflects the light emitted from the projection unit (31). The drive unit (33) changes the projection direction of the image by driving the mirror unit (32) to change the orientation of the mirror unit (32).

[0247] According to this aspect, the projection direction of the image can be changed by the projector (30).

[0248] The correction method of the eighth aspect is a correction method that corrects a projection image using installation information acquired by the installation information acquisition method of any one of aspects 1 to 7. The correction method includes a correction process that corrects a virtual installation position (P2) based on the installation information and generates a projection image based on a virtual image viewed from the corrected virtual installation position (P2).

[0249] According to this aspect, the image can be projected more accurately.

[0250] The correction method of the ninth aspect is a correction method for correcting a projection direction in which a projection image is projected by a projector (30) using angle information acquired by the installation information acquisition method of any one of the first to seventh aspects. The correction method includes a correction process for interpolating the projection direction at an arbitrary display position (P3) other than the three or more second adjustment points (AP2) using angle information acquired for each of the three or more second adjustment points (AP2) in the second acquisition process.

[0251] According to this aspect, the image can be projected more accurately.

[0252] A program according to a tenth aspect is a program for causing one or more processors to execute the installation information acquisition method according to any one of the first to seventh aspects.

[0253] According to this aspect, the degree of freedom in the installation state of the projector (30) can be increased.

[0254] A program according to an eleventh aspect is a program for causing one or more processors to execute the eighth or ninth correction method.

[0255] According to this aspect, the image can be projected more accurately.

[0256] A twelfth aspect of the installation information acquisition system (15) is an installation information acquisition system for a projector (30). The projector (30) is installed at an installation position (P4) in a real space (A1), and the projection direction is changeable relative to a reference direction. The projector (30) projects an image to be projected at a display position (P3) in a virtual space (A2) corresponding to the real space (A1), based on a virtual image as viewed from a virtual installation position (P2) in the virtual space (A2) corresponding to the installation position (P4), onto a projection position (P1) in the real space (A1). The installation information acquisition system includes a first acquisition unit (11), a projection processing unit (14), a second acquisition unit (12), and a third acquisition unit (13). The first acquisition unit (11) acquires position information regarding the positions of three or more first adjustment points (AP1) in the virtual space (A2). The projection processing unit (14) causes the projector (30) to project an index image into the real space (A1). The second acquisition unit acquires angle information regarding the angle of the projection direction of the projector (30) with respect to a reference direction in a state in which the position of the index image coincides with each of three or more second adjustment points (AP2). The third acquisition unit (13) acquires installation information regarding the installation state of the projector (30) based on the position information and angle information.

[0257] According to this aspect, since the installation information regarding the installation state of the projector (30) can be acquired in the third acquisition process, it is not necessary to install the projector (30) strictly according to the installation state assumed when generating the virtual image, thereby increasing the degree of freedom in the installation state of the projector (30).

[0258] Not limited to the above aspects, various configurations (including modified examples) of the installation information acquisition system (15) according to the above embodiment can be embodied as an installation information acquisition method using the installation information acquisition system (15), a (computer) program, or a non-temporary recording medium on which a program is recorded, etc.

[0259] The configurations according to the second to seventh aspects are not essential for the installation information acquisition method and may be omitted as appropriate.

[0260] The eighth and ninth aspects can be implemented independently and do not necessarily require any of the first to seventh aspects. For example, in the eighth aspect, correction processing may be performed based on angle information acquired by an appropriate method. In the ninth aspect, correction processing may be performed based on installation information acquired by an appropriate method.

[0261] An image adjustment method of a thirteenth aspect is an image adjustment method for an image projection system (100). The projector (30) is installed at an installation position (P4) in a real space (A1) and is capable of changing at least the projection direction. The first image (Im1) is an image including a first object (Ob5) displayed in a virtual space (A2) corresponding to the real space (A1). The second image (Im2) is an image obtained by rendering the first image (Im1) in a predetermined rendering region (Ra1) with a viewpoint set to a virtual installation position (P2) in the virtual space (A2) corresponding to the installation position (P4). The image projection system (100) projects, using the projector (30), a second object (Ob6) corresponding to the first object (Ob5) based on the second image (Im2) to a projection position in the real space (A1) corresponding to the display position of the first object (Ob5) in the virtual space (A2). The image adjustment method includes a first image generation step, a first adjusted image generation step, a first adjusted image projection step, a second adjusted image generation step, a second adjusted image projection step, and an adjustment step. In the first image generation step, a first image (Im1) is generated in which a first object (Ob5) is displayed at a second position (P6) in a virtual space (A2) corresponding to a first position (P5) in a real space (A1). In the first adjusted image generation step, a first adjusted image (Im21) in which the first object (Ob5) is located in a first direction relative to an image center (CT1) is rendered from the first image (Im1) as a second image (Im2). In the first adjusted image projection step, a projector (30) projects a second object (Ob6) onto a first projection position (PT1) in the real space (A1) based on the first adjusted image (Im21). In the second adjusted image generating step, a second adjusted image (Im22) is rendered from the first image (Im1) as a second image (Im2), in which the first object (Ob5) is positioned in a second direction opposite to the first direction with respect to the image center (CT1). In the second adjusted image projecting step, a projector (30) projects a second object (Ob6) to a second projection position (PT2) in the real space (A1) based on the second adjusted image (Im22).In the adjustment step, the projection size of the second object (Ob6) in the real space (A1) is adjusted so as to reduce the positional deviation between the first projection position (PT1) and the second projection position (PT2).

[0262] According to this aspect, by adjusting the first projection position (PT1) and the second projection position (PT2) to reduce the positional deviation, it is possible to reduce the difference between the image size when the display size of the first object (Ob5) in the virtual space (A2) is converted to the scale of the real space (A1) and the projection size of the second object (Ob6) projected onto the real space (A1). This makes it possible to reduce the deviation between the position in the real space (A1) corresponding to the display position of the first object (Ob5) in the virtual space (A2) and the projection position of the second object (Ob6) projected onto the real space (A1) by the projector (30).

[0263] The image adjustment method of the 14th aspect is the image adjustment method of the 13th aspect, in which, in the adjustment step, the image size of the third image (Im3) is adjusted by performing at least one of zoom adjustment of the projector (30) and size adjustment of the image size of the second image (Im2).

[0264] The image adjustment method of the 15th aspect is the image adjustment method of the 14th aspect, in which, in the adjustment step, if the second projection position (PT2) is deviated in a second direction with respect to the first projection position (PT1), at least one of a zoom adjustment to reduce the projection size of the second object (Ob6) projected by the projector (30) and a size adjustment to increase the rendering area (Ra1) is performed, and if the second projection position (PT2) is deviated in a first direction with respect to the first projection position (PT1), at least one of a zoom adjustment to increase the projection size of the second object (Ob6) projected by the projector (30) and a size adjustment to reduce the rendering area (Ra1) is performed.

[0265] In the image adjustment method of the 16th aspect, in the 13th or 14th aspect, a first distance (Ln1) between the image center (CT1) and the first object (Ob5) in the first adjusted image (Im21) and a second distance (Ln2) between the image center (CT1) and the first object (Ob5) in the second adjusted image (Im22) are equal. In the adjustment step, the projection size of the second object (Ob6) in the real space (A1) is adjusted so that the second object (Ob6) is projected at an intermediate position between the first projection position (PT1) and the second projection position (PT2).

[0266] The image adjustment method of a seventeenth aspect is the thirteenth or fourteenth aspect, further including a first coordinate acquisition step and a second coordinate acquisition step. In the first coordinate acquisition step, a first coordinate of the first projection position (PT1) is acquired from an image of a second object (Ob6) projected at the first projection position (PT1) captured by a camera (60) installed in a real space (A1) with a fixed shooting direction. In the second coordinate acquisition step, a second coordinate of the second projection position (PT2) is acquired from an image of the second object (Ob6) projected at the second projection position (PT2) captured by the camera (60). In the adjustment step, a projection size of the second object (Ob6) in the real space (A1) is adjusted so as to reduce a difference between the first coordinate and the second coordinate in directions along the first and second directions.

[0267] In the image adjustment method of the 18th aspect, in the 17th aspect, a first distance (Ln1) between the image center (CT1) and the first object (Ob5) in the first adjusted image (Im21) and a second distance (Ln2) between the image center (CT1) and the first object (Ob5) in the second adjusted image (Im22) are equal. In the adjustment step, the projection size of the second object (Ob6) in the real space (A1) is adjusted so that the second object (Ob6) is projected at an intermediate position between the first coordinate and the second coordinate in the directions along the first and second directions.

[0268] In the image adjustment method of the 19th aspect, in any one of the 13th to 18th aspects, the first direction and the second direction are directions along the pan direction or tilt direction of the projector (30).

[0269] A program according to a twentieth aspect is a program for causing a computer system to execute the image adjustment method according to any one of the thirteenth to nineteenth aspects.

[0270] According to the fourteenth to twentieth aspects, it is possible to reduce the deviation between the position in the real space (A1) corresponding to the display position of the first object (Ob5) in the virtual space (A2) and the projection position of the second object (Ob6) projected onto the real space (A1) by the projector (30).

[0271] A control system (10) of a 21st aspect is a control system (10) that controls a projector (30) of an image projection system (100). The projector (30) is installed at an installation position (P4) in a real space (A1) and is capable of changing at least the projection direction. The first image (Im1) is an image including a first object (Ob5) that is displayed in a virtual space (A2) corresponding to the real space (A1). The second image (Im2) is an image obtained by rendering the first image (Im1) in a predetermined rendering region (Ra1) with a virtual installation position (P2) in the virtual space (A2) that corresponds to the installation position (P4) as a viewpoint. The image projection system (100) projects, using the projector (30), a second object (Ob6) corresponding to the first object (Ob5) based on the second image (Im2) to a projection position in the real space (A1) that corresponds to the display position of the first object (Ob5) in the virtual space (A2). The control system (10) includes a first image generation unit (11), a second image generation unit (12), a projection control unit (13), and an adjustment unit (14). The first image generation unit (11) generates a first image (Im1) in which a first object (Ob5) is displayed at a second position (P6) in a virtual space (A2) corresponding to a first position (P5) in a real space (A1). The second image generation unit (12) renders a second image (Im2) from the first image (Im1). The projection control unit (13) causes a projector (30) to project a second object (Ob6) corresponding to the first object (Ob5) based on the second image (Im2). The second image generation unit (12) renders, from the first image (Im1) as the second image (Im2), a first adjusted image (Im21) in which the first object (Ob5) is located in a first direction relative to the image center (CT1). The second image generation unit (12) also renders, from the first image (Im1) as the second image (Im2), a second adjusted image (Im22) in which the first object (Ob5) is located in a second direction opposite to the first direction relative to the image center (CT1). The projection control unit (13) performs a first projection process in which the projector (30) projects the first object (Ob5) and the second object (Ob6) onto a first projection position (PT1) in the real space (A1) based on the first adjusted image (Im21).The projection control unit (13) performs a second projection process in which the projector (30) projects the second object (Ob6) onto a second projection position (PT2) in the real space (A1) based on the second adjusted image (Im22). The adjustment unit (14) adjusts the projection size of the second object (Ob6) in the real space (A1) so as to reduce the positional deviation between the first projection position (PT1) and the second projection position (PT2).

[0272] According to this aspect, by adjusting the first projection position (PT1) and the second projection position (PT2) to reduce the positional deviation, it is possible to reduce the difference between the image size when the display size of the first object (Ob5) in the virtual space (A2) is converted to the scale of the real space (A1) and the projection size of the second object (Ob6) projected onto the real space (A1). This makes it possible to reduce the deviation between the position in the real space (A1) corresponding to the display position of the first object (Ob5) in the virtual space (A2) and the projection position of the second object (Ob6) projected onto the real space (A1) by the projector (30).

[0273] In the control system (10) of the 22nd aspect, in the 21st aspect, the adjustment unit (14) adjusts the projection size of the second object (Ob6) in the real space (A1) by performing at least one of zoom adjustment of the projector (30) and size adjustment of the rendering area (Ra1).

[0274] According to this aspect, it is possible to reduce the deviation between the projection position of the image projected into the real space (A1) by the projector (30) and the position in the real space (A1) corresponding to the display position in the virtual space (A2).

[0275] An image projection system (100) of a 23rd aspect includes the control system (10) of the 22nd aspect and a projector (30) controlled by the control system (10).

[0276] According to the 22nd and 23rd aspects, it is possible to reduce the deviation between the position in the real space (A1) corresponding to the display position of the first object (Ob5) in the virtual space (A2) and the projection position of the second object (Ob6) projected onto the real space (A1) by the projector (30).

[0277] Not limited to the above aspects, various configurations (including modified examples) of the control system (10) and image projection system (100) according to the embodiment can be embodied as an image adjustment method performed by the control system (10), a (computer) program, or a non-transitory recording medium on which a program is recorded, etc.

[0278] The configurations according to the fourteenth to nineteenth aspects are not essential for the image adjustment method and may be omitted as appropriate. The configuration according to the twenty-second aspect is not essential for the control system (10) and may be omitted as appropriate.

[0279] 10 Control system 11 First image generation unit 12 Second image generation unit 13 Projection control unit 14 Adjustment unit 15 Installation information acquisition system 151 First acquisition unit 152 Second acquisition unit 153 Third acquisition unit 154 Projection processing unit 16 Correction processing unit 18 Generation unit 19 Control information generation unit 20 Input unit 21 Output unit 22 Communication unit 23 Data storage unit 30 Projector 31 Projection unit 32 Mirror unit 33 Drive unit 60 Camera 100 Image projection system 200 Article A1 Real space A2 Virtual space A11, A111, A113, A114, A115, A116 Projection surface A21, A22, A23, A211, A212, A213, A214 Virtual projection surface AP1, AP11, AP12, AP13, AP14 First adjustment point AP2, AP21, AP22, AP23, AP24, AP21A, AP23A, AP24A Second adjustment point CT1, CT2 Image center D1 Image content D2 Control information DR1, DR2, DR3 Projection direction Im1 First image Im2, Im2A, Im2B Second image Im21 First adjusted image Im22 Second adjusted image Im2r Reference image Im3, Im31, Im32, Im3A, Im3B Third image Ln1 First distance Ln2 Second distance Ob1, Ob5, Ob11, Ob12, Ob13 First object Ob2, Ob6, Ob22, Ob23 Second object P1 Projection position P2 Virtual installation position P3 Display position P4 Installation position P5 First position P6 Second position PT1 First projection position PT2 Second projection position Ra1 Rendering area V30 Virtual camera V31 Virtual shooting section V32 Virtual mirror section

Claims

1. An installation information acquisition method for a projector that is installed at an installation position in a real space, whose projection direction can be changed with respect to a reference direction, and projects a projection image based on a virtual image as viewed from a virtual installation position in the virtual space corresponding to the installation position onto a projection position in the real space, the method comprising: a first acquisition process of acquiring position information regarding positions of three or more first adjustment points in the virtual space; a projection process of projecting an index image in the real space with the projector; a second acquisition process of acquiring angle information regarding an angle of the projection direction of the projector with respect to the reference direction in a state where the position of the index image coincides with each of three or more second adjustment points respectively corresponding to the three or more first adjustment points in the virtual space; and a third acquisition process of acquiring installation information regarding the installation state of the projector based on the position information and the angle information.

2. The installation information acquisition method according to claim 1, further comprising an arrangement process of arranging a virtual camera at the virtual installation position on a display screen representing the virtual space, wherein the virtual image is an image as viewed from the virtual camera of the image projected onto the display position.

3. In the projection process, the projector projects the index image at three or more initial projection positions respectively corresponding to each of the three or more first adjustment points in the real space, and in the second acquisition process, the angle information is acquired based on an adjustment angle obtained by adjusting the projection direction of the projector to align the position of the index image projected at the initial projection position with the position of the second adjustment point. The installation information acquisition method according to claim 1 or 2.

4. The installation information acquisition method according to any one of claims 1 to 3, wherein the number of the first adjustment points and the number of the second adjustment points are each four or more.

5. The installation information acquisition method according to any one of claims 1 to 4, wherein the three or more second adjustment points include points on the contour of an article existing in the real space.

6. There are a plurality of discontinuous surfaces that are discontinuous from each other in the real space, and the three or more second adjustment points include points on at least two of the plurality of discontinuous surfaces. The installation information acquisition method according to any one of claims 1 to 5.

7. The projector includes: a projection unit that emits light for projecting an image onto the real space; a mirror unit that reflects the light emitted from the projection unit; and a drive unit that changes the projection direction of the image by driving the mirror unit so as to change the orientation of the mirror unit. The installation information acquisition method according to any one of claims 1 to 6.

8. A correction method for correcting the projection image using the installation information obtained by the installation information acquisition method according to any one of claims 1 to 7, including a correction process of correcting the virtual installation position based on the installation information and generating the projection image based on the virtual image when viewed from the corrected virtual installation position. The correction method.

9. A correction method for correcting the projection direction for projecting the projection image by the projector using the angle information obtained by the installation information acquisition method according to any one of claims 1 to 7, including a correction process of obtaining the projection direction at an arbitrary display position other than the three or more second adjustment points by interpolation using the angle information obtained by the second acquisition process for each of the three or more second adjustment points. The correction method.

10. A program for causing one or more processors to execute the installation information acquisition method according to any one of claims 1 to 7.

11. A program for causing one or more processors to execute the correction method according to claim 8 or 9.

12. An installation information acquisition system for a projector that is installed at an installation position in a real space, whose projection direction can be changed with respect to a reference direction, and projects a projection image based on a virtual image when viewed from a virtual installation position in a virtual space corresponding to the real space onto a projection position in the real space, including: a first acquisition unit that acquires position information regarding positions of three or more first adjustment points in the virtual space; a projection processing unit that projects an index image by the projector in the real space; a second acquisition unit that acquires angle information regarding an angle of the projection direction of the projector with respect to the reference direction in a state where the position of the index image coincides with each of three or more second adjustment points in the virtual space corresponding to the three or more first adjustment points in the virtual space; and a third acquisition unit that acquires installation information regarding the installation state of the projector based on the position information and the angle information. The installation information acquisition system.

Images