Image display method, information processing device, and program

Abstract

To facilitate checking whether an image has been deformed into a desired shape.SOLUTION: An image display method includes: acquiring a first image; acquiring a captured image obtained by imaging a projected object projected by projection light; and displaying a setting image including a first region in which the first image is displayed, a second region in which the captured image is displayed, and a third region in which a preview image is displayed, the preview image including at least a part of a deformed image obtained by deforming the first image according to an image of the projected object included in the captured image.SELECTED DRAWING: Figure 32

Description

Technical Field

[0001] The present invention relates to an image display method, an information processing apparatus, and a program.

Background Art

[0002] Techniques for transforming the shape of a displayed image into a desired shape have been developed. For example, Patent Document 1 discloses an image display system that captures an object to be projected with projected light and prompts a user to execute an operation of setting the shape of a projected image projected onto the object in accordance with the shape of the object recorded in the captured image. In this image display system, a captured image of the object to be projected with the projected light and an object (specifically, a rectangular image) for setting an area occupied by the projected image projected onto the object are displayed on a touch panel for the user to perform a setting operation. The user changes the position and shape of the object in accordance with the shape of the object recorded in the captured image. The projected image is arranged on the object in accordance with the position and shape of the object.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in a conventional image display system, since the projected image before the shape change and the projected image after the shape change are not displayed simultaneously, the user cannot directly compare the shapes of the projected images before and after the change. Therefore, it has been difficult for the user to accurately confirm whether the projected image has been appropriately transformed into a desired shape.

Means for Solving the Problems

[0005] One aspect of the image display method according to the present invention includes acquiring a first image, acquiring an image of an object onto which projected light is captured, and displaying a setting image having a first region on which the first image is displayed, a second region on which the image is captured, and a third region on which a preview image is displayed that includes at least a portion of a deformed image obtained by deforming the first image according to the image of the object onto which the light is captured.

[0006] One embodiment of the information processing apparatus according to the present invention comprises one or more processors, the one or more processors performing the following actions: acquiring a first image; acquiring an image of an object projected by projected light; and causing a display device to display a setting image having a first region where the first image is displayed, a second region where the image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the object projected that is included in the image.

[0007] One aspect of the program according to the present invention causes a processor to perform the following actions: acquire a first image; acquire an image of an object onto which projected light is projected; and cause a display device to display a setting image having a first region on which the first image is displayed, a second region on which the image is displayed, and a third region on which a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the object onto which the light is projected that is included in the image. [Brief explanation of the drawing]

[0008] [Figure 1] This is a schematic diagram illustrating how projected light images an object. [Figure 2] This is a schematic diagram illustrating how a projected image GS is projected onto an object onto which projected light is directed. [Figure 3] This is an explanatory diagram illustrating the setting image GC1a. [Figure 4] This is an explanatory diagram illustrating the setting image GC1b. [Figure 5]It is an explanatory diagram showing the setting image GC1c. [Figure 6] It is a schematic diagram showing the image GG2. [Figure 7] It is a schematic diagram showing the state where the image GG1 is divided into a plurality of small regions. [Figure 8] It is a schematic diagram showing the state where the captured image GP1 is divided into a plurality of small regions. [Figure 9] It is an explanatory diagram showing the setting image GC1d. [Figure 10] It is an explanatory diagram showing the setting image GC1e. [Figure 11] It is a schematic diagram showing the image GG3. [Figure 12] It is an explanatory diagram showing the setting image GC1f. [Figure 13] It is an explanatory diagram showing the setting image GC1g. [[ID=​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​It is a schematic diagram showing a mask image GM1. [Figure 28] It is an explanatory diagram illustrating a setting image GC1p. [Figure 29] It is a schematic diagram showing an image GG9. [Figure 30] It is a block diagram showing the configuration of a computer 1 according to the first embodiment. [Figure 31] It is a block diagram showing the configuration of a storage device 10 according to the first embodiment. [Figure 32] It is a flowchart for explaining the operation of the computer 1 according to the first embodiment. [Figure 33] It is a flowchart for explaining the image deformation processing of the computer 1 according to the first embodiment. [Figure 34] It is an explanatory diagram illustrating a setting image GC2.

Mode for Carrying Out the Invention

[0009] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. Note that the dimensions and scales of each part in the drawings may be different from the actual ones, and there are also parts schematically shown for easy understanding. Also, the scope of the present invention is not limited to these embodiments unless there is a description specifically limiting the present invention in the following description. Further, in this specification and the claims, when a numerical range is expressed as "Φ~Ψ" (both Φ and Ψ are numerical values), the range includes the numerical values of the upper limit (Ψ) and the lower limit (Φ). Also, the units of the upper limit (Ψ) and the lower limit (Φ) are the same.

[0010] 1. First Embodiment In the first embodiment, an example of a computer that causes a display device to display a setting image including an image before changing the shape, an imaging image obtained by imaging an object irradiated with light projected from a projector, and a preview image including part or all of the image whose shape has been changed based on the imaging image is illustrated, and an image display method, an information processing apparatus, and a program according to the present invention will be described.

[0011] 1.1. Computer Overview The following describes the overview of the computer 1 according to the first embodiment with reference to Figures 1 and 2. Figure 1 is a schematic diagram illustrating how the projected light image is captured on the object onto which the projected light is projected. Figure 2 is a schematic diagram illustrating how the projected image GS is projected onto the object onto which the projected light is projected. In this embodiment, the torso 3 is the object onto which the light projected from the projector 7 is projected. The projector 7 displays the projected image GS by projecting light onto the projection area RS on the wall surface W1 and onto the torso 3 positioned in front of the projection area RS.

[0012] Camera 5 acquires an image GP1 by imaging the imaging area RP on the wall surface W1 and the torso 3 positioned in front of the imaging area RP. Computer 1 displays a setting image GC1, which includes the image GP1 acquired from the camera 5 fixed in a predetermined position and an image GG1 acquired from an external server (not shown), on the touch panel 16 of Computer 1. Computer 1 also deforms the image GG1 based on user input on the setting image GC1 displayed on the touch panel 16. Then, Computer 1 displays a preview image GV, which includes at least a portion of the deformed image GG1, on the touch panel 16. If the user confirms the image GG1 and preview image GV and determines that the image GG1 has been appropriately deformed, Computer 1 outputs the deformed image GG1 to the projector 7 based on user input. The projector 7 displays a projected image GS, which includes the deformed image GG1, by projecting light based on the deformed image GG1.

[0013] In other words, the user can deform image GG1 according to the shape of the torso 3 recorded in the captured image GP1, thereby displaying the projected image GS to match the shape of the torso 3. Furthermore, the user can verify whether image GG1 has been deformed appropriately by comparing image GG1 with a preview image GV that includes the deformed image GG1.

[0014] 1.2. Methods for displaying and transforming images The following describes the image display method and the method of transforming image GG1 according to the first embodiment, with reference to Figures 3 to 27. In this embodiment, if the setting images GC1a to GC1p described later are not distinguished, they may be referred to as "setting image GC1". Also, if the preview images GV1 to 8 described later are not distinguished, they may be referred to as "preview image GV". Furthermore, if the transformed image GG1 based on the captured image GP1, including images GG2 to 8 described later, is not distinguished, the transformed image GG1 may be referred to as "transformed image". Also, if the points on image GG1, including points DG1 to 26 described later, are not distinguished, the points on image GG1 may be referred to as "point DG". Furthermore, if the points on the captured image GP1, including points DP1 to 26 described later, are not distinguished, the points on the captured image GP1 may be referred to as "point DP". Also, if the markers MG1 to 26 described later are not distinguished, they may be referred to as "marker MG". Furthermore, when markers MP1 to MP26, which will be described later, are not distinguished, markers MP1 to MP26 may be referred to as "marker MP".

[0015] Figure 3 is an explanatory diagram illustrating the setting image GC1a. Setting image GC1a is one aspect of setting image GC1.

[0016] The configuration image GC1 has regions R1, R2, and R3. The configuration image GC1 also has buttons B1, B2, and B3. Region R1 displays image GG1. Region R2 displays captured image GP1. Region R3 displays preview image GV.

[0017] The setting image GC1a is the first setting image GC1 displayed on the touch panel 16 after the computer 1 acquires image GG1 and captured image GP1. In the setting image GC1a, image GG1 is displayed in region R1 and captured image GP1 is displayed in region R2. With the setting image GC1a displayed on the touch panel 16, the user can specify one or more points DG on image GG1 by touching the touch panel 16. The computer 1 accepts the user's operation to specify one or more points DG on image GG1. The user can also specify one or more points DP on captured image GP1 by touching the touch panel 16. The computer 1 accepts the user's operation to specify one or more points DP on captured image GP1.

[0018] Figure 4 is an explanatory diagram illustrating the setting image GC1b. Setting image GC1b is one aspect of setting image GC1.

[0019] Computer 1, with the setting image GC1a displayed on the touch panel 16, receives user input specifying points DG1 to 21 on image GG1 and user input specifying points DP1 to 21 on captured image GP1, thereby changing the setting image GC1 displayed on the touch panel 16 from setting image GC1a to setting image GC1b. Setting image GC1b is configured similarly to setting image GC1a, except for the points on image GG1 where markers MG1 to 21 are displayed superimposed on image GG1 and the points on captured image GP1 where markers MP1 to 21 are displayed superimposed on image GP1. In Figure 4, point DPx corresponds to point DGx. Here, the value x represents an integer of 1 or greater. For example, point DP1 corresponds to point DG1, and point DP20 corresponds to point DG20.

[0020] In this embodiment, the operation of specifying point DG20 on image GG1 may be referred to as the "first operation." Similarly, the operation of specifying point DP20 on captured image GP1 may be referred to as the "second operation."

[0021] Markers MG1 to MG21 are displayed superimposed on image GG1, which is included in the setting image GC1b, to indicate the respective positions of points DG1 to DG21. In Figure 4, marker MGy corresponds to point DGy. Here, the value y represents an integer of 1 or greater. For example, marker MG1 corresponds to point DG1. Also, marker MG20 corresponds to point DG20. Furthermore, marker MGy includes a cross symbol indicating the position of point DGy and a number indicating the value y. The number indicating the value y included in marker MGy is displayed within a predetermined range from point DGy.

[0022] Markers MP1 to MP21 are displayed superimposed on the captured image GP1, which is included in the setting image GC1b, to indicate the respective positions of points DP1 to DP21. In Figure 4, marker MPz corresponds to point DPz. Here, the value z represents an integer of 1 or greater. For example, marker MP1 corresponds to point DP1. Also, marker MP20 corresponds to point DP20. Furthermore, marker MPz includes a cross symbol indicating the position of point DPz and a number indicating the value z. The number indicating the value z included in marker MPz is displayed within a predetermined range from point DPz.

[0023] Figure 5 is an explanatory diagram illustrating the setting image GC1c. Setting image GC1c is one aspect of setting image GC1.

[0024] Computer 1, with setting image GC1b displayed on the touch panel 16, accepts a user input by touching button B1, thereby changing the setting image GC1 displayed on the touch panel 16 from setting image GC1b to setting image GC1c. Setting image GC1c is configured similarly to setting image GC1b, except that a preview image GV1 is displayed in area R3.

[0025] Button B1 is a button used by the user to instruct computer 1 to transform image GG1. The user instructs computer 1 to transform image GG1 by touching button B1 displayed on touch panel 16. In other words, touching button B1 is an operation to instruct image GG1 to transform. Computer 1 receives the user's input by touching button B1 on the set image GC1b and transforms image GG1 into image GG2. Figure 6 is a schematic diagram showing image GG2.

[0026] Furthermore, computer 1 generates image GG2 by deforming image GG1, and then generates preview image GV1 by combining image GG2 and captured image GP1. Computer 1 then displays preview image GV1 in region R3. When combining image GG2 and captured image GP1, image GG2 may be made transparent at a predetermined transmittance, and the transparent image GG2 may be superimposed on captured image GP1 for display.

[0027] In this embodiment, the operation of touching button B1 to instruct the deformation of image GG1 is sometimes referred to as the "fifth operation."

[0028] The deformation of image GG1 will be explained in detail below with reference to Figures 7 and 8. Below, the deformation of image GG1 will be explained by illustrating the case in which image GG1 is deformed into image GG2. Figure 7 is a schematic diagram illustrating how image GG1 is divided into multiple small regions. Figure 8 is a schematic diagram illustrating how the captured image GP1 is divided into multiple small regions. Image GG1 is divided into multiple small regions by multiple triangles formed by connecting points DG1 to DG21 with multiple line segments. Here, it is preferable that the multiple line segments are connected so that they do not intersect each other. Also, the captured image GP1 is divided into multiple small regions by multiple triangles formed by connecting points DP1 to DG21 with multiple line segments. Here, in image GG1, if one point among points DG1 to 21 is connected to another point among points DG1 to 21, then in captured image GP1, one point among points DP1 to 21 corresponding to that point DG1 to 21 is connected to another point among points DP1 to 21 corresponding to that other point DG1 to 21. For example, in image GG1, if point DG17 and point DG20 are connected, then in captured image GP1, point DP17 corresponding to point DG17 is connected to point DP20 corresponding to point DG20. In other words, there is a one-to-one correspondence between each of the multiple triangles that divide image GG1 and each of the multiple triangles that divide captured image GP1. Computer 1 generates multiple transformation formulas for transforming image GG1 into image GG2 based on the correspondence between points DG1 to 21 and points DP1 to 21. Furthermore, computer 1 transforms image GG1 into image GG2 based on the transformation formula. These transformation formulas are used to match the shape of each of the multiple triangles that divide image GG1 with the shape of each of the multiple triangles that divide the captured image GP1, based on a correspondence. For example, the transformation formulas can be affine transformation formulas that convert the shape of each of the multiple triangles that divide image GG1 with the shape of each of the multiple triangles that divide the captured image GP1.

[0029] Figure 9 is an explanatory diagram illustrating the setting image GC1d. Setting image GC1d is one aspect of setting image GC1.

[0030] Computer 1, with setting image GC1c displayed on the touch panel 16, receives a user input to move marker MG20 from the position of point DG20 to the position of point DG22, thereby changing the setting image GC1 displayed on the touch panel 16 from setting image GC1c to setting image GC1d. Setting image GC1d is configured similarly to setting image GC1c, except that the position where marker MG20 is displayed corresponds to the position of point DG22.

[0031] For example, if a user checks the preview image GV and determines that image GG1 is not properly deformed, the user can re-deform image GG1 by specifying a replacement point DG on image GG1, or by specifying a replacement point DP on the captured image GP1.

[0032] To specify a replacement point DG for a point DG specified on image GG1, for example, the user moves the marker MG by dragging the marker MG corresponding to the specified point DG and dropping it at the target location. The cross symbol on the moved marker MG indicates the location of the replacement point DG. In other words, the user moves the marker MG corresponding to the specified point DG and designates the point DG indicated by the moved marker MG as the replacement point DG for the specified point DG. Computer 1 accepts the user's operation to move the marker MG and changes the point DG corresponding to the moved marker MG. Computer 1 also updates the correspondence between point DP and point DG in accordance with the change in the correspondence between marker MG and point DG.

[0033] Furthermore, when specifying a replacement point DP for a designated point DP on the captured image GP1, the user moves the marker MP by dragging the marker MP corresponding to the designated point DP and dropping it at the target location. The cross symbol on the moved marker MP indicates the location of the replacement point DP. In other words, the user moves the marker MP corresponding to the designated point DP and designates the point DP indicated by the moved marker MP as the replacement point DP for the designated point DP. Computer 1 accepts the user's operation to move the marker MP and changes the point DP corresponding to the moved marker MP. Computer 1 also updates the correspondence between point DG and point DP in accordance with the change in the correspondence between marker MP and point DP.

[0034] The user specifies point DG22 to replace point DG20 by moving marker MG20 displayed on touch panel 16. Computer 1 accepts the user's operation to move marker MG20 and changes the point DG corresponding to marker MG20 from point DG20 to point DG22. Computer 1 also changes the point DG corresponding to point DP20 from point DG20 to point DG22.

[0035] In this embodiment, the operation of moving the marker MG20 to specify point DG22 on image GG1, which will replace point DG20, is sometimes referred to as the "third operation."

[0036] Figure 10 is an explanatory diagram illustrating the setting image GC1e. Setting image GC1e is one aspect of setting image GC1.

[0037] Computer 1, with the setting image GC1d displayed on the touch panel 16, accepts an operation from the user who touches button B1, and changes the setting image GC1 displayed on the touch panel 16 from setting image GC1d to setting image GC1e. Setting image GC1e is configured the same as setting image GC1d, except that in area R3, preview image GV2 is displayed instead of preview image GV1.

[0038] Computer 1 accepts input from a user who touches button B1 on the setting image GC1d, and transforms image GG1 into image GG3. Figure 11 is a schematic diagram showing image GG3. The transformation formula for transforming image GG1 into image GG3 is generated based on the correspondence between points DG1-19, points DG21 and DG22, and points DP1-21.

[0039] Furthermore, computer 1 generates image GG3 by deforming image GG1, and then generates preview image GV2 by combining image GG3 with captured image GP1. Then, computer 1 displays preview image GV2 in region R3.

[0040] Figure 12 is an explanatory diagram illustrating the setting image GC1f. Setting image GC1f is one aspect of setting image GC1.

[0041] Computer 1, with setting image GC1c displayed on the touch panel 16, receives a user input to move marker MP20 from the position of point DP20 to the position of point DP22, thereby changing the setting image GC1 displayed on the touch panel 16 from setting image GC1c to setting image GC1f. Setting image GC1f is configured similarly to setting image GC1c, except that the position where marker MP20 is displayed corresponds to the position of point DP22.

[0042] The user specifies point DP22 to replace point DP20 by moving marker MP20 displayed on touch panel 16. Computer 1 accepts the user's operation to move marker MP20 and changes the point DP corresponding to marker MP20 from point DP20 to point DP22. Computer 1 also changes the point DP corresponding to point DG20 from point DP20 to point DP22.

[0043] In this embodiment, the operation of moving the marker MP20 to specify point DP22 on the captured image GP1, which will replace point DP20, is sometimes referred to as the "fourth operation."

[0044] Figure 13 is an explanatory diagram illustrating the setting image GC1g. Setting image GC1g is one aspect of setting image GC1.

[0045] Computer 1, with setting image GC1f displayed on the touch panel 16, accepts a user input by touching button B1, thereby changing the setting image GC1 displayed on the touch panel 16 from setting image GC1f to setting image GC1g. Setting image GC1g is configured similarly to setting image GC1f, except that in area R3, preview image GV3 is displayed instead of preview image GV1.

[0046] Computer 1 receives input from a user who touches button B1 on the setting image GC1f, and transforms image GG1 into image GG4. Figure 14 is a schematic diagram showing image GG4. The transformation formula for transforming image GG1 into image GG4 is generated based on the correspondence between points DG1 to DG21 and points DP1 to DG19, points DP21 and DP22.

[0047] Furthermore, computer 1 generates image GG4 by deforming image GG1, and then generates preview image GV3 by combining image GG4 with captured image GP1. Then, computer 1 displays preview image GV3 in region R3.

[0048] Figure 15 is an explanatory diagram illustrating the setting image GC1h. Setting image GC1h is one aspect of setting image GC1.

[0049] Computer 1, with setting image GC1c displayed on the touch panel 16, receives user input to move marker MG20 from point DG20 to point DG22, and to move marker MP20 from point DP20 to point DP22, thereby changing the setting image GC1 displayed on the touch panel 16 from setting image GC1c to setting image GC1h. Setting image GC1h is configured similarly to setting image GC1c, except that the position where marker MG20 is displayed corresponds to the position of point DG22, and the position where marker MP20 is displayed corresponds to the position of point DP22.

[0050] The user specifies point DG22 as a replacement for point DG20 by moving marker MG20 displayed on touch panel 16. Computer 1 accepts the user's operation of moving marker MG20 and changes the point DG corresponding to marker MG20 from point DG20 to point DG22. The user also specifies point DP22 as a replacement for point DP20 by moving marker MP20 displayed on touch panel 16. Computer 1 accepts the user's operation of moving marker MP20 and changes the point DP corresponding to marker MP20 from point DP20 to point DP22. Computer 1 also associates point DG22 with point DP22.

[0051] Figure 16 is an explanatory diagram illustrating the setting image GC1i. Setting image GC1i is one aspect of setting image GC1.

[0052] Computer 1, with the setting image GC1h displayed on the touch panel 16, accepts an operation from the user who touches button B1, and changes the setting image GC1 displayed on the touch panel 16 from setting image GC1h to setting image GC1i. Setting image GC1i is configured the same as setting image GC1h, except that in area R3, preview image GV4 is displayed instead of preview image GV1.

[0053] Computer 1 accepts input from a user who touches button B1 on the setting image GC1h, thereby transforming image GG1 into image GG5. Figure 17 is a schematic diagram showing image GG5. The transformation formula for transforming image GG1 into image GG5 is generated based on the correspondence between points DG1-19, points DG21 and DG22, and points DP1-19, points DP21 and DP22.

[0054] Furthermore, computer 1 generates image GG5 by deforming image GG1, and then generates preview image GV4 by combining image GG5 with captured image GP1. Then, computer 1 displays preview image GV4 in region R3.

[0055] Figure 18 is an explanatory diagram illustrating the setting image GC1j. Setting image GC1j is one aspect of setting image GC1.

[0056] Computer 1, with the setting image GC1c displayed on the touch panel 16, receives user input specifying points DG22-26 on image GG1 and points DP22-26 on captured image GP1, thereby changing the setting image GC1 displayed on the touch panel 16 from setting image GC1c to setting image GC1j. Setting image GC1j is configured similarly to setting image GC1c, except for points where markers MG22-26 are displayed superimposed on image GG1 and points where markers MP22-26 are displayed superimposed on captured image GP1. In Figure 18, point DPx corresponds to point DGx. As mentioned above, the value x represents an integer greater than or equal to 1. For example, point DP22 corresponds to point DG22, and point DP26 corresponds to point DG26.

[0057] In this embodiment, the operation of specifying point DG26 on image GG1 may be referred to as the "sixth operation." Also, the operation of specifying point DP26 on captured image GP1 may be referred to as the "seventh operation."

[0058] For example, if a user checks the preview image GV and determines that image GG1 is not properly deformed, the user can re-deform image GG1 by specifying a new point DG in addition to the points DG specified on image GG1, and by specifying a new point DP in addition to the points DP specified on the captured image GP1.

[0059] When specifying a new point DG, for example, the user specifies one or more points DG on image GG1 as a new point DG by touching the touch panel 16. Computer 1 accepts the user's operation to specify one or more points DG on image GG1. Computer 1 also displays one or more markers MG corresponding to each of the specified points DG so as to be superimposed on image GG1.

[0060] When specifying a new point DP, for example, the user touches the touch panel 16 to specify one or more points DP on the captured image GP1 as new points DP. Computer 1 accepts the user's operation to specify one or more points DP on the captured image GP1. Computer 1 also displays one or more markers MP corresponding to each of the specified points DP so as to be superimposed on the captured image GP1.

[0061] Markers MG1 to MG26 are displayed superimposed on image GG1, which is included in the setting image GC1j, to indicate the respective positions of points DG1 to DG26. In Figure 18, marker MGy corresponds to point DGy. As mentioned above, the value y represents an integer of 1 or greater. For example, marker MG22 corresponds to point DG22. Also, marker MG26 corresponds to point DG26. Furthermore, marker MGy includes a cross symbol indicating the position of point DGy and a number indicating the value y. The number indicating the value y included in marker MGy is displayed within a predetermined range from point DGy.

[0062] Markers MP1 to MP26, indicating the positions of points DP1 to DP26, are displayed superimposed on the captured image GP1, which is included in the setting image GC1j. In Figure 18, marker MPz corresponds to point DPz. As mentioned above, the value z represents an integer greater than or equal to 1. For example, marker MP22 corresponds to point DP22, and marker MP26 corresponds to point DP26. Furthermore, marker MPz includes a cross symbol indicating the position of point DPz and a number indicating the value z. The number indicating the value z included in marker MPz is displayed within a predetermined range from point DPz.

[0063] Figure 19 is an explanatory diagram illustrating the setting image GC1k. The setting image GC1k is one aspect of the setting image GC1.

[0064] Computer 1, with the setting image GC1j displayed on the touch panel 16, accepts an operation from the user who touches button B1, and changes the setting image GC1 displayed on the touch panel 16 from setting image GC1j to setting image GC1k. Setting image GC1k is configured the same as setting image GC1j, except that in area R3, preview image GV5 is displayed instead of preview image GV1.

[0065] Computer 1 receives input from a user who touches button B1 on the setting image GC1j, and transforms image GG1 into image GG6. Figure 20 is a schematic diagram of image GG6. The transformation formula for transforming image GG1 into image GG6 is generated based on the correspondence between points DG1 to DG26 and points DP1 to DG26.

[0066] Furthermore, computer 1 generates image GG6 by deforming image GG1, and then generates preview image GV5 by combining image GG6 with captured image GP1. Finally, computer 1 displays preview image GV5 in region R3.

[0067] Figure 21 is an explanatory diagram illustrating the setting image GC1l. Setting image GC1l is one aspect of setting image GC1.

[0068] Computer 1, while displaying the setting image GC1k on the touch panel 16, accepts a user operation to deselect point DG20 on image GG1 or point DP20 on captured image GP1, thereby changing the setting image GC1 displayed on the touch panel 16 from setting image GC1k to setting image GC1l. Setting image GC1l is configured similarly to setting image GC1k, except for points where marker MG20 is not displayed and points where marker MP20 is not displayed.

[0069] For example, if a user checks the preview image GV and determines that image GG1 is not properly deformed, the user can re-deform image GG1 by deselecting point DG specified on image GG1 or point DP specified on captured image GP1.

[0070] To deselect a point DG specified on image GG1, for example, the user performs an operation to erase the marker MG corresponding to the specified point DG. Computer 1 accepts the user's operation to erase the marker MG and deselects the point DG corresponding to the marker MG. The operation to erase the marker MG may be, for example, a double-tap operation on the marker MG to be erased, or a long press operation on the marker MG to be erased for a predetermined time or longer.

[0071] To deselect a point DP specified on the captured image GP1, for example, the user performs an operation to erase the marker MP corresponding to the specified point DP. Computer 1 accepts the user's operation to erase the marker MP and deselects the point DP corresponding to the marker MP. The operation to erase the marker MP may be, for example, a double-tap operation on the marker MP to be erased, or a long-press operation on the marker MP to be erased for a predetermined time or longer.

[0072] In this embodiment, it is assumed that if one of the corresponding markers MG and MP is deleted, the other marker MG and marker MP is also deleted. That is, if the designation of one of the corresponding points DG and point DP is canceled, the designation of the other point DG and point DP is also canceled.

[0073] The user performs an operation to erase marker MG20 or marker MP20 displayed on the touch panel 16. Upon receiving the user's operation to erase marker MG20 or marker MP20, computer 1 deselects points DG20 and DP20.

[0074] In this embodiment, the operation to deselect point DG20 or point DP20 may be referred to as the "eighth operation."

[0075] Figure 22 is an explanatory diagram illustrating the setting image GC1m. Setting image GC1m is one aspect of setting image GC1.

[0076] Computer 1, with the setting image GC1l displayed on the touch panel 16, accepts an operation from the user who touches button B1, and changes the setting image GC1 displayed on the touch panel 16 from setting image GC1l to setting image GC1m. Setting image GC1m is configured the same as setting image GC1l, except that in area R3, preview image GV6 is displayed instead of preview image GV5.

[0077] Computer 1 accepts input from a user who touches button B1 on the setting image GC1l, and transforms image GG1 into image GG7. Figure 23 is a schematic diagram showing image GG7. The transformation formula for transforming image GG1 into image GG7 is generated based on the correspondence between points DG1~19 and DG21~26 and points DP1~19 and DP21~26.

[0078] Furthermore, computer 1 generates image GG7 by deforming image GG1, and then generates preview image GV6 by combining image GG7 with captured image GP1. Then, computer 1 displays preview image GV6 in region R3.

[0079] Figure 24 is an explanatory diagram illustrating the setting image GC1n. The setting image GC1n is one aspect of the setting image GC1.

[0080] Computer 1, while displaying the setting image GC1k on the touch panel 16, accepts a user operation to deselect point DG26 on image GG1 or point DP26 on captured image GP1, thereby changing the setting image GC1 displayed on the touch panel 16 from setting image GC1k to setting image GC1n. Setting image GC1n is configured similarly to setting image GC1k, except for points where marker MG26 and marker MP26 are not displayed.

[0081] The user performs an operation to erase marker MG26 or marker MP26 displayed on the touch panel 16. Upon receiving the user's operation to erase marker MG26 or marker MP26, computer 1 deselects points DG26 and DP26.

[0082] In this embodiment, the operation to deselect point DG26 or point DP26 may be referred to as the "ninth operation."

[0083] Figure 25 is an explanatory diagram illustrating the setting image GC1o. Setting image GC1o is one aspect of setting image GC1.

[0084] Computer 1, with the setting image GC1n displayed on the touch panel 16, accepts an operation from the user who touches button B1, and changes the setting image GC1 displayed on the touch panel 16 from setting image GC1n to setting image GC1o. Setting image GC1o is configured the same as setting image GC1n, except that in area R3, preview image GV7 is displayed instead of preview image GV5.

[0085] Computer 1 receives input from a user who touches button B1 on the setting image GC1n, and transforms image GG1 into image GG8. Figure 26 is a schematic diagram showing image GG8. The transformation formula for transforming image GG1 into image GG8 is generated based on the correspondence between points DG1~25 and points DP1~25.

[0086] Furthermore, computer 1 generates image GG8 by deforming image GG1, and then generates preview image GV7 by combining image GG8 with captured image GP1. Then, computer 1 displays preview image GV7 in region R3.

[0087] Figure 27 is a schematic diagram showing the mask image GM1. The mask image GM1 is a black and white binary image generated based on the captured image GP1. Computer 1 can generate an image containing a portion of the deformed image by combining the deformed image, which is obtained by deforming image GG1, with the mask image GM1 using arithmetic operations such as addition, subtraction, multiplication, or division. In this embodiment, an image containing a portion of the deformed image may be referred to as a "partially deformed image." In other words, a partially deformed image is an image from which a portion of the deformed image has been extracted based on the mask image GM1. In this embodiment, it is assumed that computer 1 generates a partially deformed image along with the generation of the deformed image. It is also assumed that computer 1 generates a preview image GV containing a partially deformed image along with the generation of a preview image GV containing the deformed image.

[0088] Figure 28 is an explanatory diagram illustrating the setting image GC1p. Setting image GC1p is one aspect of setting image GC1.

[0089] Computer 1, with the setting image GC1c displayed on the touch panel 16, accepts a user input by touching button B2, thereby changing the setting image GC1 displayed on the touch panel 16 from setting image GC1c to setting image GC1p. Setting image GC1p is configured similarly to setting image GC1c, except that in area R3, preview image GV8 is displayed instead of preview image GV1.

[0090] Button B2 is a button used by the user to instruct computer 1 to switch between displaying a preview image GV containing a deformed image or a preview image GV containing a partially deformed image on the touch panel 16. The user instructs computer 1 to switch the preview image GV by touching button B2 displayed on the touch panel 16. In other words, touching button B2 is an operation to specify whether the preview image GV displayed in area R3 contains a deformed image or a partially deformed image. In response to the user's operation of touching button B2 on the setting image GC1c, computer 1 switches between displaying preview image GV1 containing the deformed image GG2 or preview image GV8 containing the partially deformed image GG9 on the touch panel 16. In other words, in response to the user's operation of touching button B2 on the setting image GC1c, computer 1 switches whether the preview image GV contains the deformed image GG2 or the partially deformed image GG9. Figure 29 is a schematic diagram showing image GG9. Image GG9 is generated by combining image GG2 and the mask image GM1.

[0091] In this embodiment, the operation of touching button B2 to specify whether the preview image GV includes a deformed image or a partially deformed image may be referred to as the "tenth operation."

[0092] If the user checks the preview image GV and determines that image GG1 has been properly deformed, the user can terminate the deformation process of image GG1 by touching button B3 displayed on the touch panel 16. Button B3 is a button used by the user to instruct computer 1 to complete the deformation process of image GG1. Computer 1 completes the deformation process of image GG1 by receiving the user's operation of touching button B3 on the set image GC1. Computer 1 also converts the deformed image so that the generated deformed image is displayed in a predetermined area on the projection area RS and torso 3. Then, computer 1 outputs the converted deformed image to projector 7. Projector 7 displays the projected image GS, which includes the deformed image, by projecting projection light corresponding to the converted deformed image onto the projection area RS and torso 3.

[0093] 1.3. Computer Configuration and Functions The configuration and functions of the computer 1 according to the first embodiment will be described below with reference to Figures 30 and 31.

[0094] Figure 30 is a block diagram showing the configuration of computer 1 according to the first embodiment. Computer 1 includes a storage device 10 for storing various information, a processing device 12 for controlling the operation of computer 1, a communication device 14 for performing communication with a camera 5, a projector 7, an external storage device or an external server, etc., and a touch panel 16 for displaying a setting image GC1 and accepting user operations. The processing device 12 has the functions of an acquisition unit 120, a display control unit 121, an output unit 122, and an image editing unit 130. The image editing unit 130 also has the functions of an input management unit 131, an image transformation unit 132, a preview generation unit 133, and a conversion formula generation unit 134. The touch panel 16 includes a display unit 160 and an input unit 161.

[0095] The storage device 10 is composed of, for example, volatile memory such as RAM and non-volatile memory such as ROM. Here, RAM is an abbreviation for Random Access Memory, and ROM is an abbreviation for Read Only Memory.

[0096] Figure 31 is a block diagram showing the configuration of the storage device 10 according to the first embodiment. The non-volatile memory of the storage device 10 stores a program 100 that defines the operation of the computer 1, image data 101 representing image GG1, captured image data 102 representing captured image GP1, mask image data 103 representing mask image GM1, setting image data 104 representing setting image GC1, deformed image data 105 representing deformed image and partially deformed image, preview image data 106 representing preview image GV, input information 107 representing the content of the operation received from the user, and conversion formula information 110 representing the conversion formula used when performing image deformation, etc. The input information 107 includes information regarding the correspondence between point DG and point DP. The conversion formula information 110 includes a deformation table 111 and an output table 112. The deformation table 111 represents a conversion formula for deforming image GG1 into a deformed image. Output table 112 represents a transformation formula for transforming the deformed image so that the deformed image contained in the projected image GS is displayed within a predetermined range on the projection area RS and the torso 3.

[0097] The output table 112 is generated based on multiple captured images obtained by the camera 5 capturing multiple monochrome pattern images displayed when the projector 7 projects light onto the projection area RS and the torso 3. Each of these multiple captured images contains a different pattern image. The number of pattern images displayed by the projector 7 is determined based on the resolution of the optical modulator, such as the liquid crystal panel, of the projector 7 (specifically, the number of digits when the resolution is expressed in binary). For example, if the projector 7 is equipped with a liquid crystal panel and the resolution of the liquid crystal panel is 120 x 90, then "120" and "90" are each represented by 7-digit binary numbers, so 7 images are needed to represent the horizontal coordinates and 7 images are needed to represent the vertical coordinates. Furthermore, in order to suppress the decrease in imaging accuracy due to the influence of ambient light such as illumination, it is preferable to use complementary patterns obtained by inverting the grayscale of the pattern images. In other words, if the resolution of the liquid crystal panel of the projector 7 is 120 x 90, the output table 112 is generated based on 28 captured images, for a total of 28 pattern images. The color scheme of the pattern images is not limited to black and white; for example, it may consist of two colors, such as red and green. It is preferable that the pattern images use two colors with high contrast.

[0098] Furthermore, the volatile memory of the storage device 10 is used by the processing unit 12 as a work area when executing the program 100.

[0099] Furthermore, part or all of the storage device 10 may be located on an external storage device or an external server. Also, part or all of the various information stored in the storage device 10 may be stored in the storage device 10 in advance, or it may be obtained from an external storage device or an external server.

[0100] The processing unit 12 is comprised of one or more CPUs. However, the processing unit 12 may also include a programmable logic device such as an FPGA instead of a CPU, or in addition to a CPU. Here, CPU is an abbreviation for Central Processing Unit, and FPGA is an abbreviation for Field-Programmable Gate Array.

[0101] The processing unit 12 functions as the acquisition unit 120, display control unit 121, output unit 122, and image editing unit 130 shown in Figure 30, by having its CPU and other components execute the program 100 and operating according to the program 100.

[0102] The acquisition unit 120 controls the communication device 14 to acquire various information from external terminals such as the camera 5, projector 7, external storage device, or external server, which are connected to the computer 1 in a communicative manner. The acquisition unit 120 also stores the acquired information in the storage device 10. In this embodiment, the acquisition unit 120 acquires image data 101, captured image data 102, and mask image data 103. The acquisition unit 120 then stores the acquired image data 101, captured image data 102, and mask image data 103 in the storage device 10.

[0103] In this embodiment, it is assumed that image data 101 and mask image data 103 are acquired from an external server (not shown), and captured image data 102 is acquired from the camera 5. The external server acquires captured image data 102 from the camera 5 and generates a mask image GM1 based on the captured image GP1 shown in the captured image data 102.

[0104] The display control unit 121 controls the display unit 160 of the touch panel 16 to display the setting image GC1. The display control unit 121 also displays a marker MG superimposed on the image GG1 displayed in area R1 of the setting image GC1, in response to user input. Furthermore, the display control unit 121 displays a marker MP superimposed on the captured image GP1 displayed in area R2 of the setting image GC1, in response to user input. The display control unit 121 also erases the displayed markers MG and MP, in response to user input. Additionally, the display control unit 121 moves the displayed markers MG and MP, in response to user input. That is, the display control unit 121 displays the markers MG and MP in a different position than their original location, in response to user input. Finally, the display control unit 121 displays a preview image GV in area R3 of the setting image GC1, in response to user input.

[0105] The image editing unit 130 performs various processes related to the transformation of image GG1. Specifically, the image editing unit 130 functions as an input management unit 131, an image transformation unit 132, a preview generation unit 133, and a transformation formula generation unit 134.

[0106] The input management unit 131 manages input information 107 that represents the content of operations received from the user. Specifically, the input management unit 131 acquires input information 107 that represents the content of operations received from the user by controlling the input unit 161 of the touch panel 16. The input management unit 131 also stores the acquired input information 107 in the storage device 10. Furthermore, when the computer 1 receives a new input operation from the user, the input management unit 131 updates the input information 107 stored in the storage device 10. The input management unit 131 also performs various judgments related to operations received from the user.

[0107] The transformation formula generation unit 134 generates a transformation table 111 based on the correspondence between one or more specified points DG and one or more specified points DP. The transformation table 111 represents multiple transformation formulas for transforming image GG1 into a transformed image. The transformation formula generation unit 134 also stores the generated transformation table 111 in the storage device 10.

[0108] The image deformation unit 132 deforms image GG1 into a deformed image based on the transformation formula shown in the deformation table 111. The image deformation unit 132 also generates a partially deformed image by combining the deformed image with the mask image GM1. In other words, the image deformation unit 132 generates deformed image data 105 representing the deformed image and the partially deformed image based on the deformation table 111, the image data 101 representing image GG1, and the mask image data 103 representing mask image GM1. The image deformation unit 132 also stores the generated deformed image data 105 in the storage device 10.

[0109] The preview generation unit 133 generates a preview image GV by combining the deformed image and the captured image GP1. The preview generation unit 133 also generates a preview image GV by combining a partially deformed image and the captured image GP1. In other words, the preview generation unit 133 generates preview image data 106 based on the deformed image data 105 representing the deformed image and the partially deformed image, and the captured image data 102 representing the captured image GP1. The preview generation unit 133 then stores the generated preview image data 106 in the storage device 10.

[0110] The output unit 122 transforms the deformed image based on the output table 112. Then, the output unit 122 controls the communication device 14 to output the transformed deformed image to the projector 7.

[0111] The communication device 14 includes, for example, an interface board having connectors and interface circuits, and has the function of receiving various information from the camera 5, projector 7, external storage device or external server, etc., and the function of transmitting various information to the camera 5, projector 7, external storage device or external server, etc. In this embodiment, it is assumed that the communication device 14 is connected to the camera 5, projector 7 and an external server (not shown) in a communicative manner to transmit and receive various information. When transmitting and receiving various information using wireless communication with the camera 5, projector 7 or external server, etc., the communication device 14 is configured to include an antenna for performing wireless communication compliant with a predetermined wireless communication standard.

[0112] The touch panel 16 is a device that integrates a display unit 160 for displaying images and an input unit 161 for receiving user input operations. The display unit 160 is composed of, for example, a liquid crystal panel or an organic EL panel, and displays the set image GC1 under the control of the display control unit 121. Here, organic EL is an abbreviation for organic electro-luminescence. The input unit 161 is composed of, for example, a transparent sheet-like contact sensor. The input unit 161 is provided so as to cover the display unit 160. The input unit 161 detects the touch position using an object in contact with the input unit 161 and the capacitance identified by the input unit 161, and outputs data indicating the detected touch position to the processing unit 12. The data indicating the touch position is stored in the storage device 10 as input information 107.

[0113] 1.4. Computer Operation The operation of the computer 1 according to the first embodiment will be described below with reference to Figures 32 and 33. Figure 32 is a flowchart illustrating the operation of the computer 1 according to the first embodiment. The series of operations shown in this flowchart are initiated, for example, when the power of the computer 1 is turned ON and the touch panel 16 receives an input operation from the user of the computer 1 to start the operation.

[0114] In step S101, the acquisition unit 120 controls the communication device 14 to acquire image data 101 representing image GG1 from an external server that is connected to the computer 1 in a communicative manner. The acquisition unit 120 also stores the acquired image data 101 in the storage device 10.

[0115] In step S102, the acquisition unit 120 controls the communication device 14 to acquire image data 102 representing the captured image GP1 from the camera 5, which is connected to the computer 1 in a communicative manner. The acquisition unit 120 also stores the acquired image data 102 in the storage device 10.

[0116] In step S103, the acquisition unit 120 controls the communication device 14 to acquire mask image data 103 representing the mask image GM1 from an external server that is connected to the computer 1 in a communicative manner. The acquisition unit 120 also stores the acquired mask image data 103 in the storage device 10.

[0117] In step S104, the display control unit 121 controls the display unit 160 of the touch panel 16 to display the setting image GC1 indicated by the setting image data 104. Specifically, the display control unit 121 controls the display unit 160 to display the setting image GC1a.

[0118] After the processing in step S104 is completed, the processing unit 12 performs image transformation processing in step S200. After the processing in step S200 is completed, the processing unit 12 proceeds to step S105.

[0119] Figure 33 is a flowchart illustrating the image transformation process of computer 1 according to the first embodiment. This flowchart shows a series of operations in the image transformation process of step S200.

[0120] In step S201, the input management unit 131 determines whether the input unit 161 has received an operation from the user to specify point DG on image GG1. In other words, the input management unit 131 determines whether input information 107 representing the content of the operation to specify point DG on image GG1 has been acquired. If input information 107 representing the content of the operation to specify point DG on image GG1 has been acquired, that is, if the answer in step S201 is YES, the input management unit 131 proceeds to step S202. If input information 107 representing the content of the operation to specify point DG on image GG1 has not been acquired, that is, if the answer in step S201 is NO, the input management unit 131 proceeds to step S204.

[0121] In step S202, the input management unit 131 stores the acquired input information 107 in the storage device 10. If the input information 107 is already stored in the storage device 10, the input management unit 131 updates the input information 107 stored in the storage device 10.

[0122] In step S203, the display control unit 121 displays the marker MG superimposed on the image GG1 based on the input information 107.

[0123] In step S204, the input management unit 131 determines whether the input unit 161 has received an operation from the user to specify point DP on the captured image GP1. In other words, the input management unit 131 determines whether input information 107 representing the content of the operation to specify point DP on the captured image GP1 has been acquired. If input information 107 representing the content of the operation to specify point DP on the captured image GP1 has been acquired, that is, if the answer in step S204 is YES, the input management unit 131 proceeds to step S205. If input information 107 representing the content of the operation to specify point DP on the captured image GP1 has not been acquired, that is, if the answer in step S204 is NO, the input management unit 131 proceeds to step S207.

[0124] In step S205, the input management unit 131 stores the acquired input information 107 in the storage device 10. If the input information 107 is already stored in the storage device 10, the input management unit 131 updates the input information 107 stored in the storage device 10.

[0125] In step S206, the display control unit 121 displays the marker MP superimposed on the captured image GP1 based on the input information 107.

[0126] In step S207, the input management unit 131 determines whether the input unit 161 has received an operation from the user to deselect point DG specified on image GG1 or point DP specified on captured image GP1. In other words, the input management unit 131 determines whether input information 107 representing the content of the operation to deselect point DG specified on image GG1 or point DP specified on captured image GP1 has been acquired. If input information 107 representing the content of the operation to deselect point DG specified on image GG1 or point DP specified on captured image GP1 has been acquired, that is, if the result in step S207 is YES, the input management unit 131 proceeds to step S208. If input information 107 representing the content of the operation to deselect point DG specified on image GG1 or point DP specified on captured image GP1 has not been acquired, that is, if the result in step S207 is NO, the input management unit 131 proceeds to step S210.

[0127] In step S208, the input management unit 131 updates the input information 107 stored in the storage device 10.

[0128] In step S209, the display control unit 121 erases the markers MG and MP displayed on the display unit 160 based on the input information 107.

[0129] In step S210, the input management unit 131 determines whether the input unit 161 has received an operation from the user to specify a point DG to replace a point DG specified on image GG1, or an operation from the user to specify a point DP to replace a point DP specified on captured image GP1. In other words, the input management unit 131 determines whether input information 107 representing the content of an operation to specify a point DG to replace a point DG specified on image GG1, or input information 107 representing the content of an operation to specify a point DP to replace a point DP specified on captured image GP1, has been acquired. If input information 107 representing the content of an operation to specify a point DG to replace a point DG specified on image GG1, or input information 107 representing the content of an operation to specify a point DP to replace a point DP specified on captured image GP1, that is, if the result in step S210 is YES, the input management unit 131 proceeds to step S211. Furthermore, if neither input information 107 representing the content of the operation to specify a point DG to replace the point DG specified on image GG1, nor input information 107 representing the content of the operation to specify a point DP to replace the point DP specified on captured image GP1, that is, if the answer in step S210 is NO, the input management unit 131 proceeds to step S213.

[0130] In step S211, the input management unit 131 updates the input information 107 stored in the storage device 10.

[0131] In step S212, the display control unit 121 moves the marker MG or marker MP displayed on the display unit 160 based on the input information 107.

[0132] In step S213, the input management unit 131 refers to the input information 107 and determines whether the results of the judgments in the most recent steps S201, S204, S207, and S210 are all NO. In other words, the input management unit 131 determines whether input information 107 representing the content of an operation to cancel the designation of point DG specified on image GG1 or point DP specified on captured image GP1 has been acquired, input information 107 representing the content of an operation to specify point DG on image GG1, input information 107 representing the content of an operation to specify point DP on captured image GP1, input information 107 representing the content of an operation to specify a point DG that replaces point DG specified on image GG1, or input information 107 representing the content of an operation to specify a point DP that replaces point DP specified on captured image GP1 has been acquired. If none of the following input information 107 is obtained: input information 107 representing the content of an operation to deselect point DG specified on image GG1 or point DP specified on captured image GP1; input information 107 representing the content of an operation to specify point DG on image GG1; input information 107 representing the content of an operation to specify point DP on captured image GP1; input information 107 representing the content of an operation to specify a point DG that replaces point DG specified on image GG1; and input information 107 representing the content of an operation to specify a point DP that replaces point DP specified on captured image GP1; that is, if the answer in step S213 is YES, the input management unit 131 proceeds to step S219. Furthermore, if input information 107 representing the content of an operation to deselect point DG specified on image GG1 or point DP specified on captured image GP1 is obtained, input information 107 representing the content of an operation to specify point DG on image GG1, input information 107 representing the content of an operation to specify point DP on captured image GP1, input information 107 representing the content of an operation to specify a point DG that replaces point DG specified on image GG1, or input information 107 representing the content of an operation to specify a point DP that replaces point DP specified on captured image GP1 is obtained, that is, if the answer in step S213 is NO, the input management unit 131 proceeds to step S214.

[0133] In step S214, the input management unit 131 determines whether the input unit 161 has received an operation from the user touching button B1. In other words, the input management unit 131 determines whether input information 107 representing the content of an operation to instruct the transformation of image GG1 has been acquired. If input information 107 representing the content of an operation to instruct the transformation of image GG1 has been acquired, that is, if the result in step S214 is YES, the input management unit 131 proceeds to step S215. If input information 107 representing the content of an operation to instruct the transformation of image GG1 has not been acquired, that is, if the result in step S214 is NO, the input management unit 131 proceeds to step S201.

[0134] In step S215, the input management unit 131 refers to the input information 107 and determines whether the number of points DG and DP match. If the number of points DG and DP match, i.e., if the result in step S215 is YES, the input management unit 131 proceeds to step S216. If the number of points DG and DP do not match, i.e., if the result in step S215 is NO, the input management unit 131 proceeds to step S201.

[0135] If the number of points DG and DP do not match, it is preferable for the display control unit 121 to display a message prompting the user to perform an action to match the number of points DG and DP.

[0136] In step S216, the conversion formula generation unit 134 generates a transformation table 111 based on the input information 107. The conversion formula generation unit 134 also stores the generated transformation table 111 in the storage device 10.

[0137] In step S217, the image deformation unit 132 generates deformation image data 105 representing the deformation image and the partially deformation image based on the deformation table 111, the image data 101 representing the image GG1, and the mask image data 103 representing the mask image GM1. The image deformation unit 132 also stores the generated deformation image data 105 in the storage device 10.

[0138] In step S218, the preview generation unit 133 generates preview image data 106 based on the deformed image data 105 representing the deformed image and the partially deformed image, and the captured image data 102 representing the captured image GP1. The preview generation unit 133 also stores the generated preview image data 106 in the storage device 10.

[0139] After the processing in step S218 is completed, the processing unit 12 terminates the image transformation process shown in the flowchart of Figure 33.

[0140] In step S219, the image editing unit 130 determines whether or not the preview image GV is displayed on the display unit 160. If the preview image GV is displayed on the display unit 160, that is, if the answer in step S219 is YES, the processing unit 12, which includes the image editing unit 130, terminates the image transformation process shown in the flowchart of Figure 33. If the preview image GV is not displayed on the display unit 160, that is, if the answer in step S219 is NO, the image editing unit 130 proceeds to step S201.

[0141] After completing the image transformation process shown in the flowchart of Figure 33, the processing unit 12 executes the process in step S105 shown in the flowchart of Figure 32.

[0142] If the preview image GV is not displayed on the display unit 160, that is, if no deformed image has been generated, the processing unit 12 repeats the image deformation process shown in Figure 33 until a deformed image is generated. If the determination in step S213 is YES and the preview image GV is displayed on the display unit 160, that is, if a deformed image has been generated at least once, the processing unit 12 terminates the image deformation process without generating a new deformed image.

[0143] In step S105, the display control unit 121 displays the preview image GV in area R3 of the setting image GC1. If the preview image GV is displayed in area R3 of the setting image GC1, the display control unit 121 updates the preview image GV by controlling the display unit 160.

[0144] In step S106, the input management unit 131 determines whether the input unit 161 has received an operation from the user touching button B2. In other words, the input management unit 131 determines whether input information 107 representing the content of the operation to instruct the switching of the preview image GV has been acquired. If input information 107 representing the content of the operation to instruct the switching of the preview image GV has been acquired, that is, if the result in step S106 is YES, the input management unit 131 proceeds to step S107. If input information 107 representing the content of the operation to instruct the switching of the preview image GV has not been acquired, that is, if the result in step S106 is NO, the input management unit 131 proceeds to step S108.

[0145] In step S107, the display control unit 121 switches the preview image GV displayed in area R3 of the setting image GC1. If the display unit 160 is displaying a preview image GV that includes a deformed image, the display control unit 121 controls the display unit 160 to display a preview image GV that includes a partially deformed image.

[0146] In step S108, the input management unit 131 determines whether the input unit 161 has received an operation from the user touching button B3. In other words, the input management unit 131 determines whether input information 107 representing the content of an operation instructing the completion of the image transformation process has been acquired. If input information 107 representing the content of an operation instructing the completion of the image transformation process has been acquired, that is, if the result in step S108 is YES, the input management unit 131 proceeds to step S109. If input information 107 representing the content of an operation instructing the completion of the image transformation process has not been acquired, that is, if the result in step S108 is NO, the input management unit 131 proceeds to step S200.

[0147] The processing unit 12 repeats the image transformation process of step S200 until the determination in step S108 is YES, that is, until it receives an operation to instruct the completion of the image transformation process.

[0148] In step S109, the output unit 122 transforms the deformed image based on the output table 112.

[0149] In step S110, the output unit 122 controls the communication device 14 to output the transformed image to the projector 7.

[0150] After the processing in step S110 is completed, the processing unit 12 completes the series of operations shown in the flowchart of Figure 32.

[0151] As described above, according to the first embodiment, the computer 1 can display a setting image GC1 on the touch panel 16, in which the image GG1 to be transformed, the captured image GP1 referenced during transformation, and the preview image GV including the transformed image GG1 are displayed side by side. In other words, the user can compare the image before transformation with the image after transformation.

[0152] Furthermore, according to the first embodiment, the computer 1 can deform the image GG1 based on the correspondence between one or more points DG specified on the image GG1 and one or more points DP specified on the captured image GP1. That is, the user can deform the shape of the image GG1 to match the captured image GP1 by specifying points DP according to the captured image GP1 and specifying points DG on the image GG1 to correspond to points DP.

[0153] Furthermore, according to the first embodiment, the computer 1 can generate a partially deformed image by extracting a part of the deformed image using the mask image GM1. In other words, the computer 1 can adjust the shape of the deformed image by removing unnecessary parts of the deformed image using the mask image GM1.

[0154] As described above, the image display method according to the first embodiment includes acquiring an image GG1, acquiring an captured image GP1 of a torso 3, and displaying a setting image GC1 having a region R1 where the image GG1 is displayed, a region R2 where the captured image GP1 is displayed, and a region R3 where a preview image GV is displayed which includes at least a part of a deformed image in which the image GG1 is deformed according to the image of the torso 3 contained in the captured image GP1.

[0155] Furthermore, the computer 1 according to the first embodiment includes one or more CPUs, and one or more CPUs perform the following actions: acquire image GG1; acquire captured image GP1 of torso 3; and display a setting image GC1 on the touch panel 16, which has a region R1 where image GG1 is displayed, a region R2 where captured image GP1 is displayed, and a region R3 where a preview image GV is displayed which includes at least a part of a deformed image in which image GG1 is deformed according to the image of torso 3 contained in captured image GP1.

[0156] Furthermore, the program 100 according to the first embodiment causes the CPU to acquire an image GG1, acquire an captured image GP1 of the torso 3, and display a setting image GC1 on the touch panel 16, which has a region R1 where the image GG1 is displayed, a region R2 where the captured image GP1 is displayed, and a region R3 where a preview image GV is displayed, which includes at least a part of the deformed image obtained by deforming the image GG1 according to the image of the torso 3 contained in the captured image GP1.

[0157] In other words, the computer 1 according to this embodiment can simultaneously display image GG1, which is the image before deformation, and a preview image GV that includes the deformed image of image GG1. This allows the user to accurately confirm whether the image has been appropriately deformed into the desired shape by comparing the images before and after deformation.

[0158] In the first embodiment, computer 1 is an example of an "information processing device," touch panel 16 is an example of a "display device," program 100 is an example of a "program," image GG1 is an example of a "first image," torso 3 is an example of an "object onto which projected light is projected," captured image GP1 is an example of an "captured image," region R1 is an example of a "first region," region R2 is an example of a "second region," region R3 is an example of a "third region," setting image GC1 is an example of a "setting image," preview image GV is an example of a "preview image," and CPU is an example of a "processor."

[0159] Furthermore, the image display method according to the first embodiment further includes receiving a first operation from the user to specify a point DG20 on image GG1 in a setting image GC1, receiving a second operation from the user to specify a point DP20 on captured image GP1, and displaying a preview image GV1 in region R3 that includes at least a part of image GG2, which is obtained by transforming image GG1 by associating point DG20 with point DP20, wherein image GG2 is a transformed image.

[0160] In other words, computer 1 can deform image GG1 based on the correspondence between points DG and DP. This allows the user to specify point DP according to the captured image GP1 and specify point DG on image GG1 to correspond to point DP, thereby deforming the shape of image GG1 to match the captured image GP1.

[0161] In the first embodiment, point DG20 is an example of a "first point," point DP20 is an example of a "second point," image GG2 is an example of a "second image," and preview image GV1 is an example of a "preview image including at least a part of the second image."

[0162] Furthermore, the image display method according to the first embodiment involves receiving at least one of the following from the user while the preview image GV1 is displayed: a third operation to specify point DG22 on image GG1 to replace point DG20, or a fourth operation to specify point DP22 on captured image GP1 to replace point DP20; and, if the third operation is received and a fifth operation to instruct the user to deform image GG1, a preview image GV2 including at least a portion of the image GG3, which is obtained by deforming image GG1 by associating point DG22 with point DP20, is displayed in region R3. Furthermore, if the fourth operation is accepted and the fifth operation is accepted by the user, a preview image GV3 including at least a portion of the transformed image GG4, obtained by associating point DG20 with point DP22, is displayed in region R3; and if the third and fourth operations are accepted and the fifth operation is accepted by the user, a preview image GV4 including at least a portion of the transformed image GG5, obtained by associating point DG22 with point DP22, is displayed in region R3, wherein images GG3, GG4, and GG5 are transformed images.

[0163] In other words, after generating a deformed image, computer 1 can accept operations to specify a replacement point DG for a point DG specified on image GG1, or to specify a replacement point DP for a point DP specified on captured image GP1. This allows the user to, for example, check the preview image GV and, if they determine that image GG1 is not properly deformed, re-deform image GG1.

[0164] In the first embodiment, point DG22 is an example of a "third point," point DP22 is an example of a "fourth point," image GG3 is an example of a "third image," image GG4 is an example of a "fourth image," and image GG5 is an example of a "fifth image." Furthermore, preview image GV2 is an example of a "preview image including at least a part of the third image," preview image GV3 is an example of a "preview image including at least a part of the fourth image," and preview image GV4 is an example of a "preview image including at least a part of the fifth image."

[0165] Furthermore, the image display method according to the first embodiment further includes receiving a sixth operation from the user to specify a point DG26 on image GG1 while the preview image GV1 is displayed, receiving a seventh operation from the user to specify a point DP26 on captured image GP1 while the preview image GV1 is displayed, and, when a fifth operation to instruct the user to deform image GG1 is received, displaying a preview image GV5 in region R3 that includes at least a part of the deformed image GG6, where image GG6 is a deformed image, by associating point DG20 with point DP20 and point DG26 with point DP26, wherein image GG6 is a deformed image.

[0166] In other words, after generating a deformed image, computer 1 can accept operations to specify a new point DG in addition to the point DG specified on image GG1, and to specify a new point DP in addition to the point DP specified on captured image GP1. This allows the user to, for example, check the preview image GV and, if they determine that image GG1 is not properly deformed, re-deform image GG1.

[0167] In the first embodiment, point DG26 is an example of the "fifth point," point DP26 is an example of the "sixth point," image GG6 is an example of the "sixth image," and preview image GV5 is an example of a "preview image including at least a part of the sixth image."

[0168] Furthermore, the image display method according to the first embodiment further includes receiving an eighth operation from the user to deselect point DG20 or point DP20, or a ninth operation to deselect point DG26 or point DP26, while the preview image GV5 is displayed; if the eighth operation is received and the fifth operation is received from the user, displaying a preview image GV6 in region R3 that includes at least a part of image GG7, which is obtained by transforming image GG1 by associating point DG26 with point DP26; and if the ninth operation is received and the fifth operation is received from the user, displaying a preview image GV7 in region R3 that includes at least a part of image GG8, which is obtained by transforming image GG1 by associating point DG20 with point DP20, wherein images GG7 and GG8 are transformed images.

[0169] In other words, after generating a deformed image, computer 1 can accept an operation to deselect a point DG specified on image GG1 or a point DP specified on captured image GP1. This allows the user to, for example, check the preview image GV and, if they determine that image GG1 is not properly deformed, re-deform image GG1.

[0170] In the first embodiment, image GG7 is an example of the "seventh image," and image GG8 is an example of the "eighth image." Also, preview image GV6 is an example of a "preview image including at least a part of the seventh image," and preview image GV7 is an example of a "preview image including at least a part of the eighth image."

[0171] Furthermore, in the image display method according to the first embodiment, the preview image GV is an image obtained by combining at least a part of the deformed image and the captured image GP1.

[0172] This allows users to verify whether the deformed image is properly deformed to match the captured image GP1 simply by viewing the preview image GV.

[0173] Furthermore, the image display method according to the first embodiment further includes receiving a tenth operation from the user to specify whether the preview image GV includes a deformed image or a partially deformed image in which a part of the deformed image is extracted based on a mask image GM1 generated from the captured image GP1, and switching whether the preview image GV includes a deformed image or a partially deformed image in response to the tenth operation.

[0174] In other words, the user can compare the deformed image with a partially deformed image. This allows the user to easily see, for example, the size of the area removed by the mask image GM1.

[0175] In the first embodiment, the mask image GM1 is an example of a "mask image".

[0176] Furthermore, the image display method according to the first embodiment further includes outputting a deformed image as an input image to a projector 7 that projects projection light corresponding to the input image onto the torso 3.

[0177] As a result, the computer 1 can project light corresponding to the deformed image onto the projector 7, thereby displaying the projected image GS, which includes the deformed image, within a predetermined area on the torso 3.

[0178] In the first embodiment, projector 7 is an example of a "projector".

[0179] 2. Variations The above embodiments can be modified in various ways. Specific examples of modifications are given below. Furthermore, two or more embodiments arbitrarily selected from the following examples can be combined as appropriate, within the bounds of mutual consistency. In the modified examples given below, for elements whose operation and function are equivalent to those in the above embodiments, the same reference numerals used in the above description will be reused, and detailed explanations of each will be omitted as appropriate.

[0180] 2.1. Variation 1 In the embodiments described above, an example was given in which the image GG1 to be deformed, the captured image GP1 referenced during the deformation, and the preview image GV including the deformed image or a partially deformed image are displayed side by side. However, the present invention is not limited to this embodiment. For example, the image GG1, the captured image GP1, the preview image GV including the deformed image, and the preview image GV including a partially deformed image may be displayed side by side.

[0181] Figure 34 is an explanatory diagram illustrating the setting image GC2. The setting image GC2 has regions R4, R5, R6, and R7. The setting image GC1 also has buttons B1 and B3. Region R4 displays image GG1. Region R5 displays captured image GP1. Region R6 displays preview image GV including a deformed image. In Figure 34, preview image GV1 is displayed as an example of a preview image GV including a deformed image. Region R7 displays preview image GV including a partially deformed image. In Figure 34, preview image GV8 is displayed as an example of a preview image GV including a partially deformed image. Here, the deformed image is image GG2. The partially deformed image is image GG9, which is generated by combining image GG2 and mask image GM1. Since the preview image GV including the deformed image and the preview image GV including the partially deformed image are displayed simultaneously, the setting image GC2 does not require button B2 for switching between preview images GV.

[0182] As described above, in the image display method according to Modification 1, the setting image GC2 further has region R7, and region R6 displays either preview image GV1 including image GG2, or preview image GV8 including image GG9 in which a part of image GG2 is extracted based on mask image GM1 generated from captured image GP1, and region R7 displays the other of preview image GV1 or preview image GV8.

[0183] In other words, the computer according to this modified example can simultaneously display image GG1, which is the image before deformation, preview image GV1, which includes image GG2, which is the image GG1 that has been deformed, and preview image GV8, which includes image GG9, in which a portion of image GG2 has been extracted based on mask image GM1. This allows the user to accurately confirm whether the image has been appropriately deformed into the desired shape by comparing the images before and after deformation. In addition, the user can, for example, check the size of the area that is removed by mask image GM1.

[0184] In Modification Example 1, setting image GC2 is an example of a "setting image," region R7 is an example of a "fourth region," region R6 is an example of a "third region," image GG2 is an example of a "deformed image," preview image GV1 is an example of a "preview image including a deformed image," captured image GP1 is an example of an "captured image," mask image GM1 is an example of a "mask image," image GG9 is an example of a "partially deformed image," and preview image GV8 is an example of a "preview image including a partially deformed image."

[0185] 2.2. Variation Example 2 In the embodiments and modifications described above, the image GG1 is modified by receiving an operation from the user by touching the button B1 on the setting image, but the present invention is not limited to this embodiment. For example, the image GG1 may be modified by receiving an operation from the user to specify a point DG to replace a point DG specified on the image GG1, or an operation to specify a point DP to replace a point DP specified on the captured image GP1. This eliminates the need for the user to touch the button B1 on the setting image.

[0186] As described above, the image display method according to Modification 2 further includes receiving a third operation from the user while the preview image GV1 is displayed, which is either a third operation to specify a point DG22 on the image GG1 that will replace point DG20, or a fourth operation to specify a fourth point on the captured image GP1 that will replace point DP20; if the third operation is received from the user, displaying a preview image GV2 in region R3 that includes at least a portion of the image GG3 obtained by transforming the image GG1 by associating point DG22 with point DP20; and if the fourth operation is received from the user, displaying a preview image GV3 in region R3 that includes at least a portion of the image GG4 obtained by transforming the image GG1 by associating point DG20 with point DP22, wherein images GG3 and GG4 are transformed images.

[0187] In other words, the computer according to this modified example can, after generating a deformed image, accept an operation to specify a point DG to replace a point DG specified on image GG1, or an operation to specify a point DP to replace a point DP specified on captured image GP1. This allows the user to, for example, check the preview image GV and, if they determine that image GG1 is not properly deformed, to re-deform image GG1. Furthermore, the user can re-deform image GG1 without performing any operation to instruct image GG1 to be deformed.

[0188] In Modification 2, image GG1 is an example of the "first image," captured image GP1 is an example of the "captured image," region R3 is an example of the "third region," preview image GV is an example of the "preview image," preview image GV1 is an example of the "preview image including at least a part of the second image," point DG20 is an example of the "first point," point DP20 is an example of the "second point," point DG22 is an example of the "third point," point DP22 is an example of the "fourth point," image GG3 is an example of the "third image," and image GG4 is an example of the "fourth image." Also, preview image GV2 is an example of the "preview image including at least a part of the third image," and preview image GV3 is an example of the "preview image including at least a part of the fourth image."

[0189] 2.3. Variation 3 In the embodiments and modifications described above, the image GG1 is modified by receiving an operation from the user by touching the button B1 on the setting image, but the present invention is not limited to this embodiment. For example, the image GG1 may be modified by receiving an operation from the user to specify a new point DG in addition to the point DG specified on the image GG1, and an operation to specify a new point DP in addition to the point DP specified on the captured image GP1. This eliminates the need for the user to touch the button B1 on the setting image.

[0190] As described above, the image display method according to Modification 3 further includes: receiving a sixth operation from the user to specify point DG26 on image GG1 while preview image GV1 is displayed; receiving a seventh operation from the user to specify point DP26 on captured image GP1 while preview image GV1 is displayed; and, if the sixth and seventh operations are received, displaying a preview image GV5 in region R3 that includes at least a part of the image GG6, which is a deformed image of image GG1, by associating point DG20 with point DP20 and point DG26 with point DP26, wherein image GG6 is a deformed image.

[0191] In other words, the computer according to this modified example can accept operations to specify a new point DG in addition to the point DG specified on image GG1 after generating a deformed image, and to specify a new point DP in addition to the point DP specified on the captured image GP1. This allows the user to, for example, check the preview image GV and, if they determine that image GG1 is not properly deformed, to re-deform image GG1. Furthermore, the user can re-deform image GG1 without performing any operations to instruct image GG1 to be deformed.

[0192] In Modification Example 3, image GG1 is an example of the "first image," captured image GP1 is an example of the "captured image," region R3 is an example of the "third region," preview image GV1 is an example of a "preview image including at least a part of the second image," point DG20 is an example of the "first point," point DP20 is an example of the "second point," point DG26 is an example of the "fifth point," point DP26 is an example of the "sixth point," image GG6 is an example of the "sixth image," and preview image GV5 is an example of a "preview image including at least a part of the sixth image."

[0193] 2.4. Variation 4 In the embodiments and modifications described above, examples were given in which the display of a preview image GV including a deformed image and a preview image GV including a partially deformed image are switched in response to user operation. However, the present invention is not limited to such embodiments. For example, only the preview image GV including the partially deformed image may be displayed. This eliminates the need for the user to touch the setting image button B2.

[0194] As described above, in the image display method according to Modification 4, the preview image GV is an image obtained by combining the captured image GP1 with a partially deformed image, which is obtained by extracting a part of the deformed image based on a mask image GM1 generated from the captured image GP1.

[0195] As a result, the computer in this modified example can display a preview image GV with the unnecessary parts of the deformed image removed in advance by the mask image GM1.

[0196] In Modification Example 4, preview image GV is an example of a "preview image," captured image GP1 is an example of a "captured image," and mask image GM1 is an example of a "mask image."

[0197] 2.5. Variation 5 In the embodiments and modifications described above, examples were given in which markers MG and MP include a cross symbol indicating the position of a point and numbers indicating the correspondence between points DG and DP. However, the present invention is not limited to such embodiments. For example, instead of a cross symbol, an arrow indicating the location of a dot may be included. Other symbols or pictures that clearly indicate the location of a dot may also be included. Furthermore, instead of numbers, letters such as "A," "B," "C," "D," or "α," "β," "γ," and "δ" may be included. Also, both numbers and letters may be included, such as "No.1" and "No.20." Numbers are not limited to Arabic numerals; Roman numerals, for example, may also be used.

[0198] Furthermore, markers MG and MP may be displayed in different colors. For example, marker MG may be displayed in blue and marker MP in red.

[0199] 2.6. Variation 6 In the embodiments and modifications described above, an example was given in which a mask image for generating a partially deformed image is obtained from an external server, but the present invention is not limited to such embodiments. The mask image for generating a partially deformed image may be generated in the information processing device according to the present invention. For example, the mask image may be generated by using image editing software or the like to select the contour of the torso 3 image included in the captured image GP1, and filling the inside of the contour with white and the outside with black.

[0200] 2.7. Example 7 In the embodiments and modifications described above, an example was given in which a deformed image is transformed based on the output table 112 and the transformed deformed image is output to the projector 7. However, the present invention is not limited to such embodiments. Instead of a deformed image, a partially deformed image may be transformed and the transformed partially deformed image may be output to the projector 7.

[0201] 2.8. Variation 8 In the embodiments and modifications described above, a computer 1 was used as an example of the information processing device according to the present invention. However, a smartphone or tablet terminal having similar functions to the computer 1 may be used instead. For example, if a smartphone is used, it may have functions similar to the camera 5 in addition to the functions similar to the computer 1. [Explanation of symbols]

[0202] 1...Computer, 3...Torso, 5...Camera, 7...Projector, 10...Storage device, 12...Processing device, 14...Communication device, 16...Touch panel, 100...Program, 101...Image data, 102...Imagine image data, 103...Mask image data, 104...Setting image data, 105...Transformation image data, 106...Preview image data, 107...Input information, 110...Transformation formula information, 111...Transformation table, 112...Output table, 120...Take 121... Display control unit, 122... Output unit, 130... Image editing unit, 131... Input management unit, 132... Image transformation unit, 133... Preview generation unit, 134... Conversion formula generation unit, 160... Display unit, 161... Input unit, B1... Button, DG1... Point, DP1... Point, MG1... Marker, MP1... Marker, GS... Projection image, GC1... Setting image, GG1... Image, GP1... Acquired image, GV1... Preview image, GM1... Mask image, R1... Area, RP... Acquired area.

Claims

1. To obtain the first image, Acquiring an image of an object being projected by projected light, A setting image is displayed having a first region where the first image is displayed, a second region where the captured image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the projected object included in the captured image. Includes, The first region, the second region, and the third region are displayed in a spatially arranged order. How to display images.

2. In the aforementioned setting image, The system accepts a first operation from the user to specify a first point on the first image, The system accepts a second operation from the user to specify a second point on the captured image, By making the first point correspond to the second point, a preview image including at least a portion of the second image, which is a deformed version of the first image, is displayed in the third region. It further includes, The second image is the deformed image. The method for displaying an image according to claim 1.

3. While a preview image including at least a portion of the second image is displayed, the system accepts from the user at least one of the following: a third operation to specify a third point on the first image that replaces the first point, or a fourth operation to specify a fourth point on the captured image that replaces the second point. When the third operation described above is received, and a fifth operation instructing the user to deform the first image is received, a preview image including at least a portion of the third image, which is the first image deformed by associating the third point with the second point, is displayed in the third region. When the fourth operation is received and the fifth operation is received from the user, a preview image including at least a portion of the fourth image, which is obtained by transforming the first image by making the first point correspond to the fourth point, is displayed in the third area. When the third and fourth operations are received, and the fifth operation is received from the user, a preview image including at least a portion of the fifth image, which is a deformed version of the first image by associating the third point with the fourth point, is displayed in the third region. It further includes, The third image, the fourth image, and the fifth image are the deformed images. The method for displaying an image according to claim 2.

4. While a preview image including at least a portion of the second image is displayed, the system accepts a sixth operation from the user to specify a fifth point on the first image, While a preview image including at least a portion of the second image is displayed, the system accepts a seventh operation from the user to specify a sixth point on the captured image, When a fifth operation instructing the user to deform the first image is received, a preview image including at least a portion of the sixth image, which is the deformed version of the first image by associating the first point with the second point and the fifth point with the sixth point, is displayed in the third area. It further includes, The sixth image is the deformed image. The method for displaying an image according to claim 2.

5. While a preview image including at least a portion of the sixth image is displayed, the system accepts from the user either an eighth operation to deselect the first or second point, or a ninth operation to deselect the fifth or sixth point. When the eighth operation is received and the fifth operation is received from the user, a preview image including at least a portion of the seventh image, which is a deformed version of the first image by associating the fifth point with the sixth point, is displayed in the third region. When the ninth operation is received and the fifth operation is received from the user, a preview image including at least a portion of the eighth image, which is obtained by transforming the first image by making the first point correspond to the second point, is displayed in the third region. It further includes, The seventh image and the eighth image are the deformed images. The method for displaying an image according to claim 4.

6. While a preview image including at least a portion of the second image is displayed, the system accepts from the user either a third operation to specify a third point on the first image that replaces the first point, or a fourth operation to specify a fourth point on the captured image that replaces the second point. When the third operation is received from the user, a preview image including at least a portion of the third image, which is obtained by transforming the first image by associating the third point with the second point, is displayed in the third region. When the fourth operation is received from the user, a preview image including at least a portion of the fourth image, which is obtained by transforming the first image by associating the first point with the fourth point, is displayed in the third region. It further includes, The third image and the fourth image are the deformed images. The method for displaying an image according to claim 2.

7. While a preview image including at least a portion of the second image is displayed, the system accepts a sixth operation from the user to specify a fifth point on the first image, While a preview image including at least a portion of the second image is displayed, the system accepts a seventh operation from the user to specify a sixth point on the captured image, When the sixth operation and the seventh operation are received, a preview image including at least a portion of the sixth image obtained by transforming the first image by associating the first point with the second point and the fifth point with the sixth point is displayed in the third area. It further includes, The sixth image is the deformed image. The method for displaying an image according to claim 2.

8. The aforementioned preview image is an image obtained by combining at least a portion of the deformed image with the captured image. A method for displaying an image according to any one of claims 1 to 7.

9. The aforementioned preview image is an image obtained by combining the captured image with a partially deformed image, which is obtained by extracting a part of the deformed image based on a mask image generated from the captured image. The method for displaying an image according to claim 8.

10. A tenth operation is accepted from the user to specify whether the preview image includes the deformed image or includes a partially deformed image in which a part of the deformed image is extracted based on a mask image generated from the captured image, In response to the 10th operation, the preview image is switched to include the deformed image or the partially deformed image, This also includes, The method for displaying an image according to claim 8.

11. The aforementioned setting image further comprises a fourth region, The third region displays either a preview image including the deformed image, or a preview image including a partially deformed image in which a portion of the deformed image is extracted based on a mask image generated from the captured image. The fourth region displays the other of either the preview image including the deformed image or the preview image including the partially deformed image. The method for displaying an image according to claim 8.

12. The projector, which projects light corresponding to the input image onto the object to be projected, outputs the deformed image as the input image. This also includes, A method for displaying an image according to any one of claims 1 to 11.

13. To obtain the first image, Acquiring an image of an object being projected by projected light, A setting image is displayed having a first region where the first image is displayed, a second region where the captured image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the projected object included in the captured image. In the aforementioned setting image, The system accepts a first operation from the user to specify a first point on the first image, The system accepts a second operation from the user to specify a second point on the captured image, By making the first point correspond to the second point, a preview image including at least a portion of the second image, which is a deformed version of the first image, is displayed in the third region. While a preview image including at least a portion of the second image is displayed, the system accepts from the user at least one of the following: a third operation to specify a third point on the first image that replaces the first point, or a fourth operation to specify a fourth point on the captured image that replaces the second point. When the third operation described above is received, and a fifth operation instructing the user to deform the first image is received, a preview image including at least a portion of the third image, which is the first image deformed by associating the third point with the second point, is displayed in the third region. When the fourth operation is received and the fifth operation is received from the user, a preview image including at least a portion of the fourth image, which is obtained by transforming the first image by making the first point correspond to the fourth point, is displayed in the third area. When the third and fourth operations are received, and the fifth operation is received from the user, a preview image including at least a portion of the fifth image, which is a deformed version of the first image by associating the third point with the fourth point, is displayed in the third region. Includes, The second image, the third image, the fourth image, and the fifth image are the deformed images. How to display images.

14. To obtain the first image, Acquiring an image of an object being projected by projected light, A setting image is displayed having a first region where the first image is displayed, a second region where the captured image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the projected object included in the captured image. In the aforementioned setting image, The system accepts a first operation from the user to specify a first point on the first image, The system accepts a second operation from the user to specify a second point on the captured image, By making the first point correspond to the second point, a preview image including at least a portion of the second image, which is a deformed version of the first image, is displayed in the third region. While a preview image including at least a portion of the second image is displayed, the system accepts a sixth operation from the user to specify a fifth point on the first image, While a preview image including at least a portion of the second image is displayed, the system accepts a seventh operation from the user to specify a sixth point on the captured image, When a fifth operation instructing the user to deform the first image is received, a preview image including at least a portion of the sixth image, which is the deformed version of the first image by associating the first point with the second point and the fifth point with the sixth point, is displayed in the third area. Includes, The second image and the sixth image are the deformed images. How to display images.

15. To obtain the first image, Acquiring an image of an object being projected by projected light, A setting image is displayed having a first region where the first image is displayed, a second region where the captured image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the projected object included in the captured image. In the aforementioned setting image, The system accepts a first operation from the user to specify a first point on the first image, The system accepts a second operation from the user to specify a second point on the captured image, By making the first point correspond to the second point, a preview image including at least a portion of the second image, which is a deformed version of the first image, is displayed in the third region. While a preview image including at least a portion of the second image is displayed, the system accepts from the user either a third operation to specify a third point on the first image that replaces the first point, or a fourth operation to specify a fourth point on the captured image that replaces the second point. When the third operation is received from the user, a preview image including at least a portion of the third image, which is obtained by transforming the first image by associating the third point with the second point, is displayed in the third region. When the fourth operation is received from the user, a preview image including at least a portion of the fourth image, which is obtained by transforming the first image by associating the first point with the fourth point, is displayed in the third region. Includes, The second image, the third image, and the fourth image are the deformed images. How to display images.

16. To obtain the first image, Acquiring an image of an object being projected by projected light, A setting image is displayed having a first region where the first image is displayed, a second region where the captured image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the projected object included in the captured image. In the aforementioned setting image, The system accepts a first operation from the user to specify a first point on the first image, The system accepts a second operation from the user to specify a second point on the captured image, By making the first point correspond to the second point, a preview image including at least a portion of the second image, which is a deformed version of the first image, is displayed in the third region. While a preview image including at least a portion of the second image is displayed, the system accepts a sixth operation from the user to specify a fifth point on the first image, While a preview image including at least a portion of the second image is displayed, the system accepts a seventh operation from the user to specify a sixth point on the captured image, When the sixth operation and the seventh operation are received, a preview image including at least a portion of the sixth image obtained by transforming the first image by associating the first point with the second point and the fifth point with the sixth point is displayed in the third area. Includes, The second image and the sixth image are the deformed images. How to display images.

17. To obtain the first image, Acquiring an image of an object being projected by projected light, A setting image is displayed having a first region where the first image is displayed, a second region where the captured image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the projected object included in the captured image. Includes, The aforementioned preview image is an image obtained by combining at least a part of the deformed image and the captured image. The aforementioned preview image is an image obtained by combining the captured image with a partially deformed image, which is obtained by extracting a part of the deformed image based on a mask image generated from the captured image. How to display images.

18. To obtain the first image, Acquiring an image of an object being projected by projected light, A setting image is displayed having a first region where the first image is displayed, a second region where the captured image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the projected object included in the captured image. A tenth operation is accepted from the user to specify whether the preview image includes the deformed image or includes a partially deformed image in which a part of the deformed image is extracted based on a mask image generated from the captured image, In response to the 10th operation, the preview image is switched to include the deformed image or the partially deformed image, Includes, The aforementioned preview image is an image obtained by combining at least a portion of the deformed image with the captured image. How to display images.

19. To obtain the first image, Acquiring an image of an object being projected by projected light, A setting image is displayed having a first region where the first image is displayed, a second region where the captured image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the projected object included in the captured image. Includes, The aforementioned preview image is an image obtained by combining at least a part of the deformed image and the captured image. The aforementioned setting image further comprises a fourth region, The third region displays either a preview image including the deformed image, or a preview image including a partially deformed image in which a portion of the deformed image is extracted based on a mask image generated from the captured image. The fourth region displays the other of either the preview image including the deformed image or the preview image including the partially deformed image. How to display images.

20. Equipped with one or more processors, The one or more processors described above are: To obtain the first image, Acquiring an image of an object being projected by projected light, A setting image is displayed on a display device having a first region where the first image is displayed, a second region where the captured image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the projected object included in the captured image. Execute, The first region, the second region, and the third region are displayed in a spatially arranged order. Information processing device.

21. Equipped with one or more processors, The one or more processors described above are: To obtain the first image, Acquiring an image of an object being projected by projected light, A setting image is displayed on a display device having a first region where the first image is displayed, a second region where the captured image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the projected object included in the captured image. In the aforementioned setting image, The system accepts a first operation from the user to specify a first point on the first image, The system accepts a second operation from the user to specify a second point on the captured image, By making the first point correspond to the second point, a preview image including at least a portion of the second image, which is a deformed version of the first image, is displayed in the third region. While a preview image including at least a portion of the second image is displayed, the system accepts from the user at least one of the following: a third operation to specify a third point on the first image that replaces the first point, or a fourth operation to specify a fourth point on the captured image that replaces the second point. When the third operation described above is received, and a fifth operation instructing the user to deform the first image is received, a preview image including at least a portion of the third image, which is the first image deformed by associating the third point with the second point, is displayed in the third region. When the fourth operation is received and the fifth operation is received from the user, a preview image including at least a portion of the fourth image, which is obtained by transforming the first image by making the first point correspond to the fourth point, is displayed in the third area. When the third and fourth operations are received, and the fifth operation is received from the user, a preview image including at least a portion of the fifth image, which is a deformed version of the first image by associating the third point with the fourth point, is displayed in the third region. Execute, The second image, the third image, the fourth image, and the fifth image are the deformed images. Information processing device.

22. Equipped with one or more processors, The one or more processors described above are: To obtain the first image, Acquiring an image of an object being projected by projected light, A setting image is displayed on a display device having a first region where the first image is displayed, a second region where the captured image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the projected object included in the captured image. In the aforementioned setting image, The system accepts a first operation from the user to specify a first point on the first image, The system accepts a second operation from the user to specify a second point on the captured image, By making the first point correspond to the second point, a preview image including at least a portion of the second image, which is a deformed version of the first image, is displayed in the third region. While a preview image including at least a portion of the second image is displayed, the system accepts a sixth operation from the user to specify a fifth point on the first image, While a preview image including at least a portion of the second image is displayed, the system accepts a seventh operation from the user to specify a sixth point on the captured image, When a fifth operation instructing the user to deform the first image is received, a preview image including at least a portion of the sixth image, which is the deformed version of the first image by associating the first point with the second point and the fifth point with the sixth point, is displayed in the third area. Execute, The second image and the sixth image are the deformed images. Information processing device.

23. Equipped with one or more processors, The one or more processors described above are: To obtain the first image, Acquiring an image of an object being projected by projected light, A setting image is displayed on a display device having a first region where the first image is displayed, a second region where the captured image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the projected object included in the captured image. In the aforementioned setting image, The system accepts a first operation from the user to specify a first point on the first image, The system accepts a second operation from the user to specify a second point on the captured image, By making the first point correspond to the second point, a preview image including at least a portion of the second image, which is a deformed version of the first image, is displayed in the third region. While a preview image including at least a portion of the second image is displayed, the system accepts from the user either a third operation to specify a third point on the first image that replaces the first point, or a fourth operation to specify a fourth point on the captured image that replaces the second point. When the third operation is received from the user, a preview image including at least a portion of the third image, which is obtained by transforming the first image by associating the third point with the second point, is displayed in the third region. When the fourth operation is received from the user, a preview image including at least a portion of the fourth image, which is obtained by transforming the first image by associating the first point with the fourth point, is displayed in the third region. Execute, The second image, the third image, and the fourth image are the deformed images. Information processing device.

24. Equipped with one or more processors, The one or more processors described above are: To obtain the first image, Acquiring an image of an object being projected by projected light, A setting image is displayed on a display device having a first region where the first image is displayed, a second region where the captured image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the projected object included in the captured image. In the aforementioned setting image, The system accepts a first operation from the user to specify a first point on the first image, The system accepts a second operation from the user to specify a second point on the captured image, By making the first point correspond to the second point, a preview image including at least a portion of the second image, which is a deformed version of the first image, is displayed in the third region. While a preview image including at least a portion of the second image is displayed, the system accepts a sixth operation from the user to specify a fifth point on the first image, While a preview image including at least a portion of the second image is displayed, the system accepts a seventh operation from the user to specify a sixth point on the captured image, When the sixth operation and the seventh operation are received, a preview image including at least a portion of the sixth image obtained by transforming the first image by associating the first point with the second point and the fifth point with the sixth point is displayed in the third area. Execute, The second image and the sixth image are the deformed images. Information processing device.

25. Equipped with one or more processors, The one or more processors described above are: To obtain the first image, Acquiring an image of an object being projected by projected light, A setting image is displayed on a display device having a first region where the first image is displayed, a second region where the captured image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the projected object included in the captured image. Execute, The aforementioned preview image is an image obtained by combining at least a part of the deformed image and the captured image. The aforementioned preview image is an image obtained by combining the captured image with a partially deformed image, which is obtained by extracting a part of the deformed image based on a mask image generated from the captured image. Information processing device.

26. Equipped with one or more processors, The one or more processors described above are: To obtain the first image, Acquiring an image of an object being projected by projected light, A setting image is displayed on a display device having a first region where the first image is displayed, a second region where the captured image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the projected object included in the captured image. A tenth operation is accepted from the user to specify whether the preview image includes the deformed image or includes a partially deformed image in which a part of the deformed image is extracted based on a mask image generated from the captured image, In response to the 10th operation, the preview image is switched to include the deformed image or the partially deformed image, Execute, The aforementioned preview image is an image obtained by combining at least a portion of the deformed image with the captured image. Information processing device.

27. Equipped with one or more processors, The one or more processors described above are: To obtain the first image, Acquiring an image of an object being projected by projected light, A setting image is displayed on a display device having a first region where the first image is displayed, a second region where the captured image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the projected object included in the captured image. Execute, The aforementioned preview image is an image obtained by combining at least a part of the deformed image and the captured image. The aforementioned setting image further comprises a fourth region, The third region displays either a preview image including the deformed image, or a preview image including a partially deformed image in which a portion of the deformed image is extracted based on a mask image generated from the captured image. The fourth region displays the other of either the preview image including the deformed image or the preview image including the partially deformed image. Information processing device.

28. To the processor, To obtain the first image, Acquiring an image of an object being projected by projected light, A setting image is displayed on a display device having a first region where the first image is displayed, a second region where the captured image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the projected object included in the captured image. Make it run, The first region, the second region, and the third region are displayed in a spatially arranged order. program.

29. To the processor, To obtain the first image, Acquiring an image of an object being projected by projected light, A setting image is displayed on a display device having a first region where the first image is displayed, a second region where the captured image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the projected object included in the captured image. In the aforementioned setting image, The system accepts a first operation from the user to specify a first point on the first image, The system accepts a second operation from the user to specify a second point on the captured image, By making the first point correspond to the second point, a preview image including at least a portion of the second image, which is a deformed version of the first image, is displayed in the third region. While a preview image including at least a portion of the second image is displayed, the system accepts from the user at least one of the following: a third operation to specify a third point on the first image that replaces the first point, or a fourth operation to specify a fourth point on the captured image that replaces the second point. When the third operation described above is received, and a fifth operation instructing the user to deform the first image is received, a preview image including at least a portion of the third image, which is the first image deformed by associating the third point with the second point, is displayed in the third region. When the fourth operation is received and the fifth operation is received from the user, a preview image including at least a portion of the fourth image, which is obtained by transforming the first image by making the first point correspond to the fourth point, is displayed in the third area. When the third and fourth operations are received, and the fifth operation is received from the user, a preview image including at least a portion of the fifth image, which is a deformed version of the first image by associating the third point with the fourth point, is displayed in the third region. Make it run, The second image, the third image, the fourth image, and the fifth image are the deformed images. program.

30. To the processor, To obtain the first image, Acquiring an image of an object being projected by projected light, A setting image is displayed on a display device having a first region where the first image is displayed, a second region where the captured image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the projected object included in the captured image. In the aforementioned setting image, The system accepts a first operation from the user to specify a first point on the first image, The system accepts a second operation from the user to specify a second point on the captured image, By making the first point correspond to the second point, a preview image including at least a portion of the second image, which is a deformed version of the first image, is displayed in the third region. While a preview image including at least a portion of the second image is displayed, the system accepts a sixth operation from the user to specify a fifth point on the first image, While a preview image including at least a portion of the second image is displayed, the system accepts a seventh operation from the user to specify a sixth point on the captured image, When a fifth operation instructing the user to deform the first image is received, a preview image including at least a portion of the sixth image, which is the deformed version of the first image by associating the first point with the second point and the fifth point with the sixth point, is displayed in the third area. Make it run, The second image and the sixth image are the deformed images. program.

31. To the processor, To obtain the first image, Acquiring an image of an object being projected by projected light, A setting image is displayed on a display device having a first region where the first image is displayed, a second region where the captured image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the projected object included in the captured image. In the aforementioned setting image, The system accepts a first operation from the user to specify a first point on the first image, The system accepts a second operation from the user to specify a second point on the captured image, By making the first point correspond to the second point, a preview image including at least a portion of the second image, which is a deformed version of the first image, is displayed in the third region. While a preview image including at least a portion of the second image is displayed, the system accepts from the user either a third operation to specify a third point on the first image that replaces the first point, or a fourth operation to specify a fourth point on the captured image that replaces the second point. When the third operation is received from the user, a preview image including at least a portion of the third image, which is obtained by transforming the first image by associating the third point with the second point, is displayed in the third region. When the fourth operation is received from the user, a preview image including at least a portion of the fourth image, which is obtained by transforming the first image by associating the first point with the fourth point, is displayed in the third region. Make it run, The second image, the third image, and the fourth image are the deformed images. program.

32. To the processor, To obtain the first image, Acquiring an image of an object being projected by projected light, A setting image is displayed on a display device having a first region where the first image is displayed, a second region where the captured image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the projected object included in the captured image. In the aforementioned setting image, The system accepts a first operation from the user to specify a first point on the first image, The system accepts a second operation from the user to specify a second point on the captured image, By making the first point correspond to the second point, a preview image including at least a portion of the second image, which is a deformed version of the first image, is displayed in the third region. While a preview image including at least a portion of the second image is displayed, the system accepts a sixth operation from the user to specify a fifth point on the first image, While a preview image including at least a portion of the second image is displayed, the system accepts a seventh operation from the user to specify a sixth point on the captured image, When the sixth operation and the seventh operation are received, a preview image including at least a portion of the sixth image obtained by transforming the first image by associating the first point with the second point and the fifth point with the sixth point is displayed in the third area. Make it run, The second image and the sixth image are the deformed images. program.

33. To the processor, To obtain the first image, Acquiring an image of an object being projected by projected light, A setting image is displayed on a display device having a first region where the first image is displayed, a second region where the captured image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the projected object included in the captured image. Make it run, The aforementioned preview image is an image obtained by combining at least a part of the deformed image and the captured image. The aforementioned preview image is an image obtained by combining the captured image with a partially deformed image, which is obtained by extracting a part of the deformed image based on a mask image generated from the captured image. program.

34. To the processor, To obtain the first image, Acquiring an image of an object being projected by projected light, A setting image is displayed on a display device having a first region where the first image is displayed, a second region where the captured image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the projected object included in the captured image. A tenth operation is accepted from the user to specify whether the preview image includes the deformed image or includes a partially deformed image in which a part of the deformed image is extracted based on a mask image generated from the captured image, In response to the 10th operation, the preview image is switched to include the deformed image or the partially deformed image, Make it run, The aforementioned preview image is an image obtained by combining at least a portion of the deformed image with the captured image. program.

35. To the processor, To obtain the first image, Acquiring an image of an object being projected by projected light, A setting image is displayed on a display device having a first region where the first image is displayed, a second region where the captured image is displayed, and a third region where a preview image is displayed which includes at least a portion of a deformed image obtained by deforming the first image according to the image of the projected object included in the captured image. Make it run, The aforementioned preview image is an image obtained by combining at least a part of the deformed image and the captured image. The aforementioned setting image further comprises a fourth region, The third region displays either a preview image including the deformed image, or a preview image including a partially deformed image in which a portion of the deformed image is extracted based on a mask image generated from the captured image. The fourth region displays the other of either the preview image including the deformed image or the preview image including the partially deformed image. program.

Images