Presentation method, information processing apparatus, and program

Abstract

To appropriately and easily acquire a picked-up image used in correcting the position and shape of a projection image.SOLUTION: A display method includes: controlling a projector to display a projection image on a projection surface; controlling the projector to display pointer images located at corners of the projection image; changing the positions of the corners of the projection image based on an operation from a user to move the pointer images to execute first correction to correct the shape of the projection image; and controlling the projector to display guide images indicating the positions of one or more markers to be attached by the user at predetermined positions on the projection surface with reference to the positions of the pointer images moved based on the operation from the user.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] The present invention relates to a display method, an information processing apparatus, and a program.

Background Art

[0002] Techniques have been developed to correct the position and shape of an image displayed on a projection surface by an apparatus such as a projector using a captured image of markers arranged on the projection surface. For example, Patent Document 1 discloses a detection device including a display unit, an imaging unit, an image acquisition unit that acquires a captured image captured by the imaging unit of a target area where a plurality of markers are arranged, a guide image corresponding to the number of markers arranged in the target area or the positional relationship of the plurality of markers, and a control unit that superimposes the guide image on the captured image captured by the imaging unit and causes the display unit to display the superimposed image. When a marker arranged in the target area is imaged, the detection device superimposes the guide image on the image output from the imaging unit and causes the display unit to display the superimposed image. By imaging the marker using the guide image as a reference, the user can arrange the image indicating the marker included in the captured image at an appropriate position. Further, the detection device can detect an image indicating a marker from the captured image at high speed and with high accuracy.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When a user captures an image of a marker using the detection device, it is necessary to adjust the direction in which the imaging unit is pointed based on the guide image. For example, for some users such as the elderly or children, it may not be easy to appropriately adjust the direction in which the imaging unit is pointed, and thus the captured image may not be appropriately obtained.

Means for Solving the Problem

[0005] One aspect of the display method according to the present invention includes displaying a projection image on a projection surface by controlling a projection device; displaying a pointer image located at an angle of the projection image by controlling the projection device; performing a first correction for correcting the shape of the projection image by changing the position of the angle of the projection image based on an operation from a user who moves the pointer image; and displaying a guide image indicating the attachment position of one or more markers by the user at a predetermined position on the projection surface based on the position of the pointer image moved based on an operation from the user by controlling the projection device.

[0006] One aspect of the information processing apparatus according to the present invention includes a processing device, and the processing device performs: displaying a projection image on a projection surface by controlling a projection device; displaying a pointer image located at an angle of the projection image by controlling the projection device; performing a first correction for correcting the shape of the projection image by changing the position of the angle of the projection image based on an operation from a user who moves the pointer image; and displaying a guide image indicating the attachment position of one or more markers by the user at a predetermined position on the projection surface based on the position of the pointer image moved based on an operation from the user by controlling the projection device.

[0007] One aspect of the program according to the present invention causes a processing device to display a projection image on a projection surface by controlling a projection device, display a pointer image located at an angle of the projection image by controlling the projection device, execute a first correction for correcting the shape of the projection image by changing the position of the corner of the projection image based on an operation from a user who moves the pointer image, and display a guide image indicating the attachment position of one or more markers by a user at a predetermined position on the projection surface based on the position of the pointer image moved based on an operation from the user by controlling the projection device.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Embodiments for Carrying Out the Invention

[0009] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. Note that the dimensions and scales of each part in the drawings may be different from the actual ones, and there are also some parts schematically shown for easy understanding. Further, the scope of the present invention is not limited to these forms unless otherwise specifically stated in the following description to limit the present invention.

[0010] 1. Embodiment In the embodiment, a projector that displays a guide image indicating the pasting position of a marker on a projection surface based on the position of a pointer image located at a corner of a projected image will be exemplified to describe the display method, information processing apparatus, and program according to the present invention. The projector according to the embodiment corrects the shape of the projected image by changing the position of the corner of the projected image based on an operation from a user who moves the pointer image. Further, the projector according to the embodiment corrects the shape of the projected image and the position where the projected image is displayed based on a captured image obtained by capturing a range including the marker pasted at the position indicated by the guide image.

[0011] 1.1. Outline of the Projector Hereinafter, the outline of the projector 1 according to the embodiment will be described with reference to FIGS. 1 to 3. FIG. 1 is a schematic diagram illustrating a state in which a pointer image GC1 is displayed. FIG. 2 is a schematic diagram for explaining the configuration of the pointer image GC1. FIG. 3 is a schematic diagram illustrating a state in which a guide image GC2 is displayed.

[0012] The projector 1 includes an imaging device 14 that images a range including a predetermined area of the projection surface, and a projection device 16 that projects projection light onto the projection surface. The imaging device 14 includes an imaging lens 142 for light collection, and an image sensor 140 that generates an imaging image by converting the light collected by the imaging lens 142 into an electrical signal. The image sensor 140 includes a plurality of pixels. The projection device 16 includes a light source (not shown), a light modulator 160 that modulates the light emitted from the light source into projection light for displaying a projection image on the projection surface, and a projection lens 162 that projects the projection light modulated by the light modulator 160 onto the projection surface. The light modulator 160 includes a plurality of pixels. The projector 1 displays an image on the projection surface by controlling the projection device 16. In the present embodiment, the projector 1 displays a projection image GP1 on the wall surface W1 that is the projection surface by controlling the projection device 16. The projection surface is not limited to the wall surface W1 and may be a screen.

[0013] Referring to FIG. 1, the projector 1 projects projection light onto the area R2 of the wall surface W1 to display a projection image GP1-1. The projection image GP1-1 is an example of the projection image GP1. The projection image GP1-1 is displayed in the area R1 of the wall surface W1. The area R1 is included in the area R2. The area R1 may be, for example, an area surrounded by the frame of the screen installed on the wall surface W1, or an area on the wall surface W1 surrounded by a frame line drawn on a part of the surface of the wall surface W1.

[0014] The projection image GP1 has corners CN1-1, CN1-2, CN1-3, and CN1-4. When not distinguishing the corners CN1-1 to CN1-4, the corners CN1-1 to CN1-4 may be referred to as "corner CN1".

[0015] Further, the projector 1 controls the projection device 16 to display the pointer images GC1-1, GC1-2, GC1-3, and GC1-4 on the wall surface W1. The pointer images GC1-1 to GC1-4 are, for example, in a cross shape where two straight lines intersect each other. The pointer image GC1-1 is located at the corner CN1-1. The pointer image GC1-2 is located at the corner CN1-2. The pointer image GC1-3 is located at the corner CN1-3. The pointer image GC1-4 is located at the corner CN1-4. In other words, the pointer image GC1-1 indicates the position of the corner CN1-1. Also, the pointer image GC1-2 indicates the position of the corner CN1-2. Also, the pointer image GC1-3 indicates the position of the corner CN1-3. Also, the pointer image GC1-4 indicates the position of the corner CN1-4. When not distinguishing between the pointer images GC1-1 to GC1-4, the pointer images GC1-1 to GC1-4 may be referred to as "pointer image GC1".

[0016] Refer to FIG. 2. The pointer image GC1 has a line L1 and a line L2. In FIG. 2, the lines L1 and L2 are illustrated as straight lines, but are not limited thereto. The lines L1 and L2 may be curves. Also, one of the lines L1 and L2 may be a straight line and the other of the lines L1 and L2 may be a curve. Also, the lines L1 and L2 intersect at the intersection point T1. That is, the pointer image GC1 has the intersection point T1.

[0017] Return to FIG. 1. By performing an operation to move the pointer image GC1, the user U changes the position of the corner CN1 of the projected image GP1 indicated by the pointer image GC1. For example, the user U performs an operation to move the pointer image GC1-2, thereby changing the position of the corner CN1-2 indicated by the pointer image GC1-2. The projector 1 corrects the shape of the projected image GP1 by changing the position of the corner CN1 of the projected image GP1 based on the operation from the user U who moves the pointer image GC1. For example, the projector 1 corrects the shape of the projected image GP1 by changing the position of the corner CN1-2 of the projected image GP1 based on the operation from the user U who moves the pointer image GC1-2. For example, the shape of the projected image GP1 is corrected by the user U so as to be substantially similar to the shape of the screen frame. That is, if the screen frame is rectangular, the shape of the projected image GP1 is shaped based on the operation from the user U so that the shape of the projected image GP1 also becomes rectangular. Note that correcting the projected image based on the operation from the user U may be referred to as "first correction". Also, correcting the projected image based on the captured image may be referred to as "second correction".

[0018] Refer to FIG. 3. The projector 1 controls the projection device 16 to display the projected image GP1-2. The projected image GP1-2 is another example of the projected image GP1. Specifically, the projector 1 updates the image displayed on the wall surface W1 from the projected image GP1-1 to the projected image GP1-2 by performing the first correction based on the operation from the user U. That is, the projected image GP1-2 is the projected image GP1 after being corrected by the first correction. On the other hand, the projected image GP1-1 is the projected image GP1 before being corrected by the first correction. Also, in FIG. 3, the pointer images GC1-1 to GC1-4 are located at positions moved based on the operation from the user U related to the first correction. Also, the projected image GP1-2 is displayed in the region R1.

[0019] The projector 1 captures an image of a range including the area R1 where the projection image GP1-2 is displayed by controlling the imaging device 14. Further, the projector 1 executes image processing on the captured image representing the result of capturing the range including the area R1 where the projection image GP1-2 is displayed, and detects a plurality of points corresponding to the four corners of the image representing the area R1 included in the captured image.

[0020] When something that explicitly indicates the position of the area R1 to the user U, for example, a boundary line between the inside and outside of the area R1 or a mark indicating the positions of the four corners of the area R1, exists on the wall surface W1, in the captured image representing the result of capturing the range including the area R1, the position of the image representing the area R1 is explicitly indicated. Therefore, when image processing is executed on the captured image, a plurality of points corresponding to the four corners of the image representing the area R1 included in the captured image are detected.

[0021] On the other hand, as shown in FIG. 3, when there is nothing on the wall surface W1 that explicitly indicates the position of the area R1 to the user U, in the captured image representing the result of capturing the range including the area R1, the position of the image representing the area R1 is not explicitly indicated. Therefore, when image processing is executed on the captured image, a plurality of points corresponding to the four corners of the image representing the area R1 included in the captured image may not be detected. Here, it is assumed that in the image processing, a plurality of points corresponding to the four corners of the image representing the area R1 included in the captured image are not detected.

[0022] When there is a corner in the image representing the region R1 included in the captured image where no point is detected by image processing, the projector 1 displays a pointer image GC1 indicating the position of the corner CN1 of the projected image GP1 corresponding to the corner where no point is detected, and a corresponding guide image. That is, the projector 1 controls the projection device 16 to display the guide image GC2-1, the guide image GC2-2, the guide image GC2-3, and the guide image GC2-4 on the wall surface W1. The guide images GC2-1 to GC2-4 are images indicating the attachment positions of one or more markers M by the user U. That is, the user U can grasp the attachment positions of the plurality of markers M by checking the guide images GC2-1 to GC2-4. In the present embodiment, the guide images GC2-1 to GC2-4 are images having a rectangular frame line. When not distinguishing the guide images GC2-1 to GC2-4, the guide images GC2-1 to GC2-4 may be referred to as "guide image GC2".

[0023] The guide image GC2-1 is displayed at a predetermined position based on the position of the pointer image GC1-1. Specifically, the guide image GC2-1 is displayed at a position centered on the intersection point T1 of the pointer image GC1-1. That is, the guide image GC2-1 surrounds the intersection point T1 of the pointer image GC1-1. The user U can grasp the attachment positions of one or more markers M corresponding to the position of the corner CN1-1 indicated by the pointer image GC1-1 by checking the guide image GC2-1.

[0024] The guide image GC2-2 is displayed at a predetermined position based on the position of the pointer image GC1-2. Specifically, the guide image GC2-2 is displayed at a position centered on the intersection point T1 of the pointer image GC1-2. That is, the guide image GC2-2 surrounds the intersection point T1 of the pointer image GC1-2. The user U can grasp the attachment positions of one or more markers M corresponding to the position of the corner CN1-2 indicated by the pointer image GC1-2 by checking the guide image GC2-2.

[0025] The guide image GC2-3 is displayed at a predetermined position based on the position of the pointer image GC1-3. Specifically, the guide image GC2-3 is displayed at a position centered on the intersection point T1 of the pointer image GC1-3. That is, the guide image GC2-3 surrounds the intersection point T1 of the pointer image GC1-3. By checking the guide image GC2-3, the user U can grasp the attachment position of one or more markers M corresponding to the position of the angle CN1-3 indicated by the pointer image GC1-3.

[0026] The guide image GC2-4 is displayed at a predetermined position based on the position of the pointer image GC1-4. Specifically, the guide image GC2-4 is displayed at a position centered on the intersection point T1 of the pointer image GC1-4. That is, the guide image GC2-4 surrounds the intersection point T1 of the pointer image GC1-4. By checking the guide image GC2-4, the user U can grasp the attachment position of one or more markers M corresponding to the position of the angle CN1-4 indicated by the pointer image GC1-4.

[0027] Also, the projector 1 controls the projection device 16 to display the message SW1-2 on the wall surface W1. The message SW1-2 is a message for prompting the user U to attach one or more markers M. Specifically, the message SW1-2 is a message for prompting the user U to attach one or more markers M at the position indicated by the guide image GC2-2. That is, by checking the message SW1-2, the user U can grasp that the marker M should be attached at the position indicated by the guide image GC2-2.

[0028] The message SW1-2 includes the number NL1-2. The number NL1-2 is the number "2" indicating the number of markers M to be attached at the position indicated by the guide image GC2-2. That is, by checking the number NL1-2, the user U can grasp the number of markers M to be attached at the position indicated by the guide image GC2-2.

[0029] Note that a plurality of messages prompting the user U to attach one or more markers M may be displayed. For example, in addition to the message SW1-2 corresponding to the guide image GC2-2, the message corresponding to the guide image GC2-1, the message corresponding to the guide image GC2-3, and the message corresponding to the guide image GC2-4 may be displayed. When a plurality of messages prompting the user U to attach one or more markers M are displayed, it is preferable that the plurality of messages correspond one-to-one to the plurality of guide images GC2 displayed on the wall surface W1.

[0030] The projector 1 displays the guide image GC2 based on the position of the pointer image GC1 moved based on the operation from the user U related to the first correction. By checking the guide image GC2, the user U can appropriately attach one or more markers M. Further, when one or more markers M are attached at the position indicated by the guide image GC2, the positional relationship between the one or more markers M and the angle CN1 indicated by the pointer image GC1 corresponding to the guide image GC2 becomes a predetermined positional relationship. Specifically, the one or more markers M are arranged within a predetermined range from the angle CN1. As a result, the projector 1 can image the attached one or more markers M by imaging the region R1 which is the region where the projected image GP1 is displayed. That is, the user U can cause the projector 1 to acquire the imaging image used for the second correction without finely adjusting the direction in which the imaging device 14 is directed.

[0031] 1.2. Configuration and functions of the projector Hereinafter, with reference to FIGS. 4 to 7, the configuration and functions of the projector 1 according to the embodiment will be described.

[0032] FIG. 4 is a block diagram showing the configuration of the projector 1 according to the embodiment. The projector 1 includes a storage device 10 that stores various information, a processing device 12 that controls the operation of the projector 1, an imaging device 14 that images a range including a predetermined area of the projection surface, a projection device 16 that projects projection light onto the projection surface, and an operation device 18 that receives an input operation from the user U. The processing device 12 has functions as a projection control unit 120, an imaging control unit 121, a detection unit 122, a correction unit 123, and a coordinate management unit 124. The imaging device 14 includes an image sensor 140 and an imaging lens 142 as described above. Further, the projection device 16 includes a light source (not shown), a light modulator 160, and a projection lens 162 as described above.

[0033] The storage device 10 is configured to include, for example, a volatile memory such as a RAM and a non-volatile memory such as a ROM. Here, RAM is an abbreviation for Random Access Memory. Also, ROM is an abbreviation for Read Only Memory.

[0034] FIG. 5 is a block diagram showing the configuration of the storage device 10 according to the embodiment. The non-volatile memory included in the storage device 10 stores a program 100 that defines the operation of the projector 1, projection image information 101 representing an image projected from the projection device 16, imaging image information 102 representing the result of imaging a range including an area on the projection surface where the projection image is displayed, and coordinate information 105 representing the coordinates of points included in various images.

[0035] The coordinate information 105 includes first coordinate information 106 representing the coordinates of the four corners of the image indicated by the projection image information 101 in the light modulator 160, and second coordinate information 107 representing the coordinates of a plurality of points detected from the image indicated by the imaging image information 102.

[0036] The volatile memory included in the storage device 10 is used by the processing device 12 as a work area when executing the program 100.

[0037] Note that part or all of the storage device 10 may be provided in an external storage device, an external server, or the like. Also, part or all of the various information stored in the storage device 10 may be stored in the storage device 10 in advance, or may be acquired from an external storage device, an external server, or the like.

[0038] Returning to FIG. 4, the processing device 12 is configured to include one or more CPUs. However, the processing device 12 may be provided with a programmable logic device such as an FPGA instead of or in addition to the CPU. Here, CPU is an abbreviation for Central Processing Unit, and FPGA is an abbreviation for Field-Programmable Gate Array.

[0039] When the CPU or the like included in the processing device 12 executes the program 100, the processing device 12 functions as the projection control unit 120, the imaging control unit 121, the detection unit 122, the correction unit 123, and the coordinate management unit 124 shown in FIG. 4.

[0040] The projection control unit 120 controls the projection device to project projection light for displaying an image onto the projection surface. Specifically, the projection control unit 120 causes the projection device to project projection light based on the projection image information 101, thereby displaying a projection image on the projection surface. In other words, the projection control unit 120 causes the projection device to project the image indicated by the projection image information 101, thereby displaying a projection image on the projection surface. Also, the projection control unit 120 controls the projection device to display an image for assisting the operation of the user U on the projection surface.

[0041] In the present embodiment, the projection control unit 120 controls the projection device 16 to project projection light for displaying an image onto the wall surface W1.

[0042] Specifically, the projection control unit 120 causes the projection device 16 to project projection light based on the projection image information 101, thereby displaying the projection image GP1 on the wall surface W1. In other words, the projection control unit 120 causes the projection device 16 to project the image indicated by the projection image information 101, thereby displaying the projection image GP1 on the wall surface W1.

[0043] Also, the projection control unit 120 controls the projection device 16 to display the pointer image GC1 located at the corner CN1 of the projection image GP1 on the wall surface W1.

[0044] Also, the projection control unit 120 controls the projection device 16 to display the guide image GC2 at a predetermined position based on the position of the pointer image GC1. The guide image GC2 surrounds the intersection point T1 of the pointer image GC1.

[0045] Also, the projection control unit 120 controls the projection device 16 to display a message on the wall surface W1 that prompts the user U to attach one or more markers M. For example, the projection control unit 120 controls the projection device 16 to display a message SW1-2 that prompts the user U to attach one or more markers M at the position indicated by the guide image GC2-2.

[0046] The message that prompts the user U to attach one or more markers M may include a number indicating the number of markers M to be attached. For example, the message SW1-2 includes the number NL1-2. In other words, the projection control unit 120 controls the projection device 16 to display the number NL1-2 on the wall surface W1 that indicates the number of one or more markers M to be attached by the user U. Note that the number indicating the number of one or more markers M to be attached may not be included in the message. The projection control unit 120 may display only the number without displaying the message.

[0047] The imaging control unit 121 controls the imaging device to image a range including the area on the projection surface where the projection image is displayed. Further, the imaging control unit 121 acquires an imaging image representing the result of the imaging from the imaging device. Further, the imaging control unit 121 causes the storage device 10 to store imaging image information 102 representing the acquired imaging image.

[0048] In the present embodiment, the imaging control unit 121 controls the imaging device 14 to image a range including the area R1 on the wall surface W1 where the projection image GP1 is displayed. Further, the imaging control unit 121 acquires an imaging image representing the result of the imaging from the imaging device 14. Further, the imaging control unit 121 causes the storage device 10 to store imaging image information 102 representing the acquired imaging image.

[0049] The detection unit 122 detects points included in the image by performing image processing on the image indicated by various image information. That is, the detection unit 122 acquires coordinate information 105 representing the coordinates of the detected points. Further, the detection unit 122 causes the storage device 10 to store the acquired coordinate information 105.

[0050] In the present embodiment, the detection unit 122 performs image processing on the image indicated by the imaging image information 102 to detect a plurality of points included in the image indicated by the imaging image information 102. That is, the detection unit 122 acquires second coordinate information 107 representing the coordinates of a plurality of points included in the image indicated by the imaging image information 102. Further, the detection unit 122 causes the storage device 10 to store the acquired second coordinate information 107.

[0051] In the function related to the detection of points, a known image processing technique may be used. Examples of known image processing techniques related to the detection of points include template matching, centroid detection, and an algorithm called "AKAZE". In this specification, a detailed technical explanation related to the detection of points is omitted.

[0052] The correction unit 123 corrects the position and shape of the projected image displayed on the projection surface. Specifically, the correction unit 123 executes a first correction based on an operation from the user U.

[0053] In the present embodiment, the correction unit 123 executes a first correction for correcting the projected image GP1 by changing the position of the corner CN1 of the projected image GP1 based on an operation from the user U who moves the pointer image GC1. In other words, the correction unit 123 updates the projected image information 101 representing the image projected from the projection device 16 when the projected image GP1 is displayed, based on an operation from the user U who moves the pointer image GC1. Further, the correction unit 123 updates the first coordinate information 106 representing the coordinates of the four corners of the image indicated by the projected image information 101, based on an operation from the user U who moves the pointer image GC1.

[0054] The coordinate management unit 124 executes various processes related to the coordinates of the points detected from the image.

[0055] The coordinate management unit 124 determines whether a plurality of points corresponding to the four corners of the image representing the area on the projection surface where the projected image is displayed are detected in the captured image indicated by the captured image information 102. Specifically, the coordinate management unit 124 determines whether at least one point is detected at each of the four corners. When a plurality of points corresponding to the four corners of the image representing the area on the projection surface where the projected image is displayed are not detected in the captured image indicated by the captured image information 102, specifically, when there is a corner among the four corners where the corresponding point is not detected, the coordinate management unit 124 controls the storage device 10 to erase the second coordinate information 107 representing the coordinates of the plurality of points included in the image indicated by the captured image information 102.

[0056] FIG. 6 is a schematic diagram for explaining the captured image GS1. The captured image GS1 is an example of the image indicated by the captured image information 102. In the present embodiment, the processing device 12 acquires the captured image GS1. The captured image GS1 is a captured image acquired when an object that explicitly indicates the position of the region R1 with respect to the user U does not exist on the wall surface W1, similar to the region R1 shown in FIG. 3.

[0057] The captured image GS1 includes an image GS11. The image GS11 is an image representing the wall surface W1. The image GS11 includes an image GV11.

[0058] The image GV11 is an image representing the region R1. The image GV11 has corners CN2-1, CN2-2, CN2-3, and CN2-4. The image GV11 includes an image GV12.

[0059] The image GV12 is an image representing the projected image GP1-2. The image GV12 has corners CN3-1, CN3-2, CN3-3, and CN3-4.

[0060] The corner CN3-1 corresponds to the corner CN1-1. Also, the corner CN3-1 corresponds to the corner CN2-1. That is, the corner CN1-1 corresponds to the corner CN2-1.

[0061] The corner CN3-2 corresponds to the corner CN1-2. Also, the corner CN3-2 corresponds to the corner CN2-2. That is, the corner CN1-2 corresponds to the corner CN2-2.

[0062] The corner CN3-3 corresponds to the corner CN1-3. Also, the corner CN3-3 corresponds to the corner CN2-3. That is, the corner CN1-3 corresponds to the corner CN2-3.

[0063] The corner CN3-4 corresponds to the corner CN1-4. Also, the corner CN3-4 corresponds to the corner CN2-4. That is, the corner CN1-4 corresponds to the corner CN2-4.

[0064] As described above, since there is no object on the wall surface W1 that explicitly indicates the position of the region R1 to the user U, in the captured image GS1 representing the result of capturing the range including the region R1, the position of the image GV11 representing the region R1 is not explicitly indicated. Therefore, when image processing is performed on the captured image GS1, a plurality of points corresponding to the corners CN2-1 to CN2-4 of the image GV11 are not detected. When a plurality of points corresponding to the corners CN2-1 to CN2-4 are not detected, the processing device 12 controls the projection device 16 to display the guide image GC2-1, the guide image GC2-2, the guide image GC2-3, and the guide image GC2-4 on the wall surface W1.

[0065] Hereinafter, for comparison with the captured image GS1 shown in FIG. 6, the schematic diagram shown in FIG. 7 will be described. FIG. 7 is a schematic diagram for explaining the captured image GS2. The captured image GS2 is another example of the image shown in the captured image information 102. The captured image GS2 is a captured image obtained when marks indicating the positions of three corners out of the four corners of the region R1, excluding the portion corresponding to the corner CN1-2, exist on the wall surface W1.

[0066] The captured image GS2 includes an image GS21. The image GS21 is an image representing the wall surface W1. The image GS21 includes an image GV21.

[0067] The image GV21 is an image representing the region R1. The image GV21 has corners CN4-1, CN4-2, CN4-3, and CN4-4. Further, the image GV21 has marks indicating the positions of the corners CN4-1, CN4-3, and CN4-4. The image GV21 includes an image GV22.

[0068] The image GV22 is an image representing the projection image GP1-2. The image GV22 has corners CN5-1, CN5-2, CN5-3, and CN5-4.

[0069] The corner CN5-1 corresponds to the corner CN1-1. Further, the corner CN5-1 corresponds to the corner CN4-1. That is, the corner CN1-1 corresponds to the corner CN4-1.

[0070] Angle CN5-2 corresponds to angle CN1-2. Also, angle CN5-2 corresponds to angle CN4-2. That is, angle CN1-2 corresponds to angle CN4-2.

[0071] Angle CN5-3 corresponds to angle CN1-3. Also, angle CN5-3 corresponds to angle CN4-3. That is, angle CN1-3 corresponds to angle CN4-3.

[0072] Angle CN5-4 corresponds to angle CN1-4. Also, angle CN5-4 corresponds to angle CN4-4. That is, angle CN1-4 corresponds to angle CN4-4.

[0073] In the captured image GS2, the position of the image GV21 representing the region R1 is partially indicated by the marks indicating the positions of angle CN4-1, the mark indicating the position of angle CN4-3, and the mark indicating the position of angle CN4-4. When image processing is performed on the captured image GS2, at least one point D corresponding to angle CN4-1, at least one point D corresponding to angle CN4-3, and at least one point D corresponding to angle CN4-4 are detected.

[0074] If there is an angle among the four angles of the image representing the region R1 for which the point D is not detected, the processing device 12 controls the projection device 16 to display the pointer image GC1 indicating the position of angle CN1 of the projection image GP1 corresponding to the angle for which the point D is not detected and the corresponding guide image GC2. Therefore, the processing device 12 controls the projection device 16 to display the pointer image GC1-2 indicating the position of angle CN1-2 corresponding to angle CN4-2 for which the point D is not detected and the corresponding guide image GC2-2. That is, the user U attaches one or more markers M within a predetermined range from angle CN1 of the projection image GP1 corresponding to the angle for which the point D is not detected among the four angles of the image representing the region R1. That is, for angle CN1 of the projection image GP1 corresponding to the angle for which the point D is detected among the four angles of the image representing the region R1, the user U can save the trouble of attaching the marker M.

[0075] Returning to FIG. 4, the imaging device 140 is an image sensor such as a CCD or a CMOS as an example. Here, CCD is an abbreviation of Charge Coupled Device, and CMOS is an abbreviation of Complementary Metal Oxide Semiconductor.

[0076] The imaging device 14 images a range including the area on the projection surface where the projection image is displayed under the control of the imaging control unit 121. Further, the imaging device 14 outputs imaging image information 102 representing the result of imaging a range including the area on the projection surface where the projection image is displayed to the processing device 12. In other words, the imaging device 14 outputs the imaging image indicated by the imaging image information 102 to the processing device 12.

[0077] The light modulator 160 includes, for example, one or more liquid crystal panels. Note that the light modulator 160 may include a DMD instead of the liquid crystal panel. The light modulator 160 modulates the light emitted from the light source into projection light for displaying a projection image on the projection surface based on the signal input from the processing device 12. The light source includes, for example, a halogen lamp, a xenon lamp, an ultra-high pressure mercury lamp, an LED, or a laser light source. Here, LED is an abbreviation of Light Emitting Diode, and DMD is an abbreviation of Digital Mirror Device.

[0078] The projection device 16 projects projection light for displaying a projection image on the projection surface under the control of the projection control unit 120. In other words, the projection device 16 projects the image input from the processing device 12 onto the projection surface.

[0079] The operation device 18 receives an input operation on the projector 1 from the user U of the projector 1. The operation device 18 is configured to include, for example, a touch panel or operation buttons provided on the housing of the projector 1. When the operation device 18 is configured to include a touch panel, the operation device 18 outputs data indicating the detected touch position to the processing device 12. Also, when the operation device 18 is configured to include operation buttons, the operation device 18 outputs data for identifying the pressed button to the processing device 12. Further, the operation device 18 may be configured to include a receiving device for receiving an operation signal output from the remote controller based on the operation of the user U. When the operation device 18 is configured to include a receiving device, the operation device 18 outputs data indicated by the operation signal received from the remote controller to the processing device 12. Thereby, the content of the input operation on the projector 1 is transmitted to the processing device 12.

[0080] 1.3. Operation of the Projector FIG. 8 is a flowchart for explaining the operation of the projector 1 according to the embodiment. The series of operations shown in the flowchart of FIG. 8 is started, for example, when the power of the projector 1 is turned on and the operation device 18 receives an input operation regarding the start of operation from the user U.

[0081] In step S101, the projection control unit 120 causes the projection device 16 to project projection light based on the projection image information 101, thereby displaying the projection image GP1-1 in the area R1 of the wall surface W1. In other words, the projection control unit 120 causes the projection device 16 to project the image indicated by the projection image information 101, thereby displaying the projection image GP1-1 in the area R1 of the wall surface W1.

[0082] Further, the projection control unit 120 controls the projection device 16 to display the pointer image GC1 located at the angle CN1 of the projection image GP1-1 on the wall surface W1. That is, the projection control unit 120 controls the projection device 16 to display the pointer image GC1-1 located at the angle CN1-1, the pointer image GC1-2 located at the angle CN1-2, the pointer image GC1-3 located at the angle CN1-3, and the pointer image GC1-4 located at the angle CN1-4.

[0083] In step S102, the correction unit 123 performs a first correction to correct the shape of the projection image GP1 by changing the position of the angle CN1 of the projection image GP1 based on an operation from the user U who moves the pointer image GC1. In other words, the correction unit 123 updates the projection image information 101 based on an operation from the user U who moves the pointer image GC1. Further, the correction unit 123 updates the first coordinate information 106 representing the coordinates of the four corners of the image indicated by the projection image information 101 based on an operation from the user U who moves the pointer image GC1. When the first correction is executed, the projection image GP1-2 is displayed on the wall surface W1.

[0084] In step S103, the imaging control unit 121 controls the imaging device 14 to image a range including the area R1 on the wall surface W1 where the projection image GP1 is displayed. Further, the imaging control unit 121 acquires an imaging image representing the result of the imaging from the imaging device 14. Further, the imaging control unit 121 stores the imaging image information 102 representing the acquired imaging image in the storage device 10.

[0085] Hereinafter, a series of operations shown in the flowchart of FIG. 8 will be described assuming that the imaging image GS1 is acquired in step S103.

[0086] In step S104, the detection unit 122 executes image processing on the captured image GS1 indicated by the captured image information 102 to detect a plurality of points D included in the captured image GS1. That is, the detection unit 122 acquires second coordinate information 107 representing the coordinates of the plurality of points D included in the captured image GS1. Further, the detection unit 122 stores the acquired second coordinate information 107 in the storage device 10.

[0087] In step S105, the coordinate management unit 124 determines whether a plurality of points D corresponding to the corners CN2-1 to CN2-4 of the image GV11 representing the region R1 on the wall surface W1 where the projected image GP1 is displayed in the captured image GS1 indicated by the captured image information 102 are detected. Specifically, the coordinate management unit 124 determines whether at least one point D is detected at each of the corners CN2-1 to CN2-4. When a plurality of points D corresponding to the corners CN2-1 to CN2-4 of the image GV11 representing the region R1 are detected in the captured image GS1, specifically, when at least one point D is detected at each of the corners CN2-1 to CN2-4, that is, when the result in step S105 is YES, the processing device 12 including the coordinate management unit 124 ends a series of operations shown in the flowchart of FIG. 8. When a plurality of points D corresponding to the corners CN2-1 to CN2-4 of the image GV11 representing the region R1 are not detected in the captured image GS1, specifically, when there is a corner among the corners CN2-1 to CN2-4 where the corresponding point D is not detected, that is, when the result in step S105 is NO, the coordinate management unit 124 advances the processing to step S106.

[0088] As described above, when image processing is executed on the captured image GS1, the point D is not detected. Therefore, the determination in step S105 is NO.

[0089] In step S106, the projection control unit 120 controls the projection device 16 to display the guide image GC2 at a predetermined position based on the position of the pointer image GC1. Specifically, the projection control unit 120, based on the second coordinate information 107, displays the guide image GC2 corresponding to the pointer image GC1 indicating the position of the corner CN1 of the projection image GP1 corresponding to the corner among CN2-1 to CN2-4 where the point D is not detected. That is, the projection control unit 120 controls the projection device 16 to display the guide image GC2-1, the guide image GC2-2, the guide image GC2-3, and the guide image GC2-4 on the wall surface W1. The guide image GC2-1 is displayed at a predetermined position based on the position of the pointer image GC1-1. The guide image GC2-2 is displayed at a predetermined position based on the position of the pointer image GC1-2. The guide image GC2-3 is displayed at a predetermined position based on the position of the pointer image GC1-3. The guide image GC2-4 is displayed at a predetermined position based on the position of the pointer image GC1-4.

[0090] Also, the projection control unit 120 controls the projection device 16 to display, on the wall surface W1, a message prompting the user U to attach one or more markers M. Specifically, the projection control unit 120 controls the projection device 16 to display, to the user U, a message SW1-2 prompting the attachment of one or more markers M at the position indicated by the guide image GC2-2. The message SW1-2 includes the number NL1-2. In other words, the projection control unit 120 controls the projection device 16 to display, on the wall surface W1, the number NL1-2 indicating the number of one or more markers M attached by the user U.

[0091] In step S107, the coordinate management unit 124 controls the storage device 10 to erase the second coordinate information 107.

[0092] When the guide images GC2-1 to GC2-4 are displayed in step S106, the user U attaches one or more markers M to the positions indicated by each of the guide images GC2-1 to GC2-4. That is, the user U attaches one or more markers M to the position indicated by the guide image GC2-1, attaches one or more markers M to the position indicated by the guide image GC2-2, attaches one or more markers M to the position indicated by the guide image GC2-3, and attaches one or more markers M to the position indicated by the guide image GC2-4. When the user U attaches one or more markers M to the positions indicated by each of the guide images GC2-1 to GC2-4, the processing device 12 executes the processing in step S103 to obtain a captured image representing the result of imaging the wall surface W1 to which the plurality of markers M are attached, specifically, the range including the region R1 of the wall surface W1. The captured image includes a plurality of images representing the markers M attached to the wall surface W1. Further, the processing device 12 executes the processing in step S104 to detect a plurality of points D corresponding to the plurality of images representing the markers M attached to the wall surface W1 from the captured image. That is, the processing device 12 can detect at least one point D at each of the four corners of the image representing the region R1 by executing image processing on the captured image. Thereby, the projector 1 can execute the second correction using the captured image.

[0093] As described above, according to the embodiment, the projector 1 displays the guide image GC2 based on the position of the pointer image GC1 moved based on the operation from the user U related to the first correction. That is, the user U can attach one or more markers M at appropriate positions by checking the guide image GC2. Further, when one or more markers M are attached at the positions indicated by the guide image GC2, the one or more markers M are arranged within a predetermined range from the corner CN1 of the projection image GP1 indicated by the guide image GC2 and the corresponding pointer image GC1. As a result, the projector 1 can image the attached one or more markers M by imaging the area R1, which is the area where the projection image GP1 is displayed. Therefore, the user U can cause the projector 1 to acquire the captured image used for the second correction without finely adjusting the direction in which the imaging device 14 is directed.

[0094] Also, according to the embodiment, the projector 1 controls the projection device 16 to display a message SW1-2 for prompting the attachment of one or more markers M on the wall surface W1. That is, the user U can understand that the marker M should be attached at the position indicated by the guide image GC2-2 by checking the message SW1-2. Further, the message SW1-2 includes a number NL1-2 indicating the number of markers M to be attached at the position indicated by the guide image GC2-2. That is, the user U can grasp the number of markers M to be attached at the position indicated by the guide image GC2-2 by checking the number NL1-2.

[0095] As described above, the display method according to the embodiment includes displaying a projection image GP1 on the wall surface W1 by controlling the projection device 16, displaying a pointer image GC1 located at a corner of the projection image GP1 by controlling the projection device 16, performing a first correction for correcting the shape of the projection image GP1 by changing the position of the corner of the projection image GP1 based on an operation from the user U who moves the pointer image GC1, and displaying a guide image GC2 indicating the attachment position of one or more markers M by the user U at a predetermined position on the wall surface W1 with reference to the position of the pointer image GC1 moved based on an operation from the user U by controlling the projection device 16.

[0096] Also, the projector 1 according to the embodiment includes a processing device 1 2 and the processing device 1 2 executes displaying a projection image GP1 on the wall surface W1 by controlling the projection device 16, displaying a pointer image GC1 located at a corner of the projection image GP1 by controlling the projection device 16, performing a first correction for correcting the shape of the projection image GP1 by changing the position of the corner of the projection image GP1 based on an operation from the user U who moves the pointer image GC1, and displaying a guide image GC2 indicating the attachment position of one or more markers M by the user U at a predetermined position on the wall surface W1 with reference to the position of the pointer image GC1 moved based on an operation from the user U by controlling the projection device 16.

[0097] Also, the program 100 according to the embodiment includes a processing device 1 2 By controlling the projection device 16, a projection image GP1 is displayed on the wall surface W1; by controlling the projection device 16, a pointer image GC1 located at the corner of the projection image GP1 is displayed; based on an operation from the user U that moves the pointer image GC1, the position of the corner of the projection image GP1 is changed to perform a first correction for correcting the shape of the projection image GP1; by controlling the projection device 16, a guide image GC2 indicating the attachment position of one or more markers M by the user U is displayed at a predetermined position on the wall surface W1 based on the position of the pointer image GC1 moved based on an operation from the user U.

[0098] That is, the user U can attach one or more markers M at appropriate positions by checking the guide image GC2. Also, when one or more markers M are attached at the positions indicated by the guide image GC2, the one or more markers M are arranged within a predetermined range from the corner CN1 of the projection image GP1 indicated by the pointer image GC1 corresponding to the guide image GC2. As a result, the projector 1 can capture the attached one or more markers M by capturing the area R1, which is the area where the projection image GP1 is displayed. Therefore, the user U can cause the projector 1 to acquire a captured image without finely adjusting the direction in which the imaging device 14 is directed.

[0099] Note that in the embodiment, the projector 1 is an example of an "information processing device", and the program 10 0 is an example of a "program", and the processing device 1 2An example of the "processing device", the projection device 16 is an example of the "projection device", the wall surface W1 is an example of the "projection surface", the projection image GP1 is an example of the "projection image", the pointer image GC1 is an example of the "pointer image", the user U is an example of the "user", the marker M is an example of the "marker", one or more markers M are an example of the "one or more markers", and the guide image GC2 is an example of the "guide image". Also, the "angle of the projection image" and the "corner of the projection image" take the corner CN1 as an example. More specifically, the "pointer image" takes the pointer image GC1-2 as an example. Also, the "guide image" takes the guide image GC2-2 as an example. Also, the "angle of the projection image" and the "corner of the projection image" take the corner CN1-2 as an example.

[0100] Also, in the display method according to the embodiment, the pointer image GC1 has the line L1 and the line L2, and the guide image GC2 has a rectangular frame line surrounding the intersection point T1 where the line L1 and the line L2 intersect.

[0101] That is, the projector 1 can more clearly indicate the attachment position of one or more markers M. Thereby, the user U can attach the marker M more accurately.

[0102] In the embodiment, the line L1 is an example of the "first line", the line L2 is an example of the "second line", and the intersection point T1 is an example of the "intersection point". Also, the "figure surrounding the intersection point" takes the rectangular frame line as an example.

[0103] Also, the display method according to the embodiment further includes obtaining an imaging image GS1 representing the result of imaging the range including the region R1 where the projection image GP1 is displayed, and performing image processing on the imaging image GS1 to detect a plurality of points D that correspond one-to-one to the four corners of the image GV11 representing the region R1 included in the imaging image GS1. If there is a corner where the point D is not detected among the four corners of the image GV11, the pointer image GC1 is located at the corner of the projection image GP1 corresponding to the corner where the point D is not detected.

[0104] That is, the projector 1 displays a pointer image GC1 indicating the position of the corner CN1 of the projection image GP1 corresponding to the corner where the point D among the four corners of the image GV11 representing the region R1 is not detected, and a corresponding guide image GC2. Therefore, the user U only needs to attach one or a plurality of markers M only to the necessary locations, and can save the trouble of attaching the markers M more than necessary.

[0105] In the embodiment, the region R1 is an example of a "display region", the point D is an example of a "feature point", and the plurality of points D are examples of "a plurality of feature points". Further, the "captured image" takes the captured image GS1 and the captured image GS2 as examples. Further, the "first image" takes the image GV11 and the image GV21 as examples. Further, the "four corners of the first image" take the corners CN2-1 to CN2-4 and the corners CN4-1 to CN4-4 as examples.

[0106] Also, in the display method according to the embodiment, displaying the guide image GC2-2 includes displaying a message SW1-2 that prompts the user U to attach one or a plurality of markers M.

[0107] Thereby, the user U can grasp that the marker M should be attached to the position indicated by the guide image GC2-2 by checking the message SW1-2.

[0108] In the embodiment, the message SW1-2 is an example of a "message".

[0109] Also, in the display method according to the embodiment, displaying the guide image GC2-2 includes displaying a number NL1-2 indicating the number of one or a plurality of markers M attached by the user U.

[0110] Thereby, the user U can grasp the number of markers M to be attached to the position indicated by the guide image GC2-2 by checking the number NL1-2.

[0111] In the embodiment, the numbers NL1-2 are an example of "numbers".

[0112] 2. Variations The above embodiments can be variously modified. Specific modification modes are exemplified below. Also, two or more modes arbitrarily selected from the following examples can be appropriately combined within a range where they do not conflict with each other. In the variations exemplified below, for elements whose actions and functions are equivalent to those of the above-described embodiments, the reference numerals used in the above description are reused, and the detailed description of each is appropriately omitted.

[0113] 2.1. Variation 1 In the above-described embodiment, the case where the projector 1 realizes the display method according to the present invention is exemplified, but the present invention is not limited to such a mode. For example, in a multi-projection system including two projectors and an information processing device such as a computer, the display method according to the present invention may be realized. Also, instead of including two projectors and an information processing device, in a system including a projector having the functions of the information processing device and a projector not having the functions of the information processing device, the display method according to the present invention may be realized.

[0114] For example, when the display method according to the present invention is realized in a projection system including a projector and an information processing device having a display panel such as a notebook computer, a message for prompting the user U to attach one or more markers M may be displayed on the display panel. Similarly, the number indicating the number of one or more markers M attached by the user U may be displayed on the display panel. Also, when the projection system realizing the display method according to the present invention has a display panel, the projection system may, by controlling the display panel, superimpose and display an image indicating the position of a point detected from the captured image by image processing on the captured image. Thereby, the user U can appropriately confirm whether a point has been detected from the captured image.

[0115] 2.2. Modification Example 2 In the above-described embodiments and modification examples, the case where the guide image GC2 is an image having a rectangular frame line has been exemplified. However, the present invention is not limited to such a mode. The guide image may be an image having a circular frame line, an image having a rhombus-shaped frame line, or an image having a pentagon-shaped frame line. The guide image may be an image having a polygonal frame line other than a quadrilateral.

[0116] Also, the figure of the guide image does not have to be a figure that continuously surrounds the intersection points of the pointer image. The figure of the guide image may be a figure having a gap in part, such as a so-called Landolt ring.

[0117] 2.3. Modification Example 3 In the above-described embodiments and modification examples, the case where the number NL1-2 indicating the number of markers M attached to the position indicated by the guide image GC2-2 is displayed has been exemplified. However, for example, when a plurality of guide images GC2 are displayed, a number indicating the total number of markers M attached to the positions indicated by the plurality of guide images GC2 may be displayed. The number indicating the total number may be displayed together with the number indicating the number of markers M attached to the position indicated by one of the plurality of guide images GC2.

[0118] 2.4. Modification Example 4 Some or all of the controls executed by the projector 1 may be executed by the processing device 12. Also, some or all of the controls executed by the processing device 12 may be executed by the projector 1.

[0119] 3. Supplementary Note Hereinafter, a summary of the present invention will be described as a supplementary note.

[0120] 3.1. Supplementary Note 1 By controlling the projection device, displaying a projection image on the projection surface, by controlling the projection device, displaying a pointer image located at an angle of the projection image, and based on an operation from the user to move the pointer image, by changing the position of the corner of the projection image, performing a first correction to correct the shape of the projection image, and by controlling the projection device, displaying a guide image indicating the attachment position of one or more markers by the user at a predetermined position on the projection surface based on the position of the pointer image moved based on the operation from the user. A display method including this.

[0121] That is, the user can attach one or more markers at appropriate positions by checking the guide image. Also, when one or more markers are attached at the positions indicated by the guide image, the one or more markers are arranged within a predetermined range from the corner of the projection image indicated by the pointer image corresponding to the guide image. As a result, a projector or projection system that realizes the display method described in Supplementary Note 1 can image one or more attached markers by imaging a display area that is the area where the projection image is displayed. Therefore, the user can have the projector or projection system that realizes the display method described in Supplementary Note 1 acquire an imaged image without finely adjusting the direction in which the imaging device is pointed.

[0122] 3.2. Supplementary Note 2 The pointer image has a first line and a second line, and the guide image has a figure surrounding an intersection point where the first line and the second line intersect. The display method described in Supplementary Note 1.

[0123] That is, a projector or projection system that realizes the display method described in Supplementary Note 2 can more clearly indicate the attachment position of one or more markers. Thereby, the user can attach the markers more accurately.

[0124] 3.3. Supplementary Note 3 Obtaining a captured image representing the result of capturing a range including the display area where the projected image is displayed, and detecting a plurality of feature points that correspond one-to-one to four corners of a first image representing the display area included in the captured image by performing image processing on the captured image, and further including, when there is a corner where the feature points are not detected among the four corners of the first image, the pointer image is located at the corner of the projected image corresponding to the corner where the feature points are not detected, the display method according to Appendix 1 or 2.

[0125] That is, a projector or a projection system that realizes the display method according to Appendix 3 displays a guide image corresponding to a pointer image indicating the position of the corner of the projected image corresponding to the corner where the feature points are not detected among the four corners of the first image representing the display area. Therefore, the user only needs to attach one or more markers only to the necessary places, and can save the trouble of attaching markers more than necessary.

[0126] 3.4. Appendix 4 Displaying the guide image includes displaying a message that prompts the user to attach the one or more markers, the display method according to any one of Appendices 1 to 3.

[0127] Thereby, the user can grasp that the marker should be attached to the position indicated by the guide image by checking the message.

[0128] 3.5. Appendix 5 Displaying the guide image includes displaying a number indicating the number of the one or more markers attached by the user, the display method according to any one of Appendices 1 to 4.

[0129] Thereby, the user can grasp the number of markers to be attached to the position indicated by the guide image by checking the number.

[0130] 3.6. Appendix 6 An information processing apparatus comprising a processing device, the processing device configured to: display a projection image on a projection surface by controlling a projection device; display a pointer image located at a corner of the projection image by controlling the projection device; perform a first correction to correct the shape of the projection image by changing the position of the corner of the projection image based on an operation from a user moving the pointer image; and display a guide image indicating the attachment position of one or more markers by the user at a predetermined position on the projection surface with reference to the position of the pointer image moved based on the operation from the user by controlling the projection device.

[0131] That is, the user can attach one or more markers at appropriate positions by checking the guide image. Also, when one or more markers are attached at the positions indicated by the guide image, the one or more markers are arranged within a predetermined range from the corner of the projection image indicated by the pointer image corresponding to the guide image. As a result, a projection system including the information processing apparatus described in Supplementary Note 6 can image the attached one or more markers by imaging a display area which is an area where the projection image is displayed. Therefore, the user can cause the projection system including the information processing apparatus described in Supplementary Note 6 to acquire an imaged image without finely adjusting the direction in which the imaging device is directed.

[0132] 3.7. Supplementary Note 7 A program for causing a processing device to: display a projection image on a projection surface by controlling a projection device; display a pointer image located at a corner of the projection image by controlling the projection device; perform a first correction to correct the shape of the projection image by changing the position of the corner of the projection image based on an operation from a user moving the pointer image; and display a guide image indicating the attachment position of one or more markers by the user at a predetermined position on the projection surface with reference to the position of the pointer image moved based on the operation from the user by controlling the projection device.

[0133] That is, the user can attach one or more markers at appropriate positions by checking the guide image. Also, when one or more markers are attached at the positions indicated by the guide image, the one or more markers are arranged within a predetermined range from the corners of the projected image indicated by the guide image and the corresponding pointer image. As a result, a projector or a projection system that operates according to the program described in Appendix 7 can image the attached one or more markers by imaging a display area that is an area where the projected image is displayed. Therefore, the user can cause a projector or a projection system that operates according to the program described in Appendix 7 to acquire an imaging image without finely adjusting the direction in which the imaging device is directed.

Explanation of Signs

[0134] 1... Projector, 10... Storage device, 12... Processing device, 14... Imaging device, 16... Projection device, 18... Operating device, 100... Program, 101... Projected image information, 102... Imaging image information, 105... Coordinate information, 106... First coordinate information, 107... Second coordinate information, 120... Projection control unit, 121... Imaging control unit, 122... Detection unit, 123... Correction unit, 124... Coordinate management unit, 140... Image sensor, 142... Imaging lens, 160... Light modulator, 162... Projection lens, GP1... Projected image, GC1... Pointer image, GC2... Guide image, GS1... Imaging image, CN1... Corner, D... Point, L1... Line, T1... Intersection point, R1... Region, NL1-2... Numbers, SW1-2... Messages, W1... Wall surface.

Claims

1. By controlling the projection device, displaying a projection image on a projection surface, By controlling the projection device, a pointer image located at an angle of the projection image is displayed, Based on an operation from a user and for moving the pointer image, By changing the position of the corner of the projection image, performing a first correction to correct the shape of the projection image, executing, By controlling the projection device, at a predetermined position on the projection surface based on the position of the pointer image moved based on an operation from the user, displaying a guide image indicating the pasting position of one or more markers by the user, including, A display method.

2. The pointer image has a first line and a second line, The guide image has a figure surrounding an intersection point where the first line and the second line intersect, The display method according to Claim 1.

3. Obtaining an imaging image representing the result of imaging a range including a display area where the projection image is displayed, By performing image processing on the imaging image, detecting a plurality of feature points that correspond one-to-one to four corners of a first image representing the display area included in the imaging image, further including, When there is a corner among the four corners of the first image where the feature point is not detected, The pointer image is located at the corner of the projection image corresponding to the corner where the feature point is not detected, The display method according to Claim 1 or 2.

4. Displaying the guide image includes displaying a message prompting the user to paste the one or more markers, The display method according to Claim 1.

5. Displaying the guide image includes displaying a number indicating the number of pasted one or more markers by the user, The display method according to Claim 1.

6. Comprising a processing device, The processing device, By controlling the projection device, displaying a projection image on a projection surface, By controlling the projection device, a pointer image located at an angle of the projection image is displayed, Based on an operation from a user and for moving the pointer image, By changing the position of the corner of the projection image, performing a first correction to correct the shape of the projection image, executing, By controlling the projection device, based on an operation from the user, the moved po ​ ​ ​ ​ ​ ​ ​ Displaying a guide image indicating the pasting position of one or more markers by the user at a predetermined position on the projection surface based on the position of the inter-image; and executing, an information processing apparatus.

7. Causing a processing device to display a projection image on a projection surface by controlling a projection device; displaying a pointer image located at a corner of the projection image by controlling the projection device; performing a first correction to correct the shape of the projection image by changing the position of the corner of the projection image based on an operation from the user for moving the pointer image; and displaying a guide image indicating the pasting position of one or more markers by the user at a predetermined position on the projection surface based on the position of the moved pointer image based on an operation from the user by controlling the projection device. Executing, a program. ​ ​ ​ ​

Images