Control method, projection device, and program

Abstract

It accurately detects the edges of the projection surface without using markers or other indicators. [Solution] The projection device 10 includes a projector 110 that projects an image onto a projection surface SC, a processing device 140, and a camera 120. During a first period, the processing device 140 causes the projector 110 to project a first pattern in which the brightness continuously decreases in a first direction, which is the direction from the center outward on the projection surface SC. The processing device 140 also causes the camera 20 to capture images of the projection surface SC and the first pattern projected onto the projection surface SC during the first period, thereby acquiring a first captured image. The processing device 140 then detects a plurality of first feature points in the first captured image that indicate the edges of the projection surface.

Description

Technical Field

[0001] The present disclosure relates to a control method, a projection device, and a program.

Background Art

[0002] Patent Document 1 discloses a technique for maintaining the positional relationship between a projection plane 90 and a projection image PJIM in a preset initial positional relationship without using a marker or the like indicating the position of the projection plane. In the technique disclosed in Patent Document 1, in the first period, a first imaging image IM1 including a first projection image PJIM1 and a projection plane 90 is acquired by an imaging device 141. Then, four first projection points FPP at the four corners of the first projection image PJIM1 and four first feature points Fa10 to Fa13 as feature points of the projection plane 90 are extracted from the first imaging image IM1. In the second period after the first period, a second imaging image IM2 including a second projection image PJIM2 and a projection plane 90 is acquired by the imaging device 141. Then, four second projection points SPP at the four corners of the second projection image PJIM2, the four first feature points Fa10 to Fa13, and four second feature points Fb10 to Fb13 that correspond one-to-one to the four first feature points Fa10 to Fa13 are extracted. The four first projection points FPP, the four first feature points Fa10 to Fa13, the four second projection points SPP, and the four second feature points Fb10 to Fb13 in Patent Document 1 are used, for example, to return the positional relationship between the projection plane 90 and the projection image PJIM to the initial positional relationship before the apparatus main body 19 moves, for example, in the first period.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] To improve the accuracy of extracting four first feature points Fa10 to Fa13 from the first captured image IM1, one might consider, for example, projecting white light onto a region including the edge of the projection surface 90 to increase the brightness near that edge. However, due to the influence of secondary reflection of the white light from objects surrounding the projection surface 90, the accuracy of extracting the four first feature points Fa10 to Fa13 from the first captured image IM1 may actually decrease. [Means for solving the problem]

[0005] One aspect of the control method of the present disclosure is a control method for a projection system comprising a projector that projects an image onto a projection surface, a camera, and one or more processors that control the projector and the camera, characterized in that, in a first period, the one or more processors perform the following: cause the projector to project a first pattern onto the projection surface in which the brightness decreases continuously or in steps in a direction from the center outward of the projection surface; cause the camera to capture the projection surface and the first pattern projected onto the projection surface in the first period to acquire a first captured image; and detect a plurality of first feature points in the first captured image that indicate the edges of the projection surface.

[0006] Another aspect of the present disclosure is a control method for a projection system comprising a projector that projects an image onto a projection surface, a camera, and one or more processors that control the projector and the camera, characterized in that the one or more processors perform the following actions: during a first period, cause the projector to project a first pattern onto the projection surface, the first pattern being an annular pattern located outside the image, wherein the brightness of the first pattern increases continuously or in steps in a direction from the center outward of the projection surface; cause the camera to capture the projection surface and the first pattern projected onto the projection surface during the first period to acquire a first captured image; and detect a plurality of first feature points in the first captured image that indicate the edges of the projection surface.

[0007] One embodiment of the projection apparatus of the present disclosure is a projection apparatus comprising a projector for projecting an image onto a projection surface and one or more processors, wherein the one or more processors perform the following in a first period: cause the projector to project a first pattern onto the projection surface in which the brightness decreases continuously or in steps in a first direction which is a direction from the center outward of the projection surface; obtain a first captured image by having a camera capture the projection surface and the first pattern projected onto the projection surface in the first period; and detect a plurality of first feature points in the first captured image that indicate the edges of the projection surface.

[0008] Another aspect of the present disclosure is a projection apparatus comprising a projector for projecting an image onto a projection surface and one or more processors, wherein the one or more processors perform the following in a first period: cause the projector to project onto the projection surface a first pattern which is an annular first pattern located outside the image and whose brightness increases continuously or in steps in a direction from the center outward of the projection surface; cause a camera to capture the projection surface and the first pattern projected onto the projection surface in the first period to acquire a first captured image; and detect a plurality of first feature points in the first captured image that indicate the edges of the projection surface.

[0009] One aspect of the program of this disclosure is characterized in that a computer that controls a projector that projects an image onto a projection surface and a camera is instructed to perform the following actions during a first period: cause the projector to project a first pattern onto the projection surface in which the brightness decreases continuously or in steps in a first direction which is a direction from the center outward of the projection surface; cause the camera to capture the projection surface and the first pattern projected onto the projection surface during the first period in order to acquire a first captured image; and detect a plurality of first feature points in the first captured image that indicate the edges of the projection surface.

[0010] A program in another aspect of the present disclosure is characterized in that, in a first period, a computer that controls a projector that projects an image onto a projection surface and a camera, causes the projector to project onto the projection surface a first pattern which is an annular first pattern provided outside the image and whose brightness increases continuously or in steps in a direction from the center outward of the projection surface; causes the camera to capture the projection surface and the first pattern projected onto the projection surface in the first period to acquire a first captured image; and detects a plurality of first feature points in the first captured image that indicate the edges of the projection surface. [Brief explanation of the drawing]

[0011] [Figure 1] This figure shows an example configuration of a projection device 10 according to one embodiment of the present disclosure. [Figure 2] This figure shows an example of the installation of the projection surface SC in this embodiment. [Figure 3] This is a diagram illustrating the first pattern PT1 in this embodiment. [Figure 4] This figure shows an example of the brightness gradient in the second part A1 of the first pattern PT1. [Figure 5] This diagram illustrates the influence of reflected light from other surfaces near the projection surface SC. [Figure 6] This flowchart shows the processing flow in the control method executed by the processing unit 140 according to program PR1. [Figure 7] This figure shows an example of the first pattern PT2 projected by the projection device 10 in the second embodiment. [Figure 8] This figure shows an example of the brightness gradient in section A2 of part 2. [Figure 9] This figure shows an example of the brightness gradient in the second part A1 of the first pattern in modified example (1). [Figure 10] This figure shows an example of the first pattern PT33 in modified example (2). [Modes for carrying out the invention]

[0012] The embodiments described below are subject to various technically preferred limitations. However, the embodiments of this disclosure are not limited to those described below. A. First Embodiment Figure 1 shows an example configuration of a projection device 10 according to one embodiment of the present disclosure. The projection device 10 is a device that projects an image onto a projection surface SC. The projection surface SC is installed on a wall W1 of the room where the projection device 10 is located, as shown in Figure 2. As shown in Figure 2, the ceiling TS, floor BS, wall W2 and wall W3 that partition the room are connected to wall W1, and the projection surface SC is installed near the ceiling TS. As shown in Figure 1, the projection device 10 includes a projector 110, a camera 120, a storage device 130, and a processing device 140. Each of the projector 110, camera 120, and storage device 130 is connected to the processing device 140 via a bus (not shown in Figure 1). The camera 120 may be separate from the projection device 10. The number of projection devices 10 may be one or two or more. If there are two or more projection devices 10, multi-projection may be employed, in which the images projected from each projection device 10 are tiled, stacked, or a combination thereof. The following explanation also applies to the case of multi-projection.

[0013] The processing unit 140 includes one or more processors, including, for example, a CPU (Central Processing Unit). The processing unit 140 is an example of a computer in this disclosure. As will be described in detail later, the processing unit 140 functions as the control center of the projection device 10 by operating according to the program PR1 pre-stored in the storage device 130.

[0014] The projector 110 includes a light source (internal light source), a generation unit that generates image light according to image data provided from the processing device 140, and an optical system that guides the image light generated by the generation unit to the projection surface SC. In FIG. 1, the illustration of the generation unit and the optical system is omitted. Specific examples of the generation unit include a drawing panel including a light modulation element such as an LCD (Liquid Crystal Display), an LCOS (Liquid Crystal on Silicon), or a DMD (Digital Mirror Device).

[0015] The camera 120 includes a CMOS or CCD image sensor. In the present embodiment, the imaging area of the camera 120 is preset so as to cover the entire projection surface SC. The camera 120 images the imaging area under the control of the processing device 140. The camera 120 outputs image data representing the captured image to the processing device 140.

[0016] The storage device 130 includes a non-volatile memory such as a flash ROM (Read Only Memory) and a volatile memory such as a RAM (Random Access Memory). A program PR1 for causing the processing device 140 to function as a control center of the projection device 10 is stored in the non-volatile memory of the storage device 130. The non-volatile memory of the storage device 130 is used by the processing device 140 as a work area when executing the program PR1. When the power supply of the projection device 10 (not shown in FIG. 1) is turned on, the processing device 140 reads the program PR1 from the non-volatile memory to the volatile memory and starts executing the program PR1 read into the volatile memory.

[0017] The processing device 140 operating according to the program PR1 functions as a first projection control unit 140a, an imaging control unit 140b, a detection unit 140c, a calculation unit 140d, and a second projection control unit 140e.

[0018] In the first period, the first projection control unit 140a causes the projector 110 to project the first pattern PT1 onto the projection surface. FIG. 3 is a diagram for explaining the first pattern PT1 in the present embodiment. As shown in FIG. 3, the first pattern PT1 in the present embodiment is an image including a first portion A0 and a strip-shaped second portion A1 provided above the first portion A0. The first portion A0 in the first pattern PT1 is all white, and the brightness in the first portion A0 is uniform and constant.

[0019] In the second portion A1, the brightness continuously decreases in a first direction, which is an example of a direction from the center of the projection surface SC toward the outside. In other words, the second portion A1 looks blurred. In FIG. 3, the first direction is indicated by an arrow Y1. In the present embodiment, in the second portion A1, the brightness decreases from K1 to K2 along the first direction, and in FIG. 3, this decrease is shown by drawing the arrow Y1 with a dotted line. FIG. 4 is a graph representing the gradient of the brightness decrease from the position P1 to the position P2 in FIG. 3. In the present embodiment, the ratio of the number of pixels of the second portion A1 to the total number of pixels of the entire first pattern PT1 is about 5 to 10%. The first projection control unit 140a causes the projector 110 to project the first pattern PT1 by drawing pattern data in which about 5 to 10% of the total number of pixels is the second portion A1 and the remaining pixels are the first portion A0 on the drawing panel in the projector 110.

[0020] When trying to detect the corners of the projection surface SC (the four corners of the rectangle corresponding to the contour of the projection surface SC) using a full-white image, if the corners of the projection surface SC are close to other surfaces such as the ceiling TS, as shown in FIG. 5, due to the influence of the reflected light L of the projection light by the other surface, the detection accuracy of the corners close to the other surface may decrease. The full-white image refers to an image projected onto the projection surface SC by outputting white light from the entire drawing panel in the projector 110.

[0021] The first pattern PT1 is used to suppress the influence of reflected light from other surfaces, such as the ceiling T, so that corners can be detected even when the corners of the projection surface SC are in close proximity to other surfaces. In this embodiment, as shown in Figure 2, the projection surface SC is provided on the ceiling TS side of wall W1, and the influence of reflected light from ceiling TS is a concern when detecting corners. For this reason, in this embodiment, the second part A1 is provided above the first part A0, i.e., on the ceiling TS side. If the influence of reflected light from surfaces other than ceiling TS (walls W2, wall W3, and floor surface BS) is a concern, then an image in which an annular second part A1 is provided around the first part A0 may be used as the first pattern PT1.

[0022] Furthermore, the first projection control unit 140a displays the second pattern on the projection surface SC during the first period by stopping the projection of image light by the projector 110 following the projection of the first pattern. In this embodiment, the second pattern is a black image. A black image is an image based on the ambient light in the room where the projection surface SC is installed. Since this ambient light includes light leaking from the projector 110, the second pattern can also be said to be the image projected onto the projection surface SC by the projector 110. As another example of constructing an environment for imaging the projection surface SC onto which the black image is projected, the output of the light source of the projector 110 may be reduced to zero or near zero, or the mechanical shutter of the projector 110 may be closed.

[0023] Furthermore, in the second period following the first period, the first projection control unit 140a causes the projector 110 to project the first pattern onto the projection surface, and subsequently projects the second pattern onto the projection surface SC following the projection of the first pattern.

[0024] The imaging control unit 140b acquires first image data representing the first captured image by having the camera 120 capture the projection surface SC and the first pattern projected onto the projection surface SC during the first period. The imaging control unit 140b also acquires first reference image data representing the first reference image by having the camera 120 capture the projection surface SC and the second pattern projected onto the projection surface SC during the first period.

[0025] Furthermore, the imaging control unit 140b acquires second image data representing the second captured image by having the camera 120 capture the projection surface SC and the first pattern projected onto the projection surface SC during the second period. Also, the imaging control unit 140b acquires second reference image data representing the second reference image by having the camera 120 capture the projection surface SC and the second pattern projected onto the projection surface SC during the second period.

[0026] The detection unit 140c detects a plurality of first feature points, which are markers indicating the corners of the projection surface SC in the first captured image, based on the first image data and the first reference image data. In this embodiment, the set of plurality of first feature points indicates the edges that appear in the first captured image corresponding to the contour lines of the projection surface SC. Each of the plurality of first feature points may be a natural feature point. In this embodiment, the detection unit 140c calculates a difference image representing the difference between the two images by subtracting the pixel value of the pixel corresponding to the pixel in the image represented by the first reference image data for each pixel included in the image represented by the first image data. This process removes ambient noise, and only the effects caused by the projection of the first pattern appear in the difference image. Next, the detection unit 140c obtains information about the corners of the projection surface SC by extracting dark areas from the difference image. An example of a process for extracting dark areas is a minimum value detection process. In this embodiment, the detection unit 140c applies a first-order differential filter and a second-order differential filter to the difference image in the X direction (horizontal scanning direction in the first image) and the Y direction (vertical scanning direction), respectively, and detects a set of pixels where "the result of the first derivative is below a certain threshold" and "the result of the second derivative is above a certain threshold" as pixels corresponding to the corners of the projection plane. For the Y direction, it is sufficient to use the filter that is rotated 90 degrees from the filter used in the X direction.

[0027] According to this embodiment, the corners of the projection surface SC can be detected by a first pattern in which the brightness continuously decreases from the center outward of the projection surface SC. The distribution in which the brightness continuously decreases is the distribution when the first pattern is projected onto the projection surface SC. As a result, the brightness at and near the corners of the projection surface SC becomes relatively darker than at the center of the projection surface SC, and the influence of secondary reflection from surfaces other than the projection surface SC is suppressed. Therefore, a decrease in the detection accuracy of the corners of the projection surface SC during the first period can be suppressed.

[0028] Furthermore, the detection unit 140c detects a plurality of second feature points, which are markers indicating the corners of the projection surface SC in the second captured image, based on the second image data and the second reference image data, using the same algorithm as for detecting the first feature points. In this embodiment, the plurality of second feature points indicate edges that appear in the second captured image corresponding to the contour of the projection surface SC. This suppresses a decrease in the detection accuracy of the corners of the projection surface SC during the second period.

[0029] The calculation unit 140d calculates correction data for correcting the projection image projected onto the projection surface SC in the second period, based on a plurality of first feature points detected by the detection unit 140c and a plurality of second feature points detected by the detection unit 140c. Specifically, the calculation unit 140d calculates data representing a transformation matrix that converts the positions on the drawing panel of the four corners of a rectangle whose outline is represented by a plurality of second feature points to the positions on the drawing panel of the four corners of a rectangle whose outline is represented by a plurality of first feature points, as the above-mentioned correction data. According to this embodiment, since the correction data is calculated based on the detection results of the corners in which the decrease in accuracy is suppressed, the accuracy of correcting the image is improved. The correction data in this embodiment is data for adjusting the position of the projection image on the projection surface SC in the second period to the position of the projection image on the projection surface SC in the first period. According to this embodiment, since the position of the image can be corrected based on the detection results of the corners in which the decrease in accuracy is suppressed, the accuracy of correcting the position of the image is improved.

[0030] The second projection control unit 140e projects the projected image onto the projector 110 while correcting the position relative to the projection surface SC based on the correction data calculated by the calculation unit 140d.

[0031] Furthermore, the processing unit 140, operating according to program PR1, executes a control method that prominently demonstrates the features of this disclosure. Figure 6 is a flowchart showing the processing flow in this control method. The processing in steps SA110, SA120, and SA130 is performed in the first period. The processing from step SA140 onward is performed in the second period.

[0032] In step SA110, the processing unit 140 functions as a first projection control unit 140a and an imaging control unit 140b. In step SA110, the processing unit 140 first causes the projector 110 to project the first pattern PT1 onto the projection surface SC. While the processing unit 140 is projecting the first pattern PT1 onto the projection surface SC, it further functions as an imaging control unit 140b and causes the camera 120 to image the projection surface SC and the first pattern PT1 projected onto the projection surface SC during the first period, thereby acquiring first image data representing the first captured image.

[0033] In step SA120, which is executed following step SA110, the processing unit 140 also functions as a first projection control unit 140a and an imaging control unit 140b. In step SA120, the processing unit 140 first causes the projector 110 to project the second pattern onto the projection surface SC. The processing unit 140, which is projecting the second pattern onto the projection surface, further functions as an imaging control unit 140b, and acquires first reference image data representing the first reference image by causing the camera 120 to capture the projection surface SC and the second pattern projected onto the projection surface SC.

[0034] In step SA130, the processing unit 140 functions as a detection unit 140c. In step SA130, the processing unit 140 detects a plurality of first feature points based on the first image data and the first reference image data.

[0035] In step SA140, the processing unit 140 functions as a first projection control unit 140a and an imaging control unit 140b. In step SA140, the processing unit 140 causes the projector 110 to project the first pattern PT1 onto the projection surface SC. The processing unit 140, which is projecting the first pattern PT1 onto the projection surface SC, further functions as an imaging control unit 140b, and acquires a second image data representing a second captured image by causing the camera 120 to capture the projection surface SC and the first pattern PT1 projected onto the projection surface SC during the second period.

[0036] In step SA150, which follows step SA140, the processing unit 140 causes the projector 110 to project the second pattern onto the projection surface SC. The processing unit 140, which is projecting the second pattern onto the projection surface SC, further functions as an imaging control unit 140b, and acquires a second reference image data representing the second reference image by having the camera 120 capture the projection surface SC and the second pattern projected onto the projection surface SC.

[0037] In step SA160, which follows step SA150, the processing unit 140 functions as a detection unit 140c. In step SA160, the processing unit 140 detects a plurality of second feature points based on the second image data and the second reference image data.

[0038] In step SA170, which follows step SA160, the processing unit 140 functions as a calculation unit 140d. In step SA170, the processing unit 140 calculates correction data to adjust the position of the projection image projected onto the projection surface in the second period relative to the projection surface, based on a plurality of first feature points detected in step SA130 and a plurality of second feature points detected in step SA160, to the position of the projection image projected onto the projection surface in the second period relative to the projection surface relative to the projection surface in the first period.

[0039] In step SA180, which follows step SA170, the processing unit 140 functions as a second projection control unit 140e. In step SA180, the processing unit 140 projects the projection image onto the projector 110 while correcting its position relative to the projection surface SC based on the correction data calculated in step SA170.

[0040] As described above, according to this embodiment, the edge of the projection surface can be detected by a first pattern in which the brightness continuously decreases from the center outward of the projection surface. Therefore, the influence of secondary reflection from surfaces other than the projection surface on the edge and its vicinity is suppressed, and a decrease in the accuracy of edge detection in both the first and second periods can be suppressed. In this embodiment, there is no need for markers or other indicators to show the position of the projection surface, and even when detection at the edge is difficult under conditions where the wall around the edge of the projection surface SC appears to be almost the same color, the edge of the projection surface SC can be detected without markers. Furthermore, in this embodiment, since the edge of the projection surface SC can be detected by projecting two patterns, the first pattern and the second pattern, there is also the effect of shortening the time required for edge detection.

[0041] In addition, according to this embodiment, correction data is calculated based on edge detection results with suppressed accuracy degradation, thus improving the accuracy of image correction. The correction data in this embodiment is data for adjusting the position of the projected image relative to the projection surface in the second period to the position of the projected image relative to the projection surface in the first period, thus improving the accuracy of correcting the image position in the second period.

[0042] B. Second Embodiment Next, a second embodiment of the present disclosure will be described. In the projection device 10 of the second embodiment, the first pattern PT2 projected onto the projection surface SC during the first and second periods differs from the first pattern PT1 in the first embodiment. Figure 7 is a diagram showing an example of the first pattern PT2 projected by the projection device 10 in this embodiment. As is clear from comparing Figure 3 and Figure 7, the first pattern PT2 in this embodiment has a first portion A0 and an annular second portion A2 provided around the first portion A0. At least the first portion A0 is an example of the portion onto which the image is projected, and the second portion A2 is an example of the outside of the image. That is, the annular second portion A2 is located outside the image. Figure 8 is a graph showing how the brightness in the second portion A2 increases from K2 to K1 along the first direction from position P1 to position P2. As shown in Figure 8, in the second portion A2, the brightness increases continuously in the first direction, which is the direction from the center of the projection surface SC outwards. In Figure 7, the arrow Y2, drawn with a dashed line, shows how the brightness continuously increases in the first direction from the center outward of the projection surface SC in the second portion A2. By projecting the first pattern PT2 onto the projection device 10 so that the projection surface SC fits within the first portion A0 of the second pattern in this embodiment, the influence of reflected light from other surfaces located near the corners of the projection surface SC can be mitigated, similar to the embodiment described above.

[0043] In this embodiment as well, by using the first pattern PT2 in which the brightness continuously increases from the center outward of the projection surface, the edges of the projection surface can be detected. This suppresses the influence of secondary reflections from surfaces other than the projection surface on the edges and their vicinity, thereby suppressing a decrease in the accuracy of edge detection in both the first and second periods. In this embodiment as well, correction data is calculated based on the edge detection results in which the decrease in accuracy has been suppressed, thus improving the accuracy of image correction. The correction data in this embodiment is similar to the correction data in the first embodiment, and is data for adjusting the position of the projected image on the projection surface in the second period to the position of the projected image on the projection surface in the first period. Therefore, in this embodiment as well, the accuracy of correcting the position of the image in the second period is improved.

[0044] C. Transformation The above embodiment can be modified as follows. (1) In the second part A1 of the second pattern in the first embodiment described above, the brightness decreased continuously along the first direction from position P1 to position P2, as shown in Figure 4. However, as shown in Figure 9, an image having a second part A1 in which the brightness decreases in steps from K1 to K2 along the first direction from position P1 to position P2 may be used as the first pattern. By using a first pattern in which the brightness decreases in steps from the center of the projection surface outwards, the influence of secondary reflection from surfaces other than the projection surface is suppressed from extending to the corners of the projection surface SC and its vicinity. Similarly, in the second part A2 of the second embodiment, the brightness may increase in steps along the first direction from position P1 to position P2.

[0045] (2) As shown in Figure 10, a first pattern PT3 comprising a second part A1 and a second part A2 may be used, and the first pattern PT3 may be projected so as to sandwich the corner of the projection surface SC into the region A3 between the second part A1 and the second part A2. In this embodiment as well, the influence of secondary reflection from surfaces other than the projection surface (e.g., the ceiling) on ​​the corner of the projection surface and its vicinity is suppressed. The size of region A3 may be adjusted depending on the positional relationship between the projection surface SC and surfaces other than the projection surface SC.

[0046] (3) In the first embodiment described above, the first pattern PT1 was a completely white image with a second portion A2 around it, but it may also be a monochrome image of any of R, G, or B with a second portion A2 around it. By using a first pattern PT1 with a color corresponding to the color of the wall surface of the room where the projection surface is installed (for example, a complementary color), it is expected that the influence of secondary reflection from surfaces other than the projection surface will be further suppressed from extending to the corners of the projection surface SC and its vicinity. The same applies to the first pattern PT2 in the second embodiment. When using a monochrome image of any of R, G, or B with a second portion A2 around it as the first pattern PT1, the detection unit 140c may use different differential filters on the image depending on the color.

[0047] (4) The distribution of the second portion A1 in the first pattern PT1 may be changed. Changing the distribution of the second portion A1 means changing the range (i.e., width) of the second portion A1, or changing the gradient of the brightness change in the second portion A1. The wider the second portion A2 and the gentler the gradient of the brightness change, the wider the margin that can maintain the position of the projected image on the projection surface SC. Conversely, if the gradient of the brightness change is steep, it becomes easier to find the corners of the projection surface SC. Furthermore, the projection surface SC onto which the first pattern PT1 is projected is imaged using the camera 120, and the detection unit 140c may change the differential filter according to at least one of the range and gradient of the distribution of the second portion A1 in the captured image. Similarly, the distribution of the second portion A2 may also be changed in the first pattern PT2 in the second embodiment.

[0048] (5) In the above embodiment, the first projection control unit 140a, the imaging control unit 140b, the detection unit 140c, the calculation unit 140d, and the second projection control unit 140e were software modules. However, at least one of the first projection control unit 140a, the imaging control unit 140b, the detection unit 140c, the calculation unit 140d, and the second projection control unit 140e may be a hardware module such as an ASIC (Application Specific Integrated Circuit). Even if at least one of the first projection control unit 140a, the imaging control unit 140b, the detection unit 140c, the calculation unit 140d, and the second projection control unit 140e is a hardware module, the same effects as in the above embodiment will be achieved.

[0049] (6) In the above embodiment, the first pattern was used to generate correction data to adjust the position of the projected image on the projection surface SC during the second period to the position of the projected image on the projection surface SC during the first period. However, the first pattern may also be used for shape correction to align the shape of the image with the edges of the projection surface SC.

[0050] (7) Program PR1 may be manufactured as a standalone product and may be provided for a fee or free of charge. Specific methods of providing Program PR1 include providing it by writing it to a computer-readable recording medium such as flash ROM, or providing it by downloading it via a telecommunications line such as the Internet.

[0051] D. Summary of this disclosure This disclosure is not limited to the embodiments and modifications described above, and can be implemented in various forms without departing from its spirit. For example, this disclosure can also be implemented in the following forms. The technical features in the embodiments described above that correspond to the technical features in each of the forms described below can be replaced or combined as appropriate in order to solve some or all of the problems of this disclosure or to achieve some or all of the effects of this disclosure. Furthermore, if such technical features are not described as essential in this specification, they can be deleted as appropriate. A summary of this disclosure is provided below.

[0052] (Note 1) One embodiment of the control method of the present disclosure is a control method for a projection system comprising a projector that projects an image onto a projection surface, a camera, and one or more processors that control the projector and the camera, characterized in that the one or more processors perform the following actions: during a first period, cause the projector to project a first pattern onto the projection surface in which the brightness decreases continuously or in steps toward the center of the projection surface outward; cause the camera to capture the projection surface and the first pattern projected onto the projection surface during the first period to acquire a first captured image; and detect a plurality of first feature points in the first captured image that indicate the edges of the projection surface. According to the control method of this embodiment, the edges of the projection surface can be detected by the first pattern in which the brightness decreases continuously or in steps toward the center of the projection surface. As a result, the brightness at and near the edges of the projection surface becomes relatively darker than at the center of the projection surface, and the influence of secondary reflection from surfaces other than the projection surface on the edges and near the edges can be suppressed. Therefore, a decrease in the accuracy of edge detection can be suppressed.

[0053] (Note 2) Another aspect of the present disclosure is a control method for a projection system comprising a projector for projecting an image onto a projection surface, a camera, and one or more processors for controlling the projector and the camera, characterized in that the one or more processors perform the following actions: during a first period, cause the projector to project a first pattern onto the projection surface, the first pattern being annular in shape and having a brightness that increases continuously or gradually in a direction from the center outward of the projection surface; cause the camera to capture the projection surface and the first pattern projected onto the projection surface during the first period to acquire a first captured image; and detect a plurality of first feature points in the first captured image that indicate the edges of the projection surface. According to the control method of this aspect, the edges of the projection surface can be detected by a first pattern being annular in shape and having a brightness that increases continuously or gradually in a direction from the inside outward of the projection surface and having a brightness that increases continuously or gradually in a direction from the inside outward of the projection surface. As a result, the brightness at and near the edges of the projection surface becomes relatively dimmer than at the center of the projection surface, suppressing the influence of secondary reflections from surfaces other than the projection surface on the edges and their vicinity. Therefore, a decrease in the accuracy of edge detection can be suppressed.

[0054] (Note 3) A more preferred control method is the control method described in (Note 1) or (Note 2), wherein one or more processors perform the following actions in a second period following the first period: causing the projector to project the first pattern onto the projection surface; obtaining a second image by having the camera capture the projection surface and the first pattern projected onto the projection surface in the second period; detecting a plurality of second feature points in the second image that indicate the edges of the projection surface; calculating correction data for correcting the image based on the plurality of first feature points and the plurality of second feature points; and causing the projector to project the image corrected based on the correction data onto the projection surface. According to the control method of this embodiment, the image can be corrected based on edge detection results with suppressed degradation of accuracy, thus improving the accuracy of image correction.

[0055] (Note 4) A more preferred embodiment of the control method is the control method described in (Appendix 3), wherein the correction data is data for adjusting the position of the image relative to the projection surface in the second period to the position of the image relative to the projection surface in the first period. According to the control method of this embodiment, the position of the image can be corrected based on edge detection results in which the reduction in accuracy is suppressed, and thus the accuracy of correcting the position of the image is also improved.

[0056] (Note 5) A more preferred embodiment of the control method further includes accepting an operation to adjust the brightness distribution in the first direction, projecting the first pattern onto the projection surface during the first period includes projecting the first pattern adjusted based on the operation onto the projection surface during the first period, and acquiring the first image includes acquiring the first image by having the camera image the projection surface and the first pattern adjusted based on the operation and projected onto the projection surface during the first period, as described in any of (Note 1), (Note 2), (Note 3), or (Note 4). According to this embodiment, for example, a first pattern having a brightness distribution adjusted considering the shape of the projection surface and the installation environment can detect a plurality of first feature points optimized for the shape of the projection surface and the installation environment.

[0057] (Note 6) One embodiment of the projection apparatus of the present disclosure is a projection apparatus comprising a projector for projecting an image onto a projection surface and one or more processors, wherein the one or more processors perform the following in a first period: cause the projector to project a first pattern onto the projection surface in which the brightness decreases continuously or in steps in a first direction which is a direction from the center outward of the projection surface; obtain a first captured image by having a camera capture the projection surface and the first pattern projected onto the projection surface in the first period; and detect a plurality of first feature points in the first captured image that indicate the edges of the projection surface. According to the projection apparatus of this embodiment, the edges of the projection surface can be detected by the first pattern in which the brightness decreases continuously or in steps from the center outward of the projection surface, similar to the control method described in (Appendix 1). As a result, the brightness at the edges of the projection surface and its vicinity becomes relatively darker than at the center of the projection surface, and the influence of secondary reflection from surfaces other than the projection surface on the edges and its vicinity can be suppressed. Therefore, a decrease in the accuracy of edge detection can be suppressed.

[0058] (Note 7) Another embodiment of the present disclosure is a projection apparatus comprising a projector for projecting an image onto a projection surface and one or more processors, wherein the one or more processors perform the following in a first period: cause the projector to project onto the projection surface a first pattern which is an annular first pattern located outside the image and whose brightness increases continuously or in steps in a direction from the center outward of the projection surface; cause a camera to capture the projection surface and the first pattern projected onto the projection surface in the first period to acquire a first captured image; and detect a plurality of first feature points in the first captured image that indicate the edges of the projection surface. According to the projection apparatus of this embodiment, the edges of the projection surface can be detected by a first pattern which is an annular first pattern located outside the image and whose brightness increases continuously or in steps from the inside outward of the projection surface, similar to the control method described in (Appendix 2). As a result, the brightness at and near the edges of the projection surface becomes relatively dimmer than at the center of the projection surface, suppressing the influence of secondary reflections from surfaces other than the projection surface on the edges and their vicinity. Therefore, a decrease in the accuracy of edge detection can be suppressed.

[0059] (Note 8) One aspect of the program of this disclosure is characterized in that a computer that controls a projector that projects an image onto a projection surface and a camera is instructed to perform the following actions during a first period: cause the projector to project a first pattern onto the projection surface in which the brightness decreases continuously or in steps in a first direction that is from the center outward of the projection surface; cause the camera to capture the projection surface and the first pattern projected onto the projection surface during the first period to acquire a first captured image; and detect a plurality of first feature points in the first captured image that indicate the edges of the projection surface. According to the program of this aspect, the edges of the projection surface can be detected by the first pattern in which the brightness decreases continuously or in steps from the center outward of the projection surface, similar to the control method described in (Appendix 1). As a result, the brightness at the edges of the projection surface and its vicinity becomes relatively darker than at the center of the projection surface, and the influence of secondary reflection from surfaces other than the projection surface on the edges and its vicinity can be suppressed. Therefore, a decrease in the accuracy of edge detection can be suppressed.

[0060] (Note 9) A program in another aspect of the present disclosure is characterized in that, in a first period, a computer that controls a projector that projects an image onto a projection surface and a camera, causes the projector to project onto the projection surface a first pattern which is an annular first pattern provided on the outside of the image and in which the brightness increases continuously or in steps in the direction from the center outward of the projection surface; causes the camera to capture the projection surface and the first pattern projected onto the projection surface in the first period to acquire a first captured image; and detects a plurality of first feature points in the first captured image that indicate the edges of the projection surface. According to the program in this aspect, similar to the control method described in (Appendix 2), the edges of the projection surface can be detected by a first pattern which is an annular first pattern provided on the outside of the image and in which the brightness increases continuously or in steps from the inside outward of the projection surface. As a result, the brightness at the edges of the projection surface and its vicinity becomes relatively darker than at the center of the projection surface, and the influence of secondary reflection from surfaces other than the projection surface on the edges and its vicinity can be suppressed. Therefore, it is possible to suppress the decrease in the accuracy of edge detection. [Explanation of symbols]

[0061] 10...Projection device, 110...Projector, 120...Camera, 130...Storage device, 140...Processing device, 140a...First projection control unit, 140b...Imaging control unit, 140c...Detection unit, 140d...Calculation unit, 140e...Second projection control unit, PR1...Program.

Claims

1. A control method for a projection system comprising a projector that projects an image onto a projection surface, a camera, and one or more processors that control the projector and the camera, The one or more processors described above During the first period, the projector is made to project a first pattern onto the projection surface in which the brightness decreases continuously or in steps in the direction from the center of the projection surface outwards. A first image is obtained by having the camera capture the projection surface and the first pattern projected onto the projection surface during the first period. To detect a plurality of first feature points indicating the edge of the projection surface in the first captured image, A control method characterized by performing the following.

2. A control method for a projection system comprising a projector that projects an image onto a projection surface, a camera, and one or more processors that control the projector and the camera, The one or more processors described above During the first period, the projector is made to project a first pattern onto the projection surface, the first pattern being an annular pattern located outside the image, wherein the brightness of the first pattern increases continuously or in steps in the direction from the center outward of the projection surface. A first image is obtained by having the camera capture the projection surface and the first pattern projected onto the projection surface during the first period. To detect a plurality of first feature points indicating the edge of the projection surface in the first captured image, A control method characterized by performing the following.

3. The one or more processors described above In a second period following the first period, the projector is made to project the first pattern onto the projection surface. A second image is obtained by having the camera capture the projection surface and the first pattern projected onto the projection surface during the second period. To detect a plurality of second feature points indicating the edge of the projection surface in the second captured image, Based on the plurality of first feature points and the plurality of second feature points, correction data for correcting the image is calculated. The image corrected based on the correction data is projected onto the projection surface by the projector. A control method according to claim 1 or claim 2, which performs the following:

4. The control method according to claim 3, wherein the correction data is data for adjusting the position of the image relative to the projection surface in the second period to the position of the image relative to the projection surface in the first period.

5. The one or more processors described above The system accepts an operation to adjust the brightness distribution in the first direction, Projecting the first pattern onto the projection surface during the first period includes projecting the first pattern, which has been adjusted based on the operation during the first period, onto the projection surface. The control method according to claim 1 or 2, wherein acquiring the first image image includes having the camera capture the projection surface and the first pattern adjusted based on the operation and projected onto the projection surface during the first period.

6. A projection device comprising a projector for projecting an image onto a projection surface, one or more processors, and a camera, The one or more processors described above are: During the first period, the projector is made to project a first pattern onto the projection surface in which the brightness decreases continuously or in steps in a first direction that is a direction from the center of the projection surface outwards. A first image is obtained by having the camera capture the projection surface and the first pattern projected onto the projection surface during the first period. To detect a plurality of first feature points indicating the edge of the projection surface in the first captured image, A projection device characterized by performing the following.

7. A projection device comprising a projector for projecting an image onto a projection surface, one or more processors, and a camera, The one or more processors described above are: During the first period, the projector is made to project a first pattern onto the projection surface, which is an annular first pattern provided outside the image, wherein the brightness of the first pattern increases continuously or in steps in the direction from the center outward of the projection surface. A first image is obtained by having the camera capture the projection surface and the first pattern projected onto the projection surface during the first period. To detect a plurality of first feature points indicating the edge of the projection surface in the first captured image, A projection device characterized by performing the following.

8. A projector that projects an image onto a projection surface, a camera, and a computer that controls them, During the first period, the projector is made to project a first pattern onto the projection surface in which the brightness decreases continuously or in steps in a first direction that is a direction from the center of the projection surface outwards. A first image is obtained by having the camera capture the projection surface and the first pattern projected onto the projection surface during the first period. To detect a plurality of first feature points indicating the edge of the projection surface in the first captured image, A program characterized by causing the execution of a specific action.

9. A projector that projects an image onto a projection surface, a camera, and a computer that controls them, During the first period, the projector is made to project a first pattern onto the projection surface, which is an annular first pattern provided outside the image, wherein the brightness of the first pattern increases continuously or in steps in the direction from the center outward of the projection surface. A first image is obtained by having the camera capture the projection surface and the first pattern projected onto the projection surface during the first period. To detect a plurality of first feature points indicating the edge of the projection surface in the first captured image, A program characterized by causing the execution of a specific action.

Images