Control method for projection device, projection system and program

Abstract

In a projection device, the normal vector of the target, such as a screen, is appropriately identified. [Solution] A method for controlling a projection device includes, using one or more processors, obtaining a first distance from the projection lens to a first position on the object and a second distance from the projection lens to a second position on the object based on the output from a first sensor; obtaining a gravity vector in a first three-dimensional coordinate system based on the output from a second sensor; identifying a first coordinate of the first position in the first three-dimensional coordinate system based on the first distance; identifying a second coordinate of the second position in the first three-dimensional coordinate system based on the second distance; and identifying a first normal vector of a plane related to the object in the first three-dimensional coordinate system based on the first coordinate, the second coordinate, and the gravity vector.

Description

Technical Field

[0001] The present invention relates to a projection device control method, a projection system, and a program.

Background Art

[0002] Techniques for projecting an image onto a target such as a screen by a projection device are known. Further, there is a known technique of specifying the positional relationship between the target and the projection device and then correcting the image according to the positional relationship. The positional relationship is determined, for example, based on the normal vector of the target. Patent Document 1 describes an example of such a technique.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the conventional techniques have a problem that it may be difficult to appropriately specify the normal vector of a target.

[0005] In the conventional method of specifying the normal vector of an approximate plane, it is necessary to acquire the three-dimensional coordinates of three or more target feature points. Even the method described in Patent Document 1 requires three or more target feature points. However, there are cases where the three-dimensional coordinates of three or more target feature points cannot be acquired. Alternatively, even when the three-dimensional coordinates of three or more target feature points can be acquired, the accuracy of the coordinates may be low. In these cases, it is difficult to appropriately specify the normal vector of the target.

Means for Solving the Problems

[0006] An example of a control method for a projection device according to the present invention is a control method for a projection device that projects an image onto an object via a projection lens, comprising: using one or more processors to obtain a first distance from the projection lens to a first position on the object and a second distance from the projection lens to a second position on the object based on the output from a first sensor; obtaining a gravity vector in a first three-dimensional coordinate system based on the output from a second sensor; identifying a first coordinate of the first position in the first three-dimensional coordinate system based on the first distance; identifying a second coordinate of the second position in the first three-dimensional coordinate system based on the second distance; and identifying a first normal vector of a plane related to the object in the first three-dimensional coordinate system based on the first coordinate, the second coordinate, and the gravity vector.

[0007] An example of a projection system according to the present invention is a projection system that projects an image onto an object via a projection lens, which acquires a first distance from the projection lens to a first position on the object and a second distance from the projection lens to a second position on the object based on the output from a first sensor, acquires a gravity vector in a first three-dimensional coordinate system based on the output from a second sensor, identifies a first coordinate of the first position in the first three-dimensional coordinate system based on the first distance, identifies a second coordinate of the second position in the first three-dimensional coordinate system based on the second distance, and identifies a first normal vector of the plane related to the object in the first three-dimensional coordinate system based on the first coordinate, the second coordinate, and the gravity vector.

[0008] An example of a program according to the present invention is a program for controlling a projection device that projects an image onto an object via a projection lens, the program causing one or more processors to perform the following actions: obtain a first distance from the projection lens to a first position on the object and a second distance from the projection lens to a second position on the object based on the output from a first sensor; obtain a gravity vector in a first three-dimensional coordinate system based on the output from a second sensor; identify a first coordinate of the first position in the first three-dimensional coordinate system based on the first distance; identify a second coordinate of the second position in the first three-dimensional coordinate system based on the second distance; and identify a first normal vector of a plane related to the object in the first three-dimensional coordinate system based on the first coordinate, the second coordinate, and the gravity vector. [Brief explanation of the drawing]

[0009] [Figure 1] An example of using the projection device 10 according to Embodiment 1. [Figure 2] An example configuration of the projection device 10. [Figure 3] An example configuration of projector 11 in Figure 2. [Figure 4] A flowchart illustrating an example of the operation of the projection device 10 in Figure 1. [Figure 5] Figure 4 illustrates the details of step S1. [Figure 6] A diagram illustrating the details of step S5 in Figure 4. [Figure 7] An example configuration of the distance sensor 14 according to Embodiment 2. [Figure 8] A flowchart illustrating an example of the process in step S1 according to Embodiment 2. [Figure 9] An example of using the projection device 10 according to Embodiment 3. [Figure 10] A flowchart illustrating an example of the process in step S1 according to Embodiment 3. [Figure 11] Example of an captured image. [Figure 12] A flowchart illustrating an example of the operation of the projection device 10 according to Embodiment 4. [Figure 13] A diagram explaining the details of step S31 in FIG. 12. [Figure 14] A diagram explaining the details of step S35 in FIG. 12. [Figure 15] A diagram illustrating the process of step S36 in FIG. 12. [Figure 16] A flowchart showing a process example of step S31 in Embodiment 5. [Description of Embodiments]

[0010] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. [Embodiment 1] FIG. 1 shows a usage example of a projection apparatus 10 according to Embodiment 1. The projection apparatus 10 projects a projection image 30 onto a screen 20 as an image. FIG. 1 shows coordinate axes of a three-dimensional coordinate system. The three-dimensional coordinate system is represented using three axes, namely the X-axis, Y-axis, and Z-axis. The X-axis, Y-axis, and Z-axis are mutually orthogonal. In FIG. 1, the paper surface is parallel to the XZ plane. Also in FIG. 1, the Z-axis is parallel to the optical axis 40 of the projection apparatus 10.

[0011] The projection apparatus 10 projects an image onto a target. In the present embodiment, the screen 20 is used as an example of the target. The screen 20 includes a flat surface. Preferably, the entire one side of the screen 20 substantially constitutes a single flat surface. Alternatively, the screen 20 has a shape that can be approximated as a flat surface by using an approximate flat surface. In that case, the approximate flat surface can be defined as the flat surface of the screen 20.

[0012] The screen 20 is installed parallel to the direction of gravity. That is, the flat surface of the screen 20 is installed parallel to the direction of gravity. Note that while the Y-axis is parallel to the direction of gravity in FIG. 1, the Y-axis does not have to be parallel to the direction of gravity. This will be described later in relation to the definition of the coordinate system.

[0013] Note that as a modified example, the target may be an object other than a screen. Also, the target may include a plurality of flat surfaces. Also, the target may include a curved surface. Also, the target may be an object that does not include a flat surface, that is, an object constituted only by a curved surface.

[0014] In Figure 1, a projected image 30 is projected onto a screen 20. Since the screen 20 is installed parallel to the Y-axis as described above, the projected image 30 is represented as a straight line in Figure 1.

[0015] The screen 20 is ideally arranged in a state orthogonal to the Z-axis. However, in practice, as shown in Figure 1, the screen 20 may be arranged in a state not orthogonal to the Z-axis. In the example of Figure 1, the angle θ formed by the screen 20 and the Z-axis is less than 90 degrees.

[0016] Figure 2 shows a configuration example of the projection device 10. The projection device 10 includes a projector 11, a processor 12, a storage device 13, a distance sensor 14, and an acceleration sensor 15. The distance sensor 14 is a first sensor. The acceleration sensor 15 is a second sensor.

[0017] Figure 3 shows a configuration example of the projector 11. The projector 11 includes a light source 111, a liquid crystal panel 112, a prism 113, a mirror 114, and a projection lens 115. The light source 111 emits light for projecting the projected image 30. The liquid crystal panel 112 is an example of a light modulation device. The liquid crystal panel 112 modulates the light emitted from the light source 111. The prism 113 and the mirror 114 control the optical path of the light emitted from the light source 111. The projection lens 115 forms the projected image 30 by refracting the modulated light. The optical axis of the projection lens 115 is the optical axis 40 shown in Figure 1.

[0018] In this manner, the projection device 10 projects the projected image 30 onto the screen 20 via the projection lens 115.

[0019] Let's return to the explanation in Figure 2. The processor 12 functions as a means of calculation. The processor 12 can be manufactured using an integrated circuit, ASIC, FPGA, etc. The processor 12 controls the operation of the entire projection device 10. Note that only one processor 12 is shown in Figure 2. However, the projection device 10 may have multiple processors 12.

[0020] The storage device 13 stores information. The storage device 13 includes, for example, a storage medium such as a semiconductor memory device or a magnetic disk device. Some or all of the storage medium may be a non-transitory storage medium. The storage device 13 stores the program PG1.

[0021] The processor 12 operates by reading and executing program PG1 from the storage device 13. Program PG1 is a program that controls the projection device 10. For example, the processor 12 controls the projection device 10 by executing program PG1. In other words, program PG1 causes the processor 12 to control the projection device 10.

[0022] The distance sensor 14 is a sensor for measuring distance. The distance sensor 14 can measure the distance to at least two points on an object. The distance sensor 14 can be any known distance sensor. For example, a light-based sensor such as a ToF (Time Of Flight) sensor may be used. Alternatively, a sensor that uses ultrasound may be used.

[0023] The accelerometer 15 is a sensor for measuring acceleration. The output of the accelerometer 15 includes information that can identify the direction of acceleration. For example, the accelerometer 15 measures acceleration along three orthogonal axes. That is, the accelerometer 15 outputs an acceleration vector. Here, for example, if the projection device 10 is stationary, the accelerometer 15 can be said to output a gravity vector. The gravity vector is a vector that represents the direction and magnitude of gravity.

[0024] Furthermore, the projection device 10 may include a communication device (not shown). The communication device may include a network interface. The communication device may perform wired communication. The communication device may perform wireless communication.

[0025] The projector 11, the distance sensor 14, and the acceleration sensor 15 are each calibrated to a single common three-dimensional coordinate system. This common three-dimensional coordinate system is the first three-dimensional coordinate system.

[0026] For example, the processor 12 can identify the area in space where the projected image 30 is projected by the projector 11 in a first three-dimensional coordinate system. For example, if the projected image 30 is rectangular, a square pyramid can be identified in the first three-dimensional coordinate system. Also, for example, the processor 12 can identify the coordinates of a point in space in the first three-dimensional coordinate system based on the output from the distance sensor 14 corresponding to that point. Also, for example, the processor 12 can identify the direction of gravity in the first three-dimensional coordinate system based on the output from the accelerometer 15. Alternatively, the processor 12 can identify the gravity vector in the first three-dimensional coordinate system based on the output from the accelerometer 15.

[0027] In this embodiment, the first three-dimensional coordinate system is defined as a three-dimensional coordinate system that satisfies both of the following conditions (1) and (2). (1) The origin of the first three-dimensional coordinate system is the position of the principal point of the projection lens 115 of the projection device 10. (2) The Z-axis of the first three-dimensional coordinate system is parallel to the optical axis 40 of the projection lens 115 of the projection device 10. The Y-axis of the first three-dimensional coordinate system can be taken in any direction within the plane perpendicular to the Z-axis. Generally, it is preferable to take the Y-axis so that the positive direction coincides with the upward direction when the projection device 10 is normally installed, as this makes it easier for humans to understand the coordinate system.

[0028] Using such a coordinate system simplifies coordinate calculations. For example, the amount of computation is reduced when calculating the coordinates of each point on the screen 20 where the projected image 30 is projected. However, the above coordinate system is just one example. The first three-dimensional coordinate system can be defined arbitrarily. For example, even if the origin of the coordinate system is a position other than the principal point of the projection lens 115 of the projection device 10, it is possible to calibrate the projector 11, the distance sensor 14, and the acceleration sensor 15 to match such a coordinate system. Similarly, even if the Z-axis and the optical axis 40 of the projection lens 115 are not parallel in the coordinate system, it is possible to calibrate the projector 11, the distance sensor 14, and the acceleration sensor 15 to match such a coordinate system.

[0029] Figure 4 is a flowchart illustrating an example of the operation of the projection device 10 according to this embodiment. This flowchart represents the control method of the projection device 10. The control method of the projection device 10 is executed by the processor 12. The processor 12 may execute this control method by executing the program PG1. That is, the program PG1 may be a program that causes the processor 12 to execute the control method of the projection device 10.

[0030] In this embodiment, this control method is performed by a single processor 12. However, in a modified example, this control method may be performed by a plurality of processors 12.

[0031] The process shown in Figure 4 is initiated, for example, when a user of the projection device 10 performs an operation to cause the projection device 10 to determine the orientation of the screen 20. Once the process shown in Figure 4 is initiated, in step S1, the processor 12 obtains the first distance and the second distance based on the output from the distance sensor 14.

[0032] Step S1 will be explained in detail using Figure 5. The first distance L1 is the distance from the projection lens 115 to point P1 on the screen 20. Point P1 represents the first position. The position of the projection lens 115 is represented, for example, by the position of the principal point of the projection lens 115. The second distance L2 is the distance from the projection lens 115 to point P2 on the screen 20. Point P2 represents the second position. The first position and the second position are different from each other.

[0033] Next, in step S2, the processor 12 obtains the gravity vector VG in the first three-dimensional coordinate system based on the output from the accelerometer 15.

[0034] Next, in step S3, the processor 12 determines the first and second coordinates based on the first distance L1 and the second distance L2. The first coordinate is the coordinate of point P1 in the first three-dimensional coordinate system. The second coordinate is the coordinate of point P2 in the first three-dimensional coordinate system.

[0035] Next, in step S4, the processor 12 identifies a direction vector V12 with the coordinates of point P1 and the coordinates of point P2 as its endpoints. In the example in Figure 5, point P1 is the starting point of the direction vector V12, and point P2 is the ending point of the direction vector V12. The starting and ending points of the direction vector V12 may be swapped.

[0036] Next, in step S5, the processor 12 determines a first normal vector based on the direction vector V12 and the gravity vector VG.

[0037] Step S5 will be explained in detail using Figure 6. The first normal vector N1 is the normal vector of the screen 20 in the first three-dimensional coordinate system. Since the screen 20 is installed parallel to the direction of gravity as described above, the first normal vector N1 can be identified as a vector orthogonal to both the direction vector V12 and the gravity vector VG. For example, the first normal vector N1 is calculated as the cross product of the direction vector V12 and the gravity vector VG.

[0038] Furthermore, to be more precise, including cases where screen 20 contains multiple planes, or where screen 20 contains a curved surface, etc., the "normal vector of screen 20" can be said to be the normal vector of the plane associated with screen 20. The first normal vector N1 is the normal vector of the plane associated with screen 20. Here, the plane associated with screen 20 is a plane that contains points P1 and P2 and is parallel to the gravity vector VG.

[0039] Here, the direction vector V12 is determined in step S4 based on the coordinates of point P1 and point P2. Therefore, in step S5, the first normal vector N1 is determined based on the coordinates of point P1, the coordinates of point P2, and the gravity vector VG.

[0040] As described above, in this embodiment, the first normal vector N1 is calculated using the direction vector V12. This calculation method reduces the amount of computation performed by the processor 12. For example, the first normal vector N1 can be calculated using only the coordinate subtraction when calculating the direction vector V12 and the multiplication between vector elements when calculating the first normal vector N1. However, as a modified example, it is also possible to determine the first normal vector N1 without using the direction vector V12. For example, one can first determine a straight line passing through point P1 and parallel to the gravity vector VG, then determine a plane containing that straight line and point P2, and finally determine the normal vector of that plane.

[0041] In this way, the first normal vector N1 of the plane associated with the screen 20 is determined. The processor 12 controls the projection device 10 based on the determined first normal vector N1.

[0042] As an example of controlling the projection device 10 based on the first normal vector N1, steps S6 and S7 can be performed. In step S6, the processor 12 corrects the original image based on the first normal vector N1. As an example of correction, so-called trapezoidal correction is performed. This will be explained below.

[0043] As shown in Figure 1, consider the case where the plane of the screen 20 and the Z-axis direction are not orthogonal. In this case, the optical axis 40 of the projection device 10 and the first normal vector N1 are not parallel. Therefore, if the original image formed inside the projection device 10 is rectangular, the projected image 30 projected onto the screen 20 will be a quadrilateral, not a rectangle. Considering this, in order to make the projected image 30 rectangular, it is necessary to convert the original image, which is originally rectangular, into another appropriate quadrilateral. This correction is sometimes called trapezoidal correction.

[0044] The specific process for trapezoidal correction can be designed appropriately by a person skilled in the art based on prior art. An example is described below. Trapezoidal correction is achieved, for example, by transforming the two-dimensional coordinates of each pixel of the original image according to a specific transformation formula. The specific transformation formula can be designed appropriately by a person skilled in the art. For example, first, the vertices of the projection rectangle, which is formed by projecting the original rectangle that forms the original image onto a plane parallel to the plane of the screen 20, starting from the principal point of the projection lens, are calculated. Then, a transformation formula is identified from the coordinates of each vertex of the projection rectangle in the two-dimensional coordinate system of the plane containing the projection rectangle to the coordinates of each vertex of the original rectangle in the two-dimensional coordinate system of the plane containing the original rectangle. This transformation formula is a formula that transforms the two-dimensional coordinates of each vertex of the projection rectangle to the two-dimensional coordinates of each vertex of the original rectangle. By transforming the two-dimensional coordinates of each pixel of the original image according to this transformation formula, the projected image 30 can be made into a rectangle.

[0045] In this way, the original image is corrected. Next, in step S7, the processor 12 projects the corrected original image through the projection lens 115. As a result, the projected image 30 is projected onto the screen 20 through the projection lens 115.

[0046] As described above, according to the control method of the projection device 10 in this embodiment, by utilizing the gravity vector VG, it is possible to more easily determine the appropriate first normal vector N1 of the screen 20.

[0047] For example, in this embodiment, it is sufficient to obtain the coordinates of two points, point P1 and point P2, as the coordinates of a point on the screen 20. Since it is not necessary to obtain the coordinates of three or more points, the processing becomes simpler.

[0048] Furthermore, conventional techniques using three or more points often require three points that are not on the same straight line. In such cases, a distance sensor capable of measuring distance along a surface rather than a line is required, which increases costs. In this embodiment, since only the coordinates of two points need to be obtained, a distance sensor capable of measuring distance along a line rather than a surface can be used. Therefore, there is a possibility of reducing costs.

[0049] Furthermore, in conventional techniques, when using a large number of coordinates, it is not always possible to obtain all coordinates with high accuracy. With this control method, if at least two high-accuracy coordinates can be obtained, the first normal vector N1 can be identified with high accuracy.

[0050] The control of the projection device 10 based on the first normal vector N1 is not limited to that shown in steps S6 and S7. For example, the projection device 10 may output the value of the first normal vector N1. Also, for example, the projection device 10 may output information prompting the user to adjust the orientation of the screen 20 so that the orientation of the first normal vector N1 is appropriate. Also, for example, the projection device 10 may output a warning if the orientation of the first normal vector N1 is outside the acceptable range.

[0051] [Embodiment 2] Embodiment 2 is an embodiment in which the configuration of the distance sensor 14 is further specified in Embodiment 1. Embodiment 2 will be described below. Parts common to Embodiment 1 may be omitted from the description.

[0052] Figure 7 shows an example configuration of the distance sensor 14 according to Embodiment 2. The distance sensor 14 comprises at least one light source 141 and three or more light-receiving elements 142. In Figure 7, examples of light-receiving elements 142 are shown as light-receiving element 142-1, light-receiving element 142-2, light-receiving element 142-3, and light-receiving element 142-4. Although four light-receiving elements are shown in Figure 7, as described above, in this embodiment, three or more light-receiving elements are sufficient.

[0053] The light source 141 emits incident light LI that enters the screen 20. Each light-receiving element 142 receives reflected light LR that is reflected from a specific position on the screen 20 corresponding to the light-receiving element 142. In other words, three or more light-receiving elements 142 each receive reflected light LR that is reflected from three or more positions on the screen 20 that are reflected from the incident light LI.

[0054] Each light-receiving element 142 outputs distance and intensity values. For example, light-receiving element 142-1 outputs distance D1 and intensity value W1. Light-receiving element 142-2 outputs distance D2 and intensity value W2. Light-receiving element 142-3 outputs distance D3 and intensity value W3. Light-receiving element 142-4 outputs distance D4 and intensity value W4.

[0055] Figure 8 is a flowchart showing an example of the process in step S1 according to this embodiment. In this embodiment, step S1 includes steps S11 to S13 shown in Figure 8.

[0056] In step S1, specifically in step S11, the processor 12 acquires three or more intensity values ​​W1, W2, W3, W4, ... Here, the processor 12 acquires these intensity values ​​based on the output from each of the three or more photodetectors 142. Each of these intensity values ​​W1, W2, W3, W4, ... represents the intensity of each reflected light LR. Furthermore, each of these intensity values ​​W1, W2, W3, W4, ... corresponds one-to-one with the three or more photodetectors 142-1, 142-2, 142-3, 142-4, ...

[0057] Next, in step S12, the processor 12 identifies two intensity values ​​from among three or more intensity values ​​W1, W2, W3, W4, ... that satisfy the first criterion, and two corresponding photodetectors. The first criterion can be arbitrarily defined as a criterion for selecting a photodetector suitable for use in distance measurement.

[0058] For example, the first criterion is that the intensity value is greater than or equal to a predefined threshold. Alternatively, for example, the first criterion is that the intensity value is within the second largest of all intensity values.

[0059] In the following, the two photodetectors identified in step S12 will be referred to as the first photodetector and the second photodetector. Next, in step S13, the processor 12 acquires the first distance L1 and the second distance L2. The first distance L1 is the distance acquired based on the output from the first photodetector. The second distance L2 is the distance acquired based on the output from the second photodetector.

[0060] The processing after step S13 may be performed in the same manner as steps S2 and later in Embodiment 1.

[0061] According to Embodiment 2 described above, two photodetectors exhibiting particularly large intensity values ​​are selected from among three or more photodetectors 142-1, 142-2, 142-3, 142-4, ... Here, if the intensity value of a certain photodetector is large, it is considered that the distance measurement accuracy by that photodetector is high. Therefore, a decrease in the measurement accuracy of the first distance L1 and the second distance L2 can be suppressed. Consequently, a decrease in the accuracy of the direction vector V12 can be suppressed.

[0062] [Embodiment 3] Embodiment 3 is an embodiment in which the method of acquiring distance is changed in Embodiment 1 or 2. Embodiment 3 will be described below. Parts common to Embodiment 1 or 2 may be omitted from the description.

[0063] Figure 9 shows an example of the use of the projection device 10 according to Embodiment 3. The projection device 10 includes a camera 16. The camera 16 is an imaging device that captures images. The camera 16 is a third sensor. The camera 16 is positioned so that it can capture the screen 20 as the target of imaging. Alternatively, the camera 16 is positioned so that it can capture an area of ​​the screen 20 on which the projected image 30 may be projected as the target of imaging. The camera 16 has three or more imaging pixels. Preferably, the camera 16 has a large number of imaging pixels arranged in a grid.

[0064] Camera 16 is calibrated to a first three-dimensional coordinate system. For example, the processor 12 can determine the coordinate in the first three-dimensional coordinate system for each pixel of the image captured by camera 16. Alternatively, for example, the processor 12 can determine a straight line in the first three-dimensional coordinate system for each pixel of the image captured by camera 16.

[0065] In this embodiment, the distance sensor 14 comprises at least one light source 141 and three or more light-receiving elements 142, similar to Embodiment 2. Each light-receiving element 142 outputs a distance. Unlike Embodiment 2, it is not necessary for each light-receiving element 142 to output an intensity.

[0066] Figure 10 is a flowchart showing an example of the process of step S1 according to Embodiment 3. In this embodiment, step S1 includes steps S21 to S25 shown in Figure 10.

[0067] In step S1, specifically in step S21, the processor 12 acquires the captured image taken by the camera 16. This captured image may be, for example, an image of the screen 20. Alternatively, this captured image may be an image of the area of ​​the screen 20 on which the projected image 30 may be projected.

[0068] Next, in step S22, the processor 12 determines, based on the captured image, whether or not an object different from the screen 20 exists in the space from the projection lens 115 to the screen 20. Hereinafter, an object different from the screen 20 will simply be referred to as "the object".

[0069] Figure 11 shows an example of an captured image. An example of the processing in step S22 will be explained using the example in Figure 11. In the example in Figure 11, it is determined that an object exists. That is, in the captured image, the region corresponding to the screen 20 includes a first region R1 in which no object exists and a second region R2 in which an object exists. In the example in Figure 11, the second region R2 is shown by hatching.

[0070] Here, specific criteria and processing methods for determining whether or not an object exists can be appropriately designed by a person skilled in the art based on prior art. For example, the determination may be based on the color of the pixels. For example, if there are pixels whose brightness is below a certain threshold, it may be determined that an object exists at the location of those pixels. Alternatively, edge detection may be performed on the captured image. If an edge is detected within a region corresponding to the inside of the screen 20, it may be determined that an object exists in the darker of the two regions flanking the edge.

[0071] The example in Figure 11 shows a case where the distance sensor 14 has eight light-receiving elements. Points P11 to P18 are shown as points corresponding to the eight light-receiving elements. In addition, among the imaging pixels of the camera 16, imaging pixels C11 to C18 are shown as imaging pixels corresponding to the eight light-receiving elements.

[0072] In the example shown in Figure 11, there is a one-to-one correspondence between points P11-P18 and image pixels C11-C18. However, in the modified example, there is no one-to-one correspondence between points P11-P18 and image pixels C11-C18. For example, point P11 may correspond to two image pixels. Or, for example, points P11 and P12 may correspond to the same image pixel.

[0073] If the processor determines in step S22 that an object exists, it executes steps S23 and S24. In step S23, the processor identifies two imaging pixels from among multiple imaging pixels included in the first region R1 where no object exists. In the example in Figure 11, two imaging pixels are identified from among five imaging pixels C11, C12, C13, C14, and C18 included in the first region R1. Here, the criteria for identifying two imaging pixels when the first region R1 contains three or more imaging pixels, as in Figure 11, can be arbitrarily designed. The two imaging pixels identified in step S23 are designated as the first imaging pixel and the second imaging pixel.

[0074] After step S23, in step S24, the processor 12 identifies two light-receiving elements. In particular, the processor 12 identifies two light-receiving elements based on the correspondence between three or more imaging pixels of the camera 16 and three or more light-receiving elements. For example, a first light-receiving element corresponding to a first imaging pixel and a second light-receiving element corresponding to a second imaging pixel are identified.

[0075] After step S24, in step S25, the processor 12 obtains a first distance L1 and a second distance L2. The first distance L1 is the distance obtained based on the output from the first photodetector. The second distance L2 is the distance obtained based on the output from the second photodetector.

[0076] If, in step S22 described above, it is determined that no object exists, the processor 12 proceeds to step S25. In this case, the first distance L1 and the second distance L2 can be obtained based on the outputs from any two photodetectors.

[0077] The processing after step S25 may be performed in the same manner as steps S2 and later in Embodiment 1.

[0078] According to Embodiment 3 described above, the distances in areas where no objects exist can be used as the first distance L1 and the second distance L2. Therefore, the distance to the screen 20 is measured more accurately, and a decrease in the accuracy of the direction vector V12 can be suppressed. In particular, even if the photodetector 142 does not output information indicating the reliability of the distance measurement result, a decrease in the accuracy of the direction vector V12 can be suppressed.

[0079] [Embodiment 4] Embodiment 4 is an embodiment in which the method for specifying multiple normal vectors is modified in any of Embodiments 1 to 3. Embodiment 4 will be described below. Parts common to any of Embodiments 1 to 3 may be omitted from the description.

[0080] Figure 12 is a flowchart illustrating an example of the operation of the projection device 10 according to Embodiment 4. This flowchart corresponds to the flowchart in Figure 4 in Embodiment 1.

[0081] In step S31, the processor 12 obtains the first distance, second distance, and third distance based on the output from the distance sensor 14.

[0082] Step S31 will be explained in detail using Figure 13. The first distance L1 and the second distance L2 are obtained in the same manner as in Embodiment 1. The third distance L3 is the distance from the projection lens 115 to point P3 on the screen 20. Point P3 represents a third position. The third position is a different position from the first and second positions. Also, the third position is not on the straight line passing through the first and second positions.

[0083] Next, in step S32, the processor 12 obtains the gravity vector VG in the first three-dimensional coordinate system based on the output from the accelerometer 15.

[0084] Next, in step S33, the processor 12 determines the first, second, and third coordinates based on the first distance L1, the second distance L2, and the third distance L3. In particular, the third coordinate is determined based on the third distance L3. The third coordinate is the coordinate of point P3 in the first three-dimensional coordinate system.

[0085] Next, in step S34, the processor 12 identifies direction vectors V12, V23, and V31. Direction vector V12 is the first direction vector. Direction vector V23 is the second direction vector. Direction vector V31 is the third direction vector.

[0086] Direction vector V12 is a direction vector with the coordinates of point P1 and point P2 as its endpoints. Direction vector V23 is a direction vector with the coordinates of point P2 and point P3 as its endpoints. Direction vector V31 is a direction vector with the coordinates of point P3 and point P1 as its endpoints.

[0087] In the example in Figure 13, point P1 is the starting point of direction vector V12, and point P2 is the ending point of direction vector V12. In the example in Figure 13, point P2 is the starting point of direction vector V23, and point P3 is the ending point of direction vector V23. In the example in Figure 13, point P3 is the starting point of direction vector V31, and point P1 is the ending point of direction vector V31. The starting and ending points of each direction vector can be swapped.

[0088] Next, in step S35, the processor 12 identifies three normal vectors.

[0089] Step S35 will be explained in detail using Figure 14. The first normal vector N1 is determined based on the direction vector V12 and the gravity vector VG. The second normal vector N2 is determined based on the direction vector V23 and the gravity vector VG. The third normal vector N3 is determined based on the direction vector V31 and the gravity vector VG.

[0090] The first normal vector N1, the second normal vector N2, and the third normal vector N3 are all normal vectors of the plane associated with the screen 20. Ideally, all of these normal vectors would have the same direction. However, if the screen 20 is not a perfect plane, these normal vectors may be normal vectors of different planes associated with the screen 20. Also, if there are errors in the distance measurement, the directions of these normal vectors may not coincide.

[0091] Here, similar to Embodiment 1, the direction vector V12 is determined in step S34 based on the coordinates of point P1 and point P2. Therefore, in step S35, the first normal vector N1 is determined based on the coordinates of point P1, the coordinates of point P2 and the gravity vector VG. Similarly, the second normal vector N2 is determined based on the coordinates of point P2, the coordinates of point P3 and the gravity vector VG. Furthermore, the third normal vector N3 is determined based on the coordinates of point P3, the coordinates of point P1 and the gravity vector VG.

[0092] Next, in step S36, the processor 12 determines the fourth normal vector N4 based on the first normal vector N1, the second normal vector N2, and the third normal vector N3. Figure 15 illustrates this process. For example, the fourth normal vector N4 is calculated as the average of the first normal vector N1, the second normal vector N2, and the third normal vector N3.

[0093] In this way, the fourth normal vector N4 of the plane associated with the screen 20 is identified. The processor 12 controls the projection device 10 based on the identified fourth normal vector N4.

[0094] As an example of controlling the projection device 10 based on the fourth normal vector N4, steps S37 and S38 can be performed. Step S37 may be performed in the same manner as step S6 of Embodiment 1. However, in this embodiment, the fourth normal vector N4 is used in place of the first normal vector N1 in step S37. In this way, the original image is corrected.

[0095] Step S38 may be performed in the same manner as step S7 of Embodiment 1. This causes the projected image 30 to be projected onto the screen 20 via the projection lens 115.

[0096] According to Embodiment 4 described above, the final normal vector is determined based on the three normal vectors, thus increasing the accuracy of the calculation of the final normal vector.

[0097] In Embodiment 4, three normal vectors are identified based on three points. As a variation, four or more normal vectors may be identified based on four or more points. Doing so further improves the accuracy of the final normal vector calculation.

[0098] [Embodiment 5] Embodiment 5 is an embodiment that combines Embodiment 3 and Embodiment 4. Embodiment 5 will be described below. Parts common to Embodiment 3 or 4 may be omitted from the description.

[0099] The distance sensor 14 according to Embodiment 5 comprises four or more light-receiving elements 142. In the example shown in Figure 7, four light-receiving elements 142 are shown. The four or more light-receiving elements 142 each receive reflected light LR that is reflected from four or more positions on the screen 20 by the incident light LI.

[0100] Furthermore, the camera 16 according to Embodiment 5 has four or more imaging pixels. In the example in Figure 11, the camera 16 has at least eight imaging pixels. Also, Embodiment 5 assumes that the first region R1 includes three or more imaging pixels. In the example in Figure 11, the first region R1 includes five imaging pixels.

[0101] An example of the operation of the projection device 10 according to Embodiment 5 is shown by changing the process of step S31 in the flowchart of Figure 12 according to Embodiment 4 to the process shown in Figure 16.

[0102] Figure 16 is a flowchart showing an example of the processing of step S31 in Embodiment 5. In this embodiment, step S31 includes the steps shown in Figure 16. Steps S21 and S22 may be performed in the same manner as shown in Figure 10 relating to Embodiment 3. For example, in step S21, the processor 12 acquires the captured image captured by the camera 16. Next, in step S22, the processor 12 determines whether or not an object other than the screen 20 exists.

[0103] If the processor determines in step S22 that an object exists, it executes steps S23a and S24a. In step S23a, the processor identifies three imaging pixels from among the three or more imaging pixels contained in the first region R1 in which no object exists. In the example in Figure 11, three imaging pixels are identified from among the five imaging pixels C11, C12, C13, C14, and C18 that constitute the first region R1. Here, the criteria for identifying three imaging pixels when the first region R1 contains four or more imaging pixels, as in Figure 11, can be arbitrarily designed. The three identified imaging pixels are referred to as the first imaging pixel, the second imaging pixel, and the third imaging pixel.

[0104] After step S23a, in step S24a, the processor 12 identifies three light-receiving elements. In particular, the processor 12 identifies three light-receiving elements based on the correspondence between four or more imaging pixels of the camera 16 and four or more light-receiving elements. For example, a first light-receiving element corresponding to a first imaging pixel, a second light-receiving element corresponding to a second imaging pixel, and a third light-receiving element corresponding to a third imaging pixel are identified.

[0105] After step S24a, in step S25a, the processor 12 acquires a first distance L1, a second distance L2, and a third distance L3. The first distance L1 is the distance acquired based on the output from the first photodetector. The second distance L2 is the distance acquired based on the output from the second photodetector. The third distance L3 is the distance acquired based on the output from the third photodetector.

[0106] If, in step S22 described above, it is determined that no object exists, the processor 12 proceeds to step S25a. In this case, the first distance L1, the second distance L2, and the third distance L3 can be obtained based on the outputs from any three photodetectors.

[0107] The processing after step S25a may be performed in the same manner as in step S32 and later of Embodiment 4.

[0108] According to Embodiment 5 described above, it is possible to obtain both the effects of Embodiment 3 and Embodiment 4. For example, since the distance in a region where no object exists can be used, the distance to the screen 20 can be measured more accurately, and since the final normal vector is determined based on three normal vectors, the accuracy of the calculation of the final normal vector is improved.

[0109] [Other embodiments] In each of the embodiments described above, the projection device 10 is equipped with sensors. For example, the projection device 10 is equipped with a distance sensor 14 and an acceleration sensor 15. In embodiments 3 and 5, the projection device 10 is equipped with a camera 16. As a modification, at least one of these sensors may be provided outside the projection device 10. In such a case, the processor 12 of the projection device 10 may receive output from the externally provided sensor. In such a case, the projection device 10 and at least one externally provided sensor can be said to constitute a projection system.

[0110] In each of the embodiments described above, the physical components of each sensor can be designed arbitrarily. For example, a distance sensor for measuring the distance to point P1 and a distance sensor for measuring the distance to point P2 may be provided separately. Alternatively, for example, one distance sensor may have multiple light-receiving units. In that case, for example, each of the multiple light-receiving units may have one light-receiving element, or each of the multiple light-receiving units may have multiple light-receiving elements.

[0111] In each of the embodiments described above, if a camera 16 is provided, the distance sensor 14 may be omitted. In that case, the first distance L1, the second distance L2, etc., may be measured by triangulation based on the image captured by the camera 16. In that case, the camera 16 may be the first sensor.

[0112] The specific implementation method of the triangulation method can be arbitrarily designed based on publicly known technologies, etc., but one example is described below. The projection device 10 projects a surveying image onto the screen 20. The surveying image is an image containing multiple markers. Each marker indicates the position of the point to be measured, such as point P1, point P2, etc. Based on the pixel positions of the markers in the projected original image, it is possible to determine the direction of each point as seen from the projection lens 115. Also, based on the pixel positions of the markers in the image captured by the camera 16, it is possible to determine the direction of each point as seen from the camera 16. Furthermore, based on the position of the projection lens 115, the direction of each point as seen from the projection lens 115, the position of the camera 16, and the direction of each point as seen from the camera 16, the distance to each point can be determined.

[0113] In each of the embodiments described above, the order of the processes shown in each flowchart can be modified as appropriate by those skilled in the art. For example, in Figure 4, step S2 can be performed at any point before step S5. Also, step S2 may be performed before the process in Figure 4 begins.

[0114] Furthermore, when multiple processes are executed in a single step, the order in which these processes are executed can sometimes be designed arbitrarily. For example, in step S33 of Figure 12, there are no restrictions on the order in which the processes for identifying the first coordinate, identifying the second coordinate, and identifying the third coordinate are executed.

[0115] (Note) A summary of this disclosure is provided below.

[0116] (Note 1) A control method for a projection device that projects an image onto an object via a projection lens, comprising: using one or more processors to obtain a first distance from the projection lens to a first position on the object and a second distance from the projection lens to a second position on the object based on the output from a first sensor; obtaining a gravity vector in a first three-dimensional coordinate system based on the output from a second sensor; identifying a first coordinate of the first position in the first three-dimensional coordinate system based on the first distance; identifying a second coordinate of the second position in the first three-dimensional coordinate system based on the second distance; and identifying a first normal vector of a plane related to the object in the first three-dimensional coordinate system based on the first coordinate, the second coordinate, and the gravity vector.

[0117] According to the control method described in Appendix 1, by utilizing the gravity vector VG, it becomes easier to appropriately identify the first normal vector N1 of the screen 20.

[0118] (Note 2) The control method according to Appendix 1, wherein the first sensor comprises at least one light source that emits incident light incident on the object, and three or more photodetectors that each receive reflected light from three or more positions on the object where the incident light has been reflected, and the control method includes, using one or more processors, obtaining three or more intensity values, each of which indicates the intensity of the reflected light and corresponds one-to-one with the three or more photodetectors, based on the output from each of the three or more photodetectors, and identifying a first photodetector and a second photodetector that correspond to two intensity values ​​from the three or more intensity values ​​that satisfy a first criterion, wherein the first distance is a distance obtained based on the output from the first photodetector, and the second distance is a distance obtained based on the output from the second photodetector.

[0119] According to the control method described in Appendix 2, two photodetectors showing particularly large intensity values ​​are selected from among three or more photodetectors, thereby suppressing a decrease in the measurement accuracy of the first distance L1 and the second distance L2.

[0120] (Note 3) The control method according to Appendix 1 or 2, wherein the first sensor comprises at least one light source that emits incident light incident on the target, and three or more photodetectors that each receive reflected light from three or more positions on the target where the incident light has been reflected, and the control method comprises one or more processors determining, based on an image captured by a third sensor having three or more imaging pixels, whether or not an object other than the target exists in the space from the projection lens to the target; if it is determined that the object exists in the space, identifying a first imaging pixel and a second imaging pixel from among a plurality of imaging pixels included in a first region which is a region in the image where the object does not exist; and identifying a first photodetector corresponding to the first imaging pixel and a second photodetector corresponding to the second imaging pixel based on the correspondence between the three or more imaging pixels and the three or more photodetectors, and the control method according to Appendix 1 or 2, wherein the first distance is a distance obtained based on the output from the first photodetector, and the second distance is a distance obtained based on the output from the second photodetector.

[0121] According to the control method described in Appendix 3, the distances in areas where no objects exist can be used as the first distance L1 and the second distance L2, so that the distance to the screen 20 can be measured more accurately.

[0122] (Note 4) A control method according to any one of the appendices 1 to 3, further comprising: using one or more processors to obtain a third distance from the projection lens to a third position on the object based on the output from a first sensor; determining a third coordinate of the third position in a first three-dimensional coordinate system based on the third distance; determining a second normal vector of a plane related to the object in the first three-dimensional coordinate system based on the second coordinate, the third coordinate, and the gravity vector; determining a third normal vector of a plane related to the object in the first three-dimensional coordinate system based on the third coordinate, the first coordinate, and the gravity vector; and determining a fourth normal vector of a plane related to the object based on the first normal vector, the second normal vector, and the third normal vector.

[0123] According to the control method described in Appendix 4, the final normal vector is determined based on three normal vectors, thus improving the accuracy of the calculation of the final normal vector.

[0124] (Note 5) The first sensor comprises four or more photodetectors that each receive reflected light from four or more positions on the object where the incident light has been reflected; the third sensor comprises four or more imaging pixels; and the control method, when one or more processors determine that the object exists in the space, identifies a third imaging pixel from among three or more imaging pixels included in the first region; identifies a third photodetector corresponding to the third imaging pixel based on the correspondence between the four or more imaging pixels and the four or more photodetectors; and obtains a third distance from the projection lens to the third position on the object based on the output from the third photodetector. The control method according to Appendix 3 or 4, further comprising: determining the third coordinate of the third position in the first three-dimensional coordinate system based on the third distance; determining the second normal vector of the plane related to the object in the first three-dimensional coordinate system based on the second coordinate, the third coordinate, and the gravity vector; determining the third normal vector of the plane related to the object in the first three-dimensional coordinate system based on the third coordinate, the first coordinate, and the gravity vector; and determining the fourth normal vector of the plane related to the object based on the first normal vector, the second normal vector, and the third normal vector.

[0125] According to the control method described in Appendix 5, the distance to the screen 20 can be measured more accurately because the distance in the area where no object exists can be used. Also, since the final normal vector is determined based on the three normal vectors, the accuracy of the calculation of the final normal vector is improved.

[0126] (Note 6) The control method further includes specifying a direction vector with the first coordinate and the second coordinate as endpoints using one or more processors, wherein the first normal vector is specified based on the direction vector and the gravity vector, as described in any one of appendices 1 to 5.

[0127] According to the control method described in Appendix 6, the first normal vector N1 is calculated using the direction vector V12, thus reducing the amount of computation performed by the processor 12.

[0128] (Note 7) A projection system for projecting an image onto an object via a projection lens, wherein, based on the output from a first sensor, a first distance from the projection lens to a first position on the object and a second distance from the projection lens to a second position on the object are obtained; a gravity vector in a first three-dimensional coordinate system is obtained based on the output from a second sensor; a first coordinate of the first position in the first three-dimensional coordinate system is identified based on the first distance; a second coordinate of the second position in the first three-dimensional coordinate system is identified based on the second distance; and a first normal vector of a plane related to the object in the first three-dimensional coordinate system is identified based on the first coordinate, the second coordinate, and the gravity vector.

[0129] According to the projection system described in Appendix 7, by utilizing the gravity vector VG, it becomes easier to appropriately identify the first normal vector N1 of the screen 20.

[0130] (Note 8) A program for controlling a projection device that projects an image onto an object via a projection lens, the program causing one or more processors to perform the following actions: obtain a first distance from the projection lens to a first position on the object and a second distance from the projection lens to a second position on the object based on the output from a first sensor; obtain a gravity vector in a first three-dimensional coordinate system based on the output from a second sensor; identify a first coordinate of the first position in the first three-dimensional coordinate system based on the first distance; identify a second coordinate of the second position in the first three-dimensional coordinate system based on the second distance; and identify a first normal vector of a plane related to the object in the first three-dimensional coordinate system based on the first coordinate, the second coordinate, and the gravity vector.

[0131] According to the program described in Appendix 8, by utilizing the gravity vector VG, it becomes easier to appropriately identify the first normal vector N1 of the screen 20. [Explanation of Symbols]

[0132] 10…Projection device, 11…Projector, 12…Processor, 13…Memory device, 14…Distance sensor (1st sensor), 15…Accelerometer (2nd sensor), 16…Camera (1st sensor, 3rd sensor), 20…Screen (target), 30…Projected image, 40…Optical axis, 111…Light source, 112…Liquid crystal panel, 113…Prism, 114…Mirror, 115…Projection lens, 141…Light source, 142…Photodetector, C11~C18…Image pixels, D1~ D4...Distance, L1...First distance, L2...Second distance, L3...Third distance, LI...Incident light, LR...Reflected light, N1...First normal vector, N2...Second normal vector, N3...Third normal vector, N4...Fourth normal vector, P1...Point (first position), P2...Point (second position), P3...Point (third position), P11~P18...Points, PG1...Program, R1...First region, R2...Second region, V12, V23, V31...Direction vectors, VG...Gravity vector, W1~W4...Intensity values.

Claims

1. A method for controlling a projection device that projects an image onto an object via a projection lens, With one or more processors, Based on the output from the first sensor, The first distance from the projection lens to the first position on the object, The second distance from the projection lens to the second position on the object, To obtain, Based on the output from the second sensor, the gravity vector in the first three-dimensional coordinate system is obtained, Based on the first distance, the first coordinates of the first position in the first three-dimensional coordinate system are identified, Based on the second distance, the second coordinate of the second position in the first three-dimensional coordinate system is determined, Based on the first coordinate, the second coordinate, and the gravity vector, a first normal vector of the plane related to the object in the first three-dimensional coordinate system is determined, A control method including

2. The first sensor is, At least one light source that emits incident light incident on the aforementioned object, Three or more light-receiving elements that each receive reflected light from three or more positions on the object, Equipped with, The control method described above is The aforementioned one or more processors, Based on the output from each of the three or more light-receiving elements, each element represents the intensity of the reflected light, and three or more intensity values ​​corresponding one-to-one with the three or more light-receiving elements are obtained. Identifying the first and second photodetectors that correspond to two intensity values ​​among the three or more intensity values ​​that satisfy the first criterion, Includes, The first distance is a distance obtained based on the output from the first photodetector, The second distance is a distance obtained based on the output from the second photodetector. The control method according to claim 1.

3. The first sensor is, At least one light source that emits incident light incident on the aforementioned object, Three or more light-receiving elements that each receive reflected light from three or more positions on the object, Equipped with, The control method described above is The aforementioned one or more processors, Based on the captured image captured by the third sensor having three or more imaging pixels, it is determined whether or not an object other than the target exists in the space from the projection lens to the target, If it is determined that the object exists in the space, From among a plurality of imaging pixels included in the first region, which is a region in the aforementioned captured image where the object does not exist, a first imaging pixel and a second imaging pixel are identified. Based on the correspondence between the three or more imaging pixels and the three or more light-receiving elements, the first light-receiving element corresponding to the first imaging pixel and the second light-receiving element corresponding to the second imaging pixel are identified. Includes, The first distance is a distance obtained based on the output from the first photodetector, The second distance is a distance obtained based on the output from the second photodetector. The control method according to claim 1.

4. With one or more processors, Based on the output from the first sensor, the third distance from the projection lens to the third position on the target is obtained, Based on the third distance, the third coordinate of the third position in the first three-dimensional coordinate system is determined, Based on the second coordinate, the third coordinate, and the gravity vector, the second normal vector of the plane related to the object in the first three-dimensional coordinate system is determined, Based on the third coordinate, the first coordinate, and the gravity vector, the third normal vector of the plane related to the object in the first three-dimensional coordinate system is determined. Based on the first normal vector, the second normal vector, and the third normal vector, a fourth normal vector of the plane related to the object is determined, The control method according to claim 1, further comprising:

5. The first sensor comprises four or more light-receiving elements, each receiving reflected light from four or more positions on the object where the incident light has been reflected. The third sensor comprises four or more imaging pixels, The control method described above is When one or more processors determine that the object exists in the space, Identifying a third imaging pixel from among three or more imaging pixels included in the first region, Based on the correspondence between the four or more imaging pixels and the four or more photodetectors, the third photodetector corresponding to the third imaging pixel is identified. Based on the output from the third light-receiving element, a third distance is obtained from the projection lens to the third position on the target, Based on the third distance, the third coordinate of the third position in the first three-dimensional coordinate system is determined, Based on the second coordinate, the third coordinate, and the gravity vector, the second normal vector of the plane related to the object in the first three-dimensional coordinate system is determined, Based on the third coordinate, the first coordinate, and the gravity vector, the third normal vector of the plane related to the object in the first three-dimensional coordinate system is determined. Based on the first normal vector, the second normal vector, and the third normal vector, a fourth normal vector of the plane related to the object is determined, The control method according to claim 3, further comprising:

6. The control method further includes specifying a direction vector with the first coordinate and the second coordinate as endpoints using one or more processors, The first normal vector is determined based on the direction vector and the gravity vector. The control method according to claim 1.

7. A projection system that projects an image onto an object via a projection lens, Based on the output from the first sensor, The first distance from the projection lens to the first position on the object, The second distance from the projection lens to the second position on the object, Obtain, Based on the output from the second sensor, the gravity vector in the first three-dimensional coordinate system is obtained. Based on the first distance, the first coordinates of the first position in the first three-dimensional coordinate system are identified. Based on the second distance, the second coordinate of the second position in the first three-dimensional coordinate system is identified. Based on the first coordinate, the second coordinate, and the gravity vector, a first normal vector of the plane related to the object in the first three-dimensional coordinate system is determined. Projection system.

8. A program for controlling a projection device that projects an image onto an object via a projection lens, The aforementioned program is configured on one or more processors. Based on the output from the first sensor, The first distance from the projection lens to the first position on the object, The second distance from the projection lens to the second position on the object, To obtain, Based on the output from the second sensor, the gravity vector in the first three-dimensional coordinate system is obtained, Based on the first distance, the first coordinates of the first position in the first three-dimensional coordinate system are identified, Based on the second distance, the second coordinate of the second position in the first three-dimensional coordinate system is determined, Based on the first coordinate, the second coordinate, and the gravity vector, a first normal vector of the plane related to the object in the first three-dimensional coordinate system is determined, A program that executes something.

Images