Correction method, projector, and program

Abstract

To increase the accuracy of correcting the shape of a projection image.SOLUTION: A correction method includes: determining a normal vector of a screen surface; determining a first vector orthogonal to both a gravity vector and the normal vector that are determined from output of an acceleration sensor associated with a coordinate system of an optical device of a projector; determining a second vector orthogonal to the first vector on the screen surface; and determining a correction parameter for correcting the shape of a projection image to be projected on the screen surface on the basis of the first vector and the second vector.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to a correction method, a projector, and a program. [Background technology]

[0002] For example, Patent Document 1 discloses an image projection system having an estimation unit that estimates an inclination of a projection display device with respect to a projection surface based on a projection azimuth angle detected by a first azimuth angle detection unit, an imaging azimuth angle detected by a second azimuth angle detection unit, and an image captured by an imaging device, and a correction unit that corrects the shape of a projection screen based on the inclination of the projection display device estimated by the estimation unit. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2017 / 169186 Summary of the Invention [Problem to be solved by the invention]

[0004] When the projection surface is tilted obliquely with respect to gravity, the inclination of the projection surface needs to be taken into account when correctly calculating the roll rotation angle for correcting the shape of the projection screen, but Patent Document 1 does not take this into consideration. [Means for solving the problem]

[0005] A correction method according to one embodiment of the present disclosure includes calculating a normal vector of a screen surface, calculating a first vector that is orthogonal to both a gravity vector and the normal vector that are calculated from the output of an acceleration sensor associated with a coordinate system of an optical device of a projector, calculating a second vector that is included in the screen surface and is orthogonal to the first vector, and calculating correction parameters for correcting a shape of a projection image projected onto the screen surface based on the first vector and the second vector.

[0006] A projector according to another embodiment of the present disclosure includes an optical device and a processing device, wherein the processing device executes the following operations: determining a normal vector of a screen surface; determining a first vector that is orthogonal to both a gravity vector and the normal vector determined from the output of an acceleration sensor associated with a coordinate system of the optical device; determining a second vector that is included in the screen surface and is orthogonal to the first vector; and determining correction parameters for correcting the shape of a projection image projected onto the screen surface based on the first vector and the second vector.

[0007] A program according to one embodiment of the present disclosure causes a computer to execute the following steps: calculate a normal vector of a screen surface; calculate a first vector that is orthogonal to both the normal vector and a gravity vector calculated from the output of an acceleration sensor associated with the coordinate system of an optical device of a projector; calculate a second vector that is included on the screen surface and is orthogonal to the first vector; and calculate correction parameters for correcting the shape of a projection image projected onto the screen surface based on the first vector and the second vector. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an outline of a system used in a correction method according to a first embodiment. [Diagram 2] FIG. 1 is a block diagram of a projector according to a first embodiment. [Diagram 3] 4 is a flowchart showing a flow of a correction method according to the first embodiment. [Figure 4] 3A and 3B are diagrams for explaining the pitch angle, yaw angle, and roll angle of the projector. [Diagram 5] 11 is a diagram for explaining the relationship between the attitude of the projector and the output of the acceleration sensor. FIG. [Figure 6] FIG. 4 is a diagram for explaining the relationship between a gravity vector and a pitch angle. [Figure 7] FIG. 2 is a diagram for explaining the relationship between a gravity vector and a roll angle. [Figure 8] 1 is a diagram for explaining a roll angle obtained by a conventional method based on the output of an acceleration sensor. FIG. [Figure 9] 11A and 11B are diagrams for explaining a roll angle calculated by another conventional method based on the output of an acceleration sensor. [Figure 10] 4 is a diagram for explaining the relationship between a normal vector of a screen surface and a gravity vector. FIG. [Figure 11] FIG. 2 is a diagram for explaining three-dimensional coordinates on a screen surface. [Figure 12] 2 is a diagram for explaining a plane representing a screen surface and a normal vector of the screen surface. FIG. [Figure 13] FIG. 13 is a diagram for explaining acquisition of a gravity vector. [Figure 14] FIG. 2 is a diagram for explaining a first vector and a second vector. [Figure 15] 4 is a diagram for explaining vectors obtained by standardizing a normal vector of a screen surface, a first vector, and a second vector. FIG. [Figure 16] FIG. 11 is a diagram for explaining an example of keystone distortion correction. [Figure 17] 1 is a diagram for explaining a horizontal vanishing point and a vertical vanishing point in an optical device. [Figure 18] FIG. 13 is a diagram for explaining determination of a correction shape. [Figure 19] FIG. 11 is a block diagram of a projector according to a second embodiment. [Figure 20]10 is a flowchart showing a procedure of a correction method according to a second embodiment. [Figure 21] FIG. 11 is a block diagram of a projector according to a third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, preferred embodiments of the present disclosure will be described with reference to the accompanying drawings. Note that the dimensions and scale of each part in the drawings may differ from the actual dimensions, and some parts are shown diagrammatically to facilitate understanding. In addition, the scope of the present disclosure is not limited to these forms unless otherwise specified in the following description to the effect that the present disclosure is limited. 1. First embodiment 1-1. Overview of the system used for the correction method 1 is a diagram showing an outline of a system 100 used in a correction method according to Embodiment 1. As shown in FIG.

[0010] The projector 10 is a display device that projects an image represented by image information output from a device such as a computer (not shown) onto a screen surface SC. The screen surface SC is the surface of an object such as a screen, and is generally flat. The screen surface SC does not need to be strictly flat, but may be any surface that can be regarded as a surface.

[0011] Here, the installation orientation of the screen surface SC may differ depending on the usage mode of the system 100. Therefore, the projector 10 corrects the distortion of the projected image caused by the installation orientation of the screen surface SC by keystone correction. As will be described in detail later, the projector 10 is equipped with a camera 17 and an acceleration sensor 18, and has a function of measuring the shape of the screen surface SC using the camera 17, a function of acquiring a gravity vector using the acceleration sensor 18, and a function of determining correction parameters for keystone correction based on the shape of the screen surface SC and the gravity vector.

[0012] 1-2.Projector Fig. 2 is a block diagram of a projector 10 according to the first embodiment. As shown in Fig. 2, the projector 10 has a storage device 11, a processing device 12, a communication device 13, an image processing circuit 14, an optical device 15, an operation device 16, a camera 17, and an acceleration sensor 18. The storage device 11, the processing device 12, the communication device 13, the image processing circuit 14, and the acceleration sensor 18 are disposed inside a housing (not shown) of the projector 10. These are connected to each other so as to be able to communicate with each other.

[0013] The storage device 11 is a storage device that stores programs executed by the processing device 12 and data processed by the processing device 12. The storage device 11 includes, for example, a hard disk drive or a semiconductor memory. Note that a part or all of the storage device 11 may be provided in an external storage device or server outside the projector 10.

[0014] The storage device 11 stores a program PR1 and correction parameters PA.

[0015] The program PR1 is a program for executing a correction method, which will be described later in detail. The correction parameter PA is a parameter indicating the degree of correction in the distortion correction process in the image processing circuit 14, and is generated by a correction value calculation unit 12c, which will be described later.

[0016] The processing device 12 is a processing device having a function of controlling each part of the projector 10 and a function of processing various data. The processing device 12 is configured to include a processor such as a CPU (Central Processing Unit). The processing device 12 may be configured with a single processor or multiple processors. Also, some or all of the functions of the processing device 12 may be realized by hardware such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). Also, the processing device 12 may be integrated with the image processing circuit 14.

[0017] The communication device 13 is a communication device capable of communicating with various devices, and acquires image data IMG from a device not shown. For example, the communication device 13 is a wired communication device such as a wired LAN (Local Area Network), USB (Universal Serial Bus), or HDMI (High Definition Multimedia Interface), or a wireless communication device such as LPWA (Low Power Wide Area), wireless LAN including Wi-Fi, or Bluetooth. "HDMI", "Wi-Fi", and "Bluetooth" are all registered trademarks.

[0018] The image processing circuit 14 is a circuit that performs necessary processing on the video data IMG from the communication device 13 and inputs it to the optical device 15. The image processing circuit 14 has, for example, a frame memory (not shown), and develops the video data IMG in the frame memory, appropriately performs various processing such as resolution conversion processing, resizing processing, and distortion correction processing, and inputs the data to the optical device 15. Here, the above-mentioned correction parameter PA is used for the distortion correction processing. Note that the image processing circuit 14 may perform processing such as OSD (On Screen Display) processing that generates image information for menu display or operation guide, etc., and synthesizes it with the video data IMG, as necessary.

[0019] The optical device 15 is a device that displays an image by projecting image light onto a projection region RP. The optical device 15 includes a light source 15a, a light modulator 15b, and a projection optical system 15c.

[0020] The light source 15a includes a light source such as a halogen lamp, a xenon lamp, an ultra-high pressure mercury lamp, an LED (Light Emitting Diode), or a laser light source, and emits red, green, and blue light, respectively. The light modulator 15b includes three light modulation elements provided corresponding to red, green, and blue. Each light modulation element includes, for example, a transmissive liquid crystal panel, a reflective liquid crystal panel, or a DMD (Digital Mirror Device), and generates image light of each color by modulating light of the corresponding color. The image light of each color generated by the light modulator 15b is synthesized by a color synthesis optical system to become full-color image light. The projection optical system 15c is an optical system including a projection lens that forms an image of the full-color image light from the light modulator 15b on a screen surface SC and projects it.

[0021] The operation device 16 is a device that accepts operations from a user. For example, the operation device 16 includes an operation panel and a remote control light receiving unit, not shown. The operation panel is provided on the exterior housing of the projector 10, and outputs a signal based on an operation from the user. The remote control light receiving unit receives an infrared signal from a remote control, not shown, decodes the infrared signal, and outputs a signal based on the operation of the remote control. Note that the operation device 16 is provided as necessary, and may be omitted.

[0022] The camera 17 is a digital camera having an imaging element such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS).

[0023] The acceleration sensor 18 is a sensor that detects acceleration applied to the projector 10. The acceleration sensor 18 is built into the projector 10 and fixed to a predetermined location in the housing of the projector 10. The predetermined location is, for example, a circuit board (not shown) on which the processing device 12 is mounted. The acceleration sensor 18 outputs a signal according to acceleration in a direction along each of the x-axis, y-axis, and z-axis (to be described later) that are associated with the coordinate system of the optical device 15 of the projector 10. The acceleration sensor 18 is fixed to a predetermined location in the housing of the projector 10, thereby specifying its position in the housing. In other words, the relative positional relationship between the acceleration sensor 18 and the optical device 15 is specified in advance. As a result, the acceleration sensor 18 is associated with the coordinate system of the optical device 15.

[0024] In the above-described projector 10, the processing device 12 functions as a plane detection unit 12a, an axis calculation unit 12b, and a correction value calculation unit 12c by executing the program PR1 stored in the storage device 11. Therefore, the processing device 12 includes the plane detection unit 12a, the axis calculation unit 12b, and the correction value calculation unit 12c.

[0025] The plane detection unit 12a determines a plane representing the screen surface SC by controlling the operations of the optical device 15 and the camera 17. Specifically, the plane detection unit 12a determines a plane representing the screen surface SC based on an image obtained by capturing a measurement pattern PT (described later) projected onto the screen surface SC.

[0026] The axis calculation unit 12b obtains an axis x and an axis y, which will be described later, as coordinate axes of the screen surface SC, based on the plane obtained by the plane detection unit 12a and the output of the acceleration sensor 18. Specifically, the axis calculation unit 12b obtains a normal vector N, which will be described later, of the screen surface SC, based on the plane obtained by the plane detection unit 12a. The axis calculation unit 12b also obtains a vector X, which will be described later, that is orthogonal to both a gravity vector G and a normal vector N, which will be described later, that are obtained from the output of the acceleration sensor 18. The vector X is an example of a "first vector." Furthermore, the axis calculation unit 12b obtains a vector Y, which will be described later, that is orthogonal to the vector X on the screen surface SC. The vector Y is an example of a "second vector." The vector Y is a vector included in the screen surface SC.

[0027] The correction value calculation unit 12c calculates a correction parameter PA for correcting the shape of the projection image projected onto the screen surface SC, based on vector X and vector Y, which are described below and calculated by the axis calculation unit 12b.

[0028] 1-3. Correction method 3 is a flowchart showing the flow of the correction method according to the first embodiment. The correction method is executed by the projector 10 described above.

[0029] As shown in FIG. 3, the correction method of this embodiment includes steps S10, S20, and S30 in this order.

[0030] In step S10, the plane detection unit 12a controls the operations of the optical device 15 and the camera 17 to obtain a plane representing the screen surface SC.

[0031] Specifically, step S10 includes steps S11, S12, and S13 in this order. In step S11, the plane detection unit 12a captures an image of a measurement pattern PT (described later) projected onto the screen surface SC. In step S12, the plane detection unit 12a determines coordinates on the screen surface SC based on the image captured in step S11. In step S13, the plane detection unit 12a determines a plane representing the screen surface SC based on the coordinates determined in step S12.

[0032] In step S20, the axis calculation unit 12b determines the x-axis and the y-axis (described later) as the coordinate axes of the screen surface SC based on the plane determined by the plane detection unit 12a and the output of the acceleration sensor .

[0033] Specifically, step S20 includes steps S21, S22, S23, and S24 in this order. In step S21, the axis calculation unit 12b calculates a normal vector N of the screen surface SC, described below, based on the plane calculated in step S10. In step S22, the axis calculation unit 12b acquires a gravity vector G, described below, that is calculated from the output of the acceleration sensor 18. In step S23, the axis calculation unit 12b calculates a vector X, described below, that is orthogonal to both the gravity vector G and the normal vector N acquired in step S22. In step S23, the axis calculation unit 12b calculates a vector Y, described below, that is orthogonal to the vector X on the screen surface SC.

[0034] In step S30, the correction value calculation unit 12c calculates correction parameters PA for correcting the shape of the projection image projected onto the screen surface SC, based on vectors X and Y (to be described later) calculated in step S20.

[0035] Specifically, step S30 includes steps S31, S32, and S33 in this order. In step S31, the correction value calculation unit 12c obtains a matrix R, described below, for converting from the coordinate system of the optical device 15 of the projector 10 to the coordinate system on the screen surface SC. In step S32, the correction value calculation unit 12c calculates a correction shape using the matrix R obtained in step S31. In step S33, the correction value calculation unit 12c obtains a correction parameter PA based on the correction shape obtained in step S32.

[0036] FIG. 4 is a diagram for explaining the pitch angle θ, yaw angle φ, and roll angle ψ of the projector 10. As shown in FIG. 4, the projector 10 has an x-axis, a y-axis, and a z-axis set as three mutually orthogonal axes. The x-axis is an axis along the width direction of the projector 10. The y-axis is an axis along the height direction of the projector 10. The z-axis is an axis along the projection direction of the projector 10. The z-axis is an example of a "first axis" that is the optical axis of the projection lens. The y-axis is an example of a "second axis" that is orthogonal to the first axis. The x-axis is an example of a "third axis" that is orthogonal to both the first axis and the second axis.

[0037] Here, the attitude of projector 10 is expressed by a pitch angle θ, which is the rotation angle of projector 10 about the x-axis, a yaw angle φ, which is the rotation angle of projector 10 about the y-axis, and a roll angle ψ, which is the rotation angle of projector 10 about the z-axis.

[0038] In the example shown in Fig. 4, the y-axis is parallel to the direction of gravity. Therefore, the x-axis and z-axis are each parallel to the horizontal plane. Furthermore, the horizontal axis h along the lateral direction of the screen surface SC is parallel to the direction of gravity, and the vertical axis v along the longitudinal direction of the screen surface SC is parallel to the horizontal plane. Here, when the z-axis is perpendicular to the screen surface SC, the x-axis is parallel to the horizontal axis h, and the y-axis is parallel to the vertical axis v.

[0039] 5 is a diagram for explaining the relationship between the attitude of projector 10 and the output of acceleration sensor 18. Acceleration sensor 18 is a three-axis acceleration sensor that outputs signals according to the acceleration in the directions along the x-axis, y-axis, and z-axis. Such three-axis output from acceleration sensor 18 indicates the three components of the gravitational vector G acting on projector 10 when projector 10 is installed.

[0040] Fig. 6 is a diagram for explaining the relationship between the gravity vector G and the pitch angle θ. Fig. 7 is a diagram for explaining the relationship between the gravity vector G and the roll angle ψ. When the screen surface SC is perpendicular to a plane having the gravity vector G as a normal vector, as shown in Fig. 6, the pitch angle θ can be obtained based on the output values ​​(Gx, Gy, Gz) of the acceleration sensor 18, that is, based on the gravity vector G. In this case, as shown in Fig. 7, the roll angle ψ can be obtained based on the output values ​​(Gx, Gy, Gz) of the acceleration sensor 18, that is, based on the gravity vector G.

[0041] However, since the yaw angle φ is unrelated to the gravity vector G, the yaw angle φ cannot be obtained based on the output values ​​(Gx, Gy, Gz) of the acceleration sensor 18.

[0042] Fig. 8 is a diagram for explaining the roll angle ψ calculated by a conventional method based on the output of the acceleration sensor 18. In Fig. 8, the screen surface SC when it is perpendicular to the plane having the gravity vector G as a normal vector is indicated by a dashed line as a screen surface SC-0, and the screen surface SC when it is not perpendicular to the plane having the gravity vector G as a normal vector is indicated by a dashed line as a screen surface SC-X.

[0043] As shown in Fig. 8, the roll angle ψ of the image on the actual screen surface SC-X differs from the roll angle ψ' calculated based on the output value of the acceleration sensor 18 as described above. Here, the roll angle ψ of the image on the screen surface SC-1 is the angle between the vertical axis v of the screen surface SC-X and the y axis of the projector 10. In this way, when the screen surface SC is not perpendicular to the plane having the gravity vector G as the normal vector, there is a problem that the roll angle ψ differs from the roll angle ψ'. Although not shown, the same problem occurs with the pitch angle θ.

[0044] FIG. 9 is a diagram for explaining the roll angle ψ calculated by another conventional method based on the output of the acceleration sensor 18. This conventional method uses a TOF (Time Of Flight) sensor to directly measure the relative attitude relationship between the projector 10 and the screen surface SC. By using this measurement result, the correct pitch angle θ and yaw angle φ can be calculated. However, this measurement result does not include information related to the roll angle ψ. For this reason, it is necessary to calculate the roll angle ψ based on the output value of the acceleration sensor 18.

[0045] As described above, on screen surface SC-0 that is perpendicular to a plane whose normal vector is the gravity vector G, the angle between the y axis of projector 10 and the vertical axis v of screen surface SC-0 coincides with the roll angle ψ obtained from the output of acceleration sensor 18. Therefore, first, the y axis of projector 10 is set as the virtual y axis of screen surface SC-0, and this virtual y axis is rotated by the roll angle ψ' obtained from the output of acceleration sensor 18 to obtain the y axis after roll correction of screen surface SC-0, and then the axis perpendicular to this y axis after roll correction can be obtained as the x axis of screen surface SC-0.

[0046] However, on a screen surface SC-X that is not perpendicular to a plane having the gravity vector G as its normal vector, the angle between the y axis of the projector 10 and the vertical axis v of the screen surface SC-X does not match the roll angle ψ obtained from the output of the acceleration sensor 18. For this reason, the other method cannot obtain the correct x and y axes of the screen surface SC-X. As a result, an image that is rotated in the roll direction within the plane of the screen surface SC-X is projected.

[0047] Fig. 10 is a diagram for explaining the relationship between the normal vector N of the screen surface SC-X and the gravity vector G. As shown in Fig. 10, based on the normal vector N and the gravity vector G of the screen surface SC-X, a vector X representing the x-axis and a vector Y representing the y-axis on the screen surface SC-X can be obtained. The vector X is an example of a "first vector." The vector Y is an example of a "second vector."

[0048] Here, the x-axis of the screen surface SC-X is defined so that vector X representing the x-axis is perpendicular to gravity vector G. Also, the y-axis of the screen surface SC-X is defined so that vector Y representing the y-axis, normal vector N of the screen surface SC-X, and gravity vector G are in the same plane.

[0049] More specifically, for example, by using the cross product of vectors, Vector X and vector Y can be found from the relationships X=N×G and Y=N×X=N×(N×G).

[0050] By determining the x-axis and y-axis on the screen surface SC-X in this manner, it is possible to correct the roll direction of the image regardless of the attitude of the projector 10 or the attitude of the screen surface SC.

[0051] 11 is a diagram for explaining the three-dimensional coordinates (Xs, Ys, Zs) of the screen surface SC-X. In the above-mentioned step S11, the plane detection unit 12a controls the operation of the projector 10 so as to sequentially project a plurality of measurement patterns PT onto the screen surface SC, and controls the operation of the camera 17 so as to capture each measurement pattern PT projected onto the screen surface SC. This allows the camera 17 to obtain a plurality of captured images of the measurement patterns PT.

[0052] For example, a binary code pattern is used as the measurement pattern PT. The binary code pattern refers to an image for expressing the coordinates of a display device using a binary code. The binary code is a technique for expressing the value of each digit when an arbitrary numerical value is expressed in binary by turning a switch on and off. When a binary code pattern is used as the measurement pattern PT, the image projected by the projector 10 corresponds to the switch, and measurement patterns PT are required for the number of digits of the binary number expressing the coordinate value. In addition, separate measurement patterns PT are required for each of the vertical coordinate and the horizontal coordinate. For example, when the resolution (number of pixels) of the optical device 15 of the projector 10 is 120×90, 120 and 90 are each expressed as a 7-digit binary number, so seven images are required to express the vertical coordinate and seven images are required to express the horizontal coordinate.

[0053] Furthermore, when a binary code pattern is used as the measurement pattern PT, the robustness of the measurement generally decreases due to the influence of disturbance light such as lighting. For this reason, when a binary code pattern is used as the measurement pattern PT, it is preferable to use a complementary pattern in addition to the binary code pattern in order to reduce the influence of disturbance light and improve the robustness of the measurement. A complementary pattern is an image in which black and white are inverted.

[0054] It should be noted that the measurement pattern PT is not limited to a binary code pattern, and may be other structured light patterns such as a dot pattern, a rectangular pattern, a polygonal pattern, a checkered pattern, a gray code pattern, a phase shift pattern, or a random dot pattern.

[0055] In step S12, the plane detection unit 12a measures the screen surface SC based on the multiple captured images. Here, in step S12, a correspondence relationship between the coordinates (Xc, Yc) of the coordinate system of the captured image of the camera 17 and the coordinates (Xp, Yp) in the coordinate system of the optical device 15 of the projector 10 is obtained, and the three-dimensional coordinates of each part of the measurement pattern PT projected on the screen surface SC-X are obtained from this correspondence. As a result, the three-dimensional coordinates (Xs, Ys, Zs) of each point on the screen surface SC-X onto which each point is projected are obtained for the coordinates (Xp, Yp) of each point in the coordinate system of the optical device 15.

[0056] Fig. 12 is a diagram for explaining a plane representing the screen surface SC-X and a normal vector N of the screen surface SC-X. As shown in Fig. 12, in the above-mentioned step S12, the coordinates (Xs0, Ys0, Zs0), (Xs1, Ys1, Zs1), ..., (Xsn, Ysn, Zsn) of n points on the screen surface SC-X onto which the measurement pattern PT is projected are found as three-dimensional coordinates (Xs, Ys, Zs).

[0057] In step S13, the plane detection unit 12a obtains the equation of a plane that passes through these n points, that is, aX+bY+cZ=1, to obtain the equation of the plane that represents the screen surface SC-X.

[0058] As described above, a plane representing the screen surface SC-X is obtained in step S10. After step S10, step S20 is executed.

[0059] In step S20, first, in step S21, the axis calculation unit 12b calculates the normal vector N=(Nx, Ny, Nz)=(a, b, c) of the screen surface SC-X based on the equation calculated in step S13.

[0060] Fig. 13 is a diagram for explaining acquisition of the gravity vector G. In step S22, the axis calculation unit 12b acquires the gravity vector G=(Gx, Gy, Gz) acting on the projector 10 from the output value of the acceleration sensor 18, as shown in Fig. 13. Here, since the same gravitational acceleration is also acting on the screen surface SC-X, the gravity vector of the screen surface SC-X is also G=(Gx, Gy, Gz).

[0061] Fig. 14 is a diagram for explaining vector X and vector Y. In step S23, the axis calculation unit 12b obtains a vector X that is orthogonal to both the normal vector N = (Nx, Ny, Nz) of the screen surface SC-X and the gravity vector G = (Gx, Gy, Gz) as shown in Fig. 13. Here, for example, the vector X = N × G is obtained by the cross product of the normal vector N and the gravity vector G. Note that the method of calculating the vector X is not limited to the method using the cross product, and may be, for example, a method of obtaining the vector X from the intersection line between a plane orthogonal to the normal vector N and a plane orthogonal to the gravity vector G.

[0062] In step S24, the axis calculation unit 12b obtains a vector Y that is orthogonal to both the normal vector N and the vector X. Here, for example, the vector Y=N×X=N×(N×G) is obtained by the cross product of the normal vector N and the vector X.

[0063] By determining the vectors X and Y as described above, in step S30, correction can be made in the correct roll direction using the vectors X and Y so that the x-axis coincides with the horizontal axis h and the y-axis coincides with the vertical axis v on the screen surface SC-X.

[0064] 15 is a diagram for explaining vectors Ex, Ey, and Ez obtained by standardizing the normal vector N, vector X, and vector Y of the screen surface SC-X. In step S31, the correction value calculation unit 12c first normalizes the vector X=(Xx, Xy, Xz) representing the x-axis of the screen surface SC-X, the vector Y=(Yx, Yy, Yz) representing the y-axis, and the normal vector N=(Nx, Ny, Nz) to a length of 1, thereby obtaining the vectors Ex, Ey, and Ez as an orthonormal basis on the screen surface SC-X that has been appropriately roll-corrected with respect to gravity.

[0065] If the three vectors Ex, Ey, and Ez are transposed and arranged horizontally to form a 3 × 3 matrix R, then this matrix R becomes a rotation matrix from the coordinate system of the optical device 15 of the projector 10 to the coordinate system on the screen surface SC-X, as expressed by the following equation.

[0066]

number

[0067] In this manner, in step S31, the correction value calculation unit 12c uses the three vectors Ex, Ey, and Ez to obtain the matrix R, which is a rotation matrix.

[0068] If the standard basis vectors of the coordinate system of the optical device 15 of the actual projector 10 are e_X = (1,0,0), e_Y = (0,1,0), and e_Z = (0,0,1), then by converting these with matrix R, they are converted into vectors Ex, Ey, and Ez as orthonormal definitions on the screen surface SC-X, as expressed by the following equation.

[0069]

number

[0070] In this way, the matrix R is a rotation matrix that represents the rotation of the screen surface SC-X such that the roll seen from the projector 10 in the projection direction is correctly corrected.

[0071] By determining the matrix R as described above, in step S32, the corrected shape SH can be calculated using a known keystone distortion correction method using the matrix R. An example of the calculation of the distortion corrected shape in step S32 will be described below.

[0072] Fig. 16 is a diagram for explaining an example of keystone correction. Fig. 16 shows a method of using the coordinates of a vanishing point on the optical device 15 as an example of keystone correction. The vanishing point is the coordinates on the optical device 15 when a point at infinity in the vertical and horizontal directions on the screen surface SC-X is viewed from the projector 10.

[0073] The matrix R represents the rotation of the screen surface SC-X with respect to the projector 10. When the coordinates of the origin of the screen surface SC-X are (0, 0, 1 / c), an arbitrary point P on the screen surface SC-X is expressed by the following equation using two parameters s and t.

[0074]

number

[0075] At this point P, when s → infinity, it becomes the point at horizontal infinity, and when t → infinity, it becomes the point at vertical infinity.

[0076] FIG. 17 is a diagram for explaining the horizontal vanishing point and the vertical vanishing point in the optical device 15. As shown in FIG.

[0077] A point p on the optical device 15 corresponding to this point P is expressed by the following equation.

[0078]

number

[0079] The coordinate system of the optical device 15 is not a pixel coordinate system but is called the standard coordinate system of a pinhole camera model, and is obtained by extracting two components (X, Y) from the coordinates (X, Y, 1) on a plane when the emission point of the projection light is set as the origin and the optical device 15 is regarded as the plane Z=1.

[0080] The vanishing point on the optical device 15 is a point at infinity in real space projected onto the optical modulator 15b of the optical device 15. Therefore, as expressed by the following formula, at point p in the coordinate system of the optical device 15, when s → ∞, it is the horizontal vanishing point H∞, and when t → ∞, it is the vertical vanishing point V∞.

[0081]

number

[0082] 18 is a diagram for explaining the determination of the correction shape SH. A straight line parallel to the x-axis in real space (i.e., a horizontal straight line) has the property that it passes through the horizontal vanishing point H∞ on the optical device 15. Similarly, a straight line parallel to the y-axis in real space (i.e., a vertical straight line) has the property that it passes through the vertical vanishing point V∞ on the optical device 15. The inverse of these properties also holds true.

[0083] Therefore, in step S32, the correction value calculation unit 12c calculates the corrected shape SH on the optical device 15 by calculating each of the upper and lower sides as straight lines passing through the horizontal vanishing point H∞, and each of the left and right sides as straight lines passing through the vertical vanishing point V∞.

[0084] Here, there are many types of corrective shapes SH whose upper and lower sides each pass through the horizontal vanishing point H∞ and whose left and right sides each pass through the vertical vanishing point V∞. The corrective shape SH is determined by selecting the corrective shape SH according to conditions such as the maximum area, the left-justified, right-justified, top-justified or bottom-justified shape on the optical device 15, the specified aspect ratio on the screen surface SC-X projection surface, etc., depending on the purpose of the function.

[0085] In step S33, the correction value calculation unit 12c calculates the correction parameters PA based on the correction shape SH calculated in step S32. The calculated correction parameters PA are stored in the storage device 11 as described above, and are used for keystone correction in the image processing circuit 14. As a result, a rectangular image that has been roll-corrected so that the y-axis is parallel to the direction of gravity is projected onto the screen surface SC-X.

[0086] As described above, the correction method of this embodiment includes steps S21, S23, S24, and S30. As described above, step S21 obtains the normal vector N of the screen surface SC. Step S23 obtains the vector X that is orthogonal to both the gravity vector G and the normal vector N obtained from the output of the acceleration sensor 18 associated with the coordinate system of the optical device 15 of the projector 10. The vector X is an example of a "first vector." Step S24 obtains the vector Y that is included in the screen surface SC and is orthogonal to the vector X. The vector Y is an example of a "second vector." Step S30 obtains the correction parameter PA for correcting the shape of the projection image projected on the screen surface SC based on the vector X and the vector Y.

[0087] Here, the correction method of the present embodiment is performed using the projector 10 including the optical device 15 and the processing device 12. The processing device 12 executes steps S21, S23, S24, and S30.

[0088] The display method of this embodiment is realized by the processing device 12, which is an example of a "computer," executing the program PR1. The program PR1 causes the processing device 12 to execute steps S21, S23, S24, and S30.

[0089] In the above correction method, projector 10, or program PR1, it is possible to obtain a correction parameter PA that takes into account the inclination of the screen surface SC with respect to the direction of gravity by using the gravity vector G and the normal vector N. As a result, it is possible to improve the correction accuracy of the shape of the projected image.

[0090] As described above, the correction method of this embodiment includes step S10. In step S10, a plane representing the screen surface SC is obtained. Then, in step S21, a normal vector N is obtained from the plane of the screen surface SC. This makes it possible to easily obtain the normal vector N of the screen surface SC even if the screen surface SC is not strictly a plane.

[0091] As described above, step S10 includes step S13. In step S13, the plane of the screen surface SC is obtained based on an image of the measurement pattern PT projected onto the screen surface SC. This makes it possible to obtain a plane representing the screen surface SC with high accuracy.

[0092] Furthermore, as described above, step S30 determines the correction parameters PA using the normalized vectors X and Y. This makes it possible to easily adjust the aspect ratio of the correction shape SH.

[0093] 2. Second embodiment A second embodiment of the present disclosure will be described below. In the following exemplary embodiment, for elements whose actions and functions are similar to those of the first embodiment, the reference symbols used in the description of the first embodiment will be used and detailed descriptions of each will be omitted as appropriate.

[0094] 19 is a block diagram of a projector 10A according to the second embodiment. The projector 10A includes a TOF sensor 19 instead of the camera 17 of the first embodiment, and uses a program PR2 instead of the program PR1 of the first embodiment, and is configured in the same manner as the projector 10 of the first embodiment.

[0095] The TOF sensor 19 is a time-of-flight sensor and measures the shape of the screen surface SC. The output of the TOF sensor 19 indicates the three-dimensional coordinates of the screen surface SC.

[0096] The processing device 12 of this embodiment functions as a plane detection unit 12d, an axis calculation unit 12b, and a correction value calculation unit 12c by executing the program PR2 stored in the storage device 11. Therefore, the processing device 12 of this embodiment includes the plane detection unit 12d, the axis calculation unit 12b, and the correction value calculation unit 12c.

[0097] The plane detection unit 12d determines a plane representing the screen surface SC based on the output of the TOF sensor 19. Note that the axis calculation unit 12b of this embodiment determines the axis x and the axis y as the coordinate axes of the screen surface SC based on the plane determined by the plane detection unit 12d and the output of the acceleration sensor 18.

[0098] 20 is a flowchart showing the flow of the correction method according to the second embodiment. The correction method is executed by the above-mentioned projector 10A.

[0099] The correction method of this embodiment is similar to the correction method of the first embodiment, except that it includes step S10A instead of step S10 of the first embodiment, as shown in Fig. 20. Step S10A is similar to step S10 of the first embodiment, except that it includes step S12A instead of steps S11 and S12 of the first embodiment.

[0100] In step S12A, the plane detection unit 12d measures the shape of the screen surface SC using the TOF sensor 19, and determines coordinates on the screen surface SC based on the output of the TOF sensor 19. In step S13 of this embodiment, the plane detection unit 12d determines a plane representing the screen surface SC based on the coordinates determined in step S12A.

[0101] The above-described second embodiment can also improve the accuracy of correction of the shape of the projected image. As described above, the correction method of this embodiment includes step S10A, in which the plane of the screen surface SC is obtained using a TOF sensor 19, which is a time-of-flight sensor. This allows the plane representing the screen surface SC to be obtained in a short time compared to the aspect in which the measurement pattern PT is used as in the first embodiment.

[0102] 3. Third embodiment The third embodiment of the present disclosure will be described below. In the following exemplary embodiment, the elements having the same actions and functions as those in the first embodiment will be referred to by the same reference numerals used in the description of the first embodiment, and detailed descriptions thereof will be omitted as appropriate.

[0103] 21 is a block diagram of a projector 10B according to the third embodiment. The projector 10B is configured similarly to the projector 10 of the first embodiment, except that the camera 17 and the acceleration sensor 18 of the first embodiment are omitted. However, a camera 20 is communicably connected to the projector 10B.

[0104] The camera 20 is installed outside the projector 10B and is configured similarly to the camera 17 of the first embodiment, except that it includes an acceleration sensor 21. The acceleration sensor 21 is configured similarly to the acceleration sensor 21 of the first embodiment, except that it is provided in the camera 20. Here, the acceleration sensor 21 outputs a signal according to the acceleration in the directions along the x-axis, y-axis, and z-axis that are associated with the coordinate system of the optical device 15 of the projector 10B. The camera 20 may be calibrated in advance to specify its positional relationship with the projector 10B, and may be associated with the coordinate system of the optical device 15 of the projector 10B based on the specified positional relationship.

[0105] The above third embodiment also makes it possible to improve the accuracy of correction of the shape of the projected image. In this embodiment, as described above, the acceleration sensor 21 is disposed in the camera 20 that captures the measurement pattern PT. This makes it possible to easily add the functions of the acceleration sensor 21 and the camera 20 to a projector 10B that does not have a built-in acceleration sensor and camera.

[0106] 4. Variations Each of the above-mentioned exemplary embodiments may be modified in various ways. Specific modified embodiments that may be applied to each of the above-mentioned embodiments are illustrated below. Two or more embodiments selected from the following examples may be appropriately combined as long as they are not mutually contradictory.

[0107] 4-1. Variation 1 In the first embodiment described above, the acceleration sensor 18 is provided in the projector 10, but the present invention is not limited to this, and the acceleration sensor 18 may be provided outside the projector 10. However, even in this case, the acceleration sensor 18 is associated with the coordinate system of the optical device 15 of the projector 10. Similarly, in the second embodiment, the acceleration sensor 18 may be provided outside the projector 10A.

[0108] 4-2. Variation 2 In the first embodiment described above, the camera 17 is provided in the projector 10, but this is not limiting, and the camera 17 may be provided outside the projector 10. Similarly, in the second embodiment, the camera 17 may be provided outside the projector 10A.

[0109] 5. Notes The following is a summary of this disclosure.

[0110] (Supplementary Note 1) A correction method including: determining a normal vector of a screen surface; determining a first vector orthogonal to both a gravity vector and the normal vector determined from the output of an acceleration sensor associated with a coordinate system of an optical device of a projector; determining a second vector orthogonal to the first vector on the screen surface; and determining correction parameters for correcting a shape of a projection image projected onto the screen surface based on the first vector and the second vector.

[0111] In the above aspect, by using the gravity vector and the normal vector, it is possible to obtain correction parameters that take into account the inclination of the screen surface with respect to the direction of gravity, thereby improving the accuracy of correction of the shape of the projected image.

[0112] (Supplementary Note 2) The correction method of Supplementary Note 1, further comprising: determining a plane representing the screen surface, and determining the normal vector comprises determining the normal vector from the plane. In the above aspect, even if the screen surface is not strictly flat, the normal vector of the screen surface can be easily determined.

[0113] (Supplementary Note 3) The correction method according to Supplementary Note 2, wherein obtaining the plane includes obtaining the plane based on an image obtained by capturing a measurement pattern projected onto the screen surface. In the above aspect, the plane representing the screen surface can be obtained with high accuracy.

[0114] (Supplementary Note 4) The correction method of Supplementary Note 3, wherein the acceleration sensor is provided in a camera that captures the measurement pattern. In the above aspect, even if a projector does not have a built-in acceleration sensor and camera, the functions of the acceleration sensor and camera can be easily added.

[0115] (Supplementary Note 5) The correction method of Supplementary Note 2, wherein determining the plane includes determining the plane using a time-of-flight sensor. In the above aspect, the plane representing the screen surface can be determined in a short time compared to the aspect using a measurement pattern.

[0116] (Supplementary Note 6) The correction method according to any one of Supplementary Note 1 to Supplementary Note 5, wherein obtaining the correction parameters includes obtaining the correction parameters by using the first vector and the second vector that have been normalized. In the above-mentioned aspect, it is possible to easily adjust the aspect ratio of the corrected shape.

[0117] (Supplementary Note 7) A projector including an optical device and a processing device, wherein the processing device executes the following operations: determining a normal vector of a screen surface; determining a first vector orthogonal to both a gravity vector and the normal vector determined from the output of an acceleration sensor associated with a coordinate system of the optical device; determining a second vector orthogonal to the first vector on the screen surface; and determining correction parameters for correcting a shape of a projection image projected onto the screen surface based on the first vector and the second vector.

[0118] In the above aspect, by using the gravity vector and the normal vector, it is possible to obtain correction parameters that take into account the inclination of the screen surface with respect to the direction of gravity, thereby improving the accuracy of correction of the shape of the projected image.

[0119] (Appendix 8) A program that causes a computer to execute the following operations: determining a normal vector of a screen surface; determining a first vector that is orthogonal to both a gravity vector and the normal vector that are determined from the output of an acceleration sensor associated with the coordinate system of an optical device of a projector; determining a second vector that is orthogonal to the first vector on the screen surface; and determining correction parameters for correcting the shape of a projection image projected onto the screen surface based on the first vector and the second vector.

[0120] In the above aspect, by using the gravity vector and the normal vector, it is possible to obtain correction parameters that take into account the inclination of the screen surface with respect to the direction of gravity, thereby improving the accuracy of correction of the shape of the projected image. [Explanation of symbols]

[0121] 10...projector, 10A...projector, 10B...projector, 11...storage device, 12...processing device, 12a...plane detection unit, 12b...axis calculation unit, 12c...correction value calculation unit, 12d...plane detection unit, 13...communication device, 14...image processing circuit, 15...optical device, 15a...light source, 15b...light modulator, 15c...projection optical system, 16...operation device, 17...camera, 18...acceleration sensor, 19...TOF sensor, 20...camera, 21...acceleration sensor, 100...system, Ex...vector, Ey...vector, Ez...vector, G...gravity vector, H...horizontal vanishing point, IMG...image data, N...normal vector, PA...correction parameter, PR1...program, PR2...program, PT...measurement Pattern, R...matrix, RP...projection area, S10...step, S10A...step, S11...step, S12...step, S12A...step, S13...step, S20...step, S21...step, S22...step, S23...step, S24...step, S30...step, S31...step, S32...step, S33...step, SC...screen surface, SC-0...screen surface, SC-1...screen surface, SC-X...screen surface, V...vertical vanishing point, X...vector (first vector), Y...vector (second vector), h...horizontal axis, s...parameter, t...parameter, v...vertical axis, x...axis, y...axis, θ...pitch angle, φ...yaw angle, ψ...roll angle, ψ'...roll angle.

Claims

1. Determining a normal vector of the screen surface; determining a first vector that is orthogonal to both a gravity vector and the normal vector, the first vector being determined from an output of an acceleration sensor associated with a coordinate system of an optical device of a projector; determining a second vector that is included in the screen plane and is orthogonal to the first vector; determining a correction parameter for correcting a shape of the projection image projected on the screen surface based on the first vector and the second vector. Correction method.

2. determining a plane representing the screen surface; The determination of the normal vector includes: determining the normal vector from the plane; The correction method according to claim 1 .

3. Determining the plane includes: determining the plane based on an image obtained by capturing a measurement pattern projected on the screen surface; The correction method according to claim 2 .

4. the acceleration sensor is disposed in a camera that captures an image of the measurement pattern; The correction method according to claim 3 .

5. Determining the plane includes: determining said plane using a time-of-flight sensor; The correction method according to claim 2 .

6. Obtaining the correction parameter includes: determining the correction parameter using the normalized first and second vectors; The correction method according to claim 1 .

7. An optical device; a processing device; The processing device includes: Determining a normal vector of the screen surface; determining a first vector orthogonal to both a gravity vector determined from an output of an acceleration sensor associated with a coordinate system of the optical device and the normal vector; determining a second vector that is included in the screen plane and is orthogonal to the first vector; and determining a correction parameter for correcting a shape of the projection image projected on the screen surface based on the first vector and the second vector. projector.

8. Determining a normal vector of the screen surface; determining a first vector that is orthogonal to both a gravity vector and the normal vector, the first vector being determined from an output of an acceleration sensor associated with a coordinate system of an optical device of a projector; determining a second vector that is included in the screen plane and is orthogonal to the first vector; and determining a correction parameter for correcting a shape of the projection image projected on the screen surface based on the first vector and the second vector. program.

Images