Correction method, projector and program product

Abstract

The invention provides a correction method, a projector, and a program product to improve the correction accuracy of the shape of a projected image. The correction method comprises the following steps: obtaining a normal vector of a screen surface; a first vector obtaining step for obtaining a first vector orthogonal to both a gravity vector obtained from the output of the acceleration sensor corresponding to the coordinate system of the optical device of the projector and a normal vector; obtaining a second vector orthogonal to the first vector on the screen surface; and obtaining a correction parameter for correcting the shape of the projection image projected on the screen surface on the basis of the first vector and the second vector.

Description

Technical Field

[0001] The invention relates to a calibration method, a projector and a program product. Background Art

[0002] For example, patent document 1 discloses an image projection system, which includes: an estimation unit, which estimates the inclination of a projection display device relative to a projection surface based on a projection azimuth detected by a first azimuth detection unit, a shooting azimuth detected by a second azimuth detection unit, and a captured image obtained by a shooting device; and a correction unit, which corrects the shape of a projection screen based on the inclination of the projection display device estimated by the estimation unit.

[0003] Patent Document 1: International Publication No. 2017 / 169186

[0004] When the projection plane is tilted with respect to the gravity direction, the tilt of the projection plane needs to be considered when accurately calculating a roll rotation angle for correcting the shape of the projection screen. However, Patent Document 1 does not take this into consideration. Summary of the invention

[0005] A correction method according to one embodiment of the present invention includes: obtaining a normal vector of a screen surface; obtaining a first vector orthogonal to both a gravity vector and the normal vector, wherein the gravity vector is obtained based on the output of an acceleration sensor corresponding to a coordinate system of an optical device of a projector; obtaining a second vector included in the screen surface and orthogonal to the first vector; and obtaining correction parameters for correcting the shape of a projected image projected onto the screen surface based on the first vector and the second vector.

[0006] A projector in another embodiment of the present invention includes an optical device and one or more processors, which perform the following processing: calculating a normal vector of a screen surface; calculating a first vector that is orthogonal to both a gravity vector and the normal vector, wherein the gravity vector is calculated based on the output of an acceleration sensor corresponding to a coordinate system of the optical device; calculating a second vector that is included in the screen surface and is orthogonal to the first vector; and calculating correction parameters for correcting the shape of a projected image projected onto the screen surface based on the first vector and the second vector.

[0007] A program product in one embodiment of the present disclosure causes a computer to perform the following processing: calculating a normal vector of a screen surface; calculating a first vector orthogonal to both a gravity vector and the normal vector, wherein the gravity vector is calculated based on the output of an acceleration sensor corresponding to a coordinate system of an optical device of a projector; calculating a second vector contained in the screen surface and orthogonal to the first vector; and calculating correction parameters for correcting the shape of a projected image projected onto the screen surface based on the first vector and the second vector. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 This is a diagram showing an overview of a system used in the calibration method according to the first embodiment.

[0009] Figure 2 This is a block diagram of a projector according to the first embodiment.

[0010] Figure 3 This is a flowchart showing the flow of the calibration method according to the first embodiment.

[0011] Figure 4 It is a diagram for explaining the pitch angle, yaw angle, and roll angle of the projector.

[0012] Figure 5 This is a diagram for explaining the relationship between the posture of the projector and the output of the acceleration sensor.

[0013] Figure 6 This is a diagram for explaining the relationship between the gravity vector and the pitch angle.

[0014] Figure 7 It is a diagram for explaining the relationship between the gravity vector and the roll angle.

[0015] Figure 8 This is a diagram for explaining a roll angle obtained by a conventional method based on the output of an acceleration sensor.

[0016] Fig. 9 This is a diagram for explaining a roll angle obtained by another conventional method based on the output of an acceleration sensor.

[0017] Fig.10 This is a diagram for explaining the relationship between the normal vector of the screen surface and the gravity vector.

[0018] Fig.11 This is a diagram for explaining the three-dimensional coordinates of the screen surface.

[0019] Fig.12 A diagram for explaining a plane representing a screen surface and a normal vector of the screen surface.

[0020] Fig.13This is a diagram for explaining how to obtain a gravity vector.

[0021] Fig.14 It is a diagram for explaining the first vector and the second vector.

[0022] Fig.15 It is a diagram for explaining vectors obtained by normalizing the normal vector of the screen surface, the first vector, and the second vector.

[0023] Fig.16 This is a diagram for explaining an example of keystone correction.

[0024] Fig.17 A diagram for explaining a horizontal vanishing point and a vertical vanishing point in an optical device.

[0025] Fig.18 It is a diagram for explaining determination of a correction shape.

[0026] Fig.19 This is a block diagram of a projector according to the second embodiment.

[0027] Fig. 20 This is a flowchart showing the flow of the calibration method according to the second embodiment.

[0028] Fig.21 This is a block diagram of a projector according to a third embodiment.

[0029] Description of symbols

[0030] 10: Projector; 10A: Projector; 10B: Projector; 11: Storage device; 12: Processing device; 12a: Plane detection unit; 12b: Axis calculation unit; 12c: Correction value calculation unit; 12d: Plane detection unit; 13: Communication device; 14: Image processing circuit; 15: Optical device; 15a: Light source; 15b: Light modulator; 15c: Projection optical system; 16: Operating device; 17: Camera; 18: Acceleration sensor; 19: TOF sensor; 20: Camera; 21: Acceleration sensor; 100: System; Ex: Vector, Ey: Vector; Ez: Vector; G: Gravity vector; H: Horizontal vanishing point; IMG: Image data; N: Normal vector; PA: Correction parameter; PR1: Program ;PR2: program;PT: measurement pattern;R: matrix;RP: projection area;S10: step;S10A: step;S11: step;S12: step;S12A: step;S13: step;S20: step;S21: step;S22: step;S23: step;S24: step;S30: step;S31: step;S32: step;S33: step;SC: screen surface;SC-0: screen surface;SC-1: screen surface;SC-X: screen surface;V: vertical vanishing point;X: vector (first vector);Y: vector (second vector);h: horizontal axis;s: parameter;t: parameter;v: vertical axis;x: axis;y: axis;θ: pitch angle;φ: yaw angle;ψ: roll angle;ψ': roll angle. DETAILED DESCRIPTION

[0031] Hereinafter, preferred embodiments of the present disclosure will be described with reference to the accompanying drawings. In addition, in the accompanying drawings, the size and scale of each part are appropriately different from the actual one, and there are also parts that are schematically shown for easy understanding. In addition, the scope of the present disclosure is not limited to these embodiments unless there is a record of the subject matter of the present disclosure that is particularly limited in the following description.

[0032] 1. First Implementation Method

[0033] 1-1. Overview of system used for calibration method

[0034] Figure 1 FIG. 1 is a diagram showing an overview of a system 100 used in the calibration method of the first embodiment. Figure 1 As shown, system 100 has a projector 10 .

[0035] The projector 10 is a display device that projects an image represented by image information output from a device such as a computer (not shown) onto a screen surface SC. The screen surface SC is the surface of an object such as a screen, and is generally a plane. In addition, the screen surface SC may not be a plane in the strict sense, as long as it can be regarded as a plane.

[0036] Here, the installation posture of the screen surface SC may be different depending on the usage of the system 100. Therefore, the projector 10 corrects the distortion of the projected image caused by the installation posture of the screen surface SC through trapezoidal correction. The projector 10 includes a camera 17 and an acceleration sensor 18, and has a function of measuring the shape of the screen surface SC using the camera 17, a function of obtaining a gravity vector using the acceleration sensor 18, and a function of obtaining a correction parameter for trapezoidal correction based on the shape of the screen surface SC and the gravity vector.

[0037] 1-2. Projector

[0038] Figure 2 FIG. 1 is a block diagram of a projector 10 according to the first embodiment. Figure 2 As shown, the projector 10 includes a storage device 11, a processing device 12, a communication device 13, an image processing circuit 14, an optical device 15, an operating device 16, a camera 17, and an acceleration sensor 18. The storage device 11, the processing device 12, the communication device 13, the image processing circuit 14, and the acceleration sensor 18 are arranged inside a housing (not shown) of the projector 10. They are connected so as to be able to communicate with each other.

[0039] The storage device 11 is a storage device that stores programs executed by the processing device 12 and data processed by the processing device 12. The storage device 11 is configured to include, for example, a hard disk drive or a semiconductor memory. In addition, part or all of the storage device 11 may also be provided in a storage device or server outside the projector 10.

[0040] The storage device 11 stores a program PR1 and correction parameters PA.

[0041] The program PR1 is a program for executing a correction method described in detail later. The correction parameter PA is a parameter indicating the degree of correction in the distortion correction process in the image processing circuit 14, and is generated by the correction value calculation unit 12c described later.

[0042] The processing device 12 is a processing device having the function of controlling each part of the projector 10 and the function of processing various data. The processing device 12 is configured to include a processor such as a CPU (Central Processing Unit). In addition, the processing device 12 can be composed of a single processor or a plurality of processors. In addition, part or all of the functions of the processing device 12 can also be implemented by hardware such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). In addition, the processing device 12 can also be integrated with the image processing circuit 14.

[0043] The communication device 13 is a communication device capable of communicating with various devices, and acquires image data IMG from a device not shown. For example, the communication device 13 is a wired communication device such as a wired LAN (Local Area Network), USB (Universal Serial Bus), HDMI (High Definition Multimedia Interface), a wireless communication device such as LPWA (Low Power Wide Area), a wireless LAN including Wi-Fi, and Bluetooth. "HDMI", "Wi-Fi", and "Bluetooth" are registered trademarks.

[0044] The image processing circuit 14 is a circuit that performs necessary processing on the image data IMG from the communication device 13 and inputs the image data to the optical device 15. The image processing circuit 14 has, for example, a frame memory (not shown), expands the image data IMG in the frame memory, appropriately performs various processes such as resolution conversion processing, size adjustment processing, and distortion correction processing, and then inputs the image data to the optical device 15. Here, the correction parameter PA is used in the distortion correction processing. In addition, the image processing circuit 14 can also perform processing such as OSD (On Screen Display) processing that generates image information for menu display or operation guidance and synthesizes it with the image data IMG as needed.

[0045] The optical device 15 is a device that displays an image by projecting image light onto the projection region RP. The optical device 15 includes a light source 15a, a light modulator 15b, and a projection optical system 15c.

[0046] The light source 15a is configured to include, for example, a halogen lamp, a xenon lamp, an ultra-high pressure mercury lamp, an LED (Light Emitting Diode) or a laser light source, and emits red, green and blue light, respectively. The light modulator 15b is configured to include three light modulation elements arranged corresponding to red, green and blue. Each light modulation element includes, for example, a transmissive liquid crystal panel, a reflective liquid crystal panel or a DMD (digital micromirror device), and generates image light of each color by modulating the light of the corresponding color. The image light of each color generated by the light modulator 15b is synthesized by a color synthesis optical system to become a full-color image light. The projection optical system 15c is an optical system including a projection lens, etc., which images the full-color image light from the light modulator 15b on the screen surface SC and projects it.

[0047] The operating device 16 is a device that receives operations from the user. For example, the operating device 16 includes an operating panel and a remote control light receiving unit (not shown). The operating panel is provided in the outer casing of the projector 10 and outputs a signal based on the operation from the user. The remote control light receiving unit receives an infrared signal from a remote control (not shown), decodes the infrared signal, and outputs a signal based on the operation of the remote control. In addition, the operating device 16 is provided as needed and may be omitted.

[0048] The camera 17 is a digital camera having an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor).

[0049] The acceleration sensor 18 is a sensor that detects acceleration applied to the projector 10. The acceleration sensor 18 is built into the projector 10 and fixed to a predetermined position in the housing of the projector 10. The predetermined position is, for example, a circuit substrate (not shown) on which the processing device 12 is mounted. The acceleration sensor 18 outputs a signal corresponding to acceleration in the direction of each axis of the x-axis, y-axis, and z-axis described later corresponding to the coordinate system of the optical device 15 of the projector 10. The acceleration sensor 18 is fixed to a predetermined position in the housing of the projector 10, thereby determining the position in the housing. That is, the relative positional relationship between the acceleration sensor 18 and the optical device 15 is predetermined. Thus, the acceleration sensor 18 corresponds to the coordinate system of the optical device 15.

[0050] In the above projector 10, the processing device 12 functions as a plane detection unit 12a, an axis calculation unit 12b, and a correction value calculation unit 12c by executing the program PR1 stored in the storage device 11. Therefore, the processing device 12 includes the plane detection unit 12a, the axis calculation unit 12b, and the correction value calculation unit 12c.

[0051] The plane detection unit 12a obtains the plane representing the screen surface SC by controlling the operations of the optical device 15 and the camera 17. Specifically, the plane detection unit 12a obtains the plane representing the screen surface SC based on an image obtained by capturing a measurement pattern PT described later projected onto the screen surface SC.

[0052] The axis calculation unit 12b obtains the axis x and the axis y described later as the coordinate axes of the screen surface SC based on the plane obtained by the plane detection unit 12a and the output of the acceleration sensor 18. Specifically, the axis calculation unit 12b obtains the normal vector N described later of the screen surface SC based on the plane obtained by the plane detection unit 12a. In addition, the axis calculation unit 12b obtains the vector X described later that is orthogonal to both the gravity vector G described later and the normal vector N obtained based on the output of the acceleration sensor 18. The vector X is an example of a "first vector". In addition, the axis calculation unit 12b obtains the vector Y described later that is orthogonal to the vector X in the screen surface SC. The vector Y is an example of a "second vector". The vector Y is a vector included in the screen surface SC.

[0053] The correction value calculation unit 12 c obtains a correction parameter PA for correcting the shape of a projection image projected on the screen surface SC based on a vector X and a vector Y (described later) obtained by the axis calculation unit 12 b .

[0054] 1-3. Calibration method

[0055] Figure 3 1 is a flowchart showing the flow of the calibration method according to the first embodiment. This calibration method is executed by the projector 10 described above.

[0056] like Figure 3 As shown, the calibration method of this embodiment includes step S10, step S20 and step S30 in sequence.

[0057] In step S10 , the plane detection unit 12 a controls the operations of the optical device 15 and the camera 17 to obtain a plane representing the screen surface SC.

[0058] Specifically, step S10 includes step S11, step S12, and step S13 in sequence. In step S11, the plane detection unit 12a captures the measurement pattern PT projected on the screen surface SC. In step S12, the plane detection unit 12a obtains the coordinates on the screen surface SC based on the image captured in step S11. In step S13, the plane detection unit 12a obtains the plane representing the screen surface SC based on the coordinates obtained in step S12.

[0059] In step S20 , the axis calculation unit 12 b obtains an axis x and an axis y, which will be described later, as coordinate axes of the screen surface SC based on the plane obtained by the plane detection unit 12 a and the output of the acceleration sensor 18 .

[0060] Specifically, step S20 includes step S21, step S22, step S23, and step S24 in sequence. In step S21, the axis calculation unit 12b calculates a normal vector N of the screen surface SC, which will be described later, based on the plane calculated in step S10. In step S22, the axis calculation unit 12b obtains a gravity vector G, which will be described later, calculated based on the output of the acceleration sensor 18. In step S23, the axis calculation unit 12b calculates a vector X, which will be described later, which is orthogonal to both the gravity vector G and the normal vector N obtained in step S22. In step S23, the axis calculation unit 12b calculates a vector Y, which will be described later, which is orthogonal to the vector X on the screen surface SC.

[0061] In step S30 , the correction value calculation unit 12 c obtains a correction parameter PA for correcting the shape of the projection image projected on the screen surface SC based on the vector X and the vector Y (described later) obtained in step S20 .

[0062] Specifically, step S30 includes step S31, step S32, and step S33 in sequence. In step S31, the correction value calculation unit 12c obtains a matrix R, which will be described later, for converting from the coordinate system of the optical device 15 of the projector 10 to the coordinate system on the screen surface SC. In step S32, the correction value calculation unit 12c calculates the correction shape using the matrix R obtained in step S31. In step S33, the correction value calculation unit 12c obtains the correction parameter PA based on the correction shape obtained in step S32.

[0063] Figure 4 is used to explain the pitch angle θ and the yaw angle of the projector 10. And the graph of the roll angle ψ. Figure 4 As shown, in the projector 10, the x-axis, y-axis, and z-axis are set as three axes that are orthogonal to each other. The x-axis is an axis along the width direction of the projector 10. The y-axis is an axis along the height direction of the projector 10. The z-axis is an axis along the projection direction of the projector 10. The z-axis is an example of the "first axis" which is the optical axis of the projection lens. The y-axis is an example of the "second axis" that is orthogonal to the first axis. The x-axis is an example of the "third axis" that is orthogonal to both the first axis and the second axis.

[0064] Here, the posture of the projector 10 is represented by the pitch angle θ which is the rotation angle of the projector 10 around the x-axis, the yaw angle θ which is the rotation angle of the projector 10 around the y-axis, and the pitch angle θ which is the rotation angle of the projector 10 around the y-axis. The rotation angle of the projector 10 around the z-axis is represented by a roll angle ψ.

[0065] exist Figure 4In the example shown, the y-axis is parallel to the direction of gravity. Therefore, the x-axis and the z-axis are parallel to the horizontal plane. In addition, the horizontal axis h along the lateral direction of the screen surface SC is parallel to the horizontal plane, and the vertical axis v along the longitudinal direction of the screen surface SC is parallel to the direction of gravity. Here, when the z-axis is orthogonal to the screen surface SC, the x-axis is parallel to the horizontal axis h, and the y-axis is parallel to the vertical axis v.

[0066] Figure 5 1 is a diagram for explaining the relationship between the posture of the projector 10 and the output of the acceleration sensor 18. The acceleration sensor 18 is a three-axis acceleration sensor, and outputs signals corresponding to accelerations in the directions along the x-axis, y-axis, and z-axis, respectively. Such three-axis outputs from the acceleration sensor 18 represent the three components of the gravity vector G applied to the projector 10 when the projector 10 is installed.

[0067] Figure 6 It is a diagram for explaining the relationship between the gravity vector G and the pitch angle θ. Figure 7 1 is a diagram for explaining the relationship between the gravity vector G and the roll angle ψ. When the screen surface SC is perpendicular to the plane with the gravity vector G as the normal vector, Figure 6 As shown in FIG. 1 , the pitch angle θ can be obtained based on the output values ​​(Gx, Gy, Gz) of the acceleration sensor 18, that is, based on the gravity vector G. In addition, in this case, Figure 7 As shown, the roll angle ψ can be obtained based on the output values ​​(Gx, Gy, Gz) of the acceleration sensor 18, that is, based on the gravity vector G.

[0068] However, due to the yaw angle Since it is independent of the gravity vector G, the yaw angle cannot be calculated from the output values ​​(Gx, Gy, Gz) of the acceleration sensor 18.

[0069] Figure 8 1 is a diagram for explaining the roll angle ψ obtained by a conventional method based on the output of the acceleration sensor 18. Figure 8 In the figure, the screen plane SC that is perpendicular to the plane with the gravity vector G as the normal vector is represented by a dotted line as screen plane SC-0, and the screen plane SC that is not perpendicular to the plane with the gravity vector G as the normal vector is represented by a solid line as screen plane SC-X.

[0070] like Figure 8As shown in FIG. 1 , the roll angle ψ of the image on the actual screen surface SC-X is different from the roll angle ψ' obtained from the output value of the acceleration sensor 18 as described above. Here, the roll angle ψ of the image on the screen surface SC-X is the angle formed by the vertical axis v of the screen surface SC-X and the y axis of the projector 10. In this way, when the screen surface SC is not perpendicular to the plane with the gravity vector G as the normal vector, there is a problem of generating a difference between the roll angle ψ and the roll angle ψ'. In addition, although not shown in the figure, the same problem also exists with respect to the pitch angle θ.

[0071] Fig. 9 This is a diagram for explaining the roll angle ψ obtained by another conventional method based on the output of the acceleration sensor 18. This other conventional method uses a TOF (Time Of Flight) sensor to directly measure the relative posture relationship between the projector 10 and the screen surface SC. By using this measurement result, the correct pitch angle θ and yaw angle can be obtained. However, this measurement result does not include information on the roll angle ψ. Therefore, it is necessary to obtain the roll angle ψ based on the output value of the acceleration sensor 18.

[0072] As described above, in the screen surface SC-0 perpendicular to the plane having the gravity vector G as the normal vector, the angle formed by the y-axis of the projector 10 and the vertical axis v of the screen surface SC-0 is consistent with the roll angle ψ obtained from the output of the acceleration sensor 18. Therefore, first, the y-axis of the projector 10 is set as the temporary y-axis of the screen surface SC-0, and the temporary y-axis is rotated by the amount of the roll angle ψ' obtained from the output of the acceleration sensor 18. Thus, after the y-axis of the screen surface SC-0 after the roll correction is obtained, the axis perpendicular to the y-axis after the roll correction can be obtained as the x-axis of the screen surface SC-0.

[0073] However, in the screen surface SC-X which is not perpendicular to the plane having the gravity vector G as the normal vector, the angle formed by the y-axis of the projector 10 and the vertical axis v of the screen surface SC-X does not match the roll angle ψ obtained from the output of the acceleration sensor 18. Therefore, in this other method, the correct x-axis and y-axis of the screen surface SC-X cannot be obtained. As a result, an image rotated in the roll direction within the surface of the screen surface SC-X is projected.

[0074] Fig.10 1 is a diagram for explaining the relationship between the normal vector N of the screen surface SC-X and the gravity vector G. Fig.10 As shown, based on the normal vector N of the screen surface SC-X and the gravity vector G, the vector X representing the x-axis and the vector Y representing the y-axis on the screen surface SC-X can be obtained. The vector X is an example of the "first vector". The vector Y is an example of the "second vector".

[0075] Here, the x-axis of the screen plane SC-X is determined so that the vector X representing the x-axis is perpendicular to the gravity vector G. In addition, the y-axis of the screen plane SC-X is determined so that the vector Y representing the y-axis, the normal vector N of the screen plane SC-X, and the gravity vector G enter the same plane.

[0076] More specifically, for example, the vector X and the vector Y can be obtained from the relationship of X=N×G, Y=N×X=N×(N×G) using the outer product of the vectors.

[0077] By obtaining the x-axis and the y-axis on the screen surface SC-X in this manner, it is possible to correct the rolling direction of the image regardless of the posture of the projector 10 or the posture of the screen surface SC.

[0078] Fig.11 It is a diagram for explaining the three-dimensional coordinates (Xs, Ys, Zs) of the screen surface SC-X. In the above-mentioned step S11, the plane detection unit 12a controls the operation of the projector 10 to sequentially project a plurality of measurement patterns PT onto the screen surface SC, and controls the operation of the camera 17 to capture each measurement pattern PT projected onto the screen surface SC. Thus, a plurality of captured images obtained by capturing the measurement pattern PT by the camera 17 are obtained.

[0079] As the measurement pattern PT, for example, a binary code pattern is used. The binary code pattern refers to an image used to express the coordinates of the display device using binary code. Binary code refers to a technology that expresses the value of each digit when an arbitrary numerical value is expressed in binary by turning a switch on and off. When a binary code pattern is used as the measurement pattern PT, the image projected by the projector 10 is equivalent to the switch, and a measurement pattern PT of the number of digits of the binary number representing the coordinate value is required. In addition, different measurement patterns PT are required for the longitudinal coordinate and the lateral coordinate, respectively. For example, when the resolution (number of pixels) of the optical device 15 of the projector 10 is 120×90, 120 and 90 are respectively expressed by 7-bit binary numbers, so 7 images are required to express the longitudinal coordinate, and 7 images are required to express the lateral coordinate.

[0080] In addition, when a binary code pattern is used as the measurement pattern PT, the robustness of the measurement is generally reduced due to the influence of interference light such as lighting. Therefore, when a binary code pattern is used as the measurement pattern PT, from the perspective of suppressing the influence of interference light and improving the robustness of the measurement, it is preferred to use a complementary pattern. A complementary pattern refers to an image after black and white are reversed.

[0081] In addition, the measurement pattern PT is not limited to the binary code pattern, and may be other structured light such as a dot pattern, a rectangular pattern, a polygonal pattern, a checkered pattern, a Gray code pattern, a phase shift pattern, or a random dot pattern.

[0082] In the above-mentioned step S12, the plane detection unit 12a measures the screen surface SC based on the plurality of captured images. Here, in step S12, after the correspondence relationship between the coordinates (Xc, Yc) of the coordinate system of the captured image of the camera 17 and the coordinates (Xp, Yp) in the coordinate system of the optical device 15 of the projector 10 is obtained, the three-dimensional coordinates of each part of the measurement pattern PT projected on the screen surface SC-X are obtained based on the correspondence relationship. Thus, for the coordinates (Xp, Yp) of each point in the coordinate system of the optical device 15, the three-dimensional coordinates (Xs, Ys, Zs) of each point on the screen surface SC-X to which the point is projected are obtained.

[0083] Fig.12 1 is a diagram for explaining a plane representing the screen surface SC-X and a normal vector N of the screen surface SC-X. Fig.12 As shown, in the aforementioned step S12, the coordinates (Xs0, Ys0, Zs0), (Xs1, Ys1, Zs1), ..., (Xsn, Ysn, Zsn) of n points on the screen surface SC-X projected with the measurement pattern PT are calculated as three-dimensional coordinates (Xs, Ys, Zs).

[0084] In step S13, the plane detection unit 12a obtains an equation of a plane passing through the n points: aX+bY+cZ=1, thereby obtaining an equation representing the plane of the screen surface SC-X.

[0085] As described above, in step S10, the plane representing the screen surface SC-X is obtained. After step S10, step S20 is executed.

[0086] In step S20, first, in step S21, the axis calculation unit 12b obtains the normal vector N=(Nx, Ny, Nz)=(a, b, c) of the screen surface SC-X based on the equation obtained in step S13.

[0087] Fig.13 is a diagram for explaining how to obtain the gravity vector G. In step S22, Fig.13 As shown, the axis calculation unit 12b obtains the gravity vector G=(Gx, Gy, Gz) applied to the projector 10 based on the output value of the acceleration sensor 18. Here, since the same gravitational acceleration is also applied to the screen surface SC-X, the gravity vector of the screen surface SC-X is also G=(Gx, Gy, Gz).

[0088] Fig.14 is a diagram for explaining vector X and vector Y. In step S23, as Fig.13As shown, the axis calculation unit 12b obtains a vector X that is orthogonal to both the normal vector N = (Nx, Ny, Nz) of the screen surface SC-X and the gravity vector G = (Gx, Gy, Gz). Here, for example, the vector X = N × G is obtained by the outer product of the normal vector N and the gravity vector G. In addition, the method of calculating the vector X is not limited to the method using the outer product, and for example, the method of obtaining the vector X from the intersection of the plane orthogonal to the normal vector N and the plane orthogonal to the gravity vector G may be used.

[0089] In step S24, the axis calculation unit 12b obtains a vector Y that is orthogonal to both the normal vector N and the vector X. Here, for example, by taking the outer product of the normal vector N and the vector X, a vector Y=N×X=N×(N×G) is obtained.

[0090] By obtaining the vectors X and Y as described above, in step S30, using the vectors X and Y, it is possible to perform accurate correction in the roll direction so that the x-axis coincides with the horizontal axis h and the y-axis coincides with the vertical axis v on the screen surface SC-X.

[0091] Fig.15 1 is a diagram for explaining vectors Ex, Ey, and Ez after normalization of the normal vector N, vector X, and vector Y of the screen surface SC-X. In step S31, the correction value calculation unit 12c first normalizes the vector X=(Xx, Xy, Xz) representing the x-axis of the screen surface SC-X, the vector Y=(Yx, Yy, Yz) representing the y-axis, and the normal vector N=(Nx, Ny, Nz) to a length of 1, thereby obtaining vectors Ex, Ey, and Ez as a standard orthogonal basis on the screen surface SC-X after appropriate roll correction with respect to gravity.

[0092] If a 3×3 matrix obtained by transposing and horizontally arranging the three vectors Ex, Ey, and Ez is denoted by R, the matrix R is expressed as follows, and becomes a rotation matrix from the coordinate system of the optical device 15 of the projector 10 to the coordinate system on the screen surface SC-X.

[0093]

[0094] In this way, in step S31, the correction value calculation unit 12c obtains the matrix R which is a rotation matrix using the three vectors Ex, Ey, and Ez.

[0095] When the standard basis vectors of the coordinate system of the optical device 15 of the actual projector 10 are assumed to be e_X=(1, 0, 0), e_Y=(0, 1, 0), and e_Z=(0, 0, 1), they are transformed using the matrix R and converted into vectors Ex, Ey, and Ez as the standard orthogonal basis on the screen surface SC-X as shown in the following formula.

[0096]

[0097] Thus, the matrix R is a rotation matrix representing the rotation of the screen surface SC-X after accurate roll correction as viewed from the projector 10 in the projection direction.

[0098] By obtaining the matrix R as described above, in step S32, the correction shape SH can be calculated using a known keystone correction method using the matrix R. An example of calculation of the distortion correction shape in step S32 will be described below.

[0099] Fig.16 FIG. 1 is a diagram for explaining an example of keystone correction. Fig.16 1 shows a method of using the coordinates of a vanishing point on the optical device 15 as an example of keystone correction. The vanishing point refers to the coordinates on the optical device 15 when the projector 10 observes a point at infinity in the vertical and horizontal directions on the screen surface SC-X.

[0100] The matrix R represents the rotation of the screen plane SC-X relative to the projector 10. When the coordinates of the origin of the screen plane SC-X are assumed to be (0, 0, 1 / c), an arbitrary point P on the screen plane SC-X is represented by the following equation using two parameters s and t.

[0101]

[0102] At this point P, the point where s→∞ is the point at infinity horizontally, and the point where t→∞ is the point at infinity vertically.

[0103] Fig.17 It is a diagram for explaining the horizontal vanishing point and the vertical vanishing point in the optical device 15 .

[0104] A point p on the optical device 15 corresponding to the point P is expressed by the following equation.

[0105]

[0106] The coordinate system of the optical device 15 is not a pixel coordinate system, but a coordinate system called the standard coordinate system of the pinhole camera model. It is a coordinate system that takes the emission point of the projected light as the origin and extracts the two components (X, Y) of the coordinates (X, Y, 1) on the plane when the optical device 15 is regarded as plane Z=1.

[0107] The vanishing point on the optical device 15 is obtained by projecting a point at infinity in real space onto the light modulator 15b of the optical device 15. Therefore, as shown in the following equation, at point p in the coordinate system of the optical device 15, s→∞ is the horizontal vanishing point H∞, and t→∞ is the vertical vanishing point V∞.

[0108]

[0109] Fig.18 SH is a diagram for explaining the determination of the correction shape SH. A straight line parallel to the x-axis in real space (i.e., a horizontal straight line) has the property of passing through the horizontal vanishing point H∞ on the optical device 15. Similarly, a straight line parallel to the y-axis in real space (i.e., a vertical straight line) has the property of passing through the vertical vanishing point V∞ on the optical device 15. In addition, these properties also hold true in reverse.

[0110] Therefore, in step S32 , the correction value calculation unit 12 c obtains the upper and lower sides as straight lines passing through the horizontal vanishing point H∞, and obtains the left and right sides as straight lines passing through the vertical vanishing point V∞, thereby obtaining the correction shape SH on the optical device 15 .

[0111] Here, there are various correction shapes SH whose upper and lower sides respectively pass through the horizontal vanishing point H∞ and whose left and right sides respectively pass through the vertical vanishing point V∞. For example, the correction shape SH is selected according to the functional purpose, the condition of maximizing the area, the condition of left alignment, right alignment, top alignment or bottom alignment on the optical device 15, the condition of achieving a specified aspect ratio on the screen surface SC-X projection plane, and the like to determine the correction shape SH.

[0112] In step S33, the correction value calculation unit 12c obtains the correction parameter PA based on the correction shape SH obtained in step S32. The obtained correction parameter PA is stored in the storage device 11 as described above, and is used for the trapezoidal correction in the image processing circuit 14. Thus, a rectangular image that has been subjected to the roll correction so that the y-axis is parallel to the gravity direction is projected onto the screen surface SC-X.

[0113] As described above, the correction method of the present embodiment includes step S21, step S23, step S24, and step S30. As described above, step S21 obtains the normal vector N of the screen surface SC. Step S23 obtains the vector X that is orthogonal to both the gravity vector G obtained based on the output of the acceleration sensor 18 corresponding to the coordinate system of the optical device 15 of the projector 10 and the normal vector N. Vector X is an example of a "first vector". Step S24 obtains the vector Y that is included in the screen surface SC and is orthogonal to the vector X. Vector Y is an example of a "second vector". In step S30, a correction parameter PA for correcting the shape of the projected image projected onto the screen surface SC is obtained based on the vector X and the vector Y.

[0114] Here, the calibration method of the present embodiment is performed using the projector 10 including the optical device 15 and the processing device 12. The processing device 12 executes step S21, step S23, step S24, and step S30.

[0115] The display method of the present embodiment is implemented by executing the program PR1 by the processing device 12, which is an example of a “computer.” The program PR1 causes the processing device 12 to execute step S21, step S23, step S24, and step S30.

[0116] In the above correction method, projector 10 or program PR1, correction parameters PA can be obtained that take into account the inclination of the screen surface SC with respect to the gravity direction by using the gravity vector G and the normal vector N. As a result, the correction accuracy of the shape of the projected image can be improved.

[0117] As described above, the calibration method of this embodiment includes step S10. Step S10 obtains a plane representing the screen surface SC. Then, step S21 obtains a normal vector N from the plane of the screen surface SC. Thus, even if the screen surface SC is not a strict plane, the normal vector N of the screen surface SC can be easily obtained.

[0118] As described above, step S10 includes step S13. In step S13, the plane of the screen surface SC is obtained based on the image obtained by capturing the measurement pattern PT projected onto the screen surface SC. Thus, the plane representing the screen surface SC can be obtained with high accuracy.

[0119] Then, as described above, in step S30, the correction parameter PA is obtained using the normalized vector X and vector Y. This makes it possible to easily adjust the aspect ratio of the correction shape SH.

[0120] 2. Second Implementation Method

[0121] In the following exemplary embodiments, elements having the same functions and effects as those of the first embodiment are assigned the same reference numerals as those used in the description of the first embodiment, and detailed descriptions thereof are appropriately omitted.

[0122] Fig.19 This is a block diagram of a projector 10A according to the second embodiment. The projector 10A has the same configuration as the projector 10 according to the first embodiment except that it includes a TOF sensor 19 instead of the camera 17 according to the first embodiment and uses a program PR2 instead of the program PR1 according to the first embodiment.

[0123] The TOF sensor 19 is a time-of-flight sensor, and measures the shape of the screen surface SC. The output of the TOF sensor 19 represents the three-dimensional coordinates of the screen surface SC.

[0124] The processing device 12 of this embodiment functions as a plane detection unit 12d, an axis calculation unit 12b, and a correction value calculation unit 12c by executing the program PR2 stored in the storage device 11. Therefore, the processing device 12 of this embodiment includes the plane detection unit 12d, the axis calculation unit 12b, and the correction value calculation unit 12c.

[0125] The plane detector 12d obtains a plane representing the screen surface SC based on the output of the TOF sensor 19. The axis calculator 12b of this embodiment obtains the axis x and the axis y as coordinate axes of the screen surface SC based on the plane obtained by the plane detector 12d and the output of the acceleration sensor 18.

[0126] Fig. 20 1 is a flowchart showing the flow of the calibration method according to the second embodiment. This calibration method is executed by the projector 10A described above.

[0127] like Fig. 20 As shown, the calibration method of this embodiment is the same as the calibration method of the first embodiment except that step S10A is included instead of step S10 of the first embodiment. Step S10A is the same as step S10 of the first embodiment except that step S12A is included instead of steps S11 and S12 of the first embodiment.

[0128] In step S12A, the plane detection unit 12d measures the shape of the screen surface SC using the TOF sensor 19, and obtains the coordinates on the screen surface SC based on the output of the TOF sensor 19. In step S13 of the present embodiment, the plane detection unit 12d obtains the plane representing the screen surface SC based on the coordinates obtained in step S12A.

[0129] According to the second embodiment described above, the correction accuracy of the shape of the projected image can also be improved. As described above, the correction method of this embodiment includes step S10A, and step S10A uses the TOF sensor 19, which is a sensor of the time of flight method, to obtain the plane of the screen surface SC. As a result, compared with the method using the measurement pattern PT as in the first embodiment, the plane representing the screen surface SC can be obtained in a short time.

[0130] 3. Third Implementation Method

[0131] In the following exemplary embodiments, elements having the same functions and effects as those of the first embodiment are assigned the same reference numerals as those used in the description of the first embodiment, and detailed descriptions thereof are appropriately omitted.

[0132] Fig.21This is a block diagram of a projector 10B according to the third embodiment. The projector 10B has the same configuration as the projector 10 according to the first embodiment, except that the camera 17 and the acceleration sensor 18 according to the first embodiment are omitted. However, the camera 20 is connected to the projector 10B in a communicable manner.

[0133] The camera 20 is configured similarly to the camera 17 of the first embodiment except that it is provided outside the projector 10B and includes an acceleration sensor 21. The acceleration sensor 21 is configured similarly to the acceleration sensor 18 of the first embodiment except that it is provided in the camera 20. Here, the acceleration sensor 21 outputs a signal corresponding to the acceleration along the direction of each axis of the x-axis, y-axis, and z-axis corresponding to the coordinate system of the optical device 15 of the projector 10B. The camera 20 may be calibrated in advance to determine the positional relationship with the projector 10B, and correspond to the coordinate system of the optical device 15 of the projector 10B based on the determined positional relationship.

[0134] According to the third embodiment described above, the correction accuracy of the shape of the projected image can also be improved. In this embodiment, as described above, the acceleration sensor 21 is arranged in the camera 20 that captures the measurement pattern PT. Thus, even if the projector 10B does not have an acceleration sensor and a camera built in, the functions of the acceleration sensor 21 and the camera 20 can be easily added.

[0135] 4. Modifications

[0136] The above-exemplified embodiments can be variously modified. The following examples illustrate specific modified embodiments that can be applied to the above-mentioned embodiments. Two or more embodiments arbitrarily selected from the following examples can be appropriately combined within the range that they do not contradict each other.

[0137] 4-1. Modification 1

[0138] In the first embodiment described above, the acceleration sensor 18 is provided in the projector 10, but the present invention is not limited thereto, and the acceleration sensor 18 may be provided outside the projector 10. However, in this case, the acceleration sensor 18 also corresponds to the coordinate system of the optical device 15 of the projector 10. Similarly, in the second embodiment, the acceleration sensor 18 may also be provided outside the projector 10A.

[0139] 4-2. Modification 2

[0140] In the first embodiment, the camera 17 is provided in the projector 10, but the present invention is not limited thereto, and the camera 17 may be provided outside the projector 10. Similarly, in the second embodiment, the camera 17 may be provided outside the projector 10A.

[0141] 5. Notes

[0142] The following is a summary of the present disclosure.

[0143] (Supplementary Note 1) A correction method, comprising: calculating a normal vector of a screen surface; calculating a first vector orthogonal to both a gravity vector and the normal vector, wherein the gravity vector is calculated based on the output of an acceleration sensor corresponding to a coordinate system of an optical device of a projector; calculating a second vector orthogonal to the first vector in the screen surface; and calculating correction parameters for correcting the shape of a projected image projected onto the screen surface based on the first vector and the second vector.

[0144] In the above-described embodiment, by using the gravity vector and the normal vector, it is possible to obtain a correction parameter that takes into account the inclination of the screen surface with respect to the gravity direction. As a result, it is possible to improve the correction accuracy of the shape of the projected image.

[0145] (Supplementary Note 2) The correction method according to Supplementary Note 1, further comprising: obtaining a plane representing the screen surface, and obtaining the normal vector comprises: obtaining the normal vector based on the plane. In the above manner, even if the screen surface is not a strict plane, the normal vector of the screen surface can be easily obtained.

[0146] (Supplementary Note 3) The calibration method according to Supplementary Note 2, wherein obtaining the plane includes obtaining the plane based on an image obtained by photographing the measurement pattern projected onto the screen surface. In the above aspect, the plane representing the screen surface can be obtained with high accuracy.

[0147] (Supplementary Note 4) The calibration method according to Supplementary Note 3, wherein the acceleration sensor is provided in a camera that captures the measurement pattern. In the above embodiment, even a projector that does not have an acceleration sensor and a camera built in can easily add the functions of the acceleration sensor and the camera.

[0148] (Supplementary Note 5) The calibration method according to Supplementary Note 2, wherein obtaining the plane includes obtaining the plane using a time-of-flight sensor. In the above method, the plane representing the screen surface can be obtained in a shorter time than the method using a measurement pattern.

[0149] (Supplement 6) The correction method according to any one of Supplements 1 to 5, wherein obtaining the correction parameter includes obtaining the correction parameter using the normalized first vector and the second vector. In the above embodiment, the aspect ratio of the correction shape can be easily adjusted.

[0150] (Supplementary Note 7) A projector comprises an optical device and a processing device, wherein the processing device performs the following processing: calculating a normal vector of a screen surface; calculating a first vector orthogonal to both a gravity vector and the normal vector, wherein the gravity vector is calculated based on the output of an acceleration sensor corresponding to a coordinate system of the optical device; calculating a second vector orthogonal to the first vector in the screen surface; and calculating correction parameters for correcting the shape of a projected image projected onto the screen surface based on the first vector and the second vector.

[0151] In the above-described embodiment, by using the gravity vector and the normal vector, it is possible to obtain a correction parameter that takes into account the inclination of the screen surface with respect to the gravity direction. As a result, it is possible to improve the correction accuracy of the shape of the projected image.

[0152] (Supplementary Note 8) A program that causes a computer to perform the following processing: calculating a normal vector of a screen surface; calculating a first vector that is orthogonal to both a gravity vector and the normal vector, wherein the gravity vector is calculated based on the output of an acceleration sensor corresponding to a coordinate system of an optical device of a projector; calculating a second vector that is orthogonal to the first vector in the screen surface; and calculating a correction parameter for correcting the shape of a projected image projected onto the screen surface based on the first vector and the second vector.

[0153] In the above-described embodiment, by using the gravity vector and the normal vector, it is possible to obtain a correction parameter that takes into account the inclination of the screen surface with respect to the gravity direction. As a result, it is possible to improve the correction accuracy of the shape of the projected image.

Claims

1. A calibration method, comprising: Find the normal vector of the screen surface; Calculating a first vector orthogonal to both the gravity vector and the normal vector, the gravity vector being calculated based on the output of an acceleration sensor corresponding to a coordinate system of an optical device of the projector; Calculating a second vector that is contained in the screen plane and is orthogonal to the first vector; and Based on the first vector and the second vector, a correction parameter for correcting the shape of the projection image projected onto the screen surface is obtained.

2. The calibration method according to claim 1, wherein: The calibration method further comprises: obtaining a plane representing the screen surface, Obtaining the normal vector includes: obtaining the normal vector according to the plane.

3. The calibration method according to claim 2, wherein: Finding the plane includes finding the plane based on an image obtained by photographing the measurement pattern projected onto the screen surface.

4. The calibration method according to claim 3, wherein: The acceleration sensor is disposed on a camera that captures an image of the measurement pattern.

5. The calibration method according to claim 2, wherein: Determining the plane includes determining the plane using a time-of-flight sensor.

6. The calibration method according to claim 1, wherein: Obtaining the correction parameter includes obtaining the correction parameter using the normalized first vector and the second vector.

7. A projector, comprising: Optical devices; as well as one or more processors, The one or more processors perform the following processing: Find the normal vector of the screen surface; Calculating a first vector that is orthogonal to both a gravity vector and the normal vector, wherein the gravity vector is calculated based on an output of an acceleration sensor corresponding to a coordinate system of the optical device; Calculating a second vector that is contained in the screen plane and is orthogonal to the first vector; and Based on the first vector and the second vector, a correction parameter for correcting the shape of the projection image projected onto the screen surface is obtained.

8. A program product that causes a computer to execute the following processing: Find the normal vector of the screen surface; Calculating a first vector orthogonal to both a gravity vector and the normal vector, wherein the gravity vector is calculated based on an output of an acceleration sensor corresponding to a coordinate system of an optical device of the projector; Calculating a second vector that is contained in the screen plane and is orthogonal to the first vector; and Based on the first vector and the second vector, a correction parameter for correcting the shape of the projection image projected onto the screen surface is obtained.

Images