Determining method of projection area where image is projected, and projection device

Abstract

To determine a projection area of an image on a projection surface so as to effectively use a drawing area of a panel of a projector.SOLUTION: A determining method of a projection area PAR, includes steps of: specifying coordinates of four vertices P10s, P11s, P12s, and P13s, which correspond one-to-one to four vertices P10p, P11p, P12p, and P13p of a rectangle RE10p defining a drawing area DAR1 in a panel PL, in a screen coordinate system ΣS of a projection surface SC; identifying a first intersection point coordinate which is a coordinate in the screen coordinate system ΣS of a first intersection point PI1s at which two diagonal lines of a quadrangle QU10s having vertices P10s, P11s, P12s, and P13s as vertices intersect with each other; identifying a largest rectangle which falls within a range of the quadrangle QU10s and has a first aspect ratio centered on the first intersection point coordinate as a first rectangle RE1s; and determining a projection area PAR on the projection surface SC based on the first rectangle RE1s.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The present invention relates to a method for determining a projection area onto which an image is projected, and to a projection device. [Background technology]

[0002] A projector has, for example, a panel including a drawing area in which a plurality of pixels that emit light based on an image signal are arranged. This type of projector projects an image drawn in the drawing area of ​​the panel onto a projection surface. In the technical field of projectors, there are known techniques for correcting distortion of a projected image when projecting the image. For example, Patent Document 1 discloses a keystone distortion correction method that can project an image without distortion of the pattern, even when projecting an image with an evenly balanced pattern on both sides. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-259082 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with conventional correction methods, there is a possibility that the image after distortion correction will not make full use of the drawing area of ​​the panel that the projector has. For this reason, it is desirable to determine the projection area of ​​the image on the projection surface so that the drawing area of ​​the panel that the projector has can be used effectively. [Means for solving the problem]

[0005] In order to solve the above problems, the projection area determination method of the present invention is a method for determining a projection area onto which an image drawn in a drawing area of ​​a panel provided in a projection device is projected, and includes the steps of: identifying coordinates in a first coordinate system, which is the coordinate system of the projection surface when the projection surface is viewed from the normal direction of the projection surface onto which the image is projected, for each of four vertices of a first rectangle that defines the drawing area in the coordinate system of the panel; identifying first intersection coordinates, which are the coordinates in the first coordinate system of the point where two diagonals of a second rectangle corresponding to the first rectangle and having four vertices that correspond one-to-one with the four vertices, based on the coordinates of each of the four vertices of the second rectangle; identifying as a first rectangle the largest rectangle that fits within the range of the second rectangle and has a first aspect ratio with the first intersection coordinates as its center coordinate; and determining the projection area on the projection surface based on the first rectangle.

[0006] In addition, a projection device according to the present invention comprises a panel including a drawing area in which an image is drawn, and a processing circuit, wherein the processing circuit performs the following operations: identifying the coordinates in a first coordinate system, which is the coordinate system of the projection surface when the projection surface is viewed from the normal direction of the projection surface onto which the image is projected, of each of the four vertices of a first rectangle that defines the drawing area in the coordinate system of the panel; identifying first intersection coordinates, which are the coordinates in the first coordinate system of the point where two diagonals of a second rectangle corresponding to the first rectangle and having four vertices that correspond one-to-one with the four vertices, based on the coordinates of each of the four vertices of the second rectangle; identifying as a first rectangle the largest rectangle that fits within the range of the second rectangle and has a first aspect ratio with the first intersection coordinates as its center coordinate; and determining the projection area of ​​the image on the projection surface based on the first rectangle. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram illustrating an outline of a system including a projector according to an embodiment of the invention. [Figure 2]FIG. 1 is a block diagram showing a configuration of a projector. [Figure 3] 10A and 10B are explanatory diagrams illustrating an example of a method for calculating a transformation matrix from a panel coordinate system to a screen coordinate system. [Figure 4] 10 is an explanatory diagram for explaining a method for specifying the coordinates in the screen coordinate system ΣS of four vertices on the projection surface that correspond one-to-one to the four vertices on the panel. [Figure 5] 10 is an explanatory diagram for explaining the operation of an intersection coordinate specifying unit; FIG. [Figure 6] FIG. 10 is an explanatory diagram for explaining an example of a method for specifying a first rectangle. [Figure 7] FIG. 10 is an explanatory diagram for explaining the rest of the method for specifying the first rectangle. [Figure 8] FIG. 10 is an explanatory diagram for explaining an example of a method for specifying a second rectangle. [Figure 9] FIG. 10 is an explanatory diagram for explaining the rest of the method for specifying the second rectangle. [Figure 10] 10A and 10B are explanatory diagrams for explaining a method for specifying a drawing area after correction on a panel. [Figure 11] 10 is a flowchart illustrating an example of the operation of the projector. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, in each drawing, the dimensions and scale of each part are appropriately different from those of the actual parts. Furthermore, since the embodiments described below are preferred examples of the present invention, various technically preferable limitations are applied, but the scope of the present invention is not limited to these embodiments unless otherwise specified in the following description to the effect that the present invention is limited.

[0009] [1. Embodiment] In this embodiment, a projection device will be described using a projector that projects an image onto a projection surface as an example. First, an overview of a system 10 including a projector 100 according to the embodiment will be described with reference to FIG.

[0010] FIG. 1 is a diagram showing an outline of a system 10 including a projector 100 according to an embodiment of the invention.

[0011] The projector 100 has a panel PL including a drawing area DAR1 in which an image is drawn based on image data output from a device such as a computer (not shown). The projector 100 projects the image drawn in the drawing area DAR1 of the panel PL onto a projection surface SC. The panel PL is, for example, an electro-optical panel such as a transmissive liquid crystal panel, a reflective liquid crystal panel, or a digital mirror device. In this embodiment, it is assumed that three panels PL are provided, corresponding to red, green, and blue. The drawing area DAR1 of the panel PL includes, for example, a plurality of pixels arranged in a matrix. In this embodiment, it is assumed that the drawing area DAR1 is the largest drawing area on the panel PL. In addition, it is assumed that the quadrangle defining the drawing area DAR1 of the panel PL is a rectangle RE10p. That is, in this embodiment, it is assumed that the drawing area DAR1 is a rectangular drawing area. However, the quadrangle defining the drawing area DAR1 is not limited to a rectangle. The rectangle RE10p defines the drawing area DAR1 of the panel PL in the panel coordinate system ΣP. The rectangle RE10p is an example of a "first rectangle." The projection surface SC onto which the image is projected is the surface of an object such as a screen, and is generally a plane. The projection surface SC does not need to be strictly a plane, but from the viewpoint of simplifying the process of geometrically correcting the image, it is preferable that the projection surface SC be a surface that can be regarded as a plane.

[0012] Here, the relative positional relationship between the projector 100 and the projection surface SC may vary depending on the manner in which the system 10 is used, etc. The relative positional relationship between the projector 100 and the projection surface SC includes not only the relative positional relationship of the projector 100 with respect to the projection surface SC, but also the relative attitude of the projector 100 with respect to the projection surface SC. The relative positional relationship of the projector 100 with respect to the projection surface SC changes depending on the installation positions of one or both of the projection surface SC and the projector 100. The relative attitude relationship of the projector 100 with respect to the projection surface SC changes depending on the installation attitude of one or both of the projection surface SC and the projector 100. The installation position and installation attitude of the projector 100 change depending on, for example, conditions such as the position and inclination of the installation surface on which the projector 100 is installed, adjustments made by an adjustment mechanism provided in the projector 100, and adjustments made by an adjustment mechanism provided in the stand on which the projector 100 is installed.

[0013] Depending on the relative positional relationship between the projector 100 and the projection surface SC, distortion may occur in the image projected on the projection surface SC. For this reason, the projector 100 corrects the distortion of the image on the projection surface SC by geometric correction such as keystone correction. In geometric correction, for example, the projection area of ​​the image on the projection surface SC is corrected to be rectangular.

[0014] In this embodiment, the explanation of corrections and the like performed by the projector 100 uses a screen coordinate system ΣS, which is a coordinate system for indicating the position coordinates of pixels on the projection surface SC, and a panel coordinate system ΣP, which is a coordinate system for indicating the position coordinates of pixels on the panel PL. For example, the screen coordinate system ΣS is a three-axis Cartesian coordinate system having mutually orthogonal Xs, Ys, and Zs axes, and the panel coordinate system ΣP is a three-axis Cartesian coordinate system having mutually orthogonal Xp, Yp, and Zp axes. In this embodiment, it is assumed that the Zs axis is parallel to the normal direction of the projection surface SC. Also, it is assumed that the Zp axis is parallel to the normal direction of the surface of one panel PL representing the three panels PL. Note that for each panel PL, the position on each panel PL is indicated using the Xp and Yp axes of the panel coordinate system ΣP. For example, at corresponding positions on three panels PL, the coordinates indicated by the Xp and Yp axes are the same for all three panels PL. The screen coordinate system ΣS is an example of a "first coordinate system" and a "screen coordinate system," and the panel coordinate system ΣP is an example of a "second coordinate system" and a "panel coordinate system." In addition, the direction of the arrow on each of the Xs-axis, Ys-axis, and Zs-axis is defined as the positive direction. For example, the direction of the arrow on the Xs-axis is denoted as the +Xs direction. Similarly, the direction of the arrow on each of the Xp-axis, Yp-axis, and Zp-axis is defined as the positive direction. For example, the direction of the arrow on the Xp-axis is denoted as the +Xp direction.

[0015] 1 corresponds to, for example, a rectangle RE10p that defines the drawing area DAR1 on the panel PL, and defines the largest drawing area on the projection surface SC. The four vertices P10s, P11s, P12s, and P13s of the rectangle QU10s on the projection surface SC correspond one-to-one to the four vertices P10p, P11p, P12p, and P13p of the rectangle RE10p on the panel PL, i.e., the rectangle RE10p in the panel coordinate system ΣP. That is, the rectangle RE10p in the panel coordinate system ΣP corresponds to the rectangle QU10s in the screen coordinate system ΣS. In other words, the rectangle QU10s is a rectangle obtained by converting the rectangle RE10p in the panel coordinate system ΣP into the screen coordinate system ΣS. The four vertices P10s, P11s, P12s, and P13s are an example of the "four vertices of a second quadrangle," and the quadrangle QU10s is an example of the "second quadrangle." The four vertices P10p, P11p, P12p, and P13p are the four vertices of a rectangle RE10p that defines the drawing area DAR1 in the panel coordinate system ΣP. Hereinafter, the drawing area on the projection surface SC may be referred to as the projection area.

[0016] The projection area onto which the image is projected, i.e., the projection area of ​​the image on the projection surface SC, is determined to fit within the range of the rectangle QU10s. "Fitted within the range of the rectangle QU10s" means that at least one vertex of the rectangle or quadrangle that defines the projection area converted to the panel coordinate system ΣP touches the outline of the rectangle QU10s, and the rectangle or quadrangle has a resolution or area that does not extend beyond the outline of the quadrangle QU10s. The method for determining the projection area is briefly explained in FIG. 1 and will be explained in detail in FIG. 3 and subsequent figures.

[0017] For example, the projector 100 identifies the coordinates of four vertices P10s, P11s, P12s, and P13s in the screen coordinate system ΣS. Then, the projector 100 identifies first intersection coordinates, which are the coordinates in the screen coordinate system ΣS of a first intersection PI1s where two diagonals of a quadrangle QU10s having the four vertices P10s, P11s, P12s, and P13s intersect with each other. The first intersection PI1s is an example of a "point where two diagonals of a second quadrangle intersect with each other." Hereinafter, the coordinates of the first intersection PI1s in the screen coordinate system ΣS, i.e., the first intersection coordinates, may be referred to as the coordinates of the first intersection PI1s.

[0018] The projector 100 also identifies the largest rectangle that fits within the rectangle QU10s and has a first aspect ratio centered at the coordinates of the first intersection point PI1s as the first rectangle RE1s. At least one of the four vertices of the first rectangle RE1s is inscribed in the rectangle QU10s. The first aspect ratio is, for example, the same aspect ratio as the aspect ratio based on the resolution of the panel PL. The aspect ratio based on the resolution of the panel PL is, for example, the aspect ratio of the rectangle RE10p that defines the drawing area DAR1 on the panel PL. The first aspect ratio may be, for example, an aspect ratio of a general projector, such as "16:9," "16:10," or "4:3." However, the first aspect ratio may be an aspect ratio other than the above-mentioned example aspect ratios. The projector 100 determines the projection area of ​​the image on the projection surface SC based on the first rectangle RE1s having the first aspect ratio. The first aspect ratio may be an aspect ratio that is arbitrarily set by the user within the range of the maximum resolution of the panel PL.

[0019] For example, the projector 100 may determine an area defined by the first rectangle RE1s as the projection area of ​​the image on the projection surface SC. Alternatively, the projector 100 may determine an area defined by a rectangle specified based on the first rectangle RE1s as the projection area of ​​the image on the projection surface SC. The rectangle specified based on the first rectangle RE1s is, for example, a second rectangle RE2s shown in FIG. 9, which will be described later. At least two of the four vertices of the second rectangle RE2s are inscribed in the quadrangle QU10s. By determining the projection area in this way, the drawing area DAR1 of the panel PL of the projector 100 can be used effectively.

[0020] Next, the configuration of the projector 100 will be described with reference to FIG.

[0021] FIG. 2 is a block diagram showing the configuration of the projector 100.

[0022] The projector 100 has a storage device 110, a processing device 120, a communication device 130, an image processing circuit 140, an optical device 150, an operation device 160, an acceleration sensor 170, and a distance sensor 180. For example, the storage device 110, the processing device 120, the communication device 130, the image processing circuit 140, and the acceleration sensor 170 are disposed inside a housing (not shown) of the projector 100 and are connected to each other so as to be able to communicate with each other.

[0023] The storage device 110 is a storage device that stores various types of information such as the control program PR of the projector 100. The storage device 110 is configured to include, for example, a hard disk drive or a semiconductor memory. Note that part or all of the storage device 110 may be included in the processing device 120. Alternatively, part or all of the storage device 110 may be provided in a storage device or server external to the projector 100. Note that the storage device 110 in which the control program PR is stored corresponds to a computer-readable recording medium.

[0024] The processing device 120 has a function of controlling each unit of the projector 100 and a function of processing various data. The processing device 120 includes, for example, one or more processors such as a central processing unit (CPU). Note that some or all of the functions of the processing device 120 may be implemented by hardware such as a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA). The processing device 120 may also be integrated with the image processing circuit 140. Note that if the first aspect ratio is an aspect ratio arbitrarily set by the user, the processing device 120 may receive an operation from the user to set the first aspect ratio via, for example, an operation device 160 (described later). The operation of setting the first aspect ratio is an operation of selecting one from multiple aspect ratio options, such as "16:9," "16:10," or "4:3."

[0025] The processing device 120 executes, for example, a control program PR stored in the storage device 110, thereby functioning as a vertex coordinate identification unit 122, an intersection coordinate identification unit 124, a rectangle identification unit 126, and a projection area determination unit 128. Therefore, the processing device 120 includes the vertex coordinate identification unit 122, the intersection coordinate identification unit 124, the rectangle identification unit 126, and the projection area determination unit 128. For example, the vertex coordinate identification unit 122, the intersection coordinate identification unit 124, the rectangle identification unit 126, and the projection area determination unit 128 determine the projection area of ​​the image on the projection surface SC. When the first aspect ratio is arbitrarily set by the user, the processing device 120 may cause the optical device 150, described below, to project a graphical user interface for receiving an operation from the user to set the first aspect ratio. The processing device 120 is an example of a "processing circuit." The operations of the vertex coordinate specifying unit 122, intersection coordinate specifying unit 124, rectangle specifying unit 126, and projection area determining unit 128 will be described later with reference to FIG. 3 and subsequent figures.

[0026] The communication device 130 is a communication device capable of communicating with various devices, and acquires image data IMG from devices not shown. For example, the communication device 130 may be a wired communication device such as a wired LAN (Local Area Network), or a wireless communication device such as a wireless LAN or Bluetooth, or may have the functions of both a wired communication device and a wireless communication device. "Bluetooth" is a registered trademark. Examples of wireless LANs include LPWA (Low Power Wide Area) and Wi-Fi. "Wi-Fi" is a registered trademark. Furthermore, examples of wired communication devices include, in addition to wired LANs, USB (Universal Serial Bus) and HDMI (High Definition Multimedia Interface). "HDMI" is a registered trademark.

[0027] The image processing circuit 140 is a circuit that performs necessary processing on image data IMG acquired by the communication device 130 and inputs the image data IMG to the optical device 150. For example, the image processing circuit 140 has a frame memory (not shown) and loads the image data IMG into the frame memory. The image processing circuit 140 then performs various processes, such as resolution conversion, resizing, and distortion correction, on the image data IMG loaded into the frame memory, as appropriate, and inputs the image data IMG to the optical device 150. Note that the image processing circuit 140 may also perform processes such as OSD (On Screen Display) processing, where necessary, that generates image information for menu display or operation guides, etc., and combines the image information with the image data IMG.

[0028] The optical device 150 is a device that displays an image by projecting image light onto the projection surface SC. In this embodiment, by appropriately determining the projection area of ​​the image on the projection surface SC, distortion of the projected image, which is the image that appears on the projection surface SC due to the projection of the image light by the optical device 150, is suppressed. The optical device 150 has a light source 152, an optical modulator 154, and a projection optical system 156.

[0029] The light source 152 includes a light source such as a halogen lamp, a xenon lamp, an ultra-high pressure mercury lamp, an LED (Light Emitting Diode), or a laser light source, and emits red, green, and blue light, respectively. The light modulator 154 includes three light modulation elements corresponding to red, green, and blue. Each light modulation element includes, for example, a panel PL, and generates image light of each color by modulating the light of the corresponding color. For example, if the panel PL is a transmissive liquid crystal panel, each pixel arranged in the drawing area DAR1 of the panel PL is set to a light transmittance based on the image data IMG processed by the image processing circuit 140. As a result, the light emitted from the light source 152 is modulated by passing through the drawing area DAR1 of the panel PL. The image light of each color generated by the light modulator 154 is combined by a color combining optical system to become full-color image light. The projection optical system 156 is an optical system including a projection lens and the like that forms and projects the full-color image light from the light modulator 154 onto a projection surface SC.

[0030] The operation device 160 is a device that accepts operations from a user. For example, the operation device 160 includes an operation panel (not shown). The operation panel is provided on the exterior housing of the projector 100 and outputs a signal based on an operation from the user. Note that a remote controller that transmits a signal based on an operation from the user wirelessly or via a cable may be used as the operation device 160. In this case, the projector 100 has a remote control light receiving unit as part of the operation device 160. For example, the remote control light receiving unit receives an infrared signal from a remote controller (not shown), decodes the infrared signal, and outputs a signal based on the operation of the remote controller. Note that the operation device 160 may be provided as needed and may be omitted.

[0031] The acceleration sensor 170 detects acceleration in three mutually perpendicular axes and detects acceleration acting on the projector 100. The acceleration sensor 170 is built into the projector 100 and fixed to a predetermined location within the housing of the projector 100. The predetermined location where the acceleration sensor 170 is fixed is, for example, a circuit board (not shown) on which the processing device 120 is mounted. Note that the predetermined location where the acceleration sensor 170 is fixed is not limited to the circuit board on which the processing device 120 is mounted. The acceleration sensor 170 outputs a signal corresponding to acceleration in directions along the Xp, Yp, and Zp axes of the panel coordinate system ΣP, for example. By fixing the acceleration sensor 170 to a predetermined location within the housing of the projector 100, the position of the acceleration sensor 170 within the housing is identified. As a result, the relative positional relationship between the acceleration sensor 170 and one panel PL representing the three panels PL is identified in advance. As a result, the acceleration sensor 170 is associated with the panel coordinate system ΣP. For example, as shown in FIG. 3, which will be described later, the gravity vector g based on the output of the acceleration sensor 170 is represented in the panel coordinate system ΣP.

[0032] The distance sensor 180 is a time-of-flight (TOF) distance sensor that measures the distance to the projection surface SC. The distance sensor 180 is fixed to a predetermined location on the projector 100, thereby identifying its position on the projector 100. This identifies in advance the relative positional relationship between the distance sensor 180 and one panel PL that represents the three panels PL. As a result, the distance sensor 180 is associated with the panel coordinate system ΣP. For example, the projector 100, more specifically, the vertex coordinate identification unit 122, can calculate a normal vector n of the projection surface SC shown in FIG. 3 (described later) based on a depth map of the projection surface SC based on the output of the distance sensor 180. The normal vector n of the projection surface SC is represented, for example, by the panel coordinate system ΣP. Note that the method for calculating the normal vector n of the projection surface SC is not limited to the method using the time-of-flight distance sensor 180. For example, the normal vector n of the projection surface SC may be calculated based on the results of triangulation using a camera.

[0033] Next, the operation of the vertex coordinate specification unit 122 will be described with reference to FIGS.

[0034] FIG. 3 is an explanatory diagram illustrating an example of a method for calculating a transformation matrix Aps from the panel coordinate system ΣP to the screen coordinate system ΣS.

[0035] In this embodiment, it is assumed that the Xs axis of the screen coordinate system ΣS is parallel to the horizontal plane. Also, as described above, in this embodiment, it is assumed that the Zs axis of the screen coordinate system ΣS is parallel to the normal direction of the projection surface SC. Therefore, in this embodiment, it is possible to utilize the property that the cross product of the gravity vector g indicating the direction of gravity and the normal vector n indicating the normal direction of the projection surface SC is parallel to the horizontal plane.

[0036] For example, the vertex coordinate identification unit 122 calculates a gravity vector g based on the output of the acceleration sensor 170 associated with the panel coordinate system ΣP. The vertex coordinate identification unit 122 also calculates a normal vector n based on a depth map of the projection surface SC based on the output of the distance sensor 180 associated with the panel coordinate system ΣP. The vertex coordinate identification unit 122 then calculates a transformation matrix Aps, which transforms the coordinate system from the panel coordinate system ΣP to the screen coordinate system ΣS, using the gravity vector g and the normal vector n. The positional relationship of the acceleration sensor 170 with the panel PL is calibrated, for example, when the projector 100 is manufactured. As a result, the acceleration sensor 170 is associated with the panel coordinate system ΣP.

[0037] Specifically, for example, the vertex coordinate identification unit 122 calculates the cross product of the normal vector n and the gravity vector g as a horizontal vanishing point vector H indicating the direction of the horizontal vanishing point in the screen coordinate system ΣS. The horizontal vanishing point vector H is expressed by equations (1) to (4) when the horizontal vanishing point vector H is (Hx, Hy, Hz), the normal vector n is (nx, ny, nz), and the gravity vector g is (gx, gy, gz).

[0038] H=(Hx,Hy,Hz)=(nx,ny,nz)×(gx,gy,gz) …(1) Hx=ny·gz-nz·gy …(2) Hy=nz·gx-nx·gz …(3) Hz=nx·gy-ny·gx …(4)

[0039] Furthermore, the vertex coordinate identification unit 122 calculates the cross product of the normal vector n and the horizontal vanishing point vector H as the vertical vanishing point vector V indicating the direction of the vertical vanishing point in the screen coordinate system ΣS. When the vertical vanishing point vector V is (Vx, Vy, Vz), the vertical vanishing point vector V is expressed by equations (5) to (8).

[0040] V=(Vx,Vy,Vz)=(nx,ny,nz)×(Hx,Hy,Hz) …(5) Vx=ny·Hz-nz·Hy …(6) Vy=nz·Hx-nx·Hz …(7) Vz=nx·Hy-ny·Hx …(8)

[0041] Then, the vertex coordinate specifying unit 122 specifies a transformation matrix Aps based on the horizontal vanishing point vector H, the vertical vanishing point vector V, and the normal vector n. For example, the transformation matrix Aps is expressed by equation (9).

[0042]

number

[0043] The method for calculating the transformation matrix Aps is not limited to the above example, and any known method can be used. For example, the horizontal vanishing point vector H constituting the components of the transformation matrix Aps may be the cross product of the normal vector n and a unit vector e having components only in the -Ys direction, rather than the cross product of the normal vector n and the gravity vector g. Here, the unit vector e is (0, -1, 0).

[0044] For example, as shown in FIG. 4, the vertex coordinate specifying unit 122 specifies the coordinates in the screen coordinate system ΣS of a point on the projection surface SC that corresponds to a point on the panel PL, using the transformation matrix Aps.

[0045] FIG. 4 is an explanatory diagram for explaining a method for determining the coordinates in the screen coordinate system ΣS of four vertices P10s, P11s, P12s, and P13s on the projection surface SC, which correspond one-to-one to the four vertices P10p, P11p, P12p, and P13p on the panel PL.

[0046] Four vertices P10p, P11p, P12p, and P13p on the panel PL are vertices of a rectangle RE10p that defines a drawing area DAR1 on the panel PL. The coordinates in the screen coordinate system ΣS of four vertices P10s, P11s, P12s, and P13s on the projection surface SC that correspond one-to-one to the four vertices P10p, P11p, P12p, and P13p are calculated using a transformation matrix Aps. For example, one vertex P10s, P11s, P12s, and P13s of the four vertices P10s, P11s, P12s, and P13s is n The coordinates of s in the screen coordinate system ΣS are calculated using the transformation matrix Aps and the vertex P as shown in equation (10). n It is calculated based on the product with p.

[0047] (Aps)(P n p)=α(P n s) …(10)

[0048] In addition, the vertex P in Eq. (10) n p is the vertex P among the four vertices P10p, P11p, P12p, and P13p. n s is the vertex corresponding to s. Furthermore, the value α in equation (10) is a value according to the distance between the panel PL and the projection surface SC. The value α may or may not be specified. In this embodiment, it is assumed that the value α is not particularly specified.

[0049] In this way, the coordinates in the screen coordinate system ΣS of the four vertices P10s, P11s, P12s, and P13s on the projection surface SC, which correspond one-to-one to the four vertices P10p, P11p, P12p, and P13p on the panel PL, are identified based on the transformation matrix Aps. The transformation matrix Aps is an example of a "correspondence relationship that associates the second coordinate system with the first coordinate system."

[0050] The area on the projection surface SC defined by a rectangle QU10s having four vertices P10s, P11s, P12s, and P13s corresponds to, for example, the drawing area DAR1 on the panel PL and is the largest drawing area on the projection surface SC. FIG. 4 illustrates a case in which distortion occurs in the area on the projection surface SC corresponding to the drawing area DAR1 on the panel PL. In this embodiment, the drawing area of ​​the image on the projection surface SC, i.e., the projection area of ​​the image on the projection surface SC, is corrected to become rectangular, thereby suppressing distortion of the image on the projection surface SC. For example, in this embodiment, as shown in FIGS. 5 to 9, the projection area on the projection surface SC is determined based on a first rectangle RE1s having a first aspect ratio and centered at a first intersection PI1s where the two diagonals of the rectangle QU10s intersect.

[0051] Next, with reference to FIG. 5, the operation of the intersection coordinate specifying unit 124 that specifies the coordinates of the first intersection PI1s will be described.

[0052] FIG. 5 is an explanatory diagram for explaining the operation of the intersection coordinate specifying unit 124. As shown in FIG.

[0053] The intersection coordinate identification unit 124 identifies first intersection coordinates, which are the coordinates in the screen coordinate system ΣS of a first intersection PI1s where two diagonals (a first diagonal and a second diagonal) of a quadrangle QU10s having four vertices P10s, P11s, P12s, and P13s intersect. For example, the intersection coordinate identification unit 124 calculates an equation for a line passing through the vertices P10s and P12s based on the coordinates of the vertices P10s and P12s in the screen coordinate system ΣS. At least a portion of the line passing through the vertices P10s and P12s constitutes the first diagonal of the quadrangle QU10s. That is, the intersection coordinate identification unit 124 calculates an equation representing the first diagonal of the quadrangle QU10s based on the coordinates of each of the vertices P10s and P12s. Furthermore, the intersection coordinate identification unit 124 calculates the equation of a line passing through the vertices P11s and P13s based on the coordinates of the vertices P11s and P13s in the screen coordinate system ΣS. At least a portion of the line passing through the vertices P11s and P13s forms the second diagonal of the quadrangle QU10s. That is, the intersection coordinate identification unit 124 calculates the equation representing the second diagonal of the quadrangle QU10s based on the coordinates of each of the vertices P10s and P12s. Then, based on the equation of the line passing through the vertices P10s and P12s and the equation of the line passing through the vertices P11s and P13s, the intersection coordinate identification unit 124 calculates the coordinates of the intersection of the line passing through the vertices P10s and P12s and the line passing through the vertices P11s and P13s (first intersection PI1s) as first intersection coordinates. Therefore, the intersection coordinate specifying unit 124 specifies the first intersection coordinates based on the coordinates of each of the vertices P10s, P11s, P12s, and P13s in the screen coordinate system ΣS.

[0054] Next, a method for specifying a first rectangle RE1s having a first aspect ratio and centered at the first intersection point PI1s will be described with reference to FIGS.

[0055] 6 is an explanatory diagram illustrating an example of a method for identifying a first rectangle RE1s having a first aspect ratio and centered at a first intersection point PI1s. As described in FIG. 1, the first rectangle RE1s is the largest rectangle that is centered at the first intersection point PI1s and has the first aspect ratio, fits within the range of the rectangle QU10s, and has the largest resolution or area.

[0056] For example, the rectangle identification unit 126 calculates an equation for a straight line Ld1 that passes through the start and end points of one of two diagonals (a third diagonal and a fourth diagonal) of a rectangle having a first aspect ratio and a first intersection point PI1s as its center, and an equation for a straight line Ld2 that passes through the start and end points of the other of the two diagonals. Note that a rectangle having a first aspect ratio and a first intersection point PI1s as its center is a rectangle that has, for example, two sides parallel to the Xs axis and two sides parallel to the Ys axis. In FIG. 6, an example of a rectangle having a first aspect ratio and a first intersection point PI1s as its center is shown by dashed lines to make it easier to visualize the lines Ld1 and Ld2.

[0057] The equations representing the two straight lines Ld1 and Ld2, which correspond to the two diagonals of a rectangle centered at the first intersection point PI1s and having a first aspect ratio, are calculated based on the first intersection coordinates, which are the coordinates of the first intersection point PI1s, and the first aspect ratio.

[0058] Furthermore, for example, the rectangle specification unit 126 calculates the equation of a line L10 passing through the vertices P10s and P11s based on the coordinates of the vertices P10s and P11s, calculates the equation of a line L11 passing through the vertices P11s and P12s based on the coordinates of the vertices P11s and P12s, calculates the equation of a line L12 passing through the vertices P12s and P13s based on the coordinates of the vertices P12s and P13s, and calculates the equation of a line L13 passing through the vertices P10s and P13s based on the coordinates of the vertices P10s and P13s.

[0059] Then, the rectangle identification unit 126 calculates the intersections of each of the straight lines L10, L11, L12, and L13 with each of the straight lines Ld1 and Ld2. To make the drawing easier to understand, in Fig. 6, of the intersections of each of the straight lines L10, L11, L12, and L13 with each of the straight lines Ld1 and Ld2, the intersections that are located outside the quadrangle QU10s are omitted. In other words, in Fig. 6, of the intersections of each of the straight lines L10, L11, L12, and L13 with each of the straight lines Ld1 and Ld2, only the intersections that are located on the sides of the quadrangle QU10s are shown.

[0060] For example, Fig. 6 shows, among the intersections of the straight line L10, L11, L12, or L13 with the straight line Ld1, an intersection PI10s between the straight line Ld1 connecting the vertices P10s and P13s, and an intersection PI12s between the straight line Ld1 and the straight line Ld1 connecting the vertices P11s and P12s. Also shown in Fig. 6 are, among the intersections of the straight line L10, L11, L12, or L13 with the straight line Ld2, an intersection PI13s between the straight line Ld2 connecting the vertices P10s and P13s, and an intersection PI11s between the straight line Ld2 connecting the vertices P11s and P12s. That is, Fig. 6 omits the illustration of the intersections of the straight line L10 with the straight line Ld1, the intersections of the straight line L12 with the straight line Ld1, the intersections of the straight line L10 with the straight line Ld2, and the intersections of the straight line L12 with the straight line Ld2.

[0061] As shown in FIG. 7, among the intersections of each of the straight lines L10, L11, L12, and L13 with each of the straight lines Ld1 and Ld2, the intersection closest to the first intersection point PI1s becomes one of the four vertices of the first rectangle RE1s.

[0062] FIG. 7 is an explanatory diagram for explaining the rest of the method for specifying the first rectangle RE1s.

[0063] The rectangle identification unit 126 identifies the intersection point closest to the first intersection point PI1s among the intersection points of each of the lines L10, L11, L12, and L13 with each of the lines Ld1 and Ld2 as one of the four vertices of the first rectangle RE1s. That is, the rectangle identification unit 126 identifies the intersection point closest to the first intersection point PI1s among the intersection points PI10s and PI12s between the sides of the quadrangle QU10s and the line Ld1, and the intersection points PI11s and PI13s between the sides of the quadrangle QU10s and the line Ld2 as one of the four vertices of the first rectangle RE1s. 7, among the distance DIS1 between the first intersection PI1s and the intersection PI10s, the distance DIS2 between the first intersection PI1s and the intersection PI11s, the distance DIS3 between the first intersection PI1s and the intersection PI12s, and the distance DIS4 between the first intersection PI1s and the intersection PI13s, the distance DIS4 is the shortest. Therefore, the rectangle identification unit 126 identifies the intersection PI13s of the intersections PI10s, PI11s, PI12s, and PI13s as one of the four vertices of the first rectangle RE1s.

[0064] Then, the rectangle identification unit 126 identifies as the first rectangle RE1s a rectangle having a first aspect ratio centered at the first intersection PI1s, one of whose four vertices is the intersection PI13s, and two sides parallel to the Xs axis and two sides parallel to the Ys axis. For example, the rectangle identification unit 126 identifies as the first rectangle RE1s a rectangle whose four vertices are each located on the line Ld1 or Ld2 and one of whose four vertices is the intersection PI13s. As a result, the largest rectangle that fits within the quadrangle QU10s and has the first aspect ratio centered at the first intersection coordinates that are the coordinates of the first intersection PI1s is identified as the first rectangle RE1s.

[0065] Among the intersections of each of the straight lines L10, L11, L12, and L13 with each of the straight lines Ld1 and Ld2, those located outside the quadrangle QU10s are not candidates for the intersection closest to the first intersection point PI1s. Therefore, the rectangle specification unit 126 does not need to calculate the intersections of each of the straight lines L10, L11, L12, and L13 with each of the straight lines Ld1 and Ld2 that are located outside the quadrangle QU10s.

[0066] Here, although the area defined by the first rectangle RE1s may be determined as the projection area of ​​the image on the projection surface SC, in this embodiment, it is assumed that the area defined by the second rectangle RE2s shown in Fig. 9, which will be described later, is determined as the projection area of ​​the image on the projection surface SC. The second rectangle is, for example, a rectangle that fits within the range of the rectangle QU10s, is larger than the first rectangle RE1s, has a first aspect ratio, and is specified based on the first rectangle RE1s.

[0067] Next, a method for specifying the second rectangle RE2s will be described with reference to FIGS.

[0068] FIG. 8 is an explanatory diagram for explaining an example of a method for specifying the second rectangle RE2s.

[0069] The rectangle identification unit 126 identifies the point where the first rectangle RE1s and the quadrangle QU10s intersect as the reference point. That is, the rectangle identification unit 126 identifies the intersection point between each of the lines L10, L11, L12, and L13 and each of the lines Ld1 and Ld2 that is closest to the first intersection point PI1s as the reference point. In the example shown in Fig. 8, the rectangle identification unit 126 identifies the intersection point PI13s where the first rectangle RE1s and the quadrangle QU10s intersect as the reference point.

[0070] The rectangle specification unit 126 also calculates the intersections of the sides of the quadrangle QU10s with lines passing through the reference point and each of three of the four vertices of the first rectangle RE1s other than the reference point. That is, the rectangle specification unit 126 also calculates the intersections of the sides of the quadrangle QU10s with lines passing through the intersection PI13s and each of three of the four vertices of the first rectangle RE1s other than the intersection PI13s.

[0071] Specifically, for example, the rectangle identification unit 126 calculates the intersection PI20s between a straight line Ls1, which is an extension of one of two sides of the first rectangle RE1s that are parallel to the Ys axis and passes through the intersection PI13s, and a side of the quadrangle QU10s. The Ys-axis value of the intersection PI20s is calculated, for example, by substituting the Xs-axis value of the intersection PI13s into the equation of the line L10. Note that the points identified by substituting the Xs-axis value of the intersection PI13s into the equations of the lines L11 and L12 are not located on the sides of the quadrangle QU10s, and therefore are not the intersections of the line Ls1 and the sides of the quadrangle QU10s. For example, the intersection of the straight line Ls1 with the side of the rectangle QU10s is the intersection of the straight line Ls1 with each of the lines L10, L11, and L12, where the Ys-axis value of the first intersection PI1s is included between the Ys-axis value of the intersection and the Ys-axis value of the intersection PI13s.

[0072] Furthermore, for example, the rectangle identification unit 126 calculates an intersection PI22s between a straight line Ls2 extending from one of two sides of the first rectangle RE1s that are parallel to the Xs axis and pass through the intersection PI13s, and a side of the quadrangle QU10s. The Xs-axis value of the intersection PI22s is calculated, for example, by substituting the Ys-axis value of the intersection PI13s into the equation of the line L11. Note that the points identified by substituting the Ys-axis value of the intersection PI13s into the equations of the lines L10 and L12 are not located on the sides of the quadrangle QU10s, and therefore are not intersections between the line Ls2 and the sides of the quadrangle QU10s. For example, the intersection of the line Ls2 with the side of the rectangle QU10s is the intersection of the line Ls2 with each of the lines L10, L11, and L12, where the Xs-axis value of the first intersection PI1s is included between the Xs-axis value of the intersection and the Xs-axis value of the intersection PI13s.

[0073] In this embodiment, the intersection PI21s between the line Ld2, which is an extension of one of the two diagonals of the first rectangle RE1s and which passes through the intersection PI13s, and a side of the quadrangle QU10s is the intersection PI11s described in FIG. 6 and has already been calculated. Here, the point determined based on the equations of the lines L10 and L12 and the equation of the line Ld2 is not located on a side of the quadrangle QU10s, and therefore is not the intersection of the line Ld2 and a side of the quadrangle QU10s. For example, the intersection of the line Ld2 and a side of the quadrangle QU10s is the intersection of the line Ld2 and each of the lines L10, L11, and L12 with the line Ld2, where the first intersection PI1s is located on the line connecting the intersection and the intersection PI13s.

[0074] Hereinafter, the intersection PI13s may be referred to as an intersection PI23s together with the intersections PI20s, PI21s, and PI22s.

[0075] As shown in FIG. 9, one of two vertices of the four vertices P20s, P21s, P22s, and P23s of the second rectangle RE2s is the intersection point PI23s, which is the reference point, and one of the intersection points PI20s, PI21s, and PI22s is identified as the other of the two vertices.

[0076] FIG. 9 is an explanatory diagram for explaining the rest of the method for specifying the second rectangle RE2s.

[0077] The rectangle specifying unit 126 specifies the intersection point, among the intersection points PI20s, PI21s, and PI22s, which has the smallest magnification ratio relative to the first rectangle RE1s, as one of the four vertices P20s, P21s, P22s, and P23s of the second rectangle RE2s.

[0078] For example, the rectangle identification unit 126 calculates a distance DIS10 between the start point and end point of a side of the first rectangle RE1s parallel to the Ys axis, a distance DIS20 between the start point and end point of a side of the first rectangle RE1s parallel to the Xs axis, and a distance DIS30 between the start point and end point of a diagonal of the first rectangle RE1s. Note that the distance DIS10 corresponds to the length of the side of the first rectangle RE1s parallel to the Ys axis, the distance DIS20 corresponds to the length of the side of the first rectangle RE1s parallel to the Xs axis, and the distance DIS30 corresponds to the length of the diagonal of the first rectangle RE1s. The rectangle identification unit 126 also calculates a distance DIS12 between intersections PI20s and PI23s, a distance DIS22 between intersections PI22s and PI23s, and a distance DIS32 between intersections PI21s and PI23s.

[0079] Then, the rectangle identification unit 126 calculates, for example, a first ratio which is the ratio of the distance DIS12 to the distance DIS10, a second ratio which is the ratio of the distance DIS22 to the distance DIS20, and a third ratio which is the ratio of the distance DIS32 to the distance DIS30. Hereinafter, an intersection which is one of the elements that determine each of the first ratio, the second ratio, and the third ratio may be referred to as an intersection corresponding to that ratio. For example, an intersection PI22s which is one of the elements that determine the second ratio may be referred to as an intersection PI22s corresponding to the second ratio.

[0080] The rectangle identification unit 126 identifies the intersection point PI20s, PI21s, and PI22s that corresponds to the smallest ratio among the first ratio, second ratio, and third ratio as one of the four vertices P20s, P21s, P22s, and P23s of the second rectangle RE2s. In the example shown in FIG. 9, the second ratio is the smallest ratio among the first ratio, second ratio, and third ratio. Therefore, the rectangle identification unit 126 identifies the intersection point PI22s that corresponds to the second ratio among the intersection points PI20s, PI21s, and PI22s as one of the four vertices P20s, P21s, P22s, and P23s of the second rectangle RE2s.

[0081] Then, the rectangle identification unit 126 identifies a rectangle having the first aspect ratio and including the intersection points PI22s and PI23s as two of its four vertices as the second rectangle RE2s. In the example shown in Fig. 9, the vertex P22s of the second rectangle RE2s is the intersection point PI22s, and the vertex P23s of the second rectangle RE2s is the intersection point PI23s.

[0082] For example, the coordinates of a vertex P20s of the second rectangle RE2s are determined based on the coordinates of the intersection PI23s that is the vertex P23s, the distance DIS22 between the intersections PI22s and PI23s, and the first aspect ratio. Also, for example, the coordinates of a vertex P21s of the second rectangle RE2s are determined based on the coordinates of the vertex P20s and the coordinates of the intersection PI22s that is the vertex P22s.

[0083] In this way, the rectangle identification unit 126 identifies as the second rectangle RE2s the rectangle having the first aspect ratio, one of its four vertices being the intersection point PI23s identified as the reference point, and the largest rectangle that does not protrude from the rectangle QU10s. Note that the second rectangle RE2s does not have to be the largest rectangle that does not protrude from the rectangle QU10s, as long as it has the first aspect ratio, one of its four vertices being the reference point, and is larger than the first rectangle RE1s. In other words, the rectangle identification unit 126 may identify as the second rectangle RE2s the rectangle having the first aspect ratio, one of its four vertices being the intersection point PI23s identified as the reference point, and that is larger than the first rectangle RE1s and does not protrude from the rectangle QU10s.

[0084] The method for identifying the second rectangle RE2s is not limited to the above example. For example, the rectangle identification unit 126 identifies a first candidate rectangle having a first aspect ratio and including two of its four vertices, the intersections PI23s and PI20s, which are reference points. Furthermore, the rectangle identification unit 126 identifies a second candidate rectangle having a first aspect ratio and including two of its four vertices, the intersections PI23s and PI21, and a third candidate rectangle having a first aspect ratio and including two of its four vertices, the intersections PI23s and PI22. The rectangle identification unit 126 may then identify the largest rectangle among the first, second, and third candidate rectangles that does not extend beyond the rectangle QU10s as the second rectangle RE2s.

[0085] Furthermore, in this embodiment, the projection area determination unit 128 determines the area defined by the second rectangle RE2s as the projection area PAR of the image on the projection surface SC. The area of ​​the panel PL corresponding to the area defined by the second rectangle RE2s corresponds to the corrected drawing area on the panel PL.

[0086] Next, a method for specifying the corrected drawing area DAR2 on the panel PL will be described with reference to FIG.

[0087] FIG. 10 is an explanatory diagram for explaining a method for specifying the drawing area DAR2 after correction on the panel PL.

[0088] The four vertices P20p, P21p, P22p, and P23p of the rectangle QU2p that defines the corrected drawing area DAR2 on the panel PL correspond one-to-one to the four vertices P20s, P21s, P22s, and P23s of the second rectangle RE2s in the screen coordinate system ΣS. The coordinates in the panel coordinate system ΣP of the four vertices P20p, P21p, P22p, and P23p on the panel PL that correspond one-to-one to the four vertices P20s, P21s, P22s, and P23s are calculated by the inverse matrix Aps of the transformation matrix Aps. -1 For example, one vertex P20p, P21p, P22p, and P23p is calculated using nThe coordinates of p in the panel coordinate system ΣP are expressed as the inverse matrix Aps of the transformation matrix Aps, as shown in equation (11). -1 and vertex P n It is calculated based on the product with s.

[0089] (Aps -1 )(P n s)=(1 / α)(P n p) …(11)

[0090] In addition, the vertex P in Equation (11) n s is the vertex P among the four vertices P20s, P21s, P22s, and P23s. n p is the vertex corresponding to p. Furthermore, the value α in equation (11) is the same as the value α in equation (10) above, and is a value that corresponds to the distance between the panel PL and the projection surface SC.

[0091] In this way, the coordinates in the panel coordinate system ΣP of the four vertices P20p, P21p, P22p, and P23p on the panel PL, which correspond one-to-one to the four vertices P20s, P21s, P22s, and P23s on the projection surface SC, are the inverse matrix Aps of the transformation matrix Aps -1 It is identified based on:

[0092] The drawing area DAR2 of the panel PL, defined by a quadrangle QU2p having four vertices P20p, P21p, P22p, and P23p, corresponds to, for example, an area defined by a second rectangle RE2s on the projection surface SC. For example, the projector 100 corrects the image of the drawing area DAR1 of the panel PL to an image of the drawing area DAR2. Then, the projector 100 projects the image drawn in the drawing area DAR2 of the panel PL, i.e., the corrected image, onto the projection surface SC. As described above, the area on the projection surface SC corresponding to the drawing area DAR2 of the panel PL is defined by the second rectangle RE2s. Therefore, the projection area PAR on the projection surface SC of the image drawn in the drawing area DAR2 of the panel PL is rectangular. As a result, in this embodiment, it is possible to suppress distortion of the image on the projection surface SC.

[0093] Furthermore, in this embodiment, one of the four vertices P20s, P21s, P22s, and P23s of the second rectangle RE2s is one of the four vertices of the first rectangle RE1s. The first rectangle RE1s is the largest rectangle having a first aspect ratio and centered at the first intersection point PI1s, within a range that does not extend beyond the rectangle QU10s on the projection surface SC corresponding to the rectangle RE10p that defines the largest drawing area DAR1 of the panel PL. This allows the projector 100 to effectively use the drawing area DAR1 of the panel PL in this embodiment.

[0094] Furthermore, in this embodiment, one of the four vertices P20s, P21s, P22s, and P23s of the second rectangle RE2s is one of the four vertices of the first rectangle RE1s, so the center of the second rectangle RE2s can be prevented from being far from the center of the first rectangle RE1s. As described above, the first intersection point PI1s, which is the center of the first rectangle RE1s, is the point where two diagonals of the rectangle QU10s on the projection surface SC corresponding to the rectangle RE10p that defines the drawing area DAR1 intersect. In this case, the projection area PAR defined by the second rectangle RE2s appears near the center in the direction in which the projector 100 installed by the user faces, so it is possible to prevent a loss of intuitiveness when the user views an image projected on the projection surface SC. A loss of intuitiveness when the user views an image projected on the projection surface SC can mean, for example, the occurrence of an uncomfortable feeling when the user views the image projected on the projection surface SC.

[0095] Next, with reference to FIG. 11, the operation of the projector 100 for suppressing distortion in the image projected onto the projection surface SC will be described.

[0096] Fig. 11 is a flowchart showing an example of the operation of projector 100. Fig. 11 shows the operation of projector 100 for suppressing distortion in the image projected onto projection surface SC. The timing at which the operation shown in Fig. 11 is performed is not particularly limited, but it is preferable that the operation be performed when projector 100 is used for the first time or when the relative positional relationship between projector 100 and projection surface SC is changed.

[0097] The processing of each step shown in FIG. 11 is executed by the processing device 120 included in the projector 100.

[0098] First, in step S100, the processing device 120 functions as the vertex coordinate identification unit 122 and calculates the normal vector n of the projection surface SC. For example, the vertex coordinate identification unit 122 acquires information indicating the distance from the projector 100 to each of multiple points on the projection surface SC from the distance sensor 180. Then, the vertex coordinate identification unit 122 calculates the normal vector n of the projection surface SC based on a depth map of the projection surface SC that is based on the distance from the projector 100 to each of the multiple points on the projection surface SC.

[0099] Next, in step S110, the processing device 120 functions as the vertex coordinate identification unit 122 and acquires the output of the acceleration sensor 170. For example, the output of the acceleration sensor 170 is information indicating the gravity vector g.

[0100] Next, in step S120, the processing device 120 functions as the vertex coordinate specification unit 122, and calculates a transformation matrix Aps for transforming the coordinate system from the panel coordinate system ΣP to the screen coordinate system ΣS, using the gravity vector g and the normal vector n.

[0101] Then, in step S130, the processing device 120 functions as the vertex coordinate identification unit 122 and identifies the coordinates in the screen coordinate system ΣS of four points corresponding to the four vertices P10p, P11p, P12p, and P13p of the largest drawing area DAR1 on the panel PL. For example, the vertex coordinate identification unit 122 identifies the coordinates in the screen coordinate system ΣS of four vertices P10s, P11s, P12s, and P13s that correspond one-to-one to the four vertices P10p, P11p, P12p, and P13p. Specifically, the vertex coordinate identification unit 122 calculates the coordinates in the screen coordinate system ΣS of each of the vertices P10s, P11s, P12s, and P13s based on the product of the coordinates in the panel coordinate system ΣP of each of the vertices P10p, P11p, P12p, and P13p and the transformation matrix Aps.

[0102] Next, in step S140, the processing device 120 functions as the intersection coordinate identification unit 124 to identify first intersection coordinates. For example, the intersection coordinate identification unit 124 identifies the first intersection coordinates, which are the coordinates in the screen coordinate system ΣS of a first intersection PI1s where two diagonals of a quadrangle QU10s having four vertices P10s, P11s, P12s, and P13s intersect with each other.

[0103] Next, in step S150, the processing device 120 functions as the rectangle identification unit 126 and identifies a first rectangle RE1s. For example, the rectangle identification unit 126 identifies the largest rectangle having a first aspect ratio and centered on the first intersection coordinates within a range that does not protrude from the quadrangle QU10s as the first rectangle RE1s.

[0104] Next, in step S160, the processing device 120 functions as the rectangle identification unit 126 and identifies a second rectangle RE2s. For example, the rectangle identification unit 126 identifies as the second rectangle RE2s the largest rectangle that does not extend beyond the rectangle QU10s and has a first aspect ratio, with the point where the rectangle QU10s and the first rectangle RE1s meet as one of its four vertices.

[0105] Then, in step S170, the processing device 120 functions as the projection area determination unit 128 and determines the projection area PAR of the image on the projection surface SC. For example, the projection area determination unit 128 determines the area defined by the second rectangle RE2s as the projection area PAR. Note that the second rectangle RE2s is specified based on the first rectangle RE1s, and therefore determining the area defined by the second rectangle RE2s as the projection area PAR is included in determining the projection area PAR based on the first rectangle RE1s.

[0106] Next, in step S180, the processing device 120 functions as the projection area determination unit 128, and identifies the coordinates in the panel coordinate system ΣP of four points corresponding to the four vertices P20s, P21s, P22s, and P23s of the second rectangle RE2s on the projection surface SC. For example, the vertex coordinate identification unit 122 identifies the coordinates in the panel coordinate system ΣP of four vertices P20p, P21p, P22p, and P23p that correspond one-to-one to the four vertices P20s, P21s, P22s, and P23s. Specifically, the vertex coordinate identification unit 122 calculates the coordinates in the screen coordinate system ΣS of each of the vertices P20s, P21s, P22s, and P23s and the inverse matrix Aps of the transformation matrix Aps. -1 The coordinates of each of the vertices P20p, P21p, P22p, and P23p in the panel coordinate system ΣP are calculated based on the product of these vertices. A drawing area DAR2 of the panel PL is defined by a quadrangle QU2p having the four vertices P20p, P21p, P22p, and P23p.

[0107] As described above, in this embodiment, the area of ​​the panel PL on which the image is drawn is corrected to the drawing area DAR2 so that the projection area PAR of the image on the projection surface SC becomes rectangular. For example, the projector 100 draws an image based on image data IMG in the drawing area DAR2 of the panel PL. Then, the projector 100 projects the image drawn in the drawing area DAR2 onto the projection surface SC. As a result, the image drawn in the drawing area DAR2 appears as a projected image in the projection area PAR defined by the second rectangle RE2s. In other words, the corrected rectangular projected image appears on the projection surface SC.

[0108] The operation of the projector 100 is not limited to the example shown in FIG. 11 . For example, the process of step S110 may be executed before the process of step S100, or may be executed in parallel with the process of step S100. Furthermore, for example, the process of step S160 may be omitted. In an aspect in which the process of step S160 is omitted, for example, in step S170, the projection area determination unit 128 may determine the area defined by the first rectangle RE1s as the projection area PAR. Furthermore, the process of step S180 may be executed by a functional block other than the projection area determination unit 128. For example, the process of step S180 may be executed by the vertex coordinate identification unit 122. Alternatively, the processing device 120 may function as a functional block other than the vertex coordinate identification unit 122, the intersection coordinate identification unit 124, the rectangle identification unit 126, and the projection area determination unit 128, and execute the process of step S180.

[0109] As described above, in this embodiment, the projector 100 has a panel PL including a drawing area DAR1 in which an image is drawn, a vertex coordinate specification unit 122, an intersection coordinate specification unit 124, a rectangle specification unit 126, and a projection area determination unit 128. The method for determining the projection area PAR according to this embodiment is a method for determining the projection area PAR in which an image drawn in the drawing area DAR1 of the panel PL provided in the projector 100 is projected. In the method of determining the projection area PAR, the vertex coordinate specification unit 122 specifies the coordinates in the screen coordinate system ΣS, which is the coordinate system of the projection surface SC when the projection surface SC on which the image is projected is viewed from the normal direction of the projection surface SC, of ​​each of the four vertices P10p, P11p, P12p, and P13p of the rectangle RE10p that defines the drawing area DAR1 in the coordinate system of the panel PL, and the intersection coordinate specification unit 124 specifies the coordinates in the screen coordinate system ΣS, which is the coordinate system of the projection surface SC when the projection surface SC is viewed from the normal direction of the projection surface SC on which the image is projected, of each of the four vertices P10s, P11s, P12p, and P13p. The first intersection coordinates, which are the coordinates in the screen coordinate system ΣS of the first intersection PI1s where the two diagonals of a rectangle QU10s having vertices P10s, P11s, P12s, and P13s intersect, are identified based on the coordinates of each of the four vertices P10s, P11s, P12s, and P13s of the rectangle QU10s, and the rectangle identification unit 126 identifies the largest rectangle that fits within the range of the rectangle QU10s and has a first aspect ratio with the first intersection coordinates as its center coordinate as the first rectangle RE1s, and the projection area determination unit 128 determines the projection area PAR on the projection surface SC based on the first rectangle RE1s.

[0110] As described above, in this embodiment, the projection area PAR on the projection surface SC is determined based on the first rectangle RE1s centered on the first intersection coordinates, which are the coordinates of the first intersection PI1s in the screen coordinate system ΣS. The first rectangle RE1s is the largest rectangle that fits within the range of the quadrangle QU10s on the projection surface SC corresponding to the rectangle RE10p that defines the drawing area DAR1 of the panel PL, and that has a first aspect ratio and is centered on the first intersection coordinates. This makes it possible to identify a rectangle for defining the projection area PAR of the corrected image that can effectively utilize the drawing area DAR1 of the panel PL. Therefore, in this embodiment, the drawing area DAR1 of the panel PL of the projector 100 can be effectively used.

[0111] Furthermore, in this embodiment, the rectangle specification unit 126 specifies a second rectangle RE2s that fits within the range of the rectangle QU10s and is larger than the first rectangle RE1s, and has a first aspect ratio with the point where the rectangle QU10s and the first rectangle RE1s meet as one of its four vertices. The projection area determination unit 128 determines the projection area PAR on the projection surface SC based on the second rectangle RE2s. Thus, in this embodiment, the projection area PAR on the projection surface SC is determined based on the second rectangle RE2s, which is larger than the first rectangle RE1s. Therefore, in this embodiment, the drawing area DAR1 of the panel PL can be used more effectively.

[0112] Furthermore, in this embodiment, the coordinates of each of the four vertices P10s, P11s, P12s, and P13s in the screen coordinate system ΣS are identified based on a transformation matrix Aps that associates the panel coordinate system ΣP, which is the coordinate system of the panel PL, with the screen coordinate system ΣS. As a result, in this embodiment, without using a high-resolution sensor to detect the area on the projection surface SC that corresponds to the drawing area DAR1 of the panel PL, it is possible to identify the coordinates in the screen coordinate system ΣS of the four vertices P10s, P11s, P12s, and P13s of the quadrangle QU10s that defines that area.

[0113] Furthermore, in this embodiment, the transformation matrix Aps is calculated based on the gravity vector g based on the output of the acceleration sensor 170 associated with the panel coordinate system ΣP, and the normal vector n of the projection surface SC. As a result, in this embodiment, the transformation matrix Aps can be calculated taking into account the inclination of the projector 100 with respect to the ground. For example, in this embodiment, the Xs axis of the screen coordinate system ΣS can be made parallel to the horizontal plane regardless of the inclination of the projector 100 with respect to the ground. Therefore, in this embodiment, the user can view a projected image that is erect with respect to the ground as seen by the user, regardless of the inclination of the projector 100 with respect to the ground.

[0114] Furthermore, in this embodiment, the normal vector n of the projection surface SC is calculated based on a depth map of the projection surface SC that is based on the output of the distance sensor 180. This eliminates the need for the user to visually measure the normal vector n.

[0115] [2. Modifications] Each of the above embodiments can be modified in various ways. Specific modified embodiments are exemplified below. Two or more embodiments arbitrarily selected from the following examples can be combined as appropriate within a range that does not contradict each other. In the modified examples exemplified below, elements whose actions and functions are equivalent to those of the embodiments will be designated by the same reference numerals as in the above description, and detailed descriptions of each will be omitted as appropriate.

[0116] [First Modification] In the above-described embodiment, the case where the first rectangle RE1s is identified is illustrated, but the present invention is not limited to this. For example, the rectangle identification unit 126 may identify the second rectangle RE2s without identifying the first rectangle RE1s.

[0117] In this modification, the vertex coordinate identification unit 122 identifies the coordinates of the four vertices P10s, P11s, P12s, and P13s in the screen coordinate system ΣS, as in the above-described embodiment. The intersection coordinate identification unit 124 also identifies the first intersection coordinates, as in the above-described embodiment. The rectangle identification unit 126 then identifies, as a reference point, the point where the largest rectangle that fits within the rectangle QU10s and has a first aspect ratio and is centered at the first intersection coordinates intersects with the rectangle QU10s. The rectangle identification unit 126 then identifies, as a second rectangle RE2s, the largest rectangle that fits within the rectangle QU10s and has the first aspect ratio and has the reference point as one of its four vertices. The projection area determination unit 128 then determines the projection area PAR on the projection surface SC based on the second rectangle RE2s. Note that the largest rectangle that fits within the range of the rectangle QU10s and has the first aspect ratio centered on the first intersection coordinate corresponds to the first rectangle RE1s, but as explained in Figure 8, the reference point can be identified without identifying the first rectangle RE1s.

[0118] As described above, this modification can also achieve the same effects as the above-described embodiment. Furthermore, in this modification, the second rectangle RE2s is specified without specifying the first rectangle RE1s, which reduces the processing load on the processing device 120 when determining the projection area PAR on the projection surface SC.

[0119] [Second Modification] In the above-described embodiment, the case where the transformation matrix Aps is calculated using the gravity vector g and the normal vector n is illustrated, but the present invention is not limited to this. For example, instead of the gravity vector g, a vector indicating a direction along the Yp axis of the panel coordinate system ΣP may be used. In other words, the transformation matrix Aps may be calculated using the vector indicating a direction along the Yp axis of the panel coordinate system ΣP and the normal vector n.

[0120] As described above, this modification also provides the same effects as the above-described embodiment, except for the effect obtained by calculating the transformation matrix Aps using the gravity vector g. The effect obtained by calculating the transformation matrix Aps using the gravity vector g is, for example, the effect of making the Xs axis of the screen coordinate system ΣS parallel to the horizontal plane regardless of the inclination of the projector 100 with respect to the ground. Note that, for example, if the inclination of the projector 100 with respect to the ground is small enough not to affect the user's visibility, this modification also allows the user to view a projected image that is erect or nearly erect with respect to the ground. Furthermore, in this modification, the projector 100 does not need to have the acceleration sensor 170.

[0121] [3. Notes] From the above-described exemplary embodiments, the following configurations can be understood, for example.

[0122] A method for determining a projection area according to aspect 1, which is a preferred aspect, is a method for determining a projection area onto which an image drawn in a drawing area of ​​a panel provided in a projection device is projected, and includes: identifying the coordinates in a first coordinate system, which is the coordinate system of the projection surface when the projection surface is viewed from the normal direction of the projection surface onto which the image is projected, of each of the four vertices of a first rectangle that defines the drawing area in the coordinate system of the panel; identifying first intersection coordinates, which are the coordinates in the first coordinate system of the point where two diagonals of a second rectangle corresponding to the first rectangle and having four vertices that correspond one-to-one with the four vertices, based on the coordinates of each of the four vertices of the second rectangle; identifying as a first rectangle the largest rectangle that fits within the range of the second rectangle and has a first aspect ratio with the first intersection coordinates as its center coordinate; and determining the projection area on the projection surface based on the first rectangle. According to the first aspect, it is possible to specify a rectangle for defining the projection area of ​​the corrected image that allows for effective use of the drawing area of ​​the panel. Therefore, in this aspect, it is possible to effectively use the drawing area of ​​the panel provided in the projection device.

[0123] A method for determining a projection area according to aspect 2, which is a specific example of aspect 1, further includes identifying a rectangle that fits within the range of the second rectangle and is larger than the first rectangle, and has the first aspect ratio, with the point where the second rectangle and the first rectangle intersect as one of its four vertices, and determining the projection area on the projection surface based on the second rectangle. According to the second aspect, the projection area on the projection surface is determined based on the second rectangle, which is larger than the first rectangle. Therefore, in this aspect, the drawing area of ​​the panel can be used more effectively.

[0124] A method for determining a projection area according to a third preferred aspect is a method for determining a projection area onto which an image drawn in a drawing area of ​​a panel included in a projection device is projected, the method comprising the steps of: specifying coordinates in a first coordinate system, which is the coordinate system of a projection surface when the projection surface is viewed from a normal direction to the projection surface onto which the image is projected, of each of four vertices of a first rectangle that defines the drawing area in the coordinate system of the panel; and determining a point in front of a point where two diagonals of a second rectangle corresponding to the first rectangle intersect with each other, the second rectangle having four vertices that correspond one-to-one with the four vertices of the first rectangle. The method includes identifying first intersection coordinates, which are coordinates in the first coordinate system, based on the coordinates of each of the four vertices of the second rectangle; identifying as a reference point the point where the largest rectangle that fits within the range of the second rectangle and has a first aspect ratio and has the first intersection coordinates as its center coordinates and the second rectangle intersects; identifying as a second rectangle the largest rectangle that fits within the range of the second rectangle and has the first aspect ratio and has the reference point as one of its four vertices; and determining the projection area on the projection surface based on the second rectangle. According to the first aspect, the same effects as those of the second aspect can be obtained.

[0125] In a method for determining a projection area according to aspect 4, which is a specific example of any one of aspects 1 to 3, the coordinates of each of the four vertices in the first coordinate system are identified based on a correspondence relationship that associates the first coordinate system with a second coordinate system, which is the coordinate system of the panel. According to aspect 4, it is possible to identify the coordinates in the first coordinate system of the four vertices of the second rectangle that defines the drawing area of ​​the panel without using a high-resolution sensor to detect the area on the projection surface that corresponds to the drawing area of ​​the panel.

[0126] In a method for determining a projection area according to aspect 5, which is a specific example of aspect 4, the correspondence is calculated based on a gravity vector based on the output of an acceleration sensor associated with the second coordinate system and a normal vector of the projection surface. According to the fifth aspect, it is possible to calculate a correspondence relationship that takes into account the inclination of the projection device with respect to the ground as a correspondence relationship between the second coordinate system and the first coordinate system.

[0127] In a method for determining a projection area according to aspect 6, which is a specific example of aspect 5, the normal vector of the projection plane is calculated based on a depth map of the projection plane that is based on the output of a distance sensor that detects distance. According to the sixth aspect, it is possible to save the user the trouble of visually measuring the normal vector.

[0128] Furthermore, a projection device according to aspect 7, which is a preferred aspect, comprises a panel including a drawing area in which an image is drawn, and a processing circuit, wherein the processing circuit performs the following steps: identifying the coordinates in a first coordinate system, which is the coordinate system of the projection surface when the projection surface is viewed from the normal direction of the projection surface onto which the image is projected, of each of the four vertices of a first rectangle that defines the drawing area in the coordinate system of the panel; identifying first intersection coordinates, which are the coordinates in the first coordinate system of the point where two diagonals of a second rectangle corresponding to the first rectangle and having four vertices that correspond one-to-one with the four vertices, based on the coordinates of each of the four vertices of the second rectangle; identifying as a first rectangle the largest rectangle that fits within the range of the second rectangle and has a first aspect ratio with the first intersection coordinates as its center coordinate; and determining the projection area of ​​the image on the projection surface based on the first rectangle. According to the seventh aspect, the same effects as those of the first aspect can be obtained. [Explanation of symbols]

[0129] 10...system, 100...projector, 110...storage device, 120...processing device, 122...vertex coordinate identification unit, 124...intersection coordinate identification unit, 126...rectangle identification unit, 128...projection area determination unit, 130...communication device, 140...image processing circuit, 150...optical device, 152...light source, 154...light modulator, 156...projection optical system, DAR1, DAR2...drawing area, PAR...projection area, PL...panel, QU10s...rectangle, QU2p...rectangle, RE10p...rectangle, RE1s...first rectangle, RE2s...second rectangle, SC...projection surface.

Claims

1. A method for determining a projection area onto which an image drawn in a drawing area of ​​a panel provided in a projection device is projected, comprising: Identifying coordinates in a first coordinate system, which is a coordinate system of a projection surface when the projection surface is viewed from a normal direction of the projection surface onto which the image is projected, of each of four vertices of a first rectangle that defines the drawing area in the coordinate system of the panel; Identifying first intersection coordinates, which are coordinates in the first coordinate system of a point where two diagonals of a second quadrangle corresponding to the first quadrangle and having four vertices that correspond one-to-one to the four vertices, intersect with each other, based on the coordinates of each of the four vertices of the second quadrangle; Identifying, as a first rectangle, the largest rectangle that fits within the range of the second rectangle and has a first aspect ratio and has the first intersection coordinates as its center coordinates; determining the projection area on the projection surface based on the first rectangle; A method for determining a projection area, comprising:

2. further comprising specifying a rectangle having the first aspect ratio and having a point where the second rectangle and the first rectangle meet as one of four vertices, as a second rectangle that fits within the range of the second rectangle and is larger than the first rectangle; determining the projection area on the projection surface based on the second rectangle; 2. The method for determining a projection area according to claim 1.

3. The coordinates of each of the four vertices in the first coordinate system are identified based on a correspondence relationship that associates the first coordinate system with a second coordinate system that is a coordinate system of the panel.

2. The method for determining a projection area according to claim 1.

4. the correspondence relationship is calculated based on a gravity vector based on an output of an acceleration sensor associated with the second coordinate system and a normal vector of the projection surface; 4. The method for determining a projection area according to claim 3.

5. The normal vector of the projection surface is calculated based on a depth map of the projection surface based on an output of a distance sensor that detects a distance.

5. The method for determining a projection area according to claim 4.

6. a panel containing a drawing area in which the image is drawn; a processing circuit; Equipped with The processing circuitry Identifying coordinates in a first coordinate system, which is a coordinate system of a projection surface when the projection surface is viewed from a normal direction of the projection surface onto which the image is projected, of each of four vertices of a first rectangle that defines the drawing area in the coordinate system of the panel; Identifying first intersection coordinates, which are coordinates in the first coordinate system of a point where two diagonals of a second quadrangle corresponding to the first quadrangle and having four vertices that correspond one-to-one to the four vertices, intersect with each other, based on the coordinates of each of the four vertices of the second quadrangle; Identifying, as a first rectangle, the largest rectangle that fits within the range of the second rectangle and has a first aspect ratio and has the first intersection coordinates as its center coordinates; determining a projection area of ​​the image on the projection surface based on the first rectangle; A projection device characterized by executing the above.

Images