Display method, projector and program

Abstract

To provide a new visual effect for an image projected on a projector.SOLUTION: A processing device 17 of a projector 10 displays a projection object image GX including a first portion P1 and a second portion P2 different from the first portion P1 on a screen 20. The processing device 17 stores first shape information E1 indicating a first shape being the shape of the first portion P1 when the screen 20 is viewed from a first direction. The processing device 17, when the shape of the first portion P1 viewed from the first direction changes to the shape different from the first shape, performs correction of bringing the shape of the first portion P1 viewed from the first direction close to the first shape for the projection object image GX, and displays a correction image C obtained by not performing correction to the second portion P2 on the screen 20 by using the projector 10.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present invention relates to a display method, a projector, and a program. [Background technology]

[0002] Conventionally, there are known techniques for correcting distortion in a projected image when projecting an image using a projector. For example, in Patent Document 1 below, a synthesis circuit synthesizes image data input from an image source such as a personal computer with image data for an on-screen display menu output from an on-screen display menu generation unit. A keystone distortion correction unit performs keystone distortion correction on the image data of this synthesized image. This makes it possible to reduce keystone distortion not only in the image to be projected but also in the on-screen display menu projected onto the screen. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2000-330507 Summary of the Invention [Problem to be solved by the invention]

[0004] The above-mentioned conventional technologies are based on the premise that an image is projected onto a flat screen. In contrast, when a flexible material such as fabric or thin synthetic resin is used as the screen, the position of the projection surface may change three-dimensionally due to factors such as wind. When such changes occur in the screen, it may be possible to provide new visual effects depending on how they are corrected. [Means for solving the problem]

[0005] A display method according to one aspect of the present invention includes using a projector to display on a screen a first image including a first portion and a second portion different from the first portion; storing information indicating a first shape, which is the shape of the first portion when the screen is viewed from a first direction; and, when the shape of the first portion when viewed from the first direction changes to a shape different from the first shape, correcting the first image to bring the shape of the first portion when viewed from the first direction closer to the first shape, and displaying on the screen using the projector a second image obtained by not performing the correction on the second portion.

[0006] A projector according to one aspect of the present invention includes an optical device and a control device that controls the optical device, and the control device uses the optical device to display a first image on a screen, the first image including a first portion and a second portion different from the first portion; stores information indicating a first shape, which is the shape of the first portion when the screen is viewed from a first direction; and, when the shape of the first portion when viewed from the first direction changes to a shape different from the first shape, corrects the first image to bring the shape of the first portion when viewed from the first direction closer to the first shape, and uses the optical device to display a second image obtained by not performing the correction on the second portion on the screen.

[0007] A program according to one aspect of the present invention causes a processing device to use a projector to display on a screen a first image including a first portion and a second portion different from the first portion; store information indicating a first shape, which is the shape of the first portion when the screen is viewed from a first direction; and, when the shape of the first portion when viewed from the first direction changes to a shape different from the first shape, correct the first image to bring the shape of the first portion when viewed from the first direction closer to the first shape, and display, using the projector, a second image obtained by not performing the correction on the second portion on the screen. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram showing a display system 1 including a projector 10 according to an embodiment. [Figure 2] FIG. 2 is a block diagram showing the configuration of a projector 10. [Figure 3] FIG. 2 is a diagram showing types of images related to the display system 1. [Figure 4] FIG. 2 is a diagram schematically illustrating an example of an input image I. [Figure 5] 2 is an explanatory diagram schematically showing an imaging range SA of an imaging device 15. FIG. [Figure 6] 10 is a diagram illustrating a method for accepting a designation of a first portion P1 by a designation accepting unit 172. FIG. [Figure 7] FIG. 2 is an explanatory diagram schematically showing a correction target image CX. [Figure 8] FIG. 2 is an explanatory diagram schematically illustrating an example of a first captured image S1. [Figure 9] FIG. 10 is an explanatory diagram schematically illustrating an example of a second captured image S2. [Figure 10] FIG. 10 is an explanatory diagram schematically illustrating an example of a third captured image S3. [Figure 11] FIG. 10 is an explanatory diagram schematically showing a method for determining a correction amount. [Figure 12] FIG. 10 is an explanatory diagram schematically showing a method for determining a correction amount. [Figure 13] 10 is a flowchart showing the flow of a display method executed by the processing device 17 of the projector 10 in accordance with the control program 162. DETAILED DESCRIPTION OF THE INVENTION

[0009] Preferred embodiments of the present invention will be described below with reference to the accompanying drawings. Note that the dimensions and scale of each part in the drawings may differ from the actual dimensions and some parts are shown schematically to facilitate understanding. Furthermore, the scope of the present invention is not limited to these embodiments unless otherwise specified in the following description to the effect that the present invention is limited thereto.

[0010] A: Overview of Display System 1 1 is a diagram showing a display system 1 including a projector 10 according to an embodiment. The display system 1 includes the projector 10 and a screen 20.

[0011] The projector 10 displays an image by projecting it onto the screen 20. The image projected by the projector 10 is referred to as a projected image G. The projector 10 is disposed, for example, so that the projected image G is positioned on the fabric 22 of the screen 20. In the example of FIG. 1, the projected image G is disposed so that it overlaps with the center of the fabric 22. The projector 10 may be disposed, for example, on a desk, table, or floor, or may be attached to a ceiling or wall.

[0012] The screen 20 is a component having a projection surface onto which the projection image G is projected. In this embodiment, the screen 20 is a tapestry, and includes a fabric 22, an upper bar 24A, a lower bar 24B, and a hanging string 26. The fabric 22 has a horizontally long rectangular shape, with the upper bar 24A attached along the top edge and the lower bar 24B attached along the bottom edge. Hanging strings 26 are attached to both ends of the upper bar 24A, allowing the screen 20 to be hung using hooks F or the like.

[0013] 1, when the screen 20 is suspended, the fabric 22 is stretched flat in the direction of gravity due to the weight of the lower bar 24B. However, if an external force is applied to the screen 20, for example, when wind blows around the screen 20 or when a person touches the screen 20, the fabric 22 may sag and become non-flat.

[0014] The screen 20 is not limited to a tapestry, but may be, for example, a roller blind placed near a window for sun protection, a banner, a wall, etc. The projection surface of the screen 20 is not limited to fabric 22, but may be made of synthetic resin such as vinyl, or paper. The following describes an example in which the shape of the screen 20 changes, but this is not limiting, and the relative positional relationship between the projector 10 and the screen 20 may also be changed, for example. For example, the angle of the projector 10 with respect to the wall surface may be changed by hanging the projector 10 with a string or the like.

[0015] B: Projector 10 Configuration 2 is a block diagram showing the configuration of the projector 10. The projector 10 includes an operation device 12, a communication device 13, an optical device 14, an imaging device 15, a storage device 16, and a processing device 17.

[0016] The operation device 12 is, for example, various operation buttons, operation keys, or a touch panel. The operation device 12 is provided, for example, on the housing of the projector 10. The operation device 12 may also be a remote controller provided separately from the housing of the projector 10. The operation device 12 receives input operations from the user.

[0017] The communication device 13 is an interface communicatively connected to an image supply device such as a computer (not shown). The communication device 13 receives input image data, which is data of the input image I, from the image supply device. The communication device 13 is, for example, an interface such as a wireless or wired LAN (Local Area Network), Bluetooth, USB (Universal Serial Bus), or HDMI (High Definition Multimedia Interface). Bluetooth, USB, and HDMI are all registered trademarks. The communication device 13 may also be connected to the image supply device via another network such as the Internet. The communication device 13 includes an interface such as an antenna in the case of a wireless connection or a connector in the case of a wired connection, and an interface circuit that electrically processes signals received via the interface.

[0018] The optical device 14 projects the projection image G within a projectable range NA based on an image signal from the processing device 17. The projectable range NA is shown in FIG. 5, for example. The projectable range NA is a range onto which an image can be projected by the optical device 14. In general, the housing of the projector 10 is disposed so that the projectable range NA overlaps with the screen 20. The optical device 14 has a light source 141, a light modulation device 142, and a projection optical system 143.

[0019] The light source 141 includes, for example, a halogen lamp, a xenon lamp, an ultra-high pressure mercury lamp, an LED (Light Emitting Diode), or a laser light source. The light source 141 emits, for example, red, green, and blue light, or emits white light. When the light source 141 emits white light, the light emitted from the light source 141 has its brightness distribution reduced by an integrator optical system (not shown), and is then separated into red, green, and blue light by a color separation optical system (not shown), and enters the light modulation device 142.

[0020] The light modulation device 142 includes three light modulation elements respectively corresponding to red, green, and blue. Each of the light modulation elements includes, for example, a transmissive liquid crystal panel, a reflective liquid crystal panel, or a DMD (digital mirror device). The light modulation elements modulate the red, green, and blue light, respectively, based on an image signal from the processing device 17, to generate image light of each color. The image light of each color generated by the light modulation device 142 is combined by a color combining optical system (not shown) to become full-color image light. Note that the light modulation device 142 is not limited to this, and full-color image light may be viewed by emitting image light of each color in a time-division manner using a single-plate liquid crystal panel, a DMD, or the like.

[0021] The projection optical system 143 forms and projects the full-color image light onto the screen 20. The projection optical system 143 is an optical system that includes at least one projection lens, and may include a zoom lens, a focus lens, or the like.

[0022] The imaging device 15 captures an imaging range SA, which is a space in the imaging direction, and generates captured image data corresponding to the captured image S. The imaging range SA is shown in FIG. 5, for example. The imaging device 15 includes a light receiving optical system such as a lens, and an imaging element that converts light collected by the light receiving optical system into an electrical signal. The imaging element is, for example, a CCD (Charge Coupled Device) image sensor that receives light in the visible light range. As will be described later, the imaging device 15 is positioned so that the imaging direction coincides with the projection direction of the image by the optical device 14, and so that the imaging range SA includes the entire projectible range NA.

[0023] The imaging device 15 may be provided separately from other elements of the projector 10. In this case, the imaging device 15 and the projector 10 may be connected to each other via a wired or wireless interface so that data can be transmitted and received. In this case, it is assumed that the positional relationship between the imaging range SA of the imaging device 15 and the projectable range NA of the optical device 14 has been calibrated.

[0024] The storage device 16 is a recording medium that can be read by the processing device 17. The storage device 16 includes, for example, a nonvolatile memory and a volatile memory. Examples of the nonvolatile memory include a ROM (Read Only Memory), an EPROM (Erasable Programmable Read Only Memory), and an EEPROM (Electrically Erasable Programmable Read Only Memory). Examples of the volatile memory include a RAM.

[0025] The storage device 16 stores a control program 162 executed by the processing device 17 and various data 164 used by the processing device 17. The control program 162 is executed by the processing device 17. The control program 162 includes, for example, an operating system and a plurality of application programs. The data 164 includes input image data corresponding to the input image I and shape information E, which will be described later. The data 164 also includes calibration data for associating the coordinates of the projectable range NA on the captured image S with the coordinates on the frame memory.

[0026] The processing device 17 is configured, for example, by one or more processors. As an example, the processing device 17 is configured by one or more central processing units (CPUs). Some or all of the functions of the processing device 17 may be configured by circuits such as a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA). The processing device 17 executes various processes in parallel or sequentially.

[0027] The processing device 17 reads and executes the control program 162 from the storage device 16, thereby functioning as a captured image acquisition unit 170, a projection control unit 171, a designation reception unit 172, a shape information generation unit 173, a correction processing unit 174, and a correction amount determination unit 175. The processing device 17 is an example of a control device. Details of each functional unit of the processing device 17 will be described later.

[0028] C. Details of Image Projection by Projector 10 C-1. Image type 3 is a diagram showing types of images related to the display system 1. The images related to the display system 1 include an input image I, a corrected image C, a projected image G, and a captured image S.

[0029] The input image I is an image supplied from an image supply device via, for example, the communication device 13. The input image I is essentially input image data, and the user recognizes its contents by, for example, projecting the input image I as a projection image G or displaying it on a display (not shown).

[0030] The corrected image C is an image obtained by performing a correction process on the input image I. The correction processes include keystone correction, which corrects deformation of the projected image G due to deviation of the projection direction of the projector 10 relative to the screen 20, as well as distortion correction, which corrects deformation of the projected image G due to distortion of the projection surface of the screen 20, as will be described later. The corrected image C is essentially corrected image data, and the user recognizes its contents, for example, when the corrected image C is projected as the projected image G or displayed on a display (not shown).

[0031] The correction process is not limited to the input image I, and may also be performed on the corrected image C. For example, if the screen 20 warps while the corrected image C is being projected, further correction is performed on the corrected image C being projected, and a new corrected image C is generated. Further correction includes changing the amount of correction in the corrected image C. Hereinafter, the image that is the target of the correction process is referred to as the correction target image CX. The correction target image CX is the input image I or the corrected image C.

[0032] The projection image G is an image projected onto the screen 20. The projection image G is projected onto the screen 20 as an image visible to the user as a result of the optical device 14 being driven by an image signal generated using the input image I or the corrected image C. Hereinafter, the input image I or the corrected image C that is the source of the image signal will be referred to as the projection target image GX. The projection image G is an example of the first image and the second image.

[0033] The appearance of the projected image G may differ depending on, for example, the position of the person viewing the projected image G. The appearance of the projected image G may also differ depending on, for example, the state of the screen 20. Specifically, the appearance of the projected image G projected onto the screen 20 may differ depending on whether the fabric 22 of the screen 20 is stretched flat or sagged.

[0034] The captured image S is an image captured by the imaging device 15. In this embodiment, the captured image S is mainly an image obtained by capturing a projection image G projected onto the screen 20. The captured image S, which is a captured image of the projection image G, allows the user to understand how the projection image G appears from the imaging direction of the imaging device 15. The captured image S is essentially captured image data, and the user recognizes its contents, for example, when the captured image S is projected as a projection image G or displayed on a display (not shown).

[0035] C-2. Details of input image I 4 is a diagram schematically illustrating an example of an input image I. In this embodiment, the input image I includes a first portion P1 and a second portion P2 that is different from the first portion P1. The first portion P1 is a character image showing the alphabet "OPEN." The second portion P2 is a partial area of ​​a background image that forms the background of the first portion P1 and does not include the first portion P1.

[0036] The first portion P1 is not limited to a character image, but may be any image that can be distinguished from the second portion P2 by its outline. The first portion P1 may be, for example, an illustration or photograph of a character, person, animal, or the like, or may be a logo mark or the like.

[0037] The background image that becomes the second portion P2 may be a solid color, may have a repeating pattern, may have a multi-color gradation, or may be a photograph, a painting, or the like.

[0038] In this embodiment, the first portion P1 and the second portion P2 are included in the same layer in the input image I. Therefore, the first portion P1 and the second portion P2 are also included in the same layer in the corrected image C generated based on the input image I. For example, in the process of editing the input image I, it is considered that an operation such as overlaying a text image constituting the first portion P1 on a background image constituting the second portion P2 is performed. However, when the input image I is input to the projector 10 as input image data, the text image and the background image are combined, and, for example, hue information and the like of the portion of the background image overlapping the text image is erased.

[0039] The first portion P1 and the second portion P2 may be arranged on different layers in the input image I. For example, if the projector 10 is a so-called interactive projector, it is possible to write on a background image by handwriting or the like and save the image. If the input image I is such a written image, the first portion P1 becomes a drawing image showing handwritten characters or the like, and the second portion P2 becomes a background image.

[0040] C-3.Relationship between imaging range SA and projection range NA FIG. 5 is an explanatory diagram that schematically shows the imaging range SA of the imaging device 15. Note that in FIG. 5 and subsequent figures, the hanging string 26 of the screen 20 is not shown. In FIG. 5, the imaging range SA of the imaging device 15 is indicated by a dashed-dotted line, and the projectable range NA of the optical device 14 is indicated by a dotted line. The area included in the imaging range SA is captured as the captured image S. The imaging range SA includes the entire projectable range NA. In other words, the captured image S is captured so as to include the entire range onto which the image can be projected by the projector 10.

[0041] In the example of Fig. 5, a projection image G is displayed in a part of the projectable range NA. The projection image G is an image obtained by projecting the input image I shown in Fig. 4 or a corrected image C obtained by correcting the input image I. The range of the projectable range NA onto which the projection image G is projected is referred to as the projection range GA. In general, the projection range GA is determined by, for example, the aspect ratio of the input image I corresponding to the projection image G, the magnification setting and the display position setting by the user, and the like.

[0042] C-4. Details of Processing Device 17 As described above, the processing device 17 functions as an image acquisition unit 170, a projection control unit 171, a specification receiving unit 172, a shape information generation unit 173, a correction processing unit 174, and a correction amount determination unit 175 by reading and executing the control program 162 from the storage device 16.

[0043] The captured image acquisition section 170 acquires the captured image S captured by the imaging device 15. In this embodiment, the captured image acquisition section 170 may continuously acquire the image of the screen 20, for example, while the projector 10 is projecting an image onto the screen 20. As described above, the captured image S includes the entire projectible range NA of the optical device 14. Therefore, the entire area of ​​the projected image G is captured in the captured image S.

[0044] The projection control unit 171 uses image data of the projection target image GX to generate an image signal for driving the optical device 14. The image signal generated by the projection control unit 171 is input to the optical device 14.

[0045] The designation receiving unit 172 receives an input designating the first portion P1. The designation receiving unit 172 receives designation of a range HA including the first portion P1, for example, using a projection image G projected on the screen 20. When the range HA is designated, the designation receiving unit 172 stores the inside of the contour included in the range HA as the first portion P1.

[0046] FIG. 6 is a diagram illustrating a method for accepting a designation of the first portion P1 by the designation accepting unit 172. When the user performs a predetermined operation using the operation device 12 while the projection image G is projected onto the screen 20, the designation mode for the first portion P1 is activated. In the designation mode for the first portion P1, pointers TS1 and TS2 for designating an area HA are displayed on the projection image G. The pointer TS1 designates the upper left point of the area HA. ​​The pointer TS2 designates the lower right point of the area HA. ​​The user uses the operation device 12 to move the pointers TS1 and TS2 to desired positions on the projection image G. When the user performs an operation such as "Confirm," the area HA is confirmed. For example, when the area HA is confirmed in the state shown in FIG. 6, the portion representing the letters "O" and "P" is designated as the first portion P1. In this case, the letters "E" and "N" are not used to generate the shape information E, which will be described later.

[0047] The first portion P1 may not be specified on the projected image G, but may be specified on the touch panel by using the pointers TS1 and TS2, for example, by displaying the input image I on a touch panel, which is an example of the operation device 12. Also, instead of specifying a rectangular area by using the pointers TS1 and TS2, the user may be able to specify a range of any shape by tracing the touch panel with a finger, for example.

[0048] Alternatively, the processing device 17 may automatically identify the first portion P1, rather than having the designation receiving unit 172 accept a designation of the first portion P1 from the user. For example, when the projection image G is projected, the processing device 17 performs an edge extraction process similar to that of the shape information generating unit 173, which will be described later, to identify a central object in the projection image G. The central object is the character portion of "OPEN" in the projection image G shown in FIG. 6, etc. The processing device 17 identifies the central object in the projection image G as the first portion P1. That is, the processing device 17 may determine the first portion P1 by extracting a contour included in the captured image S.

[0049] The shape information generation unit 173 generates shape information E that indicates the shape of the first portion P1 when the screen 20 is viewed from a first direction. The shape information E generated by the shape information generation unit 173 is stored in the storage device 16. The first direction is the imaging direction of the imaging device 15, and in this embodiment, coincides with the projection direction of the image by the optical device 14.

[0050] The shape information generating unit 173 sets one of the captured images S acquired by the captured image acquiring unit 170 as the first captured image S1. At the timing of capturing the first captured image S1, it is preferable that the projection image G is projected onto the screen 20 in an ideal state. The ideal state is, for example, a state in which the screen 20 is positioned perpendicular to the projection direction and no bending occurs in the fabric 22, that is, a state in which no distortion or misalignment occurs in the projection image G.

[0051] 8 is an explanatory diagram schematically illustrating an example of the first captured image S1. Note that the line segments shown inside the first captured image S1 are line segments for dividing an area M that is set to identify the correction location in the correction processing unit 174, which will be described later, and are not actually shown in the first captured image S1. In the first captured image S1, the screen 20 is positioned perpendicular to the projection direction, and no bending occurs in the fabric 22. Therefore, in the first captured image S1, no distortion or misalignment occurs in the projected image G.

[0052] The shape information E generated from the first captured image S1 is referred to as first shape information E1. Also, the shape of the first portion P1 indicated by the first shape information E1 is referred to as the first shape. The shape information generation unit 173 stores the first shape information E1 indicating the first shape, which is the shape of the first portion P1 when the screen 20 is viewed from a first direction.

[0053] The shape information generation unit 173 performs conventional edge detection processing on the captured image S using, for example, a differential filter or a Laplacian filter, to detect the four corners of the projection image G and the contour of the first portion P1. The shape information generation unit 173 generates shape information E that specifies the shape of the contour of the first portion P1. In this embodiment, the shape information E includes a set of coordinate data for the contour of the first portion P1. For example, the shape information generation unit 173 specifies the coordinates of each point that constitutes the contour of the first portion P1, using the upper left corner of the projection image G on the captured image S as the reference coordinate (0,0). This set of coordinates becomes the shape information E. Connecting the coordinates included in the shape information E forms a line segment that indicates the contour of the first portion P1. The shape information E also includes the coordinates of the four corners of the projection image G.

[0054] 6, for example, assume that the coordinates of the top left point of captured image S are (0,0), and the coordinates of the top left point of projected image G on captured image S are (300,200). Also assume that the size ratio between the entire captured image S and the projected image G appearing on captured image S is 0.8. In this case, by subtracting (300,200) from the coordinate values ​​in captured image S and multiplying by 0.8, it is possible to convert the coordinate values ​​into those with the top left point of projected image G as the reference coordinate (0,0).

[0055] That is, the imaging device 15 acquires a first captured image S1 by capturing an image of the first portion P1 from a first direction. The shape information generation unit 173 generates first shape information E1 indicating the first shape based on the first captured image S1. Note that when the processing device 17 is used as a reference, acquisition of the first captured image S1 may be performed by the captured image acquisition unit 170 acquiring the first captured image S1 from the imaging device 15. In this case, the captured image acquisition unit 170 acquires the first captured image S1 obtained by capturing an image of the first portion P1 from the first direction.

[0056] Furthermore, the shape information generation unit 173 repeatedly generates shape information E based on the captured image S, for example, while the projector 10 is projecting an image onto the screen 20. This is because the visual recognition state of the projected image G changes when, for example, the screen 20 is warped due to the influence of wind or the like. The captured image S captured after the first captured image S1 is referred to as the second captured image S2. Furthermore, the shape information E generated using the second captured image S2 is referred to as the second shape information E2. The shape of the first portion P1 indicated by the second shape information E2 is referred to as the second shape.

[0057] That is, the imaging device 15 acquires the first captured image S1 and then acquires the second captured image S2 by capturing the first portion P1 from the first direction. The shape information generation unit 173 detects the shape of the first portion P1 captured in the second captured image S2 and generates the second shape information E2. Note that when the processing device 17 is used as a reference, acquisition of the second captured image S2 may be performed by the captured image acquisition unit 170 acquiring the second captured image S2 from the imaging device 15. In this case, the captured image acquisition unit 170 acquires the second captured image S2 by capturing the first portion P1 from the first direction after acquiring the first captured image S1.

[0058] Note that a known feature point matching technique is used to identify the first portion P1 in the second captured image S2. Specifically, points in the second captured image S2 that match feature points of the shape of the first portion P1 identified by the first shape information E1 are extracted, and the first portion P1 in the second captured image S2 is identified. With this technique, it is possible to determine whether the first portion P1 matches the first shape information E1 even if the first portion P1 in the second captured image S2 is distorted.

[0059] The correction processing unit 174 performs correction processing on the correction target image CX to generate a corrected image C. The correction processing by the correction processing unit 174 is performed based on the correction amount determined by the correction amount determination unit 175, which will be described later. FIG. 7 is an explanatory diagram schematically showing the correction target image CX. As shown in FIG. 7, the correction processing unit 174 divides the correction target image CX into a plurality of rectangular regions M and sets correction points TC at the four vertices of the region M. The correction processing unit 174 can generate a corrected image C by deforming the correction target image CX by shifting the position of each correction point TC. In geometric correction, the movement amount of each correction point TC becomes the correction amount.

[0060] Here, the horizontal axis of the correction target image CX is the X axis, and the vertical axis is the Y axis. In the example of FIG. 7, the correction target image CX is divided into eight equal parts along the X axis and four equal parts along the Y axis, dividing the correction target image CX into 32 regions M. Hereinafter, when identifying each region M, it will be expressed as region M[X,Y]. For example, the region M in the upper left of the correction target image CX will be expressed as region M[1,1], and the region M in the lower right will be expressed as region M[8,4]. The regions M including the first portion P1 are regions M[2,2], M[3,2], M[4,2], M[5,2], M[6,2], M[7,2], M[2,3], M[3,3], M[4,3], M[5,3], M[6,3], and M[7,3].

[0061] The correction amount determination unit 175 determines the amount of correction in the correction processing unit 174. The correction amount determination unit 175 detects a change in the shape of the first portion P1 displayed on the screen 20 using the shape information E generated by the shape information generation unit 173. Specifically, when there is a difference between the second shape information E2 and the first shape information E1, for example, the correction amount determination unit 175 determines that the shape of the first portion P1 viewed from the imaging direction of the imaging device 15 has changed.

[0062] 9 is an explanatory diagram schematically illustrating an example of the second captured image S2. In the second captured image S2, the screen 20 is tilted with respect to the projection direction, and the projected image G appears obliquely distorted when viewed from the capturing direction. The second shape information E2 generated from such a second captured image S2 is different from the first shape information E1 generated from the first captured image S1 shown in FIG. 8. Therefore, the correction amount determination unit 175 determines that the shape of the first portion P1 viewed from the first direction has changed.

[0063] When the shape of the first portion P1 viewed from the first direction changes to a shape different from the first shape, the correction amount determination unit 175 determines a correction amount for the projection image G so as to make the shape of the first portion P1 viewed from the first direction closer to the first shape. At this time, no correction amount is set for the second portion P2. Furthermore, the correction processing unit 174 performs correction on the projection target image GX based on the correction amount determined by the correction amount determination unit 175. That is, when the shape of the first portion P1 viewed from the first direction changes to a shape different from the first shape, the correction processing unit 174 performs correction on the projection target image GX so as to make the shape of the first portion P1 viewed from the first direction closer to the first shape. Furthermore, since no correction amount is set for the second portion P2, the correction processing unit 174 does not perform correction on the second portion P2. The projection image G obtained by projecting the corrected correction image C in this manner is the second image. By performing correction on the projection target image GX, the shape of the projection image G is corrected. That is, the first image is corrected.

[0064] 11 and 12 are explanatory diagrams that schematically show a method for determining the amount of correction. The left diagram in Fig. 11 is a diagram in which a portion corresponding to the region M[5,2] is extracted from the second shape information E2. The right diagram in Fig. 11 is a diagram in which a portion corresponding to the region M[5,2] is extracted from the first shape information E1. The correction points in the region M[5,2] are designated TC1 to TC4.

[0065] The correction amount determination unit 175 detects the four corners of the contour of the first portion P1 in each region M. For example, points B1 to B4 are the four corners of the contour of the first portion P1 in region M[5,2]. Point B1 is the upper left corner, point B2 is the upper right corner, point B3 is the lower right corner, and point B4 is the lower left corner. The correction amount determination unit 175 determines the correction amounts of the correction points TC1 to TC4 so that the arrangement of the points B1 to B4 in the second shape information E2 approaches the arrangement of the points B1 to B4 in the first shape information E1.

[0066] For example, when the shape of the first portion P1 is a curved region M, such as region M[2,2], the four corners of the outline are shaped like points B5 to B8 in FIG. 12. The outline of the first portion P1 is determined so as to include the entire outline of the first portion P1 located in region M and to minimize the area of ​​the quadrangle formed by the outline. The quadrangle may be a square, rectangle, or parallelogram. Ah That's fine.

[0067] For example, the correction amount determination unit 175 first determines the correction amount for an upper left region M[2,2] of the region M including the first portion P1. More specifically, the correction amount determination unit 175 determines the correction amount for the correction point TC1 so that the coordinates of the point B1 in the second shape information E2 match as closely as possible with the coordinates of the upper left point B1 in the first shape information E1. Next, using the correction point TC1 as a reference, the correction amounts for the other correction points TC2 to TC4 are determined so that the coordinates of the points B2 to B4 in the second shape information E2 match as closely as possible with the coordinates of the points B2 to B4 in the first shape information E1.

[0068] After determining the correction amounts for the correction points TC1 to TC4 in the region M[2,2], the correction amount determination unit 175 sequentially determines the correction amounts for the correction points TC in the region M adjacent to the region M[2,2]. For example, the upper left correction point TC and the lower left correction point TC of the region M[3,2] are adjacent to the correction points TC2 and TC3 in the region M[2,2], and therefore the correction amounts have already been determined. Therefore, the correction amount determination unit 175 sequentially determines the upper right correction point TC and the lower right correction point TC of the region M[3,2]. At this time, the correction amount determination unit 175 determines the correction amount for each correction point TC so that the coordinates of the vertices of the contour of the first portion P1 in the region M[3,2] of the second shape information E2 match as closely as possible with the coordinates of the vertices of the contour of the first portion P1 in the region M[3,2] of the first shape information E1.

[0069] The correction amount determination unit 175 repeats this process to determine the correction amount at the correction point TC in the region M including the first portion P1. When all of the correction amounts at the correction points TC in the region M including the first portion P1 have been determined, the correction amount determination unit 175 outputs the correction amount for each correction point TC to the correction processing unit 174, and the correction processing unit 174 performs correction on the correction target image CX.

[0070] That is, the correction amount determination unit 175 compares the shape of the first portion P1 based on the second captured image S2 with the first shape to determine the amount of correction. More specifically, the correction amount determination unit 175 divides the area of ​​the projection target image GX that includes the first portion P1 into two or more rectangular areas M, and determines the amount of correction for each correction point TC in each area M by comparing the shape of the first portion P1 based on the second captured image S2 with the first shape.

[0071] FIG. 10 is an explanatory diagram schematically illustrating an example of a third captured image S3, which is the captured image S after correction processing. In the third captured image S3, the screen 20 is tilted with respect to the projection direction, as in the second captured image S2 shown in FIG. 9, and the contours of the projected image G appear to be distorted diagonally when viewed from the imaging direction. On the other hand, for the first portion P1, the screen 20 appears to be positioned perpendicular to the projection direction, as in the first captured image S1. This prevents the visibility of the first portion P1 of the projected image G from decreasing even if the screen 20 moves or bends with respect to the projection direction. Therefore, for example, if the first portion P1 contains text information, the text information can be easily conveyed to a viewer of the screen 20. Furthermore, because the display changes differently between the second portion P2, which is the background, and the first portion P1 as the screen 20 moves, a visual effect not previously available can be provided to the viewer. For example, in this embodiment, the overall shape of the projected image G changes as the screen 20 moves, while the character portion appears to be stationary, giving the viewer the impression that the characters are floating above the background.

[0072] C-5. Operation of the processing device 17 13 is a flowchart showing the flow of a display method executed by the processing device 17 of the projector 10 in accordance with the control program 162. The processing device 17 waits until an instruction to project an image is issued, for example, by a predetermined operation being performed on the operation device 12 (step S100: NO). The instruction to project an image may be, for example, an instruction to start image projection, or an instruction to switch the display from the currently projected projection target image GX to another projection target image GX. When an instruction to project an image is issued (step S100: YES), the processing device 17 functions as the projection control unit 171 and projects the projection target image GX onto the screen 20 (step S102). The projection target image GX may be an input image I, or may be a corrected image C that has been subjected to necessary correction such as keystone correction.

[0073] When the projection target image GX is displayed on the screen 20 as the projection image G, the processing device 17 functions as the captured image acquisition unit 170 and acquires the first captured image S1 from the imaging device 15 (step S104). The processing device 17 functions as the shape information generation unit 173 and generates first shape information E1 that indicates the shape of the first part P1 that appears in the first captured image S1. The shape indicated by the first shape information E1 is the first shape. The generated first shape information E1 is stored in the storage device 16 (step S106).

[0074] Thereafter, the processing device 17 functions as the captured image acquisition unit 170 and acquires the second captured image S2 from the imaging device 15 (step S108). The processing device 17 functions as the shape information generation unit 173 and generates second shape information E2 that specifies the shape of the first part P1 shown in the second captured image S2 (step S110).

[0075] The processing device 17 functions as the correction amount determination unit 175 and compares the second shape information E2 with the first shape information E1 to determine whether the shape of the first portion P1 has changed to a shape different from the first shape (step S112). If the shape of the first portion P1 has not changed (step S112: NO), the processing device 17 proceeds to step S118. On the other hand, if the shape of the first portion P1 has changed to a shape different from the first shape (step S112: YES), the processing device 17 functions as the correction amount determination unit 175 and determines the correction amount of each correction point TC in the region M including the first portion P1 in the projection target image GX so that the shape of the first portion P1 approaches the shape in the first shape information E1 (step S114). The processing device 17 functions as the correction processing unit 174 and performs a correction process on the projection target image GX based on the correction amount determined in step S114 to generate a corrected image C (step S116). The processing device 17 functions as the projection control unit 171 and projects the corrected image C onto the screen 20 as a new projection target image GX (step S118).

[0076] The processing device 17 returns to step S108 and repeats the subsequent processes until an instruction to end image projection is given, for example, by a predetermined operation being performed on the operation device 12 (step S120: NO). Then, when an instruction to end image projection is given (step S120: YES), the processing device 17 ends the processing according to this flowchart.

[0077] D. Summary of embodiments As described above, in the display method according to the embodiment, when the shape of the first portion P1 displayed on the screen 20 changes to a shape different from the first shape, the display method corrects the shape of the first portion P1 as viewed from the first direction to approximate the first shape, while not correcting the shape of the second portion P2. By doing so, even if the screen 20 moves or warps relative to the projection direction, the visibility of the first portion P1 of the projected image G is unlikely to decrease. Therefore, for example, if the first portion P1 contains text information, the text information can be easily conveyed to viewers of the screen 20. Furthermore, because the display changes differently between the second portion P2, which is the background, and the first portion P1 as the screen 20 moves, viewers can experience a visual effect not previously seen.

[0078] Furthermore, the display method according to the embodiment acquires a first captured image S1 obtained by capturing a first portion P1 from a first direction, and generates first shape information E1 based on the first captured image S1. Furthermore, the display method according to the embodiment generates second shape information E2 based on a second captured image S2 captured after the first captured image S1, and determines the amount of correction by comparing the shape of the first portion P1 based on the second shape information E2 with the first shape. This allows the amount of correction to be determined based on the actual appearance of the first portion P1, thereby improving the accuracy of the correction.

[0079] Furthermore, the display method according to the embodiment divides an area of ​​the projection target image GX that includes the first portion P1 into two or more rectangular areas M, and determines the amount of correction for the correction point TC in each area M. This makes it possible to correct only the first portion P1 of the projection target image GX that includes the first portion P1 and the second portion P2, thereby imparting a visual effect to the projection image G that has not been seen before.

[0080] Furthermore, in the display method according to the embodiment, the designation receiving unit 172 receives an input from the user designating the first portion P1. This allows the user to designate any portion as the first portion P1, thereby improving the degree of freedom in the display form of the projection image G.

[0081] Furthermore, in the display method according to the embodiment, by extracting the contours contained in the projection image G and automatically determining the first part P1, the user is saved the trouble of specifying the first part P1, thereby improving convenience.

[0082] Furthermore, in the display method according to the embodiment, when the first part P1 and the second part P2 are arranged on the same layer, different visual effects can be imparted to the parts on the same layer, thereby improving the degree of freedom in the display form of the projected image G.

[0083] Furthermore, when the shape of the first portion P1 displayed on the screen 20 changes to a shape different from the first shape, the projector 10 according to the embodiment corrects the shape of the first portion P1 as viewed from the first direction in the projected image G so that it approaches the first shape, but does not correct the shape of the second portion P2. This prevents the visibility of the first portion P1 of the projected image G from decreasing even when the screen 20 moves or warps relative to the projection direction. Therefore, for example, when the first portion P1 contains text information, the text information can be easily conveyed to viewers of the screen 20. Furthermore, because the display changes differently between the second portion P2, which is the background, and the first portion P1 as the screen 20 moves, viewers can experience a visual effect not previously seen.

[0084] Furthermore, by executing the control program 162, the processing device 17 according to the embodiment corrects the shape of the first portion P1 of the projected image G when viewed from the first direction to approximate the first shape when the shape of the first portion P1 displayed on the screen 20 changes to a shape different from the first shape, while not correcting the shape of the second portion P2. This prevents the visibility of the first portion P1 of the projected image G from decreasing even when the screen 20 moves or warps relative to the projection direction. Therefore, for example, when the first portion P1 contains text information, the text information can be easily conveyed to viewers of the screen 20. Furthermore, because the display changes differently between the second portion P2 (the background) and the first portion P1 as the screen 20 moves, viewers can experience a visual effect not previously seen.

[0085] The processing of the processing device 17 in this embodiment may be performed by a plurality of processing devices. For example, an image processing circuit may be provided separately from the processing device that controls the entire projector 10. The image processing circuit performs image processing on input image data and converts it into an image signal. The image processing circuit is configured, for example, by an integrated circuit. Integrated circuits include LSI (Large Scale Integration), ASIC, PLD, FPGA, SoC (System on Chip), etc. Furthermore, an analog circuit may be included as part of the configuration of the integrated circuit. [Explanation of symbols]

[0086] 1...display system, 10...projector, 12...operation device, 13...communication device, 14...optical device, 15...imaging device, 16...storage device, 17...processing device, 20...screen, 170...captured image acquisition unit, 171...projection control unit, 172...designation reception unit, 173...shape information generation unit, 174...correction processing unit, 175...correction amount determination unit, C...corrected image, CX...image to be corrected, E...shape information, G...projected image, GX...image to be projected, I...input image, P1...first part, P2...second part, S...captured image.

Claims

1. displaying a first image on a screen using a projector, the first image including a first portion and a second portion different from the first portion; storing information indicating a first shape that is a shape of the first portion when the screen is viewed from a first direction; when the shape of the first portion as viewed from the first direction has changed to a shape different from the first shape, correcting the first image so that the shape of the first portion as viewed from the first direction approaches the first shape, and not correcting the second portion, thereby displaying a second image on the screen using the projector; The storing includes: acquiring a first captured image by capturing an image of the first portion from the first direction; generating information indicating the first shape based on the first captured image; Display method.

2. Displaying the second image includes: After acquiring the first captured image, acquiring a second captured image by capturing an image of the first portion from the first direction; determining a correction amount by comparing a shape of the first portion based on the second captured image with the first shape; The display method according to claim 1.

3. Determining the correction amount includes: Dividing an area of ​​the first image including the first portion into two or more rectangular areas; and determining the correction amount for each correction point in each of the rectangular regions by said comparing. The display method according to claim 2.

4. further comprising receiving an input specifying the first portion. The display method according to any one of claims 1 to 3.

5. determining the first portion by extracting a contour included in the first image; The display method according to any one of claims 1 to 3.

6. the first portion and the second portion are included in the same layer in the first image; The display method according to any one of claims 1 to 5.

7. displaying a first image on a screen using a projector, the first image including a first portion and a second portion different from the first portion; storing information indicating a first shape that is a shape of the first portion when the screen is viewed from a first direction; when the shape of the first portion as viewed from the first direction has changed to a shape different from the first shape, correcting the first image so that the shape of the first portion as viewed from the first direction approaches the first shape, and not correcting the second portion, thereby displaying a second image on the screen using the projector; further comprising receiving an input specifying the first portion. Display method.

8. displaying a first image on a screen using a projector, the first image including a first portion and a second portion different from the first portion; determining the first portion by extracting a contour included in the first image; storing information indicating a first shape that is a shape of the first portion when the screen is viewed from a first direction; when the shape of the first portion as viewed from the first direction has changed to a shape different from the first shape, correcting the first image so that the shape of the first portion as viewed from the first direction approaches the first shape, and not correcting the second portion, thereby displaying a second image on the screen using the projector; determining the first portion by extracting a contour included in the first image; Display method.

9. an optical device; a control device for controlling the optical device, The control device displaying a first image on a screen using the optical device, the first image including a first portion and a second portion different from the first portion; storing information indicating a first shape that is a shape of the first portion when the screen is viewed from a first direction; when the shape of the first portion as viewed from the first direction has changed to a shape different from the first shape, correcting the first image so that the shape of the first portion as viewed from the first direction approaches the first shape, and displaying a second image obtained by not performing the correction on the second portion on the screen using the optical device; The storing includes: acquiring a first captured image by capturing an image of the first portion from the first direction; generating information indicating the first shape based on the first captured image; projector.

10. The processing device displaying a first image on a screen using a projector, the first image including a first portion and a second portion different from the first portion; storing information indicating a first shape that is a shape of the first portion when the screen is viewed from a first direction; when the shape of the first portion as viewed from the first direction has changed to a shape different from the first shape, correcting the first image so that the shape of the first portion as viewed from the first direction approaches the first shape, and displaying a second image obtained by not performing the correction on the second portion on the screen using the projector; The storing includes: acquiring a first captured image by capturing an image of the first portion from the first direction; generating information indicating the first shape based on the first captured image; program.

Images