Method for adjusting projection image, projection system, and control unit

Abstract

To provide a method for adjusting a projection image for accurately adjusting a projection image to a shape of a projection surface.SOLUTION: A second control unit 250 of a projector 200, based on a projection area 51 in a first picked-up image 70A obtained by picking up an image of a screen 5, a first image 10 in the first picked-up image 70A, and the first image 10 on a liquid crystal panel 235, specifies a first area on the liquid crystal panel 235 corresponding to the projection area 51, draws a second image 20 larger than the first image 10 in the first area, and based on the projection area 51 in a second picked-up image 70B obtained by picking up an image of the screen 5 on which image light corresponding to the second image 20 is projected, the second image 20 in the second picked-up image 70B, and the second image 20 on the liquid crystal panel 235, specifies a second area on the liquid crystal panel 235 corresponding to the projection area 51.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a method for adjusting a projected image, a projection system, and a control device. [Background technology]

[0002] Conventionally, there is known an adjustment method for adjusting the shape of an image projected by a projector to the shape of a projection surface. For example, Patent Document 1 discloses a projector device that projects a test pattern onto a screen, calculates the distances between the projector device and multiple points on the display screen from an image captured by an image sensor, and detects the positional relationship between the projector device and the screen. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-246502 Summary of the Invention [Problem to be solved by the invention]

[0004] However, when capturing an image of the projection surface onto which the image is projected and correcting the display position and shape of the projected image based on the captured image, if the image projected onto the projection surface extends beyond the projection surface, the accuracy of adjusting the image may decrease. [Means for solving the problem]

[0005] The present disclosure provides a method for drawing a first image of a first size on a drawing panel, projecting the first image onto a projection surface, detecting an area of ​​the projection surface and the first image from a first captured image of the projection surface onto which the first image is projected, specifying a first area on the drawing panel corresponding to the area of ​​the projection surface based on the area of ​​the projection surface in the first captured image, the first image in the first captured image, and the first image on the drawing panel, and drawing a second image of a second size larger than the first size within the first area. a second image on the projection surface; detecting an area of ​​the projection surface and the second image from a second captured image of the projection surface onto which the second image is projected; specifying a second area on the drawing panel that corresponds to the area of ​​the projection surface based on the area of ​​the projection surface in the second captured image, the second image in the second captured image, and the second image on the drawing panel; drawing the projection image within the second area; and projecting the projection image on the projection surface.

[0006] The present disclosure provides a projector including a drawing panel that projects a first image of a first size drawn on the drawing panel onto a projection surface; an imaging device that captures an image of the projection surface onto which the first image is projected to generate a first captured image; and a control device that executes the following: detecting an area of ​​the projection surface and the first image from the first captured image; specifying a first area on the drawing panel that corresponds to the area of ​​the projection surface based on the area of ​​the projection surface in the first captured image, the first image in the first captured image, and the first image on the drawing panel; and transmitting first information indicating the first area to the projector; A projection system that performs the following: drawing a second image of a second size larger than the first size within the first area; and projecting the second image onto the projection surface; wherein the imaging device images the projection surface onto which the second image is projected to generate a second captured image; and the control device detects the area of ​​the projection surface and the second image from the second captured image; identifies a second area of ​​the drawing panel corresponding to the area of ​​the projection surface based on the area of ​​the projection surface in the second captured image, the second image in the second captured image, and the second image on the drawing panel; and transmits second information indicating the second area to the projector.

[0007] The present disclosure provides a method for projecting a first image of a first size onto a projection surface, the method comprising: receiving, from an imaging device, a first captured image of a projection surface onto which a first image of a first size is projected; receiving, from a projector that projects the first image onto the projection surface, information on the first image drawn on a drawing panel of the projector; detecting, from the first captured image, an area of ​​the projection surface and the first image; specifying, based on the area of ​​the projection surface in the first captured image, the first image in the first captured image, and information on the first image, a first area on the drawing panel corresponding to the area of ​​the projection surface; a control device that executes the following operations: transmitting information to the projector; receiving from the imaging device a second captured image of the projection surface onto which a second image of a second size larger than the first size is projected; detecting an area of ​​the projection surface and the second image from the second captured image; identifying a second area on the drawing panel corresponding to the area of ​​the projection surface based on the area of ​​the projection surface in the second captured image, the second image in the second captured image, and the first information; and transmitting second information indicating the second area to the projector. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing the system configuration of a projection system. [Figure 2] FIG. 2 is a block diagram showing the configuration of an image projection unit. [Figure 3] FIG. 10 is a diagram showing an example of a screen. [Figure 4] FIG. 4 is a diagram showing an example of a first image drawn on a liquid crystal panel. [Figure 5] FIG. 4 is a diagram showing an example of a first captured image. [Figure 6] 4 is a diagram showing the positions of the four vertices of the projection area in the panel coordinate system. [Figure 7] FIG. 10 is a diagram showing an example of a second image drawn on the liquid crystal panel. [Figure 8] FIG. 10 is a diagram showing an example of a second image projected onto a screen. [Figure 9] FIG. 10 is a diagram showing an example of a second captured image. [Figure 10] 4 is a diagram showing the positions of the four vertices of the projection area in the panel coordinate system. [Figure 11] FIG. 10 is a diagram showing an example of a projected image projected onto a screen. [Figure 12] FIG. 4 is a diagram showing an example of an operation screen displayed on a touch panel. [Figure 13] 10 is a flowchart showing the operation of the projector. [Figure 14] 10A and 10B are diagrams showing other examples of the first image and the second image. [Figure 15] FIG. 10 is a diagram showing another system configuration of the projection system. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

[0010] 1. Explanation of the projection system configuration FIG. 1 is a diagram showing the system configuration of a projection system 1A. The projection system 1A includes an information processing device 100 and a projector 200.

[0011] The information processing device 100 is used as a device that supplies the projector 200 with image data that is the source of the image projected by the projector 200, and accepts operations to change the shape and display position of the image projected by the projector 200. The information processing device 100 may be, for example, a smartphone, a tablet computer, or a notebook computer.

[0012] The projector 200 generates image light based on image data supplied from the information processing device 100, and projects the generated image light onto the screen 5.

[0013] 2. Configuration of information processing device The configuration of the information processing device 100 will be described. The information processing device 100 includes a first wireless interface 110, a camera 120, a touch panel 130, and a first control unit 150. Hereinafter, the interface 110 will be abbreviated as I / F.

[0014] The first wireless I / F 110 is a wireless communication device including an interface circuit compatible with standards such as Bluetooth and Wi-Fi. Bluetooth and Wi-Fi are registered trademarks. The first wireless I / F 110 is wirelessly connected to the projector 200 and performs mutual data communication with the projector 200. Furthermore, the information processing device 100 in this embodiment may include a wired I / F 140 included in the information processing device 100 constituting a projection system 1B shown in a modified example described later, and may perform data communication with the projector 200 through wired communication using the wired I / F 140. The wired I / F 140 may include an interface circuit.

[0015] The camera 120 includes an imaging lens, an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary MOS), and a data processing circuit. The imaging lens, imaging element, and data processing circuit are not shown. The camera 120 captures an image under the control of the first control unit 150, and generates a captured image 70. The camera 120 outputs the generated captured image 70 to the first control unit 150.

[0016] The touch panel 130 includes a display panel and a touch sensor, and receives touch operations from the user. The display panel and the touch sensor are not shown. The display panel may be, for example, a liquid crystal panel or an organic EL (Electro Luminescence) panel. The touch sensor detects the user's touch operation on the display panel. A coordinate system is set in advance on the display panel, and the touch sensor outputs an operation signal to the first control unit 150, the operation signal including coordinate information indicating the position in the coordinate system touched by the user.

[0017] The first control unit 150 is a computer device that includes a first storage unit 160 and a first processor 170 .

[0018] The first storage unit 160 includes a non-volatile memory such as a ROM (Read Only Memory) and a volatile memory such as a RAM (Random Access Memory). The ROM stores a control program 165 such as an OS (Operating System) or an application program, and a setting file that defines the operation of the information processing device 100. The RAM is used as a calculation area for the first processor 170.

[0019] The first processor 170 is an arithmetic processing device including one or more CPUs (Central Processing Units), MPUs (Micro-Processing Units), etc. The first processor 170 executes a control program 165 to control the operation of each part of the information processing device 100.

[0020] 3.Projector configuration Next, the configuration of the projector 200 will be described. The projector 200 includes a second wireless I / F 210, an image processing unit 220, a frame memory 225, an image projection unit 230, and a second control unit 250.

[0021] The second wireless I / F 210 is a wireless communication device including an interface circuit compatible with standards such as Bluetooth and Wi-Fi. The second wireless I / F 210 is wirelessly connected to the information processing device 100 and performs mutual data communication with the information processing device 100. The projector 200 may also perform data communication with the information processing device 100 through wired communication using a wired I / F (not shown). The wired I / F may include an interface circuit.

[0022] A frame memory 225 is connected to the image processing unit 220. The image processing unit 220 expands image data input from the second control unit 250 or the second wireless I / F 210 in the frame memory 225. The frame memory 225 is configured by, for example, a Synchronous Dynamic Random Access Memory (SDRAM).

[0023] The image processing unit 220 performs image processing such as resolution conversion or resizing, distortion correction, shape correction, digital zooming, and adjustment of image color and brightness on the image data stored in the frame memory 225. The image processing unit 220 executes image processing specified by the second control unit 250, and performs processing using parameters input from the second control unit 250 as necessary. Of course, the image processing unit 220 can also execute a combination of multiple image processing operations from the above. The image processing unit 220 reads the processed image data from the frame memory 225 and outputs the read image data to the light modulation device 233 of the image projection unit 230.

[0024] The image processing unit 220 and the frame memory 225 are configured, for example, by an integrated circuit. The integrated circuit includes an LSI (Large Scale Integrated Circuit), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field-Programmable Gate Array), an SoC (System-on-a-chip), etc. An analog circuit may be included as part of the integrated circuit configuration, or the second control unit 250 may be configured in combination with an integrated circuit.

[0025] FIG. 2 is a block diagram showing the configuration of the image projection unit 230. As shown in FIG. The configuration of the image projection unit 230 will be described with reference to FIG. The image projection unit 230 includes a light source 231, a light modulation device 233, and an optical unit 241. The light modulation device 233 of this embodiment includes three transmissive liquid crystal panels 235R, 235G, and 235B corresponding to the three colors of red, green, and blue as light modulation elements, and a drive circuit 239 that drives these liquid crystal panels 235R, 235G, and 235B. R is an abbreviation for Red, G is an abbreviation for Green, and B is an abbreviation for Blue. Hereinafter, the liquid crystal panels 235R, 235G, and 235B included in the projector 200 will be collectively referred to as liquid crystal panels 235. The light modulation elements included in the light modulation device 233 are not limited to the transmissive liquid crystal panels 235R, 235G, and 235B, and may be, for example, reflective liquid crystal panels or DMDs (Digital Micromirror Devices). The liquid crystal panels 235R, 235G, and 235B, the reflective liquid crystal panels, and the DMDs correspond to drawing panels.

[0026] The light source 231 includes a discharge light source lamp such as an extra-high pressure mercury lamp or a metal halide lamp, or a solid-state light source such as a light-emitting diode or a semiconductor laser. Light emitted from the light source 231 is incident on the liquid crystal panel 235. The liquid crystal panels 235R, 235G, and 235B are each composed of a transmissive liquid crystal panel with liquid crystal sealed between a pair of transparent substrates. The liquid crystal panel 235R modulates red light, the liquid crystal panel 235G modulates green light, and the liquid crystal panel 235B modulates blue light. Each liquid crystal panel 235 has a panel region 237 consisting of a plurality of pixels arranged in a matrix. The panel region 237 of the liquid crystal panel 235R is referred to as panel region 237R, the panel region 237 of the liquid crystal panel 235G is referred to as panel region 237G, and the panel region 237 of the liquid crystal panel 235B is referred to as panel region 237B. Furthermore, the panel areas 237R, 237G, and 237B are collectively referred to as the panel area 237.

[0027] Image data is input to the drive circuit 239 from the image processing unit 220. The drive circuit 239 applies a drive voltage corresponding to the input image data to each pixel in the panel area 237, changing the light transmittance of each pixel to a transmittance corresponding to the image data. Light emitted from the light source 231 is modulated by passing through the panel area 237, and image light corresponding to the image data is formed for each color light of red, green, and blue. The formed image light of each color is combined by a color combining optical system (not shown) to become image light representing a color image. The optical unit 241 includes a projection lens and the like, and enlarges and projects the image light modulated by the liquid crystal panel 235 onto the projection area 51. As a result, a projected image corresponding to the image light is displayed in the projection area 51.

[0028] FIG. 3 is a diagram showing an example of the screen 5. As shown in FIG. Here, the screen 5 onto which the projector 200 projects an image will be described with reference to FIG. The screen 5 includes a projection area 51 and a background area 53. The projection area 51 corresponds to a projection surface. The projection area 51 is an area onto which image light is projected and onto which a projected image corresponding to the projected image light is displayed. The background area 53 is an area that forms the background of the projection area 51 and is adjacent to the projection area 51 on the outside. The background area 53 is also an area that forms the background of the projection area 51, and is an area where an image is not visibly displayed even if image light is projected onto the background area 53. There is a difference in luminance between the projection area 51 and the background area 53. For example, the projection area 51 is an area with a high luminance value such as white, and the background area 53 is an area with a low luminance value such as black. The background area 53 only needs to have a luminance difference of a certain value or more with the projection area 51, and may be substituted, for example, by a wall in a room or a curtain.

[0029] Returning to FIG. 1, the configuration of projector 200 will be further described. The second control unit 250 is a computer device that includes a second storage unit 260 and a second processor 270 .

[0030] The second storage unit 260 includes a volatile memory such as a RAM, and a non-volatile memory such as a ROM. The RAM is used for temporary storage of various data, and the ROM stores a control program 265 used to control the operation of the projector 200 and various setting information.

[0031] The second processor 270 is an arithmetic processing unit including one or more CPUs and MPUs. The second processor 270 executes the control program 265 to control each unit of the projector 200.

[0032] The second control unit 250 starts the image adjustment when, for example, it receives an instruction signal to start the image adjustment from the information processing device 100. This image adjustment is a process of adjusting distortion of the shape of the projected image and the display position and display size of the projected image displayed on the screen 5.

[0033] FIG. 4 is a diagram showing an example of the first image 10 drawn in the panel area 237. As shown in FIG. The second control unit 250 first causes the first image 10 to be displayed on the screen 5. As shown in Fig. 4, the first image 10 is a pattern image in which first rectangular figures 11 and second rectangular figures 13, which have the same shape and size, are alternately arranged in the vertical and horizontal directions of the first image 10 on the panel area 237. The first rectangular figures 11 are black rectangles, and the second rectangular figures 13 are white rectangles. The color of the first rectangular figures 11 is not limited to black, and the color of the second rectangular figures 13 is not limited to white. The colors of the first rectangular figures 11 and the second rectangular figures 13 may be different; for example, colors that are complementary in a color wheel may be used for the first rectangular figures 11 and the second rectangular figures 13.

[0034] The second control unit 250 generates the first image 10. The first image 10 may be stored in advance in the second storage unit 260. After generating the first image 10, the second control unit 250 controls the image processing unit 220 and the image projection unit 230 to draw the generated first image 10 in the panel area 237 of the liquid crystal panel 235. Light emitted from the light source 231 passes through the panel area 237 and is modulated into image light corresponding to the first image 10, and the modulated image light is displayed in the projection area 51.

[0035] The first image 10 is formed in a first size. The size of the first image 10 refers to, for example, the number of display pixels that constitute the first image 10 when the first image 10 is rendered on the liquid crystal panel 235. In this case, the size of the first image 10 can be rephrased as the display resolution of the first image 10 or the resolution of the first image 10. The size of the first image 10 may be the total number of display pixels that constitute the first image 10. Alternatively, the size of the first image 10 may be the number of display pixels in the horizontal direction or the number of display pixels in the vertical direction that constitute the first image 10. The first size is expressed by the resolution of the first image 10, or the number of display pixels of the liquid crystal panel 235 on which the first image 10 is drawn. The same applies to the second size, which will be described later.

[0036] The resolution of the first image 10 is, for example, a preset resolution, which is sufficiently smaller than the panel resolution of the liquid crystal panel 235. Here, the reason why the resolution of the first image 10 is set to a resolution sufficiently smaller than the panel resolution of the liquid crystal panel 235 will be described.

[0037] When capturing an image of the screen 5 using the camera 120 provided in the information processing device 100 and detecting the projection image or the boundary between the projection area 51 and the background area 53 using the captured image 70 generated, it is preferable to display the projection image near the boundary between the projection area 51 and the background area 53. This is because the values ​​of the distortion parameters of the camera 120 are unknown, and the greater the distance between the projected image and the boundary between the projection area 51 and the background area 53, the greater the error tends to increase due to the influence of the distortion parameters.

[0038] However, if the size of the first image 10 is increased indiscriminately in order to display the first image 10 near the boundary between the projection area 51 and the background area 53, part of the first image 10 may fall outside the projection area 51, resulting in failure to detect the first image 10. For this reason, in this embodiment, the resolution of the first image 10 is set to a size that is sufficiently smaller than the panel resolution of the liquid crystal panel 235, thereby preventing part of the projected first image 10 from falling outside the projection area 51. The camera 120 captures an image of the screen 5 on which the first image 10 is displayed, and generates a captured image 70. The captured image 70, which is obtained by capturing an image of the screen 5 on which the first image 10 is displayed, is called a first captured image 70A. The first image 10 and the projection area 51 are detected from the first captured image 70A, and the second image 20 is projected again onto the screen 5 based on the detection results. At this time, since the projection area 51 has been detected from the first captured image 70A, the shape and resolution of the second image 20 can be changed based on the detection results, so that the second image 20 can be displayed near the boundary line between the projection area 51 and the background area 53. The second image 20 will be described later.

[0039] The first image 10 includes four contact points: a first contact point T1, a second contact point T2, a third contact point T3, and a fourth contact point T4. The four contact points, the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4, included in the first image 10 are an example of an image of at least four preset shapes.

[0040] The first contact point T1 is located at a position corresponding to the upper left vertex of the first image 10. The first contact point T1 is the point where the lower right vertex of the first rectangular shape 11A, the lower left vertex of the second rectangular shape 13B, the upper right vertex of the second rectangular shape 13C, and the upper left vertex of the first rectangular shape 11D meet in the drawing. The first rectangular figure 11A, the second rectangular figure 13B, the second rectangular figure 13C, and the first rectangular figure 11D correspond to images of a preset shape included in the first image 10. The first rectangular figure 11A is a figure located at the upper left of the first image 10 when viewed from the drawing. The second rectangular figure 13B is a figure that is in contact with the first rectangular figure 11A on the right side of the first rectangular figure 11A when viewed from the drawing. The second rectangular figure 13C is a figure that is in contact with the first rectangular figure 11A below the first rectangular figure 11A in the drawing. The first rectangular figure 11D is a figure that is in contact with the second rectangular figure 13C on the right side of the second rectangular figure 13C when viewed from the drawing. Here, viewing the drawing means viewing the drawing in an orientation in which the origin O1 of a panel coordinate system, which will be described later, or the origin O2 of an image capture coordinate system is at the upper left.

[0041] The second contact point T2 is located at a position corresponding to the top right vertex of the first image 10 as viewed in the drawing. The second contact point T2 is a point where the lower left vertex of the second rectangular figure 13E meets the lower right vertex of the first rectangular figure 11F, the upper left vertex of the first rectangular figure 11G, and the upper right vertex of the second rectangular figure 13H. The second rectangular figure 13E, the first rectangular figure 11F, the first rectangular figure 11G, and the second rectangular figure 13H correspond to images of predetermined shapes included in the first image 10. The second rectangular figure 13E is a figure located at the upper right of the first image 10 in the drawing. The first rectangular figure 11F is a figure that is on the left side of the second rectangular figure 13E and is in contact with the second rectangular figure 13E when viewed from the drawing. The first rectangular figure 11G is a figure that is in contact with the second rectangular figure 13E below the second rectangular figure 13E in the drawing. The second rectangular figure 13H is a figure that is in contact with the first rectangular figure 11G on the left side of the first rectangular figure 11G when viewed from the drawing.

[0042] The third contact point T3 is located at a position corresponding to the bottom left vertex of the first image 10 as viewed in the drawing. The third contact point T3 is the point where the upper right vertex of the second rectangular figure 13I meets the upper left vertex of the first rectangular figure 11J, the lower right vertex of the first rectangular figure 11K, and the lower left vertex of the second rectangular figure 13L. The second rectangular figure 13I, the first rectangular figure 11J, the first rectangular figure 11K, and the second rectangular figure 13L correspond to images of a predetermined shape included in the first image 10. The second rectangular figure 13I is a figure located at the bottom left of the first image 10 when viewed from the drawing. The first rectangular figure 11J is a figure that is in contact with the second rectangular figure 13I on the right side of the second rectangular figure 13I when viewed from the drawing. The first rectangular figure 11K is a figure that is in contact with the second rectangular figure 13I above the second rectangular figure 13I in the drawing. The second rectangular figure 13L is a figure that is in contact with the first rectangular figure 11K on the right side of the first rectangular figure 11K when viewed from the drawing.

[0043] The fourth contact point T4 is a figure located at the bottom right of the first image 10 in the drawing. The fourth contact point T4 is the point where the upper left vertex of the first rectangular figure 11M meets the upper right vertex of the second rectangular figure 13N, the lower left vertex of the second rectangular figure 13O, and the lower right vertex of the first rectangular figure 11P. The first rectangular figure 11M, the second rectangular figure 13N, the second rectangular figure 13O, and the first rectangular figure 11P correspond to images of a predetermined shape included in the first image 10. The first rectangular figure 11M is a figure located at the bottom right of the first image 10 when viewed from the drawing. The second rectangular figure 13N is a figure that is in contact with the first rectangular figure 11M on the left side of the first rectangular figure 11M when viewed from the drawing. The second rectangular figure 13O is a figure that is in contact with the first rectangular figure 11M above the first rectangular figure 11M in the drawing. The first rectangular figure 11P is a figure that is on the left side of the second rectangular figure 13O and is in contact with the second rectangular figure 13O when viewed from the drawing.

[0044] When the second control unit 250 displays the first image 10 on the screen 5, it transmits an image capture instruction and an acquisition request for the first captured image 70A to the information processing device 100. When the first control unit 150 receives the image capture instruction and the acquisition request from the projector 200, it displays an image capture guide on the touch panel 130. The image capture guide is a display that guides the user to capture an image of the screen 5 by operating the camera 120.

[0045] When the imaging guide is displayed on the touch panel 130, the user holds the information processing device 100 in their hand and moves to the imaging position. This imaging position is a position where the entire screen 5 is included within the angle of view of the camera 120. More preferably, the imaging position is a position where the center of the horizontal direction of the angle of view of the camera 120 is located at the center of the horizontal direction of the screen 5. When the user moves to the imaging position, the user causes the camera 120 to capture an image. When the first captured image 70A is input from the camera 120, the first control unit 150 transmits the input first captured image 70A to the projector 200.

[0046] FIG. 5 is a diagram showing an example of the first captured image 70A. When the second control unit 250 receives the first captured image 70A from the information processing device 100, the second control unit 250 performs image analysis on the received first captured image 70A to detect the first image 10 and the projection area 51.

[0047] The second control unit 250 sets a coordinate system on the input first captured image 70A. This coordinate system is called the captured coordinate system. As shown in FIG. 5, the captured coordinate system has an origin O2 at the top left of the first captured image 70A, the horizontal direction of the first captured image 70A as the u-axis, and the vertical direction of the first captured image 70A as the v-axis. The second control unit 250 detects four contact points, the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4 shown in FIG. 4, from the first captured image 70A as the first image 10. The second control unit 250 acquires the coordinates of the four contact points, the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4, in the captured coordinate system set on the first captured image 70A. The four contact points, the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4, in the captured coordinate system set on the first captured image 70A correspond to four points included in the first image in the first captured image.

[0048] Furthermore, the second control unit 250 detects the projection area 51 from the first captured image 70A. The second control unit 250 detects the positions of the four vertices of the projection area 51. The second control unit 250 detects the projection area 51 based on changes in luminance values ​​of the first captured image 70A. Because the white projection area 51 is white and the background area 53 is black, there is a large change in luminance value at the boundary between the projection area 51 and the background area 53. The second control unit 250 compares the change in luminance value of pixels in the first captured image 70A with a preset threshold value, and identifies pixels whose change in luminance value is equal to or greater than the threshold value.

[0049] For example, the second control unit 250 detects a pixel where the difference in luminance value between the target pixel and the previous target pixel is equal to or greater than a threshold value while shifting the target pixel from the vertical center of the first captured image 70A to the vertical outside. When the second control unit 250 detects a pixel where the difference in luminance value is equal to or greater than the threshold value, the second control unit 250 determines that the target pixel is a pixel located in the background area 53 outside the projection area 51. The second control unit 250 also determines that the pixel previous to the target pixel is a pixel located within the projection area 51 and on the boundary line between the projection area 51 and the background area 53. The second control unit 250 acquires the coordinates in the imaging coordinate system of the pixel determined to be located on the boundary line.

[0050] Similarly, the second control unit 250 detects a pixel where the difference in luminance value between the target pixel and the previous target pixel is equal to or greater than a threshold value while shifting the target pixel from the horizontal center of the first captured image 70A to the horizontal outer side. When the second control unit 250 detects a pixel where the difference in luminance value is equal to or greater than the threshold value, it determines that the target pixel is a pixel located in the background area 53 outside the projection area 51. Furthermore, the second control unit 250 determines that the pixel just before the target pixel is a pixel located within the projection area 51 and on the boundary line between the projection area 51 and the background area 53. The second control unit 250 acquires the coordinates in the imaging coordinate system of the pixel determined to be located on the boundary line.

[0051] After detecting pixels located on the boundary lines of the projection area 51 on each of the four sides of the projection area 51, the second control unit 250 identifies pixels located on the four vertices of the projection area 51. The upper left of the projection area 51 is denoted as vertex M1, the upper right as vertex M2, the lower left as vertex M3, and the lower right as vertex M4. Vertex M1 corresponds to the pixel located on the boundary line between projection area 51 and background area 53 that has the smallest coordinate values ​​on the u-axis and v-axis. Vertex M2 corresponds to the pixel located on the boundary line between projection area 51 and background area 53 that has the largest u-axis coordinate value and the smallest v-axis coordinate value. Vertex M3 corresponds to the pixel located on the boundary line between projection area 51 and background area 53 that has the smallest u-axis coordinate value and the largest v-axis coordinate value. Vertex M4 corresponds to the pixel located on the boundary line between the projection area 51 and the background area 53 that has the largest coordinate values ​​on the u axis and v axis.

[0052] Next, the second control unit 250 calculates the determinant of the projective transformation. The second control unit 250 calculates the determinant of the projective transformation using the coordinates in the panel coordinate system of the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4 in the first image 10 and the coordinates in the imaging coordinate system of the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4 in the first captured image 70A. The panel coordinate system is a coordinate system set in the panel area 237 of the liquid crystal panel 235, and as shown in FIG. 4, the origin O1 is at the upper left of the panel area 237, and the horizontal direction is the s-axis and the vertical direction is the t-axis when viewed from the drawing.

[0053] The coordinates of the first contact T1, the second contact T2, the third contact T3, and the fourth contact T4 in the panel coordinate system are coordinates that indicate the positions of the first contact T1, the second contact T2, the third contact T3, and the fourth contact T4 in the first image 10 drawn on the liquid crystal panel 235. Furthermore, the coordinates of the first contact T1, the second contact T2, the third contact T3, and the fourth contact T4 in the imaging coordinate system are coordinates of the first contact T1, the second contact T2, the third contact T3, and the fourth contact T4 detected from the first captured image 70A. The four contacts of the first contact T1, the second contact T2, the third contact T3, and the fourth contact T4 in the first image 10 in the panel coordinate system correspond to four points included in the first image on the drawing panel. Furthermore, each of the four contact points, the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4, of the first image 10 in the panel coordinate system corresponds to each of the four contact points, the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4, in the imaging coordinate system set on the first captured image 70A.

[0054] FIG. 6 is a diagram showing the positions of the four vertices of the projection area 51 in the panel coordinate system. The second control unit 250 converts the coordinates of the four vertices M1, M2, M3, and M4 of the projection area 51 from the coordinates of the imaging coordinate system to the coordinates of the panel coordinate system using the determinant of the projective transformation calculated. The second control unit 250 converts the coordinates of the four vertices M1, M2, M3, and M4 of the projection area 51 in the imaging coordinate system using the determinant of the projective transformation to convert them into the coordinates of the four vertices in the panel coordinate system. The four vertices M1, M2, M3, and M4 of the projection area 51 in the panel coordinate system are defined as vertices N1, N2, N3, and N4. The area defined by the four vertices N1, N2, N3, and N4 in the panel coordinate system converted using the determinant of the projective transformation calculated based on the first captured image 70A and the first image 10 corresponds to the first area.

[0055] FIG. 7 is a diagram showing an example of the second image 20 drawn on the liquid crystal panel 235. As shown in FIG. The second control unit 250 generates the second image 20 based on vertices N1, N2, N3, and N4 of the four vertices M1, M2, M3, and M4 of the projection area 51 in the panel coordinate system. The second image 20 is generated with a second size larger than the first size of the first image 10. The second control unit 250 generates the second image 20 so that the four vertices of the second image 20 are located at the positions of vertices N1, N2, N3, and N4, respectively. Like the first image 10, the second image 20 is a pattern image in which first rectangular shapes 11 and second rectangular shapes 13 are alternately arranged in the vertical and horizontal directions of the first image 10. However, the multiple first rectangular shapes 11 and multiple second rectangular shapes 13 included in the second image 20 do not have the same shape or size. This is because the shape of the second image 20 is deformed to correspond to the distortion of the first image 10 displayed in the projection area 51.

[0056] Furthermore, the second control unit 250 may generate a determinant of the projective transformation, and generate the second image 20 by transforming the first image 10 using the generated determinant of the projective transformation. The second control unit 250 generates a determinant of a projective transformation based on the coordinates in the panel coordinate system of the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4 of the first image 10, and the coordinates in the panel coordinate system of the vertices N1, N2, N3, and N4. The second control unit 250 may generate the second image 20 by transforming the coordinates of each vertex of the first rectangular figure 11 and the second rectangular figure 13 included in the first image 10 using the generated determinant of the projective transformation.

[0057] FIG. 8 is a diagram showing an example of the second image 20 projected onto the screen 5. As shown in FIG. When the second control unit 250 generates the second image 20, it controls the image processing unit 220 and the image projection unit 230 to draw the generated second image 20 in the panel area 237 of the liquid crystal panel 235. Light emitted from the light source 231 passes through the panel area 237 and is modulated into image light corresponding to the second image 20, and the modulated image light is displayed in the projection area 51. The second image 20 is displayed within the projection area 51 of the screen 5 and closer to the boundary line between the projection area 51 and the background area 53 than the first image 10. This reduces errors due to the influence of the distortion parameters of the camera 120, and makes it possible to accurately detect the coordinates of the panel coordinate system of the second image 20 and the projection area 51 based on the captured image 70 of the camera 120.

[0058] FIG. 9 is a diagram showing an example of the second captured image 70B. Hereinafter, the captured image 70 obtained by capturing the screen 5 on which the second image 20 is displayed will be referred to as a second captured image 70B. The second control unit 250 performs the same processing as when the first image 10 described above is displayed on the screen 5, and receives the second captured image 70B from the information processing device 100. The second control unit 250 performs image analysis on the received second captured image 70B, and detects the second image 20 and the projection area 51.

[0059] The second control unit 250 detects four contact points, i.e., a first contact point T1, a second contact point T2, a third contact point T3, and a fourth contact point T4 shown in FIG. 9, from the second captured image 70B as the second image 20. Specifically, the second control unit 250 acquires the coordinates of the four contact points, i.e., the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4, in an imaging coordinate system set on the second captured image 70B. The second control unit 250 also detects the coordinates of four vertices M5, M6, M7, and M8 of the projection area 51 based on changes in the luminance value of the second captured image 70B. The four contact points, i.e., the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4, in the imaging coordinate system set on the second captured image 70B correspond to four points included in the second image in the second captured image.

[0060] Next, the second control unit 250 calculates the determinant of the projection transformation based on the four coordinates in the panel coordinate system of the first contact T1, the second contact T2, the third contact T3, and the fourth contact T4 in the second image 20 and the four coordinates in the imaging coordinate system of the first contact T1, the second contact T2, the third contact T3, and the fourth contact T4 in the second captured image 70B. The four contact points, i.e., the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4, of the second image 20 in the panel coordinate system correspond to four points included in the second image on the drawing panel. Also, each of the four contact points, i.e., the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4, of the second image 20 in the panel coordinate system corresponds to each of the four contact points, i.e., the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4, in the imaging coordinate system set on the second captured image 70B.

[0061] Fig. 10 is a diagram showing the positions of the four vertices of the projection area in the panel coordinate system, and Fig. 11 is a diagram showing an example of a projected image 30 displayed on the screen 5. The second control unit 250 uses the calculated determinant of the projective transformation to convert the coordinates of the four vertices M5, M6, M7, and M8 of the projection area 51 into coordinates in the panel coordinate system. The four vertices converted into the coordinates in the panel coordinate system are denoted as N5, N6, N7, and N8. The second control unit 250 generates the projection image 30 so that the four vertices are located at the positions of the vertices N5, N6, N7, and N8, respectively. After generating the projection image 30, the second control unit 250 controls the image processing unit 220 and the image projection unit 230 to draw the generated projection image 30 on the panel area 237 of the liquid crystal panel 235. Light emitted from the light source 231 passes through the panel area 237 and is modulated into image light corresponding to the projection image 30, and the modulated image light is displayed in the projection area 51. The area defined by the four vertices N1, N2, N3, and N4 in the panel coordinate system, transformed by the determinant of the projective transformation calculated based on the first captured image 70B and the first image 20, corresponds to the second area.

[0062] FIG. 12 is a diagram showing an example of an operation screen 131 displayed on the touch panel 130 of the information processing device 100. As shown in FIG. After the second control unit 250 displays the projection image 30 on the screen 5, it transmits an instruction to the information processing device 100 to start quick corner correction. When the first control unit 150 receives an instruction to start quick corner correction from the projector 200, it causes the touch panel 130 to display an operation screen 131 shown in FIG. On the operation screen 131, a selection screen 133 for selecting a vertex to be corrected, a cross key 135, and a confirmation key 137 are displayed.

[0063] The user operates the cross key 135 to select a vertex of the projected image 30 that is to be the target of quick corner correction. When a vertex of the projected image 30 is selected and the enter key 137 is pressed, the first control unit 150 determines that the vertex of the projected image 30 that is to be the target of quick corner correction has been selected. The first control unit 150 transmits information about the selected vertex of the projected image 30 to the projector 200.

[0064] Next, the user presses the cross key 135 in the direction in which the user wants to move the vertex of the selected projected image 30. While the cross key 135 is being pressed, the first control unit 150 transmits to the projector 200 an operation signal corresponding to the pressed cross key 135. While receiving the operation signal from the information processing device 100, the second control unit 250 moves the vertex of the selected projected image 30 in the direction corresponding to the pressed cross key 135. In other words, the second control unit 250 moves the position of the panel area 237 in which the vertex of the selected projected image 30 is drawn in the direction corresponding to the pressed cross key 135. When the operation of the cross key 135 is no longer detected, the first control unit 150 stops transmitting the operation signal to the projector 200. Furthermore, when the enter key 137 is pressed, the first control unit 150 transmits to the projector 200 an operation signal instructing the end of quick corner correction.

[0065] 4. Projector operation 13 is a flowchart showing the operation of the projector 200. The operation of the projector 200 will be described with reference to the flowchart shown in FIG.

[0066] The second control unit 250 determines whether or not an instruction signal instructing the start of image adjustment has been received from the information processing device 100 (step S1). If the instruction signal has not been received (step S1 / NO), the second control unit 250 waits to start processing until the instruction signal is received.

[0067] When the second control unit 250 receives an instruction signal from the information processing device 100 (step S1 / YES), it causes the first image 10 to be displayed on the screen 5 (step S2).

[0068] Next, the second control unit 250 transmits an instruction to capture the first captured image 70A and a request to acquire the first captured image 70A to the information processing device 100 (step S3). After that, the second control unit 250 determines whether or not the first captured image 70A has been received from the information processing device 100 (step S4), and if the first captured image 70A has not been received (step S4 / NO), waits to start the next process until the first captured image 70A is received.

[0069] When the second control unit 250 receives the first captured image 70A from the information processing device 100 (step S4 / YES), the second control unit 250 performs image analysis on the received first captured image 70A. The second control unit 250 detects four contact points of the first image 10, namely, the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4, from the first captured image 70A (step S5).

[0070] Next, the second control unit 250 analyzes the first captured image 70A and detects the four vertices M1, M2, M3, and M4 of the projection area 51 (step S6).

[0071] Next, the second control unit 250 calculates the determinant of the projective transformation (step S7). The second control unit 250 calculates the determinant of the projection transformation based on the four coordinates of the first contact T1, the second contact T2, the third contact T3, and the fourth contact T4 in the panel coordinate system and the four coordinates of the first contact T1, the second contact T2, the third contact T3, and the fourth contact T4 in the imaging coordinate system.

[0072] Next, the second control unit 250 uses the calculated determinant of the projective transformation to transform the coordinates of the four vertices M1, M2, M3, and M4 of the projection area 51 into the coordinates of the four vertices N1, N2, N3, and N4 of the panel coordinate system (step S8).

[0073] Next, the second control unit 250 generates the second image 20 based on the calculated coordinates in the panel coordinate system of the four vertices N1, N2, N3, and N4 of the projection area 51 (step S9). The second control unit 250 generates the second image 20 so that the four vertices are located at the positions of the four vertices N1, N2, N3, and N4.

[0074] Next, the second control unit 250 displays the generated second image 20 on the screen 5 (step S10). Next, the second control unit 250 transmits an instruction to capture the second captured image 70B and a request to acquire the second captured image 70B to the information processing device 100 (step S11). The second control unit 250 determines whether or not the second captured image 70B has been received from the information processing device 100 (step S12 / NO). If the second control unit 250 has not received the second captured image 70B from the information processing device 100 (step S12 / NO), the second control unit 250 waits until the second captured image 70B is received.

[0075] When the second control unit 250 receives the second captured image 70B from the information processing device 100 (step S12 / YES), it detects four contact points, namely, the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4, of the second image 20 (step S13), similar to step S5. Furthermore, the second control unit 250 detects four vertices of the projection area 51 (step S14), similar to steps S6 and S7, and calculates the determinant of the projective transformation (step S15).

[0076] Furthermore, similar to steps S8 and S9, the second control unit 250 uses the determinant of the projective transformation to transform the four vertices M1, M2, M3, and M4 of the projection area 51 into four vertices N1, N2, N3, and N4 of the panel coordinate system, and calculates the coordinates of the four vertices in the panel coordinate system (step S16).The second control unit 250 also generates the projection image 30 based on the four vertices N1, N2, N3, and N4 of the transformed panel coordinate system (step S17).

[0077] The second control unit 250 displays the generated projection image 30 on the screen 5 (step S18), and transmits an instruction to start quick corner correction to the information processing device 100 (step S19). After that, the second control unit 250 determines whether or not an operation signal indicating the vertex to be corrected has been received from the information processing device 100 (step S20).

[0078] If the second control unit 250 has not received an operation signal indicating the vertex to be corrected (step S20 / NO), it determines whether or not it has received an operation signal instructing the end of quick corner correction from the information processing device 100 (step S21). If the second control unit 250 has not received an operation signal instructing the end of quick corner correction (step S21 / NO), it returns to the determination of step S20. On the other hand, if the second control unit 250 has received an operation signal instructing the end of quick corner correction (step S21 / YES), it ends this processing flow.

[0079] Furthermore, when the second control unit 250 receives an operation signal indicating a vertex to be corrected (step S20 / YES), it determines whether or not it has received an operation signal corresponding to the operated cross key (step S22). When the second control unit 250 has not received an operation signal corresponding to the operation of the cross key (step S22 / NO), it proceeds to the determination of step S21.

[0080] Furthermore, when the second control unit 250 receives an operation signal corresponding to the operated cross key (step S22 / YES), it moves the position of the panel area 237 depicting the vertices of the selected projection image 30 in the direction indicated by the operation signal (step S23). After that, the second control unit 250 determines whether or not the reception of the operation signal corresponding to the cross key 135 continues (step S22).

[0081] 5. Effects of adjusting the projected image As described above, the second control unit 250 of the projector 200 of this embodiment executes the following processing. The second control unit 250 draws the first image 10 of the first size on the liquid crystal panel 235 and projects image light corresponding to the first image 10 drawn on the liquid crystal panel 235 onto the screen 5. Furthermore, the second control unit 250 detects the projection area 51 and the first image 10 from a first captured image 70A obtained by capturing an image of the screen 5 onto which image light corresponding to the first image 10 is projected. Then, the second control unit 250 identifies a first area on the liquid crystal panel 235 corresponding to the projection area 51 based on the projection area 51 in the first captured image 70A, the first image 10 in the first captured image 70A, and the first image 10 on the liquid crystal panel 235.

[0082] In addition, the second control unit 250 draws a second image 20 of a second size larger than the first size within the first area, and projects image light corresponding to the second image 20 drawn on the liquid crystal panel 235 onto the screen 5. The second control unit 250 detects the projection area 51 and the second image 20 from a second captured image 70B obtained by capturing an image of the screen 5 onto which the image light corresponding to the second image 20 is projected. Then, the second control unit 250 identifies a second area on the liquid crystal panel 235 corresponding to the projection area 51 based on the projection area 51 in the second captured image 70B, the second image 20 in the second captured image 70B, and the second image 20 on the liquid crystal panel 235. Furthermore, the second control unit 250 draws a projection image within the second region, and projects image light corresponding to the projection image onto the screen 5.

[0083] According to this configuration, the first size of the first image 10 is smaller than the second size of the second image 20. By first displaying the first image 10, which is smaller than the panel area 237 of the liquid crystal panel 235, on the screen 5, the first image 10 can be displayed on the screen 5 so as not to extend beyond the projection area 51. This reduces the probability of failing to detect the first image 10 when detecting it from the first captured image 70A.

[0084] Furthermore, the projection area 51 and the first image 10 are detected from the first captured image 70A, the area of ​​the liquid crystal panel 235 corresponding to the projection area 51 is identified, and the second image 20 is drawn in the area of ​​the liquid crystal panel 235 corresponding to the identified projection area 51. Then, the second image 20 drawn on the liquid crystal panel 235 is displayed on the screen 5. This reduces the influence of lens distortion of the camera 120 that captures the first captured image 70A and the second captured image 70B, and makes it possible to accurately identify the area of ​​the liquid crystal panel 235 that corresponds to the projection area 51. Therefore, the projection image projected by the projector 200 can be accurately matched to the shape of the projection area 51.

[0085] Moreover, the first image 10 and the second image 20 are rectangular images, and the first image 10 and the second image 20 include images of at least four preset shapes.

[0086] According to this configuration, by detecting an image of a predetermined shape from the first captured image and the second captured image, it is possible to detect the area of ​​the first image 10 and the second image 20 with high accuracy.

[0087] Furthermore, detecting the projection area 51 and the first image 10 from the first captured image 70A means detecting the vertices of the projection area 51 and the points of the first image 10.

[0088] According to this configuration, the vertices of the projection area 51 and the points of the first image 10 can be detected from the first captured image 70A, and the projection area 51 and the first image 10 can be detected.

[0089] The first image 10 is a rectangular image. The second control unit 250 calculates a determinant of the projective transformation based on four points included in the first image 10 in the first captured image 70A and four points included in the first image 10 on the liquid crystal panel 235 that correspond to the four points in the first captured image. The second control unit 250 detects the positions on the liquid crystal panel 235 of the four vertices of the projection area 51 detected from the first captured image 70A using the calculated determinant of the projective transformation.

[0090] According to this configuration, the area of ​​the liquid crystal panel 235 that corresponds to the projection area 51 can be identified with high accuracy.

[0091] The second image 20 is a rectangular image. The second control unit 250 calculates a determinant of the projective transformation based on four points included in the second image 20 in the second captured image 70B and four points included in the second image 20 on the liquid crystal panel 235 that correspond to the four points in the second captured image. The second control unit 250 detects the positions on the liquid crystal panel 235 of the four vertices of the projection area 51 detected from the second captured image 70B based on the calculated determinant of the projective transformation.

[0092] According to this configuration, the area of ​​the liquid crystal panel 235 that corresponds to the projection area 51 can be identified with high accuracy.

[0093] 6. Variations FIG. 14 is a diagram showing the configuration of a projection system 1B, which is another embodiment of the projection system 1A shown in FIG. 14, the information processing device 100 does not include the camera 120, but includes a wired I / F 140 and is connected to an external camera 300 by wire. The connection between the information processing device 100 and the camera 300 may be wireless, using short-range wireless communication such as Wi-Fi or Bluetooth. The camera 300 corresponds to an imaging device, and the information processing device 100 corresponds to a control device.

[0094] In this modification, the processing performed by the second control unit 250 of the projector 200 described above is executed by the first control unit 150 of the information processing device 100. The first control unit 150 receives from the camera 300 a first captured image 70A obtained by capturing an image of the screen 5 on which the first image 10 is displayed. Furthermore, the first control unit 150 receives, from the projector 200, information about the first image 10 drawn on the liquid crystal panel 235 of the projector 200. The information about the first image 10 includes coordinates of a first contact point T1, a second contact point T2, a third contact point T3, and a fourth contact point T4 included in the first image 10. These coordinates are coordinates in the panel coordinate system.

[0095] The first control unit 150 performs image analysis on the received first captured image 70A to detect the first image 10 and the projection area 51. More specifically, the first control unit 150 detects the coordinates of the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4 of the first image 10 in the imaging coordinate system. The first control unit 150 also detects the coordinates of the four vertices M1, M2, M3, and M4 of the projection area 51 in the imaging coordinate system.

[0096] The first control unit 150 calculates the determinant of the projective transformation based on the coordinates of the first contact T1, the second contact T2, the third contact T3, and the fourth contact T4 in the imaging coordinate system and the coordinates of the first contact T1, the second contact T2, the third contact T3, and the fourth contact T4 in the panel coordinate system received from the projector 200.

[0097] Next, the first control unit 150 converts the coordinates of the four vertices M1, M2, M3, and M4 of the projection area 51 into the coordinates of the four vertices N1, N2, N3, and N4 of the panel coordinate system using the calculated determinant of projective transformation. The first control unit 150 transmits information on the converted coordinates of the four vertices N1, N2, N3, and N4 of the panel coordinate system to the projector 200.

[0098] The projector 200 generates the second image 20 based on the coordinates of the four vertices N1, N2, N3, and N4 received from the information processing device 100, and displays the generated second image 20 on the screen 5. The information processing device 100 receives from the camera 300 a second captured image 70B obtained by capturing an image of the screen 5 on which the second image 20 is displayed.

[0099] The first control unit 150 performs image analysis on the second captured image 70B in the same manner as in the case of image analysis on the first captured image 70A, and calculates the coordinates in the panel coordinate system of the four vertices N5, N6, N7, and N8 of the projection area 51. The first control unit 150 transmits the calculated coordinates in the panel coordinate system of the four vertices N5, N6, N7, and N8 of the projection area 51 to the projector 200. The projector 200 generates a projection image 30 based on the coordinates of the four vertices N5, N6, N7, and N8 of the projection area 51 in the panel coordinate system received from the information processing device 100, and displays the generated projection image 30 on the screen 5.

[0100] 7. Projection System Effects As described above, the projection system 1A of this embodiment includes the projector 200, and the information processing device 100 including the camera 120 and the first control unit 150.

[0101] The projector 200 includes a liquid crystal panel 235, and projects image light corresponding to a first image 10 of a first size drawn on the liquid crystal panel 235 onto the screen 5. The camera 120 captures an image of the screen 5 onto which image light corresponding to the first image 10 is projected, and generates a first captured image 70A. The first control unit 150 detects the projection area 51 and the first image 10 from the first captured image 70A. The first control unit 150 identifies a first area on the liquid crystal panel 235 corresponding to the projection area 51 based on the projection area 51 in the first captured image 70A, the first image 10 in the first captured image 70A, and the first image on the drawing panel 235. The first control unit 150 transmits to the projector 200 first information indicating the first area.

[0102] Furthermore, the projector 200 draws a second image 20 of a second size larger than the first size within the first area indicated by the first information, and projects image light corresponding to the second image 20 onto the screen 5. The camera 120 captures an image of the screen 5 onto which the image light corresponding to the second image 20 is projected, and generates a second captured image 70B. The first control unit 150 detects the projection area 51 and the second image 20 from the second captured image 70B. The first control unit 150 identifies the second area of ​​the liquid crystal panel 235 corresponding to the projection area 51 based on the projection area 51 in the second captured image 70B, the second image 20 in the second captured image 70B, and the second image 20 on the liquid crystal panel 235. The first control unit 150 transmits to the projector 200 second information indicating the second area.

[0103] According to this configuration, the first size of the first image 10 is smaller than the second size of the second image 20. By first displaying the first image 10, which is smaller than the panel area 237 of the liquid crystal panel 235, on the screen 5, the first image 10 can be displayed on the screen 5 so as not to extend beyond the projection area 51. This reduces the probability of failing to detect the first image 10 when detecting it from the first captured image 70A.

[0104] Furthermore, the projection area 51 and the first image 10 are detected from the first captured image 70A, the area of ​​the liquid crystal panel 235 corresponding to the projection area 51 is identified, and the second image 20 is drawn in the area of ​​the liquid crystal panel 235 corresponding to the identified projection area 51. Then, the second image 20 drawn on the liquid crystal panel 235 is displayed on the screen 5. This reduces the influence of lens distortion of the camera 120 that captures the first captured image 70A and the second captured image 70B, and makes it possible to accurately identify the area of ​​the liquid crystal panel 235 that corresponds to the projection area 51. Therefore, the projection image projected by the projector 200 can be accurately matched to the shape of the projection area 51.

[0105] 8.Effects of control devices As described above, the first control unit 150 of this embodiment receives the first captured image 70A, which is an image of the screen 5 onto which image light corresponding to the first image 10 of a first size is projected, from the camera 300. The first control unit 150 also receives information about the first image 10 drawn on the liquid crystal panel 235 of the projector 200 from the projector 200 that projects the image light corresponding to the first image 10. The first control unit 150 detects the projection area 51 and the first image from the first captured image 70A. The first control unit 150 identifies the first area on the drawing panel 235 that corresponds to the projection area 51 based on the projection area 51 in the first captured image 70A, the first image in the first captured image 70A, and information about the first image. The first control unit 150 transmits to the projector 200 first information indicating the first area.

[0106] The first control unit 150 also receives from the camera 300 a second captured image 70B obtained by capturing an image of the screen 5 onto which image light corresponding to a second image 20 of a second size larger than the first size is projected. The first control unit 150 detects the projection area 51 and the second image 20 from the second captured image 70B. The first control unit 150 identifies a second area on the drawing panel 235 corresponding to the projection area 51 based on the projection area 51 in the second captured image 70B, the second image 20 in the second captured image 70B, and the first information. The first control unit 150 transmits to the projector 200 second information indicating the second area.

[0107] According to this configuration, the first size of the first image 10 is smaller than the second size of the second image 20. By first displaying the first image 10, which is smaller than the panel area 237 of the liquid crystal panel 235, on the screen 5, the first image 10 can be displayed on the screen 5 so as not to extend beyond the projection area 51. This reduces the probability of failing to detect the first image 10 when detecting it from the first captured image 70A.

[0108] Furthermore, the projection area 51 and the first image 10 are detected from the first captured image 70A, the area of ​​the liquid crystal panel 235 corresponding to the projection area 51 is identified, and the second image 20 is drawn in the area of ​​the liquid crystal panel 235 corresponding to the identified projection area 51. Then, the second image 20 drawn on the liquid crystal panel 235 is displayed on the screen 5. This reduces the influence of lens distortion of the camera 120 that captures the first captured image 70A and the second captured image 70B, and makes it possible to accurately identify the area of ​​the liquid crystal panel 235 that corresponds to the projection area 51. Therefore, the projection image projected by the projector 200 can be accurately matched to the shape of the projection area 51.

[0109] The above-described embodiment is a preferred embodiment of the present invention, but the present invention is not limited to the above-described embodiment and various modifications are possible within the scope of the gist of the present invention. For example, the first image 10 and the second image 20 may be images shown in FIG. 15. The first image 10 shown in FIG. 15 is an image in which dots 15A, 15B, 15C, and 15D, which are images of preset shapes, are arranged at positions corresponding to the four corners of the first image 10. Although FIG. 14 shows a circular image as the preset shape, the shape of the image may be triangular or rectangular. Furthermore, in the first image 10, the dots 15A, 15B, 15C, and 15D do not have to be arranged at positions corresponding to the four corners. It is sufficient to use an image that includes at least four images of preset shapes.

[0110] The image patterns formed in the first image 10 and the second image 20 may be different. For example, an image in which dots 15A, 15B, 15C, and 15D are arranged at positions corresponding to the four corners may be used as the first image 10, and the pattern image shown in FIG. 8 may be used as the second image 20. Furthermore, the first image 10 and the second image 20 may have a difference in size in any one of the diagonal size, vertical size, horizontal size, etc. of the first rectangular figure 11 and the second rectangular figure 13.

[0111] The light modulation element included in the light modulation device 233 may be a transmissive liquid crystal panel or a reflective liquid crystal panel. The light modulation element may be configured using a DMD or a combination of a DMD and a color wheel. The light modulation device 233 may also employ a configuration capable of modulating light emitted from a light source, in addition to a liquid crystal panel and a DMD. In the above-described embodiment, three liquid crystal panels 235R, 235G, and 235B corresponding to the three colors of red, green, and blue are used, but this is not limited thereto. A configuration using one liquid crystal panel or a configuration using three DMDs corresponding to the three colors of red, green, and blue may also be used. The drawing panel may be any device that can adjust the projection light from the light source by reflection, transmission, or modulation to form a projected image displayed by the projector 200. Drawing by the drawing panel corresponds to adjusting the projection light from the light source by reflection, transmission, or modulation.

[0112] Furthermore, the functional units of the projector 200 and the information processing device 100 shown in FIG. 1 indicate functional configurations, and the specific implementation form is not particularly limited. In other words, it is not necessary to implement hardware corresponding to each functional unit individually, and it is of course possible to configure the system so that one or more processors execute a program to realize the functions of multiple functional units. Furthermore, some of the functions realized by software in the above embodiments may be realized by hardware, and some of the functions realized by hardware may be realized by software. In addition, the specific detailed configurations of the other units of the projector may also be changed as desired without departing from the spirit of the present invention.

[0113] Furthermore, when the projection image adjustment method is implemented by a computer included in the projector 200 or a computer included in the information processing device 100, the program executed by the computer can be configured in the form of a recording medium or a transmission medium for transmitting the program. The recording medium can be a magnetic or optical recording medium or a semiconductor memory device. Specific examples include portable or fixed recording media such as flexible disks, HDDs (Hard Disk Drives), CD-ROMs, DVDs, Blu-ray Discs, magneto-optical disks, flash memories, and card-type recording media. The recording medium can also be a non-volatile storage device such as RAM, ROM, or HDD, which is an internal storage device included in a server device. Blu-ray is a registered trademark. [Explanation of symbols]

[0114] 1A...Projection system, 1B...Projection system, 5...Screen, 10...First image, 11, 11A, 11D, 11F, 11G, 11J, 11K, 11M, 11P...First rectangular figure, 13, 13B, 13C, 13E, 13H, 13I, 13L, 13N, 13O, Second rectangular figure, 15A, 15B, 15C...Dots, 20...Second image, 30...Projected image, 51 ...projection area, 53...background area, 70...captured image, 70A...captured image, 70A...first captured image, 70B...second captured image, 100...information processing device, 110...first wireless I / F, 120...camera, 130...touch panel, 131...operation screen, 133...selection screen, 135...cross key, 137...confirmation key, 140...wired I / F, 150...first control unit, 160 ...First memory unit, 165...Control program, 170...First processor, 200...Projector, 210...Second wireless I / F, 220...Image processing unit, 225...Frame memory, 230...Image projection unit, 231...Light source, 233...Light modulation device, 235, 235R, 235G, 235B LCD panels, 237...237R, 237G, 237B...Panel area, 239...Drive circuit, 241...Optical unit, 250...Second control unit, 260...Second memory unit, 265...Control program, 270...Second processor, 300...Camera, M1, M2, M3, M4, M5, M6, M7...Vertices, N1, N2, N3, N4, N5, N6, N7...Vertices, T1...First contact, T2...Second contact, T3...Third contact, T4...Fourth contact.

Claims

1. drawing a first image of a first size on a drawing panel; projecting the first image onto a projection surface; Detecting an area of ​​the projection surface and the first image from a first captured image obtained by capturing an image of the projection surface onto which the first image is projected; Identifying a first area on the drawing panel corresponding to the area of ​​the projection surface based on an area of ​​the projection surface in the first captured image, the first image in the first captured image, and the first image on the drawing panel; Drawing a second image having a second size larger than the first size within the first area; projecting the second image onto the projection surface; detecting an area of ​​the projection surface and the second image from a second captured image obtained by capturing an image of the projection surface onto which the second image is projected; Identifying a second area on the drawing panel corresponding to the area of ​​the projection surface based on an area of ​​the projection surface in the second captured image, the second image in the second captured image, and the second image on the drawing panel; rendering a projected image within the second area; projecting the projection image onto the projection surface; A method for adjusting a projected image comprising:

2. the first image and the second image are rectangular images, The method for adjusting a projected image according to claim 1 , wherein the first image and the second image include images of at least four preset shapes.

3. The projection image adjusting method according to claim 2 , wherein detecting the area of ​​the projection surface and the first image from the first captured image comprises detecting vertices of the projection surface and points of the first image.

4. the first image is a rectangular image, calculating a determinant of a projective transformation based on four points included in the first image in the first captured image and four points included in the first image on the drawing panel that correspond to each of the four points in the first captured image; detecting positions on the drawing panel of the four vertices of the projection surface detected from the first captured image using a determinant of the projective transformation; 2. The method for adjusting a projected image according to claim 1.

5. the second image is a rectangular image, calculating a determinant of a projective transformation based on four points included in the second image in the second captured image and four points included in the second image on the drawing panel that correspond to each of the four points in the second captured image; 2. The method for adjusting a projection image according to claim 1, further comprising detecting positions on the drawing panel of four vertices of the projection surface detected from the second captured image based on a determinant of the projective transformation.

6. a projector including a drawing panel that projects a first image of a first size drawn on the drawing panel onto a projection surface; an imaging device that captures an image of the projection surface onto which the first image is projected and generates a first captured image; Detecting an area of ​​the projection surface and the first image from the first captured image; Identifying a first area on the drawing panel corresponding to the area of ​​the projection surface based on an area of ​​the projection surface in the first captured image, the first image in the first captured image, and the first image on the drawing panel; and transmitting first information indicating the first area to the projector; The projector includes: drawing a second image having a second size larger than the first size within the first area indicated by the first information; projecting the second image onto the projection surface; The imaging device is capturing an image of the projection surface onto which the second image is projected to generate a second captured image; The control device Detecting an area of ​​the projection surface and the second image from the second captured image; Identifying a second area of ​​the drawing panel corresponding to the area of ​​the projection surface based on an area of ​​the projection surface in the second captured image, the second image in the second captured image, and the second image on the drawing panel; and transmitting second information indicating the second area to the projector.

7. receiving a first captured image obtained by capturing an image of a projection surface onto which a first image of a first size is projected from an imaging device; receiving, from a projector that projects the first image onto the projection surface, information about the first image drawn on a drawing panel of the projector; Detecting an area of ​​the projection surface and the first image from the first captured image; Identifying a first area on the drawing panel corresponding to the area of ​​the projection surface based on an area of ​​the projection surface in the first captured image, the first image in the first captured image, and information about the first image; transmitting first information indicating the first area to the projector; receiving, from the imaging device, a second captured image obtained by capturing an image of the projection surface onto which a second image having a second size larger than the first size is projected; Detecting an area of ​​the projection surface and the second image from the second captured image; Identifying a second area on the drawing panel corresponding to the area of ​​the projection surface based on an area of ​​the projection surface in the second captured image, the second image in the second captured image, and the first information; transmitting second information indicating the second area to the projector; A control device that executes the above.

Images