Methods for adjusting projected images, projection system, and control device
Patent Information
- Application Number
- CN202310207551.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2022-02-28
- Filing Date
- 2023-02-27
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-02-27
AI Technical Summary
In existing technologies, the methods for adjusting projected images suffer from low image adjustment accuracy, especially when the image exceeds the projection surface, making it difficult to achieve precise matching.
By drawing images of different sizes on the drawing panel and using the imaging and control devices to detect and adjust the image area, the projection surface area is determined, and the projection position and shape of the image are then adjusted.
It improves the adjustment accuracy of the projected image, ensuring that the image can be accurately projected onto the projection surface and avoiding image deviation.
Smart Images

Figure CN116668649B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for adjusting projected images, a projection system, and a control device. Background Technology
[0002] Previously, there were known adjustment methods to match the shape of the image projected by the projector with the shape of the projection surface.
[0003] For example, Patent Document 1 discloses a projector device that projects a test pattern onto a screen, calculates the distance between multiple points on the displayed screen and the projector device based on images captured by an image sensor, and detects the positional relationship between the projector device and the screen.
[0004] Patent Document 1: Japanese Patent Application Publication No. 2006-246502
[0005] However, when the projection surface on which the image is projected is photographed and the display position and shape of the projected image are corrected based on the photographed image, if the image projected onto the projection surface exceeds the projection surface, the adjustment accuracy of the image may be reduced. Summary of the Invention
[0006] This disclosure discloses a method for adjusting a projected image, comprising: drawing a first image of a first size on a drawing panel; projecting the first image onto a projection surface; detecting a region of the projection surface and the first image from a first captured image obtained by taking a picture of the projection surface on which the first image is projected; determining a first region in the drawing panel corresponding to the region of the projection surface based on the region of the projection surface in the first captured image, the first image in the first captured image, and the first image in the drawing panel; drawing a second image of a second size larger than the first size within the first region; projecting the second image onto the projection surface; detecting a region of the projection surface and the second image from a second captured image obtained by taking a picture of the projection surface on which the second image is projected; determining a second region in the drawing panel corresponding to the region of the projection surface based on the region of the projection surface in the second captured image, the second image in the second captured image, and the second image in the drawing panel; drawing a projected image within the second region; and projecting the projected image onto the projection surface.
[0007] This disclosure discloses a projection system comprising: a projector having a drawing panel for projecting a first image of a first size drawn on the drawing panel onto a projection surface; an imaging device for capturing an image of the projection surface on which the first image is projected to generate a first captured image; and a control device that performs: detecting a region of the projection surface and the first image from the first captured image; determining a first region in the drawing panel corresponding to the region of the projection surface based on the region of the projection surface in the first captured image, the first image in the first captured image, and the first image in the drawing panel; and sending first information representing the first region to the projector. The projector performs the following actions: within the first region represented by the first information, depicting a second image of a second size larger than the first size; and projecting the second image onto the projection surface. The imaging device captures an image of the projection surface onto which the second image is projected to generate a second captured image. The control device further performs the following actions: detecting the region of the projection surface and the second image from the second captured image; determining a second region of the depicting panel corresponding to the region of the projection surface based on the region of the projection surface in the second captured image, the second image in the second captured image, and the second image in the depicting panel; and sending second information representing the second region to the projector.
[0008] This disclosure discloses a control device that performs the following actions: receiving a first captured image from an imaging device obtained by capturing an image on a projection surface onto which a first image of a first size is projected; receiving information about the first image depicted on a drawing panel of a projector from a projector onto the projection surface; detecting a region of the projection surface and the first image from the first captured image; and determining a first image in the drawing panel corresponding to the region of the projection surface based on the region of the projection surface in the first captured image, the first image in the first captured image, and the information of the first image. The process involves: 1) sending first information representing the first region to the projector; 2) receiving a second captured image from the imaging device, obtained by capturing an image of a projection surface projecting a second image of a second size larger than the first size; 3) detecting the region of the projection surface and the second image from the second captured image; 4) determining a second region in the drawing panel corresponding to the region of the projection surface based on the region of the projection surface in the second captured image, the second image in the second captured image, and the first information; and 5) sending second information representing the second region to the projector. Attached Figure Description
[0009] Figure 1 It is a diagram showing the system structure of the projection system.
[0010] Figure 2 It is a block diagram representing the structure of the image projection section.
[0011] Figure 3 This is a diagram representing an example of a screen.
[0012] Figure 4 This is a diagram showing an example of the first image depicted on a liquid crystal panel.
[0013] Figure 5 This is an example of the first captured image.
[0014] Figure 6 It is a diagram showing the positions of the four vertices of the projection area in the panel coordinate system.
[0015] Figure 7 This is a diagram showing an example of a second image depicted on a liquid crystal panel.
[0016] Figure 8 This is an example of a second image projected onto a screen.
[0017] Figure 9 This is an example of a second captured image.
[0018] Figure 10 It is a diagram showing the positions of the four vertices of the projection area in the panel coordinate system.
[0019] Figure 11 This is a diagram showing an example of a projected image onto a screen.
[0020] Figure 12 This is an example of an operation screen displayed on a touch panel.
[0021] Figure 13 This is a flowchart illustrating the actions of the projector.
[0022] Figure 14 This is a diagram representing another system structure of the projection system.
[0023] Figure 15 This is a diagram representing another example of the first and second images.
[0024] Label Explanation
[0025] 1A: Projection system; 1B: Projection system; 5: Screen; 10: First image; 11, 11A, 11D, 11F, 11G, 11J, 11K, 11M, 11P: First rectangular graphic; 13, 13B, 13C, 13E, 13H, 13I, 13L, 13N, 13O: Second rectangular graphic; 15A, 15B, 15C, 15D: Dots; 20: Second image; 30: Projected image Image; 51: Projection area; 53: Background area; 70: Captured image; 70A: First captured image; 70B: Second captured image; 100: Information processing device; 110: First wireless I / F; 120: Camera; 130: Touch panel; 131: Operation screen; 133: Selection screen; 135: Cross key; 137: OK key; 140: Wired I / F; 150: First control unit; 160 165: First storage unit; 170: Control program; 200: First processor; 210: Projector; 220: Second wireless I / F; 225: Image processing unit; 226: Frame memory; 230: Image projection unit; 231: Light source; 233: Light modulation device; 235, 235R, 235G, 235B: Liquid crystal panel; 237, 237R, 237G, 237B: Panel area; 239: Driver 241: Optical unit; 250: Second control unit; 260: Second storage unit; 265: Control program; 270: Second processor; 300: Camera; M1, M2, M3, M4, M5, M6, M7, M8: Vertices; N1, N2, N3, N4, N5, N6, N7, N8: Vertices; T1: First contact; T2: Second contact; T3: Third contact; T4: Fourth contact. Detailed Implementation
[0026] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.
[0027] 1. Description of the system structure of the projection system
[0028] Figure 1 This is a diagram showing the system structure of projection system 1A.
[0029] The projection system 1A includes an information processing device 100 and a projector 200.
[0030] The information processing device 100 is used as a means to provide image data, which will become the source of the image projected by the projector 200, to the projector 200, or to handle operations that change the shape or display position of the image projected by the projector 200. The information processing device 100 may be, for example, a smartphone, a tablet computer, or a laptop computer.
[0031] The projector 200 generates image light based on image data provided from the information processing device 100 and projects the generated image light onto the screen 5.
[0032] 2. Structure of Information Processing Device
[0033] The structure of the information processing device 100 will be described.
[0034] The information processing device 100 includes a first wireless interface 110, a camera 120, a touch panel 130, and a first control unit 150. Hereinafter, the interface will be abbreviated as I / F.
[0035] The first wireless I / F 110 is, for example, a wireless communication device having an interface circuit corresponding to standards such as Bluetooth and Wi-Fi. Bluetooth and Wi-Fi are registered trademarks. The first wireless I / F 110 is wirelessly connected to the projector 200 and performs mutual data communication with the projector 200. Alternatively, the information processing device 100 in this embodiment may have the wired I / F 140 present in the information processing device 100 constituting the projection system 1B shown in the modified example described later, and may also perform data communication with the projector 200 via wired communication based on the wired I / F 140. The wired I / F 140 may also have an interface circuit.
[0036] The camera 120 includes a shooting lens, a shooting element such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide Semiconductor), and data processing circuitry. Illustrations of the shooting lens, shooting element, and data processing circuitry are omitted. The camera 120 performs shooting according to the control of the first control unit 150, generating a captured image 70. The camera 120 outputs the generated captured image 70 to the first control unit 150.
[0037] The touch panel 130 includes a display panel and a touch sensor, and accepts user touch operations. Illustrations of the display panel and touch sensor are omitted. The display panel may be, for example, a liquid crystal panel or an organic EL (Electro-luminescent) panel. The touch sensor detects user touch operations on the display panel. A coordinate system is pre-set on the display panel, and the touch sensor outputs an operation signal containing coordinate information to the first control unit 150, whereby the coordinate information indicates the position of the user's touch within the coordinate system.
[0038] The first control unit 150 is a computer device having a first storage unit 160 and a first processor 170.
[0039] The first storage unit 160 includes non-volatile memory such as ROM (Read Only Memory) and volatile memory such as RAM (Random Access Memory). ROM stores control programs 165 such as OS (Operating System) and application programs, as well as setting files that define the operation of the information processing device 100. RAM is used as the arithmetic area of the first processor 170.
[0040] The first processor 170 is an arithmetic processing device that includes one or more CPUs (Central Processing Units) or MPUs (Microprocessors). The first processor 170 executes the control program 165 to control the operation of various parts of the information processing device 100.
[0041] 3. Structure of the projector
[0042] Next, the structure of the projector 200 will be explained.
[0043] The projector 200 includes a second wireless I / F 210, an image processing unit 220, a frame memory 225, an image projection unit 230, and a second control unit 250.
[0044] The second wireless I / F 210 is, for example, a wireless communication device with interface circuitry corresponding to standards such as Bluetooth and Wi-Fi. The second wireless I / F 210 is wirelessly connected to the information processing device 100 and performs data communication with it. Alternatively, the projector 200 can also perform data communication with the information processing device 100 via wired communication based on a wired I / F (not shown). The wired I / F may also have interface circuitry.
[0045] The image processing unit 220 is connected to a frame memory 225. The image processing unit 220 expands image data input from the second control unit 250 and the second wireless I / F 210 into the frame memory 225. The frame memory 225 is, for example, composed of SDRAM (Synchronous Dynamic Random Access Memory).
[0046] The image processing unit 220 performs image processing on the image data expanded in the frame memory 225, such as resolution transformation or size adjustment, distortion and aberration correction, shape correction, digital zoom, and adjustment of image tone or brightness. The image processing unit 220 executes the image processing specified by the second control unit 250, using parameters input from the second control unit 250 as needed. Furthermore, the image processing unit 220 can also perform multiple image processing operations in combination. The image processing unit 220 reads the processed image data from the frame memory 225 and outputs the read image data to the light modulation device 233 of the image projection unit 230.
[0047] The image processing unit 220 and the frame memory 225 are, for example, constructed from integrated circuits. Integrated circuits include LSI (Large Scale Integrated Circuit), ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), FPGA (Field-Programmable Gate Array), SoC (System-on-a-chip), etc. Furthermore, a portion of the integrated circuit structure may include analog circuitry, or it may be a structure combining the second control unit 250 and the integrated circuit.
[0048] Figure 2 This is a block diagram showing the structure of the image projection unit 230.
[0049] Reference Figure 2 Explain the structure of the image projection unit 230.
[0050] The image projection unit 230 includes a light source 231, a light modulation device 233, and an optical unit 241. In this embodiment, the light modulation device 233 includes three transmissive liquid crystal panels 235R, 235G, and 235B corresponding to the three colors red, green, and blue, and a driving circuit 239 for driving these liquid crystal panels 235R, 235G, and 235B as light modulation elements. R is an abbreviation for Red, G for Green, and B for Blue. Hereinafter, the liquid crystal panels 235R, 235G, and 235B included in the projector 200 will be referred to as liquid crystal panel 235. The light modulation element included in the light modulation device 233 is not limited to the transmissive liquid crystal panels 235R, 235G, and 235B; for example, it may be a reflective liquid crystal panel or a DMD (Digital Micromirror Device). LCD panels include 235R, 235G, 235B, reflective LCD panels, and DMD, which is equivalent to a drawing panel.
[0051] The light source 231 can be a discharge-type light source such as an ultra-high pressure mercury lamp or a metal halide lamp, or a solid-state light source such as a light-emitting diode or a semiconductor laser. Light emitted from the light source 231 is incident on the liquid crystal panel 235. Liquid crystal panels 235R, 235G, and 235B are each composed of a transmissive liquid crystal panel, etc., in which liquid crystal is sealed between a pair of transparent substrates. Liquid crystal panel 235R modulates red light, liquid crystal panel 235G modulates green light, and liquid crystal panel 235B modulates blue light. Each liquid crystal panel 235 has a panel region 237 composed of multiple pixels arranged in a matrix. The panel region 237 of liquid crystal panel 235R is referred to as panel region 237R, the panel region 237 of liquid crystal panel 235G is referred to as panel region 237G, and the panel region 237 of liquid crystal panel 235B is referred to as panel region 237B. Furthermore, when collectively referred to as panel regions 237R, 237G, and 237B, it is referred to as panel region 237.
[0052] Image data is input from the image processing unit 220 to the driving circuit 239. The driving circuit 239 applies a driving voltage corresponding to the input image data to each pixel of the panel area 237, changing the transmittance of each pixel to the transmittance corresponding to the image data. Light emitted from the light source 231 is modulated as it passes through the panel area 237, forming image light corresponding to the image data according to each color of light: red, green, and blue. The formed image lights of each color are combined by a color synthesis optical system (not shown) to become image light representing a color image. The optical unit 241 includes a projection lens, etc., which amplifies the image light modulated by the liquid crystal panel 235 and projects it onto the projection area 51. Thus, an image corresponding to the image light, i.e., a projected image, is displayed on the projection area 51.
[0053] Figure 3 This is a diagram representing an example of screen 5.
[0054] Here, refer to Figure 3 The screen 5 on which the projected image is projected by the projector 200 will be described.
[0055] Screen 5 has a projection area 51 and a background area 53. The projection area 51 is equivalent to the projection surface. The projection area 51 is the area where the projected image light is projected and the projected image corresponding to the projected image light is displayed.
[0056] Background area 53 is the area that serves as the background for projection area 51, and it is located outside and adjacent to projection area 51. Furthermore, background area 53 is the area that serves as the background for projection area 51, and it is the area where the image will not be visible even if image light is projected onto it. Projection area 51 and background area 53 have a brightness difference. For example, projection area 51 is a high-brightness area such as white, and background area 53 is a low-brightness area such as black. Background area 53 only needs to have a brightness difference of a certain value from projection area 51; for example, it can be replaced by an indoor wall, curtains, etc.
[0057] return Figure 1 Next, the structure of the projector 200 will be explained.
[0058] The second control unit 250 is a computer device having a second storage unit 260 and a second processor 270.
[0059] The second storage unit 260 has volatile memory such as RAM and non-volatile memory such as ROM. RAM is used for temporary storage of various data, etc., and ROM stores the control program 265 used in the motion control of the projector 200 and various setting information.
[0060] The second processor 270 is an arithmetic processing device that includes one or more CPUs or MPUs. The second processor 270 executes the control program 265 to control the various parts of the projector 200.
[0061] For example, when the information processing device 100 receives an instruction signal to start image adjustment, the second control unit 250 starts image adjustment. This image adjustment is a process that adjusts the distortion of the shape of the projected image, the display position of the projected image displayed on the screen 5, and the display size.
[0062] Figure 4 This is a diagram showing an example of the first image 10 depicted in panel area 237.
[0063] The second control unit 250 first displays the first image 10 on the screen 5.
[0064] like Figure 4 As shown, the first image 10 is a pattern image on panel area 237, in which a first rectangular graphic 11 and a second rectangular graphic 13 of the same shape and size are alternately arranged in the vertical and horizontal directions. The first rectangular graphic 11 is a black rectangle, and the second rectangular graphic 13 is a white rectangle. The color of the first rectangular graphic 11 is not limited to black, and the color of the second rectangular graphic 13 is not limited to white. It is acceptable as long as the color of the first rectangular graphic 11 is different from the color of the second rectangular graphic 13. For example, the first rectangular graphic 11 and the second rectangular graphic 13 can also use colors that are complementary colors on the color wheel.
[0065] The second control unit 250 generates the first image 10. Alternatively, the first image 10 can be pre-stored in the second storage unit 260. When generating the first image 10, the second control unit 250 controls the image processing unit 220 and the image projection unit 230 to depict the generated first image 10 in the panel area 237 of the liquid crystal panel 235. Light emitted from the light source 231 is modulated into image light corresponding to the first image 10 as it passes through the panel area 237, and the modulated image light is displayed in the projection area 51.
[0066] The first image 10 is formed at a first size. The size of the first image 10, for example, refers to the number of display pixels constituting the first image 10 when the first image 10 is depicted in the liquid crystal panel 235. In this case, the size of the first image 10 can be referred to as the display resolution of the first image 10 or the resolution of the first image 10. The size of the first image 10 can also be the total number of display pixels constituting the first image 10. Alternatively, the size of the first image 10 can also be the number of display pixels in the horizontal direction or the number of display pixels in the vertical direction constituting the first image 10.
[0067] The first size is represented by the resolution of the first image 10, or the number of display pixels on the liquid crystal panel 235 depicting the first image 10. The second size, described later, is the same.
[0068] The resolution of the first image 10 is, for example, a preset resolution that is sufficiently small compared to the panel resolution of the liquid crystal panel 235. Here, the reason for setting the resolution of the first image 10 to be sufficiently small compared to the panel resolution of the liquid crystal panel 235 will be explained.
[0069] When the screen 5 is captured by the camera 120 of the information processing device 100 and the boundary between the projected image, the projection area 51 and the background area 53 is detected by the generated captured image 70, it is preferable to display the projected image near the boundary line between the projection area 51 and the background area 53.
[0070] This is because the distortion parameter value of camera 120 is unknown. There is a tendency that the greater the distance between the projected image and the boundaries of the projection area 51 and the background area 53, the greater the error will be due to the influence of the distortion parameter.
[0071] However, if the size of the first image 10 is arbitrarily increased in order to display the first image 10 near the boundary between the projection area 51 and the background area 53, there is a possibility that a portion of the first image 10 may deviate from the projection area 51, resulting in detection failure of the first image 10. Therefore, in this embodiment, by setting the resolution of the first image 10 to a size that is sufficiently small compared to the panel resolution of the liquid crystal panel 235, deviation of a portion of the projected first image 10 from the projection area 51 is suppressed.
[0072] Camera 120 captures an image 70 on screen 5 displaying the first image 10. This captured image 70 is referred to as the first captured image 70A. The first image 10 and the projection area 51 are detected from the first captured image 70A, and based on the detection result, the second image 20 is projected back onto screen 5. Since the projection area 51 has already been detected from the first captured image 70A, by changing the shape and resolution of the second image 20 based on the detection result, the second image 20 can be displayed near the boundary between the projection area 51 and the background area 53. The second image 20 will be described later.
[0073] The first image 10 includes four contacts: a first contact T1, a second contact T2, a third contact T3, and a fourth contact T4. The four contacts T1, T2, T3, and T4 included in the first image 10 are an example of at least four images with predefined shapes.
[0074] When viewed in the attached diagram, the first connection point T1 is positioned corresponding to the upper left vertex of the first image 10. When viewed in the attached diagram, the first connection point T1 is the point where the lower right vertex of the first rectangular shape 11A, the lower left vertex of the second rectangular shape 13B, the upper right vertex of the second rectangular shape 13C, and the upper left vertex of the first rectangular shape 11D meet.
[0075] The first rectangular graphic 11A, the second rectangular graphic 13B, the second rectangular graphic 13C, and the first rectangular graphic 11D are images of a predefined shape contained in the first image 10.
[0076] The first rectangular figure 11A is the figure located in the upper left of the first image 10 when viewed in the attached figure.
[0077] The second rectangular figure 13B is the figure that connects to the right side of the first rectangular figure 11A when viewed in the attached figure.
[0078] The second rectangular shape 13C is the shape that connects to the first rectangular shape 11A on the lower side when viewed in the attached figure.
[0079] The first rectangular shape 11D is the shape that connects to the right of the second rectangular shape 13C when viewed in the attached figure.
[0080] Here, "observing the attached diagram" refers to viewing the attached diagram with the origin O1 of the panel coordinate system or the origin O2 of the shooting coordinate system (described later) as the upper left.
[0081] The second contact point T2, when viewed in the attached diagram, is positioned corresponding to the upper right vertex of the first image 10. When viewed in the attached diagram, the second contact point T2 is the point where the lower left vertex of the second rectangular shape 13E, the lower right vertex of the first rectangular shape 11F, the upper left vertex of the first rectangular shape 11G, and the upper right vertex of the second rectangular shape 13H meet. The second rectangular shape 13E, the first rectangular shape 11F, the first rectangular shape 11G, and the second rectangular shape 13H correspond to images of a pre-defined shape contained in the first image 10.
[0082] The second rectangular figure 13E is located in the upper right of the first image 10 when viewed in the attached figure.
[0083] The first rectangular figure 11F is the figure that connects to the second rectangular figure 13E on the left side when viewed in the attached figure.
[0084] The first rectangular shape 11G is the shape that connects to the second rectangular shape 13E on the lower side when viewed in the attached figure.
[0085] The second rectangular figure 13H is the figure that connects to the left of the first rectangular figure 11G when viewed in the attached figure.
[0086] The third contact point T3, when viewed in the attached diagram, is positioned corresponding to the lower left vertex of the first image 10. When viewed in the attached diagram, the third contact point T3 is the point where the upper right vertex of the second rectangular shape 13I, the upper left vertex of the first rectangular shape 11J, the lower right vertex of the first rectangular shape 11K, and the lower left vertex of the second rectangular shape 13L meet. The second rectangular shape 13I, the first rectangular shape 11J, the first rectangular shape 11K, and the second rectangular shape 13L correspond to images of a pre-defined shape contained in the first image 10.
[0087] The second rectangular figure 13I is located in the lower left of the first image 10 when viewed in the attached figure.
[0088] The first rectangular figure 11J is the figure that connects to the right of the second rectangular figure 13I when viewed in the attached figure.
[0089] The first rectangular shape 11K is the shape that connects to the second rectangular shape 13I on the upper side when viewed in the attached figure.
[0090] The second rectangular figure 13L is the figure that connects to the right of the first rectangular figure 11K when viewed in the attached figure.
[0091] Point T4 is the figure located in the lower right of the first image 10 when viewed in the attached diagram.
[0092] When viewed in the attached diagram, the fourth connection point T4 is the point where the upper left vertex of the first rectangular shape 11M, the upper right vertex of the second rectangular shape 13N, the lower left vertex of the second rectangular shape 13O, and the lower right vertex of the first rectangular shape 11P meet. The first rectangular shape 11M, the second rectangular shape 13N, the second rectangular shape 13O, and the first rectangular shape 11P correspond to images of a pre-defined shape contained in the first image 10.
[0093] The first rectangular figure 11M is located in the lower right of the first image 10 when viewed in the attached figure.
[0094] The second rectangular figure 13N is the figure that connects to the left of the first rectangular figure 11M when viewed in the attached figure.
[0095] The second rectangular figure 13O is the figure that connects to the first rectangular figure 11M on the upper side when viewed in the attached figure.
[0096] The first rectangular figure 11P is the figure that connects to the second rectangular figure 13O on the left side when viewed in the attached figure.
[0097] When the second control unit 250 displays the first image 10 on the screen 5, it sends a shooting instruction and a request to acquire the first captured image 70A to the information processing device 100. Upon receiving the shooting instruction and acquisition request from the projector 200, the first control unit 150 displays a shooting guide on the touch panel 130. The shooting guide instructs the user to operate the camera 120 to capture the display on the screen 5.
[0098] When the shooting guide is displayed on the touch panel 130, the user-held information processing device 100 moves to the shooting position. This shooting position is a position that includes the entire screen 5 within the field of view of the camera 120. More preferably, the shooting position is a position where the lateral center of the screen 5 is located at the lateral center of the field of view of the camera 120.
[0099] When the user moves to the shooting position, the camera 120 performs a shooting action. When the first captured image 70A is input from the camera 120, the first control unit 150 sends the input first captured image 70A to the projector 200.
[0100] Figure 5 This is a diagram showing an example of the first captured image 70A.
[0101] When the second control unit 250 receives the first captured image 70A from the information processing device 100, it performs image analysis on the received first captured image 70A and detects the first image 10 and the projection area 51.
[0102] The second control unit 250 sets a coordinate system on the input first captured image 70A. This coordinate system is called the capturing coordinate system. Figure 5 As shown, the shooting coordinate system is a coordinate system in which the upper left corner of the first captured image 70A is set as the origin O2, the horizontal axis of the first captured image 70A is set as the u-axis, and the vertical axis is set as the v-axis, when viewed in the attached diagram. As the first image 10, the second control unit 250 detects from the first captured image 70A... Figure 4 The four contacts shown are the first contact T1, the second contact T2, the third contact T3, and the fourth contact T4. The second control unit 250 obtains the coordinates of these four contacts in the shooting coordinate system. The four contacts T1, T2, T3, and T4 in the shooting coordinate system set on the first captured image 70A correspond to the four points contained in the first image.
[0103] Additionally, the second control unit 250 detects the projection area 51 from the first captured image 70A. The second control unit 250 detects the positions of the four vertices of the projection area 51. The second control unit 250 detects the projection area 51 based on changes in the brightness values of the first captured image 70A. Since the white projection area 51 is white and the background area 53 is black, there is a significant change in brightness value at the boundary between the projection area 51 and the background area 53. The second control unit 250 compares the changes in the brightness values of the pixels in the first captured image 70A with a preset threshold, and determines pixels whose brightness values change above the threshold.
[0104] For example, the second control unit 250 moves the target pixel from the vertical center of the first captured image 70A to the vertical outer edge while detecting pixels whose brightness value difference with the previous target pixel is greater than or equal to a threshold. When the second control unit 250 detects a pixel whose brightness value difference is greater than or equal to the threshold, it determines that the target pixel is a pixel located in the background region 53 outside the projection region 51. Furthermore, the second control unit 250 determines that the previous pixel of the target pixel is a pixel located within the projection region 51 and at the boundary line between the projection region 51 and the background region 53. The second control unit 250 obtains the coordinates of the pixel determined to be located at the boundary line in the shooting coordinate system.
[0105] Similarly, the second control unit 250 moves the target pixel from the horizontal center to the horizontal outer side of the first captured image 70A while detecting pixels whose brightness value difference with the previous target pixel is greater than or equal to a threshold. When the second control unit 250 detects a pixel whose brightness value difference is greater than or equal to the threshold, it determines that the target pixel is a pixel located in the background region 53 outside the projection region 51. Furthermore, the second control unit 250 determines that the previous pixel of the target pixel is a pixel located within the projection region 51 and at the boundary line between the projection region 51 and the background region 53. The second control unit 250 obtains the coordinates of the pixel determined to be located at the boundary line in the shooting coordinate system.
[0106] When the second control unit 250 detects pixels located on the boundary lines of the projection area 51 on the four sides of the projection area 51, it determines the pixels located at the four vertices of the projection area 51.
[0107] The upper left of the projection region 51 is marked as vertex M1, the upper right as vertex M2, the lower left as vertex M3, and the lower right as vertex M4.
[0108] Vertex M1 corresponds to the pixel with the smallest u-axis and v-axis coordinate values among the pixels located on the boundary line between projection region 51 and background region 53.
[0109] Vertex M2 corresponds to the pixel with the largest u-axis coordinate value and the smallest v-axis coordinate value among the pixels located on the boundary line between the projection region 51 and the background region 53.
[0110] Vertex M3 corresponds to the pixel with the smallest u-axis coordinate value and the largest v-axis coordinate value among the pixels located on the boundary line between the projection region 51 and the background region 53.
[0111] Vertex M4 corresponds to the pixel with the largest u-axis and v-axis coordinate values among the pixels located on the boundary line between projection region 51 and background region 53.
[0112] Next, the second control unit 250 calculates the rows and columns of the projection transformation. The second control unit 250 uses the coordinates of the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4 in the first image 10 in the panel coordinate system and the coordinates of the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4 in the shooting coordinate system in the first captured image 70A to calculate the rows and columns of the projection transformation. The panel coordinate system is a coordinate system set in the panel area 237 of the liquid crystal panel 235, such as... Figure 4 As shown, the coordinate system is set with the upper left corner of panel area 237 as the origin O1, and the horizontal axis as the s-axis and the vertical axis as the t-axis when viewed in the attached figure.
[0113] The coordinates of the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4 in the panel coordinate system represent the positions of the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4 of the first image 10 depicted on the liquid crystal panel 235. Furthermore, the coordinates of the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4 in the imaging coordinate system are the coordinates of the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4 detected from the first captured image 70A. These four contact points—the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4—of the first image 10 in the panel coordinate system correspond to the four points contained in the first image depicted on the panel. In addition, the four contacts T1, T2, T3 and T4 of the first image 10 in the panel coordinate system correspond to the four contacts T1, T2, T3 and T4 in the shooting coordinate system set on the first captured image 70A, respectively.
[0114] Figure 6 This is a diagram showing the positions of the four vertices of the projection area 51 in the panel coordinate system.
[0115] The second control unit 250 uses the calculated projection transformation column to transform the coordinates of the four vertices M1, M2, M3, and M4 of the projection area 51 from the shooting coordinate system to the panel coordinate system. The second control unit 250 transforms the coordinates of the four vertices M1, M2, M3, and M4 of the projection area 51 in the shooting coordinate system into the coordinates of the four vertices in the panel coordinate system using the projection transformation column. The four vertices M1, M2, M3, and M4 of the projection area 51 in the panel coordinate system are designated as vertices N1, N2, N3, and N4. The area defined by the four vertices N1, N2, N3, and N4 in the panel coordinate system, obtained by transforming the projection transformation column calculated based on the first captured image 70A and the first image 10, corresponds to the first area.
[0116] Figure 7 This is a diagram showing an example of the second image 20 depicted on the liquid crystal panel 235.
[0117] The second control unit 250 generates a second image 20 based on the vertices N1, N2, N3, and N4 of the four vertices M1, M2, M3, and M4 of the projection area 51 in the panel coordinate system. The second image 20 is generated with a second size larger than the first image 10's first size. The second control unit 250 generates the second image 20 such that the four vertices of the second image 20 are located at the positions of vertices N1, N2, N3, and N4, respectively. Like the first image 10, the second image 20 is a pattern image in which the first rectangular graphic 11 and the second rectangular graphic 13 are alternately arranged in the vertical and horizontal directions. However, the multiple first rectangular graphics 11 and second rectangular graphics 13 included in the second image 20 are not of the same shape or size. This is because the shape of the second image 20 is deformed corresponding to the distortion of the first image 10 displayed in the projection area 51.
[0118] Alternatively, the second control unit 250 may also generate rows and columns of projection transformations, and deform the first image 10 using the generated rows and columns of projection transformations to generate the second image 20.
[0119] The second control unit 250 generates a projection transformation row and column based on the coordinates of the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4 in the panel coordinate system, and the coordinates of vertices N1, N2, N3, and N4 in the panel coordinate system of the first image 10. The second control unit 250 may also use the generated projection transformation row and column to transform the coordinates of each vertex of the first rectangular graphic 11 and the second rectangular graphic 13 contained in the first image 10, thereby generating the second image 20.
[0120] Figure 8 This is a diagram showing an example of the second image 20 projected onto screen 5.
[0121] When generating the second image 20, the second control unit 250 controls the image processing unit 220 and the image projection unit 230 to depict the generated second image 20 in the panel area 237 of the liquid crystal panel 235. Light emitted from the light source 231 is modulated into image light corresponding to the second image 20 by passing through the panel area 237, and the modulated image light is displayed in the projection area 51. The second image 20 is located within the projection area 51 of the screen 5, and compared with the first image 10, it is displayed near the boundary line between the projection area 51 and the background area 53. Therefore, the error caused by the distortion parameters of the camera 120 can be reduced, and the coordinates of the second image 20 and the projection area 51 in the panel coordinate system can be detected with high precision based on the image 70 captured by the camera 120.
[0122] Figure 9 This is a diagram showing an example of the second captured image 70B.
[0123] Hereinafter, the captured image 70 obtained by taking a picture of the screen 5 displaying the second image 20 will be referred to as the second captured image 70B.
[0124] The second control unit 250 performs the same processing as when displaying the first image 10 on the screen 5, and receives the second captured image 70B from the information processing device 100. The second control unit 250 performs image analysis on the received second captured image 70B, and detects the second image 20 and the projection area 51.
[0125] As the second image 20, the second control unit 250 detects from the second captured image 70B. Figure 9 The four contacts shown are the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4. Specifically, the second control unit 250 acquires the coordinates of these four contacts in the shooting coordinate system set on the second captured image 70B. Furthermore, the second control unit 250 detects the coordinates of four vertices M5, M6, M7, and M8 of the projection area 51 based on changes in the brightness value of the second captured image 70B. The four contacts T1, T2, T3, and T4 in the shooting coordinate system set on the second captured image 70B correspond to four points contained in the second image.
[0126] Next, the second control unit 250 calculates the rows and columns of the projection transformation based on the four coordinates of the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4 in the panel coordinate system in the second image 20, and the four coordinates of the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4 in the shooting coordinate system in the second captured image 70B.
[0127] The four points T1, T2, T3, and T4 of the second image 20 in the panel coordinate system correspond to the four points contained in the second image on the drawing panel. Furthermore, these four points T1, T2, T3, and T4 of the second image 20 in the panel coordinate system correspond to the four points T1, T2, T3, and T4 in the shooting coordinate system set on the second captured image 70B, respectively.
[0128] Figure 10 This is a diagram showing the positions of the four vertices of the projection area in the panel coordinate system. Additionally, Figure 11This is a diagram showing an example of a projected image 30 displayed on screen 5.
[0129] The second control unit 250 uses the calculated projection transformation rows and columns to transform the coordinates of the four vertices M5, M6, M7, and M8 of the projection area 51 into coordinates in the panel coordinate system. The four vertices transformed into panel coordinates are labeled N5, N6, N7, and N8. The second control unit 250 generates the projected image 30 with the four vertices located at the positions of vertices N5, N6, N7, and N8, respectively. When generating the projected image 30, the second control unit 250 controls the image processing unit 220 and the image projection unit 230 to depict the generated projected image 30 in the panel area 237 of the liquid crystal panel 235. Light emitted from the light source 231 is modulated into image light corresponding to the projected image 30 as it passes through the panel area 237, and the modulated image light is displayed in the projection area 51. The region defined by the four vertices N5, N6, N7, and N8 in the panel coordinate system, obtained by transforming the rows and columns of the projection transformation calculated based on the first captured image 70B and the first image 10, corresponds to the second region.
[0130] Figure 12 This is a diagram showing an example of an operation screen 131 displayed on the touch panel 130 of the information processing device 100.
[0131] When the projected image 30 is displayed on the screen 5, the second control unit 250 sends a start instruction for rapid four-corner correction to the information processing device 100.
[0132] When the first control unit 150 receives a start instruction for rapid four-corner correction from the projector 200, it causes... Figure 12 The operation screen 131 shown is displayed on the touch panel 130.
[0133] The operation screen 131 displays the selection screen 133 for selecting the vertex of the calibration object, the cross key 135, and the OK key 137.
[0134] The user operates the directional pad 135 to select a vertex of the projected image 30 as the target for quick four-corner correction. The first control unit 150 determines that a vertex of the projected image 30 has been selected as the target for quick four-corner correction by selecting the vertex of the projected image 30 and pressing the confirm button 137. The first control unit 150 sends the information of the selected vertex of the projected image 30 to the projector 200.
[0135] Next, the user presses the cross key 135 in the direction in which the vertex of the selected projected image 30 is to be moved. While the cross key 135 is pressed, the first control unit 150 sends an operation signal corresponding to the pressed cross key 135 to the projector 200. While receiving the operation signal from the information processing device 100, the second control unit 250 moves the vertex of the selected projected image 30 in the direction corresponding to the pressed cross key 135. That is, the second control unit 250 moves the position of the panel area 237 depicting the vertex of the selected projected image 30 in the direction corresponding to the pressed cross key 135. When the first control unit 150 no longer detects operation of the cross key 135, it stops sending operation signals to the projector 200. Furthermore, when the confirmation key 137 is pressed, the first control unit 150 sends an operation signal indicating the end of quick four-corner correction to the projector 200.
[0136] 4. Projector operation
[0137] Figure 13 This is a flowchart illustrating the operation of projector 200. (See attached diagram.) Figure 13 The flowchart shown illustrates the operation of the projector 200.
[0138] The second control unit 250 determines whether it has received an instruction signal from the information processing device 100 to indicate the start of image adjustment (step S1). If no instruction signal is received (step S1 / No), the second control unit 250 waits for the start of processing until an instruction signal is received.
[0139] When the second control unit 250 receives an instruction signal from the information processing device 100 (step S1 / Yes), it displays the first image 10 on the screen 5 (step S2).
[0140] Next, the second control unit 250 sends the shooting instruction for the first captured image 70A and the acquisition request for the first captured image 70A to the information processing device 100 (step S3). Then, the second control unit 250 determines whether the first captured image 70A has been received from the information processing device 100 (step S4). If the first captured image 70A has not been received (step S4 / No), it waits for the start of the next process until the first captured image 70A is received.
[0141] When the second control unit 250 receives the first captured image 70A from the information processing device 100 (step S4 / Yes), it performs image analysis on the received first captured image 70A. The second control unit 250 detects four contacts of the first image 10 from the first captured image 70A: the first contact T1, the second contact T2, the third contact T3, and the fourth contact T4 (step S5).
[0142] Next, the second control unit 250 analyzes the first captured image 70A and detects the four vertices M1, M2, M3 and M4 of the projection area 51 (step S6).
[0143] Next, the second control unit 250 calculates the rows and columns of the projection transformation (step S7).
[0144] The second control unit 250 calculates the rows and columns of the projection transformation based on the four coordinates of the first contact T1, the second contact T2, the third contact T3, and the fourth contact T4 in the panel coordinate system, and the four coordinates of the first contact T1, the second contact T2, the third contact T3, and the fourth contact T4 in the shooting coordinate system.
[0145] Next, the second control unit 250 uses the calculated projection transformation rows and columns to transform the coordinates of the four vertices M1, M2, M3 and M4 of the projection area 51 into the coordinates of the four vertices N1, N2, N3 and N4 of the panel coordinate system (step S8).
[0146] Next, the second control unit 250 generates a second image 20 based on the calculated coordinates of the four vertices N1, N2, N3, and N4 of the projection area 51 in the panel coordinate system (step S9). The second control unit 250 generates the second image 20 with the four vertices located at the positions of the four vertices N1, N2, N3, and N4.
[0147] Next, the second control unit 250 displays the generated second image 20 on the screen 5 (step S10). Then, the second control unit 250 sends a shooting instruction for the second captured image 70B and a request to acquire the second captured image 70B to the information processing device 100 (step S11). The second control unit 250 determines whether the second captured image 70B has been received from the information processing device 100 (step S12). If the second control unit 250 has not received the second captured image 70B from the information processing device 100 (step S12 / No), it waits until the second captured image 70B is received.
[0148] When the second control unit 250 receives the second captured image 70B from the information processing device 100 (step S12 / Yes), it detects the four nodes of the second image 20—the first node T1, the second node T2, the third node T3, and the fourth node T4—similar to step S5 (step S13). Additionally, similar to steps S6 and S7, the second control unit 250 detects the four vertices of the projection area 51 (step S14) and calculates the rows and columns of the projection transformation (step S15).
[0149] Then, similarly to steps S8 and S9, the second control unit 250 uses the rows and columns of the projection transformation to transform the four vertices M1, M2, M3, and M4 of the projection area 51 into four vertices N1, N2, N3, and N4 of the panel coordinate system, and calculates the coordinates of the four vertices in the panel coordinate system (step S16). In addition, the second control unit 250 generates the projected image 30 based on the four vertices N1, N2, N3, and N4 of the transformed panel coordinate system (step S17).
[0150] The second control unit 250 displays the generated projected image 30 on the screen 5 (step S18) and sends a start instruction for rapid four-corner correction to the information processing device 100 (step S19). Then, the second control unit 250 determines whether it has received an operation signal indicating the vertex to be corrected from the information processing device 100 (step S20).
[0151] If the second control unit 250 does not receive an operation signal indicating the vertex to be corrected (step S20 / No), it determines whether it has received an operation signal indicating the end of rapid four-corner correction from the information processing device 100 (step S21). If the second control unit 250 does not receive an operation signal indicating the end of rapid four-corner correction (step S21 / No), it returns to the determination in step S20. Alternatively, if the second control unit 250 receives an operation signal indicating the end of rapid four-corner correction (step S21 / Yes), it terminates the processing flow.
[0152] Furthermore, if the second control unit 250 receives an operation signal indicating a vertex to be corrected (step S20 / Yes), it determines whether an operation signal corresponding to the operated cross key has been received (step S22). If the second control unit 250 does not receive an operation signal corresponding to the operation of the cross key (step S22 / No), it proceeds to the determination in step S21.
[0153] Furthermore, upon receiving an operation signal corresponding to the operated cross key (step S22 / Yes), the second control unit 250 moves the position of the panel area 237 depicting the vertex of the selected projection image 30 in the direction indicated by the operation signal (step S23). Then, the second control unit 250 determines whether to continue receiving operation signals corresponding to the cross key 135 (step S22).
[0154] 5. The effect of the projected image adjustment method
[0155] As explained above, the second control unit 250 of the projector 200 in this embodiment performs the following processing.
[0156] The second control unit 250 draws a first image 10 of a first size on the liquid crystal panel 235 and projects image light corresponding to the first image 10 drawn on the liquid crystal panel 235 onto the screen 5.
[0157] In addition, the second control unit 250 detects the projection area 51 and the first image 10 from the first captured image 70A obtained by capturing the image light corresponding to the first image 10 on the screen 5.
[0158] Furthermore, the second control unit 250 determines the first region in the liquid crystal panel 235 corresponding to the projection region 51 based on the projection region 51 in the first captured image 70A, the first image 10 in the first captured image 70A, and the first image 10 in the liquid crystal panel 235.
[0159] In addition, the second control unit 250 depicts a second image 20 that is larger than the first image in the first area, and projects image light corresponding to the second image 20 depicted on the liquid crystal panel 235 onto the screen 5.
[0160] The second control unit 250 detects the projection area 51 and the second image 20 from the second captured image 70B obtained by capturing the image light corresponding to the second image 20 on the screen 5.
[0161] Furthermore, the second control unit 250 determines the second region in the liquid crystal panel 235 corresponding to the projection region 51 based on the projection region 51 in the second captured image 70B, the second image 20 in the second captured image 70B, and the second image 20 in the liquid crystal panel 235.
[0162] Furthermore, the second control unit 250 depicts the projected image in the second area and projects the image light corresponding to the projected image onto the screen 5.
[0163] According to this structure, the first size of the first image 10 is smaller than the second size of the second image 20. By displaying the first image 10, which is smaller than the panel area 237 of the liquid crystal panel 235, on the screen 5 first, the first image 10 can be displayed on the screen 5 in a manner that does not extend beyond the projection area 51. Therefore, when detecting the first image 10 from the first captured image 70A, the probability of detection failure of the first image 10 can be reduced.
[0164] Furthermore, by detecting the projection area 51 and the first image 10 from the first captured image 70A, a region of the liquid crystal panel 235 corresponding to the projection area 51 is determined, and a second image 20 is drawn on the liquid crystal panel 235 corresponding to the determined projection area 51. Then, the second image 20 drawn on the liquid crystal panel 235 is displayed on the screen 5. Therefore, the influence of lens distortion of the camera 120 that captures the first captured image 70A and the second captured image 70B can be reduced, and the region of the liquid crystal panel 235 corresponding to the projection area 51 can be determined with high precision. Therefore, the projected image projected by the projector 200 can be matched with the shape of the projection area 51 with high precision.
[0165] In addition, the first image 10 and the second image 20 are rectangular images, and the first image 10 and the second image 20 contain images of at least four predefined shapes.
[0166] According to this structure, by detecting images of a pre-defined shape from the first captured image and the second captured image, it is possible to detect the regions of the first image 10 and the second image 20 with high precision.
[0167] Furthermore, detecting the projection region 51 and the first image 10 from the first captured image 70A means detecting the vertices of the projection region 51 and the points of the first image 10.
[0168] According to this structure, the vertices of the projection region 51 and the points of the first image 10 are detected from the first captured image 70A, thereby enabling the detection of the projection region 51 and the first image 10.
[0169] Image 10 is a rectangular image.
[0170] The second control unit 250 calculates the rows and columns of the projection transformation based on the four points contained in the first image 10 in the first captured image 70A and the four points contained in the first image 10 in the liquid crystal panel 235 corresponding to the four points in the first captured image. The second control unit 250 then detects the positions of the four vertices of the projection area 51 detected from the first captured image 70A on the liquid crystal panel 235 according to the calculated rows and columns of the projection transformation.
[0171] Based on this structure, the area of the liquid crystal panel 235 corresponding to the projection area 51 can be determined with high precision.
[0172] Image 20 is a rectangular image.
[0173] The second control unit 250 calculates the rows and columns of the projection transformation based on the four points contained in the second image 20 in the second captured image 70B and the four points contained in the second image 20 in the liquid crystal panel 235 corresponding to the four points in the second captured image. The second control unit 250 then detects the positions of the four vertices of the projection area 51 detected from the second captured image 70B on the liquid crystal panel 235 according to the calculated rows and columns of the projection transformation.
[0174] Based on this structure, the area of the liquid crystal panel 235 corresponding to the projection area 51 can be determined with high precision.
[0175] 6. Variations
[0176] Figure 14 It means as Figure 1 A diagram showing the structure of another projection system 1B, which is an alternative to the projection system 1A shown.
[0177] exist Figure 14 In the projection system 1B shown, the information processing device 100 does not have a camera 120, but it has a wired I / F 140 for wired connection to an external camera 300. The connection between the information processing device 100 and the camera 300 can also be a wireless connection based on short-range wireless communication such as Wi-Fi or Bluetooth. The camera 300 acts as a shooting device, and the information processing device 100 acts as a control device.
[0178] In addition, in this modified example, the first control unit 150 of the information processing device 100 performs the processing performed by the second control unit 250 of the projector 200.
[0179] The first control unit 150 receives a first captured image 70A from the camera 300, which is obtained by capturing a picture of the screen 5 displaying the first image 10.
[0180] Additionally, the first control unit 150 receives information about the first image 10 depicted on the liquid crystal panel 235 of the projector 200 from the projector 200. The information about the first image 10 includes the coordinates of the first contact T1, the second contact T2, the third contact T3, and the fourth contact T4 contained within the first image 10. These coordinates are coordinates in the panel coordinate system.
[0181] The first control unit 150 performs image analysis on the received first captured image 70A, detecting the first image 10 and the projection area 51. More specifically, the first control unit 150 detects the coordinates of the first contact point T1, the second contact point T2, the third contact point T3, and the fourth contact point T4 of the first image 10 in the shooting coordinate system. Additionally, the first control unit 150 detects the coordinates of the four vertices M1, M2, M3, and M4 of the projection area 51 in the shooting coordinate system.
[0182] The first control unit 150 calculates the rows and columns of the projection transformation. The first control unit 150 calculates the rows and columns of the projection transformation based on the coordinates of the first contact T1, the second contact T2, the third contact T3, and the fourth contact T4 in the shooting coordinate system, and the coordinates of the first contact T1, the second contact T2, the third contact T3, and the fourth contact T4 in the panel coordinate system received from the projector 200.
[0183] Next, the first control unit 150 transforms the coordinates of the four vertices M1, M2, M3, and M4 of the projection area 51 into the coordinates of the four vertices N1, N2, N3, and N4 of the panel coordinate system based on the calculated rows and columns of the projection transformation. The first control unit 150 then sends the transformed coordinate information of the four vertices N1, N2, N3, and N4 of the panel coordinate system to the projector 200.
[0184] The projector 200 generates a second image 20 based on the coordinates of the four vertices N1, N2, N3 and N4 received from the information processing device 100, and displays the generated second image 20 on the screen 5.
[0185] The information processing device 100 receives a second captured image 70B from the camera 300, which is obtained by capturing a picture of the screen 5 displaying the second image 20.
[0186] Similar to the image analysis performed on the first captured image 70A, the first control unit 150 performs image analysis on the second captured image 70B, calculating the coordinates of the four vertices N5, N6, N7, and N8 of the projection area 51 in the panel coordinate system. The first control unit 150 then sends the calculated coordinates of the four vertices N5, N6, N7, and N8 of the projection area 51 in the panel coordinate system to the projector 200.
[0187] The projector 200 generates a projected image 30 based on the coordinates of the four vertices N5, N6, N7 and N8 of the projection area 51 in the panel coordinate system received from the information processing device 100, and displays the generated projected image 30 on the screen 5.
[0188] 7. The effect of the projection system
[0189] As described above, the projection system 1A of this embodiment includes a projector 200 and an information processing device 100 having a camera 120 and a first control unit 150.
[0190] The projector 200 has a liquid crystal panel 235 that projects image light corresponding to a first image 10 of a first size depicted on the liquid crystal panel 235 onto the screen 5.
[0191] The camera 120 captures an image of the screen 5 on which image light corresponding to the first image 10 is projected, thereby generating a first captured image 70A. The first control unit 150 detects the projection area 51 and the first image 10 from the first captured image 70A.
[0192] The first control unit 150 determines the first region in the liquid crystal panel 235 corresponding to the projection region 51 based on the projection region 51 in the first captured image 70A, the first image 10 in the first captured image 70A, and the first image in the drawing panel 235.
[0193] The first control unit 150 sends the first information representing the first area to the projector 200.
[0194] In addition, the projector 200 depicts a second image 20 of a second size that is larger than the first size in the first area shown in the first information, and projects the image light corresponding to the second image 20 onto the screen 5.
[0195] The camera 120 captures a second captured image 70B by photographing the screen 5 on which image light corresponding to the second image 20 is projected.
[0196] The first control unit 150 detects the projection area 51 and the second image 20 from the second captured image 70B.
[0197] The first control unit 150 determines the second region of the liquid crystal panel 235 corresponding to the projection region 51 based on the projection region 51 in the second captured image 70B, the second image 20 in the second captured image 70B, and the second image 20 in the liquid crystal panel 235.
[0198] The first control unit 150 sends the second information representing the second area to the projector 200.
[0199] According to this structure, the first size of the first image 10 is smaller than the second size of the second image 20. By displaying the first image 10, which is smaller than the panel area 237 of the liquid crystal panel 235, on the screen 5 first, the first image 10 can be displayed on the screen 5 in a manner that does not extend beyond the projection area 51. Therefore, when detecting the first image 10 from the first captured image 70A, the probability of detection failure of the first image 10 can be reduced.
[0200] Furthermore, by detecting the projection area 51 and the first image 10 from the first captured image 70A, a region of the liquid crystal panel 235 corresponding to the projection area 51 is determined, and a second image 20 is drawn on the liquid crystal panel 235 corresponding to the determined projection area 51. Then, the second image 20 drawn on the liquid crystal panel 235 is displayed on the screen 5. Therefore, the influence of lens distortion of the camera 120 that captures the first captured image 70A and the second captured image 70B can be reduced, and the region of the liquid crystal panel 235 corresponding to the projection area 51 can be determined with high precision. Therefore, the projected image projected by the projector 200 can be matched with the shape of the projection area 51 with high precision.
[0201] 8. The effect of the control device
[0202] As explained above, in this embodiment, the first control unit 150 receives from the camera 300 a first captured image 70A obtained by capturing an image on a screen 5 that projects an image light corresponding to the first image 10 of the first size. Additionally, the first control unit 150 receives information about the first image 10 depicted on the liquid crystal panel 235 of the projector 200 from the projector 200 that projects the image light corresponding to the first image 10.
[0203] The first control unit 150 detects the projection area 51 and the first image from the first captured image 70A. Based on the projection area 51 in the first captured image 70A, the first image in the first captured image 70A, and the information of the first image, the first control unit 150 determines the first region in the drawing panel 235 corresponding to the projection area 51.
[0204] The first control unit 150 sends the first information representing the first area to the projector 200.
[0205] Additionally, the first control unit 150 receives from the camera 300 a second captured image 70B obtained by capturing an image light projected onto a screen 5 that corresponds to a second image 20 of a second size larger than the first size.
[0206] The first control unit 150 detects the projection area 51 and the second image 20 from the second captured image 70B.
[0207] The first control unit 150 determines the second region in the drawing panel 235 corresponding to the projection region 51 based on the projection region 51 in the second captured image 70B, the second image 20 in the second captured image 70B, and the first information.
[0208] The first control unit 150 sends the second information representing the second area to the projector 200.
[0209] According to this structure, the first size of the first image 10 is smaller than the second size of the second image 20. By displaying the first image 10, which is smaller than the panel area 237 of the liquid crystal panel 235, on the screen 5 first, the first image 10 can be displayed on the screen 5 in a manner that does not extend beyond the projection area 51. Therefore, when detecting the first image 10 from the first captured image 70A, the probability of detection failure of the first image 10 can be reduced.
[0210] Furthermore, by detecting the projection area 51 and the first image 10 from the first captured image 70A, a region of the liquid crystal panel 235 corresponding to the projection area 51 is determined, and a second image 20 is drawn on the liquid crystal panel 235 corresponding to the determined projection area 51. Then, the second image 20 drawn on the liquid crystal panel 235 is displayed on the screen 5. Therefore, the influence of lens distortion of the camera 120 that captures the first captured image 70A and the second captured image 70B can be reduced, and the region of the liquid crystal panel 235 corresponding to the projection area 51 can be determined with high precision. Therefore, the projected image projected by the projector 200 can be matched with the shape of the projection area 51 with high precision.
[0211] The above embodiments are preferred embodiments of the present invention. However, the invention is not limited to the above methods, and various modifications can be made without departing from the spirit of the invention.
[0212] For example, image 10 and image 20 could also be Figure 15 The image shown. Figure 15 The first image 10 shown is an image in which points 15A, 15B, 15C, and 15D, representing images of pre-defined shapes, are arranged at positions corresponding to the four corners of the first image 10. Figure 15 In the first image 10, a circular image is shown as a predefined shape, but the image shape can also be a triangle or a rectangle. Additionally, points 15A, 15B, 15C, and 15D in the first image 10 may not be positioned at the corresponding corners. An image containing at least four predefined shapes can be used.
[0213] Furthermore, the patterns formed in the first image 10 and the second image 20 can also be different. For example, the first image 10 could use an image with points 15A, 15B, 15C, and 15D arranged at positions corresponding to the four corners, and the second image 20 could use... Figure 8 The pattern image shown.
[0214] Furthermore, the first image 10 and the second image 20 only need to differ in any one of the sizes of the diagonal, vertical, or horizontal dimensions of the first rectangular graphic 11 and the second rectangular graphic 13.
[0215] Furthermore, the light modulation element in the light modulation device 233 can be either a transmissive liquid crystal panel or a reflective liquid crystal panel. Additionally, the light modulation element can be a structure using a DMD or a structure combining a DMD and a color wheel. Furthermore, in addition to the liquid crystal panel and DMD, the light modulation device 233 can also employ a structure capable of modulating light emitted from a light source. Furthermore, in the above embodiment, three liquid crystal panels 235R, 235G, and 235B corresponding to the three colors red, green, and blue are used, but this is not a limitation. A structure using a single liquid crystal panel is also possible, as is a structure using three DMDs corresponding to the three colors red, green, and blue. The drawing panel can be any panel capable of adjusting the projected light from the light source through reflection, transmission, or modulation in order to form the projected image displayed by the projector 200. Drawing by the drawing panel corresponds to adjusting the projected light from the light source through reflection, transmission, or modulation.
[0216] in addition, Figure 1 The projector 200 and information processing device 100 shown represent functional structures, and there are no particular restrictions on the specific installation method. That is, it is not necessary to install hardware corresponding to each functional unit individually; of course, it is also possible to configure multiple functional units to implement their functions by executing programs from one or more processors. Furthermore, in the above embodiments, some functions implemented by software can be implemented by hardware, and some functions implemented by hardware can be implemented by software. In addition, the specific details of the structure of other parts of the projector can be arbitrarily changed without departing from the spirit of the present invention.
[0217] Furthermore, when the method for adjusting the projected image is implemented using the computer included in the projector 200 or the computer included in the information processing device 100, the program that the computer can execute can be configured using a recording medium or a transmission medium for transmitting the program. The recording medium can be a magnetic, optical, or semiconductor storage device. Specifically, examples include removable or fixed recording media such as floppy disks, HDDs (Hard Disk Drives), CD-ROMs, DVDs, Blu-ray discs, magneto-optical discs, flash memory, and card-type recording media. Additionally, the aforementioned recording media can also be non-volatile storage devices such as RAM, ROM, and HDDs, which are internal storage devices of server devices. Blu-ray is a registered trademark.
Claims
1. An adjustment method of a projected image, wherein, The method for adjusting the projected image has the following characteristics: Draw the first image of size 1 on the drawing panel; The first image is projected onto the projection surface; From a first captured image obtained by taking a picture of the projection surface on which the first image is projected, detect the region of the projection surface and the first image; Based on the region of the projection surface in the first captured image, the first image in the first captured image, and the first image in the drawing panel, a first region in the drawing panel corresponding to the region of the projection surface is determined; Within the first region, a second image of a second size larger than the first size is depicted; The second image is projected onto the projection surface; From a second captured image obtained by taking a picture of the projection surface on which the second image is projected, detect the region of the projection surface and the second image; Based on the region of the projection surface in the second captured image, the second image in the second captured image, and the second image in the drawing panel, a second region in the drawing panel corresponding to the region of the projection surface is determined; A projected image is depicted within the second region; and The projected image is projected onto the projection surface. The first image is a rectangular image. Based on the four points contained in the first captured image and the four points contained in the first image in the drawing panel corresponding to the four points in the first captured image, calculate the rows and columns of the projection transformation. Based on the rows and columns of the projection transformation, the positions of the four vertices of the projection surface detected from the first captured image are detected in the depiction panel. The second image is generated by having the four vertices of the second image in the drawing panel located in the positions of the four vertices of the projection surface detected from the first captured image in the drawing panel, respectively, using the row and column detection of the projection transformation. This results in a pattern image that is deformed in correspondence with the distortion of the first image projected onto the projection surface.
2. The method for adjusting the projected image according to claim 1, wherein, The second image is a rectangular image. The first image and the second image contain images of at least four predefined shapes.
3. The method for adjusting the projected image according to claim 2, wherein, Detecting the region of the projection surface from the first captured image and the first image refers to detecting the vertices of the projection surface and the points of the first image.
4. The method for adjusting the projected image according to claim 1, wherein, The second image is a rectangular image. Based on the four points contained in the second image and the four points contained in the second image in the drawing panel corresponding to the four points in the second image, the rows and columns of the projection transformation are calculated. Based on the rows and columns of the projection transformation, the positions of the four vertices of the projection surface detected from the second captured image are detected in the depiction panel.
5. A projection system, wherein, The projection system has the following features: A projector having a drawing panel that projects a first image of a first size drawn on the drawing panel onto a projection surface; An imaging device that captures an image of the projection surface on which the first image is projected to generate a first captured image; and Control device, The control device performs: Detect the region of the projection surface and the first image from the first captured image; Based on the region of the projection surface in the first captured image, the first image in the first captured image, and the first image in the drawing panel, a first region in the drawing panel corresponding to the region of the projection surface is determined; and The first information representing the first region is sent to the projector. The projector performs: Within the first region represented by the first information, a second image of a second size larger than the first size is depicted; and The second image is projected onto the projection surface. The imaging device captures an image of the projection surface onto which the second image is projected, thereby generating a second captured image. The control device also performs: Detect the region of the projection surface and the second image from the second captured image; Based on the region of the projection surface in the second captured image, the second image in the second captured image, and the second image in the drawing panel, a second region of the drawing panel corresponding to the region of the projection surface is determined; and The second information representing the second region is sent to the projector. The first image is a rectangular image. Based on the four points contained in the first captured image and the four points contained in the first image in the drawing panel corresponding to the four points in the first captured image, calculate the rows and columns of the projection transformation. Based on the rows and columns of the projection transformation, the positions of the four vertices of the projection surface detected from the first captured image are detected in the depiction panel. The second image is generated by having the four vertices of the second image in the drawing panel located in the positions of the four vertices of the projection surface detected from the first captured image in the drawing panel, respectively, using the row and column detection of the projection transformation. This results in a pattern image that is deformed in correspondence with the distortion of the first image projected onto the projection surface.
6. A control device comprising one or more processors, wherein, The processor performs the following processes by executing a program: The first captured image is obtained by capturing an image of a projection surface on which a first image of the first size is projected from the imaging device; Information about the first image depicted on the drawing panel of the projector is received from the projector that projects the first image onto the projection surface; Detect the region of the projection surface and the first image from the first captured image; Based on the region of the projection surface in the first captured image, the first image in the first captured image, and the information of the first image, a first region in the drawing panel corresponding to the region of the projection surface is determined; The first information representing the first region is sent to the projector; The second captured image is obtained by receiving the image from the imaging device and capturing the projection surface onto which a second image of a second size larger than the first size is projected; Detect the region of the projection surface and the second image from the second captured image; Based on the region of the projection surface in the second captured image, the second image in the second captured image, and the first information, a second region in the drawing panel corresponding to the region of the projection surface is determined; and The second information representing the second region is sent to the projector. The first image is a rectangular image. Based on the four points contained in the first captured image and the four points contained in the first image in the drawing panel corresponding to the four points in the first captured image, calculate the rows and columns of the projection transformation. Based on the rows and columns of the projection transformation, the positions of the four vertices of the projection surface detected from the first captured image are detected in the depiction panel. The second image is generated by having the four vertices of the second image in the drawing panel located in the positions of the four vertices of the projection surface detected from the first captured image in the drawing panel, respectively, using the row and column detection of the projection transformation. This results in a pattern image that is deformed in correspondence with the distortion of the first image projected onto the projection surface.
Citation Information
Patent Citations
Projector with distortion correcting function
JP2006246502A
Projection system
US20050168705A1
Image Projecting Method, Projector, and Computer Program Product
US20080284987A1
