Control method, projection system, and program

Abstract

To provide a control method that reduces a processing load of a user.SOLUTION: A control method comprises an optical modulator provided to a projection device, comprising steps of: drawing a drawn image including a first portion having a first number of pixels; projecting the drawn image onto a projection target as a projected image by the projection device; acquiring an imaging image by an imaging device that images the projected image having a second portion corresponding to the first portion; and determining a minimum width of a stripe pattern containing a bright part or a dark part that can be detected by the imaging device on the basis of the first number of pixels and a second number of pixels that is the number of pixels of a portion corresponding to the second portion in the projected image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a control method, a projection system, and a program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there is known a technique in which a pattern projected onto a projection target by a projection device is captured by a camera, and various processes are performed based on the captured image.

[0003] For example, Patent Document 1 discloses a measurement device that performs measurements based on a captured image of a subject onto which pattern light having bright and dark areas is projected. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2011-47931 Summary of the Invention [Problem to be solved by the invention]

[0005] However, if the user provides a separate camera, the minimum width of the pattern light having light and dark areas that the camera can distinguish varies depending on the camera's performance, so this minimum width must be determined, which requires time and effort in processing. [Means for solving the problem]

[0006] The present disclosure provides a control method for a projection system including a projection device that includes an optical modulator including a plurality of pixels and projects one or more stripe patterns including bright and dark areas onto a projection target, and an imaging device that captures the one or more stripe patterns projected onto the projection target, the control method including: the optical modulator drawing a drawn image including a first portion having a first number of pixels; the projection device projecting the drawn image onto the projection target as a projected image; the imaging device capturing the projected image having a second portion corresponding to the first portion to obtain a captured image; and determining the minimum width of the bright or dark area that can be detected by the imaging device based on the first number of pixels and a second number of pixels, which is the number of pixels of a third portion of the projected image included in the captured image that corresponds to the second portion.

[0007] The present disclosure provides a projection system comprising: a projection device including an optical modulator having a plurality of pixels, which projects one or more stripe patterns including bright and dark areas onto a projection target; and an imaging device which captures the one or more stripe patterns projected onto the projection target, wherein the optical modulator draws a drawn image including a first portion having a first number of pixels; the projection device projects the drawn image onto the projection target as a projected image; the imaging device captures the projected image having a second portion corresponding to the first portion to obtain a captured image; and the imaging device determines the minimum width of the bright or dark area that can be detected based on the first number of pixels and a second number of pixels, which is the number of pixels of a third portion of the projected image included in the captured image that corresponds to the second portion.

[0008] The present disclosure relates to a program in which a processor mounted on a projection device that includes an optical modulator including a plurality of pixels and projects one or more stripe patterns including bright and dark areas onto a projection target executes the following steps: causes the optical modulator to draw a drawn image including a first portion having a first number of pixels; projects the drawn image onto the projection target as a projection image; obtains from an imaging device a captured image that captures the projected image having a second portion corresponding to the first portion; and determines the minimum width of the bright or dark area that can be detected by the imaging device based on the first number of pixels and a second number of pixels, which is the number of pixels of a third portion of the projected image included in the captured image that corresponds to the second portion. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing the system configuration of a projection system. [Figure 2] FIG. 10 is a diagram showing a pattern image drawn on a liquid crystal panel. [Figure 3] FIG. 4 is a diagram showing a pattern image projected onto a projection target. [Figure 4] FIG. 2 is a diagram showing an image captured by a camera. [Figure 5] FIG. 10 is a diagram showing a case where four panel pixels are imaged by one pixel of the imaging element. [Figure 6] FIG. 10 is a diagram showing a case where two panel pixels are imaged using one pixel of an imaging element. [Figure 7] 10 is a flowchart showing the operation of the information processing device. [Figure 8] FIG. 4 is a diagram showing a pattern image projected onto a projection target. [Figure 9] FIG. 2 is a diagram showing an image captured by a camera. DETAILED DESCRIPTION OF THE INVENTION

[0010] [1. Projection system configuration] FIG. 1 is a diagram showing the system configuration of a projection system 1. As shown in FIG. The projection system 1 includes an information processing device 100, a projector 200, and a camera 300. The projector 200 and the camera 300 are connected to the information processing device 100. While FIG. 1 shows an example in which the projector 200 and the camera 300 are connected to the information processing device 100 via a wired connection, the information processing device 100, the projector 200, and the camera 300 may be connected via wireless communication such as Bluetooth or Wi-Fi. Bluetooth and Wi-Fi are registered trademarks. The projector 200 corresponds to a projection device. The camera 300 corresponds to an imaging device.

[0011] [2. Configuration of information processing device] The information processing device 100 is configured by, for example, a desktop or notebook personal computer, or may be a smartphone or a tablet terminal. The information processing device 100 includes a first interface 110, an operation unit 120, a display unit 130, and a first control unit 150. Hereinafter, the first interface 110 will be abbreviated as first_I / F110.

[0012] The first I / F 110 includes a connection terminal compliant with standards such as HDMI (High-Definition Multimedia Interface) and USB (Universal Serial Bus), and an I / F circuit. HDMI is a registered trademark. The connection terminal and I / F circuit are not shown in the drawings. The first I / F 110 is connected to external devices such as the camera 300 and the projector 200 via a wired connection, and transmits and receives data to and from the connected external devices. The first I / F 110 may include a communication antenna to establish a wireless connection with the external devices.

[0013] The operation unit 120 includes operation devices such as a mouse and a keyboard, and functions as a reception unit that receives user operations.

[0014] The display unit 130 includes a display panel such as a liquid crystal panel or an organic EL (Electro Luminescence) panel. The display panel is not shown. The display unit 130 displays an image based on image data input from the first control unit 150 on the display panel.

[0015] The first control unit 150 is a computer device that includes a first storage unit 160 and a first processor 170 .

[0016] The first storage unit 160 includes a volatile memory such as a RAM (Random Access Memory) and a non-volatile memory such as a ROM (Read Only Memory). The first storage unit 160 may also include an auxiliary storage device such as an SSD (Solid State Drive) or an HDD (Hard Disk Drive).

[0017] The RAM is used for temporarily storing various data and the like, and the ROM stores a control program 161 used to control the operation of the information processing device 100.

[0018] The first processor 170 is an arithmetic processing device including a CPU (Central Processing Unit) and an MPU (Micro Processing Unit). The first processor 170 may be configured with a single processor or multiple processors. The first processor 170 may also be configured with a SoC (System-on-a-chip) integrated with part or all of the first storage unit 160 or other circuits. The first processor 170 may also be configured with a combination of a CPU that executes programs and a DSP (Digital Signal Processor) that executes predetermined arithmetic processing. Furthermore, all of the functions of the first processor 170 may be implemented in hardware, or may be configured using a programmable device.

[0019] [3. Projector configuration] The projector 200 includes a remote control receiver 210, a second interface 220, an image processor 231, a frame memory 233, a projection unit 240, and a second controller 250. Hereinafter, the second interface 220 will be referred to as second_I / F220.

[0020] The remote control light receiving unit 210 receives an infrared signal transmitted from the remote control 5. The remote control light receiving unit 210 is, for example, a photodiode. The remote control 5 has a plurality of operation buttons for operating the projector 200, and functions as a reception unit that receives user operations. The remote control 5 transmits to the projector 200 an infrared signal corresponding to the operation button operated by the user. The remote control light receiving unit 210 decodes the received infrared signal and generates an operation signal corresponding to the received infrared signal. The remote control light receiving unit 210 outputs the generated operation signal to the second control unit 250.

[0021] The second_I / F 220 includes, for example, a connection terminal and an I / F circuit. The connection terminal and the I / F circuit are not shown. The second_I / F 220 is connected to the information processing device 100, which is an external device, via a wired connection, and transmits and receives data to and from the connected information processing device 100. The second_I / F 220 may include a communication antenna to connect to the external device via wireless communication.

[0022] A frame memory 233 is connected to the image processing unit 231. The image processing unit 231 expands image data input from the second I / F 220 in the frame memory 233. The frame memory 233 is configured by, for example, a Synchronous Dynamic Random Access Memory (SDRAM) or the like.

[0023] The image processing unit 231 performs image processing such as resolution conversion or resizing, distortion correction, shape correction, digital zooming, and adjustment of image color and brightness on the image data stored in the frame memory 233. The image processing unit 231 executes image processing specified by the second control unit 250, and performs processing using parameters input from the second control unit 250 as necessary. Of course, the image processing unit 231 can also execute a combination of multiple image processing operations from the above. The image processing unit 231 reads the processed image data from the frame memory 233 and outputs the read image data to the light modulation device 243 of the projection unit 240.

[0024] The image processing unit 231 and the frame memory 233 are configured, for example, by an integrated circuit. The integrated circuit includes an LSI (Large Scale Integrated Circuit), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field-Programmable Gate Array), an SoC, etc. An analog circuit may be included as part of the integrated circuit configuration, or the second control unit 250 may be configured in combination with an integrated circuit.

[0025] The projection unit 240 includes a light source 241 , a light modulation device 243 , and an optical unit 247 .

[0026] The light source 241 is configured by a lamp such as a halogen lamp, a xenon lamp, or an ultra-high pressure mercury lamp, or a solid-state light source such as an LED (Light Emitting Diode) or a laser light source.

[0027] The light modulation device 243 modulates light emitted by the light source. There are no limitations on the specific configuration of the light modulation device 243. For example, the light modulation device 243 can be configured with a transmissive liquid crystal panel, a reflective liquid crystal panel, or a digital micromirror device. In this embodiment, a case will be described in which the light modulation device 243 includes a transmissive liquid crystal panel 245.

[0028] The optical unit 247 includes projection lenses, such as a focus lens for focus adjustment and a zoom lens for zoom adjustment. The projection lenses are not shown. The optical unit 247 forms an image of the image light emitted by the light modulation device 243 onto the projection target 20 using the projection lenses. As a result, an image based on the image light is enlarged and projected onto the projection target 20.

[0029] The second control unit 250 is a computer device that includes a second storage unit 260 and a second processor 270 .

[0030] The second storage unit 260 includes a volatile memory such as a RAM, and a non-volatile memory such as a ROM. The RAM is used for temporarily storing various data, and the ROM stores a control program 261 used to control the operation of the projector 200 and various setting information.

[0031] The second processor 270 is an arithmetic processing unit including a CPU and an MPU. The second processor 270 executes the control program 261 to control each unit of the projector 200. The second processor 270 may be configured with a single processor or multiple processors. The second processor 270 may also be configured with an SoC integrated with part or all of the second storage unit 260 or other circuits. The second processor 270 may also be configured with a combination of a CPU that executes a program and a DSP that executes predetermined arithmetic processing. Furthermore, all of the functions of the second processor 270 may be implemented in hardware, or may be configured using a programmable device.

[0032] [4. Camera configuration] The camera 300 corresponds to an imaging device and includes a photographing lens, an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary MOS), and a data processing circuit. The photographing lens, imaging element, and data processing circuit are not shown. In response to an instruction from the information processing device 100, the camera 300 photographs an area including at least the projection target 20 and generates a photographed image. The camera 300 outputs the generated photographed image to the information processing device 100.

[0033] [5. Projection System Operation] The first control unit 150 of the information processing device 100 uses the Gray code method to determine the correspondence between the imaging pixels of the imaging element of the externally connected camera 300 and the panel pixels of the liquid crystal panel 245 included in the projector 200. More specifically, a transformation matrix is determined to convert the imaging coordinates set in the captured image 50 captured by the camera 300 into the panel coordinates of the liquid crystal panel 245 of the projector 200.

[0034] At this time, before projecting the Gray code pattern of the Gray code method onto the projection target 20, the minimum bit width of the light or dark part of the Gray code pattern that can be detected from the image 50 captured by the camera 300 is calculated in advance. This reduces processing time by not projecting onto the projection target 20 Gray code patterns that cannot be detected by the camera 300. Furthermore, if it is determined that the resolution of the camera 300 is excessive and that it will take time to detect the Gray code pattern, a message is displayed to the user to lower the resolution of the camera 300. The Gray code pattern corresponds to one or more stripe patterns.

[0035] First, the first control unit 150 transmits pattern image data to the projector 200. This pattern image data is, for example, an image in which the entire surface of the image is formed in a single color such as white or black. In this embodiment, a case will be described in which the pattern image data is image data in which the entire surface is white. In addition, the first control unit 150 acquires information on the panel resolution of the liquid crystal panel 245 from the projector 200, and stores the acquired panel resolution information in the first storage unit 160.

[0036] Fig. 2 is a diagram showing a drawn image 30A drawn on the liquid crystal panel 245 provided in the projector 200. Fig. 3 is a diagram showing a pattern image 30B projected onto the projection target 20 by the projector 200. When the second control unit 250 of the projector 200 receives the pattern image data from the information processing device 100, it draws the received pattern image data on the liquid crystal panel 245 to generate a drawn image 30A. Image light is generated by light emitted from the light source 241 passing through the liquid crystal panel 245, and the generated image light is projected onto the projection target 20 by the projection lens. As a result, a pattern image 30B based on the pattern image data is projected onto the projection target 20 as a projected image 33. The pattern image 30B corresponds to a first image having only the first color.

[0037] The second control unit 250 draws a pattern image based on the pattern image data on the entire surface of the liquid crystal panel 245, thereby forming a drawn image 30A on the entire surface of the liquid crystal panel 245. In FIG. 2, the four vertices of the liquid crystal panel 245 are referred to as an upper left vertex 31A, an upper right vertex 31B, a lower left vertex 31C, and a lower right vertex 31D. Further, the side of the liquid crystal panel 245 that connects the upper left vertex 31A and the upper right vertex 31B in the drawing is referred to as an upper side 32C. Further, the side of the liquid crystal panel 245 connecting the lower left vertex 31C and the lower right vertex 31D in the drawing is referred to as a lower side 32D. Further, the side of the liquid crystal panel 245 connecting the upper left vertex 31A and the lower left vertex 31C in the drawing is referred to as the left side 32A. Further, the side of the liquid crystal panel 245 connecting the upper right vertex 31B and the lower right vertex 31D in the drawing is referred to as the right side 32B.

[0038] When the second control unit 250 draws the pattern image 30B based on the pattern image data on the entire surface of the liquid crystal panel 245, the top side 32C or the bottom side 32D of the liquid crystal panel 245, or the left side 32A or the right side 32B of the liquid crystal panel 245 corresponds to the first portion. Moreover, the number of pixels on the top side 32C or the bottom side 32D, or the left side 32A or the right side 32B corresponds to the first number of pixels.

[0039] 3, from the perspective of the drawing, side 34C constituting pattern image 30B is referred to as upper side 34C, and side 34D constituting pattern image 30B is referred to as lower side 34D. Also, from the perspective of the drawing, side 34A constituting pattern image 30B is referred to as left side 34A, and side 34B constituting pattern image 30B is referred to as right side 34B. When the upper edge 32C or the lower edge 32D of the liquid crystal panel 245, or the left edge 32A or the right edge 32B of the liquid crystal panel 245 corresponds to the first part, the upper edge 34C or the lower edge 34D of the pattern image 30B, or the left edge 34A or the right edge 34B of the pattern image 30B corresponds to the second part corresponding to the first part.

[0040] FIG. 4 is a diagram showing an image 50 captured by the camera 300. As shown in FIG. Next, the first control unit 150 causes the camera 300 to capture an image of the projection target 20, and receives the captured image 50 captured by the camera 300 from the camera 300. The captured image 50 includes an area where the pattern image 30B projected by the projector 200 onto the projection target 20 is captured. The area of the captured image 50 where the pattern image 30B is captured is referred to as a pattern area 30C.

[0041] The first control unit 150 detects the pattern area 30C from the acquired captured image 50. The upper left vertex of the pattern area 30C is referred to as the upper left vertex 35A, the upper right vertex as the upper right vertex 35B, the lower left vertex as the lower left vertex 35C, and the lower right vertex as the lower right vertex 35D. Furthermore, from a drawing perspective, the side connecting the upper left vertex 35A and the upper right vertex 35B is referred to as the upper side 36C, and the side connecting the lower left vertex 35C and the lower right vertex 35D is referred to as the lower side 36D. Furthermore, from a drawing perspective, the side connecting the upper left vertex 35A and the lower left vertex 35C is referred to as the left side 36A, and the side connecting the upper right vertex 35B and the lower right vertex 35D is referred to as the right side 36B.

[0042] When the upper side 34C or the lower side 34D, or the left side 34A or the right side 34B of the pattern image 30B corresponds to the second part corresponding to the first part, the upper side 36C or the lower side 36D, or the left side 36A or the right side 36B of the pattern area 30C corresponds to the third part corresponding to the second part.

[0043] Here, when detecting pattern area 30C from captured image 50, first control unit 150 determines whether or not two corners constituting pattern area 30C have been detected. Since pattern area 30C is an area of captured image 50 obtained by capturing pattern image 30B, multiple corners are detected in captured image 50. However, if the size of pattern image 30B projected onto projection target 20 is larger than projection target 20, it may not be possible to detect multiple corners of pattern image 30B from captured image 50.

[0044] Therefore, when the number of corners of the pattern image 30B detected from the captured image 50 is one or less, the first control unit 150 instructs the projector 200 to reduce the size of the pattern image 30B to be projected onto the projection target 20. Furthermore, it is preferable that the first control unit 150 detects two or more corners of the pattern image 30B from the captured image 50, and that at least two corners are on both sides of the sides that form the outline of the pattern image 30B. Furthermore, when projecting a vertical stripe pattern having light and dark areas as a gray code pattern, it is preferable that the two corners of the pattern image 30B be the upper left vertex 35A and the upper right vertex 35B, or the lower left vertex 35C and the lower right vertex 35D, in order to detect the number of panel pixels of the liquid crystal panel 245 that express the width of the vertical stripes. In other words, it is preferable that the upper side 34C or the lower side 34D, which are the horizontal sides of the pattern image 30B, be detected.

[0045] When two or more corners are detected from the captured image 50, the first control unit 150 next identifies the number of pixels of the liquid crystal panel 245 used to draw the drawn image 30A. This number of pixels is called the panel pixel count. The first control unit 150 acquires information about the panel resolution from the first storage unit 160, and identifies the number of panel pixels used to draw the pattern image 30B based on the acquired information. The first control unit 150 identifies the number of panel pixels on the side of the liquid crystal panel 245 that corresponds to the side connecting two of the two or more detected corners. The identified number of panel pixels corresponds to the first pixel count. For example, when the upper left vertex 35A and the upper right vertex 35B of the pattern area 30C are detected from the captured image 50, the first control unit 150 identifies the number of panel pixels on the top side 32C of the liquid crystal panel 245 that corresponds to the top side 36C connecting these vertices. Furthermore, when the upper left vertex 35A and the lower left vertex 35C of the pattern area 30C are detected from the captured image 50, the first control unit 150 identifies the number of panel pixels on the left side 32A of the liquid crystal panel 245 that corresponds to the left side 36A connecting these vertices.

[0046] Next, the first control unit 150 specifies the number of pixels on the corresponding side of the pattern area 30C. The number of pixels on the side that constitutes the pattern area 30C is referred to as the number of captured pixels. The number of captured pixels on the side that constitutes the pattern area 30C corresponds to the second number of pixels. For example, when the upper left vertex 35A and the upper right vertex 35B of the pattern area 30C are detected from the captured image 50, the number of captured pixels of the upper side 36C of the pattern area 30C is detected. Also, when the upper left vertex 35A and the lower left vertex 35C of the pattern area 30C are detected from the captured image 50, the number of captured pixels of the left side 34A of the pattern area 30C is detected.

[0047] Next, the first control unit 150 calculates the ratio between the number of panel pixels of the liquid crystal panel 245 and the number of captured pixels of the pattern area 30C. For example, when the upper left vertex 35A and the upper right vertex 35B of the pattern area 30C are detected from the captured image 50, the first control unit 150 calculates the ratio between the number of panel pixels of the upper side 32C of the liquid crystal panel 245 and the number of captured pixels of the upper side 36C of the pattern area 30C. In addition, when the upper left vertex 35A and the lower left vertex 35C of the pattern area 30C are detected from the captured image 50, the ratio of the number of panel pixels on the left side 32A of the liquid crystal panel 245 to the number of captured pixels on the left side 34A of the pattern area 30C is calculated.

[0048] Fig. 5 is a diagram showing a case where four panel pixels are imaged by one pixel of the imaging element, and Fig. 6 is a diagram showing a case where two panel pixels are imaged by one pixel of the imaging element. Next, the first control unit 150 calculates an approximate value of the width of the stripes of the Gray code pattern that can be detected with the resolution of the camera 300 based on the calculated ratio. For example, assume that each of the four vertices of pattern area 30C can be detected from captured image 50, and the ratio between the number of panel pixels on top edge 32C of liquid crystal panel 245 and the number of captured pixels on top edge 36C of pattern area 30C is 4:1. If the four vertices of pattern area 30C can be detected from captured image 50, it can be determined that the four pixels of liquid crystal panel 245 can be resolved down to a single pixel of the imaging element. Therefore, first control unit 150 determines that it is possible to correctly determine a stripe pattern in which the width of one stripe is represented by four panel pixels. Furthermore, it determines that stripes in a stripe pattern whose width is narrower than four pixels of liquid crystal panel 245 cannot be detected even if they are displayed. Furthermore, if the ratio of the number of panel pixels on the upper side 32C of the liquid crystal panel 245 to the number of imaging pixels on the upper side 36C of the pattern area 30C is 4:1, the sampling theorem determines that up to twice the number of panel pixels, or 8 pixels, can be resolved down to a single pixel of the imaging element.

[0049] For example, when the ratio between the number of panel pixels of the liquid crystal panel 245 and the number of imaging pixels of the pattern area 30C is 4:1, the first control unit 150 determines that the minimum width of the stripes of the Gray code pattern that can be detected with the current resolution of the camera 300 is 4. The first control unit 150 determines whether or not there is a Gray code pattern whose stripe width is smaller than the determined minimum width. If there is no Gray code pattern whose stripe width is smaller than the determined minimum width, the first control unit 150 projects all Gray code patterns stored in the first storage unit 160 onto the projection target 20 and starts measurement using the Gray code method.

[0050] Furthermore, if there is a Gray code pattern whose stripe width is smaller than the determined minimum width, the first control unit 150 excludes the Gray code pattern whose stripe width is smaller than the minimum width from the objects to be projected onto the projection target 20. For example, as shown in FIG. 6, if the ratio between the number of panel pixels of the liquid crystal panel 245 and the number of imaging pixels in the pattern area 30C is 2:1, the first control unit 150 excludes this Gray code pattern from the objects to be projected onto the projection target 20. Furthermore, when the stripe pattern with the smallest width of the light or dark portions in a Gray code pattern is defined as the first stripe pattern, if the minimum width is smaller than the width of the light or dark portions of the first stripe pattern, a message may be displayed on the display unit 130. This message may be, for example, a message indicating that a Gray code pattern with a stripe width smaller than the minimum width has been detected, and therefore this Gray code pattern will be excluded from the objects to be projected onto the projection target 20.

[0051] Furthermore, if the camera 300 is capable of changing the resolution of the image sensor, the first control unit 150 may instruct the camera 300 to change the resolution. For example, if a Gray code pattern with a stripe width smaller than the minimum width exists among the Gray code patterns stored in advance in the first storage unit 160, the first control unit 150 instructs the camera 300 to increase the resolution of the camera 300. Furthermore, if there is no Gray code pattern with a width greater than twice the determined minimum width, the first control unit 150 instructs the camera 300 to lower the resolution of the camera 300.

[0052] FIG. 7 is a flowchart showing the operation of the information processing device 100. The operation of the information processing device 100 will be described with reference to the flowchart shown in FIG. First, the first control unit 150 transmits pattern image data to the projector 200 (step S1). When the projector 200 receives the pattern image data from the information processing device 100, the projector 200 projects a pattern image 30B based on the received pattern image data onto the projection target 20.

[0053] Next, the first control unit 150 instructs the camera 300 to capture an image (step S2). The angle of view of the camera 300 has been adjusted by the user so that the projection target 20 is included within the angle of view of the camera 300. When the camera 300 is instructed to capture an image by the information processing device 100, it captures the image and generates a captured image 50. The camera 300 transmits the generated captured image 50 to the information processing device 100.

[0054] Next, the first control unit 150 determines whether or not the captured image 50 has been received from the camera 300 (step S3). If the captured image 50 has not been received from the camera 300 (step S3 / NO), the first control unit 150 waits until the captured image 50 is received (step S4).

[0055] When the first control unit 150 receives the captured image 50 from the camera 300 (step S3 / YES), it detects the pattern area 30C, which is the area in which the pattern image 30B is captured, from the captured image 50 (step S4).

[0056] Based on the detection result of the pattern region 30C, the first control unit 150 determines whether or not two or more corners included in the pattern image 30B have been detected from the captured image 50 (step S5). If the first control unit 150 is unable to identify two or more corners contained in the pattern image 30B from the captured image 50 (step S5 / NO), it instructs the projector 200 to reduce the projection size of the pattern image 30B (step S6) and proceeds to processing of step S2.

[0057] Furthermore, when two or more corners included in pattern image 30B are detected from captured image 50 (step S5 / YES), first control unit 150 calculates the ratio between the number of panel pixels and the number of captured pixels (step S6). The number of captured pixels is the number of captured pixels of camera 300 used to capture the sides of pattern area 30C sandwiched between the two corners detected in step S4. The number of panel pixels is the number of panel pixels on the sides of liquid crystal panel 245 relative to the sides of pattern area 30C. Here, it is assumed that the number of captured pixels of camera 300 used to capture the top side 36C of pattern area 30C and the number of panel pixels on top side 32C of liquid crystal panel 245 have been calculated.

[0058] Next, the first control unit 150 calculates the ratio between the number of panel pixels on the upper side 32C of the liquid crystal panel 245 and the number of imaging pixels of the camera 300 used to image the upper side 36C of the pattern area 30C (step S7), where the first control unit 150 divides the number of panel pixels on the upper side 32C of the liquid crystal panel 245 by the number of imaging pixels of the camera 300 used to image the upper side 36C.

[0059] Next, the first control unit 150 determines the calculated ratio as the minimum width (step S8). Next, the first control unit 150 determines whether or not there is a Gray code pattern in which the stripe width is smaller than the determined minimum width among the Gray code patterns used in the Gray code method (step S8).

[0060] If there is a Gray code pattern whose stripe width is smaller than the determined minimum width (step S8 / YES), the first control unit 150 excludes the Gray code pattern whose stripe width is smaller than the minimum width from the objects to be projected onto the projection target 20 (step S9).

[0061] If there is no Gray code pattern whose stripe width is smaller than the determined minimum width (step S8 / NO), or if the first control unit 150 has excluded the Gray code pattern whose stripe width is smaller than the minimum width (step S9), the first control unit 150 proceeds to the next determination. If there is a Gray code pattern whose stripe width is larger than twice the minimum width (step S8 / YES), the first control unit 150 excludes this Gray code pattern whose stripe width is larger than twice the minimum width from the object to be projected onto the projection target 20 (step S11).

[0062] When the first control unit 150 excludes Gray code patterns whose stripe width is greater than twice the minimum width (step S11), or when there are no Gray code patterns whose stripe width is greater than twice the minimum width (step S10 / NO), it proceeds to the next process.

[0063] Next, the first control unit 150 causes the projector 200 to project the patterns excluding the excluded Gray code pattern onto the projection target 20, and causes the camera 300 to capture an image to obtain a captured image (step S12).

[0064] Next, the first control unit 150 executes processing based on the acquired captured image (step S13). Here, the first control unit 150 calculates a transformation matrix based on the captured image that transforms the imaging coordinates set in the captured image 50 captured by the camera 300 into the panel coordinates of the liquid crystal panel 245 of the projector 200. The first control unit 150 may also execute three-dimensional measurement based on the captured image.

[0065] [6. Modifications] [Variation 1] In the embodiment described above, the first control unit 150 causes the projector 200 to project the pattern image 30B in which the entire surface of the image is formed in a single color such as white or black. In Modification 1, processing according to the Gray code method is performed without projecting the pattern image 30B onto the projection target 20, and the line width of the Gray code pattern projected onto the projection target 20 is gradually changed from a thick pattern to a thin pattern. In other words, the pattern image 30B projected by the projector 200 onto the projection target 20 is a Gray code pattern having one or more stripe patterns. When the second control unit 250 draws the Gray code pattern over the entire surface of the liquid crystal panel 245, the first portion, that is, the top side 32C or the bottom side 32D of the liquid crystal panel 245, or the left side 32A or the right side 32B of the liquid crystal panel 245, includes multiple light or dark parts of the Gray code pattern.

[0066] Then, the first control unit 150 causes the camera 300 to capture an image of the Gray code pattern, and detects the number of imaging pixels of the camera 300 that captured the line width of the Gray code pattern drawn on the liquid crystal panel 245. In other words, the first control unit 150 calculates the ratio between the number of panel pixels of the liquid crystal panel 245 on which the line width of the Gray code pattern is drawn and the number of imaging pixels of the camera 300 that captured the line width of this Gray code pattern. The first control unit 150 narrows the line width of the Gray code pattern to be projected onto the projection target 20 until the ratio between the number of panel pixels of the liquid crystal panel 245 and the number of imaging pixels of the camera 300 reaches a predetermined ratio.

[0067] The first control unit 150 controls the projector 200 so that when the ratio of the number of panel pixels of the liquid crystal panel 245 to the number of imaging pixels of the camera 300 is 1:1, that is, when one pixel of the imaging pixels of the camera 300 captures one pixel of the gray code pattern drawn on the liquid crystal panel 245, the gray code pattern with a line width thinner than this is not projected onto the projection target 20.

[0068] [Variation 2] Furthermore, the first control unit 150 may change the content of the message displayed on the display unit 130 depending on the minimum width. For example, if the camera 300 is capable of changing the resolution of the image sensor, the first control unit 150 may instruct the camera 300 to change the resolution. For example, if there is a Gray code pattern whose stripe width is smaller than the minimum width among the Gray code patterns stored in advance in the first storage unit 160, the first control unit 150 may cause the display unit 130 to display a message instructing the camera 300 to increase the resolution. Furthermore, the first control unit 150 may instruct the camera 300 to increase the resolution. Furthermore, if there is no Gray code pattern that is wider than twice the determined minimum width, the first control unit 150 may cause the display unit 130 to display a message instructing the camera 300 to lower the resolution. Alternatively, the first control unit 150 may instruct the camera 300 to lower the resolution.

[0069] [Variation 3] FIG. 8 is a diagram showing a pattern image 30B projected onto the projection target 20. As shown in FIG. FIG. 9 is a diagram showing a captured image 50 captured by the camera 300. As shown in FIG. If the shape of the pattern image 30B projected onto the projection target 20 by the projector 200 is distorted, the shape of the pattern image 30B captured in the captured image 50 will also be distorted.

[0070] As shown in Fig. 4, when the lengths of the top side 36C and the bottom side 36D of the pattern image 30B are substantially the same, the first control unit 150 may use either the top side 36C or the bottom side 36D as the number of pixels of the imaging element that captured the pattern image 30B. However, as shown in Fig. 9, when the lengths of the top side 36C and the bottom side 36D are different, the first control unit 150 detects the number of pixels of the top side 36C, which is the shortest side of the top side 36C and the bottom side 36D, as the number of captured pixels. The minimum limit value of the minimum width can be determined by calculating the minimum number of captured pixels and determining the minimum width based on the calculated minimum value.

[0071] [Variation 4] In the above-described embodiment, when the stripe width of a Gray code pattern is smaller than the minimum width, the Gray code pattern having a stripe width smaller than this minimum width is excluded from the objects to be projected onto the projection target 20. However, since the resolution of the camera 300 can be changed by changing the distance between the camera 300 and the projection target 20, if there is a gray code pattern with a stripe width smaller than the minimum width, a message guiding the user to reduce the distance between the camera 300 and the projection target 20 may be displayed on the display unit 130.

[0072] Furthermore, when there is a Gray code pattern that is wider than twice the minimum width, the first control unit 150 may cause the display unit 130 to display a message instructing the user to increase the distance between the camera 300 and the projection target 20.

[0073] [Variation 5] In the above-described embodiment, when the stripe width of a Gray code pattern is smaller than the minimum width, the Gray code pattern having a stripe width smaller than this minimum width is excluded from the objects to be projected onto the projection target 20. Alternatively, the first control unit 150 may perform linear interpolation on the captured image received from the camera 300, and interpolate the pixel values of pixels where an erroneous pixel value has been detected.

[0074] 7. Other Embodiments The above-described embodiment and each modification are preferred embodiments of the present invention, but the present invention is not limited to these and various modifications are possible within the scope of the gist of the present invention. For example, in the above-described embodiment, the projection system 1 includes the information processing device 100, the projector 200, and the camera 300. However, the present invention is not limited to this. For example, the camera 300 may be a built-in camera of the information processing device 100 or the projector 200.

[0075] Furthermore, in the above-described embodiment, the case where the information processing device 100 and the projector 200 are configured as separate entities has been described, but it is also possible to operate the projector 200 as the information processing device 100. In other words, the second control unit 250 of the projector 200 may be configured to execute the operation of the information processing device 100 shown in the flowchart of FIG.

[0076] 7 are divided according to the main processing content in order to facilitate understanding of the processing of the information processing device 100. The present invention is not limited by the manner in which the processing units are divided or the names of the processing units shown in the flowchart of FIG. 7. Furthermore, the processing of the information processing device 100 can be divided into more processing units according to the processing content, or one processing unit can be divided so as to include more processes. Furthermore, the processing order of the above flowchart is not limited to the example shown in the figure. The same applies to the processing unit of the projector 200 shown in FIG.

[0077] 1 indicates a functional configuration realized by cooperation between hardware and software, and the specific implementation form is not particularly limited. Therefore, it is not necessarily necessary to implement hardware corresponding to each functional unit individually, and it is of course possible to implement a configuration in which a single processor executes a program to realize the functions of multiple functional units. Furthermore, some of the functions realized by software in the above embodiments may be realized by hardware, or some of the functions realized by hardware may be realized by software.

[0078] Furthermore, when the control method and program of the present disclosure are implemented using a computer installed in the information processing device 100 or a computer installed in the projector 200, the program executed by the computer can be configured in the form of a recording medium. The program executed by the computer can also be configured in the form of a transmission medium for transmitting the program. The recording medium can be a magnetic or optical recording medium or a semiconductor memory device. Specific examples include portable or fixed recording media such as flexible disks, HDDs, CD-ROMs (Compact Disc Read-Only Memory), DVDs (Digital Versatile Discs), Blu-ray Discs, magneto-optical disks, flash memories, and card-type recording media. The recording medium can also be a non-volatile storage device such as RAM, ROM, or HDD, which is an internal storage device of a server device. Blu-ray is a registered trademark.

[0079] 8. Summary of the Disclosure A summary of this disclosure is provided below.

[0080] (Appendix 1) A control method for a projection system comprising: a projection device including an optical modulator having a plurality of pixels, which projects one or more stripe patterns including bright and dark areas onto a projection target; and an imaging device which captures the one or more stripe patterns projected onto the projection target, the control method comprising the steps of: the optical modulator drawing a drawn image including a first portion having a first number of pixels; the projection device projecting the drawn image onto the projection target as a projected image; the imaging device capturing the projected image having a second portion corresponding to the first portion to obtain a captured image; and determining the minimum width of the bright or dark area that can be detected by the imaging device based on the first number of pixels and a second number of pixels which is the number of pixels of a third portion of the projected image included in the captured image that corresponds to the second portion.

[0081] According to this method, the imaging device captures a projection image projected by the projection device onto a projection target to obtain the captured image, and determines the minimum width of a bright or dark area that the imaging device can detect based on the first number of pixels and the second number of pixels of the projection image included in the captured image. Therefore, even if a user provides the imaging device, the minimum width of a bright or dark area that the imaging device can detect can be determined based on the first number of pixels and the second number of pixels. This reduces the processing burden on the user. Furthermore, when determining the correspondence between the pixels of the optical modulator and the pixels of the imaging device, processing time can be reduced by not projecting a stripe pattern onto the projection target that cannot be determined from the captured image of the imaging device.

[0082] (Appendix 2) 2. The control method of claim 1, wherein the projected image is a first image having only a first color and different from the one or more stripe patterns.

[0083] According to this, the first image having only the first color is projected as the projected image, which makes it easy to determine the minimum width of the bright or dark area.

[0084] (Appendix 3) The control method described in Appendix 2, wherein the first image included in the captured image comprises a plurality of sides that form the outline of the first image and are included in the captured image, and the third part is a first side that is the shortest side of the plurality of sides.

[0085] According to this, the minimum width is determined based on the first number of pixels and the second number of pixels of the third portion, which is the first side, which is the shortest side of the multiple sides of the first image. In other words, since the minimum width is determined based on the minimum value of the second number of pixels, it is possible to determine the minimum limit value of the minimum width.

[0086] (Appendix 4) 4. The control method of claim 3, wherein the first image includes a first corner, a second corner different from the first corner, and a side that passes through the first corner and the second corner and forms a contour of the first image, and the third part is the side, and the control method includes determining whether the captured image includes the first corner and the second corner, and if it is determined that the captured image includes the first corner or the second corner, changing the size of the first image so that both the first corner and the second corner are included in the captured image.

[0087] According to this, the size of the first image is changed so that the first corner, the second corner, and the sides that pass through the first and second corners and form the outline of the first image are captured in the projected image. Therefore, the size of the first image can be changed so that the minimum width of the light or dark area can be determined based on the first pixel count and the second pixel count.

[0088] (Appendix 5) 2. The control method according to claim 1, wherein the projected image is the one or more stripe patterns, and the first portion includes a light portion or a dark portion.

[0089] According to this method, one or more stripe patterns are projected as the projection image, and the minimum width is determined based on the number of pixels in the light or dark areas included in the first portion and the number of pixels in the third portion, which is an image of the second portion of the projection image of the first portion. Therefore, one or more stripe patterns can be used as the projection image for determining the minimum width.

[0090] (Appendix 6) 2. The control method of claim 1, further comprising: displaying a first message when the stripe pattern having the smallest width among the plurality of stripe patterns, the stripe pattern having the smallest width of the light or dark portions of the first stripe pattern, is defined as the first stripe pattern, and the smallest width is smaller than the width of the light or dark portions of the first stripe pattern.

[0091] According to this, if the minimum width is smaller than the width of the light or dark portion of the first stripe pattern, the first message is displayed. This makes it possible to prevent a stripe pattern that does not have the required correspondence between the pixels of the optical modulator and the pixels of the imaging device from being projected onto the projection target.

[0092] (Appendix 7) 7. The control method of claim 6, further comprising: modifying content of the first message according to the minimum width.

[0093] According to this, the content of the first message is changed depending on the minimum width, which allows the user to understand the stripe patterns that can be detected in the image captured by the imaging device.

[0094] (Appendix 8) 2. The control method according to claim 1, further comprising changing the number of the plurality of stripe patterns projected from the projection device according to the result of determining the minimum width.

[0095] According to this, the number of stripe patterns projected from the projection device is changed depending on the result of determining the minimum width, so that a stripe pattern corresponding to the minimum width detectable in the image captured by the imaging device can be projected.

[0096] (Appendix 9) a projection device including an optical modulator including a plurality of pixels, which projects one or more stripe patterns including bright and dark areas onto a projection target; and an imaging device which captures the one or more stripe patterns projected onto the projection target, wherein the optical modulator draws a drawn image including a first portion having a first number of pixels; the projection device projects the drawn image onto the projection target as a projected image; the imaging device captures the projected image having a second portion corresponding to the first portion to obtain a captured image; and determining the minimum width of the bright or dark area that can be detected by the imaging device based on the first number of pixels and a second number of pixels which is the number of pixels of a third portion of the projected image included in the captured image that corresponds to the second portion.

[0097] According to this method, the imaging device captures a projection image projected by the projection device onto a projection target to obtain the captured image, and determines the minimum width of a bright or dark area that the imaging device can detect based on the first number of pixels and the second number of pixels of the projection image included in the captured image. Therefore, even if a user provides the imaging device, the minimum width of a bright or dark area that the imaging device can detect can be determined based on the first number of pixels and the second number of pixels. This reduces the processing burden on the user. Furthermore, when determining the correspondence between the pixels of the optical modulator and the pixels of the imaging device, processing time can be reduced by not projecting a stripe pattern onto the projection target that cannot be determined from the captured image of the imaging device.

[0098] (Appendix 10) A program in which a processor mounted on a projection device including an optical modulator including a plurality of pixels and which projects one or more stripe patterns including bright and dark areas onto a projection target executes the following: causing the optical modulator to draw a drawn image including a first portion having a first number of pixels; projecting the drawn image onto the projection target as a projected image; obtaining from an imaging device a captured image that captures the projected image having a second portion corresponding to the first portion; and determining the minimum width of the bright or dark area that can be detected by the imaging device based on the first number of pixels and a second number of pixels that is the number of pixels of a third portion of the projected image included in the captured image that corresponds to the second portion.

[0099] According to this method, the imaging device captures a projection image projected by the projection device onto a projection target to obtain the captured image, and determines the minimum width of a bright or dark area that the imaging device can detect based on the first number of pixels and the second number of pixels of the projection image included in the captured image. Therefore, even if a user provides the imaging device, the minimum width of a bright or dark area that the imaging device can detect can be determined based on the first number of pixels and the second number of pixels. This reduces the processing burden on the user. Furthermore, when determining the correspondence between the pixels of the optical modulator and the pixels of the imaging device, processing time can be reduced by not projecting a stripe pattern onto the projection target that cannot be determined from the captured image of the imaging device. [Explanation of symbols]

[0100] 1...projection system, 5...remote control, 20...projection target, 30A...drawing image, 30B...pattern image, 30C...pattern area, 31A...upper left vertex, 31B...upper right vertex, 31C...lower left vertex, 31D...lower right vertex, 32A...left side, 32B...right side, 32C...upper side, 32D...lower side, 33...projected image, 34A...left side, 34B...right side, 34C...upper side, 34D...lower side, 35A...upper left vertex, 35B...upper right vertex, 35C...lower left vertex, 35D...lower right vertex, 36A...left side, 36B...right side, 36C...upper side, 36D...lower side, 50...captured image, 10 0...information processing device, 110...first I / F, 120...operation unit, 150...first control unit, 160...first memory unit, 161...control program, 170...first processor, 200...projector, 210...remote control light receiving unit, 220...second I / F, 231...image processing unit, 233...frame memory, 240...projection unit, 241...light source, 243...light modulation device, 245...liquid crystal panel, 247...optical unit, 250...second control unit, 260...second memory unit, 261...control program, 270...second processor, 300...camera.

Claims

1. A control method for a projection system including a projection device that includes a light modulator including a plurality of pixels and projects one or more stripe patterns including light and dark areas onto a projection target, and an imaging device that images the one or more stripe patterns projected onto the projection target, comprising: the light modulator rendering a rendered image including a first portion having a first number of pixels; the projection device projects the drawn image onto the projection target as a projected image; the imaging device captures the projection image having a second portion corresponding to the first portion to obtain a captured image; determining a minimum width of the bright portion or the dark portion that can be detected by the imaging device based on the first pixel number and a second pixel number that is the number of pixels of a third portion of the projection image included in the captured image and that corresponds to the second portion; A control method comprising:

2. The control method of claim 1 , wherein the projected image is a first image having only a first color and different from the one or more stripe patterns.

3. the first image included in the captured image includes a plurality of sides that form a contour of the first image and are included in the captured image, The control method according to claim 2 , wherein the third portion is a first side that is the shortest side of the plurality of sides.

4. the first image includes a first corner, a second corner different from the first corner, and a side that passes through the first corner and the second corner and forms an outline of the first image; the third portion is the side, The control method includes: determining whether the captured image includes the first corner and the second corner; The control method of claim 3, further comprising: if it is determined that the captured image includes the first corner or the second corner, changing the size of the first image so that both the first corner and the second corner are included in the captured image.

5. the projected image is the one or more stripe patterns, The control method according to claim 1 , wherein the first portion includes a light portion or a dark portion.

6. When the stripe pattern of the bright portion or the dark portion having the smallest width among the plurality of stripe patterns is defined as a first stripe pattern, The control method according to claim 1 , further comprising displaying a first message when the minimum width is smaller than the width of the light portion or the dark portion of the first stripe pattern.

7. The control method according to claim 6 , further comprising: modifying content of the first message in accordance with the minimum width.

8. 2. The control method according to claim 1, wherein the number of the plurality of stripe patterns projected from the projection device is changed according to the result of determining the minimum width.

9. a projection device including a light modulator including a plurality of pixels, which projects one or more stripe patterns including light and dark areas onto a projection target; an imaging device that captures an image of the one or more stripe patterns projected onto the projection target, the light modulator rendering a rendered image including a first portion having a first number of pixels; the projection device projects the drawn image onto the projection target as a projected image; the imaging device captures the projection image having a second portion corresponding to the first portion to obtain a captured image; determining a minimum width of the bright portion or the dark portion that can be detected by the imaging device based on the first pixel number and a second pixel number that is the number of pixels of a third portion of the projection image included in the captured image and that corresponds to the second portion; a projection system including:

10. A processor mounted in a projection device that includes a light modulator including a plurality of pixels and projects one or more stripe patterns including light and dark areas onto a projection target, causing the light modulator to render a rendered image including a first portion having a first number of pixels; projecting the drawn image onto the projection target as a projection image; acquiring, from an imaging device, a captured image obtained by capturing the projected image having a second portion corresponding to the first portion; determining a minimum width of the bright portion or the dark portion that can be detected by the imaging device based on the first pixel number and a second pixel number that is the number of pixels of a third portion of the projection image included in the captured image and that corresponds to the second portion; Run the program.

Images