Control method, program, and projector

Abstract

To accurately perform video distortion correction in a multi-projection system.SOLUTION: A second position of a projection image by a projector 10B corresponding to a first position of a projection image by a projector 10A is identified on the basis of an imaging result of first structured light projected from the projector 10A to a first projection range and an imaging result of second structured light projected from the projector 10B to a second projection range. A geometric correction value of a composite image is identified on the basis of an imaging result of third structured light projected from the projector 10A to a third projection range, which is a range smaller than the first projection range and to which a part of a content is to be projected, and an imaging result of fourth structured light projected from the projector 10B to a fourth projection range, which is a range smaller than the second projection range and to which another part of the content is to be projected.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a control method, a program, and a projector. [Background technology]

[0002] One example of a display device for displaying an image is a projector equipped with a projection unit that projects an image onto a projection target such as a screen. Depending on the relative positional relationship between the projector and the projection target, distortions such as trapezoidal distortion may occur in the image displayed on the projection target. To avoid such distortions, projectors may apply various corrections to the image on the projection target, such as geometric correction, to adjust the display aspect. Various technologies for accurately performing such corrections have been proposed, such as the technology disclosed in Patent Document 1. Patent Document 1 discloses an invention related to a projection control device that controls a projection unit that projects an image onto a screen. This projection control device includes a specifying unit and a projection control unit. The specifying unit specifies an image area. The image area is an area on the screen where image information projected by the projection unit is displayed. The projection control unit projects an adjustment pattern onto the image area specified by the specifying unit. The adjustment pattern is an adjustment pattern for obtaining adjustment values used to adjust the display aspect of the image information, and is generated to be projected onto the image area. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-77773 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, multi-projection systems that combine images projected by multiple projectors horizontally or vertically to create a larger image have become popular. In multi-projection systems, simply projecting an adjustment pattern onto a specified image area, as in the technology disclosed in Patent Document 1, does not allow for the determination of correspondence between the multiple projected images, and problems such as an inability to accurately correct distortion may arise. [Means for solving the problem]

[0005] One aspect of a control method of the present disclosure is a control method for a multi-projection system including a first projector that projects a first projection image into a first projection range, and a second projector that projects a second projection image that forms content by combining the first projection image into a second projection range that at least partially overlaps the first projection range, the control method including: projecting first structured light into the first projection range; capturing an image of the first structured light projected into the first projection range; projecting second structured light into the second projection range; capturing an image of the second structured light projected into the second projection range; and capturing the first projection image based on the captured images of the first structured light and the second structured light. projecting third structured light onto a third projection range that is smaller than the first projection range and onto which a portion of the content is projected; capturing an image of the third structured light projected onto the third projection range; projecting fourth structured light onto a fourth projection range that is smaller than the second projection range and onto which another portion of the content is projected; capturing an image of the fourth structured light projected onto the fourth projection range; and determining a geometric correction value of a composite image of the first projection image and the second projection image based on the capturing results of the third structured light and the fourth structured light.

[0006] A program according to an aspect of the present disclosure includes a program that causes a computer to: project first structured light onto a first projection range from a first projector that projects a first projection image onto the first projection range; capture an image of the first structured light projected onto the first projection range; project second structured light onto a second projection range that at least partially overlaps with the first projection range to form a second projection image that is combined with the first projection image to form content; capture an image of the second structured light projected onto the second projection range; and capture an image of the second structured light projected onto the second projection range based on the results of capturing the first structured light and the second structured light. projecting third structured light from the first projector onto a third projection range that is smaller than the first projection range and onto which a portion of the content is projected, capturing an image of the third structured light projected onto the third projection range, projecting fourth structured light from the second projector onto a fourth projection range that is smaller than the second projection range and onto which another portion of the content is projected, capturing an image of the fourth structured light projected onto the fourth projection range, and determining a geometric correction value of a composite image of the first projected image and the second projected image based on the captured images of the third structured light and the fourth structured light. According to this aspect, highly accurate correction can be performed even when projection extends beyond the frame.

[0007] Also, a projector according to one aspect of the present disclosure includes: a projection device that projects a first projection image into a first projection range; a communication device that communicates with another projector that projects a second projection image that forms content by combining the first projection image into a second projection range that at least partially overlaps the first projection range; and a processing device, wherein the processing device projects first structured light from the projection device into the first projection range, captures an image of the first structured light projected into the first projection range, projects second structured light from the other projector into the second projection range, captures an image of the second structured light projected into the second projection range, and generates the first projection image based on the captured image of the first structured light and the second structured light. projecting third structured light from the projection device onto a third projection range that is smaller than the first projection range and onto which a portion of the content is projected, capturing an image of the third structured light projected onto the third projection range; projecting fourth structured light from the other projector onto a fourth projection range that is smaller than the second projection range and onto which another portion of the content is projected, capturing an image of the fourth structured light projected onto the fourth projection range; and determining a geometric correction value of a composite image of the first projection image and the second projection image based on the results of capturing the third structured light and the fourth structured light. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an example of the configuration of a multi-projection system 1 according to an embodiment of the present disclosure. [Figure 2] FIG. 1 is a diagram illustrating an example of the configuration of a projector 10. [Figure 3] FIG. 2 is a diagram illustrating an example of the configuration of a control device 20. [Figure 4] FIG. 10 is a diagram illustrating a UI screen for adjusting the shape of a projection image. [Figure 5] 10 is a flowchart showing the flow of processing in a control method executed by a processing device 210 of a control device 20 in accordance with a program P2. [Figure 6] FIG. 10 is a diagram for explaining frame overflow. [Figure 7] 10A and 10B are diagrams for explaining structured light projected by the projector 10 when frame overflow occurs. [Figure 8] FIG. 10 is a diagram for explaining Modification 4. DETAILED DESCRIPTION OF THE INVENTION

[0009] The embodiments described below are subject to various technically preferable limitations, but the embodiments of the present disclosure are not limited to the following embodiments. Embodiment Fig. 1 is a diagram showing an example configuration of a multi-projection system 1 according to an embodiment of the present disclosure. As shown in Fig. 1, the multi-projection system 1 includes a projector 10A and a projector 10B that each project an image onto a projection target SC, and a control device 20 that communicates with the projector 10A and the projector 10B via a network NW. Specific examples of the network NW include a wired or wireless local area network (LAN), a wired or wireless USB, or Bluetooth (registered trademark).

[0010] In this embodiment, the projection target SC is, for example, one of the interior walls of a room in which a user of the multi-projection system 1 lives. The image GC displayed on the projection target SC by the multi-projection system 1 is obtained by combining an image GA projected onto the projection target SC by the projector 10A and an image GB projected onto the projection target SC by the projector 10B. The arrow z in FIG. 1 represents the vertical direction, and the arrow x represents one of two directions perpendicular to the vertical direction (hereinafter referred to as the horizontal direction). Each of the projectors 10A and 10B has a generation unit (described below) such as a liquid crystal panel.

[0011] As shown in FIG. 1 , in this embodiment, an image GC corresponding to the entire content is synthesized by having projector 10A project an image representing a portion of the content and having projector 10B project an image representing another portion of the content so that the projection range of the image by projector 10A and the projection range of the image by projector 10B partially overlap in the horizontal direction. That is, a portion of the projection range of the image by projector 10A overlaps a portion of the projection range of the image by projector 10B in the horizontal direction. In this embodiment, image GC is a so-called tiling image. Note that image GC may also be a stacking image in which the entire projection range of the image by projector 10A overlaps the entire projection range of the image by projector 10B. Projector 10A is an example of a first projector in the present disclosure. Projector 10B is an example of a second projector in the present disclosure.

[0012] In this embodiment, the configuration of projector 10A is the same as the configuration of projector 10B. Hereinafter, when there is no need to distinguish between projector 10A and projector 10B, projector 10A and projector 10B will be referred to as "projector 10."

[0013] The wall surface of a room is not flat like a screen, and distortion may occur in the image displayed on the wall surface by the projector 10 depending on depressions or protrusions in the wall surface. Also, if the projection lens of the projector 10 is not directly facing the wall surface, distortion will occur in the image displayed on the wall surface by the projector 10. In this embodiment, a correction value for correcting the image distortion is calculated using a measurement pattern such as a Gaussian dot pattern or a phase shift pattern, as in the conventional case.

[0014] Fig. 2 is a diagram showing an example of the configuration of the projector 10. As shown in Fig. 2, the projector 10 includes a processing device 110, a communication device 120, an input device 130, a projection device 140, an imaging device 150, and a storage device 160. Each of the communication device 120, the input device 130, the projection device 140, the imaging device 150, and the storage device 160 is connected to the processing device 110 via a bus (not shown in Fig. 2).

[0015] The processing device 110 is, for example, a computer such as a CPU (Central Processing Unit). As will be described in detail later, the processing device 110 functions as the control center of the projector 10 by operating in accordance with a program P1 pre-stored in the storage device 160.

[0016] The communication device 120 is a device that communicates with other devices via the network NW, and includes, for example, an interface circuit. A specific example of another device that communicates with the communication device 120 is the control device 20. The communication device 120 provides data received from the network NW to the processing device 110.

[0017] Image data representing an image to be projected by the projector 10 onto the projection target SC is input from the outside to the input device 130. In this embodiment, the input device 130 is an HDMI interface, but it may also be a VGA interface. Video data may also be input to the projector 10 via a network NW, in which case the communication device 120 may also serve as the input device 130. Note that HDMI is a registered trademark.

[0018] The projection device 140 includes a generation unit that generates image light according to image data, and an optical system that guides the image light generated by the generation unit to a projection target SC and realizes optical zoom and lens shift. The generation unit and the optical system are not shown in FIG. 1. Specific examples of the generation unit include a light modulation element such as an LCD (Liquid Crystal Display), an LCOS (Liquid Crystal on Silicon), or a DMD (Digital Mirror Device). The projection device 140 also includes an image processing unit 140a, such as an image processing circuit, that adjusts the shape of an image represented by the video data based on a correction value calculated by a calculation unit 110b (described later).

[0019] The imaging device 150 includes a CMOS or CCD image sensor. In this embodiment, the imaging area of the imaging device 150 is set in advance so as to cover the entire projection area onto which the image is projected by the projection device 140, but the imaging area may also be set so as to capture an image of at least a portion of the projection area. In this embodiment, as will be described in detail later, structured light such as a Gaussian dot pattern or a phase shift pattern is projected from the projector 10 onto the projection target SC in order to calculate a correction value for distortion correction, and the imaging device 150 is used to capture an image of the projection target SC onto which the structured light is projected. The type of structured light is not particularly limited as long as it can at least achieve the purpose of calculating the correction value for distortion correction.

[0020] The storage device 160 includes a nonvolatile memory such as a flash ROM (Read Only Memory) and a volatile memory such as a RAM (Random Access Memory). The nonvolatile memory of the storage device 160 stores a program P1 that causes the processing device 110 to function as the control center of the projector 10. The nonvolatile memory of the storage device 160 is used by the processing device 110 as a work area when executing the program P1. When the power (not shown in FIG. 2) of the projector 10 is turned on, the processing device 110 reads the program P1 from the nonvolatile memory to the volatile memory and starts executing the program P1 read into the volatile memory.

[0021] The processing device 110, operating in accordance with the program P1, functions as a pattern analysis unit 110a, a calculation unit 110b, and a shape adjustment unit 110c. The pattern analysis unit 110a analyzes the image capture results from the image capture device 150 to associate positions in a camera coordinate system that defines positions on the captured image with positions in a panel coordinate system that defines positions on the image drawn by the generation unit. This association can be achieved using any known technology. The calculation unit 110b calculates a correction value for distortion correction based on the analysis results from the pattern analysis unit 110a. The shape adjustment unit 110c causes the projection device 140 to project a UI image for adjusting the shape of the projected image. Details of this UI image will be explained later.

[0022] Fig. 3 is a diagram showing an example configuration of the control device 20. The control device 20 is, for example, a personal computer. As shown in Fig. 3, the control device 20 includes a processing device 210, a communication device 220, a display device 230, an input device 240, and a storage device 250. Each of the communication device 220, the display device 230, the input device 240, and the storage device 250 is connected to the processing device 210 via a bus (not shown in Fig. 3).

[0023] The processing device 210 is a computer such as a CPU, similar to the processing device 110. The processing device 210 executes various programs stored in the storage device 250.

[0024] The communication device 220 includes an interface circuit, similar to the communication device 120. A specific example of another device that communicates with the communication device 220 is the projector 10. The communication device 220 provides the processing device 110 with data received from the network NW.

[0025] The display device 230 includes, for example, a liquid crystal display and its driving circuit (both of which are not shown in FIG. 3). The driving circuit includes a video memory in which the image to be displayed on the liquid crystal display is drawn by the processing device 210.

[0026] The input device 240 includes a keyboard with multiple operators and a pointing device such as a mouse. The input device 240 receives user operations such as pressing operators, and provides operation content data representing the received operations to the processing device 210. In this way, the user operation content is transmitted to the processing device 210.

[0027] The storage device 250 includes a nonvolatile memory such as a hard disk and a volatile memory such as RAM. Various programs are stored in the nonvolatile memory of the storage device 250. Examples of programs stored in the nonvolatile memory of the storage device 250 include a kernel program (not shown in FIG. 3) for implementing an OS (Operating System) and a program P2 for implementing a control method that prominently demonstrates the features of the present disclosure. The volatile memory of the storage device 250 is used as a work area when these programs are executed.

[0028] When the power supply (not shown in FIG. 3) of the control device 20 is turned on, the processing device 210 reads the kernel program from the nonvolatile memory to the volatile memory and starts executing the kernel program read into the volatile memory. The processing device 210, operating in accordance with the kernel program, can read a program that is instructed to be executed by an operation on the input device 240 from the nonvolatile memory to the volatile memory and execute it. For example, when an operation on the input device 240 instructs execution of a program P2, the processing device 210 reads the program P2 from the nonvolatile memory to the volatile memory and starts executing the program P2 that has been read into the volatile memory.

[0029] The processing device 210, operating in accordance with the program P2, functions as an image generation unit 210a and a shape adjustment unit 210b. The image generation unit 210a generates an image of a UI screen (hereinafter, UI image) that prompts an operation for executing the control method of the present disclosure, and displays the UI image on the display device 230. An example of this UI image is an image GU in which cross anchors M1 to M6 are evenly arranged along the periphery of a content image, as shown in FIG. 4. The user can instruct the control device 20 to adjust the shape of the content image displayed on the projection target SC by performing an operation to change the positions of the cross anchors M1 to M6 in the image GUA displayed on the display device 230. The image GUA is an example of a first adjustment image for adjusting the first projection image of the present disclosure. The cross anchors M1 to M6 are an example of a plurality of adjustment points of the present disclosure. The cross anchors M1, M2, M4, and M5 included in the image GUA are an example of a plurality of first adjustment points of the present disclosure. The cross anchors M3 and M6 included in the image GUB are an example of the plurality of second adjustment points in the present disclosure. The shape of each of the cross anchors M1 to M6 is not limited to a cross, but may be a circle, a dot, or a polygon including a rectangle.

[0030] The shape adjustment unit 210b adjusts the shape of the content image displayed on the projection target SC in accordance with a user instruction. More specifically, the shape adjustment unit 210b transmits position information indicating the positions of each of the cross anchors M1 to M6 designated by an operation on the image GU to the projector 10 using the communication device 220. The projector 10 displays the cross anchors M1 to M6 on the projection target SC in accordance with the position information. Here, as shown in FIG. 4, the projector 10A displays a UI image GUA in which the cross anchors M1, M2, M4, and M5 are arranged in accordance with the position information. On the other hand, as shown in FIG. 4, the projector 10B displays a UI image GUB in which the cross anchors M3 and M6 are arranged in accordance with the position information. Therefore, in the overlapping region where the projection ranges of the projectors 10A and 10B overlap, the cross anchors M3 and M5 are displayed only by the projector 10A. This is to eliminate redundant adjustments by the user. The user adjusts the positions of the cross anchors M3 and M5 by operating the UI image GUB based on the UI image GUA, thereby roughly adjusting the overall shape of the image GU. The image GUB is an example of a second adjustment image for adjusting the second projection image in this disclosure.

[0031] Furthermore, the shape adjustment unit 210b executes a control method that prominently exhibits the features of the present disclosure. Fig. 5 is a flowchart showing the flow of processing in this control method. As shown in Fig. 5, this control method includes a first detection process SA110, a first calculation process SA120, a determination process SA130, a second detection process SA140, a second calculation process SA150, and a reflection process SA160.

[0032] In the first detection process SA110, the shape adjustment unit 210b causes each of the projectors 10A and 10B to project the above-described structured light and causes the image capture device 150 to capture the structured light, thereby establishing a correspondence between the camera coordinate system and the panel coordinate system in each projector. For example, in the first detection process SA110, the shape adjustment unit 210b establishes a correspondence between the camera coordinate system and the panel coordinate system in the following procedure. First, structured light is projected only from the projector 10A. Next, the image capture device 150 captures the structured light projected only from the projector 10A. At this timing, a correspondence (first correspondence) between the panel coordinate system of the projector 10A and the camera coordinate system of the image capture device 150 may be generated. Next, structured light is projected only from the projector 10B. Next, the image capture device 150 captures at least a portion of the structured light projected only from the projector 10B that overlaps with the image GA. At this timing, a correspondence relationship (second correspondence relationship) between the panel coordinate system of projector 10B and the camera coordinate system of image capture device 150 may be generated. The projection range onto which projector 10A projects structured light in first detection processing SA110 is an example of a first projection range in the present disclosure, and the structured light is an example of the first structured light. The projection range onto which projector 10B projects structured light in first detection processing SA110 is an example of a second projection range in the present disclosure, and the structured light is an example of the second structured light.

[0033] In the first calculation process SA120, the shape adjustment unit 210b connects the cross anchors M1 to M6 based on the correspondence between the camera coordinate system and the panel coordinate system of each of the projectors 10A and 10B, and the positions of the cross anchors M1 to M6. At this timing, one of the projectors 10A and 10B can grasp the position of the cross anchor displayed by the other of the projectors 10A and 10B. That is, the projector 10B can identify the position in the panel coordinate system of the projector 10B of each of the cross anchors M2 and M5, in other words, the cross anchor located at the right end in the image projected by the projector 10A. The positions of the cross anchors M2 and M5 in the image projected by the projector 10A are an example of a first position in the present disclosure, and the positions of the cross anchors M2 and M5 in the panel coordinate system of the projector 10B are an example of a second position in the present disclosure.

[0034] That is, in the first calculation process SA120, for example, based on the first correspondence relationship and the second correspondence relationship calculated in the first detection process SA110, a correspondence relationship (panel correspondence relationship) between the panel coordinate system of projector 10A and the panel coordinate system of projector 10B is calculated. Then, based on this correspondence relationship, the shape adjustment unit 210b specifies which positions of the cross anchors M2 and M5 in the image projected by projector 10A (first adjusted image) correspond to which positions in the image projected by projector 10B (second adjusted image). Note that when image GC is a stacking image, all parts of image GB are manually adjusted by the user based on all parts of image GA, so it is not necessarily necessary to generate a panel correspondence relationship.

[0035] Furthermore, in the first calculation process SA120, shape adjustment unit 210b may also calculate a correction value for ensuring linearity within the composite image. The timing at which the correction value is calculated is not particularly limited as long as it is in the first calculation process SA120 or a step after the first calculation process SA120.

[0036] In the determination process SA130, the shape adjustment unit 210b determines whether or not the projection is an out-of-frame projection. Out-of-frame projection refers to projecting an image beyond the projection target SC, such as onto the ceiling or side. In a usage scenario in which a composite image is displayed across the entire projection target SC, projecting an image beyond the ceiling or side may be necessary depending on the installation conditions of the projector 10 relative to the projection target SC. FIG. 6 shows an example of an image GA projected beyond the ceiling and side. When an image is projected beyond the ceiling or side, secondary reflection light, which is the image light reflected from the ceiling or side, is added to the structured light as noise, which may cause errors in identifying the correspondence and reduce the accuracy of overlaying the projected images. The diagonal hatching in FIG. 6 represents noise due to the secondary reflection light. In this embodiment, the shape adjustment unit 210b displays a message on the display device 230 inquiring the user about whether or not the projection is an out-of-frame projection, and determines whether or not the projection is an out-of-frame projection based on the user's response to the message. The shape adjustment unit 210b displays a message inquiring whether or not the projection is outside the frame on the display device 230, for example, before the first detection process SA110, and accepts a user's response. The shape adjustment unit 210b stores the content of the response accepted before the first detection process SA110 in the storage device 250. In the determination process SA130, the shape adjustment unit 210b retrieves the content of the response stored in the storage device 250 and judges whether or not the projection is outside the frame. At least one of the timing of displaying the message and the timing of accepting the user's response is not particularly limited, as long as it is a step before the determination process SA130. In other words, the timing of accepting the user's response before the determination process SA130 may be different from or the same as the timing of displaying the message.

[0037] If the determination result of the determination process SA130 is "Yes", i.e., if the projection is outside the frame, the shape adjustment unit 210b executes the second detection process SA140 and the second calculation process SA150, and then executes the reflection process SA160. If the determination result of the determination process SA130 is "No", i.e., if the projection is not outside the frame, the shape adjustment unit 210b executes the reflection process SA160 without executing the second detection process SA140 and the second calculation process SA150.

[0038] In the second detection process SA140, the shape adjustment unit 210b causes each of the projectors 10A and 10B to project structured light in a projection range narrower than the projection range of the structured light in the first detection process SA110 and to capture the structured light, thereby causing each of the projectors to again associate the camera coordinate system with the panel coordinate system. In this embodiment, the projector 10A projects structured light with a mask whose four corners are defined by cross anchors M1, M2, M4, and M5 based on the correction value calculated in the first calculation process SA120. The projector 10B projects structured light with a mask whose four corners are defined by cross anchors M2, M3, M5, and M6 based on the correction value calculated in the first calculation process SA120. That is, in this embodiment, masks are set for the image GUA projected from the projector 10A and the image GUB projected from the projector 10B based on the positions of multiple adjustment points.

[0039] In this embodiment, as shown in FIG. 7, projector 10 draws an image GP1 representing structured light on a first layer in a video memory, and draws a mask GM defined by cross anchors on a second layer overlaid on the first layer, thereby generating a masked structured light image GP2. However, image GP2 may also be generated by drawing image GP1 and mask GM on the same layer. The four corners of the white areas in mask GM correspond to the positions of the cross anchors, and the transmittance of these white areas is 100%. Furthermore, when the composite image is a tiling image, as shown in FIG. 7, the area from cross anchors M2 to M5 in image GP1 (image GUA) projected from projector 10A is not masked (i.e., the area where the image from projector 10A and the image from projector 10B overlap). Similarly, although not shown, the image GP1 (image GUB) projected from projector 10B is not masked in the area from cross anchors M2 to M5 (i.e., the area where the image from projector 10A and the image from projector 10B overlap). This prevents a portion of the composite image from being hidden by the mask in the area where the image from projector 10A and the image from projector 10B overlap. Furthermore, the light transmittance of the area surrounding the white portion of the mask GM (the area with diagonal hatching) is 0%. That is, in this embodiment, the area surrounding the white portion of the mask GM is a black image. The area surrounding the white portion of the mask GM may also be an image composed of a single color, such as red, blue, or green. By projecting structured light masked by the mask GM, the influence of reflected light that extends beyond the ceiling or side surfaces can be mitigated, and the correspondence between the camera coordinate system and the panel coordinate system can be accurately determined. The projection range onto which projector 10A projects structured light in second detection process SA140 is an example of a third projection range in the present disclosure, and this structured light is an example of the third structured light. The projection range onto which projector 10B projects structured light in second detection process SA140 is an example of a fourth projection range in the present disclosure, and this structured light is an example of the fourth structured light.The mask GM shown in FIG. 7 is for the case where the composite image is a tiling image, but if the composite image is a stacking image, the area from the cross anchors M2 to M5 may also be masked.

[0040] In the second calculation process SA150, the shape adjustment unit 210b connects the cross anchors based on the correspondence between the panel coordinate system of the projector 10A and the panel coordinate system of the projector 10B identified in the second detection process SA140 and the positions of the cross anchors M1 to M6.

[0041] If the determination result of the determination process SA130 is "Yes", the correction value calculated in the second calculation process SA150 is set in the image processing unit 130a in the reflection process SA160. If the determination result of the determination process SA130 is "No", the correction value calculated in the first calculation process SA120 is set in the image processing unit 130a in the reflection process SA160. The present embodiment will be summarized below.

[0042] In first detection processing SA110, each of projector 10A and projector 10B projects unmasked structured light. Next, in first calculation processing SA120, processing device 210 identifies the positions of cross anchors M2 and M5 in the second adjusted image projected from projector 10B based on the correspondence between the panel coordinate system of projector 10A and the panel coordinate system of projector 10B. If processing device 210 determines in determination processing SA130 that out-of-frame projection is being performed, then in second detection processing SA140, processing device 210 causes each of projector 10A and projector 10B to project masked structured light. The mask is based on the positions of each of cross anchors M1 to M6, including the positions of cross anchors M2 and M5, in the second adjusted image calculated in first calculation processing SA120. In the second calculation process SA150, the processing device 210 connects the cross anchors based on the correspondence between the panel coordinate system of the projector 10A and the panel coordinate system of the projector 10B identified in the second detection process SA140 and the positions of the cross anchors M1 to M6.

[0043] That is, in this embodiment, structured light is projected twice from each of projector 10A and projector 10B. The first time (first detection process SA110), unmasked structured light is projected, and the second time (second detection process SA140), masked structured light is projected. When overhang projection is performed, the structured light projected the second time is masked based on the structured light projected the first time, improving the accuracy of the correspondence between the panel coordinate system of projector 10A and the panel coordinate system of projector 10B, which is determined in second detection process SA140. Although the panel correspondence determined in first detection process SA110 may be affected by secondary reflected light, the purpose of approximately determining the positions of each of the cross anchors M1 to M6 (particularly the positions of M2 and M5) for defining the mask can be achieved. Thereafter, the panel correspondence relationship is generated again based on structured light masked according to the positions of each cross anchor M1 to M6 identified in the second detection process SA140, so the influence of secondary reflected light is reduced and the accuracy of the geometric correction is improved compared to when the panel correspondence relationship generated in the first detection process SA110 is used for geometric correction.

[0044] Furthermore, because the structured light projected in the first detection process SA110 is not masked, it includes not only the positions of the cross anchors M1-M6 but also the positions of the surrounding pixels. Therefore, although the panel correspondence relationships identified in the first detection process SA110 may be affected by the secondary reflected light, it is possible to prevent a decrease in the accuracy of roughly identifying the positions of the cross anchors M1-M6 (especially the positions of M2 and M5) due to a lack of information about the surrounding pixels.

[0045] Furthermore, because the structured light projected in the first detection process SA110 is not masked, the user can easily determine the extent to which the structured light is affected by the secondary reflected light, particularly when the composite image is a stacked image. In other words, this embodiment can improve user convenience even in the case of stacked images, for which panel correspondence does not necessarily need to be calculated.

[0046] As described above, according to this embodiment, in the case of out-of-frame projection, the influence of reflected light of light that has spilled onto the ceiling or sides can be mitigated, and geometric correction of the composite image can be performed with high precision.

[0047] 2-1. Variation 1 In the above-described embodiment, a composite image was obtained by projecting images from two projectors 10 arranged horizontally onto the projection target SC. However, the projected images from each projector may be arranged vertically, or three or more projectors may be used. Furthermore, instead of arranging the projected images from multiple projectors to synthesize a composite image, i.e., constructing a tiling image, a composite image may be synthesized by superimposing the projected images, i.e., constructing a stacking image. Furthermore, the display surface of the image on the projection target SC is not limited to a flat surface and may be a curved surface. If the display surface of the composite image is a curved surface, the number of cross anchors may be increased to, for example, five horizontally and three vertically to ensure linearity within the composite image. Furthermore, the composite image may be an image that combines a tiling image and a stacking image.

[0048] 2-2. Variation 2 The control device 20 may be included in either the projector 10A or the projector 10B. For example, if the control device 20 is included in the projector 10A, the projector 10A becomes the main projector that controls the projector 10B. If the control device 20 is included in the projector 10A, the user can instruct the adjustment of the shape of the composite image using an OSD menu or the like of the projector 10A.

[0049] 2-3. Variation 3 In the determination process SA130 in the above-described embodiment, whether or not there is out-of-frame projection is determined based on a user instruction. However, whether or not there is out-of-frame projection may also be determined based on whether or not a component of secondary reflected light from a ceiling or the like is included in the image captured by the image capture device 120. In this aspect, if it is determined that there is out-of-frame projection, a notification may be given to the user, such as by displaying a message informing the user of the out-of-frame projection. Furthermore, in an aspect in which whether or not there is out-of-frame projection is determined based on whether or not there is a component of secondary reflected light from a ceiling or the like in the image captured by the image capture device 120, the third projection range and the fourth projection range may be automatically set based on the image captured by the image capture device 120 so as not to include secondary reflected light. In this case, the difference between the automatically set third projection range and the first projection range, and the difference between the automatically set fourth projection range and the second projection range may be colored red, for example, to notify the user of these differences. This notification may be performed before executing the second calculation process SA150, i.e., before specifying the correction value.

[0050] 2-4. Variation 4 Image GP2 may be generated by reducing image GP1, as shown in FIG. 8. In other words, image GP2 does not have to be an image generated by masking image GP1. Furthermore, when image GP2 is generated by reduction, the user may be notified according to the reduction ratio of image GP2 relative to image GP1. In this modification, image GP1 may be subjected to a predetermined reduction ratio in at least one of the horizontal and vertical directions, or pixels may be thinned out. The predetermined reduction ratio may be a fixed value or a variable that can be set by the user.

[0051] 2-5. Variation 5 In the above-described embodiment, the first detection processing SA110, the first calculation processing SA120, the determination processing SA130, the second detection processing SA140, the second calculation processing SA150, and the reflection processing SA160 are all executed by the control device 20, but this is not limited to this. For example, the processing device 110 of the projector 10A may execute the first detection processing SA110, the first calculation processing SA120, and the determination processing SA130, and the control device 20 may execute the second detection processing SA140, the second calculation processing SA150, and the reflection processing SA160. For example, the processing device 110 of the projector 10A may execute the first detection processing SA110, the first calculation processing SA120, and the determination processing SA130, and the processing device 110 of the projector 10B may execute the second detection processing SA140, the second calculation processing SA150, and the reflection processing SA160. That is, the operations (steps) executed based on the program P2 may be appropriately shared among the control device 20, the projector 10A, and the projector 10B. In this case, the program P2 may be stored in the storage device 160 of the projector 10A or the storage device 160 of the projector 10B.

[0052] 2-6. Variation 6 In the above-described embodiment, at least the first detection process SA110 and the second detection process SA140 may be executed while projector 10A is communicating with projector 10B. For example, after structured light is projected only from projector 10A, projector 10A may send a command signal to projector 10B to cause only projector 10B to project structured light, or projector 10A may receive at least part of the information necessary to generate the panel correspondence relationship from projector 10B. At least part of the information necessary to generate the panel correspondence relationship is, for example, data indicating the number of pixels and pixel coordinates of the generation unit of projector 10B.

[0053] 3. Summary of this disclosure The present disclosure is not limited to the above-described embodiments and modifications, and can be realized in various forms without departing from the spirit thereof. For example, the present disclosure can also be realized in the following forms. The technical features in the above embodiments corresponding to the technical features in each form described below can be replaced or combined as appropriate to solve some or all of the problems of the present disclosure or to achieve some or all of the effects of the present disclosure. Furthermore, if a technical feature is not described as essential in this specification, it can be deleted as appropriate. A summary of this disclosure is provided below.

[0054] (Appendix 1) A control method according to one aspect of the present disclosure is a control method for a multi-projection system including a first projector that projects a first projection image into a first projection range, and a second projector that projects a second projection image that forms content by combining the first projection image into a second projection range that at least partially overlaps the first projection range, the control method including: projecting first structured light into the first projection range; capturing an image of the first structured light projected into the first projection range; projecting second structured light into the second projection range; capturing an image of the second structured light projected into the second projection range; and capturing the first projection image based on the captured images of the first structured light and the second structured light. projecting third structured light onto a third projection range that is smaller than the first projection range and onto which a portion of the content is projected, capturing an image of the third structured light projected onto the third projection range, projecting fourth structured light onto a fourth projection range that is smaller than the second projection range and onto which another portion of the content is projected, capturing an image of the fourth structured light projected onto the fourth projection range, and determining a geometric correction value of a composite image of the first projected image and the second projected image based on the captured images of the third structured light and the fourth structured light. According to this aspect, highly accurate correction can be performed even when projection extends beyond the frame.

[0055] (Appendix 2) A more preferred embodiment of the control method is the control method described in (Supplementary Note 1), which includes: accepting a user operation to designate the third projection range; and accepting a user operation to designate the fourth projection range. According to this embodiment, the user can designate the third projection range and the fourth projection range.

[0056] (Appendix 3) Another preferred embodiment of the control method is the control method described in Supplementary Note 1, which further includes setting the third projection range and the fourth projection range based on an imaging result of the first structured light projected into the first projection range and the second structured light projected into the second projection range. According to this embodiment, out-of-frame projection is automatically detected, and the third projection range and the fourth projection range are automatically set.

[0057] (Appendix 4) Another preferred embodiment of the control method is the control method described in (Supplementary Note 3), which further includes notifying the user of a difference between the first projection range and the third projection range and a difference between the second projection range and the fourth projection range before specifying the geometric correction value. According to this embodiment, the user can grasp the difference between the automatically set third projection range and the first projection range, and the difference between the automatically set fourth projection range and the second projection range.

[0058] (Appendix 5) In another preferred embodiment of the control method, the composite image is a tiling image in which a portion of the first projection range overlaps a portion of the second projection range, the first position is a position in the portion of the first projection range, and the second position is a position in the portion of the second projection range. According to this embodiment, distortion at the boundary between the first projection image and the second projection image can be corrected with high accuracy.

[0059] (Appendix 6) In another preferred embodiment of the control method, the third structured light is generated by masking a portion of the first structured light that corresponds to the difference between the first projection range and the third projection range, and the fourth structured light is generated by masking a portion of the second structured light that corresponds to the difference between the second projection range and the fourth projection range. According to this embodiment, the spacing between the measurement patterns is unchanged between the third structured light and the first structured light, and the spacing between the measurement patterns is also unchanged between the fourth structured light and the second structured light, so it is possible to prevent the processing load of projector 10 in second detection process SA140 from being higher than the processing load of projector 10 in first detection process SA110.

[0060] (Appendix 7) Another preferred embodiment of the control method is the control method described in any one of (Supplementary Note 1) to (Supplementary Note 5), in which the third structured light is generated by reducing the first structured light, and the fourth structured light is generated by reducing the second structured light. According to this embodiment, the number of measurement patterns is constant between the third structured light and the first structured light, and the number of measurement patterns is constant between the fourth structured light and the second structured light, so that the correspondence between the camera coordinate system and the panel coordinate system can be identified without causing any loss of information in the second detection process SA140.

[0061] (Appendix 8) Another preferred embodiment of the control method is the control method described in (Supplementary Note 7), in which a user is notified according to the reduction ratio of the third structured light relative to the first structured light and the reduction ratio of the fourth structured light relative to the second structured light. According to this embodiment, the user can grasp the reduction ratio of the third structured light relative to the first structured light and the reduction ratio of the fourth structured light relative to the second structured light, and an excessive increase in processing load due to excessive reduction can be avoided.

[0062] (Appendix 9) A program according to an aspect of the present disclosure includes a program that causes a computer to: project first structured light onto a first projection range from a first projector that projects a first projection image onto the first projection range; capture an image of the first structured light projected onto the first projection range; project second structured light onto a second projection range that at least partially overlaps with the first projection range to form a second projection image that is combined with the first projection image to form content; capture an image of the second structured light projected onto the second projection range; and capture an image of the second structured light projected onto the second projection range based on the results of capturing the first structured light and the second structured light. projecting third structured light from the first projector onto a third projection range that is smaller than the first projection range and onto which a portion of the content is projected, capturing an image of the third structured light projected onto the third projection range, projecting fourth structured light from the second projector onto a fourth projection range that is smaller than the second projection range and onto which another portion of the content is projected, capturing an image of the fourth structured light projected onto the fourth projection range, and determining a geometric correction value of a composite image of the first projected image and the second projected image based on the captured images of the third structured light and the fourth structured light. According to this aspect, highly accurate correction can be performed even when projection extends beyond the frame.

[0063] (Appendix 10) Also, a projector according to one aspect of the present disclosure includes: a projection device that projects a first projection image into a first projection range; a communication device that communicates with another projector that projects a second projection image that forms content by combining the first projection image into a second projection range that at least partially overlaps the first projection range; and a processing device, wherein the processing device projects first structured light from the projection device into the first projection range, captures an image of the first structured light projected into the first projection range, projects second structured light from the other projector into the second projection range, captures an image of the second structured light projected into the second projection range, and generates the first projection image based on the captured image of the first structured light and the second structured light. a projection device for projecting third structured light onto a third projection range that is smaller than the first projection range and onto which a portion of the content is projected, capturing an image of the third structured light projected onto the third projection range, a projection device for projecting fourth structured light onto a fourth projection range that is smaller than the second projection range and onto which another portion of the content is projected, capturing an image of the fourth structured light projected onto the fourth projection range, and a projection device for projecting the fourth structured light onto the fourth projection range based on the captured images of the third structured light and the fourth structured light. [Explanation of symbols]

[0064] 1...multi-projection system, 10, 10A, 10B...projector, 20...control device, 110, 210...processing device, 120, 220...communication device, 130, 240...input device, 140...projection device, 150...imaging device, 160, 250...storage device, 230...display device, 110a...pattern analysis unit, 110b...calculation unit, 110c, 210b...shape adjustment unit, 140a...image processing unit, 210a...image generation unit, P1, P2...program.

Claims

1. A control method for a multi-projection system including a first projector that projects a first projection image into a first projection range, and a second projector that projects a second projection image that forms content by combining the first projection image with the first projection image into a second projection range that at least partially overlaps the first projection range, the method comprising: projecting a first structured light onto the first projection area; capturing an image of the first structured light projected onto the first projection range; projecting second structured light into the second projection area; capturing an image of the second structured light projected onto the second projection range; Identifying a second position in the second projection image corresponding to a first position in the first projection image based on the imaging results of the first structured light and the second structured light; projecting third structured light onto a third projection range that is smaller than the first projection range and onto which a portion of the content is projected; capturing an image of the third structured light projected onto the third projection range; projecting a fourth structured light onto a fourth projection range that is smaller than the second projection range and onto which another portion of the content is projected; capturing an image of the fourth structured light projected onto the fourth projection range; determining a geometric correction value of a composite image of the first projected image and the second projected image based on the imaging results of the third structured light and the fourth structured light; A control method comprising:

2. receiving a user operation to designate the third projection range; The control method according to claim 1 , further comprising: receiving an operation by the user to designate the fourth projection range.

3. and setting the third projection range and the fourth projection range based on an imaging result of the first structured light projected onto the first projection range and the second structured light projected onto the second projection range. The control method according to claim 1 .

4. The control method according to claim 3 , further comprising notifying the user of a difference between the first projection range and the third projection range and a difference between the second projection range and the fourth projection range before specifying the geometric correction value.

5. the composite image is a tiling image in which a part of the first projection range overlaps a part of the second projection range, The control method according to claim 1 , wherein the first position is a position in a portion of the first projection range, and the second position is a position in a portion of the second projection range.

6. the third structured light is generated by masking a portion of the first structured light corresponding to a difference between the first projection range and the third projection range, The control method according to claim 1 , wherein the fourth structured light is generated by masking a portion of the second structured light that corresponds to a difference between the second projection range and the fourth projection range.

7. the third structured light is generated by reducing the first structured light; The method of claim 1 , wherein the fourth structured light is generated by reducing the second structured light.

8. The control method according to claim 7 , further comprising the step of notifying a user of a reduction ratio of the third structured light relative to the first structured light and a reduction ratio of the fourth structured light relative to the second structured light.

9. On the computer, projecting a first structured light onto a first projection range from a first projector that projects a first projection image onto the first projection range; capturing an image of the first structured light projected onto the first projection range; projecting second structured light from a second projector onto a second projection range that at least partially overlaps with the first projection range, the second projection image being combined with the first projection image to form content; capturing an image of the second structured light projected onto the second projection range; Identifying a second position in the second projection image corresponding to a first position in the first projection image based on the imaging results of the first structured light and the second structured light; projecting third structured light from the first projector onto a third projection range that is smaller than the first projection range and onto which a portion of the content is projected; capturing an image of the third structured light projected onto the third projection range; projecting fourth structured light from the second projector onto a fourth projection range that is smaller than the second projection range and onto which another portion of the content is projected; capturing an image of the fourth structured light projected onto the fourth projection range; determining a geometric correction value of a composite image of the first projected image and the second projected image based on the imaging results of the third structured light and the fourth structured light; A program that executes the following.

10. a projection device that projects a first projection image onto a first projection range; a communication device that communicates with another projector that projects a second projection image, which forms content by combining with the first projection image, into a second projection range that at least partially overlaps with the first projection range; a processing unit; The processing device includes: projecting first structured light from the projection device onto the first projection area; capturing an image of the first structured light projected onto the first projection range; projecting second structured light from the other projector onto the second projection area; capturing an image of the second structured light projected onto the second projection range; Identifying a second position in the second projection image corresponding to a first position in the first projection image based on the imaging results of the first structured light and the second structured light; projecting third structured light from the projection device onto a third projection range that is smaller than the first projection range and onto which a portion of the content is projected; capturing an image of the third structured light projected onto the third projection range; projecting fourth structured light from the other projector onto a fourth projection range that is smaller than the second projection range and onto which another portion of the content is projected; capturing an image of the fourth structured light projected onto the fourth projection range; determining a geometric correction value of a composite image of the first projected image and the second projected image based on the imaging results of the third structured light and the fourth structured light; Run the projector.

Images