Control method, projector and program product

Abstract

A control method, a projector, and a program product are provided to improve image quality in an overlapping region of a projected image. The control method includes: projecting, onto a projection surface, a first image group having a first state in which the brightness of the first projection image is greater than zero and the brightness of the second projection image is zero in a corresponding region corresponding to an overlapping region in which a part of the first projection image and a part of the second projection image overlap on the projection surface; an imaging device that acquires first imaging data by imaging the corresponding region in a state where the first image group is projected onto the projection surface; projecting, onto the projection surface, a second image group having a second state in which the brightness of the first projection image is zero and the brightness of the second projection image is greater than zero in the corresponding region; the imaging device captures an image of the corresponding region in a state where the second image group is projected on the projection surface to acquire second imaging data. The first imaging data and the second imaging data are analyzed to detect a shift between the first projection image and the second projection image in a range of the overlapping region.

Description

Technical Field

[0001] The present disclosure relates to a control method, a projector, and a program product. Background Art

[0002] For example, Patent Document 1 discloses a projection control device including: an acquisition unit that acquires a captured image obtained by capturing a region of a projection surface, the region of the projection surface including at least an overlapping region where a first projection image projected by a first projector onto the projection surface overlaps with a second projection image projected by a second projector onto the projection surface; and a detection unit that analyzes the captured image and detects the magnitude and direction of the shift in the positions of the first projection image and the second projection image in the overlapping region. The detection unit detects the shift based on a frequency spectrum image obtained by applying two-dimensional Fourier transform processing to the captured image.

[0003] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2021-61510

[0004] In the technology described in Patent Document 1, depending on the type of the projection image, an accurate frequency spectrum may not be generated during the above two-dimensional Fourier transform processing, and as a result, there is a problem that the detection accuracy of the detection unit is reduced. Summary of the Invention

[0005] A control method according to one aspect of the present disclosure is a control method in a multi-projection system in which an overlapping region is set, and in the overlapping region, a part of a first projection image projected from a first projector overlaps with a part of a second projection image projected from a second projector on a projection surface. The control method includes: projecting a first image group having a first state onto the projection surface, the first state being such that in a corresponding region corresponding to the overlapping region, the brightness of the first projection image is greater than zero and the brightness of the second projection image is zero; acquiring first captured data by capturing the corresponding region in a state where the first image group is projected onto the projection surface; projecting a second image group having a second state onto the projection surface, the second state being such that in the corresponding region, the brightness of the first projection image is zero and the brightness of the second projection image is greater than zero; acquiring second captured data by capturing the corresponding region in a state where the second image group is projected onto the projection surface; and detecting a shift between the first projection image and the second projection image in the range of the overlapping region by analyzing the first captured data and the second captured data.

[0006] Another aspect of the projector of the present disclosure is used as the first projector in a multi-projector system having an overlapping area where a part of the first projection image projected from the first projector and a part of the second projection image projected from the second projector overlap on the projection surface. The projector includes an optical device and a processing device. The processing device performs the following processes: controlling the operation of the optical device and controlling the operation of the second projector so that a first image group having a first state is projected onto the projection surface, where the first state is that in the corresponding area corresponding to the overlapping area, the brightness of the first projection image is greater than zero and the brightness of the second projection image is zero; obtaining first shooting data by having a shooting device shoot the corresponding area in a state where the first image group is projected onto the projection surface; controlling the operation of the optical device and controlling the operation of the second projector so that a second image group having a second state is projected onto the projection surface, where the second state is that in the corresponding area, the brightness of the first projection image is zero and the brightness of the second projection image is greater than zero; obtaining second shooting data by having the shooting device shoot the corresponding area in a state where the second image group is projected onto the projection surface; and detecting the offset between the first projection image and the second projection image in the range of the overlapping area by analyzing the first shooting data and the second shooting data.

[0007] A program product according to an aspect of the present disclosure is a program product used in a multi-projector system having an overlapping area where a part of the first projection image projected from the first projector and a part of the second projection image projected from the second projector overlap on the projection surface. The program product causes a computer to perform the following processes: projecting a first image group having a first state onto the projection surface, where the first state is that in the corresponding area corresponding to the overlapping area, the brightness of the first projection image is greater than zero and the brightness of the second projection image is zero; obtaining first shooting data by having a shooting device shoot the corresponding area in a state where the first image group is projected onto the projection surface; projecting a second image group having a second state onto the projection surface, where the second state is that in the corresponding area, the brightness of the first projection image is zero and the brightness of the second projection image is greater than zero; obtaining second shooting data by having the shooting device shoot the corresponding area in a state where the second image group is projected onto the projection surface; and detecting the offset between the first projection image and the second projection image in the range of the overlapping area by analyzing the first shooting data and the second shooting data. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1It is a schematic diagram of a multi-projection system used in the control method of the first embodiment.

[0009] Figure 2 It is a block diagram of the projector of the first embodiment.

[0010] Figure 3 It is a flowchart showing the process of the control method of the first embodiment.

[0011] Figure 4 It is a diagram for explaining the initial alignment.

[0012] Figure 5 It is a diagram for explaining the first image group.

[0013] Figure 6 It is a diagram for explaining the second image group.

[0014] Figure 7 It is a diagram for explaining the analysis area for calculating the correction amount and the coordinate transformation of the analysis area.

[0015] Figure 8 It is a diagram for explaining the matching process between the first partial area and the analysis area.

[0016] Figure 9 It is a diagram for explaining the calculation of the offset between the first projected image and the second projected image within the range of the overlapping area.

[0017] Figure 10 It is a diagram for explaining the mixing range during the calculation of the correction amount.

[0018] Figure 11 It is a block diagram of the projector of the second embodiment.

[0019] Figure 12 It is a flowchart showing the process of the control method of the second embodiment.

[0020] Figure 13 It is a diagram for explaining the mixing range during the calculation of the correction amount.

[0021] Reference numeral description

[0022] 10: Projector; 10-1: First Projector (Projector); 10-2: Second Projector; 11: Storage Device; 12: Processing Device; 12a: Projection Control Unit; 12b: Shooting Control Unit; 12c: Correction Unit; 12d: Projection Control Unit; 13: Communication Device; 14: Image Processing Circuit; 15: Optical Device; 15a: Light Source; 15b: Light Modulator; 15c: Projection Optical System; 15c-1: Projection Optical System (First Optical System); 15c-2: Projection Optical System (Second Optical System); 16: Operation Device; 17: Shooting Device; 18: Temperature Sensor; 20: Processing Device; 30: Terminal Device; 100: Multi-Projection System; AB: Outlier; BR1: Brightness; BR2: Brightness; D1: First Shooting Data; D2: Second Shooting Data; DR: Direction; G1: First Projected Image; G2: Second Projected Image; GG: Image Group; GG0: Image Group; GG1: First Image Group; GG2: Second Image Group; GG3: Image Group; IMG1: Video Data; IMG2: Video Data; PA: Correction Amount Information; PR1: Program; PR2: Program; PT: Color Encoding Pattern; R: Overlap Region; R1: First Partial Region; R1a: First State Region; R1b: First Mixing Region; R1c: Second State Region; R2: Second Partial Region; R2a: First State Region; R2b: Second Mixing Region; R2c: Second State Region; RA: Analysis Region; RA1: Analysis Region; RA2: Analysis Region; RC: Corresponding Region; RN1: Non-Overlap Region; RN2: Non-Overlap Region; S10: Step; S20: Step; S21: Step; S30: Step; S31: Step; S40: Step; S50: Step; S60: Step; S61: Step; S62: Step; S63: Step; S64: Step; S70: Step; S80: Step; S90: Step; S100: Step; S110: Step; S120: Step; S130: Step; SC: Projection Surface; α: First Range; β: Second Range. Detailed Embodiments

[0023] Hereinafter, preferred embodiments of the present disclosure will be described with reference to the drawings. In addition, in the drawings, the dimensions and scales of each part are appropriately different from the actual ones, and there are also parts schematically shown for easy understanding. In addition, the scope of the present disclosure is not limited to these embodiments as long as the gist of the present disclosure is not particularly limited in the following description.

[0024] 1. First Embodiment

[0025] 1-1. Outline of the Multi-Projection System

[0026] Figure 1 It is a schematic diagram of the multi-projection system 100 used in the control method of the first embodiment. AsFigure 1 As shown in Figure 1 , the multi-projection system 100 includes a first projector 10-1, a second projector 10-2, and a terminal device 30. The first projector 10-1 is an example of a "projector". Hereinafter, the first projector 10-1 and the second projector 10-2 may sometimes be collectively referred to as the projector 10 without distinction.

[0027] The multi-projection system 100 projects an image group GG onto a projection surface SC using a plurality of projectors 10. In Figure 1 the example shown in Figure 1 , the multi-projection system 100 projects the image group GG including a first projected image G1 and a second projected image G2 onto the projection surface SC using two projectors 10. The projection surface SC is the surface of an object such as a screen. In Figure 1 the example shown in Figure 1 , the projection surface SC is a plane.

[0028] In addition, the projection surface SC is not limited to a plane and may be a curved surface, for example. Also, in the present embodiment, an example in which the number of projectors 10 included in the multi-projection system 100 is two is illustrated, but the present invention is not limited to this, and the number may be three or more. That is, the image group GG may also include images projected from three or more projectors 10.

[0029] The first projector 10-1 is a display device that projects the first projected image G1 shown in the video data IMG1 output from the terminal device 30 onto the projection surface SC. On the other hand, the second projector 10-2 is a display device that projects the second projected image G2 shown in the video data IMG2 output from the terminal device 30 onto the projection surface SC.

[0030] The first projected image G1 and the second projected image G2 are arranged in the arrangement direction DR in this order. Here, the first projected image G1 and the second projected image G2 are projected onto the projection surface SC in a connected state so that the image group GG displays a single image. In Figure 1 the example shown in Figure 1 , the first projected image G1 is projected onto the Figure 1 left region of the projection surface SC, while the second projected image G2 is projected onto the Figure 1 right region of the projection surface SC. And the Figure 1 right end of the first projected image G1 and the Figure 1 left end of the second projected image G2 are connected to each other. That is, the Figure 1 right end of the first projected image G1 and the Figure 1 left end of the second projected image G2 overlap.

[0031] A part of the first projected image G1 and the second projected image G2 overlaps with each other in the overlapping region R. The overlapping region R is an area where the following blending process for making the connection between the first projected image G1 and the second projected image G2 inconspicuous is implemented. Thus, in the multi-projection system 100, an overlapping region R is set such that a part of the first projected image G1 projected from the first projector 10-1 and the second projected image G2 projected from the second projector 10-2 overlap with each other on the projection plane SC.

[0032] In the present embodiment, the first projector 10-1 is the host and controls the operation of the second projector 10-2 as a slave. In addition, the first projector 10-1 has a correction function for correcting the positional deviation between the second projected image G2 and the first projected image G1. The second projector 10-2 is configured in the same manner as the first projector 10-1 except that it does not have a correction function. In addition, the second projector 10-2 may be any structure that can be controlled by the first projector 10-1, and may also be a structure different from the first projector 10-1. When the number of projectors 10 included in the multi-projection system 100 is three or more, one of the three or more projectors 10 is the host, and the other two or more projectors 10 are slaves respectively.

[0033] The terminal device 30 is a device having a function of dividing the video data representing one image into a plurality of video data to be projected by a plurality of projectors 10 and a function of supplying each of the video data obtained by the division process to the corresponding projector 10.

[0034] After dividing the video data representing one image into the video data IMG1 and the video data IMG2, the terminal device 30 of the present embodiment supplies the video data IMG1 to the first projector 10-1 and supplies the video data IMG2 to the second projector 10-2.

[0035] In Figure 1 In the example shown, the terminal device 30 is a notebook computer. In addition, the terminal device 30 is not limited to a notebook computer, and may be, for example, a desktop computer, a smart phone, or a tablet terminal, etc., and may also be a video playback device, a DVD (Digital Versatile Disk) player, a Blu-ray Disc player, a hard disk recorder, a television tuner device, a set-top box for CATV (Cable television), a video game console, etc.

[0036] 1-2. Projector

[0037] Figure 2 is a block diagram of the first projector 10-1 of the first embodiment. In Figure 2In addition to the first projector 10-1, the connection state in which the second projector 10-2 and the terminal device 30 are connected to the first projector 10-1 is also shown. Further, in Figure 2 the structure of the first projector 10-1 is representatively shown. However, the structure of the second projector 10-2 is the same as that of the first projector 10-1 except that it does not have a correction function. In the following description of the structural elements, the image data IMG1 may be replaced with the image data IMG2. Hereinafter, regarding the structural elements of the projector 10, the structural elements of the first projector 10-1 and the second projector 10-2 may be distinguished by adding the suffix “-1” to the reference numerals of the structural elements of the first projector 10-1 or adding the suffix “-2” to the reference numerals of the structural elements of the second projector 10-2.

[0038] As Figure 2 shown, the first projector 10-1 includes a storage device 11, a processing device 12, a communication device 13, an image processing circuit 14, an optical device 15, an operation device 16, a photographing device 17, and a temperature sensor 18. They are connected in a manner that enables mutual communication.

[0039] The storage device 11 is a storage device that stores programs executed by the processing device 12 and data processed by the processing device 12. The storage device 11 is configured to include, for example, a hard disk drive or a semiconductor memory. In addition, part or all of the storage device 11 may be provided in an external storage device or server of the first projector 10-1.

[0040] In the storage device 11, a program PR1, first photographing data D1, second photographing data D2, and correction amount information PA are stored.

[0041] The program PR1 is a program for executing a control method described in detail later. The first photographing data D1 is data representing an image obtained by the photographing device 17 photographing a corresponding region RC described later corresponding to the overlapping region R in a state where a first projection image G1 described later is projected onto the projection surface SC. The second photographing data D2 is data representing an image obtained by the photographing device 17 photographing the corresponding region RC described later in a state where a second projection image G2 described later is projected onto the projection surface SC. The correction amount information PA is information representing a correction amount for correcting the deviation between the first projection image G1 and the second projection image G2 in the range of the overlapping region R.

[0042] The processing device 12 is a processing device having functions of controlling each part of the first projector 10-1 and processing various data. The processing device 12 is configured to include a processor such as a CPU (Central Processing Unit), for example. In addition, the processing device 12 may be composed of a single processor or multiple processors. In addition, part or all of the functions of the processing device 12 may also be implemented by hardware such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). In addition, the processing device 12 may be integrated with the image processing circuit 14.

[0043] The communication device 13 is a communication device capable of communicating with various devices, obtaining the image data IMG1 from the terminal device 30, or communicating with the second projector 10-2. For example, the communication device 13 is a wired communication device such as a wired LAN (Local Area Network), a USB (Universal Serial Bus), or an HDMI (High Definition Multimedia Interface), or a wireless communication device such as an LPWA (Low Power Wide Area), a wireless LAN including Wi-Fi, or Bluetooth. "HDMI", "Wi-Fi", and "Bluetooth" are registered trademarks, respectively.

[0044] The image processing circuit 14 is a circuit that performs necessary processing on the image data IMG1 from the communication device 13 and inputs it to the optical device 15. The image processing circuit 14 has, for example, a frame memory (not shown), loads the image data IMG1 into the frame memory, appropriately performs various processes such as resolution conversion processing, size adjustment processing, and distortion correction processing, and inputs them to the optical device 15. In addition, the image processing circuit 14 performs processing to correct at least the offset between the first projection image G1 and the second projection image G2 in the range of the overlapping region R according to the correction amount information PA stored in the storage device 11. In addition, the image processing circuit 14 may perform OSD (On Screen Display) processing or other processing such as generating image information for menu display or operation guidance as needed and synthesizing it into the image data IMG1. In addition, the image processing circuit 14 may perform processing to correct the offset between the entire first projection image G1 and the entire second projection image G2 according to the correction amount information PA stored in the storage device 11.

[0045] The optical device 15 is a device that projects image light onto the projection surface SC. The optical device 15 includes a light source 15a, a light modulator 15b, and a projection optical system 15c. The projection optical system 15c of the first projector 10-1 is an example of the "first optical system". In addition, although not shown, the second projector 10-2 also has a projection optical system 15c in the same manner as the first projector 10-1, and the projection optical system 15c of the second projector 10-2 is an example of the "second optical system".

[0046] The light source 15a is configured to include, for example, a light source such as a halogen lamp, a xenon lamp, an ultra-high pressure mercury lamp, an LED (Light Emitting Diode), or a laser light source, and emits red, green, and blue light respectively. The light modulator 15b depicts an image based on the video data IMG1 supplied from the terminal device 30. The light modulator 15b of the first projector 10-1 is an example of the display panel of the first projector. In addition, the light modulator 15b of the first projector 10-1 may be referred to as the first display panel, and the light modulator 15b of the second projector 10-2 may be referred to as the second display panel. The light modulator 15b is configured to include three light modulation elements provided corresponding to red, green, and blue. Each light modulation element includes, for example, a transmissive liquid crystal panel, a reflective liquid crystal panel, or a DMD (Digital Micromirror Device), etc., and generates image light of each color by modulating the light of the corresponding color. The image light of each color generated by the light modulator 15b is synthesized by a color synthesis optical system to become full-color image light. The projection optical system 15c is an optical system including a projection lens that images and projects the full-color image light from the light modulator 15b onto the projection surface SC. The image depicted by the light modulator 15b, that is, the depicted image, is projected onto the projection surface SC via the projection lens.

[0047] The operation device 16 is a device that accepts operations from the user. For example, the operation device 16 includes an operation panel (not shown) and an infrared receiver for the remote control. The operation panel is provided on the outer casing of the first projector 10-1 and outputs a signal based on the operation from the user. The infrared receiver for the remote control receives an infrared signal from a remote control (not shown), decodes the infrared signal, and outputs a signal based on the operation of the remote control. In addition, the operation device 16 is provided as needed and may be omitted.

[0048] The imaging device 17 is a digital camera having an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The imaging element has a plurality of pixels.

[0049] The temperature sensor 18 is a temperature sensor such as a thermistor that detects the temperature of the projection optical system 15c. The temperature sensor 18 is built into the projector 10 and fixed at a specified position near the projection optical system 15c within the housing of the projector 10. Additionally, in the case of using the temperature sensors 18 of both the first projector 10-1 and the second projector 10-2, it is possible to detect the temperatures of the projection optical systems 15c of both the first projector 10-1 and the second projector 10-2.

[0050] In the above-described first projector 10-1, the processing device 12 functions as a projection control unit 12a, a shooting control unit 12b, and a correction unit 12c by executing the program PR1 stored in the storage device 11. Therefore, the processing device 12 includes a projection control unit 12a, a shooting control unit 12b, and a correction unit 12c.

[0051] The projection control unit 12a controls the operations of the image processing circuits 14 and the optical devices 15 of the first projector 10-1 and the second projector 10-2, respectively. More specifically, the projection control unit 12a causes the first image group GG1 described below to be projected onto the projection surface SC, or causes the second image group GG2 described below to be projected onto the projection surface SC.

[0052] The shooting control unit 12b controls the operation of the shooting device 17 of one or both of the first projector 10-1 and the second projector 10-2. More specifically, the shooting control unit 12b causes the shooting device 17 to shoot the corresponding region RC described below in a state where the first image group GG1 described below is projected onto the projection surface SC, or causes the shooting device 17 to shoot the corresponding region RC in a state where the second image group GG2 described below is projected onto the projection surface SC, thereby obtaining the first shooting data D1 and the second shooting data D2.

[0053] The correction unit 12c generates correction amount information PA based on the first shooting data D1 and the second shooting data D2. More specifically, the correction unit 12c analyzes the first shooting data D1 and the second shooting data D2, detects the shift between the first projection image G1 and the second projection image G2 in the range of the overlapping region R, and generates correction amount information PA based on the detection result.

[0054] 1-3. Control Method

[0055] Figure 3 is a flowchart showing the flow of the control method of the first embodiment. This control method is performed using the above-described multi-projection system 100.

[0056] As Figure 3As shown, in step S10 of the control method of this embodiment, the processing device 12 performs the initial alignment of the coordinate system of the imaging device 17 and the coordinate system of the optical device 15. This initial alignment is performed by establishing the correspondence between the coordinate system of the imaging device 17 of the first projector 10-1 and the coordinate system of the optical device 15 of the first projector 10-1, and establishing the correspondence between the coordinate system of the imaging device 17 of the first projector 10-1 and the coordinate system of the optical device 15 of the second projector 10-2. Details will be provided later based on Figure 4 for detailed description.

[0057] After step S10, in step S20, the projection control unit 12a projects the first image group GG1 described later onto the projection surface SC. The first image group GG1 is an image group GG having a first state in which the brightness BR1 of the first projection image G1 is greater than zero and the brightness BR2 of the second projection image G2 is zero in at least a part of the corresponding region RC corresponding to the overlapping region R. Details will be provided later according to Figure 5 for detailed description. Step S20 of this embodiment includes a step S21 of reducing the range of the mixing process of the first projection image G1 and the second projection image G2, that is, the mixing range.

[0058] After step S20, in step S30, the imaging control unit 12b acquires the first imaging data D1. This acquisition is performed by causing the imaging device 17 to image the corresponding region RC in the state where the first image group GG1 is projected onto the projection surface SC. Details will be provided later according to Figure 5 for detailed description. Step S30 of this embodiment includes a step S31 of imaging the first state region R1a described later.

[0059] After step S30, in step S40, the projection control unit 12a projects the second image group GG2 described later onto the projection surface SC. The second image group GG2 is an image group GG having a second state in which the brightness BR1 of the first projection image G1 is zero and the brightness BR2 of the second projection image G2 is greater than zero in at least a part of the corresponding region RC. Details will be provided later according to Figure 6 for detailed description. Step S40 of this embodiment includes a step S41 of reducing the range of the mixing process of the first projection image G1 and the second projection image G2, that is, the mixing range.

[0060] After step S40, in step S50, the imaging control unit 12b acquires the second imaging data D2. This acquisition is performed by causing the imaging device 17 to image the corresponding region RC in the state where the second image group GG2 is projected onto the projection surface SC. Details will be provided later according to Figure 6 for detailed description. Step S50 of this embodiment includes a step S51 of imaging the second state region R2c described later.

[0061] After step S50, in step S60, the correction unit 12c detects the shift between the first projection image G1 and the second projection image G2 in the range of the overlapping region R. This detection is performed by analyzing the first captured data D1 and the second captured data D2, which will be described in detail later based on Figures 7 to 8 This will be elaborated upon. Step S60 of the present embodiment sequentially includes: a step S61 of cutting out an analysis region RA1 described later; a step S62 of performing coordinate transformation on the analysis region RA1 described later; a step S63 of performing a matching process; and a step S64 of detecting the offset amount and the like.

[0062] After step S60, in step S70, the correction unit 12c calculates a correction amount based on the offset amount detected in step S60 and the like. The details of this calculation will be described later based on Figure 9 This will be explained.

[0063] After step S70, in step S80, the correction unit 12c generates correction amount information PA based on the correction amount calculated in step S70. The generated correction amount information PA is stored in the storage device 11. Thus, the shift between the first projection image G1 and the second projection image G2 in the range of the overlapping region R is corrected.

[0064] After step S80, in step S90, the correction unit 12c detects the temperature of the projection optical system 15c-1 according to the output from the temperature sensor 18.

[0065] In this way, in step S90, according to the output from the temperature sensor 18, the temperature of the projection optical system 15c-1 used for the first projector 10-1 to project the first projection image G1 is detected.

[0066] After step S90, in step S100, the correction unit 12c determines whether a specified time has elapsed, and repeats step S100 until the specified time has elapsed (step S100: No).

[0067] When it is determined that the specified time has elapsed (step S100: Yes), in step S110, the correction unit 12c determines whether the target temperature has been reached based on the temperature detected in step S90.

[0068] In the case where it is determined that the target temperature has not been reached (step S110: No), the aforementioned step S20 is executed. Thus, the aforementioned steps S20 to S110 are repeatedly executed until it is determined that the target temperature has been reached. At this time, the steps other than step S100 among the aforementioned steps S20 to S110 are executed at the specified time in step S100. Thus, before the optical characteristics of the projection optical system 15c become stable, the correction amount information PA is updated at the specified time interval to correct the shift between the first projection image G1 and the second projection image G2 caused by changes in the optical characteristics of the projection optical system 15c and the like.

[0069] In this way, "No" in step S110 is based on the output from one or more temperature sensors 18, and the detection of the shift and the correction of the shift are intermittently performed until the temperature of the projection optical system 15c-1 reaches the target temperature.

[0070] In the case where it is determined that the target temperature has been reached (step S110: Yes), in step S120, the projection control unit 12a expands the mixing range, which is the range where the mixing process of the first projection image G1 and the second projection image G2 is performed, to the entire area of the overlapping region R. After step S120, the process for detecting and correcting the shift ends. In addition, step S120 is executed as needed and can also be omitted. In this case, in the case where it is determined that the target temperature has been reached (step S110: Yes), the process for detecting and correcting the shift ends while the mixing range is maintained in the second range β.

[0071] In this way, when "Yes" in step S110, based on the output from one or more temperature sensors 18, after the temperature of the projection optical system 15c-1 reaches the target temperature, the processing device 12 stops detecting and correcting the shift. This is because the main reason for the shift between the first projection image G1 and the second projection image G2 is the change in optical characteristics caused by the temperature change of the projection optical system 15c. The change in optical characteristics is caused, for example, by the change in the deformation amount of the projection optical system 15c.

[0072] As described above, the control method of the multi-projection system 100 is executed. Hereinafter, steps S10 to S80 will be sequentially described in detail.

[0073] Figure 4 It is a diagram for explaining the initial alignment. As Figure 4As shown, in step S10, first, in a state where the image group GG0 is projected onto the projection surface SC, the imaging device 17 of the first projector 10-1 captures the corresponding region RC. The corresponding region RC is the region on the projection surface SC corresponding to the overlapping region R. Specifically, the corresponding region RC is a region having the same number of pixels or width as the overlapping region R in the arrangement direction DR. The width is expressed in, for example, metric units or imperial units. In the corresponding region RC, as will be described later, there is a case where the luminance of one of the first projection image G1 and the second projection image G2 is greater than zero and the luminance of the other is zero. For example, if the luminance of the first projection image G1 is greater than zero throughout the corresponding region RC and the luminance of the second projection image G2 is zero throughout the corresponding region RC, that is, the second projection image G2 is a black image, then in the corresponding region RC, substantially only the first projection image G1 is projected onto the projection surface SC. Therefore, the first projection image G1 does not have an overlapping region R with the second projection image G2. On the other hand, during normal use when the user projects the desired content as the image group GG onto the projection surface SC, in the corresponding region RC, an image group GG in a state where the luminances of both the first projection image G1 and the second projection image G2 are greater than zero can be projected. Therefore, the first projection image G1 has an overlapping region R with the second projection image G2. Thus, the corresponding region RC is a region that can become the overlapping region R during normal use. The image group GG0 is an image group GG in a state where the first projection image G1 and the second projection image G2 each contain a color-coded pattern PT.

[0074] In addition, before step S10, the user adjusts the width by which the first projection image G1 overlaps the second projection image G2. For example, the user adjusts the position of the second projector 10-2 relative to the first projector 10-1 so that a part of the first projection image G1 overlaps a part of the second projection image G2. After that, the user inputs information indicating the width of the overlapping region R, such as the number of pixels, to the terminal device 30, and sets the width of the overlapping region R for the first projector 10-1 and the second projector 10-2. Thus, in a state where the first projector 10-1 and the second projector 10-2 know the number of pixels of the overlapping region R, the control method of the multi-projection system 100 is executed. In the control method described later, the mixed region is changed to various sizes, and such control also uses the information indicating the number of pixels of the overlapping region R.

[0075] The color-coded pattern PT in the first projection image G1 and the color-coded pattern PT in the second projection image G2 are visibly displayed in the image group GG0. The position of the color-coded pattern PT in the first projection image G1 is known as coordinate values in the coordinate system of the optical device 15 of the first projector 10-1. On the other hand, the position of the color-coded pattern PT in the second projection image G2 is known as coordinate values in the coordinate system of the optical device 15 of the second projector 10-2. Additionally, in the coordinate system of the optical device 15 of the first projector 10-1, each pixel of the light modulation element included in the optical device 15 of the first projector 10-1 is represented by coordinate values. In the coordinate system of the optical device 15 of the second projector 10-2, each pixel of the light modulation element included in the optical device 15 of the second projector 10-2 is represented by coordinate values.

[0076] In step S10, based on the captured image obtained by capturing the image group GG0 with the imaging device 17 of the first projector 10-1 in this manner, the correspondence between the coordinate system of the imaging device 17 of the first projector 10-1 and the coordinate systems of the optical devices 15 of the first projector 10-1 and the second projector 10-2 is established. Here, the coordinate system of the imaging device 17 of the first projector 10-1 is a two-dimensional coordinate system of the captured image acquired by this imaging device 17, and the coordinate systems of the optical devices 15 of the first projector 10-1 and the second projector 10-2 are two-dimensional coordinate systems of the respective display panels of the first projector 10-1 and the second projector 10-2. Additionally, in the coordinate system of the imaging device 17 of the first projector 10-1, each pixel of the imaging element included in the imaging device 17 of the first projector 10-1 is represented by coordinate values.

[0077] Here, as described above, the color-coded pattern PT in the first projection image G1 and the color-coded pattern PT in the second projection image G2 are visibly displayed in the image group GG, and thus, based on this captured image, the positions of the color-coded pattern PT in the first projection image G1 and the color-coded pattern PT in the second projection image G2 can be respectively detected as coordinate values in the coordinate system of the imaging device 17 of the first projector 10-1.

[0078] In addition, as described above, the position of the color-coded pattern PT in the first projected image G1 is known as a coordinate value in the coordinate system of the optical device 15 of the first projector 10-1. Therefore, the correspondence between the coordinate system of the imaging device 17 of the first projector 10-1 and the coordinate system of the optical device 15 of the first projector 10-1 can be established. Similarly, as described above, the position of the color-coded pattern PT in the second projected image G2 is known as a coordinate value in the coordinate system of the optical device 15 of the second projector 10-2. Therefore, the correspondence between the coordinate system of the imaging device 17 of the first projector 10-1 and the coordinate system of the optical device 15 of the second projector 10-2 can be established.

[0079] In addition, in Figure 4 , a method of using the color-coded pattern PT in the image group GG0 is illustrated, but it is not limited to this method. Various patterns that can establish the correspondence between the coordinate system of the imaging device 17 of the first projector 10-1 and the coordinate systems of the optical devices 15 of the first projector 10-1 and the second projector 10-2 can be used instead of the color-coded pattern PT.

[0080] Figure 5 is a diagram for explaining the first image group GG1. In step S20, as Figure 5 shown, the first image group GG1 is projected onto the projection surface SC. The first image group GG1 is an image group GG having a first state in which the brightness BR1 of the first projected image G1 is greater than zero and the brightness BR2 of the second projected image G2 is zero in the corresponding region RC. In addition, "brightness is zero" means the luminance value of a black image. In Figure 5 , for ease of explanation, the characters "ABC" are displayed in the corresponding region RC of the first image group GG1.

[0081] The first projected image G1 has a non-overlapping region RN1 and a first partial region R1. The non-overlapping region RN1 is a region that does not overlap with the second projected image G2. The first partial region R1 is a region for overlapping with the second projected image G2. At least a part of the first partial region R1 is a region where the mixing process is performed, that is, the first mixing region R1b. The mixing process changes the brightness in the arrangement direction DR in which the first projected image G1 and the second projected image G2 are arranged, so that the brightness of the overlapping region R when the first projected image G1 and the second projected image G2 are projected so as to overlap with each other in the overlapping region R is the same as the brightness of the non-overlapping region RN1.

[0082] On the other hand, the second projected image G2 has a non-overlapping region RN2 and a second partial region R2. The non-overlapping region RN2 is a region that does not overlap with the first projected image G1. The second partial region R2 is a region for overlapping with the first projected image G1. At least a part of the second partial region R2 is a region where the mixing process is performed, that is, the second mixing region R2b.

[0083] In step S21 of step S20, the processing device 12 changes the mixing range where the mixing process is performed in the first partial region R1 from the first range α to a second range β smaller than the first range α by controlling the first drawing panel. Thus, in step S21, the first partial region R1 is adjusted to include, in addition to the first mixing region R1b, a first state region R1a having a first state. The first state region R1a includes a region corresponding to the analysis region RA. That is, the analysis region RA is a partial region of the first partial region R1 and is a region where the character "ABC" in the corresponding region RC is displayed on the projection surface SC. In the first state region R1a, the brightness BR1 of the first projected image G1 is greater than zero. In Figure 5 In the example shown, the position of the first state region R1a with respect to the first mixing region R1b is arranged at a position in the direction opposite to the arrangement direction DR. In other words, in step S21 of step S20, the processing device 12 moves the first mixing region R1b to a position on one side (right side) with respect to the first state region R1a in the arrangement direction DR, that is, makes it closer.

[0084] Similarly, in step S21 of step S20, the processing device 12 changes the mixing range where the mixing process is performed in the second partial region R2 from the first range α to the second range β by controlling the second drawing panel. Thus, in step S21, the second partial region R2 is adjusted to include, in addition to the second mixing region R2b, a first state region R2a having a first state. The first state region R2a includes a region corresponding to the analysis region RA of the first partial region R1. In the first state region R2a, the brightness BR2 of the second projected image G2 is zero. Therefore, in the first state region R1a or the first state region R2a, when observed from the user or the imaging device 17, it is observed that only the first projected image G1 is projected. In addition, the brightness BR2 of the analysis region RA of the first state region R2a is zero. In Figure 5 In the example shown, the position of the first state region R2a with respect to the second mixing region R2b is arranged at a position in the direction opposite to the arrangement direction DR. In other words, in step S21 of step S20, the processing device 12 moves the second mixing region R2b to the position on the said one side (right side) with respect to the first state region R2a in the arrangement direction DR, that is, makes it closer.

[0085] The first range α is, for example, the blending range applied in step S120. In Figure 5 the example shown, the first range α is the entire area of the first partial area R1. Additionally, the first range α is not limited to Figure 5 the example shown, and as long as it is larger than the second range β, it may also be smaller than the first partial area R1 or the second partial area R2.

[0086] In this way, in step S21, the blending range, that is, the range where the blending process of the first projected image G1 and the second projected image G2 is performed, is reduced in the arrangement direction DR.

[0087] In the first blending area R1b, the brightness BR1 of the first projected image G1 changes from the brightness of the non-overlapping area RN1 to zero within the second range β as it approaches the arrangement direction DR. In contrast, although not shown, when the blending range in the first blending area R1b is the first range α, the brightness BR1 of the first projected image G1 changes from the brightness of the non-overlapping area RN1 to zero within the first range α, which is larger than the second range β, as it approaches the arrangement direction DR. Therefore, in the first blending area R1b, the distribution of brightness in the arrangement direction DR when the blending range is the second range β is different from the distribution of brightness in the arrangement direction DR when the blending range is the first range α. Additionally, in the first blending area R1b, the distribution of brightness BR1 in the arrangement direction DR, that is, the blending curve, only needs to be set such that the brightness of the overlapping area R is the same as the brightness of the non-overlapping area RN1, and is not limited to Figure 5 the example shown.

[0088] On the other hand, in the second blending area R2b, the brightness BR2 of the second projected image G2 changes from the brightness of the non-overlapping area RN2 to zero within the second range β as it approaches the direction opposite to the arrangement direction DR. In contrast, although not shown, when the blending range in the second blending area R2b is the first range α, the brightness BR2 of the second projected image G2 changes from the brightness of the non-overlapping area RN2 to zero within the first range α, which is larger than the second range β, as it approaches the direction opposite to the arrangement direction DR. Therefore, in the second blending area R2b, the distribution of brightness in the arrangement direction DR when the blending range is the second range β is different from the distribution of brightness in the arrangement direction DR when the blending range is the first range α. Additionally, in the second blending area R2b, the distribution of brightness BR2 in the arrangement direction DR, that is, the blending curve, only needs to be set such that the brightness of the overlapping area R is the same as the brightness of the non-overlapping area RN2, and is not limited to Figure 5 the example shown.

[0089] In step S31 of step S30, the imaging device 17 images the corresponding region RC, i.e., the first state region R1a or the first state region R2a. Thus, in step S30, in a state where the first image group GG1 is projected onto the projection surface SC, the imaging device 17 images the corresponding region RC, whereby the first imaging data D1 is obtained. The first imaging data D1 at least includes an image of the analysis region RA that is part of the first partial region R1.

[0090] As described above, in the first state region R1a or the first state region R2a, the luminance of the first projection image G1 is greater than zero, and the luminance of the second projection image G2 is zero. Therefore, the imaging device 17 can obtain the first imaging data D1 without being affected by the luminance of the second projection image G2. In step S31 of step S30, for at least a part of the period during which the imaging device 17 images the first state region R1a or the first state region R2a, the state where the mixing range is the second range β is maintained. Additionally, in step S30, the region imaged by the imaging device 17 only needs to include a region corresponding to the analysis region RA1 described later.

[0091] Figure 6 It is a diagram for explaining the second image group GG2. In step S40, as Figure 6 shown, the second image group GG2 is projected onto the projection surface SC. The second image group GG2 is an image group GG having a second state in which the luminance BR1 of the first projection image G1 is zero and the luminance BR2 of the second projection image G2 is greater than zero in the corresponding region RC. In Figure 6 order to facilitate explanation, similar to the first image group GG1 described above Figure 5 , the characters "ABC" are displayed in the corresponding region RC of the second image group GG2.

[0092] In step S40, the first partial region R1 of the first projection image G1 is adjusted to include, in addition to the first mixing region R1b, a second state region R1c having a second state. The second state region R1c includes a region corresponding to the analysis region RA. In the second state region R1c, the luminance BR1 of the first projection image G1 is zero. Additionally, in step S40, the range of the first mixing region R1b is the same as that in step S20 described above, which is the second range β. In the example shown in Figure 6 , the second state region R1c is arranged at a position in the arrangement direction DR with respect to the first mixing region R1b. In other words, in step S41 of step S40, the processing device 12 moves the first mixing region R1b to a position on the other side (left side) with respect to the second state region R1c in the arrangement direction DR, i.e., makes it approach.

[0093] Similarly, in step S40, the second partial region R2 of the second projection image G2 is adjusted to include, in addition to the second mixing region R2b, a second state region R2c having a second state. The second state region R2c includes a region corresponding to the analysis region RA of the second partial region R2. That is, the analysis region RA is a partial region of the second partial region R2 and is a region where the characters "ABC" in the corresponding region RC are displayed on the projection plane SC. In the second state region R2c, the brightness BR2 of the second projection image G2 is greater than zero. In addition, as described above, in the second state region R1c, the brightness BR1 of the first projection image G1 is zero. Therefore, in the second state region R1c or the second state region R2c, when observed from the user or the imaging device 17, it is observed that only the second projection image G2 is projected. In addition, in step S40, the range of the second mixing region R2b is the same as that in step S20 above, which is the second range β. In Figure 6 In the example shown, the position of the second state region R2c with respect to the second mixing region R2b is arranged at the position in the arrangement direction DR. In other words, in step S41 of step S40, the processing device 12 moves the second mixing region R2b in the arrangement direction DR to a position on the other side (left side) with respect to the second state region R2c, that is, closer.

[0094] In step S50, the imaging device 17 images the second state region R1c or the second state region R2c. Thus, in step S50, in a state where the second image group GG2 is projected onto the projection plane SC, the imaging device 17 images the corresponding region RC, thereby obtaining the second imaging data D2. The second imaging data D2 at least includes an image of the analysis region RA that is a part of the second partial region R2.

[0095] As described above, in the second state region R1c or the second state region R2c, the brightness of the first projection image G1 is zero, and the brightness of the second projection image G2 is greater than zero. Therefore, the imaging device 17 can obtain the second imaging data D2 without being affected by the brightness of the first projection image G1. In step S51 of step S50, during at least a part of the period when the imaging device 17 images the second state region R1c or the second state region R2c, the state where the mixing range is the second range β is maintained. In addition, in step S50, the region imaged by the imaging device 17 only needs to include a region corresponding to the analysis region RA1 described later.

[0096] Figure 7 It is a diagram for explaining the coordinate transformation of the analysis region RA1 and the analysis region RA2 for calculating the correction amount. In step S60, first, in step S61, as Figure 7As shown on the left side in [reference], an analysis region RA2, which is a part of the second captured data D2, is cut out from the second captured data D2. The analysis region RA2 is a region in the image shown in the second captured data D2 that corresponds to the above-mentioned analysis region RA. Here, since the image shown in the second captured data D2 is represented by the coordinate system of the imaging device 17, the analysis region RA2 is represented by the coordinate system of the imaging device 17.

[0097] After such a step S61, in step S62, as Figure 7 shown on the right side in [reference], the analysis region RA2 is transformed into the coordinate system of the first projector 10-1. Specifically, the analysis region RA2 is transformed into the coordinate system of the light modulator 15b that the first projector 10-1 has. This transformation is performed based on the correspondence relationship obtained in the above-mentioned step S10. The analysis region RA2 after being transformed into the coordinate system of the first projector 10-1 is an example of transformed data.

[0098] Figure 8 is a diagram for explaining the matching process between the first partial region R1 and the analysis region RA2. In step S63, a matching process such as the phase-limiting method is performed on the first partial region R1 and the analysis region RA2. In Figure 8 [reference], an example of the way in which the analysis region RA1 in the first partial region R1 that corresponds to the above-mentioned analysis region RA is used for this matching is illustrated. The analysis region RA1 is represented by the coordinate system of the first projector 10-1, that is, the coordinate system of the first drawing panel. The analysis region RA1 is a region in the first projected image G1 that corresponds to the analysis region RA of the first partial region R1. The analysis region RA1 is an example of drawing data. That is, the analysis region RA1 is the data that the first projector 10-1 inputs to the first drawing panel in order to draw the first partial region R1 including the analysis region RA on the first drawing panel. The analysis region RA1 is an example of drawing data, and it is sufficient to include at least the data corresponding to the analysis region RA of the first partial region R1.

[0099] In this way, the analysis region RA1 and the analysis region RA2 are each represented by the coordinate values (x, y) of the coordinate system of the first projector 10-1. That is, in the present embodiment, the matching process for calculating the offset between the first projected image G1 and the second projected image G2 and the coordinate system of the light modulator 15b (the first drawing panel) of the first projector 10-1 are calculated in a unified manner.

[0100] In addition, the analysis region RA1 can also be obtained by cutting it out from the first captured data D1 and then transforming it into the coordinate system of the first projector 10-1.

[0101] As described above, in the first partial region R1 of the first captured data D1, the brightness BR1 of the first projected image G1 is greater than zero, and the brightness BR2 of the second projected image G2 is zero. Therefore, by using the analysis region RA1, the position of the first projected image G1 can be detected without being affected by the second projected image G2. On the other hand, as described above, in the second partial region R2 of the second captured data D2, the brightness BR1 of the first projected image G1 is zero, and the brightness BR2 of the second projected image G2 is greater than zero. Therefore, by using the analysis region RA2, the position of the second projected image G2 can be detected without being affected by the first projected image G1.

[0102] In step S64, based on the result of the matching process, one or both of the offset amount and direction between the first projected image G1 and the second projected image G2 are detected. In Figure 8 the example shown, detection values (dx, dy) representing the amount and direction of the offset between the first projected image G1 and the second projected image G2 are obtained. The detection values (dx, dy) are obtained at multiple positions in the analysis regions RA1 and RA2.

[0103] Figure 9 is a diagram for explaining the calculation of the offset between the first projected image G1 and the second projected image G2 within the range of the overlapping region R. In step S70, the first projector 10-1 calculates a correction amount for correcting the first projected image G1 based on the offset detected in step S60. More specifically, in step S70, values around the outlier AB among the detection values (dx, dy) of the offsets of multiple parts calculated in step S60, excluding the outlier AB, are supplemented according to the relationship with other detection values. Thereby, a more accurate correction amount can be obtained. Information representing this correction amount is stored in the storage device 11 as correction amount information PA.

[0104] In step S70 of the present embodiment, the mixing range may also be maintained as the second range β. In addition, during the execution of step S70, the mixing range may be biased to the left of the corresponding region RC as Figure 5 shown, may be biased to the right of the corresponding region RC as Figure 6 shown, or may be located at the center of the corresponding region RC as shown in Figure 10 described later. Additionally, the mixing range may move within the corresponding region RC. In this case, it is preferable that the movement of the mixing range is synchronized with the vertical synchronization signal of the video data IMG1. Also, during the execution of step S70, the mixing range may be smaller than the second range β.

[0105] Figure 10This is a diagram for explaining the mixing range during the period of calculating the correction amount. In step S80, the first projector 10-1 is controlled according to the correction amount calculated in step S70. Thus, in step S80, as Figure 10 shown, the image group GG3 is projected onto the projection surface SC. The image group GG3 is the image group GG obtained by correcting the offset between the first projection image G1 and the second projection image G2 according to the correction amount information PA. This correction is performed by adjusting the position of the first projection image G1 relative to the position of the second projection image G2. For example, in such a way that the position of the first projection image G1 coincides with the position of the second projection image G2, at least any one of the shape, position, or orientation of the drawn image on the light modulator 15b of the first projector 10-1 is corrected.

[0106] In Figure 10 the example shown, the first mixing region R1b and the second mixing region R2b are arranged at the center of the corresponding region RC. The mixing range of the first mixing region R1b and the second mixing region R2b is the second range β. In addition, the mixing range in step S80 is not limited to Figure 10 the example shown, and it can be biased to the left side of the corresponding region RC as Figure 5 shown, or it can be biased to the right side of the corresponding region RC as Figure 6 shown.

[0107] As described above, the control method of the multi-projection system 100 includes step S20, step S30, step S40, step S50, and step S60.

[0108] Here, the first projector 10-1 used in the control method of the present embodiment has an optical device 15 and a processing device 12. The processing device 12 of the first projector 10-1 executes step S20, step S30, step S40, step S50, and step S60. Here, the processing device 20 controls the operation of the optical device 15 of the first projector 10-1 in step S20, and controls the operation of the second projector 10-2 so that the first image group GG1 is projected onto the projection surface SC. And, in step S40, the processing device 12 of the first projector 10-1 controls the operation of the optical device 15 of the first projector 10-1, and controls the operation of the second projector 10-2 so that the second image group GG2 is projected onto the projection surface SC.

[0109] In addition, the control method of the present embodiment is implemented by the processing device 12 executing the program PR1 as an example of a "computer". The program PR1 causes the processing device 12 to execute step S20, step S30, step S40, step S50, and step S60.

[0110] In the above control method, the first projector 10-1, and the program PR1, in the first state of step S20, the brightness BR1 of the first projected image G1 is greater than zero. On the other hand, the brightness BR2 of the second projected image G2 is zero. Therefore, the first captured data D1 obtained in step S30 does not include noise caused by the second projected image G2. Similarly, in the second state of step S40, the brightness BR1 of the first projected image G1 is zero. On the other hand, the brightness BR2 of the second projected image G2 is greater than zero. Therefore, the second captured data D2 obtained in step S50 does not include noise caused by the first projected image G1. In this way, the noise included in the first captured data D1 and the second captured data D2 can be reduced. As a result, the accuracy of the result of analyzing the first captured data D1 and the second captured data D2 in step S60 can be improved. As a result, a decrease in the detection accuracy of the shift between the first projected image G1 and the second projected image in the range of the overlapping region R can be suppressed.

[0111] As described above, the first projected image G1 has a non-overlapping region RN1 and a first partial region R1. The non-overlapping region RN1 is a region that does not overlap with the second projected image G2. The first partial region R1 is a region for overlapping with the second projected image G2. At least a part of the first partial region R1 is a region where the mixing process is performed. The mixing process changes the brightness in the arrangement direction DR in which the first projected image G1 and the second projected image G2 are arranged, so that the brightness of the overlapping region R when the first projected image G1 and the second projected image G2 are projected so as to overlap each other in the overlapping region R is the same as the brightness of the non-overlapping region RN1. Step S20 includes step S21. In step S21, the mixing range where the mixing process is performed in the first partial region R1 is changed from the first range α to a second range β smaller than the first range α, thereby adjusting the first partial region R1 to include a first state region R1a having the first state and a first mixing region R1b where the mixing process is performed. Step S30 includes step S31. In step S31, the imaging device 17 images the first state region R1a. In this way, since the first partial region R1 includes not only the first state region R1a but also the first mixing region R1b, even during the detection of the shift between the first projected image G1 and the second projected image in the range of the overlapping region R, a seamless composite image can be displayed in the corresponding region RC as part of the image group GG.

[0112] In addition, as described above, the brightness distribution in the arrangement direction DR when the mixing range is the second range β is different from the brightness distribution in the arrangement direction DR when the mixing range is the first range α. Therefore, the brightness distribution can be changed according to the range where the mixing process is performed in the first partial region R1.

[0113] In addition, as described above, step S60 includes step S61, step S62, step S63, and step S64. In step S61, an analysis region RA1 represented by the coordinate system of the imaging device 17 is cut out from the second captured data D2. In step S62, the analysis region RA1 is transformed into the coordinate system of the first projector 10-1. In step S63, a matching process is performed on a first partial region R1, which is a region for overlapping with the second projection image G2 in the first projection image G1 and is represented by the coordinate system of the first projector 10-1, and the analysis region RA1 transformed into the coordinate system of the first projector 10-1. In step S64, based on the result of the matching process, one or both of the amount and direction of the shift between the first projection image G1 and the second projection image G2 are detected. In this way, step S60 includes step S61, step S62, step S63, and step S64, whereby the shift between the first projection image G1 and the second projection image G2 in the range of the overlapping region R can be appropriately detected.

[0114] In addition, as described above, the control method of the present embodiment includes step S70 and step S80. In step S70, a correction amount for correcting one or both of the first projection image G1 and the second projection image G2 is calculated based on the detected shift. Step S80 controls one or both of the first projector 10-1 and the second projector 10-2 based on the correction amount. In the above manner, the shift between the first projection image G1 and the second projection image G2 in the range of the overlapping region R can be reduced. As a result, the image quality of the overlapping region can be improved.

[0115] Moreover, as described above, in step S70, the mixing range is maintained as the second range β. That is, the control method of the present embodiment includes maintaining the state where the mixing range is the second range β for a first period. In the above manner, compared with the case where the first period is not set, the change in the mixing range can be made less noticeable to the user. The first period is, for example, 10 seconds, but is not particularly limited as long as it is a finite time.

[0116] In addition, as described above, the control method of the present embodiment includes step S90 and step S110. Step S90 is a step of detecting the temperature of the projection optical system 15c-1, which is an example of the "first optical system" for the first projector 10-1 to project the first projection image G1, based on the output from the temperature sensor 18, which is an example of one or more "sensors". The temperature sensor 18 is, for example, a thermistor. "No" in step S110 is to intermittently perform the detection of the offset and the correction of the offset based on the output from one or more temperature sensors 18 until the temperature of the projection optical system 15c-1 reaches the target temperature. "Yes" in step S110 is to stop the detection of the offset and the correction of the offset based on the output from one or more temperature sensors 18 after the temperature of the projection optical system 15c reaches the target temperature. Through such step S90 and step S110, the correction of the offset can be performed regularly until the temperature of the projection optical system 15c-1 stabilizes. Thus, the appropriate detection of the offset can be performed in a timely manner.

[0117] 2. Second Embodiment

[0118] Hereinafter, a second embodiment of the present disclosure will be described. In the embodiments exemplified below, for elements having the same functions and actions as those in the first embodiment, the reference numerals used in the description of the first embodiment are used and the detailed description of each is appropriately omitted.

[0119] Figure 11 is a block diagram of the first projector 10-1 of the second embodiment. The first projector 10-1 of the present embodiment is configured in the same manner as the first projector 10-1 of the first embodiment, except that the program PR2 is used instead of the program PR1 of the first embodiment.

[0120] In the first projector 10-1 of the present embodiment, the processing device 12 functions as a projection control unit 12d, a shooting control unit 12b, and a correction unit 12c by executing the program PR2 stored in the storage device 11.

[0121] The projection control unit 12d controls the operations of the image processing circuits 14 and the optical devices 15 of the first projector 10-1 and the second projector 10-2 in the same manner as the projection control unit 12a of the first embodiment, except that the mixing range is expanded after the projection of the second image group GG2.

[0122] Figure 12 is a flowchart showing the flow of the control method of the second embodiment. The control method of the present embodiment is the same as the control method of the first embodiment, except that step S120 of the first embodiment is omitted and step S130 is added.

[0123] Step S130 is executed between step S50 and step S60. In step S130, the projection control unit 12d expands the mixing range, which is the range for performing the mixing process of the first projection image G1 and the second projection image G2, to the entire area of the overlapping region R. That is, in step S130, the projection control unit 12d changes the mixing range from the second range β to the first range α.

[0124] As described above, the control method of the present embodiment includes changing the mixing range in the first partial region R1 from the second range β to the first range α in step S130.

[0125] Figure 13 It is a diagram for explaining the mixing range during the period of calculating the correction amount. Steps S60 to S110 of the present embodiment are executed after performing the aforementioned step S130. Therefore, as Figure 13 shown, the control method of the present embodiment includes maintaining the mixing range in the state of the first range α during the entire period of calculating the correction amount in step S70. In addition, in Figure 13 the example shown, the first range α is the entire area of the corresponding region RC, but it is not limited thereto. The first range α only needs to be larger than the second range β.

[0126] According to the above second embodiment, it is also possible to improve the image quality in the overlapping region R of the projection image. As described above, the control method of the present embodiment further includes maintaining the mixing range in the state of the first range α during the entire period of calculating the correction amount. Therefore, during periods other than when necessary, appropriate mixing processing can be performed by expanding the mixing range. Thereby, the image quality of the overlapping region R can be improved.

[0127] 3. Variation

[0128] Each of the above-exemplified modes can be variably deformed in various ways. The following exemplify specific deformation modes that can be applied to the above-mentioned modes. Two or more modes arbitrarily selected from the following exemplifications can be appropriately combined within a non-contradictory range.

[0129] 3-1. Variation 1

[0130] In the above embodiment, a mode of performing the mixing process is exemplified, but it is not limited to this mode, and the first mixing region R1b and the second mixing region R2b can also be omitted.

[0131] 3-2. Variation 2

[0132] In the above-described embodiment, the method of obtaining the first captured data D1 and the second captured data D2 by using the imaging device 17 of the first projector 10-1 is illustrated, but it is not limited to this method. For example, the acquisition of the first captured data D1 and the second captured data D2 may be performed by using the imaging device 17 of the second projector 10-2 instead of or in addition to the imaging device 17 of the first projector 10-1.

[0133] 3-3. Variant Example 3

[0134] In step S70 of the above-described embodiment, the first projector 10-1 calculates the correction amount for correcting the first projected image G1 based on the offset detected in step S60, but it is not limited to this method. For example, the second projector 10-2 may have a correction function, and in step S70, the second projector 10-2 calculates the correction amount for correcting the second projected image G2 based on the offset detected in step S60, and corrects the drawn image of the light modulator 15b of the second projector 10-2 according to the correction amount. Alternatively, in step S70, both the first projector 10-1 and the second projector 10-2 may have a correction function, and both the first projector 10-1 and the second projector 10-2 calculate the correction amount for correcting the second projected image G2 based on the offset detected in step S60. When both the first projector 10-1 and the second projector 10-2 have a correction function, the correction amount is preferably half of the correction amount when only one of the first projector 10-1 and the second projector 10-2 has a correction function. Alternatively, the first projector 10-1 may calculate the correction amount for correcting the first projected image G1 and correct the drawn image of the light modulator 15b of the second projector 10-2 according to the correction amount. In this case, for example, it is preferable to reverse the sign of the correction amount calculated by the first projector 10-1 to transform the correction amount for the first projector 10-1 to correct the first projected image G1 into the correction amount for the second projector 10-2 to correct the drawn image of the light modulator 15b of the second projector 10-2.

[0135] In addition, part or all of the method of the present disclosure including the calculation of the correction amount may be performed separately by the terminal device 30, the processing device 12, and the processing device of the second projector 10-2 having the same function as the processing device 12, or they may share the respective steps included in the method of the present disclosure.

[0136] 3-4. Variant Example 4

[0137] In step S80 of the above-described embodiment, the first projector 10-1 is controlled according to the correction amount calculated in step S70, but it is not limited to this method. For example, the second projector 10-2 may have a correction function, and the correction amount information PA is sent from the first projector 10-1 to the second projector 10-2, and the second projector 10-2 is controlled based on the correction amount information PA. In this case, the image group GG is corrected by adjusting the positions of the first projected image G1 and the second projected image G2.

[0138] 3-5. Variant Example 5

[0139] At least one of the program PR1 of the first embodiment or the program PR2 of the second embodiment may also be provided in a state recorded on a computer-readable and non-transitory recording medium. The computer is, for example, the processing device 12 or the terminal device 30. In addition, at least one of the program PR1 of the first embodiment or the program PR2 of the second embodiment may also be provided in a manner of being downloaded from a server to a computer via a network.

[0140] 3-6. Variant Example 6

[0141] The first projector 10-1 of the first embodiment has a storage device 11, a processing device 12, a communication device 13, an image processing circuit 14, an optical device 15, an operation device 16, a photographing device 17, and a temperature sensor 18, but it is not limited to this method. For example, the first projector 10-1 may also have a storage device 11, a processing device 12, a communication device 13, an image processing circuit 14, an optical device 15, and an operation device 16, and does not have a photographing device 17 and a temperature sensor 18. That is, the photographing device 17 and the temperature sensor 18 may also be able to communicate with the first projector 10-1 and be separated from the first projector 10-1. The same applies to the second projector 10-2.

[0142] 3-7. Variant Example 7

[0143] In the case where it is "Yes" in step S110 of the first embodiment, according to the output from one or more temperature sensors 18, after the temperature of the projection optical system 15c-1 or both the temperature of the projection optical system 15c-1 and the temperature of the projection optical system 15c-2 reach the target temperature, the processing device 12 stops the detection of the offset and the correction of the offset, but it is not limited to this method. For example, the processing device 12 may also stop the detection of the offset and the correction of the offset when the elapsed time from the start of the detection of the offset and the correction of the offset exceeds the target time. For example, the processing device 12 may also stop the detection of the offset and the correction of the offset when the correction amount is below a specified value.

[0144] 3-8. Variant Example 8

[0145] The brightness BR1 in the first embodiment has a smooth change in the first mixing region R1b, but is not limited to this mode. For example, the brightness BR1 in the first embodiment may also have a stepwise change in the first mixing region R1b. Additionally, the brightness BR1 in the first embodiment may not have the first mixing region R1b. That is, the brightness BR1 in the first embodiment may also change in a stepped manner.

[0146] 3-9. Modification Example 9

[0147] In the first embodiment, the matching process for calculating the offset between the first projected image G1 and the second projected image G2 and the coordinate system of the light modulator 15b (the first display panel) of the first projector 10-1 are calculated in a unified manner, but are not limited to this mode. For example, the matching process may also be calculated in a unified manner with the coordinate system of the imaging device 17. In this case, the offset between the first projected image G1 and the second projected image G2 in the coordinate system of the imaging device 17 may also be transformed into the coordinate system of the light modulator 15b of the first projector 10-1 according to the correspondence obtained in step S10.

[0148] 3-10. Modification Example 10

[0149] In step S90 of the first embodiment, the temperature of the projection optical system 15c-1 is detected based on the output from the temperature sensor 18, but is not limited to this mode. For example, in step S90, the temperature of the projection optical system 15c-2 may be detected, or the temperatures of both the projection optical system 15c-1 and the projection optical system 15c-2 may be detected. That is, in step S90, at least one of the temperatures of the projection optical system 15c-1 and the projection optical system 15c-2 may also be detected.

[0150] Similarly, in step S110 of the first embodiment, it is determined whether the temperature of the projection optical system 15c-1 has reached the target temperature, but is not limited to this mode. For example, in step S110, it may be determined whether the temperature of the projection optical system 15c-2 has reached the target temperature, or it may be determined whether the temperatures of both the projection optical system 15c-1 and the projection optical system 15c-2 have reached the target temperature. That is, in step S110, it may also be determined whether at least one of the temperatures of the projection optical system 15c-1 and the projection optical system 15c-2 has reached the target temperature.

[0151] 4. Supplementary Note

[0152] Hereinafter, a summary of the present disclosure is appended.

[0153] (Supplementary Note 1) The control method of the first mode as a preferred example of the present disclosure is a control method in a multi-projection system. In this multi-projection system, an overlapping area is set. In this overlapping area, a part of the first projection image projected from the first projector overlaps with a part of the second projection image projected from the second projector on the projection surface. The control method includes: projecting a first image group having a first state onto the projection surface. The first state is that in a corresponding area corresponding to the overlapping area, the brightness of the first projection image is greater than zero and the brightness of the second projection image is zero; obtaining first shooting data by shooting the corresponding area in a state where the first image group is projected onto the projection surface; projecting a second image group having a second state onto the projection surface. The second state is that in the corresponding area, the brightness of the first projection image is zero and the brightness of the second projection image is greater than zero; obtaining second shooting data by shooting the corresponding area in a state where the second image group is projected onto the projection surface; and detecting an offset between the first projection image and the second projection image in the range of the overlapping area by analyzing the first shooting data and the second shooting data.

[0154] In the above mode, in the first state, the brightness of the first projection image is greater than zero. On the other hand, the brightness of the second projection image is zero. Therefore, in the area where the brightness of the second projection image is zero, the first shooting data does not contain noise caused by the second projection image. Similarly, in the area where the brightness of the first projection image is zero, the second shooting data does not contain noise caused by the first projection image. In this way, this control method can generate an area with reduced noise included in the first shooting data and the second shooting data. Thereby, the accuracy of the result of analyzing the first shooting data and the second shooting data can be improved. As a result, a decrease in the detection accuracy of the offset between the first projection image and the second projection image in the range of the overlapping area can be suppressed.

[0155] (Supplementary Note 2) In the second mode, which is a preferred example of the first mode, the first projected image has a non-overlapping region, i.e., a non-overlap region, that does not overlap with the second projected image, and a first partial region for overlapping with the second projected image. At least a part of the first partial region is a region where a blending process is performed. The blending process changes the brightness in the arrangement direction in which the first projected image and the second projected image are arranged, so that the brightness of the overlapping region when the first projected image and the second projected image are projected to overlap with each other in the overlapping region is the same as the brightness of the non-overlapping region; projecting the first image group onto the projection surface includes changing the blending range where the blending process is performed in the first partial region from a first range to a second range smaller than the first range, thereby adjusting the first partial region to include a first state region having the first state and a first blending region where the blending process is performed; obtaining the first captured data includes capturing the first state region. In the above mode, since the first partial region includes not only the region of the first state but also the first blending region, a seamless composite image can be displayed in the corresponding region even during the period of detecting the offset between the first projected image and the second projected image in the range of the overlapping region.

[0156] (Supplementary Note 3) In the third mode, which is a preferred example of the second mode, the brightness distribution in the arrangement direction when the blending range is the second range is different from the brightness distribution in the arrangement direction when the blending range is the first range. In the above mode, the brightness distribution can be changed according to the range where the blending process is performed in the first partial region.

[0157] (Supplementary Note 4) In the fourth mode, which is a preferred example of any one of the first to third modes, detecting the offset between the first projected image and the second projected image in the range of the overlapping region includes: cutting out an analysis region represented by the coordinate system of the imaging device from the second captured data; generating transformation data obtained by transforming the analysis region into the coordinate system of the rendering panel of the first projector; performing a matching process on the rendering data and the transformation data, where the rendering data represents the region corresponding to the analysis region in the first projected image and the region is represented by the coordinate system of the rendering panel; and detecting one or both of the amount and direction of the offset between the first projected image and the second projected image based on the result of the matching process. In the above mode, the offset between the first projected image and the second projected image in the range of the overlapping region can be appropriately detected.

[0158] (Supplementary Note 5) In the fifth mode, which is a preferred example of the second or third mode, it further includes: calculating a correction amount for correcting one or both of the first projection image and the second projection image based on the detected offset; and controlling one or both of the first projector and the second projector according to the correction amount. In the above mode, the offset between the first projection image and the second projection image in the range of the overlapping region can be reduced. Thereby, the image quality of the overlapping region can be improved.

[0159] (Supplementary Note 6) In the sixth mode, which is a preferred example of the fifth mode, it further includes: changing the mixing range from the second range to the first range in the first partial region; and maintaining the state where the mixing range is the first range for a first period. In the above mode, in the first period, which is a period other than when necessary, by setting a relatively wide mixing range, appropriate mixing processing can be performed from the user's perspective. Thereby, the image quality of the overlapping region can be improved.

[0160] (Supplementary Note 7) In the seventh mode, which is a preferred example of the fifth or sixth mode, it further includes: maintaining the state where the mixing range is the second range for a first period. In the above mode, by setting the first period as the period for maintaining the state of the second range, the change in the mixing range can be made unobtrusive.

[0161] (Supplementary Note 8) In the eighth mode, which is a preferred example of any one of the fifth to seventh modes, it further includes: detecting the temperature of the first optical system of the first projector for projecting the first projection image according to the output from one or more sensors; intermittently performing the detection and correction of the offset according to the output from the one or more sensors until the temperature of the first optical system reaches the target temperature; and stopping the detection and correction of the offset according to the output from the one or more sensors after the temperature of the first optical system reaches the target temperature. In the above mode, the correction of the offset can be performed regularly until the temperatures of the first optical system and the second optical system are stable. Thereby, appropriate detection of the offset can be performed in a timely manner.

[0162] (Supplementary Note 9) The projector of the 9th mode as a preferred example of the present disclosure is used as a first projector in a multi-projection system with an overlapping area set, and the overlapping area is used for a first projection image projected from the first projector and a second projection image projected from the second projector to overlap each other partially on a projection surface, and the projector includes an optical device and a processing device, and the processing device performs the following processing: controlling the action of the optical device and controlling the action of the second projector so that a first image group having a first state is projected onto the projection surface, and the first state is that in a corresponding area corresponding to the overlapping area, the brightness of the first projection image is greater than zero and the brightness of the second projection image is zero; by The method comprises the steps of: controlling the operation of the optical device and the operation of the second projector so as to project a second image group having a second state onto the projection surface, wherein the brightness of the first projection image is zero and the brightness of the second projection image is greater than zero in the corresponding area; obtaining second shooting data by causing the shooting device to shoot the corresponding area when the second image group is projected onto the projection surface; and detecting the displacement of the first projection image and the second projection image in the range of the overlapping area by analyzing the first shooting data and the second shooting data.

[0163] In the above manner, in the first state, the brightness of the first projection image is greater than zero, while the brightness of the second projection image is zero, so the first shooting data does not contain noise caused by the second projection image. Similarly, the second shooting data does not contain noise caused by the first projection image. In this way, the noise contained in the first shooting data and the second shooting data can be reduced. As a result, the accuracy of the result of analyzing the first shooting data and the second shooting data can be improved. As a result, the reduction in the detection accuracy of the offset between the first projection image and the second projection image in the range of the overlapping area can be suppressed.

[0164] (Supplementary Note 10) A program product as a 10th mode of a preferred example of the present disclosure is a program product used in a multi-projection system, wherein the multi-projection system is set with an overlapping area, and the overlapping area is used for overlapping a portion of a first projection image projected from a first projector and a portion of a second projection image projected from a second projector on a projection surface, wherein the program product causes a computer to execute the following processing: projecting a first image group having a first state onto the projection surface, wherein the first state is that in a corresponding area corresponding to the overlapping area, the brightness of the first projection image is greater than zero and the brightness of the second projection image is zero; The first shooting data is obtained by shooting the corresponding area by a shooting device in a state of being projected onto the projection surface; a second image group having a second state is projected onto the projection surface, wherein the second state is that in the corresponding area, the brightness of the first projection image is zero and the brightness of the second projection image is greater than zero; the second shooting data is obtained by shooting the corresponding area by the shooting device in a state of the second image group being projected onto the projection surface; and the displacement of the first projection image and the second projection image in the range of the overlapping area is detected by analyzing the first shooting data and the second shooting data.

[0165] In the above manner, in the first state, the brightness of the first projection image is greater than zero, while the brightness of the second projection image is zero, so the first shooting data does not contain noise caused by the second projection image. Similarly, the second shooting data does not contain noise caused by the first projection image. In this way, the noise contained in the first shooting data and the second shooting data can be reduced. As a result, the accuracy of the result of analyzing the first shooting data and the second shooting data can be improved. As a result, the reduction in the detection accuracy of the offset between the first projection image and the second projection image in the range of the overlapping area can be suppressed.

Claims

1. A control method in a multi-projection system, wherein the multi-projection system is set with an overlapping area, in which a portion of a first projection image projected from a first projector overlaps a portion of a second projection image projected from a second projector on a projection surface, the control method comprising: Projecting a first image group having a first state onto the projection surface, the first state being that in a corresponding area corresponding to the overlapping area, the brightness of the first projection image is greater than zero and the brightness of the second projection image is zero; Acquiring first photographing data by photographing the corresponding area in a state where the first image group is projected onto the projection surface; Projecting a second image group having a second state onto the projection surface, the second state being that in the corresponding area, the brightness of the first projection image is zero and the brightness of the second projection image is greater than zero; Acquiring second photographing data by photographing the corresponding area in a state where the second image group is projected onto the projection surface; as well as By analyzing the first imaging data and the second imaging data, a shift between the first projection image and the second projection image in the overlap region is detected.

2. The control method according to claim 1, wherein: The first projection image includes a non-overlapping area which is an area not overlapping with the second projection image and a first partial area for overlapping with the second projection image. At least a part of the first partial area is an area where a blending process is performed, and the blending process is to change the brightness in the arrangement direction of the first projection image and the second projection image so that the brightness of the overlap area when the first projection image and the second projection image are projected in a manner that they overlap each other in the overlap area is consistent with the brightness of the non-overlap area; Projecting the first image group onto the projection surface includes: changing the mixing range in which the mixing process is performed in the first partial area from a first range to a second range that is smaller than the first range, thereby adjusting the first partial area to include a first state area having the first state and a first mixing area in which the mixing process is performed, Acquiring the first imaging data includes imaging the first state area.

3. The control method according to claim 2, wherein: The distribution of brightness in the arrangement direction when the mixing range is the second range is different from the distribution of brightness in the arrangement direction when the mixing range is the first range.

4. The control method according to claim 1, wherein: Detecting the offset between the first projected image and the second projected image in the overlapping area includes: cutting out an analysis area represented by a coordinate system of the imaging device from the second imaging data; generating transformation data obtained by transforming the analysis area into a coordinate system of a drawing panel of the first projector; performing matching processing on rendering data and the transformation data, the rendering data representing an area corresponding to the analysis area in the first projection image, and the area being represented by a coordinate system of the rendering panel; as well as Based on the result of the matching process, one or both of the amount and direction of the shift between the first projected image and the second projected image are detected.

5. The control method according to claim 2, wherein: The control method further comprises: calculating a correction amount for correcting one or both of the first projection image and the second projection image based on the detected offset; and One or both of the first projector and the second projector are controlled based on the correction amount.

6. The control method according to claim 5, wherein: The control method further comprises: changing the mixing range from the second range to the first range in the first partial region; and The state in which the mixing range is the first range is maintained for a first period.

7. The control method according to claim 5, wherein: The control method further includes maintaining the state in which the mixing range is the second range for a first period.

8. The control method according to claim 5, wherein: The control method further comprises: detecting a temperature of a first optical system of the first projector for projecting the first projection image based on outputs from one or more sensors; intermittently performing the detection of the offset and the correction of the offset according to outputs from the one or more sensors until the temperature of the first optical system reaches a target temperature; as well as After the temperature of the first optical system reaches a target temperature based on the output from the one or more sensors, the detection of the offset and the correction of the offset are stopped.

9. A projector used as a first projector in a multi-projection system having an overlapping area set so that a first projection image projected from the first projector and a second projection image projected from a second projector partially overlap each other on a projection surface, The projector comprises an optical device and a processing device. The processing device performs the following processing: Controlling the operation of the optical device and the operation of the second projector so as to project a first image group having a first state onto the projection surface, the first state being that in a corresponding area corresponding to the overlapping area, the brightness of the first projected image is greater than zero and the brightness of the second projected image is zero; Acquiring first photographing data by causing a photographing device to photograph the corresponding area in a state where the first image group is projected onto the projection surface; Controlling the operation of the optical device and the operation of the second projector so as to project a second image group having a second state onto the projection surface, wherein the second state is that in the corresponding area, the brightness of the first projection image is zero and the brightness of the second projection image is greater than zero; Acquiring second photographing data by causing the photographing device to photograph the corresponding area in a state where the second image group is projected onto the projection surface; as well as By analyzing the first imaging data and the second imaging data, a shift between the first projection image and the second projection image in the overlap region is detected.

10. A program product for use in a multi-projection system, the multi-projection system being set with an overlapping area for partially overlapping a first projection image projected from a first projector and a second projection image projected from a second projector on a projection surface, the program product causing a computer to execute the following processing: Projecting a first image group having a first state onto the projection surface, the first state being that in a corresponding area corresponding to the overlapping area, the brightness of the first projection image is greater than zero and the brightness of the second projection image is zero; Acquiring first photographing data by photographing the corresponding area with a photographing device in a state where the first image group is projected onto the projection surface; Projecting a second image group having a second state onto the projection surface, the second state being that in the corresponding area, the brightness of the first projection image is zero and the brightness of the second projection image is greater than zero; Acquiring second photographing data by photographing the corresponding area with the photographing device in a state where the second image group is projected onto the projection surface; as well as By analyzing the first imaging data and the second imaging data, a shift between the first projection image and the second projection image in the overlap region is detected.

Images