Projection method and projection system

Abstract

This invention provides a projection method that improves the display accuracy of images displayed by tiling projection. [Solution] The projection method includes, when superimposing a portion of the image projected by projector 100A and a portion of the image projected by projector 100B, acquiring information indicating the resolution of the tiling area 20 of the projection surface 10 on which the image is displayed, based on a first captured image and a second captured image; acquiring parameters for correcting the image projected by projector 100A, based on a first captured image and a second captured image; generating a first projected image based on an input image, information indicating the resolution, and a first correction parameter; and generating a second projected image based on an input image, information indicating the resolution, and a second correction parameter.

Description

Technical Field

[0001] The present invention relates to a projection method and a projection system.

Background Art

[0002] Conventionally, tiling projection is known in which images projected by a plurality of projectors are arranged on a projection surface to display a single image. For example, the projector described in Patent Document 1 acquires resolution information from other connected projectors, and notifies an image supply device that supplies image data of the resolution of the projection area in tiling projection calculated by adding the resolutions of each projector.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the resolution of the projection area calculated by adding the resolutions may be different from the resolution of the actual projection area, and in order to display an image more accurately on the projection surface, it is necessary to obtain the resolution of the projection area more accurately.

Means for Solving the Problems

[0005] The projection method of this disclosure includes: acquiring a first image captured by a first projector capturing a projection surface on which a first image is projected; acquiring a second image captured by a second projector capturing the projection surface on which a second image is projected; and, when displaying a tiling image in the projection area of ​​the projection surface using the first image projected by the first projector onto the projection surface and the second image projected by the second projector onto the projection surface, acquiring information indicating the resolution of the projection area based on the first and second images; and setting a first correction parameter for correcting the image projected by the first projector. The projection method includes: acquiring a first image and a second image based on the first image and the second image; acquiring a second correction parameter for correcting the image projected by the second projector based on the first image and the second image; the first projector projecting the first projection image, which is generated based on the input image, the resolution information, and the first correction parameter, onto the projection surface; and the second projector projecting the second projection image, which is generated based on the input image, the resolution information, and the second correction parameter, onto the projection surface.

[0006] The projection system of this disclosure includes an imaging device that acquires a first image captured of a projection surface on which a first projector projects a first image, and a second image captured of the projection surface on which a second projector projects a second image, and when a tiling image is displayed in the projection area of ​​the projection surface by the first projection image projected by the first projector onto the projection surface and the second projection image projected by the second projector onto the projection surface, information indicating the resolution of the projection area is acquired based on the first image and the second image, and a first correction parameter for correcting the image projected by the first projector. The projection system comprises: a control device that performs the actions of acquiring a second correction parameter based on the first and second captured images and correcting the image projected by the second projector based on the first and second captured images; a first projector that projects a first projection image onto the projection surface, which is generated based on an input image, information indicating the resolution, and the first correction parameter; and a second projector that projects a second projection image, which is generated based on the input image, information indicating the resolution, and the second correction parameter.

[0007] The projection system of this disclosure includes an imaging device that acquires a first image captured of a projection surface on which a first projector projects a first image, and a second image captured of the projection surface on which a second projector projects a second image, and when displaying a tiling image in the projection area of ​​the projection surface using the first image projected by the first projector and the second image projected by the second projector, it acquires information indicating the resolution of the projection area based on the first image and the second image, acquires a first correction parameter for correcting the image projected by the first projector based on the first image and the second image, and acquires a second correction parameter for correcting the image projected by the second projector based on the first image A projection system comprising: a control device that performs the following actions: acquiring based on the second captured image; transmitting information indicating the resolution of the projection area and the first correction parameter to the first projector; transmitting information indicating the resolution of the projection area and the second correction parameter to the second projector; transmitting an EDID including the information indicating the resolution to the control device when connected to the control device; the first projector that projects a first projection image generated based on an input image, the information indicating the resolution and the first correction parameter; and the second projector that projects a second projection image generated based on the input image, the information indicating the resolution and the second correction parameter onto the projection surface. [Brief explanation of the drawing]

[0008] [Figure 1] System configuration diagram of the projection system. [Figure 2] A block diagram showing the configuration of the projector. [Figure 3] A diagram showing the configuration of the image projection unit. [Figure 4] A block diagram showing the configuration of an information processing device. [Figure 5] A diagram showing the projection surface. [Figure 6] A flowchart illustrating the operation of an information processing device. [Figure 7] A flowchart illustrating the operation of a projector. [Figure 8] A flowchart showing the details of step SA1. [Figure 9] A diagram showing an example of a pattern image laid out in a panel area. [Figure 10] This figure shows the second coordinate system, plotting the four vertices of the pattern image. [Figure 11] This diagram shows the state where two lines L1 and L2 are set on the second coordinate system. [Figure 12] This diagram shows the state after translating two lines L1 and L2, which are set on the second coordinate system. [Figure 13] A diagram showing the intersections of lines L1 and L2 with the pattern image. [Figure 14] A diagram showing the projection transformation from the second coordinate system to the first coordinate system. [Figure 15] A diagram showing the projection transformation from the second coordinate system to the third coordinate system. [Figure 16] A diagram showing the projection transformation from the second coordinate system to the third coordinate system. [Modes for carrying out the invention]

[0009] [Projection system configuration] Figure 1 shows the system configuration of projection system 1. The projection system 1 comprises multiple projectors 100, a camera 200, and an information processing device 300. The camera 200 corresponds to an imaging device, and the information processing device 300 corresponds to a control device.

[0010] This embodiment describes a case where the projection system 1 includes two projectors 100, projector 100A and projector 100B. However, the number of projectors 100 is not limited to two, and may be three or more. In the following, projectors 100A and projector 100B will be collectively referred to as projector 100.

[0011] The camera 200 and the information processing device 300 are wired-connected by a cable 3 compliant with a standard such as USB (Universal Serial Bus). Also, the information processing device 300, the projector 100A, and the projector 100B are daisy-chain connected by a cable 5. For the cable 5, a cable compliant with a standard such as HDMI (High-Definition Multimedia Interface), DisplayPort, or USB Type-C is used, for example. HDMI is a registered trademark.

[0012] Also, the projector 100A, the projector 100B, and the information processing device 300 are connected to a wireless network 9 constructed by a wireless router 7. The wireless router 7 functions as an access point and relays the transmission and reception of data between devices connected to the wireless network 9. Also, the wireless router 7 functions as a router, connects to a network such as the Internet via a modem not shown, and relays the transmission and reception of data between a device connected to the network and a device connected to the wireless network 9.

[0013] The camera 200 includes an imaging lens and an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary MOS), and generates an imaging image according to a user operation or an instruction from the information processing device 300. The camera 200 transmits the generated imaging image to the information processing device 300. The camera 200 is installed so that the entire projection surface 10 is included within the angle of view. Therefore, the imaging image of the camera 200 includes an image obtained by imaging the entire projection surface 10. The illustration of the imaging lens and the image sensor is omitted.

[0014] The information processing device 300 is a device that supplies an image signal containing image data to the projector 100A. The projector 100A outputs the image signal received from the information processing device 300 to the projector 100B, which is daisy-chained. Projectors 100A and 100B extract the image data contained in the received image signal and project image light based on the extracted image data onto the projection surface 10.

[0015] The information processing device 300 can be, for example, a notebook PC (Personal Computer), a desktop PC or tablet PC, a smartphone, or a PDA (Personal Digital Assistant).

[0016] [Projector Configuration] Figure 2 is a block diagram showing the configuration of projector 100A. Projectors 100A and 100B have substantially the same configuration. Therefore, the configuration of projector 100A will be described below, and the configuration of projector 100B will be omitted. In addition, to distinguish between the configurations of projector 100A and projector 100B, the configuration of projector 100A will be denoted with the designation "A", and the configuration of projector 100B will be denoted with the designation "B".

[0017] The projector 100A includes a remote control receiver 101A, a first wireless interface 110A, a first wired interface 120A, an image processing unit 130A, a frame memory 135A, an image projection unit 140A, and a first control unit 150A. Hereinafter, interfaces will be abbreviated as I / F.

[0018] The remote control receiver 101A receives an infrared signal transmitted from the remote control 105 and outputs an operation signal corresponding to the operation indicated by the received infrared signal to the first control unit 150A.

[0019] The first wireless interface 110A is an interface for wireless communication with external devices, including the information processing device 300. For example, the first wireless interface 110A may include a wireless LAN (Local Area Network) card and communicate wirelessly with external devices to send and receive various types of information. The first wireless interface 110A may also be configured to include an interface circuit and a wireless antenna.

[0020] The first wired I / F 120A is an interface that is connected to the information processing device 300 and the projector 100B in a communicative manner, and receives image signals transmitted from the information processing device 300 and transmits the received image signals to the projector 100B. The first wired interface, I / F120A, uses interfaces that support standards such as HDMI, DisplayPort, and USB Type-C.

[0021] The first wired interface 120A includes an input terminal 121A, a receiving circuit 123A, a transmitting circuit 125A, and an output terminal 127A.

[0022] Input terminal 121A is connected to cable 5 and is a terminal that receives image signals via cable 5. The receiving circuit 123A receives the image signal input to the input terminal 121A. The receiving circuit 123A extracts the synchronization signal and image data contained in the received image signal. The receiving circuit 123A outputs the extracted synchronization signal and image data to the image processing unit 130A. The receiving circuit 123A also outputs the received image signal to the transmitting circuit 125A.

[0023] Output terminal 127A is connected to cable 5 and outputs the image signal to cable 5. The transmitting circuit 125A transmits data input from the first control unit 150A and image signals input from the receiving circuit 123A to the subsequent projector 100B via cable 5.

[0024] The image processing unit 130A is connected to the frame memory 135A. The image processing unit 130A processes the image data input from the first wired interface 120A into the frame memory 135A. The frame memory 135A is composed of, for example, SDRAM (Synchronous Dynamic Random Access Memory).

[0025] The image processing unit 130A performs image processing on the image data expanded in the frame memory 135A, such as resolution conversion or resizing, distortion correction, shape correction, digital zoom, and adjustment of image hue and brightness. The image processing unit 130A executes the image processing specified by the first control unit 150A and, if necessary, uses parameters input from the first control unit 150A to perform the processing. The image processing unit 130A can also, of course, combine and execute multiple of the above image processing operations. The image processing unit 130A reads the processed image data from the frame memory 135A and outputs the read image data to the image projection unit 140A.

[0026] The image processing unit 130A and the frame memory 135A are composed of, for example, integrated circuits. Integrated circuits include LSIs (Large Scale Integrated Circuits), ASICs (Application Specific Integrated Circuits), PLDs (Programmable Logic Devices), FPGAs (Field-Programmable Gate Arrays), SoCs (System-on-a-chip), etc. Furthermore, analog circuits may be included as part of the configuration of the integrated circuit, and the first control unit 150A may be configured in combination with an integrated circuit.

[0027] Figure 3 shows the configuration of the image projection unit 140A. Here, the configuration of the image projection unit 140A will be explained with reference to Figure 3. The image projection unit 140A includes a light source 141A, an optical modulator 142A, an optical unit 145A, and a panel drive unit 146A.

[0028] The light source 141A includes a discharge-type lamp such as an ultra-high pressure mercury lamp or a metal halide lamp, or a solid-state light source such as a light-emitting diode or a semiconductor laser. The light emitted from the light source 141A is incident on the optical modulator 142A.

[0029] The optical modulator 142A includes a color separation optical system that separates light incident from the light source 141A into red, blue, and green light, a plurality of optical modulators that modulate each color light, and a color synthesis optical system that synthesizes each color light to generate image light. The illustrations of the color separation optical system and the color synthesis optical system are omitted. The optical modulator 142A of this embodiment includes a transmissive liquid crystal panel 143A as an optical modulator. The liquid crystal panel 143A is composed of, for example, a transmissive liquid crystal panel in which liquid crystal is sealed between a pair of transparent substrates. Liquid crystal panels 143A are provided for each of the red, green, and blue colors. In Figure 3, the liquid crystal panel 143A into which red color light is incident is denoted as liquid crystal panel 143A(R). Similarly, in Figure 3, the liquid crystal panel 143A into which green color light is incident is denoted as liquid crystal panel 143A(G). Similarly, in Figure 3, the liquid crystal panel 143A into which blue colored light is incident is denoted as liquid crystal panel 143A(B). Furthermore, when referring collectively to liquid crystal panels 143A(R), 143A(G), and 143A(B), it is denoted as liquid crystal panel 143A.

[0030] Liquid crystal panels 143A(R), 143A(G), and 143A(B) are provided with panel regions 144A(R), 144A(G), and 144A(B), respectively, in which multiple pixels are arranged in a matrix. Hereinafter, when referring to panel regions 144A(R), 144A(G), and 144A(B) collectively, they will be referred to as panel region 144A. Panel regions 144A(G) and 144A(B) are not shown in the diagram.

[0031] Image data is input to the optical modulator 142A from the image processing unit 130A. A drive voltage corresponding to the input image data is applied to each pixel of the panel area 144A by the panel drive unit 146A. As a result, each pixel of the panel area 144A is set to a light transmittance corresponding to the image data. Light emitted from the light source 141A passes through the liquid crystal panel 143A, modulating the light pixel by pixel and generating colored light corresponding to the image data. The generated colored light is combined pixel by pixel by a color synthesis optical system (not shown) to form image light representing a color image.

[0032] The optical modulation element of the optical modulation device 142A is not limited to a transmissive liquid crystal panel; for example, it may be a reflective liquid crystal panel or a DMD (Digital Micromirror Device).

[0033] The optical unit 145A, equipped with a projection lens (not shown), projects the image light modulated by the optical modulator 142A onto the projection surface 10. As a result, an image corresponding to the image light is displayed on the projection surface 10.

[0034] The first control unit 150A is a computer device comprising a first storage unit 160A and a first processor 170A.

[0035] The first storage unit 160A includes, for example, ROM (Read Only Memory) and RAM (Random Access Memory). The ROM stores the control program 161 that controls the operation of projector 100A, as well as various setting information. This setting information includes information on the number and placement of projectors 100 used for tiling projection, which will be described later. The unit count information indicates the number of projectors (100) participating in the tiling projection. The placement information indicates the placement of each projector 100. In this embodiment, the information indicates that projector 100A is positioned to the left of the projection surface 10, and displays an image in the projection area 11, which is the area to the left of the projection surface 10. Furthermore, the ROM stores the pattern image 30 that the projector 100A displays on the projection surface 10. Details of the pattern image 30 will be explained with reference to Figure 9. The RAM is used as the arithmetic area of ​​the first processor 170A.

[0036] The first processor 170A is an arithmetic processing unit equipped with a processor such as a CPU (Central Processing Unit) or an MPU (Micro-Processing Unit). The first processor 170A may be composed of a single processor or of multiple processors. Furthermore, the first processor 170A may be composed of part or all of the first storage unit 160A or an SoC integrated with other circuits. Also, the first processor 170A may be composed of a combination of a CPU that executes programs and a DSP (Digital Signal Processor) that performs predetermined arithmetic processing. Moreover, all of the functions of the first processor 170A may be implemented in hardware, or it may be configured using programmable devices.

[0037] [Configuration of the information processing device] Figure 4 is a block diagram showing the configuration of the information processing device 300. The configuration of the information processing device 300 will be explained with reference to Figure 4. The information processing device 300 includes a second wireless I / F 310, a second wired I / F 320, a third wired I / F 330, a display unit 340, an operation unit 350, and a second control unit 360.

[0038] The second wireless interface 310 is an interface for wireless communication with external devices, including the projector 100. For example, the second wireless interface 310 may be equipped with a wireless LAN card and communicate wirelessly with external devices to send and receive various information. The second wireless interface 310 may also be configured to include an interface circuit and a wireless antenna.

[0039] The second wired interface 320 is connected to the camera 200 via cable 3 and receives captured images transmitted from the camera 200. The second wired interface 320 may be equipped with a terminal compliant with the USB standard, and a data communication interface such as USB may be used. The second wired interface 320 may also be configured to include terminals and interface circuits compliant with other communication standards.

[0040] The third wired interface 330 is connected to the projector 100A via cable 5 and transmits image signals to the projector 100A. The third wired interface 330 uses an interface equipped with terminals compatible with standards such as HDMI, DisplayPort, and USB Type-C. The third wired interface 330 may also be configured to include an interface circuit.

[0041] The display unit 340 includes a display panel 345 such as a liquid crystal panel or an organic EL (Electro-Luminescence) panel. The display unit 340 displays a display screen on the display panel 345 according to the control of the second control unit 360.

[0042] The operation unit 350 includes, for example, an input device such as a mouse or keyboard, and accepts user input. The operation unit 350 outputs an operation signal corresponding to the received operation to the second control unit 360. Alternatively, the operation unit 350 may be configured with a touch sensor that detects touch operations on the display panel 345 of the display unit 340. In this case, the operation unit 350 outputs coordinate information indicating the position of the display panel 345 where the touch operation was detected to the second control unit 360.

[0043] The second control unit 360 is a computer device comprising a second storage unit 370 and a second processor 380.

[0044] The second storage unit 370 includes, for example, ROM and RAM. The ROM stores control programs 371 that control the operation of each part of the information processing device 300, content files 373, and various setting information. The content files 373 are, for example, files containing content such as videos, still images, and documents. The RAM is used as the arithmetic area of ​​the second processor 380. The second storage unit 370 may also be configured to include auxiliary storage devices such as HDD (Hard Disk Drive) or SSD (Solid State Drive).

[0045] The second processor 380 is an arithmetic processing unit equipped with a processor such as a CPU or MPU. The second processor 380 may be composed of a single processor or multiple processors. Furthermore, the second processor 380 may be composed of part or all of the second memory unit 370, or an integrated SoC with other circuits. Also, the second processor 380 may be composed of a combination of a CPU that executes programs and a DSP that performs predetermined arithmetic processing. Moreover, all the functions of the second processor 380 may be implemented in hardware, or they may be configured using programmable devices.

[0046] [About tiling projection] Figure 5 shows the projection surface 10. Here, we will describe the area of ​​the projection surface 10 onto which projectors 100A and 100B project image light. The maximum area of ​​image light that projector 100A can project onto the projection surface 10 is called the projection area 11. The projection area 11 is the area of ​​the projection surface 10 where the image is displayed when the image is projected onto the entire surface of the panel area 144A of the liquid crystal panel 143A. The maximum area of ​​image light that projector 100B can project onto the projection surface 10 is called the projection area 13. The projection area 13 is the area of ​​the projection surface 10 where the image is displayed when the image is projected onto the entire surface of the panel area 144B of the liquid crystal panel 143B. As shown in Figure 1, projector 100A is positioned on the left side of the projection surface 10 in the drawing view, and projector 100B is positioned on the right side of the projection surface 10 in the drawing view. Therefore, the projection area 11 is the area on the left side of the projection surface 10, and the projection area 13 is the area on the right side of the projection surface 10.

[0047] In this embodiment, the display system 1 causes projectors 100A and 100B to perform tiling projection. Tiling projection is a projection method that uses multiple projectors 100 to display an image on a projection surface 10, and combines the images displayed by each projector 100 on the projection surface 10 to display a single large image.

[0048] In tiling projection, it is preferable that each projector 100 is adjusted so that the images displayed by each projector 100 overlap. This is done to make the boundaries between the images displayed on the projection surface 10 less noticeable. The area where the projectionable area 11 and the projectionable area 13 overlap is called the overlapping area 15. In this embodiment, as seen in the drawing, the right edge of the projectionable area 11 and the left edge of the projectionable area 13 overlap to form the overlapping area 15.

[0049] Furthermore, the area in which projectors 100A and 100B actually display images through tiling projection is called the tiling area 20. The tiling area 20 corresponds to the projection area of ​​the projection surface 10. If projector 100A displays an image in projection area 11 and projector 100B displays an image in projection area 13, the images displayed on the projection surface 10 will not be combined into a single large screen image. Therefore, the information processing device 300 generates parameters to correct the display range and shape of the image for projectors 100A and 100B, and transmits the generated parameters to projectors 100A and 100B via wireless communication. Projectors 100A and 100B project the corrected images onto the projection surface 10 according to the received parameters, so that the images are displayed in the tiling area 20 of the projection surface 10. The images displayed in the tiling area 20 correspond to the tiling image. The image that projector 100A projects onto the projection surface 10, which has been corrected according to the received parameters, corresponds to the first projection image. The image that projector 100B projects onto the projection surface 10, which has been corrected according to the received parameters, corresponds to the second projection image.

[0050] [Regarding the operation of the information processing device and projector] Figure 6 is a flowchart showing the operation of the information processing device 300. First, the operation of the information processing device 300 will be explained with reference to the flowchart shown in Figure 6. The second control unit 360 of the information processing device 300 generates parameters (step SA1). The parameters generated by the second control unit 360 include three parameters: a parameter indicating the resolution of the tiling area 20, a parameter indicating the width of the superimposed area 15, and a parameter for shape correction. The parameters generated by the second control unit 360 correspond to the first correction parameter and the second correction parameter. The details of step SA1 will be explained in detail with reference to the flowchart in Figure 8.

[0051] Next, the second control unit 360 transmits the generated parameters to projectors 100A and 100B via the second wireless interface 310 (step SA2).

[0052] Next, the second control unit 360 determines whether or not it has received EDID (Extended Display Identification Data) from the projector 100A via the third wired I / F 330 (step SA3). When projector 100A receives parameters from information processing device 300, it stores the received parameters in the first storage unit 160A. Projector 100A also acquires a parameter indicating the resolution of the tiling area 20 and sets the acquired resolution parameter as the EDID. Projector 100A transmits the set EDID to information processing device 300 via the first wired interface 120A. If the second control unit 360 has not received the EDID (step SA3 / NO), it waits until the EDID is received.

[0053] When the second control unit 360 receives the EDID (step SA3 / YES), it recognizes the projector 100A connected by cable 5 as a display screen with the resolution set by the EDID.

[0054] Next, the second control unit 360 determines whether or not the operation unit 350 has instructed it to play the content file 373 (step SA4). If the second control unit 360 has not been instructed to play the content file 373 (step SA4 / NO), it waits until it is instructed to play it.

[0055] When the second control unit 360 is instructed to play the content file 373 (step SA4 / YES), it acquires image data from the content file 373 and generates an image signal containing the acquired image data. The second control unit 360 then transmits the generated image signal to the projector 100A (step SA5). Here, the second control unit 360 converts the resolution of the image data contained in the content file 373 to the resolution corresponding to the EDID if the resolution of the image data is different from the display resolution of the projector 100A recognized by the EDID. The second control unit 360 then transmits an image signal containing the converted image data to the projector 100A.

[0056] Next, the second control unit 360 determines whether playback of the content file 373 has finished or whether it has received an instruction to end playback (step SA6). If playback of the content file 373 has not finished, or if an instruction to stop playback has not been received (step SA6 / NO), the second control unit 360 waits until playback of the content file 373 finishes or an instruction to stop playback is received. Furthermore, the second control unit 360 terminates the transmission of the image signal (step SA7) when playback of the content file 373 is completed or when it receives an instruction to terminate playback (step SA6 / YES).

[0057] Figure 7 is a flowchart showing the operation of projector 100A. Next, the operation of projector 100A will be explained with reference to the flowchart shown in Figure 7. The first control unit 150A determines whether or not it has received a parameter from the information processing device 300 (step SB1). If the first control unit 150A has not received a parameter from the information processing device 300 (step SB1 / NO), it waits until it receives a parameter.

[0058] When the first control unit 150A receives a parameter (step SB1 / YES), it stores the received parameter in the first storage unit 160A. Next, the first control unit 150A obtains a parameter indicating the resolution of the tiling area 20 from the received parameter and sets the obtained resolution parameter as the EDID (step SB2). The first control unit 150A transmits the set EDID to the information processing device 300 via the first wired I / F 120A (step SB3).

[0059] Next, the first control unit 150A determines the area that the projector 100A is responsible for displaying, based on the received parameters (step SB4). The first control unit 150A stores information such as the number of projectors 100 used for tiling projection and their arrangement as setting information. Based on the setting information and the received parameters, the first control unit 150A determines the area in the tiling area 20 that each projector 100A is responsible for displaying.

[0060] Next, the first control unit 150A determines whether the first wired I / F 120A has received the image signal transmitted from the information processing device 300 (step SB5). If the first control unit 150A has not received the image signal (step SB5 / NO), it waits until it receives the image signal.

[0061] When the first wired interface 120A receives an image signal (step SB5 / YES), the first control unit 150A transmits the received image signal to the subsequent projector 100B (step SB6).

[0062] Next, the projector 100A extracts the image data contained in the image signal using the receiving circuit 123A of the first wired I / F 120A (step SB7). The extracted image data is output to the image processing unit 130A.

[0063] The image processing unit 130A extracts image data corresponding to the area determined in step SB4 from the input image data (step SB8). Next, the image processing unit 130A applies transparency processing to the region corresponding to the superimposed region 15 of the extracted image data (step SB9). Transparency processing is a process that adjusts the brightness of the region corresponding to the superimposed region 15 of the image data.

[0064] Furthermore, the image processing unit 130A corrects the shape of the transparency-processed image data based on the shape correction parameters received in step SB1 (step SB10). The image processing unit 130A outputs the shape-corrected image data to the image projection unit 140A.

[0065] The image projection unit 140A projects the input image data onto the panel area 144A of the liquid crystal panel 143A. Light emitted from the light source 141A passes through the panel area 144A of the liquid crystal panel 143A, generating image light corresponding to the image data. The generated image light is projected onto the projection surface 10 by the optical unit 145A (step SB11).

[0066] Next, the first control unit 150A determines whether or not the reception of the image signal has finished (step SB12). If the reception of the image signal has not finished (step SB12 / NO), the first control unit 150A returns to the process in step SB6. Also, if the reception of the image signal has finished (step SB12 / YES), the first control unit 150A terminates this processing flow.

[0067] Figure 8 is a flowchart detailing step SA1. Next, we will describe the details of step SA1, namely the operation in which the information processing device 300 generates three parameters: a parameter indicating the resolution of the tiling area 20, a parameter indicating the width of the superimposed area 15, and a parameter for shape correction.

[0068] First, the second control unit 360 transmits a projection instruction for the pattern image 30 to the projector 100A (step SA101). In this embodiment, the case in which the projectors 100A and 100B have the pattern image 30 stored in advance is described, but the second control unit 360 may transmit the pattern image 30 to the projectors 100A and 100B.

[0069] When projector 100A receives a projection instruction for pattern image 30 from information processing device 300, it unfolds the pattern image 30 across the entire surface of panel area 144A and generates image light corresponding to the pattern image 30. Projector 100A projects the generated image light onto projection surface 10 using optical unit 145A. As a result, the pattern image 30 is displayed in the projectionable area 11 of projection surface 10. The pattern image 30 displayed in the projectionable area 11 by projector 100A corresponds to the first image.

[0070] Figure 9 shows an example of a pattern image 30 deployed on the panel area 144A of the projector 100A. In the following, the coordinates set in panel area 144A will be referred to as the first coordinates. The top left of the first coordinates will be referred to as the origin O, the horizontal axis as the G axis, and the vertical axis as the H axis. As shown in Figure 9, pattern image 30 is a pattern image in which a first rectangular figure 31 and a second rectangular figure 33, which are identical in shape and size, are alternately arranged in the vertical and horizontal directions of the panel area 144A. The first rectangular figure 31 is a black rectangle, and the second rectangular figure 33 is a white rectangle. The color of the first rectangular figure 31 is not limited to black, and the color of the second rectangular figure 33 is not limited to white. The colors of the first rectangular figure 31 and the second rectangular figure 33 just need to be different; for example, complementary colors on the color wheel may be used for the first rectangular figure 31 and the second rectangular figure 33.

[0071] Next, the second control unit 360 transmits an imaging instruction to the camera 200 (step SA102). Upon receiving the imaging instruction, the camera 200 performs imaging and generates an image. The generated image is called the first image. The first image captures the projection surface 10 with the pattern image 30 displayed in the projectionable area 11. The camera 200 transmits the generated first image to the information processing device 300.

[0072] The second control unit 360 determines whether or not it has received the first captured image from the information processing device 300 (step SA103). If the second control unit 360 has not received the first captured image (step SA103 / NO), it waits until it receives the first captured image.

[0073] When the second control unit 360 receives the first captured image (step SA103 / YES), it stores the received first captured image in the second storage unit 370. Next, the second control unit 360 sends a projection instruction for the pattern image 30 to the projector 100B (step SA104), and then sends an imaging instruction to the camera 200 (step SA105).

[0074] The projector 100B displays a pattern image 30 in the projection area 13 of the projection surface 10, and the camera 200 performs imaging to generate an image. The generated image is called the second image. The second image captures the projection surface 10 with the pattern image 30 displayed in the projection area 13. The camera 200 transmits the generated second image to the information processing device 300. The pattern image 30 displayed in the projection area 13 by the projector 100B corresponds to the second image.

[0075] The second control unit 360 determines whether or not it has received the second captured image from the information processing device 300 (step SA106). If the second control unit 360 has not received the second captured image (step SA106 / NO), it waits until it receives the second captured image. When the second control unit 360 receives the second captured image (step SA106 / YES), it stores the received second captured image in the second storage unit 370.

[0076] Next, the second control unit 360 analyzes the first captured image and detects the coordinates of the four vertices that are the four corners of the pattern image 30 (step SA107). For example, the second control unit 360 detects the coordinates of the intersection points where the four vertices of the first rectangular figure 31 and the second rectangular figure 33, which are adjacent in the same row, and the second rectangular figure 33 and the first rectangular figure 31, which are adjacent in the row below these figures, intersect. Here, the detected coordinates are the second coordinates. The first rectangular shape 31 and the second rectangular shape 33, which are placed at the four corners of the pattern image 30 and are adjacent to each other in the same row, and the second rectangular shape 33 and the first rectangular shape 31, which are adjacent to each other in the row below these shapes, correspond to pre-defined shapes.

[0077] The second coordinate system is the coordinate set for the images captured by camera 200, that is, the first and second captured images. The second coordinate system has the upper left corner of the captured image as the origin O, with the horizontal axis being the J axis and the vertical axis being the K axis. The imaging position of camera 200 is fixed, and the settings such as the field of view are not changed when camera 200 captures the first and second captured images. Therefore, the same position in the first and second captured images corresponds to the same coordinate in the second coordinate system.

[0078] The second control unit 360 detects four intersections among the detected intersection coordinates where the values ​​of the J axis and K axis are (minimum, minimum), (maximum, minimum), (minimum, maximum), and (maximum, maximum). The second control unit 360 detects the four vertices of the pattern image 30 based on the coordinates of these four detected intersections in the second coordinate system.

[0079] Similarly, the second control unit 360 analyzes the second captured image to detect the coordinates of the four vertices of the pattern image 30 (step SA107). The second control unit 360 then determines the coordinate values ​​of the second coordinates, which indicate the positions of the four vertices of the pattern image 30 detected from the second captured image. The four vertices of the pattern image 30 captured in the first image and the four vertices of the pattern image 30 captured in the second image may be specified by the user through operation of the control unit 350 or by the user's touch operation.

[0080] Figure 10 shows the second coordinate system, plotting the four vertices of the pattern image 30 detected from the first and second captured images. The four vertices of the pattern image 30 detected from the first captured image are called vertices a1, a2, a3, and a4. Furthermore, these four vertices a1, a2, a3, and a4 are collectively referred to as the first vertex group. Furthermore, the four vertices of the pattern image 30 detected from the second captured image are referred to as vertices b1, b2, b3, and b4. When these four vertices b1, b2, b3, and b4 are referred to collectively as the second vertex group. Vertex a1 is the vertex located in the upper left corner of pattern image 30 in the drawing view. Vertex a2 is the vertex located in the upper right corner of pattern image 30 when viewed from the drawing. Vertex a3 is the vertex located in the lower left corner of pattern image 30 when viewed from the drawing. Vertex a4 is located in the lower right corner of pattern image 30 when viewed from the drawing perspective. Vertex b1 is the vertex located in the upper left corner of pattern image 30 in the drawing view. Vertex b2 is the vertex located in the upper right corner of pattern image 30 when viewed from the drawing. Vertex b3 is the vertex located in the lower left corner of pattern image 30 when viewed from the drawing. Vertex b4 is located in the lower right corner of pattern image 30 when viewed from the drawing perspective. In this context, the drawing view in Figure 10 refers to the state where the origin O is positioned in the upper left. The second control unit 360 stores the coordinates of the first group of vertices of the pattern image 30 detected from the first captured image in the second storage unit 370. The second control unit 360 also stores the coordinates of the second group of vertices of the pattern image 30 detected from the second captured image in the second storage unit 370.

[0081] Next, the second control unit 360 classifies the first vertex group a1, a2, a3, and a4, which are the four vertices of the pattern image 30 detected from the first captured image, into two vertices located on the upper edge of the pattern image 30 and two vertices located on the lower edge (step SA108). The second control unit 360 compares the K coordinate values ​​of the first vertex group a1, a2, a3, and a4, and determines that the two vertices a1 and a2 with smaller K coordinate values ​​are the vertices located on the upper edge.

[0082] Similarly, the second control unit 360 classifies the second vertex group b1, b2, b3, and b4, which are the four vertices of the pattern image 30 detected from the second captured image, into two vertices located on the upper edge of the pattern image 30 and two vertices located on the lower edge (step SA109). The second control unit 360 compares the K coordinate values ​​of the second vertex group b1, b2, b3, and b4, and determines that the two vertices b1 and b2, which have smaller K coordinate values, are the vertices located on the upper edge.

[0083] Figure 11 shows the state in which two lines L1 and L2 are set on the second coordinate system. Next, the second control unit 360 sets a straight line L1 in the second coordinate system and calculates the slope r1 of the straight line L1 in order to determine the upper edge of the tiling region 20 to be used for tiling projection from the region determined by the first and second vertex groups (step SA110).

[0084] The second control unit 360 sets a straight line L1 that passes through any two of the four vertices a1, a2, b1, and b2 located on the upper edge. In this embodiment, the straight line L1 is set to pass through the leftmost vertex a1 of the pattern image 30 detected from the first captured image and the rightmost vertex b2 of the pattern image 30 detected from the second captured image. The second control unit 360 determines the slope r1 of the straight line L1 and stores the determined slope r1 in the second storage unit 370.

[0085] Similarly, the second control unit 360 sets a line L2 in the second coordinate system and calculates the slope r2 of the line L2 in order to determine the lower edge of the tiling region 20 to be used for tiling projection from the region determined by the first and second vertex groups (step SA111).

[0086] The second control unit 360 sets a straight line L2 that passes through any two of the four vertices a3, a4, b3, and b4 located on the bottom edge. In this embodiment, the straight line L2 is set to pass through the leftmost vertex a3 of the pattern image 30 detected from the second captured image and the rightmost vertex b4 of the pattern image 30 detected from the second captured image. The second control unit 360 determines the slope r2 of the straight line L2 and stores the determined slope r2 in the second storage unit 370.

[0087] Figure 12 shows the state after translating two lines L1 and L2, which are set on the second coordinate system. Next, the second control unit 360 moves the lines L1 and L2 in parallel (step SA112). The second control unit 360 compares the K coordinate values ​​of vertices a1 and a2 located on the upper edge of the first vertex group with those of vertices b1 and b2 located on the upper edge of the second vertex group, and selects the vertex with the largest K coordinate value, that is, the vertex located furthest down in the direction of the arrow on the K axis. In the example shown in Figure 12, vertex b1 is selected. The second control unit 360 moves the line L1 so that it passes through vertex b1.

[0088] Furthermore, the second control unit 360 compares the K coordinate values ​​of vertices a3 and a4 located on the lower edge of the first vertex group with those of vertices b3 and b4 located on the lower edge of the second vertex group, and selects the vertex with the smallest K coordinate value, that is, the vertex located at the top in the direction of the arrow on the K axis. In the example shown in Figure 12, vertex a4 is selected. The second control unit 360 moves the line L2 so that it passes through vertex a4.

[0089] Figure 13 shows the intersection points of lines L1 and L2 and pattern image 30. Next, the second control unit 360 determines the coordinates of the intersection points c1 and c2 between the line L1 and the pattern image 30 (step SA113). The second control unit 360 sets intersection point c1 as the intersection point between the line segment defined by vertices a1 and a3 and the line L1. The second control unit 360 also sets intersection point c2 as the intersection point between the line segment defined by vertices b2 and b4 and the line L1.

[0090] Next, the second control unit 360 determines the coordinates of the intersection points c3 and c4 between the line L2 and the pattern image 30 (step SA114). The second control unit 360 sets intersection point c3 as the intersection point between the line segment defined by vertices a1 and a3 and the line L2. The second control unit 360 also sets intersection point c4 as the intersection point between the line segment defined by vertices b2 and b4 and the line L2.

[0091] Next, the second control unit 360 calculates the projection transformation matrix H1 (step SA115). The second control unit 360 calculates a projection transformation matrix H1 based on the coordinates of the four vertices a1, a2, a3, and a4 of the first vertex group and the coordinates of the four vertices of the pattern image 30 unfolded on the liquid crystal panel 143A. The coordinates of the four vertices of the first vertex group are coordinates on the second coordinate system, and the coordinates of the four vertices of the pattern image 30 unfolded on the panel area 144A are coordinates on the first coordinate system. The second control unit 360 obtains a projection transformation matrix H1 that transforms the coordinates of the four vertices a1, a2, a3, and a4 of the first vertex group to the four vertices of the pattern image 30 on the first coordinate system. The second control unit 360 stores the obtained projection transformation matrix H1 in the second storage unit 370. In step SA115, the second control unit 360 may calculate a projection transformation matrix H3 that transforms the coordinates of the four vertices b1, b2, b3, and b4 of the second vertex group to the coordinates of the four vertices of the pattern image 30 set in the panel area 144B, based on the coordinates of the four vertices b1, b2, b3, and b4 of the second vertex group and the coordinates of the four vertices of the pattern image 30 unfolded on the liquid crystal panel 143B, rather than the projection transformation matrix H1. Alternatively, the second control unit 360 may calculate both the projection transformation matrix H1 and the projection transformation matrix H3.

[0092] Next, the second control unit 360 projects the intersections c1, c2, c3, and c4 using the calculated projection transformation matrix H1 (step SA116). Next, the second control unit 360 calculates the resolution of the tiling region 20 based on the coordinates of the intersections c1, c2, c3, and c4 after the projection transformation. In step SA116, the second control unit 360 may also project the intersections c1, c2, c3, and c4 using the calculated projection transformation matrix H3. Furthermore, the second control unit 360 may also project the intersections c1, c2, c3, and c4 using both the projection transformation matrix H1 and the projection transformation matrix H3. In other words, the projection transformation matrix used when projecting the intersections c1, c2, c3, and c4 can be appropriately selected according to the purpose.

[0093] Figure 14 shows the projection transformation from the second coordinate system to the first coordinate system. In particular, Figure 14 shows the four intersection points c1, c2, c3, and c4 on the second coordinate system, and the coordinates of these four intersection points c1, c2, c3, and c4 on the first coordinate system after the projection transformation. The position of intersection c1 after projection transformation is denoted as d1, the position of intersection c2 after projection transformation is denoted as point d2, the position of intersection c3 after projection transformation is denoted as point d3, and the position of intersection c4 after projection transformation is denoted as point d4.

[0094] Next, the second control unit 360 determines the midpoint e1 as the position midway between points d1 and d2 after the projection transformation, and determines the midpoint e2 as the position midway between points d3 and d4 after the projection transformation. Furthermore, the second control unit 360 determines the midpoint e3 as the position midway between points d1 and d3 after the projection transformation, and determines the midpoint e4 as the position midway between points d2 and d4 after the projection transformation.

[0095] Next, the second control unit 360 calculates the resolution of the tiling area 20 (step SA117). The second control unit 360 finds the distance of the line segment D1 connecting the midpoint e1 and the midpoint e2, and obtains the resolution of the line segment D1 corresponding to the found distance. The resolution of the line segment D1 corresponds to the vertical resolution of the tiling area 20 and the second-direction resolution of the projection area. The first coordinate is the coordinate set in the panel area 144A of the liquid crystal panel 143A, and the resolution of the panel area 144A is known. Therefore, the second control unit 360 can obtain the resolution based on the coordinate values ​​of the first coordinate. Similarly, the second control unit 360 calculates the distance of the line segment D2 connecting midpoint e3 and midpoint e4, and obtains the resolution of the line segment D2 corresponding to the calculated distance. Line segments D1 and D2 are in mutually orthogonal directions. The resolution of line segment D2 corresponds to the lateral resolution of the tiling area 20 and the resolution of the projection area in the first direction. The resolution of the tiling region 20 is indicated by the resolution of line segment D1 and the resolution of line segment D2.

[0096] Next, the second control unit 360 generates shape correction parameters (step SA118). The second control unit 360 generates parameters for shape correction based on the projection transformation matrix H1, which include a parameter to move vertex a1, shown in Figure 13, to the position of intersection c1, and a parameter to move vertex b2, to the position of intersection c2. In step SA118, the second control unit 360 may generate the shape correction parameters based on the projection transformation matrix H3. Furthermore, the second control unit 360 may generate the shape correction parameters based on the projection transformation matrix H1 and the projection transformation matrix H3.

[0097] Figure 15 shows the projection transformation from the second coordinate system to the third coordinate system. In particular, Figure 15 shows the state after the projection transformation of the intersection points c1, c2, c3, and c4 on the second coordinate system to the third coordinate system. First, the second control unit 360 calculates a projection transformation matrix H2 that projects the intersection points c1, c2, c3, and c4 of the first coordinate system onto the four points on the third coordinate system (step SA119). The third coordinate system is a set of coordinates for the image supplied from the information processing device 300 to the projector 100. The upper left corner of the image is defined as the origin O, with the horizontal axis of the third coordinate system being the M axis and the vertical axis being the N axis.

[0098] Furthermore, the four points of the third coordinate system are set based on the resolution of the image data supplied by the information processing device 300 to the projector 100. For example, let the points after projection transformation of intersection points c1, c2, c3, and c4 be points f1, f2, f3, and f4, respectively. Also, assume that the image resolution is 3840 × 2160. In this case, the coordinates of point f1 are (0, 0), the coordinates of point f2 are (3840, 0), the coordinates of point f3 are (0, 2160), and the coordinates of point f4 are (3840, 2160). Here, the image resolution matches the resolution of the tiling area 20. The image resolution may be arbitrarily changed by user settings.

[0099] Figure 16 shows the projection transformation from the second coordinate system to the third coordinate system. In particular, Figure 16 shows the state after projecting vertices a2, a4, b1, and b3 on the second coordinate system to the third coordinate system. Next, the second control unit 360 performs a projection transformation on vertices a2, a4, b1, and b3 using the calculated matrix H2 (step SA120). The position of vertex b1 after projection transformation is denoted as point g1, the position of vertex a2 after projection transformation is denoted as point g2, the position of vertex b3 after projection transformation is denoted as point g3, and the position of vertex a4 after projection transformation is denoted as point g4.

[0100] Next, the second control unit 360 calculates a parameter indicating the width of the superimposed region 15 (step SA121). The second control unit 360 determines the distance on the M axis between points g1 and g2 obtained by projecting vertices b1 and a2 belonging to the first vertex group, and the distance on the M axis between points g3 and g4 obtained by projecting vertices b3 and a4 belonging to the second vertex group. The distance on the M axis between points g1 and g2 is called distance D3, and the distance on the M axis between points g3 and g4 is called distance D4. The second control unit 360 selects the shorter distance D3 from distances D3 and D4 as the width of the superimposed region 15. As described above, the resolution of the tiling region 20 is known, so it is possible to obtain the resolution based on the distance on the third coordinate system. The second control unit 360 stores the parameter that represents the calculated width of the superimposed region 15 in terms of resolution in the second storage unit 370.

[0101] [Example 1] In the embodiment described above, the resolution of line segment D1 and the resolution of line segment D2 were transmitted to the projector 100A as parameters indicating the resolution of the tiling area 20. In the modified example 1, based on the calculated resolutions of line segment D1 and line segment D2, an arbitrary resolution is set as the resolution of the tiling area 20, in accordance with the specifications of the information processing device 300 and the projector 100. The second control unit 360 transmits the set resolution as a parameter to the projector 100A.

[0102] First, the second control unit 360 calculates the aspect ratio of the resolution of the tiling region 20, which is shown by the resolution of the calculated line segment D1 and the resolution of line segment D2. Let Wt be the resolution of line segment D2, which is the widthwise resolution of the tiling region 20, and let Ht be the resolution of line segment D1, which is the heightwise resolution of the tiling region 20. The widthwise direction is the direction of D2 as shown in Figure 14, and the heightwise direction is the direction of D1 as shown in Figure 14. The aspect ratio is Wt / Ht.

[0103] Next, the second control unit 360 calculates the resolution based on the aspect ratio and the specifications set for the information processing device 300 and the projector 100. For example, let's assume that the resolution that can be set for projector 100A is any even number for the width direction, and a fixed value of 1080 (Full HD) for the height direction. 1080 (Full HD), which is the height resolution, is just one example of a pre-set resolution. The height resolution is not limited to 1080 (Full HD); for example, 2160 (4K) could be used, or the user could specify any resolution. If the user specifies a resolution, it is preferable that it be an even number. Furthermore, the pre-set resolution may also be the width resolution. Here, if we denote the width resolution as Wd and the height resolution as Hd, Wd = Aspect Ratio × Hd = Aspect Ratio × 1080 The Wd and Hd values ​​that satisfy the conditions are set as the resolution of the tiling area 20, and the set resolutions Wd and Hd are sent to the projector 100A as parameters.

[0104] [Differentiation 2] In Modification 2, a resolution selected from the resolutions that can be set for projector 100A and projector 100B is set as the resolution of the tiling area 20. The second control unit 360 transmits the set resolution as a parameter to projector 100A. First, the second control unit 360 obtains a list of resolutions that can be set for projector 100A from projector 100A. The second control unit 360 also obtains a list of resolutions that can be set for projector 100B from projector 100B. The list obtained from projector 100A is called list information A, and the list obtained from projector 100B is called list information B.

[0105] Next, the second control unit 360 compares list information A and list information B and selects a resolution common to both list information A and list information B. The selected resolutions are called the common resolution group.

[0106] Next, the second control unit 360 calculates the aspect ratio of all resolutions included in the common resolution group. It also calculates the aspect ratio of the resolution of the tiling region 20, which is represented by the resolutions of line segment D1 and line segment D2, in the same manner as in the modified example 1. The aspect ratio of all resolutions included in a common resolution group is called the common aspect ratio. Furthermore, the aspect ratio calculated using the resolution of the tiling area 20 is called the tiling aspect ratio.

[0107] Next, the second control unit 360 calculates the absolute difference between each common aspect ratio and the tiling aspect ratio. The second control unit 360 selects the common aspect ratio that has the smallest absolute difference between it and the tiling aspect ratio. The common aspect ratio selected here may be one or more. The selected common aspect ratio corresponds to the selected aspect ratio.

[0108] Next, the second control unit 360 acquires the common resolution corresponding to the selected common aspect ratio and calculates the number of pixels of the acquired common resolution. The number of pixels is obtained by multiplying the resolution in the width direction and the resolution in the height direction of the common resolution.

[0109] Next, the second control unit 360 calculates the number of pixels in the tiling area 20. This number of pixels is obtained by multiplying the resolution of line segment D2, which is the widthwise resolution of the tiling area 20, and the resolution of line segment D1, which is the heightwise resolution.

[0110] Next, the second control unit 360 selects the number of pixels of the common resolution that is closest to the calculated number of pixels of the tiling area 20. The second control unit 360 selects the common resolution that has the smallest absolute difference between the number of pixels of the tiling area 20 and the number of pixels of the common resolution. The second control unit 360 sets the selected common resolution as the resolution of the tiling area 20 and transmits the set common resolution as a parameter to the projector 100A.

[0111] Furthermore, the second control unit 360 may generate a first aspect ratio group, which consists of aspect ratios that can be set on the projector 100A, based on the resolutions registered in the list information A. Furthermore, the second control unit 360 may generate a second group of aspect ratios that can be set on the projector 100B based on the resolutions registered in the list information B. The aspect ratios registered in the first and second aspect ratio groups may be one or multiple.

[0112] Next, the second control unit 360 acquires the aspect ratios common to the first aspect ratio group and the second aspect ratio group as a common aspect ratio group. The common aspect ratio group may consist of one aspect ratio or multiple aspect ratios. The aspect ratios registered in the common aspect ratio group are called common aspect ratios.

[0113] Next, the second control unit 360 calculates the absolute difference between each common aspect ratio included in the common aspect ratio group and the tiling aspect ratio. The second control unit 360 selects the common aspect ratio with the smallest absolute difference between the common aspect ratio and the tiling aspect ratio. The common aspect ratio selected here may be one or more. The selected common aspect ratio corresponds to the selected aspect ratio.

[0114] Next, the second control unit 360 acquires a common resolution corresponding to the selected common aspect ratio and calculates the number of pixels of the acquired common resolution. Next, the second control unit 360 calculates the number of pixels of the tiling area 20. This number of pixels is obtained by multiplying the resolution in the width direction and the resolution in the height direction of the tiling area 20.

[0115] [Summary of this disclosure] A summary of this disclosure is provided below. (Note 1) The first projector acquires a first image of the projection surface from which it projects the first image, The second projector acquires a second image of the projection surface from which it projects the second image, When a tiling image is displayed on the projection area of ​​the projection surface by a first projected image projected by the first projector onto the projection surface and a second projected image projected by the second projector onto the projection surface, information indicating the resolution of the projection area is acquired based on the first and second captured images. A first correction parameter for correcting the image projected by the first projector is obtained based on the first captured image and the second captured image, A second correction parameter for correcting the image projected by the second projector is obtained based on the first captured image and the second captured image, The first projector projects the first projection image, which is generated based on the input image, the resolution information, and the first correction parameter, onto the projection surface. The second projector projects the second projection image, which is generated based on the input image, the resolution information, and the second correction parameter, onto the projection surface. A projection method, including

[0116] According to this, in tiling projection, where a portion of the image projected by the first projector and a portion of the image projected by the second projector are superimposed, the resolution of the projection area on the projection surface displaying the image is determined based on the first captured image and the second captured image. Therefore, the projection area on which the image is displayed can be precisely determined by tiling projection, thereby improving the accuracy of tiling projection. Furthermore, the first and second correction parameters are acquired based on the first and second captured images. Therefore, the first projection image corresponding to the projection area can be accurately generated using the first correction parameter. Similarly, the second projection image corresponding to the projection area can be accurately generated using the second correction parameter.

[0117] (Note 2) Information indicating the resolution of the projection area is obtained based on the first captured image and the second captured image. Obtaining the positions of the four corners of the first image from the first image, Obtain the positions of the four corners of the second image from the second image, The projection area is determined based on the positions of the four corners of the first image and the positions of the four corners of the second image. To obtain the resolution of the aforementioned projection area, The projection method described in Appendix 1, including the method described in Appendix 1.

[0118] According to this method, the projection area to be used for tiling projection is determined based on the positions of the four corners of the image in the first captured image and the positions of the four corners of the image in the second captured image. Therefore, the projection area to be used for tiling projection can be determined with high accuracy.

[0119] (Note 3) To obtain the resolution of the projection area, To obtain the resolution in the first direction of the projection area, To obtain the resolution in the second direction of the projection area that is orthogonal to the first direction, The projection method described in Appendix 2, including the method described in Appendix 2.

[0120] This method allows us to obtain the resolution in a first direction of the projection area and in a second direction perpendicular to the first direction. Therefore, the resolution of the projection area used for tiling projection can be precisely defined.

[0121] (Note 4) To obtain the resolution of the projection area, Obtain an even resolution as the resolution in the first direction. The projection method described in Appendix 3, including the method described in Appendix 3.

[0122] According to this, an even resolution can be obtained as the resolution in the first direction. Therefore, the resolution in the first direction can be set to a resolution that can be displayed by the projector.

[0123] (Note 5) To obtain the resolution in the first direction, Based on the resolution of the projection area, the aspect ratio of the projection area is obtained, The resolution in the second direction is to obtain a pre-set resolution, The resolution in the first direction is obtained based on the aspect ratio and the preset resolution. The projection method described in Appendix 3 or 4, including the projection method described in Appendix 4.

[0124] According to this, if the resolution in the second direction is a preset resolution, the resolution in the first direction can be obtained based on the aspect ratio of the projection area. Therefore, the resolution in the first direction can be set to an appropriate resolution according to the resolution in the second direction.

[0125] (Note 6) To obtain the resolution of the projection area, To obtain information on the first aspect ratio group that can be set for the first projector, To obtain information on the second aspect ratio group that can be set for the second projector, The aspect ratios that are included in both the first aspect ratio group and the second aspect ratio group are obtained as a common aspect ratio group, Among the aspect ratios included in the aforementioned common aspect ratio group, the aspect ratio that has the smallest difference from the aspect ratio calculated based on the resolution of the projection area is acquired as the selected aspect ratio. Based on the selected aspect ratio, the resolution in the first direction and the resolution in the second direction are obtained. A projection method described in any one of the appendices 3 to 5, including the above.

[0126] According to this, an aspect ratio that can be set for both the first and second projectors and has the smallest difference from the aspect ratio calculated based on the resolution of the projection area can be selected as the selected aspect ratio. Furthermore, the resolution of the first and second directions of the projection area can be set based on the selected aspect ratio. Therefore, it is possible to set the resolution of the first and second directions to be suitable for the first and second projectors.

[0127] (Note 7) Obtaining the resolution in the first direction and the resolution in the second direction based on the selected aspect ratio is: Based on the resolution of the projection area, the number of pixels in the projection area is obtained, To obtain a plurality of resolutions having the selected aspect ratio and settable for the first and second projectors, Obtaining the number of pixels for each of the aforementioned multiple resolutions, Among the multiple resolutions, the resolution with the smallest difference from the number of pixels in the projection area is selected. Based on the resolution selected as the resolution with the smallest difference from the number of pixels in the projection area, the resolution in the first direction and the resolution in the second direction are obtained. The projection method described in Appendix 6, including the method described in Appendix 6.

[0128] According to this, the resolution for the first direction and the resolution for the second direction can be obtained based on the resolution that has a selectable aspect ratio and is settable for the first and second projectors, and which has the smallest difference from the number of pixels in the projection area. Therefore, appropriate resolutions for the first and second directions can be set according to the resolution of the projection area.

[0129] (Note 8) The aforementioned first projector, The information indicating the resolution and the first correction parameter are stored. When the input image is connected via a wired connection to a control device that supplies the first projector, the system transmits EDID (Extended Display Identification Data) including information indicating the resolution of the projection area to the control device. The projection method described in any one of the appendices 1 to 7, which performs the following.

[0130] According to this, the resolution of the projection area can be notified to the control device. Therefore, the control device can obtain information indicating the resolution of the projection area.

[0131] (Note 9) The control device is To obtain information indicating the resolution of the projection area, the first correction parameter, and the second correction parameter, The information indicating the resolution of the projection area and the first correction parameter are transmitted to the first projector. The information indicating the resolution of the projection area and the second correction parameter are transmitted to the second projector. The projection method described in Appendix 8, including the method described in Appendix 8.

[0132] According to this, information indicating the resolution of the projection area and a first correction parameter are transmitted to the first projector, and information indicating the resolution of the projection area and a second correction parameter are transmitted to the second projector. Therefore, the first projector can generate the first projected image, and the second projector can generate the second projected image.

[0133] (Note 10) The first image is an image that includes a predetermined shape at positions corresponding to at least four corners of the first image, The second image is an image that includes pre-set shapes at positions corresponding to at least the four corners of the second image. The projection method described in any one of the items 1 through 9 in the appendix.

[0134] According to this, the first image includes pre-set shapes at the positions corresponding to the four corners, and the second image also includes pre-set shapes at the positions corresponding to the four corners. Therefore, detecting the positions of the four corners of the first image becomes easier, and detecting the positions of the four corners of the second image also becomes easier.

[0135] (Note 11) An imaging device that acquires a first image obtained by capturing the projection surface on which a first projector projects a first image, and a second image obtained by capturing the projection surface on which a second projector projects a second image. When a tiling image is displayed on the projection area of ​​the projection surface by a first projected image projected by the first projector onto the projection surface and a second projected image projected by the second projector onto the projection surface, information indicating the resolution of the projection area is acquired based on the first and second captured images. A first correction parameter for correcting the image projected by the first projector is obtained based on the first captured image and the second captured image, A control device that performs the following: acquiring a second correction parameter for correcting the image projected by the second projector, based on the first captured image and the second captured image; A first projector that projects a first projection image onto the projection surface, which is generated based on the input image, the information indicating the resolution, and the first correction parameter, A second projector that projects a second projection image generated based on the input image, the information indicating the resolution, and the second correction parameter, A projection system equipped with [the following features].

[0136] In this configuration, in tiling projection where a portion of the image projected by the first projector and a portion of the image projected by the second projector are superimposed, the resolution of the projection area on the projection surface displaying the image is determined based on the first captured image and the second captured image. Therefore, the projection area on which the image is displayed can be precisely determined by tiling projection, thereby improving the accuracy of tiling projection. Furthermore, the first and second correction parameters are acquired based on the first and second captured images. Therefore, a first projection image corresponding to the projection area can be accurately generated using the first correction parameter. Furthermore, a second projection image corresponding to the projection area can be accurately generated using the second correction parameter.

[0137] (Note 12) An imaging device that acquires a first image obtained by capturing the projection surface on which a first projector projects a first image, and a second image obtained by capturing the projection surface on which a second projector projects a second image. When a tiling image is displayed on the projection area of ​​the projection surface by a first projected image projected by the first projector onto the projection surface and a second projected image projected by the second projector onto the projection surface, information indicating the resolution of the projection area is acquired based on the first and second captured images. A first correction parameter for correcting the image projected by the first projector is obtained based on the first captured image and the second captured image, A second correction parameter for correcting the image projected by the second projector is obtained based on the first captured image and the second captured image, The first projector is to transmit information indicating the resolution of the projection area and the first correction parameter. The information indicating the resolution of the projection area and the second correction parameter are transmitted to the second projector. A control device that performs the following: When connected to the control device, it transmits an EDID including information indicating the resolution to the control device, A first projector that projects a first projection image generated based on an input image, information indicating the resolution, and a first correction parameter, A projection system comprising: a second projector that projects a second projection image onto a projection surface, which is generated based on the input image, the information indicating the resolution, and the second correction parameter.

[0138] In this configuration, in tiling projection where a portion of the image projected by the first projector and a portion of the image projected by the second projector are superimposed, the resolution of the projection area on the projection surface displaying the image is determined based on the first captured image and the second captured image. Therefore, the projection area on which the image is displayed can be precisely determined by tiling projection, thereby improving the accuracy of tiling projection. Furthermore, the first and second correction parameters are acquired based on the first and second captured images. Therefore, a first projection image corresponding to the projection area can be accurately generated using the first correction parameter. Furthermore, a second projection image corresponding to the projection area can be accurately generated using the second correction parameter. Furthermore, when the control unit is connected, the first projector transmits an EDID containing information indicating the resolution to the control unit. Therefore, the control device can be made to recognize the first projector as a display device and notify it of the resolution of the image displayed by the first projector.

[0139] The embodiments described above are preferred embodiments of the present invention. However, the invention is not limited to the embodiments described above, and various modifications can be made without departing from the spirit of the invention. For example, in the embodiments described above, the case in which the camera 200 is provided outside the information processing device 300 and the projector 100 was described, but the information processing device 300 or the projector 100 may be configured to mount the camera 200.

[0140] Furthermore, in the embodiments described above, the case in which the information processing device 300 generates parameters such as the resolution of the tiling area 20, the width of the superimposed area 15, and shape correction was explained, but these parameters may also be generated by the camera 200. Alternatively, these parameters may be generated by the projector 100A or 100B.

[0141] Furthermore, if the second control unit 360 finds that the calculated resolution of the tiling area 20 is not a resolution that can be recognized as an EDID value, it may change the resolution of the tiling area 20 to a value that can be recognized as an EDID. For example, depending on the specifications, it may not be possible to set an odd resolution for the display. The second control unit 360 may, for example, correct an odd resolution to an even resolution if at least one of the calculated width and height resolutions of the tiling area 20 is odd.

[0142] Furthermore, although the above-described embodiment illustrates an optical modulator 142A equipped with a liquid crystal panel 143A, the liquid crystal panel 143A may be a transmissive liquid crystal panel or a reflective liquid crystal panel. Also, the optical modulator 142A may use a digital mirror device instead of the liquid crystal panel 143A. Alternatively, a configuration combining a digital mirror device and a color wheel may be used. In addition, the optical modulator 142A may employ a configuration capable of modulating light emitted from a light source, in addition to the liquid crystal panel 143A and the digital mirror device.

[0143] Furthermore, the functional units of the projector 100A shown in Figures 2 and 3 represent functional configurations, and the specific implementation form is not particularly limited. In other words, it is not necessarily required that hardware corresponding to each functional unit be implemented individually, and it is certainly possible to have a configuration in which a single processor executes a program to realize the functions of multiple functional units. Also, some of the functions realized by software in the above embodiment may be realized by hardware, and some of the functions realized by hardware may be realized by software. In addition, the specific detailed configuration of other parts of the projector can also be arbitrarily changed without departing from the spirit of the present invention.

[0144] Furthermore, the functional units of the information processing device 300 shown in Figure 4 represent functional configurations, and the specific implementation form is not particularly limited. In other words, it is not necessarily required that hardware corresponding to each functional unit be implemented individually, and it is certainly possible to have a configuration in which a single processor executes a program to realize the functions of multiple functional units. Also, some of the functions realized by software in the above embodiment may be realized by hardware, and some of the functions realized by hardware may be realized by software. In addition, the specific detailed configurations of other parts of the projector can also be arbitrarily changed without departing from the spirit of the present invention.

[0145] Furthermore, the processing units in the flowcharts shown in Figures 6 and 8 are divided according to their main processing content in order to facilitate understanding of the processing of the information processing device 300. The present invention is not limited by the way the processing units are divided or the names of the units shown in the flowcharts in Figures 6 and 8. Furthermore, the processing of the information processing device 300 can be divided into even more processing units depending on the processing content, or it can be divided so that one processing unit contains even more processing. Also, the processing order in the flowchart above is not limited to the example shown.

[0146] Furthermore, the processing units in the flowchart shown in Figure 7 are divided according to their main processing content in order to facilitate understanding of the processing of Projector 100A. The present invention is not limited by the way the processing units are divided or the names of the processing units shown in the flowchart in Figure 7. In addition, the processing of Projector 100A can be further divided into many more processing units depending on the processing content, or each processing unit can be divided to include even more processing. Moreover, the processing order in the flowchart above is not limited to the example shown.

[0147] Furthermore, when the projection method is implemented using a computer provided by the information processing device 300, the program to be executed by the computer may be configured as a recording medium or a transmission medium for transmitting this program. Magnetic, optical, or semiconductor memory devices can be used as the recording medium. Specifically, portable or fixed recording media such as flexible disks, HDDs (Hard Disk Drives), CD-ROMs, DVDs, Blu-ray Discs, magneto-optical disks, flash memory, and card-type recording media can be used. Alternatively, the above-mentioned recording media may be non-volatile storage devices such as RAM, ROM, or HDDs, which are internal storage devices provided by the server device. Blu-ray is a registered trademark. [Explanation of symbols]

[0148] 1…Projection system, 3…Cable, 5…Cable, 7…Wireless router, 9…Wireless network, 10…Projection surface, 11…Projection area, 13…Projection area, 15…Superimposed area, 20…Tiling area, 30…Pattern image, 31…First rectangular shape, 33…Second rectangular shape, 100, 100A, 100B…Projector, 101A…Remote control receiver, 105…Remote control, 110A…First wireless I / F, 120A…First wired I / F, 121A…Input terminal, 123A…Receiver circuit, 125A…Transmitter circuit, 127A…Output terminal, 130A…Image processing unit, 135A…Frame memory, 140A…Image projection unit ,141A...light source, 142A...optical modulator, 143A, 143B...liquid crystal panel, 144A, 144B...panel area, 145A...optical unit, 146A...panel drive unit, 150A...first control unit, 160A...first memory unit, 161...control program, 170A...first processor, 200...camera, 300...information processing device, 310...second wireless I / F, 320...second wired I / F, 330...third wired I / F, 340...display unit, 345...display panel, 350...operation unit, 360...second control unit, 370...second memory unit, 371...control program, 373...content file, 380...second processor.

Claims

1. The first projector acquires a first image of the projection surface from which it projects the first image, The second projector acquires a second image of the projection surface from which it projects the second image, When a tiling image is displayed on the projection area of ​​the projection surface by a first projector projecting a first projected image onto the projection surface and a second projector projecting a second projected image onto the projection surface, information indicating the resolution of the projection area is acquired based on the first and second captured images. A first correction parameter for correcting the image projected by the first projector is obtained based on the first captured image and the second captured image, A second correction parameter for correcting the image projected by the second projector is obtained based on the first captured image and the second captured image, The first projector projects the first projection image, which is generated based on the input image, the resolution information, and the first correction parameter, onto the projection surface. The second projector projects the second projection image, which is generated based on the input image, the resolution information, and the second correction parameter, onto the projection surface. A projection method, including

2. Information indicating the resolution of the projection area is obtained based on the first captured image and the second captured image. Obtaining the positions of the four corners of the first image from the first image, Obtaining the positions of the four corners of the second image from the second image, The projection area is determined based on the positions of the four corners of the first image and the positions of the four corners of the second image. To obtain the resolution of the aforementioned projection area, The projection method according to claim 1, including the method described in claim 1.

3. To obtain the resolution of the projection area, To obtain the resolution in the first direction of the projection area, To obtain the resolution in the second direction of the projection area that is orthogonal to the first direction, The projection method according to claim 2, including the method described in claim 2.

4. To obtain the resolution of the projection area, Obtain an even resolution as the resolution in the first direction. The projection method according to claim 3, including the method described in claim 3.

5. To obtain the resolution in the first direction, Based on the resolution of the projection area, the aspect ratio of the projection area is obtained, The resolution in the second direction is to acquire a pre-set resolution, The resolution in the first direction is obtained based on the aspect ratio and the preset resolution. The projection method according to claim 3, including the method described in claim 3.

6. To obtain the resolution of the projection area, To obtain information on the first aspect ratio group that can be set for the first projector, To obtain information on a second aspect ratio group that can be set for the second projector, The aspect ratios that are included in both the first aspect ratio group and the second aspect ratio group are obtained as a common aspect ratio group, Among the aspect ratios included in the aforementioned common aspect ratio group, the aspect ratio that has the smallest difference from the aspect ratio calculated based on the resolution of the projection area is acquired as the selected aspect ratio. Based on the selected aspect ratio, the resolution in the first direction and the resolution in the second direction are obtained. A projection method according to any one of claims 3 to 5, including the following:

7. Obtaining the resolution in the first direction and the resolution in the second direction based on the selected aspect ratio is: Based on the resolution of the projection area, the number of pixels in the projection area is obtained, To obtain a plurality of resolutions having the selected aspect ratio and settable for the first and second projectors, Obtaining the number of pixels for each of the aforementioned multiple resolutions, Among the multiple resolutions, the resolution with the smallest difference from the number of pixels in the projection area is selected. Based on the resolution selected as the resolution with the smallest difference from the number of pixels in the projection area, the resolution in the first direction and the resolution in the second direction are obtained. The projection method according to claim 6, including the method described in claim 6.

8. The first projector is The information indicating the resolution and the first correction parameter are stored. When the input image is connected via a wired connection to a control device that supplies the first projector, the EDID (Extended Display Identification Data) including information indicating the resolution of the projection area is transmitted to the control device, A projection method according to any one of claims 1 to 5, which performs the following:

9. The control device is To obtain information indicating the resolution of the projection area, the first correction parameter, and the second correction parameter, The information indicating the resolution of the projection area and the first correction parameter are transmitted to the first projector. The information indicating the resolution of the projection area and the second correction parameter are transmitted to the second projector. The projection method according to claim 8, including the method described in claim 8.

10. The first image is an image that includes pre-set shapes at positions corresponding to at least four corners of the first image, The projection method according to any one of claims 1 to 5, wherein the second image is an image that includes a predetermined figure at positions corresponding to at least four corners of the second image.

11. An imaging device that acquires a first image obtained by capturing the projection surface on which a first projector projects a first image, and a second image obtained by capturing the projection surface on which a second projector projects a second image. When a tiling image is displayed on the projection area of ​​the projection surface by a first projector projecting a first projected image onto the projection surface and a second projector projecting a second projected image onto the projection surface, information indicating the resolution of the projection area is acquired based on the first and second captured images. A first correction parameter for correcting the image projected by the first projector is obtained based on the first captured image and the second captured image, A control device that performs the following: acquiring a second correction parameter for correcting the image projected by the second projector, based on the first captured image and the second captured image; A first projector that projects a first projection image onto the projection surface, which is generated based on the input image, the information indicating the resolution, and the first correction parameter, A second projector that projects a second projection image generated based on the input image, the information indicating the resolution, and the second correction parameter, A projection system equipped with [the following features].

12. An imaging device that acquires a first image obtained by capturing the projection surface on which a first projector projects a first image, and a second image obtained by capturing the projection surface on which a second projector projects a second image. When a tiling image is displayed on the projection area of ​​the projection surface by a first projector projecting a first projected image onto the projection surface and a second projector projecting a second projected image onto the projection surface, information indicating the resolution of the projection area is acquired based on the first and second captured images. A first correction parameter for correcting the image projected by the first projector is obtained based on the first captured image and the second captured image, A second correction parameter for correcting the image projected by the second projector is obtained based on the first captured image and the second captured image, The information indicating the resolution of the projection area and the first correction parameter are transmitted to the first projector. The information indicating the resolution of the projection area and the second correction parameter are transmitted to the second projector. A control device that performs the following: When connected to the control device, it transmits an EDID containing information indicating the resolution to the control device, A first projector that projects a first projection image generated based on an input image, information indicating the resolution, and a first correction parameter, A projection system comprising: a second projector that projects a second projection image onto a projection surface, which is generated based on the input image, the information indicating the resolution, and the second correction parameter.

Images