Exposure adjustment method, image correction method, and exposure adjustment system

Abstract

The present invention provides an exposure adjustment method and an exposure adjustment system that improve the accuracy of correction using captured images captured by an imaging device. [Solution] The information processing device 200 acquires an image captured from the first projector 100A that includes the projection surface 10 on which the image is projected, displays the acquired image on the touch panel 230, accepts a first operation to correct the shape and position of the unmasked area of ​​the image, sets the unmasked area in the image based on the first operation, calculates a target value for exposure adjustment based on the image of the unmasked area, outputs the calculated target value to the first projector 100A, and the first projector 100A performs exposure adjustment based on the target value.

Description

Technical Field

[0001] The present invention relates to an exposure adjustment method, an image correction method, and an exposure adjustment system.

Background Art

[0002] A technique for correcting the position and shape of a projection image projected by a projection device using a captured image captured by an imaging device is known.

[0003] For example, Patent Document 1 discloses a projector including an acquisition unit, a first derivation unit, a second derivation unit, and a correction unit. The acquisition unit acquires a first captured image and a second captured image. The first derivation unit derives a first mapping relationship and a second mapping relationship. The second derivation unit derives projection conversion parameters. The correction unit corrects the projection image based on a first projection relationship, a second projection relationship, and the projection conversion parameters.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, when the projector disclosed in Patent Document 1 acquires a captured image including, for example, external light incident indoors in a part of the captured image captured by the imaging device and corrects the projection image, the accuracy of correcting the projection image may decrease. The reason for this is that the captured image includes a region having a higher luminance than the image projected by the projection device, and in the captured image in which exposure adjustment is performed based on the luminance of this high-luminance region, the image projected by the projection device is displayed darkly, and the detection accuracy of the image projected by the projection device from the captured image decreases. [Means for solving the problem]

[0006] The exposure adjustment method of the present disclosure is an exposure adjustment method in which a control device performs the following actions: acquires an image from an imaging device that captures an area including a projection surface on which an image is projected; displays the acquired image; accepts a first operation to correct the shape and position of the unmasked area of ​​the image; sets the unmasked area in the image based on the first operation; calculates a target value for exposure adjustment based on the unmasked area of ​​the image; outputs the calculated target value to the imaging device; and the imaging device adjusts at least one of the settings of the shutter speed, gain, and aperture of the imaging device based on the target value.

[0007] The image correction method of the present disclosure comprises: a first projection device projecting a guide image that guides the setting of an unmasked area; a control device acquiring a first image captured from an imaging device, which captures the area including the projected guide image; the control device displaying the acquired first image; the control device receiving a first operation to correct the shape and position of the unmasked area, which is set based on the guide image displayed on the first image; the control device setting the unmasked area in the first image based on the first operation; the control device calculating a target value for exposure adjustment based on the unmasked area in the first image; the control device outputting the calculated target value to the imaging device; and the imaging device shuttering the imaging device based on the target value. The image correction method comprises: adjusting at least one of the settings of shutter speed, gain, and aperture; a first projection device projecting a first pattern image having a preset pattern; a control device acquiring a second image image including the first pattern image, which is captured by the imaging device whose settings have been adjusted based on the target value; a second projection device projecting a second pattern image having a preset pattern; a control device acquiring a third image image including the second pattern image, which is captured by the imaging device whose settings have been adjusted based on the target value; and a control device correcting the shape of the image projected by at least one of the first and second projection devices based on the second and third image images.

[0008] The exposure adjustment system of the present disclosure is an exposure adjustment system comprising: an acquisition unit that acquires an image from an imaging device that captures an area including a projection surface on which an image is projected; a display unit that displays the acquired image; an operation unit that accepts a first operation to correct the shape and position of an unmasked area in the image; a control unit that performs the following actions: setting the unmasked area in the image based on the first operation; calculating a target value for exposure adjustment based on the unmasked area in the image; and outputting the calculated target value to the imaging device; and an imaging device that adjusts at least one setting value of the shutter speed, gain, and aperture of the imaging device based on the target value. [Brief explanation of the drawing]

[0009] [Figure 1] A diagram showing the system configuration. [Figure 2] A block diagram showing the configuration of the projector. [Figure 3] A block diagram showing the configuration of an information processing device. [Figure 4] A flowchart illustrating the operation of an information processing device. [Figure 5] A diagram showing an example of the first app screen. [Figure 6] A diagram showing an example of the second app screen. [Figure 7] This figure shows the first and second projected images after adjusting the projection position. [Figure 8] A diagram showing an example of the third app screen. [Figure 9] A diagram showing an example of a guide image. [Figure 10] A diagram showing an example of the 4th APP screen. [Modes for carrying out the invention]

[0010] [1. System Configuration] Figure 1 shows the system configuration of a system to which the present invention is applied. Figure 1 shows a system comprising multiple projectors 100 and an information processing device 200 that operates as a control device.

[0011] The system shown in Figure 1 comprises two projectors 100: a first projector 100A and a second projector 100B. The number of projectors 100 constituting the system is not limited to two. For example, a configuration with three or more projectors 100 is also possible. In the following, the first projector 100A and the second projector 100B will be collectively referred to as projector 100. The first projector 100A corresponds to the first projection device, and the second projector 100B corresponds to the second projection device.

[0012] The first projector 100A, the second projector 100B, and the information processing device 200 are connected to a wireless network 5 provided by a wireless router 3. The wireless router 3 functions as an access point and relays data transmission and reception between devices connected to the wireless network 5. The wireless router 3 also functions as a router, connecting to a network such as the Internet via a modem (not shown), and relays data transmission and reception between devices connected to the network and devices connected to the wireless network 5. The information processing device 200, the first projector 100A, and the second projector 100B transmit and receive control information and captured images, which will be described later, via the wireless network 5. The first projector 100A and the information processing device 200 constitute an exposure adjustment system.

[0013] In addition, the information processing device 200, the first projector 100A, and the second projector 100B are daisy-chain connected by a cable 7. For the cable 7, for example, a cable compliant with standards such as HDMI (High-Definition Multimedia Interface), DisplayPort, or USB Type-C is used. HDMI is a registered trademark. The information processing device 200 transmits an image signal to the first projector 100A via the cable 7. The first projector 100A receives and processes the image signal transmitted from the information processing device 200, and transmits the received image signal to the second projector 100B via the cable 7.

[0014] The connection of the information processing device 200, the first projector 100A, and the second projector 100B may be only by a wireless network 5. That is, the information processing device 200 may be configured to transmit control information and an image signal to the first projector 100A and the second projector 100B via the wireless network 5. Also, the information processing device 200, the first projector 100A, and the second projector 100B may be connected by a wired LAN (Local Area Network).

[0015] For the information processing device 200, for example, a notebook PC (Personal Computer), a desktop PC, a tablet PC, a smartphone, a PDA (Personal Digital Assistant), or the like is used. The information processing device 200 transmits an image signal including image data to the first projector 100A via the cable 7.

[0016] The first projector 100A and the second projector 100B extract the image data included in the image signal received from the information processing apparatus 200, and generate image light based on the extracted image data. The first projector 100A and the second projector 100B project the generated image light onto the projection surface 10. Thereby, an image corresponding to the image data is displayed on the projection surface 10. The projection surface 10 may utilize an indoor wall surface or a screen may be installed.

[0017] A small window 9 is installed in the room where the information processing apparatus 200, the first projector 100A, and the second projector 100B are installed, and external light enters through this small window 9. Due to the external light entering through the small window 9, in a plan view, the area on the right side of the room where the small window 9 is provided is brighter than the area on the left side where the projection surface 10 is provided.

[0018] [2. Configuration of the Projector] FIG. 2 is a block diagram showing the configuration of the first projector 100A. The first projector 100A and the second projector 100B have substantially the same configuration. Therefore, hereinafter, the configuration of the first projector 100A will be described, and the description of the configuration of the second projector 100B will be omitted. Also, hereinafter, in order to distinguish the configuration of the first projector 100A from that of the second projector 100B, the configuration of the first projector 100A is denoted by the symbol "A", and the configuration of the second projector 100B is denoted by the symbol "B".

[0019] The first projector 100A includes an infrared receiver 110A, a first wireless interface 120A, a first wired interface 130A, an image processing unit 140A, a frame memory 145A, an image projection unit 150A, an imaging unit 160A, and a first control unit 170A. Hereinafter, the interface is abbreviated as I / F.

[0020] The infrared receiver 110A receives the infrared signal transmitted from the remote control 115, and outputs an operation signal corresponding to the operation content indicated by the received infrared signal to the first control unit 170A.

[0021] The first wireless interface 120A is an interface for wireless communication with external devices, including the information processing device 200. The first wireless interface 120A may include, for example, a wireless LAN card, and communicate wirelessly with external devices to send and receive various types of information. The first wireless interface 120A may also be configured to include an interface circuit and a wireless antenna.

[0022] The first wired interface 130A is connected to the information processing device 200 and the second projector 100B in a communicative manner, and is an interface that receives image signals transmitted from the information processing device 200 and transmits the received image signals to the second projector 100B. The first wired interface, I / F130A, uses interfaces that support standards such as HDMI, DisplayPort, and USB Type-C.

[0023] The image processing unit 140A is connected to the frame memory 145A. The image processing unit 140A processes the image data input from the first wired interface 130A into the frame memory 145A. The frame memory 145A is composed of, for example, SDRAM (Synchronous Dynamic Random Access Memory).

[0024] The image processing unit 140A performs image processing on the image data expanded in the frame memory 145A, such as resolution conversion or resizing, distortion correction, shape correction, digital zoom, and adjustment of image hue and brightness. The image processing unit 140A executes the image processing specified by the first control unit 170A and, if necessary, uses parameters input from the first control unit 170A to perform the processing. The image processing unit 140A can also, of course, combine and execute multiple of the above image processing operations. The image processing unit 140A reads the processed image data from the frame memory 145A and outputs the read image data to the image projection unit 150A.

[0025] The image processing unit 140A and the frame memory 145A are composed of, for example, integrated circuits. Integrated circuits include LSIs (Large Scale Integrated Circuits), ASICs (Application Specific Integrated Circuits), PLDs (Programmable Logic Devices), FPGAs (Field-Programmable Gate Arrays), SoCs (System-on-a-chip), etc. Furthermore, analog circuits may be included as part of the configuration of the integrated circuit, and the first control unit 170A may be configured in combination with an integrated circuit.

[0026] The image projection unit 150A includes a light source 151A, an optical modulator 153A, and an optical unit 155A. Light source 151A includes discharge-type light sources such as ultra-high pressure mercury lamps and metal halide lamps, or solid-state light sources such as light-emitting diodes and semiconductor lasers. Light emitted from light source 151A is separated into red, green, and blue light by a color separation optical system (not shown).

[0027] The optical modulator 153A comprises three liquid crystal panels corresponding to the red, green, and blue color components. The optical modulation elements of the optical modulator 153A are not limited to transmissive liquid crystal panels; for example, reflective liquid crystal panels or DMDs (Digital Micromirror Devices) may also be used.

[0028] The color light, separated into red, green, and blue color components by the color separation optical system, is incident on three liquid crystal panels corresponding to the red, green, and blue color components. Each liquid crystal panel is composed of a transmissive liquid crystal panel in which liquid crystal is sealed between a pair of transparent substrates. The liquid crystal panel has a pixel region formed with multiple pixels arranged in a matrix, and a driving voltage can be applied to each pixel of the liquid crystal.

[0029] The optical modulator 153A includes a panel drive unit that drives each of the three liquid crystal panels corresponding to the color components. The panel drive unit applies a drive voltage corresponding to the input image data to each pixel in the pixel area, setting each pixel of the three liquid crystal panels to a light transmittance corresponding to the image data. Light emitted from the light source 151A is modulated pixel by pixel as it passes through the pixel area of ​​the liquid crystal panel, forming image light corresponding to the image data for each color. The formed image light of each color is combined pixel by a color synthesis optical system (not shown) to form image light representing a color image. The optical unit 155A includes a projection lens and the like, and magnifies and projects the image light modulated by the liquid crystal panel onto the projection surface 10. As a result, an image based on the image data is displayed on the projection surface 10.

[0030] The imaging unit 160A corresponds to an imaging device. The imaging unit 160A includes an imaging lens and an image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary MOS). The imaging lens and image sensor are not shown in the diagram. The imaging unit 160A captures an area including the projection surface 10 and generates an image at the direction of the first control unit 170A. The imaging unit 160A outputs the generated image to the first control unit 170A. Note that it is not necessary for both the first projector 100A and the second projector 100B to have imaging units 160A and 160B; it is sufficient if at least one of the first projector 100A and the second projector 100B has an imaging unit 160A. In this embodiment, the case in which the first projector 100A has an imaging unit 160A will be described.

[0031] The first control unit 170A is a computer device comprising a first storage unit 180A and a first processor 190A.

[0032] The first storage unit 180A includes, for example, ROM (Read Only Memory) and RAM (Random Access Memory). The ROM stores the control program 185A that controls the operation of the first projector 100A, as well as various setting information. The RAM is used as the arithmetic area for the first processor 190A.

[0033] The first processor 190A is an arithmetic processing unit equipped with a processor such as a CPU (Central Processing Unit) or an MPU (Micro-Processing Unit). The first processor 190A may be composed of a single processor or multiple processors. Furthermore, the first processor 190A may be composed of part or all of the first storage unit 180A or an SoC integrated with other circuits. Also, the first processor 190A may be composed of a combination of a CPU that executes programs and a DSP (Digital Signal Processor) that performs predetermined arithmetic processing. Moreover, all the functions of the first processor 190A may be implemented in hardware, or they may be configured using programmable devices.

[0034] [3. Configuration of the Information Processing Device] Figure 3 is a block diagram showing the configuration of the information processing device 200. The configuration of the information processing device 200 will be explained with reference to Figure 3. The information processing device 200 includes a second wireless I / F 210, a second wired I / F 220, a touch panel 230, an operation unit 240, and a second control unit 250.

[0035] The second wireless interface 210 is an interface for wireless communication with external devices, including the projector 100. The second wireless interface 210 corresponds to the data acquisition unit. For example, the second wireless interface 210 may be equipped with a wireless LAN card and perform wireless communication with external devices to send and receive various information. The second wireless interface 210 may also be configured to include an interface circuit and a wireless antenna.

[0036] The second wired interface 220 is connected to the first projector 100A via cable 7 and transmits image signals to the first projector 100A. The second wired interface 220 may have an interface with terminals compatible with standards such as HDMI, DisplayPort, or USB Type-C. The second wired interface 220 may also have an interface circuit.

[0037] The touch panel 230 corresponds to the display unit. The touch panel 230 comprises a display panel such as a liquid crystal panel or an organic EL (Electro-Luminescence) panel, and a touch sensor that detects touch operations on the touch panel 230. The touch sensor detects the user's touch operation and outputs coordinate values ​​indicating the position of the detected touch operation to the second control unit 250. These coordinates are coordinate values ​​in the coordinate system set on the display panel.

[0038] The operation unit 240 is equipped with an input device such as a mouse or keyboard and accepts user input. The operation unit 240 outputs an operation signal corresponding to the received operation to the second control unit 250.

[0039] The second control unit 250 is a computer device comprising a second storage unit 260 and a second processor 270. The second control unit 250 corresponds to the control unit.

[0040] The second storage unit 260 includes, for example, ROM and RAM. The ROM stores control programs 265 that control the operation of each part of the information processing device 200, as well as various setting information. The control programs include application programs 265A. Hereinafter, application programs 265A will be referred to as APP265A. The RAM is used as the arithmetic area of ​​the second processor 270. The second storage unit 260 may also be configured to include auxiliary storage devices such as HDDs (Hard Disk Drives) or SSDs (Solid State Drives).

[0041] The second processor 270 is an arithmetic processing unit equipped with a processor such as a CPU or MPU. The second processor 270 may be composed of a single processor or multiple processors. Alternatively, the second processor 270 may be composed of part or all of the second memory unit 260 or an SoC integrated with other circuits. Furthermore, the second processor 270 may be composed of a combination of a CPU that executes programs and a DSP that executes predetermined arithmetic processing. In addition, all the functions of the second processor 270 may be implemented in hardware, or it may be configured using programmable devices.

[0042] [4. Operation of the Information Processing Device] Next, the operation of the information processing device 200 for achieving stacked projection using multiple projectors 100 will be explained with reference to the flowchart shown in Figure 4 and Figures 5 to 10. Stacking projection is a projection method in which the same image is projected by overlapping multiple projectors 100, resulting in a brighter image than when projecting a single image with only one projector 100. This section describes the operation of the information processing device 200 when stacking projection is performed using two projectors 100, a first projector 100A and a second projector 100B.

[0043] First, the user selects APP265A by operating the control unit 240 or the touch panel 230. The following describes the case where the user performs all operations by touch operation on the touch panel 230. The APP265A selected here is a program that sets the projector 100 to stacking projection under the control of the information processing device 200.

[0044] When the second control unit 250 receives an operation to select APP265A, it executes the APP265A selected by the received operation and displays the initial screen of the APP screen 300 on the touch panel 230. Although the initial screen of the APP screen 300 is not shown in the diagram, this initial screen allows settings such as stacking projection, tiling projection, and edge blending. The user selects stacking projection by touch operation.

[0045] Figure 5 shows an example of the first APP screen 310. When stacking projection is selected via touch operation, the second control unit 250 displays the first APP screen 310 shown in Figure 5 (step S1). The first APP screen 310 is an example of an APP screen 300 that is displayed when the second processor 270 executes APP265A.

[0046] The first APP screen 310 displays a projector selection unit 311, an OK button 312, and a back button 313. The projector selection unit 311 displays identification information for projectors 100 connected to the wireless network 5. The identification information for projectors 100 includes, for example, the name and IP address of the projector 100. The user selects the projector 100 to be used for stacking projection from among the projectors 100 whose identification information is displayed on the projector selection unit 311 by touch operation and presses the OK button 312. Of course, the user can select multiple projectors 100. The second control unit 250 accepts the selection of projectors 100 to be used for stacking projection when the OK button 312 is pressed (step S2).

[0047] When the OK button 312 on the first APP screen 301 is pressed, the second control unit 250 displays the second APP screen 320 on the touch panel 230 (step S3). Next, the second control unit 250 instructs the first projector 100A to project the first position adjustment image 20 and the second projector 100B to project the second position adjustment image 30 (step S4).

[0048] Figure 6 shows an example of the second APP screen 320. Figure 7 shows an example of the first position adjustment image 20 projected onto the projection surface 10 by the first projector 100A, and the second position adjustment image 30 projected onto the projection surface 10 by the second projector 100B.

[0049] First, let's explain the second APP screen 320 shown in Figure 6. The second app screen 320 is a screen that accepts operations to adjust the position of the image projected by the projector 100 relative to the projection surface 10. The second app screen 320 displays the projector selection section 321, the image adjustment section 330, the OK button 326, and the back button 327. The image adjustment unit 330 includes a focus setting unit 322, a distortion setting unit 323, a zoom setting unit 324, and a lens shift setting unit 325.

[0050] The projector selection unit 321 accepts an operation to select the projector 100 to be operated on. The projector selection unit 321 displays the identification information of the projector 100 selected by the projector selection unit 311 shown in Figure 4. The user selects the projector 100 to adjust the image projection position by touch operation.

[0051] The focus setting unit 322 accepts operations to change the focus of the projection lens of the projector 100 selected by the projector selection unit 321. The focus setting unit 322 includes an operator 3221 that accepts user operations, a bar display 3223 that shows the range of changes that can be made by operating the operator 3221, and the current focus position.

[0052] The distortion setting unit 323 accepts operations to change the distortion of the projection lens of the projector 100 selected by the projector selection unit 321. The distortion setting unit 323 includes an operator 3231 that accepts user operations, a range of changes that can be made by operating the operator 3231, and a bar display 3233 that shows the current distortion.

[0053] The zoom setting unit 324 accepts operations to change the zoom of the projection lens of the projector 100 selected by the projector selection unit 321. The zoom setting unit 324 includes an operator 3241 that accepts user operations, and a bar display 3243 that shows the range that can be changed by operating the operator 3241, as well as the current zoom position.

[0054] The lens shift setting unit 325 accepts operations to change the lens position of the projection lens of the projector 100 selected by the projector selection unit 321. The lens shift setting unit 325 includes up, down, left, and right keys as control elements 3251.

[0055] Next, with reference to Figure 7, we will explain the first position adjustment image 20 and the second position adjustment image 30, and the adjustment of the projection position performed using these first and second position adjustment images 20 and 30. The first position adjustment image 20 is an image with rectangular shapes 21, 22, 23, and 24 placed at the four corners, as shown in Figure 7, and the second position adjustment image 30 is also an image with rectangular shapes 31, 32, 33, and 34 placed at the four corners, as shown in Figure 7.

[0056] The projection position adjustment performed using the second APP screen 320 can be performed, for example, as shown in Figure 7, by ensuring that figures 21 and 31, 22 and 32, 33 and 33, and 24 and 34 partially overlap, and that the first position adjustment image 20 and the second position adjustment image 30 are projected to cover the entire projection surface 10. In other words, the first position adjustment image 20 and the second position adjustment image 30 do not need to overlap in such a way that the vertices of the four corners of the first position adjustment image 20 and the second position adjustment image 30 perfectly coincide.

[0057] The user operates the image adjustment unit 330 on the second APP screen 320 while referring to the first position adjustment image 20 and the second position adjustment image 30 projected onto the projection surface 10, and adjusts the projection position of the first position adjustment image 20 and the second position adjustment image 30 (step S5). In other words, the user operates the image adjustment unit 330 so that the shapes 21 and 31, 22 and 32, 33 and 33, and 24 and 34 of the first position adjustment image 20 and the second position adjustment image 30 partially overlap. Similarly, the user operates the image adjustment unit 330 so that the first position adjustment image 20 and the second position adjustment image 30 cover the entire projection surface 10.

[0058] When the second control unit 250 receives an operation to change the focus, distortion, zoom, or lens shift via the second APP screen 320, it transmits a control signal corresponding to the received operation to the corresponding projector 100. Here, we will explain assuming that the first projector 100A is selected by the projector selection unit 321 and the second control unit 250 transmits a control signal to the first projector A. The first control unit 170A of the first projector 100A changes the focus, distortion, zoom, or lens position of the projection lens according to the control signal received from the information processing device 200.

[0059] Figure 8 shows an example of the third APP screen 340. The third APP screen 340 is an example of the APP screen 300 that is displayed when the second processor 270 executes APP265A. When the projection position adjustment performed using the second APP screen 320 is completed and the OK button 326 is pressed, the second control unit 250 displays the third APP screen 340 shown in Figure 8 on the touch panel 230 (step S6). The third APP screen 340 shown in Figure 8 displays a guidance display 341 asking whether or not to perform mask processing to limit the exposure adjustment range, as well as a YES button 343, a NO button 345, an OK button 347, and a back button 359.

[0060] If the user wants to perform masking, they select the YES button 343 and press the OK button 347. If the user does not want to perform masking, they select the NO button 345 and press the OK button 347. The second control unit 250 determines whether masking has been selected based on the operation received from the third APP screen 340 (step S7).

[0061] The second control unit 250 determines that mask processing was not selected when the NO button 345 is selected and the OK button 347 is pressed on the third APP screen 340 (step S7 / NO). In this case, the process proceeds to step S17. Details of the process in step S17 will be described later.

[0062] The second control unit 250 determines that mask processing has been selected when the YES button 343 is selected on the third APP screen 340 and the OK button 347 is pressed (step S7 / YES). In this case, the second control unit 250 displays the fourth APP screen 350 on the touch panel 230 and instructs the projection and imaging of the guide image 40 (step S9). This flowchart describes the case where the guide image 40 is projected onto the first projector 100A, but the guide image 40 may also be projected onto the second projector 100B.

[0063] Figure 9 shows an example of a guide image 40 projected by the first projector 100A onto the projection surface 10. The guide image 40 is an image with black borders formed on the edges of all four sides of the guide image 40. The guide image 40 is not limited to the example shown in Figure 9, and may be, for example, an image in which a shape of any shape is placed within the area including the four corners of the guide image 40.

[0064] The first projector 100A projects the guide image 40 onto the projection surface 10 according to the instructions of the information processing device 200. In step S5, during the adjustment of the projection position, the first position adjustment image 20 is adjusted to cover the entire projection surface 10. Therefore, the guide image 40 projected by the first projector 100A is also projected to cover the entire projection surface 10.

[0065] Next, the first projector 100A captures an image of the area including the projection surface 10 onto which the guide image 40 is projected, and generates an image. This image is called the first image 400. The first projector 100A transmits the generated first image 400 to the information processing device 200. The second control unit 250 receives the first captured image 400 transmitted by the first projector 100A (step S10) and displays the received first captured image 400 on the fourth APP screen 350 (step S11).

[0066] Figure 10 shows an example of the fourth APP screen 350. The fourth APP screen 350 is an example of the APP screen 300 that is displayed when the second processor 270 executes APP265A. Now, let's explain the fourth app screen, 350. The fourth APP screen 350 displays the image display unit 351, the control point change operation unit 355, the re-image button 356, the exposure adjustment unit 357, the OK button 358, and the back button 359.

[0067] The second control unit 250 displays the first captured image 400 received from the first projector 100A on the image display unit 351 of the fourth APP screen 350. The first captured image 400 is an image that includes the guide image 40. Furthermore, the second control unit 250 superimposes the setting frame 360 ​​onto the first captured image 400 displayed on the image display unit 351 (step S12).

[0068] The setting frame 360 ​​shown in Figure 10 includes control points 361, 362, 363, and 364 positioned at the four vertices of the setting frame 360, and four line segments 365, 366, 367, and 368 connecting these control points 361, 362, 363, and 364.

[0069] Setting frame 360 ​​is an image of a frame that defines the unmasked areas that will not be masked. The area outside the setting frame 360 ​​is the mask area where masking is performed, and the area inside the setting frame 360 ​​is the non-masked area where masking is not performed. For example, if the first captured image 400 includes ambient light entering through the small window 9 shown in Figure 10, exposure adjustment is performed based on the brightness of the ambient light. As a result, the brightness of the projected image captured in the first captured image 400 may appear dark, and it may be difficult to recognize the positional relationship between the projected image and the projection surface 10.

[0070] The second control unit 250 receives a first operation to correct the shape and position of the setting frame 360 ​​via touch operation (step S13). For example, the user performs the first operation to correct the position and shape of the setting frame 360 ​​via touch operation while referring to the guide image 40 displayed on the first captured image 400. The user changes the position and shape of the setting frame 360 ​​by changing the positions of control points 361, 362, 363, and 364 via touch operation and sets the non-masked area.

[0071] As described later, the second control unit 250 masks the area of ​​the first captured image 400 outside the setting frame 360 ​​and calculates the average value of the brightness of the pixels in the area of ​​the first captured image 400 inside the setting frame 360 ​​as the target value for exposure adjustment. In this case, it is preferable to set the setting frame 360 ​​so that the ratio of the area of ​​the guide image 40 to the area of ​​the setting frame 360 ​​set by the user is high. This is because if the ratio of the area of ​​the guide image 40 to the area of ​​the setting frame 360 ​​set by the user is low, the target value for exposure adjustment will become smaller than the brightness value of the guide image 40, which suppresses a decrease in the accuracy of exposure adjustment.

[0072] Next, the second control unit 250 determines whether or not the OK button 358 has been pressed (step S14). If the OK button 358 has not been pressed (step S14 / NO), the second control unit 250 returns to the process in step S10 and accepts the first operation. When a non-masked area is set in the first captured image 400 by user operation and the OK button 358 is pressed (step S14 / YES), the second control unit 250 calculates the target value for exposure adjustment (step S15).

[0073] The second control unit 250 masks the area of ​​the first captured image 400 outside the setting frame 360. For example, the second control unit 250 calculates the average value of the brightness of the pixels in the area of ​​the first captured image 400 inside the setting frame 360. The second control unit 250 sets the calculated average value of brightness to the target value for exposure adjustment. This suppresses a decrease in the detection accuracy of the image projected by the projection device from the captured image due to the brightness value of the projection surface 10. Subsequently, the second control unit 250 transmits the set target value for exposure adjustment to the first projector 100A as control information (step S16). Alternatively, the second control unit 250 may generate control information to adjust at least one of the settings of the shutter speed, gain, and aperture of the imaging unit 160A so that the average value of the brightness of the first captured image 400 of the imaging unit 160A becomes the target value for exposure adjustment, and transmit the generated control information to the first projector 100A. Furthermore, the target value for exposure adjustment may be the average value of the brightness values ​​of pixels in the region of the first captured image 400 inside the setting frame 360, where the brightness value of the pixels is equal to or greater than a first predetermined value. Here, the first predetermined value is, for example, the lower limit of the brightness value of the projection surface 10.

[0074] The first control unit 170A of the first projector 100A adjusts at least one of the settings of the shutter speed, gain, and aperture of the imaging unit 160A based on the target exposure adjustment value indicated by the control information received from the information processing device 200. The first control unit 170A also adjusts at least one of the settings of the shutter speed, gain, and aperture of the imaging unit 160A according to the control information received from the information processing device 200. The first control unit 170A adjusts at least one of the settings of the shutter speed, gain, and aperture of the imaging unit 160A to generate a first image 400, and then transmits the generated first image 400 to the information processing device 200.

[0075] Figure 10 shows a rectangular setting frame 360, but the shape of the setting frame 360 ​​is not limited to a rectangle. The control point number change operation unit 355 includes an operator 3551 that increases the number of control points and an operator 3553 that decreases the number of control points. The number of control points can be changed by operating operator 3551 or operator 3553. Operating operator 3551 or operator 3553 corresponds to a third operation. For example, the shape of the setting frame 360 ​​can be a triangle, a pentagon, a hexagon, or other polygon. The user presses operator 3551 and then touches any of the line segments 365, 366, 367, and 368 that make up the setting frame 360, and a new control point is added at the position of the touched line segment 365, 366, 367, and 368.

[0076] The re-imaging button 356 is a button that causes the first projector 100A to capture the first image 400 again. When the re-imaging button 356 is pressed, the second control unit 250 instructs the first projector 100A to capture the image again. As a result, the image display unit 351 displays the first image 400 that has been re-captured by the first projector 100A.

[0077] The exposure adjustment unit 357 includes an operator 3571 that accepts operations to raise or lower the calculated target brightness value, and a bar display 3573 that shows the exposure adjustment range and the current exposure setting position. The user can change the target brightness value by operating the operator 3571. Operating the operator 3571 and the re-image button 356 constitute a second operation. When the operator 3571 is operated and the re-image button 356 is operated, the changed target value of the exposure adjustment is transmitted to the first projector 100A as control information.

[0078] The first control unit 170A of the first projector 100A adjusts at least one of the shutter speed, gain, and aperture settings of the imaging unit 160A based on the target exposure value indicated by the control information received from the information processing device 200. The first control unit 170A also adjusts at least one of the shutter speed, gain, and aperture settings of the imaging unit 160A according to the control information received from the information processing device 200. After adjusting at least one of the shutter speed, gain, and aperture settings of the imaging unit 160A, the first control unit 170A generates a first image 400 and transmits the generated first image 400 to the information processing device 200. As a result, the image display unit 351 displays the first image 400, which has been exposure-adjusted according to the control information generated by the second control unit 250.

[0079] Next, the second control unit 250 instructs the first projector 100A to project the first pattern image and generate the captured image (step S17). The first pattern image is, for example, an image in which figures of a predetermined shape are arranged at predetermined intervals in the vertical and horizontal directions of the pattern image. The figures may be, for example, circles, triangles, or squares.

[0080] The first projector 100A projects a pattern image onto the projection surface 10 according to instructions from the information processing device 200, and the imaging unit 160A captures the projection surface 10 onto which the pattern image has been projected to generate an image. This image is called the second image. The first projector 100A transmits the generated second image to the information processing device 200.

[0081] Next, the second control unit 250 instructs the second projector 100B to project the second pattern image and instructs the first projector 100A to generate the captured image (step S18). The second pattern image may be the same as the first pattern image or it may be a different pattern. The first and second pattern images may be patterns in which figures whose positions can be identified in the first and second pattern images are arranged, and for example, the shapes of the figures arranged in the first and second pattern images may be different.

[0082] The second projector 100B projects the second pattern image onto the projection surface 10 according to instructions from the information processing device 200. The first projector 100A generates an image by having the imaging unit 160A capture the projection surface 10 onto which the second pattern image has been projected. This image is called the third image. The first projector 100A transmits the generated third image to the information processing device 200.

[0083] The second control unit 250 receives the second and third captured images transmitted from the first projector 100A (step S19). Having acquired the second and third captured images, the second control unit 250 performs a shape detection process (step S20).

[0084] The second control unit 250 performs image analysis of the second captured image as a shape detection process, detects shapes included in the pattern image, and identifies the position of the detected shape in the second captured image. Similarly, the second control unit 250 performs image analysis on the third captured image as a shape detection process to detect shapes included in the pattern image and to determine the position of the detected shapes in the third captured image.

[0085] Next, the second control unit 250 generates correction information (step S21). This correction information is information that matches the position of each figure included in the second captured image with the position of each figure included in the third captured image. For example, the second control unit 250 generates correction information that corrects the position of a figure included in the second captured image to the position of the corresponding figure included in the third captured image.

[0086] When the second control unit 250 generates correction information, it outputs the generated correction information to at least one of the first projector 100A and the second projector 100B (step S22). Here, we will explain assuming that the correction information is output only to the first projector 100A.

[0087] Next, the second control unit 250 transmits an image signal including image data to the first projector 100A via the cable 7 (step S23). When the first projector 100A receives the image signal from the information processing device 200, it captures the received image signal and transmits it to the second projector 100B.

[0088] When the first projector 100A receives an image signal from the information processing device 200, it acquires image data contained in the received image signal and corrects the acquired image data based on correction information. The first projector 100A generates image light based on the image data corrected based on the correction information. The first projector 100A projects the generated image light onto the projection surface 10. The second projector 100B also generates image light based on image data and projects the generated image light onto the projection surface 10. The second projector 100B generates image light based on image data and projects the generated image light onto the projection surface 10, causing the image projected by the first projector 100A and the image projected by the second projector 100B to overlap, thereby achieving stacked projection.

[0089] [5. Variant] In the embodiment described above, the case of stacking projection using two projectors 100, a first projector 100A and a second projector 100B, was explained as an example. Even when correcting the position and shape of the image projected by a single projector 100, exposure adjustment can be appropriately performed even when ambient light is captured in the captured image by executing the processes in steps S1 to S13 of the flowchart shown in Figure 4. Therefore, by capturing the pattern image projected onto the projection surface 10 with the exposure-adjusted imaging unit 160A, the position and shape of the image projected by a single projector 100 can be corrected with high accuracy.

[0090] The embodiments described above are preferred embodiments of the present invention. However, the invention is not limited to the embodiments described above, and various modifications can be made without departing from the spirit of the invention.

[0091] For example, in the above-described embodiment, the case in which the information processing device 200 is operated as a control device was explained, but the first projector 100A or the second projector 100B may also be operated as a control device. Also, in the above-described embodiment, the case in which the first projector 100A is equipped with an imaging unit 160A was explained, but the imaging unit 160B may be provided in the information processing device 200, or an imaging device may be provided outside the first projector 100A or the information processing device 200.

[0092] Furthermore, although the above-described embodiment illustrates an optical modulator 153A equipped with a liquid crystal panel, the liquid crystal panel may be a transmissive liquid crystal panel or a reflective liquid crystal panel. Also, the optical modulator 153A may use a digital mirror device instead of a liquid crystal panel. Alternatively, a configuration combining a digital mirror device and a color wheel may be used. In addition, the optical modulator 153A may employ a configuration capable of modulating light emitted from a light source, in addition to the liquid crystal panel A and the digital mirror device.

[0093] Furthermore, the functional units of the information processing device 200 shown in Figure 3 represent functional configurations, and the specific implementation form is not particularly limited. In other words, it is not necessarily required that hardware corresponding to each functional unit be implemented individually, and it is certainly possible to have a configuration in which a single processor executes a program to realize the functions of multiple functional units. Also, some of the functions realized by software in the above embodiment may be realized by hardware, or some of the functions realized by software may be realized by software. In addition, the specific detailed configurations of other parts of the projector can also be arbitrarily changed without departing from the spirit of the present invention. The same applies to the configurations of the first projector 100A and the second projector 100B shown in Figure 2.

[0094] Furthermore, the processing units in the flowchart shown in Figure 4 are divided according to their main processing content in order to facilitate understanding of the processing of the information processing device 200. The present invention is not limited by the way the processing units are divided or the names of the processing units shown in the flowchart in Figure 4. In addition, the processing of the information processing device 200 can be further divided into more processing units depending on the processing content, or it can be divided so that one processing unit contains even more processing. Also, the processing order in the flowchart above is not limited to the example shown.

[0095] [6. Summary of this disclosure] A summary of this disclosure is provided below. (Note 1) An exposure adjustment method comprising: a control device acquiring an image from an imaging device that captures an area including a projection surface on which an image is projected; displaying the acquired image; receiving a first operation to correct the shape and position of the unmasked area of ​​the image; setting the unmasked area in the image based on the first operation; calculating a target value for exposure adjustment based on the unmasked area of ​​the image; and outputting the calculated target value to the imaging device; and the imaging device adjusting at least one setting value of the shutter speed, gain, and aperture of the imaging device based on the target value.

[0096] As a result, even if ambient light or other light enters areas outside the unmasked region and the brightness becomes higher than the projected image, the target exposure value is calculated based on the captured image of the unmasked region, and the imaging device performs exposure adjustment based on the calculated target value. Therefore, the exposure adjustment of the imaging device can be optimized, and a decrease in the accuracy of detecting the projection surface range from the captured image of the imaging device can be suppressed.

[0097] (Note 2) The exposure adjustment method according to Appendix 1, wherein, before the control device sets the unmasked area in the captured image, the control device accepts a second operation to change the target value, outputs the changed target value to the imaging device, the imaging device adjusts the setting value of the imaging device based on the changed target value, captures the area including the projection surface to generate a captured image, outputs the captured image to the control device, and the control device displays the captured image.

[0098] This allows the user to change the target value for exposure adjustment and display the image captured by the imaging device after exposure adjustment based on the changed target value, thereby improving the visibility of the projection surface area in the captured image when the user corrects the shape and position of the setting frame.

[0099] (Note 3) The exposure adjustment method according to Appendix 1 or 2, wherein the unmasked area is a polygon with five or more control points as vertices.

[0100] This allows the unmasked area to be set according to the shape of the projection surface, even when the projection surface has a polygon or curved shape with five or more vertices, by setting the unmasked area as a polygon with five or more control points as vertices.

[0101] (Note 4) An exposure adjustment method according to Appendix 1 or 2, comprising: accepting a third operation to change the number of control points that define the shape of the unmasked region; and setting the unmasked region in the captured image by a polygon whose vertices are the control points whose number of points has been changed by the third operation.

[0102] This allows for the creation of unmasked areas using polygons that correspond to complex shapes, even if the projection surface has a complex shape, by increasing or decreasing the number of control points.

[0103] (Note 5) The first projection device projects a guide image that guides the setting of the unmasked area; the control device acquires a first image captured from the imaging device, which captures the area including the projected guide image; the control device displays the acquired first image; the control device accepts a first operation to correct the shape and position of the unmasked area set based on the guide image displayed on the first image; the control device sets the unmasked area in the first image based on the first operation; the control device calculates a target value for exposure adjustment based on the unmasked area in the first image; the control device outputs the calculated target value to the imaging device; and the imaging device performs a shutter adjustment based on the target value. An image correction method comprising: adjusting at least one setting value among shutter speed, gain, and aperture; a first projection device projecting a first pattern image having a predetermined pattern; a control device acquiring a second image image including the first pattern image, which is captured by the imaging device whose setting value has been adjusted based on the target value; a second projection device projecting a second pattern image having a predetermined pattern; a control device acquiring a third image image including the second pattern image, which is captured by the imaging device whose setting value has been adjusted based on the target value; and a control device correcting the shape of the image projected by at least one of the first projection device and the second projection device based on the second image image and the third image image.

[0104] As a result, the imaging device with adjusted settings captures a second image containing the first pattern image, and a third image containing the second pattern image. Based on the second and third images, the shape of the image projected by at least one of the first and second projection devices is corrected. Therefore, the shape of the image projected by at least one of the first and second projection devices can be corrected with high accuracy.

[0105] (Note 6) An exposure adjustment system comprising: an acquisition unit that acquires an image from an imaging device that captures an area including a projection surface onto which an image is projected; a display unit that displays the acquired image; an operation unit that accepts a first operation to correct the shape and position of the unmasked area of ​​the image; a control unit that performs the following actions: setting the unmasked area in the image based on the first operation; calculating a target value for exposure adjustment based on the unmasked area of ​​the image; and outputting the calculated target value to the imaging device; and an imaging device that adjusts at least one setting value of the shutter speed, gain, and aperture of the imaging device based on the target value.

[0106] As a result, even if ambient light or other light enters areas outside the unmasked region and the brightness becomes higher than the projected image, the target exposure value is calculated based on the captured image of the unmasked region, and the imaging device performs exposure adjustment based on the calculated target value. Therefore, the exposure adjustment of the imaging device can be optimized, and a decrease in the accuracy of detecting the projection surface range from the captured image of the imaging device can be suppressed.

[0107] (Note 7) The exposure adjustment system according to Appendix 6, wherein, before the control device sets the unmasked area in the captured image, the control unit receives a second operation to change the target value via the operation unit, outputs the changed target value to the imaging device, the imaging device adjusts the setting value of the imaging device based on the changed target value, captures the area including the projection surface to generate a captured image, outputs the captured image to the control device, and the control unit displays the captured image on the display unit.

[0108] This allows the user to change the target value for exposure adjustment and display the image captured by the imaging device that has adjusted the exposure based on the changed target value. This improves the visibility of the projection surface area in the captured image when the user corrects the shape and position of the setting frame.

[0109] (Note 8) The exposure adjustment system according to Appendix 6 or 7, wherein the unmasked region is a polygon having five or more vertices.

[0110] This allows the unmasked area to be set according to the shape of the projection surface, even when the projection surface has a polygon or curved shape with five or more vertices, by setting the unmasked area as a polygon with five or more control points as vertices. [Explanation of Symbols]

[0111] 3...Wireless router, 5...Wireless network, 7...Cable, 9...Small window, 10...Projection surface, 20...First projected image, 30...Second projected image, 100...Projector, 100A...First projector, 100B...Second projector, 110A...Remote control receiver, 115...Remote control, 120A...First wireless I / F, 130A...First wired I / F, 140A...Image processing unit, 145A...Frame memory, 150A...Image projection unit, 151A...Light source, 153A...Optical modulation device, 155A...Optical unit, 160A...Imaging unit, 170A...First control unit, 180A...First storage unit, 185A...Control program, 190A...First processor, 200...Information processing device, 210...Second wireless I / F, 220...Second wired I / F, 230...Touch panel, 240...Operation unit, 250...Second control unit, 260...Second storage unit, 2 65...Control program, 265A...Application program, 270...Second processor, 300...APP screen, 311...Projector selection section, 312...OK button, 313...Back button, 321...Projector selection section, 322...Focus setting section, 323...Distortion setting section, 324...Zoom setting section, 325...Lens shift setting section, 326...OK button, 327...Back button 341...Guidance display, 343...YES button, 345...NO button, 347...OK button, 349...Back button, 351...Image display unit, 355...Control point change operation unit, 356...Re-imaging button, 357...Exposure adjustment unit, 358...OK button, 359...Back button, 360...Setting frame, 361, 362, 363, 364...Control points, 365, 366, 367, 368...Line segments, 400...First captured image.

Claims

1. The control device Acquiring an image from an imaging device that captures an area including the projection surface onto which the image is projected, Displaying the acquired image, The system accepts a first operation to correct the shape and position of the non-masked region of the captured image, Setting the non-masked region in the captured image based on the first operation, The target value for exposure adjustment is calculated based on the non-masked region of the captured image. The calculated target value is output to the imaging device, and the following is performed: The aforementioned imaging device Adjusting at least one of the settings of the shutter speed, gain, and aperture of the imaging device based on the aforementioned target value, A method for adjusting exposure to perform this action.

2. Before the control device sets the unmasked area in the captured image, The control device, The system accepts a second operation to change the aforementioned target value, Outputting the modified target value to the imaging device, The aforementioned imaging device Adjusting the setting value of the imaging device based on the changed target value, The process involves capturing an image of the area including the projection surface and generating an captured image, Outputting the captured image to the control device, The control device, Displaying the aforementioned captured image, The exposure adjustment method according to claim 1, which performs the following:

3. The exposure adjustment method according to claim 1, wherein the unmasked region is a polygon with five or more control points as vertices.

4. The system accepts a third operation to change the number of control points that define the shape of the non-masked region, The non-masked region is set in the captured image by a polygon whose vertices are the control points whose points have been changed by the third operation, The exposure adjustment method according to claim 1, comprising having

5. The first projection device projects a guide image that guides the setting of the non-masked area, The control device acquires a first image from the imaging device that captures the area including the projected guide image, The control device displays the acquired first image, The control device receives a first operation to correct the shape and position of the unmasked region, which is set based on the guide image displayed in the first captured image. The control device sets the non-masked region in the first captured image based on the first operation, The control device calculates a target value for exposure adjustment based on the unmasked region of the first captured image, The control device outputs the calculated target value to the imaging device, The imaging device adjusts at least one of the settings of the shutter speed, gain, and aperture of the imaging device based on the target value. The first projection device projects a first pattern image having a pre-set pattern, The control device acquires a second image, which includes the first pattern image, captured by the imaging device, whose set value has been adjusted based on the target value. The second projection device projects a second pattern image having a pre-set pattern, The control device acquires a third image, which includes the second pattern image, by imaging the imaging device, whose set value is adjusted based on the target value. The control device corrects the shape of the image projected by at least one of the first projection device and the second projection device based on the second captured image and the third captured image. An image correction method that performs this action.

6. An acquisition unit that acquires an image from an imaging device, which captures an area including the projection surface onto which the image is projected, A display unit that displays the acquired image, An operating unit that receives a first operation to correct the shape and position of the non-masked region of the captured image, Setting the non-masked region in the captured image based on the first operation, The target value for exposure adjustment is calculated based on the non-masked region of the captured image. Outputting the calculated target value to the imaging device, A control unit that performs the following: A control device comprising, An imaging device that adjusts at least one of the settings of the shutter speed, gain, and aperture of the imaging device based on the aforementioned target value, An exposure adjustment system equipped with this system.

7. Before the control device sets the unmasked area in the captured image, The control unit, The operation unit accepts a second operation to change the target value, Outputting the modified target value to the imaging device, The imaging device is Adjusting the setting value of the imaging device based on the changed target value, The process involves capturing an image of the area including the projection surface and generating an captured image, Outputting the captured image to the control device, The control unit, The captured image is displayed on the display unit, The exposure adjustment system according to claim 6, which performs the following:

8. The exposure adjustment system according to claim 6, wherein the unmasked region is a polygon having five or more vertices.

Images