Control method, control system, and information processing program

Abstract

To execute multi-projection by causing a user to designate a projection range on a display 150 included in an electronic device different from a projector 30 in order to designate the projection range of a projection image.SOLUTION: A control method includes: acquiring a captured image SI generated by a camera 110, the captured image SI including a first projection image PI1 and a second projection image PI2; causing a display 150 to display a screen including the captured image SI; receiving, via the screen, operation of designating a range in which a composite image TI obtained by combining the first projection image PI1 and the second projection image PI2 is formed on a projection object PO; generating first information AI1 including at least one first adjustment amount AA1 of the first projection image PI1 required for forming the composite image TI on the basis of the operation; and transmitting the first information AI1 to a first projector 30A.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a control method, a control system, and an information processing program. [Background technology]

[0002] Conventionally, when projecting an image using a projector, there is a technology in which a user specifies the projection range of the projected image on a projection target onto which the projected image is projected, using a display provided on an information processing device other than the projector.

[0003] For example, Patent Document 1 discloses an information processing device that displays an image of a room on a display based on room information indicating the three-dimensional shape of the room, and accepts a user's operation to specify a wall surface onto which the projection image will be projected via a touch panel installed on the display, thereby identifying the wall surface specified by the user as the projection surface. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-126127 Summary of the Invention [Problem to be solved by the invention]

[0005] Patent Document 1 does not anticipate cases where stacking projection or tiling projection is performed, in which a plurality of projection images are projected. [Means for solving the problem]

[0006] A control method according to one embodiment of the present invention is a control method for an electronic device capable of communicating with a first projector that projects a first projection image onto a projection target, and includes the steps of: acquiring a captured image generated by a camera, the captured image including the first projection image and a second projection image projected from a second projector different from the first projector; displaying a screen including the captured image on a display; accepting an operation via the screen to specify an area in which a composite image formed by combining the first projection image and the second projection image is to be formed on the projection target; generating first information including at least one first adjustment amount of the first projection image required to form the composite image based on the operation; and transmitting the first information to the first projector.

[0007] A control system according to one embodiment of the present invention is a control system comprising a first projector that projects a first projection image onto a projection target and an electronic device capable of communicating with the first projector, wherein the electronic device performs the following operations: acquiring a captured image generated by a camera, the captured image including the first projection image and a second projection image projected from a second projector different from the first projector; displaying a screen including the captured image on a display; accepting an operation via the screen to specify an area in which a composite image formed by combining the first projection image and the second projection image is to be formed on the projection target; generating first information including at least one first adjustment amount of the first projection image required to form the composite image based on the operation; and transmitting the first information to the first projector; and wherein the first projector adjusts the first projection image based on the first adjustment amount.

[0008] An information processing program according to one embodiment of the present invention causes a computer that controls an electronic device capable of communicating with a first projector that projects a first projection image onto a projection target to acquire an image generated by a camera, the image including the first projection image and a second projection image projected from a second projector different from the first projector; display a screen including the image on a display; accept, via the screen, an operation that specifies an area in which a composite image formed by combining the first projection image and the second projection image is to be formed on the projection target; generate, based on the operation, first information including at least one first adjustment amount of the first projection image required to form the composite image; and transmit the first information to the first projector. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing the overall configuration of a control system 1. [Figure 2] FIG. 1 is a block diagram showing an example of the configuration of an information processing device 10. [Figure 3] 3 is a diagram showing an example of a captured image SI acquired by an acquisition unit 122 from an imaging device 110. FIG. [Figure 4] 10 is a diagram showing an example of a display screen DI that the display control unit 123 causes the display 150 to display and an example of an operation that the reception unit 124 receives. FIG. [Figure 5] 10 is a diagram showing an example of a display screen DI that the display control unit 123 causes the display 150 to display and an example of an operation that the reception unit 124 receives. FIG. [Figure 6] 10 is a diagram showing an example of a display screen DI that the display control unit 123 causes the display 150 to display and an example of an operation that the reception unit 124 receives. FIG. [Figure 7] 10 is a diagram showing an example of a display screen DI that the display control unit 123 causes the display 150 to display and an example of an operation that the reception unit 124 receives. FIG. [Figure 8] 10 is a diagram showing an example of a display screen DI that the display control unit 123 causes the display 150 to display and an example of an operation that the reception unit 124 receives. FIG. [Figure 9]10 is a diagram showing an example of a display screen DI that the display control unit 123 causes the display 150 to display and an example of an operation that the reception unit 124 receives. FIG. [Figure 10] 10 is a diagram showing an example of a display screen DI that the display control unit 123 causes the display 150 to display and an example of an operation that the reception unit 124 receives. FIG. [Figure 11] 10 is a diagram showing an example of a display screen DI that the display control unit 123 causes the display 150 to display and an example of an operation that the reception unit 124 receives. FIG. [Figure 12] FIG. 2 is a block diagram showing the configuration of a first projector 30A. [Figure 13] FIG. 2 is a sequence diagram showing the basic operation of the control system 1 according to the first embodiment. [Figure 14] FIG. 2 is a sequence diagram showing the basic operation of the control system 1 according to the first embodiment. [Figure 15] FIG. 2 is a diagram showing an example of a pattern image PT. [Figure 16] FIG. 10 is a diagram showing an example of a second display screen DI2. [Figure 17] FIG. 10 is a diagram showing an example of a second display screen DI2. [Figure 18] FIG. 13 is a diagram showing a captured image SI in a sixth modification example. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, in each drawing, the dimensions and scale of each part are appropriately different from those of the actual parts. Furthermore, since the embodiments described below are preferred specific examples of the present invention, various technically preferable limitations are applied, but the scope of the present invention is not limited to these embodiments unless otherwise specified in the following description to the effect that the present invention is limited.

[0011] 1: First embodiment 1-1: Configuration of the first embodiment 1-1―1: Overall composition 1 is a block diagram showing the overall configuration of a control system 1 according to the first embodiment. The control system 1 includes one information processing device 10, a first projector 30A, and a second projector 30B. The information processing device 10, the first projector 30A, and the second projector 30B are connected to each other via a communication network NET so as to be able to communicate with each other.

[0012] The information processing device 10 causes the first projector 30A to project a first projection image PI1 onto a projection target PO. The information processing device 10 also causes the second projector 30B to project a second projection image PI2 onto the projection target PO. The projection target PO has a three-dimensional shape and is the projection target onto which the first projection image PI1 and the second projection image PI2 are projected. The projection target PO is, for example, a wall or a screen. The information processing device 10 performs multi-projection by causing the first projector 30A to project the first projection image PI1 and the second projector 30B to project the second projection image PI2. The multi-projection includes stacking and tiling. "Stacking" is a technique for displaying a single image by overlapping a first projected image PI1 and a second projected image PI2 on a projection target PO. Note that "stacking" includes "simple stacking." "Simple stacking" is a technique for increasing the brightness of a composite image TI of the first projected image PI1 and the second projected image PI2, where the first projected image PI1 and the second projected image PI2 are the same image and are projected in an overlapping manner at the same position. On the other hand, "tiling" is a technique for displaying a first projection image PI1 and a second projection image PI2 side by side on the projection target PO. When a user of the control system 1 executes multi-projection using the control system 1, the user executes settings for the multi-projection using the information processing device 10. A specific setting method will be described later.

[0013] The information processing device 10 is, for example, a smartphone. However, the information processing device 10 is not limited to a smartphone. For example, the information processing device 10 may be a tablet or a PC (Personal Computer). The information processing device 10 is also an example of an "electronic device."

[0014] In FIG. 1, the control system 1 includes two projectors 30, a first projector 30A and a second projector 30B. However, the control system 1 can include any number of projectors 30 greater than or equal to two. The information processing device 10 can perform multi-projection using any number of these projectors 30. For ease of explanation, the following describes a case where the control system 1 includes two projectors 30, a first projector 30A and a second projector 30B. However, the operation when the information processing device 10 performs multi-projection using three or more projectors 30 is basically the same as the operation when performing multi-projection using two projectors, a first projector 30A and a second projector 30B, which will be described below.

[0015] Furthermore, while the information processing device 10 adjusts the projection positions of the first projection image PI1 and the second projection image PI2 on the projection target PO using a method described below, it is preferable that the first projection image PI1 is a white image displayed on the projection target PO as a result of white light being projected from the first projector 30A. Similarly, it is preferable that the second projection image PI2 is a white image displayed on the projection target PO as a result of white light being projected from the second projector 30B.

[0016] 1-1-2: Configuration of information processing device 2 is a block diagram showing an example configuration of the information processing device 10. The information processing device 10 includes an imaging device 110, a processing device 120, a storage device 140, a display 150, an input device 160, and a communication device 170. The elements of the information processing device 10 are connected to each other by a single bus or multiple buses for communicating information.

[0017] The imaging device 110 is a device that captures a first projected image PI1 and a second projected image PI2. In this embodiment, as described below, the imaging device 110 generates a single captured image SI that includes both the first projected image PI1 and the second projected image PI2. The imaging device 110 captures various images under the control of the processing device 120. For example, cameras provided in PCs, tablet terminals, and smartphones are preferably used as the imaging device 110, but the present invention is not limited to this. The imaging device 110 may also be an external camera such as a web camera.

[0018] In the following description, it is assumed that the screen adjustment functions, such as the screen enlargement / reduction function and the geometric correction function, of the first projector 30A and the second projector 30B have been initialized when the imaging device 110 generates the captured image SI. However, as will be described later, the screen adjustment functions do not have to be initialized.

[0019] The processing device 120 is a processor that controls the entire information processing device 10, and is configured, for example, by one or more chips. The processing device 120 is configured, for example, by a central processing unit (CPU) that includes an interface with peripheral devices, an arithmetic unit, a register, etc. Note that some or all of the functions of the processing device 120 may be realized by hardware such as a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), and a field programmable gate array (FPGA). The processing device 120 executes various processes in parallel or sequentially.

[0020] The storage device 140 is a recording medium that can be read and written by the processing device 120, and stores a plurality of programs including the control program PR1 executed by the processing device 120. The storage device 140 may be configured by at least one of, for example, a read-only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), and a random access memory (RAM). The storage device 140 may also be called a register, a cache, a main memory, a primary storage device, or the like.

[0021] The display 150 is a device that displays images and text information. The display 150 displays various images under the control of the processing device 120. For example, various display panels such as a liquid crystal display panel and an organic EL (Electro Luminescence) display panel are suitably used as the display 150.

[0022] The input device 160 is a device that accepts operations from a user of the control system 1. For example, the input device 160 includes a keyboard, a touchpad, a touch panel, or a pointing device such as a mouse. Here, if the input device 160 includes a touch panel, it may also serve as the display 150. In the following description, it is assumed that the input device 160 includes a touch panel and also serves as the display 150, but the aspect of this embodiment is not limited to this.

[0023] The communication device 170 is hardware serving as a transmitting / receiving device for communicating with other devices. The communication device 170 is also called, for example, a network device, a network controller, a network card, or a communication module. The communication device 170 includes a wireless communication interface. Examples of the wireless communication interface include those compliant with wireless LAN and Bluetooth (registered trademark). The communication device 170 may also include a connector for wired connection and an interface circuit corresponding to the connector. Examples of the connector and interface circuit for wired connection include those compliant with wired LAN, IEEE1394, and USB.

[0024] The processing device 120 reads and executes the control program PR1 from the storage device 140, thereby functioning as a projection control unit 121, an acquisition unit 122, a display control unit 123, a reception unit 124, a determination unit 125, an output unit 126, a first generation unit 127, a second generation unit 128, and a communication control unit 129. The control program PR1 may be transmitted from another device, such as a server that manages the information processing device 10, via the communication network NET.

[0025] The projection control section 121 causes the first projector 30A to project the first projection image PI1, and also causes the second projector 30B to project the second projection image PI2.

[0026] The acquisition unit 122 causes the imaging device 110 to generate a captured image SI and acquires the captured image SI from the imaging device 110. Fig. 3 is a diagram showing an example of the captured image SI acquired by the acquisition unit 122 from the imaging device 110. As shown in Fig. 3, the captured image SI includes a first projection image PI1 and a second projection image PI2 on the projection target PO.

[0027] For ease of explanation, the following description assumes that the information processing device 10, the first projector 30A, and the second projector 30B face the projection target PO, such as a wall or a screen, and that the first projector 30A and the second projector 30B are installed side by side in the horizontal direction relative to the projection target PO. The following description also assumes that the first projector 30A and the second projector 30B project the first projection image PI1 and the second projection image PI2 onto the projection target PO at the same magnification. Therefore, in FIG. 3, the first projection image PI1 and the second projection image PI2 are rectangular and of the same shape and aligned in the horizontal direction. However, as described below, at least one of the information processing device 10, the first projector 30A, and the second projector 30B does not necessarily face the projection target PO. As a result, the shapes of the first projection image PI1 and the second projection image PI2 may be rectangular or other shapes.

[0028] In FIG. 2, a display control unit 123 causes a display 150 to display various images and text information.

[0029] The reception unit 124 receives an operation from the user of the information processing device 10 to the input device 160. In the present embodiment, as an example, as described above, the input device 160 is configured to include a touch panel and also serves as the display 150. Therefore, the reception unit 124 receives an operation from the user to the display 150 on which the touch panel is stacked.

[0030] 4 to 11 are diagrams showing examples of a display screen DI that the display control unit 123 causes the display 150 to display and examples of operations that the reception unit 124 receives.

[0031] As an example, the display control unit 123 causes the display 150 to display a first display screen DI1 shown in FIG. 4. The first display screen DI1 includes a first message M1, a first button BT1, and a second button BT2. The first message M1 includes a message asking the user whether stacking or tiling is to be performed as multi-projection. The first message M1 is a message output to the first display screen DI1 by the output unit 126, which will be described later. The first button BT1 is a button that allows the user to select stacking as a response to the first message M1. The second button BT2 is a button that allows the user to select tiling as a response to the first message M1.

[0032] The reception unit 124 receives either an operation indicating a selection of stacking or an operation indicating a selection of tiling based on a touch on the first button BT1 or the second button BT2 by an indicator F, such as a user's finger, that indicates the first button BT1 or the second button BT2.

[0033] When the user touches the first button BT1 or the second button BT2, the display control unit 123 causes the display 150 to display the second display screen DI2 shown in FIG. 5, for example.

[0034] The second display screen DI2 includes a captured image SI. As described above, the captured image SI includes a first projected image PI1 and a second projected image PI2. The user swipes the pointer F on the second display screen DI2 to surround an area CR on the projection target PO, onto which a composite image TI obtained by combining the first projected image PI1 and the second projected image PI2 is desired to be projected. The trajectory of the swipe is a frame line RL in FIG. 5. The frame line RL includes at least one of a straight line and a curved line. The accepting unit 124 accepts a swipe operation indicating the frame line RL from the user on the display 150.

[0035] In FIG. 2, a determination unit 125 determines whether or not the operation from the user received by the reception unit 124 is an appropriate input.

[0036] The output unit 126 outputs a message M to the user on the display 150 in accordance with the determination result by the determination unit 125.

[0037] Fig. 6 is a diagram showing a first example of an operation to indicate the frame line RL accepted by the accepting unit 124. Fig. 7 is a diagram showing an example of a third display screen DI3 that the display control unit 123 causes the display 150 to display.

[0038] In Fig. 6, the overlapping area between the area RR surrounded by the frame line RL and the area of ​​the first projection image PI1 is defined as an overlapping area DR1. The determination unit 125 calculates the ratio of the overlapping width DW1 of the overlapping area DR1 to the width PW1 of the area of ​​the first projection image PI1. When the reception unit 124 receives an operation in which the second button BT2 is selected by the user on the first display screen DI1 shown in Fig. 4, and when the ratio of the overlapping width DW1 to the width PW1 on the second display screen DI2 shown in Fig. 6 is equal to or greater than a first value, the determination unit 125 determines that the operation received by the reception unit 124 is abnormal. The first value is, for example, 70%.

[0039] In this case, the display control unit 123 displays the third display screen DI3 illustrated in FIG. 7. The third display screen DI3 includes a second message M2, a first button BT1, and a second button BT2. The second message M2 is a message that the output unit 126 outputs to the third display screen DI3, and includes a message recommending multi-projection using stacking to the user. The first button BT1 is a button that allows the user to select stacking as a response to the first message M1, similar to the first button BT1 on the first display screen DI1. The second button BT2 is a button that allows the user to select tiling as a response to the first message M1, similar to the second button BT2 on the first display screen DI1. When the user touches the first button BT1 or the second button BT2 with an indicator F such as a finger, the display control unit 123 displays the second display screen DI2 on the display 150, similar to the above.

[0040] 6, if the ratio of the overlapping width DW1 to the horizontal width PW1 again becomes equal to or greater than the first value, the display control unit 123 may not cause the display 150 to display the third display screen DI3 again. For example, the display control unit 123 may cause the display 150 to display a part of the first projection image PI1 and a part of the second projection image PI2 side by side within the range of the region RR.

[0041] Fig. 8 is a diagram showing a second example of an operation to indicate the frame line RL accepted by the accepting unit 124. Fig. 9 is a diagram showing an example of a fourth display screen DI4 that the display control unit 123 causes the display 150 to display.

[0042] 8, the determination unit 125 determines the positional relationship between the vertical width PH1 of the first projection image PI1 and the vertical width RH of the region RR surrounded by the frame line RL. As shown in Fig. 8, for example, if the side RS of the region RR protrudes beyond the vertical width PH1 of the first projection image PI1, the determination unit 125 determines that the operation received by the reception unit 124 is abnormal.

[0043] More specifically, if at least one of the end points of the side RS is located outside the region RR and the distance between the end point and the region RR is greater than a predetermined value set in advance, the judgment unit 125 judges that the operation received by the reception unit 124 is abnormal.

[0044] In this case, the display control unit 123 displays a fourth display screen DI4 illustrated in FIG. 9. The fourth display screen DI4 includes a third message M3, a first button BT1, and a second button BT2. The third message M3 is a message output by the output unit 126 to the fourth display screen DI4. The third message M3 includes a message informing the user that a composite image TI exceeding the range of the first projection image PI1 cannot be generated. The first button BT1 is a button used by the user to select "stacking" as a response to the third message M3, similar to the first button BT1 on the first display screen DI1. The second button BT2 is a button used by the user to select "tiling" as a response to the third message M3, similar to the second button BT2 on the first display screen DI1. When the user touches the first button BT1 or the second button BT2 with an indicator F such as a finger, the display control unit 123 displays the second display screen DI2 on the display 150, similar to the above.

[0045] Note that the above description has been given regarding the determination result of the determination unit 125 based on a comparison between the area of ​​the first projection image PI1 and the frame line RL or the area RR surrounded by the frame line RL. The same applies to the determination result of the determination unit 125 based on a comparison between the area of ​​the second projection image PI2 and the frame line RL or the area RR surrounded by the frame line RL. That is, if the side RS of the area RR protrudes beyond at least one of the vertical width PH1 of the first projection image PI1 and the vertical width of the second projection image PI2, the determination unit 125 may determine that the operation accepted by the accepting unit 124 is abnormal.

[0046] 2, when the operation received by the receiving unit 124 is normal, the first generating unit 127 generates first information AI1 based on the operation received by the receiving unit 124. The first information AI1 includes at least one first adjustment amount AA1 related to the first projection image PI1 required to form the composite image TI.

[0047] 10 is an explanatory diagram of a method for calculating the first adjustment amount AA1 by the first generation unit 127. The first generation unit 127 detects the outer frame of the first projection image PI1 on the second display screen DI2 displayed on the display 150. Next, the first generation unit 127 detects the width PW1 of the first projection image PI1 on the second display screen DI2 based on the resolution of the captured image SI included in the second display screen DI2. Next, the first generation unit 127 detects the vertical frame HL1 of the region RR and detects the width FW1 of the region FR1 where the region of the first projection image PI1 does not overlap with the region RR. The first generation unit 127 calculates the ratio of the width FW1 of the region FR1 to the width PW1 of the first projection image PI1 as the first adjustment amount AA1. Specifically, the first generation unit 127 calculates the first adjustment amount AA1 by dividing the value of the width FW1 on the second display screen DI2 by the value of the width PW1 on the second display screen DI2.

[0048] 2, when the operation received by the receiving unit 124 is normal, the second generating unit 128 generates second information AI2 based on the operation received by the receiving unit 124. The second information AI2 includes at least one second adjustment amount AA2 related to the second projection image PI2 required to form the composite image TI.

[0049] 11 is an explanatory diagram of a method for calculating the second adjustment amount AA2 by the second generation unit 128. The second generation unit 128 detects the outer frame of the second projection image PI2 on the second display screen DI2 displayed on the display 150. Next, the second generation unit 128 detects the width PW2 of the second projection image PI2 on the second display screen DI2 based on the resolution of the captured image SI included in the second display screen DI2. Next, the second generation unit 128 detects the vertical frame HL2 of the region RR and detects the width FW2 of the region FR2 where the region of the second projection image PI2 does not overlap with the region RR. The second generation unit 128 calculates the ratio of the width FW2 of the region FR2 to the width PW2 of the region RR as the second adjustment amount AA2. Specifically, the second generation unit 128 calculates the second adjustment amount AA2 by dividing the value of the width FW2 on the second display screen DI2 by the value of the width PW2 on the second display screen DI2.

[0050] The perimeter of the frame line RL and the widths FW1, PW1, FW2, and PW2 are all expressed in units of pixels. In this embodiment, the resolution of the display 150 is the same as the resolution (number of pixels) of the captured image SI, i.e., the pixels of the display 150 correspond one-to-one to the pixels of the captured image SI. Therefore, in this embodiment, the number of pixels of the perimeter of the frame line RL and the widths FW1, PW1, FW2, and PW2 on the display 150 are the same as the number of pixels of the perimeter of the frame line RL and the widths FW1, PW1, FW2, and PW2 on the captured image SI. Note that the resolution of the display 150 does not have to be the same as the resolution (number of pixels) of the captured image SI. In this case, a correspondence between the pixels of the display 150 and the pixels of the captured image SI may be stored in advance in the storage device 140. This correspondence is converted from one of the coordinate system of the display 150 and the coordinate system of the captured image SI to the other. As a result, the number of pixels on the display 150 is converted into the number of pixels in the captured image SI. Based on the correspondence relationship, the perimeter of the frame line RL, the width FW1, the width PW1, the width FW2, and the width PW2 converted into the coordinate system of the captured image SI may be used.

[0051] 2, the communication control unit 129 causes the first projector 30A to transmit first information AI1 including the first adjustment amount AA1 generated by the first generation unit 127 to the communication device 170. In addition, the communication control unit 129 causes the second projector 30B to transmit second information AI2 including the second adjustment amount AA2 generated by the second generation unit 128 to the communication device 170.

[0052] 1-1-3: Projector configuration 12 is a block diagram showing the configuration of the first projector 30A. The first projector 30A includes a projection device 310, a processing device 320, a storage device 330, and a communication device 340. The elements of the first projector 30A are connected to each other by one or more buses for communicating information. Furthermore, the elements of the first projector 30A are configured by one or more devices, and some elements of the first projector 30A may be omitted. It should be noted that the second projector 30B has the same configuration as the first projector 30A, and therefore illustration and description thereof will be omitted.

[0053] The projection device 310 is a device that projects a first projection image PI1 generated by a projection image generation unit 322 (described later) onto a projection target PO such as a screen or a wall. The projection device 310 projects various images under the control of the processing device 320. The projection device 310 includes, for example, a light source, a liquid crystal panel, and a projection lens, and modulates light from the light source using the liquid crystal panel and projects the modulated light onto the projection target PO via the projection lens.

[0054] The processing device 320 is a processor that controls the entire first projector 30A, and is configured, for example, by one or more chips. The processing device 320 is configured, for example, by a central processing unit (CPU) that includes an interface with peripheral devices, an arithmetic unit, a register, etc. Note that some or all of the functions of the processing device 320 may be realized by hardware such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). The processing device 320 executes various processes in parallel or sequentially.

[0055] The storage device 330 is a recording medium readable by the processing device 320, and stores a plurality of programs including the control program PR3 executed by the processing device 320. The storage device 330 may be configured with at least one of, for example, a read-only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), and a random access memory (RAM). The storage device 330 may also be called a register, a cache, a main memory, a primary storage device, or the like. The storage device 330 may store an original image RI1 that is the source of the first projection image PI1 projected by the first projector 30A.

[0056] The communication device 340 is hardware serving as a transmitting / receiving device for communicating with other devices. The communication device 340 is also called, for example, a network device, a network controller, a network card, a communication module, etc. The communication device 340 includes a wireless communication interface. Examples of wireless communication interfaces include those that comply with wireless LAN and Bluetooth (registered trademark). The communication device 340 may also include a connector for wired connection and an interface circuit corresponding to the connector. Examples of connectors and interface circuits for wired connection include those that comply with wired LAN, IEEE1394, and USB.

[0057] The processing device 320 reads and executes the control program PR3 from the storage device 330, thereby functioning as an image acquisition unit 321, a projection image generation unit 322, a projection control unit 323, and an information acquisition unit 324. Note that the control program PR3 may be transmitted from another device, such as a server that manages the first projector 30A, via the communication network NET.

[0058] The image acquisition unit 321 acquires an original image RI1 that is the source of the first projection image PI1 projected by the first projector 30A. The image acquisition unit 321 may acquire the original image RI1 from the storage device 330. Alternatively, the image acquisition unit 321 may acquire the original image RI1 from the information processing device 10 via the communication device 340.

[0059] The projection image generation unit 322 generates a first projection image PI1 using the original image RI1 acquired by the image acquisition unit 321. As an example, the projection image generation unit 322 generates the first projection image PI1 by performing geometric correction on the original image RI1. As described above, the following description is based on the premise that the screen adjustment functions, such as the geometric correction function, performed by the projection image generation unit 322 have been initialized when the imaging device 110 of the information processing device 10 generates the captured image SI.

[0060] The projection control unit 323 causes the projection device 310 to project the first projection image PI1 generated by the projection image generation unit 322 onto the projection target PO. As described above, the following description is based on the premise that the screen adjustment functions such as the zoom function performed by the projection control unit 323 have been initialized when the imaging device 110 of the information processing device 10 generates the captured image SI.

[0061] The information acquisition unit 324 acquires first information AI1 including the first adjustment amount AA1 from the information processing device 10 via the communication device 340. When the information acquisition unit 324 acquires the first information AI1, the projection control unit 323 adjusts the projection position of the first projection image PI1 with respect to the projection target PO, using the first adjustment amount AA1 included in the first information AI1.

[0062] More specifically, the projection control unit 323 calculates the horizontal movement amount of the projection position of the first projection image PI1 relative to the projection target PO using the ratio of the width FW1 of the region FR1 to the width PW1 of the first projection image PI1, which is indicated by the first adjustment amount AA1. Specifically, the projection control unit 323 calculates the number of pixels corresponding to the horizontal movement amount of the projection position of the first projection image PI1 by multiplying the number of pixels corresponding to the horizontal length of the liquid crystal panel provided in the projection device 310 by the ratio. As an example, if the number of pixels corresponding to the horizontal length of the liquid crystal panel is 1920 pixels and the ratio is 0.3, the projection control unit 323 calculates the horizontal movement amount of the projection position of the first projection image PI1 as 1920 × 0.3 = 576 pixels. The projection control unit 323 moves the projection position of the first projection image PI1 relative to the projection target PO in the horizontal direction by the calculated movement amount. In the example shown in Fig. 10, the projection control unit 323 moves the projection position of the first projection image PI1 to the right by the calculated movement amount.

[0063] If the amount of movement calculated by the projection control unit 323 exceeds the movable range of the first projection image PI1, the projection control unit 323 may cause the projection device 310 to project a projection image PI including an error message notifying that adjustment is impossible onto the projection target PO. Furthermore, in this case, the processing device 320 may cause the information processing device 10 to transmit information including the error message to the communication device 340, and the information processing device 10 may display the error message on the display 150.

[0064] In the above description of the configuration of the first embodiment, it is assumed that the first projected image PI1 and the second projected image PI2 are aligned horizontally on the projection target PO. However, the operation of each component is similar to that described above when the positional relationship between the first projected image PI1 and the second projected image PI2 on the projection target PO is different. For example, the first projected image PI1 and the second projected image PI2 may be aligned vertically.

[0065] 1-2: Operation of the first embodiment 13 and 14 are sequence diagrams showing the basic operation of the control system 1 according to the first embodiment. In the following, for the sake of simplicity, description of the operation of the processing device 120 provided in the information processing device 10 as the determination unit 125 and the operation based on the determination result by the operation as the determination unit 125 will be omitted.

[0066] In step S1, the processing device 320 provided in the first projector 30A functions as the projection control unit 323. The processing device 320 causes the projection device 310 to project the first projection image PI1 onto the projection target PO.

[0067] In step S2, the processing device 320 provided in the second projector 30B functions as a projection control unit 323. The processing device 320 causes the projection device 310 to project the second projection image PI2 onto the projection target PO.

[0068] Note that the projection of the first projection image PI1 from the first projector 30A in step S1 may be performed by the information processing device 10 controlling the first projector 30A. Alternatively, the projection may be performed by a control device other than the information processing device 10 controlling the first projector 30A. Similarly, the projection of the second projection image PI2 from the second projector 30B in step S2 may be performed by the information processing device 10 controlling the second projector 30B. Alternatively, the projection may be performed by a control device other than the information processing device 10 controlling the second projector 30B.

[0069] In step S3, the processing device 120 included in the information processing device 10 functions as an acquisition unit 122. The processing device 120 causes the imaging device 110 to generate a captured image SI, and acquires the captured image SI from the imaging device 110. The captured image SI includes a first projection image PI1 and a second projection image PI2 on the projection target PO.

[0070] In step S4, the processing device 120 included in the information processing device 10 functions as the display control unit 123. The processing device 120 displays a first display screen DI1 on the display 150. The first display screen DI1 includes a first message M1, a first button BT1, and a second button BT2.

[0071] In step S5, the processing device 120 included in the information processing device 10 functions as the receiving unit 124. The processing device 120 receives either an operation indicating a selection of stacking or an operation indicating a selection of tiling, based on a touch on the first button BT1 or the second button BT2 by an indicator F, such as a user's finger, that indicates the first button BT1 or the second button BT2.

[0072] In step S6, the processing device 120 included in the information processing device 10 functions as the display control unit 123. The processing device 120 displays the second display screen DI2 on the display 150. The second display screen DI2 includes the captured image SI.

[0073] In step S7, the processing device 120 included in the information processing device 10 functions as the reception unit 124. The user surrounds an area CR on the projection target PO of the composite image TI of the first projection image PI1 and the second projection image PI2 by swiping the pointer F on the second display screen DI2. The processing device 120 receives a swipe from the user indicating a frame line RL on the display 150.

[0074] In step S8, the processing device 120 included in the information processing device 10 functions as the first generation unit 127. If the operations accepted in steps S5 and S6 are normal, the processing device 120 generates first information AI1 based on the operations. The first information AI1 includes at least one first adjustment amount AA1 related to the first projection image PI1 required to form the composite image TI.

[0075] In step S9, the processing device 120 included in the information processing device 10 functions as the second generation unit 128. If the operations accepted in steps S5 and S6 are normal, the processing device 120 includes second information AI2 based on the operations. The second information AI2 includes at least one second adjustment amount AA2 related to the second projection image PI2 required to form the composite image TI.

[0076] In step S10, the processing device 120 included in the information processing device 10 functions as the communication control unit 129. The processing device 120 causes the first projector 30A to transmit the first information AI1 generated in step S8 to the communication device 170. Furthermore, the processing device 320 provided in the first projector 30A functions as an information acquisition unit 324. The processing device 320 causes the communication device 340 to receive the first information AI1 from the information processing device 10, thereby acquiring the first information AI1.

[0077] In step S11, the processing device 120 included in the information processing device 10 functions as the communication control unit 129. The processing device 120 causes the second projector 30B to transmit the second information AI2 generated in step S9 to the communication device 170. Furthermore, the processing device 320 provided in the second projector 30B functions as an information acquisition unit 324. The processing device 320 causes the communication device 340 to receive and acquire the second information AI2 from the information processing device 10.

[0078] In step S12, the processing device 320 included in the first projector 30A functions as the projection control unit 323. The processing device 320 calculates the horizontal movement amount of the projection position of the first projection image PI1 relative to the projection target PO, using the ratio of the width FW1 of the region FR1 to the width PW1 of the region of the first projection image PI1, which is indicated by the first adjustment amount AA1. Furthermore, the processing device 320 moves the projection position of the first projection image PI1 relative to the projection target PO in the horizontal direction by the calculated movement amount.

[0079] In step S13, the processing device 320 included in the second projector 30B functions as the projection control unit 323. The processing device 320 calculates the horizontal movement amount of the projection position of the second projection image PI2 relative to the projection target PO, using the ratio of the width FW2 of the region FR2 to the width PW2 of the region of the second projection image PI2, which is indicated by the second adjustment amount AA2. Furthermore, the processing device 320 moves the projection position of the second projection image PI2 relative to the projection target PO in the horizontal direction by the calculated movement amount.

[0080] 2: Variation The above embodiments can be modified in various ways. Specific modified embodiments are exemplified below. The embodiments exemplified below and the embodiments described above can be combined as appropriate within the scope of not mutually contradicting each other. Note that for elements in the modified embodiments exemplified below that have the same actions and functions as the embodiments, the reference numerals referenced in the above explanation will be used and detailed explanations of each element will be omitted as appropriate.

[0081] 2-1: Variation 1 In the above embodiment, the information processing device 10 calculates the first adjustment amount AA1 and the second adjustment amount AA2. Then, the first projector 30A uses the first adjustment amount AA1 to calculate the movement amount of the first projection image PI1 on the projection target PO, and the second projector 30B uses the second adjustment amount AA2 to calculate the movement amount of the second projection image PI2 on the projection target PO. However, the first projector 30A may independently calculate the movement amount of the first projection image PI1 on the projection target PO without using the first adjustment amount AA1, for example, by performing the following method. Similarly, the second projector 30B may independently calculate the movement amount of the second projection image PI2 on the projection target PO without using the second adjustment amount AA2, for example, by performing the following method.

[0082] In this modification, each of the first projector 30A and the second projector 30B includes a stereo camera.

[0083] A user of the control system 1 first swipes to surround an area CR on the projection target PO in a composite image TI of the first projection image PI1 and the second projection image PI2 with a frame line RL, in the same manner as shown in Fig. 5. The information processing device 10 causes the first projector 30A or the second projector 30B to project a pattern image PT corresponding to the area CR onto the projection target PO. Fig. 15 is a diagram showing an example of the pattern image PT.

[0084] The first projector 30A uses a stereo camera to capture an image of the projection target PO onto which the first projection image PI1, the second projection image PI2, and the pattern image PT are projected. The first projector 30A also detects a side TL1 extending in the vertical direction of the pattern image PT and a side PL1 extending in the vertical direction of the first projection image PI1 from the captured image SI of the projection target PO. The first projector 30A calculates, by triangulation, the difference between the horizontal position of the side TL1 and the horizontal position of the side PL1 as the amount of movement of the first projection image PI1. The same applies to the second projector 30B.

[0085] 2-2: Variation 2 In the above embodiment, it is assumed that the information processing device 10, the first projector 30A, and the second projector 30B all face the projection target PO. However, among these, the information processing device 10 may be positioned obliquely with respect to the projection target PO.

[0086] Fig. 16 is a diagram showing an example of the second display screen DI2 when the information processing device 10 is positioned obliquely with respect to the projection target PO. As shown in Fig. 16, when the information processing device 10 is positioned obliquely with respect to the projection target PO, the shapes of the first projection image PI1 and the second projection image PI2 on the display 150, together with the shape of the projection target PO, become distorted rectangles that are different from a rectangle.

[0087] In this case, the information processing device 10 may perform projective transformation on the shapes of the first projection image PI1 and the second projection image PI2 to correct the shapes of the first projection image PI1 and the second projection image PI2 to shapes that are seen in a planar view, and then display them on the display 150. The same applies to the shape of the projection target PO.

[0088] In this case, the information processing device 10 may include a ToF (Time of Flight) sensor, generate a depth map of the projection surface of the projection target PO based on measurement data from the ToF sensor, and detect, based on the depth map, that the information processing device 10 is tilted with respect to the projection target PO. Alternatively, the information processing device 10 may include an acceleration sensor and a gyro sensor, calculate attitude information indicating the attitude of the information processing device 10 based on measurement data from these sensors, and detect, based on the attitude information, that the information processing device 10 is tilted with respect to the projection target PO.

[0089] 2-3: Variation 3 In the above embodiment, it is assumed that the information processing device 10, the first projector 30A, and the second projector 30B all face the projection target PO. However, at least one of the first projector 30A and the second projector 30B may be positioned obliquely with respect to the projection target PO.

[0090] 17 is a diagram showing an example of the second display screen DI2 when the first projector 30A and the second projector 30B are positioned obliquely with respect to the projection target PO. As shown in Fig. 17, when the first projector 30A and the second projector 30B are positioned obliquely with respect to the projection target PO, the shapes of the first projection image PI1 and the second projection image PI2 on the display 150 are distorted rectangles that are different from rectangles.

[0091] In this case, when the user of the control system 1 swipes to surround an area CR on the projection target PO in a composite image TI of the first projected image PI1 and the second projected image PI2 with a frame line RL, the pixels in the area SA shown by hatching in Fig. 17 will not be used in the composite image TI and will become wasted pixels. In such a case, the output unit 126 may output a message M to the display 150 to notify the user that some of the pixels of the first projected image PI1 or the second projected image PI2 may not be used in the composite image TI.

[0092] For this reason, the first projector 30A includes a ToF sensor, as an example. The first projector 30A generates a depth map of the projection surface of the projection target PO based on measurement data from the ToF sensor, and calculates a first plane equation of the projection surface based on the depth map. Next, the first projector 30A calculates a first normal vector of the projection surface based on the first plane equation. Next, the first projector 30A calculates the angle between the first normal vector and the optical axis vector of the projection lens provided in the projection device 310. Next, if the calculated angle exceeds a predetermined angle set in advance, the first projector 30A determines that the first projector 30A is tilted with respect to the projection target PO. On the other hand, if the calculated angle is equal to or smaller than the predetermined angle set in advance, the first projector 30A determines that the first projector 30A is facing directly toward the projection target PO. The first projector 30A generates a first flag indicating these determination results. The first projector 30A transmits the first flag to the information processing device 10. The ToF sensor is an example of a "distance measuring sensor."

[0093] Similarly, the second projector 30B is equipped with a ToF sensor. The second projector 30B generates a depth map of the projection surface of the projection target PO based on measurement data from the ToF sensor, and calculates a second plane equation for the projection surface based on the depth map. Next, the second projector 30B calculates a second normal vector of the projection surface based on the second plane equation. Next, the second projector 30B calculates the angle between the second normal vector and the optical axis vector of the projection lens provided in the projection device 310. Next, if the calculated angle exceeds a predetermined angle set in advance, the second projector 30B determines that the second projector 30B is tilted with respect to the projection target PO. On the other hand, if the calculated angle is equal to or smaller than the predetermined angle set in advance, the second projector 30B determines that the second projector 30B is facing directly toward the projection target PO. The second projector 30B generates a second flag indicating these determination results. The second projector 30B transmits the second flag to the information processing device 10.

[0094] The information processing device 10 outputs a message M to the display 150 based on at least one of the first flag received from the first projector 30A and the second flag acquired from the second projector 30B. The message M is a message notifying the user that some of the pixels of the first projected image PI1 or the second projected image PI2 may not be used in the composite image TI.

[0095] Alternatively, instead of the first projector 30A and the second projector 30B detecting the inclination of the first projector 30A and the second projector 30B, the information processing device 10 may detect that at least one of the first projector 30A and the second projector 30B is positioned obliquely with respect to the projection target PO. Specifically, as shown in FIG. 17 , when the outline of the projection target PO is rectangular on the second display screen DI2, the information processing device 10 determines that the information processing device 10 is facing the projection target PO directly, based on the outline of the projection target PO. When it is determined that the information processing device 10 is facing the projection target PO directly, but the shape of the first projection image PI1 is a distorted rectangle that is different from a rectangle, the information processing device 10 may determine that the first projector 30A is positioned obliquely with respect to the projection target PO. Similarly, if the information processing device 10 is determined to be facing the projection target PO directly, but the shape of the second projection image PI2 is a distorted rectangle that is different from a rectangle, the information processing device 10 may determine that the second projector 30B is positioned at an angle to the projection target PO.

[0096] Note that only one of the first projector 30A and the second projector 30B may detect the tilt. Therefore, only the first projector 30A may generate the first flag and transmit the first flag to the information processing device 10. In this case, the second projector 30B may not need to calculate the second plane equation and the second normal vector. Also, the information processing device 10 may detect the tilt of only one of the first projector 30A and the second projector 30B. The message M may be a message notifying the user that there is a possibility that only a portion of the pixels of the first projection image PI1 will not be used in the composite image TI.

[0097] 2-4: Variation 4 In the above-mentioned variant example 3, as explained with reference to Figure 17, if the shape of the first projection image PI1 is a distorted rectangle that is different from a rectangle because the first projector 30A is positioned at an angle to the projection target PO, the shape of the first projection image PI1 on the projection target PO may be adjusted to a rectangle by performing geometric correction on the first projection image PI1 on the first projector 30A side. Similarly, if the second projector 30B is positioned at an angle to the projection target PO and the shape of the second projection image PI2 is a distorted rectangle that is different from a rectangle, the shape of the second projection image PI2 on the projection target PO may be adjusted to a rectangle by performing geometric correction on the second projection image PI2 on the second projector 30B side.

[0098] In this case, the display control unit 123 may cause the display 150 to display the second display screen DI2 after the shapes of the first projection image PI1 and the second projection image PI2 on the projection target PO have been adjusted to be rectangular.

[0099] Alternatively, if the shape of the first projection image PI1 on the second display screen DI2 is a distorted rectangle that is different from a rectangle, as an example, based on an operation of the information processing device 10 by a user of the control system 1, a control signal may be output from the information processing device 10 to the first projector 30A to adjust the shape of the first projection image PI1 on the projection target PO to a rectangle. Similarly, if the shape of the second projection image PI2 on the second display screen DI2 is a distorted rectangle that is different from a rectangle, as an example, based on an operation of the information processing device 10 by a user of the control system 1, a control signal may be output from the information processing device 10 to the second projector 30B to adjust the shape of the second projection image PI2 on the projection target PO to a rectangle.

[0100] 2-5: Variation 5 In the above embodiment, as explained with reference to Figures 8 and 9, as an example, if the vertical width RH of region RR is longer than the vertical width PH1 of the first projection image PI1 and therefore exceeds the vertical width PH1 of the first projection image PI1, the judgment unit 125 judges that the operation accepted by the acceptance unit 124 is abnormal.

[0101] However, in this case, the first projector 30A may enlarge the first projected image PI1 so that the length of the vertical width PH1 of the first projected image PI1 matches the length of the vertical width RH of the region RR. When the first projector 30A enlarges the first projected image PI1, on the second display screen DI2 shown in Fig. 8, the length of the vertical width RH of the region RR matches the length of the vertical width PH1 of the first projected image PI1, and the vertical width RH of the region RR does not protrude beyond the vertical width PH1 of the first projected image PI1.

[0102] In this case, the first information AI1 includes the enlargement ratio of the first projection image PI1 in addition to the first adjustment amount AA1.

[0103] The same applies to the relationship between the second projection image PI2 and the region RR. In this case, the second information AI2 includes the enlargement ratio of the second projection image PI2 in addition to the second adjustment amount AA2.

[0104] 2-6: Variation 6 In the above embodiment, as explained with reference to Figures 6 and 7, if the ratio of the overlap width DW1 of the overlap area DR1 between the area RR surrounded by the frame line RL and the first projection image PI1 is equal to or greater than a first value, the receiving unit 124 determines that the operation received is abnormal.

[0105] In the above embodiment, it is assumed that the first projected image PI1 and the second projected image PI2 are spaced apart within the projection target PO. In this case, even if the initial first projected image PI1 and the second projected image PI2 are both simply white light, the information processing device 10 can detect the position of the area of ​​the first projected image PI1 and the position of the overlapping area DR1 in FIG. 6.

[0106] However, if the first projection image PI1 and the second projection image PI2 are partially overlapping from the beginning and both the initial first projection image PI1 and the initial second projection image PI2 are white light, it becomes difficult for the information processing device 10 to detect the position of the area of ​​the first projection image PI1 in Figure 6 and the position of the overlap area DR1.

[0107] Therefore, the information processing device 10 sets the first projection image PI1 projected from the first projector 30A and the second projection image PI2 projected from the second projector 30B as raster images with different colors.

[0108] FIG. 18 is a diagram showing a captured image SI in Modification Example 6. As shown in FIG. 18, the first projected image PI1 and the second projected image PI2 are different raster images. Therefore, in the initial state, the area where only the first projected image PI1 exists, the area where only the second projected image PI2 exists, and the overlapping area DA where the first projected image PI1 and the second projected image PI2 overlap are displayed in different colors. As a result, the information processing device 10 can easily detect the position of the area of ​​the first projected image PI1 and the position of the overlapping area DR1.

[0109] Furthermore, in the above embodiment, if the ratio of the overlap width DW1 to the width PW1 is equal to or greater than a first value, the determination unit 125 determines that the operation received by the reception unit 124 is abnormal. However, in this sixth modification, if the ratio of the value obtained by subtracting the overlap width DD of the initial overlap area DA from the overlap width DW1 to the width PW1 is equal to or greater than a first value, the determination unit 125 preferably determines that the operation received by the reception unit 124 is abnormal.

[0110] 2-7: Variation 7 In the above embodiment, it is assumed that the shape of the region RR surrounded by the frame line RL is rectangular. However, the shape of the region RR may be other shapes. For example, the shape of the region RR may be a triangle or a polygon with pentagons or more sides.

[0111] For example, when simple stacking is performed as multi-projection in the control system 1, the user of the control system 1 may draw a triangular or pentagonal frame line RL by swiping the pointer F on the second display screen DI2.

[0112] In this case, the first projector 30A changes the shape of the first projected image PI1 to match the shape of the frame line RL. Similarly, the second projector 30B changes the shape of the second projected image PI2 to match the shape of the frame line RL. Then, the first projector 30A and the second projector 30B project the first projected image PI1 and the second projected image PI2 at the same position.

[0113] 2-8: Variation 8 In the above embodiment, the communication control unit 129 causes the first projector 30A to transmit the first information AI1 to the communication device 170, and causes the second projector 30B to transmit the second information AI2 to the communication device 170. However, for example, the communication control unit 129 may only cause the first projector 30A to transmit the first information AI1 to the communication device 170. This is because, depending on the position of the frame line RL on the second display screen DI2, it may be possible to construct a composite image by simply adjusting the first projected image PI1 with respect to the second projected image PI2. In this case, the processing device 120 does not need to include the second generation unit 128.

[0114] 3: Summary of this disclosure A summary of this disclosure is provided below.

[0115] (Appendix 1) A control method for an electronic device capable of communicating with a first projector that projects a first projection image onto a projection target, the control method comprising: acquiring a captured image generated by a camera, the captured image including the first projection image and a second projection image projected from a second projector different from the first projector; displaying a screen including the captured image on a display; accepting, via the screen, an operation to specify an area in which a composite image formed by combining the first projection image and the second projection image is to be formed on the projection target; generating, based on the operation, first information including at least one first adjustment amount of the first projection image required to form the composite image; and transmitting the first information to the first projector.

[0116] This allows the user to specify the projection range of the projection image PI on the display 150 provided in an electronic device different from the projector 30, thereby enabling multi-projection.

[0117] More specifically, by using the above control method, if a user of the electronic device performs an operation on the display 150 to specify an area in which the composite image TI is to be formed on the projection target PO, the electronic device can generate information based on the operation. As a result, by using the electronic device, the user of the electronic device can easily generate information for forming the composite image TI to be projected in multi-projection such as stacking or tiling.

[0118] (Supplementary Note 2) The control method of Supplementary Note 1, wherein the operation is an operation of surrounding at least a portion of the captured image on the screen with a frame line including at least one of straight lines and curved lines.

[0119] This allows the user of the electronic device to specify, with a simple operation, the area on the projection target PO where the composite image TI is to be formed.

[0120] (Supplementary Note 3) The control method described in Supplementary Note 1 or Supplementary Note 2, further comprising: accepting a selection as to whether the composite image is to be generated by stacking or by tiling; and, when accepting that the composite image is to be generated by tiling, outputting a message recommending that the composite image be generated by stacking, depending on the overlap width between the area of ​​the first projected image on the screen and the area enclosed by the frame line, or the overlap width between the area of ​​the second projected image and the area enclosed by the frame line.

[0121] This makes it possible to prompt the user of the electronic device to perform multi-projection by stacking when the operation on the display 150 by the user of the electronic device is suitable for multi-projection by stacking.

[0122] (Appendix 4) The control method described in any one of Appendices 1 to 3, characterized in that the captured image displayed on the display is displayed in a planar view by projective transformation.

[0123] This makes it easier for the user of the electronic device to operate the display 150 even if the first projection image PI1 and the second projection image PI2 are captured obliquely by a camera provided on the electronic device.

[0124] (Appendix 5) A control method described in any one of Appendices 2 to 4, characterized in that, when at least a portion of the frame line extends beyond the area of ​​the first projected image included in the captured image displayed on the display, a message is output indicating that a composite image exceeding the area of ​​the first projected image cannot be formed.

[0125] This allows the user of the electronic device to easily understand that it is not possible to construct a connected image with a range larger than that of the white image as the projected image PI.

[0126] (Appendix 6) The control method described in any one of Appendices 1 to 5, further comprising: acquiring a first flag from the first projector when it is determined that the first projector is tilted with respect to the projection target; and outputting, when the first flag is acquired, a message indicating that at least a portion of the multiple pixels constituting the first projection image may not be used in the composite image.

[0127] This allows the user of the electronic device to recognize that at least one of the first projector 30A and the second projector 30B is installed at an angle to the projection target PO.

[0128] (Appendix 7) The control method described in Appendix 6, characterized in that the first flag is generated based on a first normal vector of the projection surface of the projection target calculated by the first projector based on the measurement value of a first ranging sensor and an optical axis vector of a projection lens included in the first projector.

[0129] This allows the first projector 30A itself to generate information indicating that the first projector 30A is installed at an angle to the projection target PO. Similarly, the second projector 30B itself can generate information indicating that the second projector 30B is installed at an angle to the projection target PO.

[0130] (Appendix 8) A control system comprising a first projector that projects a first projection image onto a projection target and an electronic device capable of communicating with the first projector, wherein the electronic device performs the following operations: acquiring a captured image generated by a camera, the captured image including the first projection image and a second projection image projected from a second projector different from the first projector; displaying a screen including the captured image on a display; accepting, via the screen, an operation that specifies an area in which a composite image formed by combining the first projection image and the second projection image is to be formed on the projection target; generating, based on the operation, first information including at least one first adjustment amount of the first projection image required to form the composite image; and transmitting the first information to the first projector; and wherein the first projector adjusts the first projection image based on the first adjustment amount.

[0131] This allows the user to specify the projection range of the projection image PI on a display 150 provided in an electronic device different from the projector 30, thereby enabling stacking projection or tiling projection as multi-projection.

[0132] More specifically, by using the above control method, if a user of the electronic device performs an operation on the display 150 to specify an area in which the composite image TI is to be formed on the projection target PO, the electronic device can generate information based on the operation. As a result, by using the electronic device, the user of the electronic device can easily generate information for forming the composite image TI to be projected in multi-projection such as stacking or tiling.

[0133] (Appendix 9) An information processing program characterized by causing a computer that controls an electronic device capable of communicating with a first projector that projects a first projection image onto a projection target to acquire a captured image generated by a camera, the captured image including the first projection image and a second projection image projected from a second projector different from the first projector; displaying a screen including the captured image on a display; accepting, via the screen, an operation that specifies an area in which a composite image formed by combining the first projection image and the second projection image is to be formed on the projection target; generating, based on the operation, first information including at least one first adjustment amount of the first projection image required to form the composite image; and transmitting the first information to the first projector.

[0134] This allows the user to specify the projection range of the projection image PI on a display 150 provided in an electronic device different from the projector 30, thereby enabling stacking projection or tiling projection as multi-projection.

[0135] More specifically, by using the above control method, if a user of the electronic device performs an operation on the display 150 to specify an area in which the composite image TI is to be formed on the projection target PO, the electronic device can generate information based on the operation. As a result, by using the electronic device, the user of the electronic device can easily generate information for forming the composite image TI to be projected in multi-projection such as stacking or tiling. [Explanation of symbols]

[0136] 1: control system, 10: information processing device, 30: projector, 30A: first projector, 30B: second projector, 110: imaging device, 120: processing device, 121: projection control unit, 122: acquisition unit, 123: display control unit, 124: reception unit, 125: determination unit, 126: output unit, 127: first generation unit, 128: second generation unit, 129: communication control unit, 140: storage device, 15 0: display, 160: input device, 170: communication device, 310: projection device, 320: processing device, 321: image acquisition unit, 322: projection image generation unit, 323: projection control unit, 324: information acquisition unit, 330: storage device, 340: communication device, AA1: first adjustment amount, AA2: second adjustment amount, AI1: first information, AI2: second information, BT1: first button, BT2: second button, CR: area, DA: overlap area, DD: overlap width, DI: display screen, DI1: first display screen, DI2: second display screen, DI3: third display screen, DI4: fourth display screen, DR1: overlap area, DW1: overlap width, F: indicator, FR1: area, FR2: area, FW1: width, FW2: width, HL1: vertical frame, HL2: vertical frame, M: message, M1: first message, M2: second message, M3: third message, NET: communication network, PH1: vertical width, PI: projected image, PI1: first projected image, PI2: second projected image, PL1: edge, PO: projection target, PR1: control program, PR3: control program, PT: pattern image, PW1: width, PW2: width, RH: height, RI1: original image, RL: frame line, RR: area, RS: edge, SA: area, SI: captured image, TI: composite image, TL1: edge

Claims

1. A control method for an electronic device capable of communicating with a first projector that projects a first projection image onto a projection target, comprising: acquiring a captured image generated by a camera, the captured image including the first projected image and a second projected image projected from a second projector different from the first projector; displaying a screen including the captured image on a display; receiving, via the screen, an operation to designate an area in which a composite image obtained by combining the first projection image and the second projection image is to be formed on the projection target; generating first information including at least one first adjustment amount of the first projection image required to form the composite image based on the operation; transmitting the first information to the first projector; A control method comprising:

2. the operation is an operation of surrounding at least a part of the captured image on the screen with a frame line including at least one of a straight line and a curved line. The control method according to claim 1 .

3. accepting a selection as to whether the composite image is generated by stacking or tiling; when it is accepted that the composite image is to be generated by tiling, outputting a message recommending that the composite image be generated by stacking, depending on an overlapping width between an area of ​​the first projection image on the screen and an area enclosed by the frame line, or an overlapping width between an area of ​​the second projection image on the screen and an area enclosed by the frame line; Further comprising: The control method according to claim 2 .

4. The captured image displayed on the display is displayed in a planar view state by projective transformation. The control method according to any one of claims 1 to 3.

5. The method further comprises outputting a message indicating that a composite image exceeding the area of ​​the first projection image cannot be formed when at least a part of the frame line extends beyond the area of ​​the first projection image included in the captured image displayed on the display. The control method according to claim 2 .

6. acquiring a first flag from the first projector when it is determined that the first projector is tilted with respect to the projection target; outputting a message indicating that there is a possibility that at least a part of a plurality of pixels constituting the first projection image will not be used in the composite image when the first flag is acquired; Further comprising: The control method according to claim 1 .

7. The first flag is a first normal vector of a projection surface of the projection target, the first normal vector being calculated by the first projector based on the measurement value of a first distance measurement sensor; an optical axis vector of a projection lens included in the first projector; Generated based on characterized in that The control method according to claim 6.

8. A control system including a first projector that projects a first projection image onto a projection target, and an electronic device that can communicate with the first projector, acquiring a captured image generated by a camera, the captured image including the first projected image and a second projected image projected from a second projector different from the first projector; displaying a screen including the captured image on a display; receiving, via the screen, an operation to designate an area in which a composite image obtained by combining the first projection image and the second projection image is to be formed on the projection target; generating first information including at least one first adjustment amount of the first projection image required to form the composite image based on the operation; transmitting the first information to the first projector; The electronic device executes the adjusting the first projection image based on the first adjustment amount; A control system executed by the first projector.

9. a computer that controls an electronic device that can communicate with a first projector that projects a first projection image onto a projection target; acquiring a captured image generated by a camera, the captured image including the first projected image and a second projected image projected from a second projector different from the first projector; displaying a screen including the captured image on a display; receiving, via the screen, an operation to designate an area in which a composite image obtained by combining the first projection image and the second projection image is to be formed on the projection target; generating first information including at least one first adjustment amount of the first projection image required to form the composite image based on the operation; transmitting the first information to the first projector; An information processing program characterized by causing the program to execute the above.

Images