Setting support method and setting support device

Abstract

To enable a user who is not used to setting up the position and / or the size of a projection image to a projected object, to set up them.SOLUTION: A setup support device 1 comprises: a touch panel 10 including a display device, an imaging apparatus 50, and a processing device 40. The processing device 40 causes the display device to display an image obtained by imaging, by the imaging apparatus 50, a projection image projected on a projected object SC by a projector 2 and causes the display device to display a guide image accelerating execution of an operation to set up the position and / or the size of the projection image, such that the guide image is superimposed on the image obtained by the imaging.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a setting assistance method and a setting assistance device. [Background technology]

[0002] When an image is projected by a projector or the like onto the surface of a projection object having a three-dimensional shape, as in projection mapping, it is necessary to measure the position of the projected image and adjust the shape of the image in advance so that the image appears as if it is pasted onto the surface of the projection object. Patent Document 1 describes a method of photographing the projection object onto which the projection image is to be projected with a camera, measuring the position of the projection image using the photographed image, and performing geometric correction of the projection image according to the measurement result. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2006-33357 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the technology disclosed in Patent Document 1, depending on the shooting position of the camera capturing the projected image, the size of the projected image in the captured image becomes small. If the projected image is not large enough in the captured image, the resolution of measuring the position of the projected image decreases, and geometric correction cannot be performed with high accuracy. In other words, the technology disclosed in Patent Document 1 has a problem that if a user is inexperienced with camera installation, the measurement accuracy of the position of the projected image decreases. Similarly, it may be difficult for an inexperienced user to appropriately set the position of the projected image in the captured image.

[0005] The present disclosure has been made in consideration of the above-mentioned problems, and aims to provide a technique that enables an inexperienced user to easily set at least one of the position and size of a projected image. [Means for solving the problem]

[0006] In order to solve the above problems, the setting assistance method disclosed herein is a setting assistance method that assists in the setting of a projection image, and includes generating a captured image by capturing at least a portion of the projection image projected by a projector onto a projection target, and prompting the user to perform an operation to set at least one of the position and size of the projection image in the captured image.

[0007] In addition, in order to solve the above problems, the setting assistance device disclosed herein is a setting assistance device that assists in the setting of a projection image, and includes a processing device that causes an imaging device to capture at least a portion of the projection image projected by a projector onto a projection target, thereby causing the imaging device to generate an captured image, and prompts the user to perform an operation to set at least one of the position and size of the projection image in the captured image. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram showing an example of the configuration of an image display system 5 including a setting support device 1 and a projector 2 according to an embodiment of the present disclosure. [Diagram 2] FIG. 13 is a diagram showing a display example of an installation support screen. [Diagram 3] FIG. 13 is a diagram showing a display example of a region setting support screen. [Figure 4] FIG. 13 is a diagram showing a display example of a region setting support screen. [Diagram 5] FIG. 13 is a diagram showing a display example of a region setting support screen. [Figure 6] 13 is a diagram showing an example of a measurement pattern generated by a measurement unit 430 of the setting support device 1. FIG. [Figure 7]2 is a diagram showing a flow of processing executed by a processing device 40 of the setting assistance device 1 according to a program PA stored in a storage device 30. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Various technically preferable limitations are attached to the embodiments described below. However, the embodiments of the present disclosure are not limited to the forms described below.

[0010] 1. Embodiment FIG. 1 is a block diagram showing an example of the configuration of an image display system 5 including a setting support device 1 and a projector 2 according to an embodiment of the present disclosure. The projector 2 is connected to a network 3. Specific examples of the network 3 include a wired LAN (Local Area Network), a wireless LAN, and Bluetooth. Bluetooth is a registered trademark. In addition to the projector 2, an image supply device 4 is connected to the network 3. The image supply device 4 supplies image data to the projector 2. In the image display system 5 shown in FIG. 1, the projector 2 projects an image G01 corresponding to image data supplied from the image supply device 4 or the setting support device 1 onto the surface of a projection target SC. Although detailed illustration is omitted in FIG. 1, the projector 2 includes a light source, three display panels as light modulation devices, a projection lens, and a display panel drive unit. Each of the three display panels corresponds to a color of red, green, and blue. In this embodiment, the display panel is a liquid crystal light valve. The display panel drive unit controls the light transmittance of each of the three display panels according to the image data supplied from the image supply device 4 or the setting support device 1. The projector 2 modulates light emitted from a light source by each of three display panels to form image light, and projects this image light from a projection lens to display an image on the surface of a projection object SC. The projection object SC in this embodiment is a rectangular parallelepiped as shown in Fig. 1, but may also have a three-dimensional shape formed by connecting polygons such as a triangular pyramid or a quadrangular pyramid.

[0011] The setting support device 1 is, for example, a smartphone. The setting support device 1 has a communication function for communicating with the projector 2 and an imaging function. The setting support device 1 is not limited to a smartphone, and may be, for example, a tablet terminal having the above communication function and the above imaging function. When the setting support device 1 and the projector 2 are connected to a network, the setting support device 1 communicates with the projector 2 and acquires projector information from the projector 2. In this embodiment, the network connection between the setting support device 1 and the projector 2 is performed during calibration of the projector 2. The projector information includes a model code indicating the model of the projector 2, version information indicating the version of the firmware installed in the projector 2, resolution information, and compression format information indicating a compression format that can be decoded in the projector 2. The resolution information is information indicating the resolution of the display panel in the projector 2. The model code or version information may be used when the setting support device 1 determines whether the projector is compatible with the setting support method of the present disclosure. The resolution information is used by the setting support device 1 when generating a measurement pattern for measuring the position of a projection image projected by the projector 2 onto a projection target SC. Although details will be described later, in this embodiment, a binary code pattern is used as the measurement pattern. The compression format information is used to determine the compression format of the measurement pattern data when the measurement pattern data indicating the measurement pattern is compressed and transmitted from the setting support device 1 to the projector 2. When the measurement pattern data is compressed and transmitted from the setting support device 1 to the projector 2, the compression format is preferably a lossless compression format such as run length, LZH, PNG, or GIF. When it is possible to specify other information from one of the model code, version information, resolution information, and compression format information, such as when the version information, resolution information, and compression format information are specified from the model indicated by the model code, only the one piece of information needs to be included in the projector information. In other words, it is sufficient that the projector information includes at least one of the model code, version information, resolution information, and compression format information.

[0012] The setting support device 1 generates measurement pattern data representing each of the multiple measurement patterns using the resolution information acquired from the projector 2, and provides the generated measurement pattern data to the projector 2. The projector 2 projects an image of the measurement pattern indicated by the measurement pattern data provided from the setting support device 1 onto the projection object SC. This image of the measurement pattern is an example of a pattern image in the present disclosure. Note that the setting support device 1 may transmit a command to the projector 2, and the projector 2 may generate the measurement pattern data based on this command. In this case, it is not necessary to include the compression format information in the projector information. If the measurement pattern data is generated by the projector 2, an effect is achieved in that the communication time for transmitting the measurement pattern from the setting support device 1 to the projector 2 is shortened. The setting support device 1 captures each of the multiple measurement patterns projected by the projector 2 onto the projection object SC using the above-mentioned imaging function, and measures the position of the projection image from the captured captured images. Measuring the position of a projected image means associating the coordinates of the projected image in the captured image captured by the imaging device 50 with the coordinates of the projected image on the display panel of the projector 2, and more specifically, means generating a transformation matrix for performing projective transformation between the former coordinates and the latter coordinates. As described above, when measuring the position of a projected image, it is necessary that acquisition of projector information has been completed. Acquisition of projector information is a process that is executed after the setting assistance device 1 and projector 2 are connected to the network, and therefore measurement of the position of a projected image is also performed. , established This is a process that is executed after the configuration support device 1 and the projector 2 are connected to the network.

[0013] As described above, if the projection object SC is not captured in a captured image at a sufficient size, the measurement accuracy of the position of the projection image based on the captured image decreases. However, the setting support device 1 of this embodiment is configured to prompt the user to perform an operation to adjust the position or zoom of the setting support device 1 so that the projection object SC is captured at a sufficient size in the captured image, so that even a user who is unfamiliar with the setting support device 1 can accurately measure the position of the projection image. The following description focuses on the setting support device 1, which prominently shows the features of this embodiment.

[0014] As shown in FIG. 1, the setting support device 1 includes a touch panel 10, a communication device 20, a storage device 30, a processing device 40, and an imaging device 50. The communication device 20 is a wireless communication module or a wired communication module. When the communication device 20 is a wired communication module, the communication device 20 is connected to the projector 2 via a communication line. The connection between the setting support device 1 and the projector 2 may be a direct connection without a relay device such as a wireless access point device or a router, or may be an indirect connection via a relay device. When the communication device 20 is a wireless communication module, a specific example of the direct connection is an ad-hoc connection, and a specific example of the indirect connection is an access point connection via a wireless access point device. When the communication device 20 is a wired communication module, a specific example of the direct connection is a peer-to-peer connection, and a specific example of the indirect connection is a connection via a wired router or a wired hub. The communication device 20 communicates with the projector 2 under the control of the processing device 40. The imaging device 50 is a camera. The imaging device 50 captures images under the control of the processing device 40, and supplies image data representing the captured images to the processing device 40.

[0015] The touch panel 10 is a device that integrates a display device that displays images and an input device into which information is input by a user. The input device is, for example, a transparent sheet-like contact sensor. The input device is provided so as to cover the display surface of the display device. The input device detects a touch position using electrostatic capacitance determined by an object that touches the input device and the input device, and outputs data indicating the detected touch position to the processing device 40. In this way, the content of a user's operation on the touch panel 10 is transmitted to the processing device 40.

[0016] The storage device 30 is a recording medium readable by the processing device 40. The storage device 30 includes, for example, a nonvolatile memory and a volatile memory. The nonvolatile memory is, for example, a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM), or an Electrically Erasable Programmable Read Only Memory (EEPROM). The volatile memory is, for example, a Random Access Memory (RAM).

[0017] The non-volatile memory of the storage device 30 stores a program PA to be executed by the processing device 40. The volatile memory of the storage device 30 is used by the processing device 40 as a work area when executing the program PA. The program PA may also be called an "application program," "application software," or an "app." The program PA is obtained, for example, from a server (not shown) via the communication device 20, and then stored in the storage device 30. The program PA may be stored in advance in the storage device 30.

[0018] The processing device 40 includes a processor such as a CPU (Central Processing Unit). The processing device 40 may include a single processor or multiple processors. When an operation to instruct the start of execution of the program PA is performed on the touch panel 10, the processing device 40 reads the program PA from the non-volatile memory to the volatile memory and starts execution of the program PA. The processing device 40 operating according to the program PA functions as an acquisition unit 410, a setting support unit 420, and a measurement unit 430 shown in FIG. 1. The acquisition unit 410, the setting support unit 420, and the measurement unit 430 shown in FIG. 1 are software modules realized by operating the processing device 40 according to the program PA.

[0019] When an instruction to acquire projector information is given by operating the touch panel 10 while the setting support device 1 and the projector 2 are connected through a network, the acquisition unit 410 communicates with the projector 2 and acquires the projector information from the projector 2. As described above, the network connection between the setting support device 1 and the projector 2 is established when the projector 2 is calibrated. In other words, the acquisition process of the projector information by the acquisition unit 410 is executed after the calibration. There are various methods for realizing the network connection between the setting support device 1 and the projector 2.

[0020] For example, a printed matter of a QR code including setting information indicating the network setting of the projector 2 may be attached to the housing of the projector 2, and the QR code may be imaged by the imaging device 50 to cause the processing device 40 to acquire the setting information, and the projector 2 may be connected to the network according to the acquired setting information. Note that QR code is a registered trademark. Alternatively, the QR code may be projected by the projector 2 and imaged by the imaging device 50. When the QR code is projected by the projector 2, if the QR code is not captured in the captured image at a size equal to or larger than a predetermined size, or if the QR code is captured too large and a part of the QR code is not included in the captured image, there may be a problem in acquiring the setting information from the captured QR code. Therefore, if the QR code in the captured image is captured at a size smaller than a predetermined size or if a part of the QR code is not captured, the optical zoom of the imaging device 50 may be adjusted by a touch operation on the touch panel 10 to enlarge or reduce the size of the QR code. Also, instead of adjusting the optical zoom, the size of the QR code in the captured image may be enlarged or reduced by moving the user holding the setting support device 1 to capture the QR code, or by changing the distance between the QR code and the setting support device 1, such as by moving the setting support device 1 away from the QR code or by moving the setting support device 1 closer to the QR code. Furthermore, in both the mode of adjusting the optical zoom and the mode of changing the distance between the QR code and the setting support device 1, a guide image of the above-mentioned predetermined size may be displayed superimposed on the captured image as a reference for enlarging or reducing the size of the QR code.

[0021] In the case where the setting support device 1 has a communication function for communicating with an NFC tag, an NFC tag storing the above setting information may be built into the projector 2, and the setting support device 1 may be held up to the location where the NFC tag is built in to allow the setting support device 1 to acquire the above setting information. , multipleA list of setting information for multiple projectors may be stored in advance in storage device 30, and the list may be displayed on the display device of touch panel 10 to allow the user to select a projector to connect to. In this case, an image of an encryption key or the like used for authentication at the time of network connection may be projected onto each of the multiple projectors, and the user may input the encryption key projected by the projector to which the user wishes to connect by operating touch panel 10, and authentication may be performed using the input encryption key.

[0022] The setting support unit 420 generates a captured image by capturing an image of the projection target SC onto which the projection image is projected by the projector 2, and displays a guide image superimposed on the captured image, which prompts the user to perform an operation to set at least one of the position and size of the projection image in the captured image. Displaying this guide image is also a process that is executed after the setting support device 1 and the projector 2 are connected to the network, that is, after calibration.

[0023] More specifically, when the setting support device 1 and the projector 2 are connected to a network, the setting support unit 420 provides the projector 2 with installation pattern image data indicating an installation pattern image upon receiving an instruction to display the installation support screen by operating the touch panel 10, and causes the projector 2 to project the installation pattern image. The installation pattern image is an image for allowing the user to grasp the projection range of the projector 2. The installation pattern image may be, for example, an image of monochromatic light such as white, red, blue, or green, or may be an image of a checkered pattern. Prior to capturing the final image, the user of the setting support device 1 appropriately sets the focus adjustment of the projector 2 and the focus adjustment and exposure adjustment of the imaging device 50. After these settings are completed, the setting support unit 420 causes the imaging device 50 to capture the projection target SC onto which the installation pattern image is projected by the projector 2, and generates the captured image. Then, the setting support unit 420 generates an installation support screen by superimposing a guide image that prompts the user to perform an operation to set the position and size of the projection image onto the image captured by the imaging device 50, and displays the installation support screen on the display device of the touch panel 10.

[0024] 2 to 5 are diagrams showing examples of the display of the installation support screen in the setting support device 1. In FIG. 2, the guide image for prompting the user to execute an operation for setting the position and size of the projection image is a rectangular image G02. The user of the setting support device 1 adjusts the position and attitude of the setting support device 1 and the optical zoom of the imaging device 50 so that the installation pattern image PG projected on the projection target SC appears larger than the image G02, and fixes the setting support device 1 using a tripod or the like. In this embodiment, if the projection target SC appears larger than the image G02 in the captured image, the size of the image G02 is determined in advance so that sufficient measurement accuracy can be obtained in the three-dimensional measurement described later. After the setting support device 1 is fixed using a tripod or the like, the imaging device 50 captures the projection target SC according to the image G02, and in this embodiment, the measurement of the position of the projection image and the setting of the area in which the projection image is arranged are performed based on the captured image of the projection target SC captured according to the image G02.

[0025] In this embodiment, the area in which the projection image is to be placed in the captured image of the projection target SC, i.e., the area occupied by the projection image, is set by a polygonal connecting operation. The setting support unit 420 also displays a guide image that prompts the user to perform the connecting operation, i.e., the operation of setting the shape of the projection image. For example, as shown in Figs. 3 to 5, when the surfaces S01, S02, and S03 of the rectangular parallelepiped of the projection target SC are the projection surfaces, the area in which the projection image is to be placed is set by connecting the rectangles corresponding to each of these three surfaces. In this case, the setting support unit 420 , covered A region setting support screen is generated by superimposing a rectangular object corresponding to each of the surfaces S01, S02, and S03 of the projectile SC on the captured image, and the region setting support screen is displayed on the display device of the touch panel 10. Here, the role of the rectangular object is as follows. In this embodiment, the user of the setting support device 1 creates content of the same size as the display panel of the projector 2 in the setting support device 1, transfers the content to the projector 2, and projects an image corresponding to the content. The user of the setting support device 1 determines an area in the content in which an image is to be displayed and an area in which an image is not to be displayed. The rectangular object corresponds to an area in which an image is to be displayed. An image is assigned to the area in which an image is to be displayed, and an image that has been subjected to geometric correction is pasted so as to overlap the area. An image of a predetermined color such as black is projected on the area in which an image is not to be displayed. The format of the content data is preferably a multimedia compression format for moving images or still images such as JPEG, GIF, PNG, Motion JPEG, H.264, H.265, etc. As another embodiment, area information for displaying an image on the display panel of the projector 2 and image data to be displayed therein may be transmitted to the projector 2, and the projector 2 may be caused to carry out geometric correction of the image data.

[0026] More specifically, the setting support unit 420, upon receiving an area setting instruction by operating the touch panel 10, displays an area setting support screen on the display device of the touch panel 10, which is generated by superimposing an object OB1, which is a rectangular image, on a captured image of the projection target SC, as shown in FIG. 3. Note that image data of the object OB1 may be transmitted from the setting support device 1 to the projector 2, and the projector 2 may project the object OB1 onto the projection target SC according to the image data. Any one of the four vertices of the object OB1 is an example of the first vertex in the present disclosure, and the object OB1 is an example of the first image in the present disclosure. As shown in FIG. 3, the setting support unit 420 displays a circle C01, a circle C02, a circle C03, and a circle C04 with each vertex at the center for the four vertices of the object OB1. The user sets an area corresponding to the surface S01 by moving and deforming the object OB1 by operating the touch panel 10 so that each vertex of the surface S01 and each vertex of the object OB1 overlap with each other. 4 shows that the object OB1 is set in the area corresponding to the surface S01. Each time the object OB1 is moved or transformed, image data of the object OB1 after the movement or transformation may be transmitted from the setting support device 1 to the projector 2, and the object OB1 projected by the projector 2 onto the projection target SC may be updated. The object OB1 and the circles C01, C02, C03, and C04 arranged at the four vertices of the object OB1 are examples of a guide image that prompts the user to perform an operation to set the shape of the projection image.

[0027] When the setting support unit 420 is given a region setting instruction again by an operation on the touch panel 10, as shown in FIG. 4, the setting support unit 420 displays an object OB2 for the second surface by superimposing the region corresponding to the surface S02 on the set captured image. The object OB2 is a rectangular image like the object OB1. For the object OB2, image data of the object OB2 may be transmitted from the setting support device 1 to the projector 2, and the projector 2 may project the image data on the projection target SC. Any one of the four vertices of the object OB2 is an example of the second vertex in the present disclosure, and the object OB2 is an example of the second image in the present disclosure. As shown in FIG. 4, the setting support unit 420 displays a circle C11, a circle C12, a circle C13, and a circle C14 with each vertex as the center for each vertex of the object OB2. The user sets the region corresponding to the surface S02 by moving and deforming the object OB2 by an operation on the touch panel 10 so that each vertex of the surface S02 and each vertex of the object OB2 overlap. FIG. 5 shows that the object OB2 is set in the region corresponding to the surface S02. The object OB2 and the circles C11, C12, C13, and C14 arranged at the four vertices of the object OB2 are also examples of guide images that prompt the user to perform an operation to set the shape of the projection image.

[0028] When moving and transforming the object OB2, it is assumed that an operation of moving the vertex corresponding to the circle C11 so that the distance between the vertex corresponding to the circle C01 and the vertex corresponding to the circle C11 becomes equal to or smaller than a threshold, specifically, an operation of dragging the vertex corresponding to the circle C11 into the circle C01, is performed on the touch panel 10. When the setting support unit 420 receives the operation via the touch panel 10, it matches the position of the vertex corresponding to the circle C11 with the position of the vertex corresponding to the circle C01, and connects both vertices.

[0029] Similarly, when an operation is performed to move the vertex corresponding to the circle C14 so that the distance between the vertex corresponding to the circle C02 and the vertex corresponding to the circle C14 is equal to or less than a threshold value, the setting support unit 420 matches the position of the vertex corresponding to the circle C14 with the position of the vertex corresponding to the circle C02 and connects both vertices. As a result, as shown in FIG. 5, one area corresponding to the faces S01 and S02 is set. In FIG. 5, one area corresponding to the faces S01 and S02 is shown by hatching. The area corresponding to the third face S03 is set in the same manner and connected to the areas corresponding to the faces S01 and S02, respectively, and one area corresponding to the faces S01, S02, and S03 is set. Note that, when it is desired to perform the vertex movement operation in detail, a configuration may be used in which the displayed captured image is enlarged by a touch operation and the vertex movement operation is performed on the enlarged captured image. A specific example of a method for enlarging and displaying a captured image is digital zoom. By performing a vertex movement operation on an enlarged captured image, usability is improved when responding to an arbitrary shape.

[0030] Conventionally, when the area in which the projection image is to be placed in the captured image, i.e., the area occupied by the projection target object, is set by connecting multiple polygons, etc., there is a problem that if the user is not familiar with the connecting operation, the area cannot be easily set. Specifically, it may be difficult for a user who is not familiar with touch operations to adjust the position and shape of one object so that the positions of the vertices of the other object match the positions of the vertices of the other object. In contrast, in this embodiment, the vertices can be easily connected, and even a user who is not familiar with touch operations can easily set the area in which the projection image is to be placed in the captured image by connecting multiple polygons.

[0031] When the setting assistance device 1 and the projector 2 are connected to a network, the measurement unit 430 generates measurement pattern data from the resolution information included in the projector information acquired by the acquisition unit 410, triggered by an instruction to start measurement by an operation on the touch panel 10. The measurement unit 430 controls the projector 2 to sequentially project each of the multiple measurement patterns represented by the measurement pattern data. The measurement unit 430 also controls the imaging device 50 to capture each of the multiple measurement patterns projected sequentially onto the projection target SC, and measures the position of the projection image from the multiple captured images captured by the imaging device 50.

[0032] More specifically, the measurement unit 430 generates a transformation matrix from the multiple captured images for performing projective transformation between the coordinates of the projection image in the captured image captured by the imaging device 50 and the coordinates of the projection image on the display device of the projector 2. This transformation matrix is ​​used when transforming the coordinates of the vertices of each surface of the projection target SC captured in the captured image into coordinates on the display panel. This transformation matrix is ​​also used when transforming to achieve geometric correction of the projection image to be projected onto the projection target SC. In this embodiment, when projecting an image of a content or the like from the projector 2 onto the projection target SC, image data representing the projection image that has been geometrically corrected using the transformation matrix is ​​provided from the setting support device 1 to the projector 2, and the projector 2 projects the image according to the image data.

[0033] As described above, in this embodiment, a binary code pattern is used as the measurement pattern. The binary code pattern refers to an image for expressing the coordinates of a display device using a binary code. The binary code is a technique for expressing the value of each digit when an arbitrary numerical value is expressed in binary by turning a switch on / off. When a binary code pattern is used as the measurement pattern, the image projected by the projector 2 corresponds to the above-mentioned switch, and images for the number of digits of the binary number expressing the coordinate value are required. In addition, separate images are required for the X coordinate and the Y coordinate. For example, when the resolution of the display panel of the projector 2 is 120×90, 120 and 90 are each expressed in a 7-digit binary number, so 7 images are required to express the X coordinate and 7 images are required to express the Y coordinate.

[0034] In addition, it is generally known that when a binary code pattern is used as a measurement pattern, the robustness of the measurement is reduced due to the influence of disturbance light such as lighting. For this reason, when a binary code pattern is used as a measurement pattern, a complementary pattern is generally used in combination to suppress the influence of disturbance light and improve the robustness of the measurement. A complementary pattern is an image in which black and white are inverted. Hereinafter, a binary code pattern in which 1 is represented by white and 0 is represented by black is called a "positive pattern," and a complementary pattern in which it is inverted is called a "negative pattern." In this embodiment, when the resolution information indicates a resolution of 120×90, as shown in FIG. 6, measurement pattern data representing a total of 28 measurement patterns, including 14 positive patterns and 14 negative patterns, is generated by the measurement unit 430.

[0035] In this embodiment, the position of the projected image is measured using a binary code pattern for three-dimensional measurement, so that even if the projection target SC has a complex three-dimensional shape, the three-dimensional shape can be detected with high accuracy, and the projected image can be deformed and attached with high accuracy to a desired range within the shape. In this embodiment, a binary code pattern is used as the measurement pattern, but other structured light such as a dot pattern, a rectangular pattern, a polygonal pattern, a checkered pattern, a gray code pattern, a phase shift pattern, or a random dot pattern may also be used.

[0036] Moreover, the processing device 40 operating according to the program PA executes the setting support method of the present disclosure. The setting support method of the present disclosure is executed after the calibration of the projector 2 is executed, that is, after the setting support device 1 and the projector 2 are connected to the network. Fig. 7 is a diagram showing the flow of the setting support method executed by the processing device 40 according to the program PA. As shown in Fig. 7, the setting support method of the present disclosure includes an acquisition process SA110, a first setting support process SA120, a measurement process SA130, and a second setting support process SA140.

[0037] The acquisition process SA110 is executed when an instruction to acquire projector information is given by the user U through an operation on the touch panel 10. In the acquisition process SA110, the processing device 40 functions as an acquisition unit 410. In the acquisition process SA110, the processing device 40 requests the projector 2 to transmit the projector information, and acquires the projector information returned from the projector 2.

[0038] The first setting support process SA120 is a process that is executed when a display instruction for an installation support screen is given by the user U through an operation on the touch panel 10. In the first setting support process SA120, the processing device 40 functions as a setting support unit 420. In the first setting support process SA120, the processing device 40 causes the projector 2 to project an installation pattern image, and causes the imaging device 50 to capture an image of the projection target SC onto which the installation pattern image is projected by the projector 2 to generate a captured image. Then, the processing device 40 generates an installation support screen by superimposing a guide image that prompts the user to perform an operation for setting the position and size of the projection image in the captured image on the captured image, and causes the display device of the touch panel 10 to display the installation support screen.

[0039] The measurement process SA130 is a process that is executed when a user U instructs the start of measurement by operating the touch panel 10. In the measurement process SA130, the processing device 40 functions as a measurement unit 430. To explain in more detail, in the measurement process SA130, the processing device 40 first generates measurement pattern data from the resolution information included in the projector information acquired in the acquisition process SA110. Next, the processing device 40 controls the projector 2 to sequentially project each of the multiple measurement patterns represented by the measurement pattern data. In addition, the processing device 40 controls the imaging device 50 to capture each of the multiple measurement patterns projected sequentially onto the projection target SC, and measures the position of the projection image from the multiple captured images captured by the imaging device 50.

[0040] The second setting support process SA140 is a process that is executed when a display instruction for a region setting support screen is given by the user U through an operation on the touch panel 10. In the second setting support process SA140, the processing device 40 functions as a setting support section 420. In the second setting support process SA140, every time a region setting instruction is given by the user U through an operation on the touch panel 10, the processing device 40 displays a rectangular object superimposed on the captured image of the projection object SC, and moves the position of the object and deforms the object in response to the operation on the touch panel 10.

[0041] According to the setting support device 1 of this embodiment, prior to measuring the position of the projection image based on the captured image of the projection target SC, the installation position of the setting support device 1 is adjusted so that the projection image projected on the projection target SC appears larger than the image G02 in the captured image by the imaging device 50. Therefore, according to this embodiment, even if the user of the projector 2 is unfamiliar with the operation of the setting support device 1, it is possible to install the setting support device 1 at an appropriate imaging position and measure the position of the projection image with high accuracy. Furthermore, according to this embodiment, the operation after associating the coordinates of the captured image and the displayed image can be performed by looking only at the displayed image of the setting support device 1 without checking the actual projected image, so that the viewpoint movement is reduced and usability is improved.

[0042] In this embodiment, it is necessary to set the area in which the projection image is arranged in the captured image of the projection object SC, that is, the area occupied by the projection object SC, by a quadrangular connection operation. However, when the second vertex is moved so that the distance between the first vertex of the first image and the second vertex of the second image is equal to or less than a threshold, the setting support device 1 automatically matches the position of the second vertex with the position of the first vertex. Therefore, according to the setting support device 1 of this embodiment, even a user who is not familiar with touch operations can easily connect the vertices and set the shape and size of the projection image simply and accurately. In addition, according to the setting support device 1 of this embodiment, once the captured image of the projection object SC is generated, the area in which the projection image is arranged can be easily set by only operating the captured image. Finally, by determining the shape of the area in which the projected image will be placed and assigning the image to that area, the created content can be transferred to a memory area built into projector 2. The created content can then be written via a network to a USB memory or SD card built into or external to projector 2, and thereafter the content mapped onto the projected object SC can be played back without using the setting support device 1.

[0043] 2. Variations The above-described embodiments may be appropriately combined with the following modifications. (1) In the above embodiment, the setting support device 1 provides the projector 2 with transformed image data based on a transformation matrix for performing projective transformation between the coordinates of the projection image in the captured image captured by the imaging device 50 and the coordinates of the projection image on the display device of the projector 2. However, the setting support device 1 may transmit to the projector 2 pairs of coordinates of the vertices of each surface of the projection object SC captured in the captured image and the coordinates on the display panel, or the transformation matrix, and cause the projector 2 to perform the transformation.

[0044] (2) In the above embodiment, the first image and the second image are both rectangular images, but they may be images of other polygons other than a rectangle, such as a triangle or a pentagon. Also, image analysis such as edge detection may be performed on the captured image of the projection object SC to detect the shape of the projection object SC, and polygonal images according to the detection results may be used as the first image and the second image.

[0045] (3) CAD (Computer-Aided Design) data indicating the shape of the projected object SC may be stored in the setting support device 1, and the setting support device 1 may execute a process of deforming the projection image to match the shape of the projection surface indicated by the CAD data. Also, the setting support device 1 may execute a process of matching the shape of the projection surface measured by three-dimensional measurement with the shape of the projection surface indicated by the CAD data, and further adjust the shape of the deformed projection image according to the matching result.

[0046] (4) In the above embodiment, the program PA is already stored in the storage device 30. However, the program PA may be manufactured or distributed as a standalone device. Specific methods for distributing the program PA include writing the program PA to a computer-readable recording medium such as a flash ROM (Read Only Memory) and distributing it, or distributing it by downloading it via a telecommunications line such as the Internet. By installing the program PA in a general information processing device and operating a computer such as a CPU of the information processing device in accordance with the program, it becomes possible to cause the information processing device to function as the setting support device of the present disclosure.

[0047] (5) In the above embodiment, the acquisition unit 410, the setting support unit 420, and the measurement unit 430 are software modules, but some or all of the acquisition unit 410, the setting support unit 420, and the measurement unit 430 may be hardware. Examples of this hardware include a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). Even if some or all of the acquisition unit 410, the setting support unit 420, and the measurement unit 430 are hardware, the same effects as those of the above embodiment can be achieved.

[0048] (6) The setting support device 1 in the above embodiment is a mobile terminal such as a smartphone or a tablet terminal. However, the acquisition unit 410, the setting support unit 420, and the measurement unit 430 may be provided in a server device that communicates with the mobile terminal via a LAN or the Internet, and the server device may function as the setting support device 1. In this case, the above smartphone that communicates with the server device functions as an imaging device that captures an image of a projection target onto which a projected image is projected by a projector, and a display device that displays a guide image superimposed on the captured image, under the control of the setting support device 1. In other words, the imaging device and the display device are not essential components of the setting support device of the present disclosure, and may be external elements accessible from the setting support device.

[0049] Furthermore, when the present disclosure is applied only to the installation support of the imaging device, the above-mentioned measurement process SA130 and second setting support process SA140 may be omitted. Similarly, when the present disclosure is applied only to the setting support of the area to which the image is projected, the above-mentioned first setting support process SA120 and measurement process SA130 may be omitted. When the measurement process SA130 is omitted, the measurement unit 430 can be omitted. Furthermore, the setting support device of the present disclosure may be incorporated in a projector, in which case the acquisition unit 410 can be omitted. If the setting support device 1 is incorporated in the projector, the projector can be installed and adjusted without using a mobile terminal, improving usability.

[0050] (7) In the image display system 5 of the above embodiment, the image supply device 4 supplies image data to the projector 2. However, the image supply device 4 may supply control data to the projector 2 in addition to image data, or a device for supplying image data and a device for supplying control data may be provided separately. The control data may include image quality adjustments, image geometric adjustments, power adjustments, etc., in the projector. etc.The image data includes data for controlling the projector. The image data includes the content to be displayed by the projector. The image supply device 4, or the device supplying the image data and the device supplying the control data, can also process the image data and the control data in cooperation with the projector. The projector or the image display system 5 can also be configured to include a storage device in which the above-mentioned content is stored, and once the content is stored in the storage device, the image supply device 4 and the like can be removed from the image display system 5.

[0051] 3. Aspects grasped from at least one of the embodiments and each modified example The present disclosure is not limited to the above-mentioned embodiments and modifications, and can be realized in various aspects without departing from the spirit of the present disclosure. For example, the present disclosure can also be realized in the following aspects. The technical features in the above-mentioned embodiments corresponding to the technical features in each aspect described below can be appropriately replaced or combined in order to solve some or all of the problems of the present disclosure or to achieve some or all of the effects of the present disclosure. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted.

[0052] The setting support method of the present disclosure is a setting support method that supports the setting of a projection image, and generates a captured image by capturing at least a part of the projection image projected by a projector onto a projection target, and prompts the user to execute an operation to set at least one of the position and size of the projection image in the captured image. According to this aspect, the user can set at least one of the position and size of the projection image by executing an operation prompted by the setting support method of the present disclosure. Therefore, the setting support method of the present disclosure allows an inexperienced user to easily set at least one of the position and size of the projection image.

[0053] In the setting support method disclosed herein, a pattern image may be projected onto the projection object by the projector, and in this case, the captured image is generated by capturing an image of the projection object onto which the pattern image is projected.

[0054] In the setting support method of the present disclosure, a guide image for encouraging the user to perform the operation may be displayed on the display device superimposed on the captured image. According to this aspect, the user can perform the operation for setting at least one of the position and size of the projection image by following the guide screen. Therefore, even a user who is unfamiliar with the operation can easily set at least one of the position and size of the projection image.

[0055] In the setting support method according to the present disclosure, the guide image may be an image that prompts a user to perform an operation of setting at least one of a position, a shape, and a size of the projection image in the captured image. According to this aspect, it becomes possible for an inexperienced user to easily set at least one of a position, a shape, and a size of the projection image in the captured image.

[0056] In the setting support method disclosed herein, the projection object may be imaged according to the guide image superimposed on the captured image, and the position of the projection image may be measured. According to this aspect, it is possible to associate the coordinates on the display panel of the projector and the coordinates in the captured image from the captured image of the projection object onto which the pattern image is projected, and use the association to set the position of the projection image.

[0057] In the setting support method of the present disclosure, the guide image may include a first image having a first vertex and a second image having a second vertex, and when an operation is received to move the second vertex so that the distance between the first vertex and the second vertex is equal to or less than a threshold, the position of the second vertex may be moved to coincide with the position of the first vertex. According to this aspect, when the distance between the first vertex and the second vertex is equal to or less than the threshold, the position of the second vertex is automatically corrected to the position of the first vertex, so that even a user who is unfamiliar with the operation of moving the second vertex can easily set the area of ​​the projected image that occupies the captured image by connecting the first image and the second image.

[0058] In the setting support method of the present disclosure, a mobile terminal may capture an image of the projection object and display the captured image and the guide image. According to this aspect, it is possible to set at least one of the position, shape, and size of the projection image in the captured image using the mobile terminal.

[0059] The setting assistance device according to the present disclosure is a setting assistance device that assists in setting a projection image, and includes the following processing device: The processing device causes an imaging device to capture at least a part of the projection image projected by a projector onto a projection target, thereby causing the imaging device to generate a captured image, and prompts the user to execute an operation to set at least one of a position and a size of the projection image in the captured image. vinegar. According to this aspect, the user can set at least one of the position and size of the projection image by performing an operation prompted by the setting support device of the present disclosure. Therefore, the setting support device of the present disclosure allows an inexperienced user to easily set at least one of the position and size of the projection image. [Explanation of symbols]

[0060] 1...setting support device, 2...projector, 10...touch panel, 20...communication device, 30...storage device, 40...processing device, 410...acquisition unit, 420...setting support unit, 430...measurement unit, 50...imaging device, PA...program, SC...projected object, 3...network, 4...image supply device, 5...image display system.

Claims

1. A setting support method for supporting settings of a projection image, comprising the steps of: generating a captured image by capturing at least a portion of the projection image projected by a projector onto a projection target; displaying, on a display device, a guide image that prompts the user to perform an operation to set at least one of the position, the shape, and the size of the projection image on the captured image, superimposed on the captured image; Including, the guide image is a plurality of polygonal images that correspond one-to-one to a plurality of surfaces of the projection target, the plurality of polygonal images include a first polygonal image for setting an area corresponding to a first surface among the plurality of surfaces, and a second polygonal image for setting an area corresponding to a second surface among the plurality of surfaces; the first polygonal image has a first vertex; the second polygonal image has a second vertex; When an operation of moving the second vertex is received such that the distance between the first vertex and the second vertex is equal to or smaller than a threshold, the position of the second vertex is made to coincide with the position of the first vertex. Setting support method.

2. projecting a pattern image onto the projection target using the projector; The captured image is generated by capturing an image of the projection object onto which the pattern image is projected. The setting support method according to claim 1 .

3. projecting a pattern image onto the projection target using the projector; measuring a position of the projected image based on a result of imaging the projection object onto which the pattern image is projected in accordance with the guide image superimposed on the captured image; The setting support method according to claim 1 .

4. The setting support method according to claim 1 , further comprising: a mobile terminal taking an image of the projection object and displaying the taken image and the guide image.

5. A setting support device that supports settings of a projection image, comprising: causing an imaging device to capture at least a part of the projection image projected by a projector onto a projection target, thereby causing the imaging device to generate a captured image; a processing device that displays, on a display device, a guide image that prompts the user to perform an operation to set at least one of a position, a shape, and a size of the projection image in the captured image, superimposed on the captured image; the guide image is a plurality of polygonal images that correspond one-to-one to a plurality of surfaces of the projection target, the plurality of polygonal images include a first polygonal image for setting an area corresponding to a first surface among the plurality of surfaces, and a second polygonal image for setting an area corresponding to a second surface among the plurality of surfaces; the first polygonal image has a first vertex; the second polygonal image has a second vertex; When an operation of moving the second vertex is received such that the distance between the first vertex and the second vertex is equal to or smaller than a threshold, the position of the second vertex is made to coincide with the position of the first vertex. Setting support device.

Images