ADJUSTMENT METHOD, MEASURING METHOD, PROJECTION SYSTEM, INFORMATION PROCESSING APPARATUS, AND PROGRAM

Abstract

To avoid the occurrence of white or black crushing in a picked-up image of a projection target object on which a projection image is projected.SOLUTION: There is provided an information processing apparatus 30 having an identification unit 321, a detection unit 322, an estimation unit 323, and an adjustment unit 324. The identification unit 321 identifies the correspondence relation between a projector coordinate system and a camera coordinate system based on a pattern image and a picked-up image of a projection target object on which the pattern image is projected. The detection unit 322 detects the luminance of a first area and a second area in the picked-up image of the projection target object on which a monochromatic image with a uniform luminance is projected. The estimation unit 323 estimates the range of luminance not saturating the luminance in the picked-up image for a third area associated with the first area and a fourth area associated with the second area in the monochromatic image. The adjustment unit 324 adjusts the luminance of a fifth area corresponding to the third area and a sixth area corresponding to the fourth area in the projection image based on a result of estimation performed by the estimation unit 323.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an adjustment method, a measurement method, a projection system, an information processing device, and a program. [Background technology]

[0002] Patent Document 1 discloses a technique for measuring the three-dimensional shape of a measurement object by a phase shift method using an image of a stripe pattern in which the brightness changes sinusoidally. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2009-115612 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the phase shift method, an image of a stripe pattern is projected onto a measurement object. Then, the three-dimensional shape of the measurement object is measured based on a captured image obtained by capturing an image of the measurement object with the stripe pattern image projected on it. In the captured image of the measurement object with the stripe pattern image projected on it, the brightness of the stripe pattern may differ depending on the region even within the same captured image due to the reflection characteristics of the measurement object or the influence of the surrounding external light, etc. In a region where the stripe pattern is not captured within the appropriate brightness range, for example, where the brightness of the stripe pattern is too high or too low, there is a risk of a decrease in the measurement accuracy of the three-dimensional shape. [Means for solving the problem]

[0005] In order to solve the above-described problems, an adjustment method of the present disclosure is a method for adjusting a projection image projected from a projector onto a projection object, the method including: determining a correspondence relationship between a projector coordinate system indicating a position within a first image projected from the projector onto the projection object and a camera coordinate system indicating a position within the first captured image, based on a first image projected from the projector onto the projection object and a first captured image obtained by capturing an image of the projection object with a first camera while the first image is being projected; detecting a luminance of a first region in a second captured image obtained by capturing an image of the projection object with the first camera while a second image is being projected from the projector and a luminance of a second region in the second captured image; For a third region that corresponds to the first region through the correspondence relationship, a first luminance range that does not saturate the luminance of the first region above or below is estimated based on the luminance detected for the first region and the luminance of the third region, and for a fourth region of the second image that corresponds to the second region through the correspondence relationship, a second luminance range that does not saturate the luminance of the second region above or below is estimated based on the luminance detected for the second region and the luminance of the fourth region, the luminance of a fifth region of the projected image that corresponds to the third region is adjusted to fall within the first luminance range, and the luminance of a sixth region of the projected image that corresponds to the fourth region is adjusted to fall within the second luminance range.

[0006] In order to solve the above problem, a measurement method of the present disclosure projects a projection image, in which luminance changes periodically within the image, from a projector onto a projection object, and measures a three-dimensional shape of the projection object based on a captured image obtained by capturing an image of the projection object in a state in which the projection image is projected by a first camera and a captured image obtained by capturing an image of the projection object in a state in which the projection image is projected by a second camera different from the first camera, the measurement method comprising the steps of: determining a correspondence relationship between a projector coordinate system indicating a position within the first image and a camera coordinate system indicating a position within the first captured image based on a first image projected from the projector onto the projection object and a first captured image obtained by capturing an image of the projection object in a state in which the first image is projected by the first camera; the luminance of a first region and the luminance of a second region in a second captured image obtained by capturing an image of the projection object with the first camera is detected; for a third region in the second image that corresponds to the first region through the correspondence relationship, a first luminance range in which the luminance of the first region is not saturated above or below is estimated based on the luminance detected for the first region and the luminance of the third region; for a fourth region in the second image that corresponds to the second region through the correspondence relationship, a second luminance range in which the luminance of the second region is not saturated above or below is estimated based on the luminance detected for the second region and the luminance of the fourth region; the luminance of a fifth region in the projected image that corresponds to the third region is adjusted to fall within the first luminance range, and the luminance of a sixth region in the projected image that corresponds to the fourth region is adjusted to fall within the second luminance range.

[0007] Further, in order to solve the above-mentioned problems, a measurement method of the present disclosure projects a projection image, in which luminance changes periodically within the image, from a projector onto a projection object, and measures a three-dimensional shape of the projection object based on a captured image obtained by capturing an image of the projection object in a state in which the projection image is projected by a first camera and the projection image, the measurement method comprising: determining a correspondence relationship between a projector coordinate system indicating a position within the first image and a camera coordinate system indicating a position within the first captured image based on a first captured image obtained by capturing an image of the projection object in a state in which the first image is projected by the projector and the first image; and determining a correspondence relationship between a projector coordinate system indicating a position within the first captured image and a camera coordinate system indicating a position within the first captured image based on a second captured image obtained by capturing an image of the projection object in a state in which the second image is projected by the projector. the luminance of a first region and the luminance of a second region in the projected image are detected; for a third region of the second image that corresponds to the first region through the correspondence relationship, a first luminance range in which the luminance of the first region is not saturated above or below is estimated based on the luminance detected for the first region and the luminance of the third region; for a fourth region of the second image that corresponds to the second region through the correspondence relationship, a second luminance range in which the luminance of the second region is not saturated above or below is estimated based on the luminance detected for the second region and the luminance of the fourth region; the luminance of a fifth region of the projected image that corresponds to the third region is adjusted to fall within the first luminance range, and the luminance of a sixth region of the projected image that corresponds to the fourth region is adjusted to fall within the second luminance range.

[0008] In order to solve the above problem, the projection system of the present disclosure includes a projector that projects a projection image onto a projection object, a first camera that captures an image of the projection object, and an information processing device. The information processing device determines a correspondence relationship between a projector coordinate system that indicates a position in the first image and a camera coordinate system that indicates a position in the first captured image based on a first captured image obtained by capturing an image of the projection object with the first camera while a first image is being projected from the projector and the first image, detects the luminance of a first region and the luminance of a second region in a second captured image obtained by capturing an image of the projection object with the first camera while a second image is being projected from the projector, and calculates the luminance of the first region for a third region of the second image that corresponds to the first region based on the correspondence relationship. The method includes estimating a first luminance range that does not cause saturation vertically based on the luminance detected for the first region and the luminance of the third region; estimating a second luminance range that does not cause saturation vertically for a fourth region of the second image that corresponds to the second region by the correspondence relationship based on the luminance detected for the second region and the luminance of the fourth region; and adjusting the luminance of a fifth region of the projected image that corresponds to the third region so that it falls within the first luminance range, and adjusting the luminance of a sixth region of the projected image that corresponds to the fourth region so that it falls within the second luminance range.

[0009] In order to solve the above problem, an information processing device according to the present disclosure specifies a correspondence relationship between a projector coordinate system indicating a position in a first image and a camera coordinate system indicating a position in a first captured image, based on a first captured image obtained by capturing an image of a projection object with a first camera while a first image is being projected from a projector that projects a projection image onto the projection object, and the first image; detects luminance of a first region and luminance of a second region in a second captured image obtained by capturing an image of the projection object with the first camera while a second image is being projected from the projector; and specifies a correspondence relationship between a projector coordinate system indicating a position in the first captured image and a camera coordinate system indicating a position in the first captured image, based on the first image and the first captured image; The present invention executes the following steps for estimating a first luminance range that does not saturate the luminance of the first region above or below, based on the luminance detected for the first region and the luminance of the third region; for a fourth region of the second image that corresponds to the second region by the correspondence relationship, estimating a second luminance range that does not saturate the luminance of the second region above or below, based on the luminance detected for the second region and the luminance of the fourth region; and adjusting the luminance of a fifth region of the projected image that corresponds to the third region so that it falls within the first luminance range, and adjusting the luminance of a sixth region of the projected image that corresponds to the fourth region so that it falls within the second luminance range.

[0010] In order to solve the above-mentioned problem, a program disclosed herein causes a computer to: determine a correspondence relationship between a projector coordinate system indicating a position in a first image and a camera coordinate system indicating a position in the first captured image, based on a first captured image obtained by capturing an image of a projection target with a first camera while a first image is being projected from a projector that projects a projection image onto the projection target, and the first image; detect a luminance of a first region and a luminance of a second region in a second captured image obtained by capturing an image of the projection target with the first camera while a second image is being projected from the projector; and detect a luminance of a third region in the second image that is associated with the first region based on the correspondence relationship. The projection apparatus executes the following steps: for a region, estimating a first luminance range that does not saturate the luminance of the first region above or below based on the luminance detected for the first region and the luminance of the third region; for a fourth region of the second image that corresponds to the second region by the correspondence relationship, estimating a second luminance range that does not saturate the luminance of the second region above or below based on the luminance detected for the second region and the luminance of the fourth region; and adjusting the luminance of a fifth region of the projected image that corresponds to the third region so that it falls within the first luminance range, and adjusting the luminance of a sixth region of the projected image that corresponds to the fourth region so that it falls within the second luminance range. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram showing an example configuration of a measurement system 1 including an information processing device 30 according to an embodiment of the present disclosure. [Diagram 2] 1 is a diagram showing an example of an image of a dot pattern projected from a projector 10 onto a measurement target SC. [Diagram 3] 11 is a diagram showing an example of a captured image obtained by imaging the measurement target SC with the image of a dot pattern projected thereon by the imaging device 20. FIG. [Figure 4] FIG. 13 is a diagram showing an example of a flowchart illustrating the flow of a measurement method executed by a processing device 320 of an information processing device 30 in accordance with a program 311. [Diagram 5]FIG. 13 is a diagram for explaining the effect of the present disclosure. [Figure 6] FIG. 13 is a diagram for explaining the effect of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Various technically preferable limitations are attached to the embodiments described below. However, the embodiments of the present disclosure are not limited to the forms described below.

[0013] 1. Embodiment FIG. 1 is a diagram showing an example of the configuration of a measurement system 1 according to an embodiment of the present disclosure. As shown in FIG. 1, the measurement system 1 includes a projector 10, an imaging device 20, and an information processing device 30. The measurement system 1 shown in FIG. 1 is a system for measuring the three-dimensional shape of an object. In the measurement system 1, under the control of the information processing device 30, a pattern image for three-dimensional measurement is projected from the projector 10 onto a measurement target SC, the three-dimensional shape of which is to be measured. In the measurement system 1, an image of the measurement target SC in a state in which the pattern image is projected is captured by the imaging device 20. In the measurement system 1, a calculation is performed by the information processing device 30 to measure the three-dimensional shape of the measurement target SC based on the pattern image and the captured image by the imaging device 20. The measurement system 1 is an example of a projection system in the present disclosure. The measurement target SC is an example of a projection target in the present disclosure.

[0014] The projector 10 includes, for example, a liquid crystal light valve for projecting an image, a projection lens, a liquid crystal driving unit, and an extra-high pressure mercury lamp as a light source. The light source in the projector 10 may be a metal halide lamp. The projector 10 is connected to the information processing device 30 by, for example, a cable so as to be able to communicate with the information processing device 30. The projector 10 acquires image data representing a pattern image from the information processing device 30 by communication via the cable. In this embodiment, the communication between the projector 10 and the information processing device 30 is, for example, wired communication conforming to a standard such as Ethernet or USB (Universal Serial Bus). Note that Ethernet is a registered trademark. The communication between the projector 10 and the information processing device 30 may be wireless communication conforming to a standard such as Wi-Fi. The projector 10 projects a pattern image indicated by the acquired image data onto the measurement target SC.

[0015] The imaging device 20 is, for example, a camera equipped with a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) that is an imaging element that converts collected light into an electric signal. In the following, for the sake of simplicity of explanation, it is assumed that the imaging device 20 captures a still image. Note that the imaging device 20 may capture a moving image instead of capturing a still image. The imaging device 20 captures an image of a measurement target SC. The imaging device 20 is connected to the information processing device 30 by, for example, a cable in a manner similar to the projector 10 so as to be able to communicate with the information processing device 30. The imaging device 20 transmits image data representing the captured image to the information processing device 30. In this embodiment, the communication between the imaging device 20 and the information processing device 30 is, for example, wired communication conforming to a standard such as Ethernet or USB, but may be wireless communication conforming to a standard such as Wi-Fi. The imaging device 20 is an example of a first camera in the present disclosure.

[0016] In this embodiment, the projector 10 and the imaging device 20 are disposed at different fixed positions. The respective positions of the projector 10 and the imaging device 20 have been adjusted in advance to be suitable for three-dimensional measurement. In addition, the lens distortion of the projection lens of the projector 10 has been corrected. Similarly, the lens distortion of the imaging device 20 has been corrected. In addition, the exposure amount and gain of the imaging device 20 have been adjusted to be suitable for three-dimensional measurement. In the measurement system 1 of this embodiment, the three-dimensional shape of the measurement object SC is measured by the phase shift method. To explain in more detail, in the measurement system 1, first, under the control of the information processing device 30, an image of a stripe pattern whose luminance changes sinusoidally is projected from the projector 10 onto the measurement object SC. The information processing device 30 shifts the phase of the stripe pattern projected onto the measurement object SC by a predetermined phase shift amount. The phase shift is repeated multiple times until the phase of the stripe pattern moves by one period. The number of times of phase shift of the stripe pattern is generally three or four, and is four in this embodiment. That is, the phase shift amount in this embodiment is π / 2 [rad]. Note that π is the circular constant. Every time the information processing device 30 shifts the phase of the stripe pattern, it causes the imaging device 20 to capture the measurement target SC with the image of the stripe pattern projected on it, and obtains image data representing the captured image from the imaging device 20. Therefore, in this embodiment, the information processing device 30 obtains four pieces of image data that correspond one-to-one to the four stripe patterns having phase shift amounts of 0, π / 2, π, and 3π / 2.

[0017] The luminance values ​​I0(x,y), I1(x,y), I2(x,y), and I3(x,y) of the pixel located at the coordinates (x,y) in each of the four images obtained by imaging the measurement target SC with stripe patterns projected with phase shift amounts of 0, π / 2, π, and 3π / 2 are expressed by the following formulas (1), (2), (3), and (4), respectively. Note that A is the amplitude of the luminance of the stripe pattern. G is the luminance of the ambient light such as external light. R is the object reflectance of the measurement target SC. φ is the phase of the luminance at the coordinates (x,y) that changes sinusoidally.

number

[0018] Even if the absolute brightness at the same position on the four images changes depending on the surface shape or color of the measurement object SC at that position, the brightness difference always has a value corresponding to the phase difference of the stripe pattern. For example, the difference between I0(x,y) and I1(x,y) has a value corresponding to π / 2. Therefore, the phase φ(x,y) at the coordinates (x,y) in the image of the measurement object SC is calculated by having the information processing device 30 execute a calculation to eliminate A, G, and R from equations (1), (2), (3), and (4), specifically, the calculation shown in the following equation (5).

number

[0019] The phase φ(x,y) calculated according to formula (5) is a value between -π and π calculated for each period of each stripe pattern. The phase φ(x,y) calculated according to formula (5) is also called a relative phase because it indicates the position in one period of the stripe pattern. When a stripe pattern of multiple periods is projected onto the measurement target SC and a pixel located at coordinates (x,y) corresponds to the n-th stripe, the phase that uniquely indicates the coordinates (x,y) is expressed as φ(x,y)+2nπ. n is an integer equal to or greater than 0 and is called the stripe order. The stripe order indicates the number of periods of the stripe pattern counted from one end of the stripe pattern of multiple periods toward the other end. The phase considering the stripe order is also called the absolute phase. In order to measure the three-dimensional shape of the measurement target SC by the phase shift method, it is necessary to calculate the absolute phase of the coordinates (x,y). In order to calculate the absolute phase at the coordinates (x,y), it is necessary to estimate the stripe order at the coordinates (x,y). As a method of estimating the fringe order, for example, as disclosed in Patent Document 1, the amplitude of the fringe pattern is changed for each period, the amplitude for each period is written in a table or the like, and the fringe order is estimated based on the amplitude A calculated from equations (1), (2), (3), and (4) and the table. The information processing device 30 measures the three-dimensional shape of the measurement object SC by calculating the height at the coordinates (x, y) according to the principle of triangulation based on the absolute phase at the coordinates (x, y).

[0020] The outline of the measurement of the three-dimensional shape by the phase shift method is as above. However, in the captured image of the measurement object SC in a state where the image of the stripe pattern is projected, the luminance of the stripe pattern may differ depending on the region even in the same captured image due to the influence of the reflection characteristics of the measurement object SC or the surrounding external light. The reflection characteristics of the measurement object SC are determined according to the surface shape or color of the measurement object SC. In the region where the stripe pattern is not captured within the appropriate luminance range, such as when the luminance of the stripe pattern is too high or too low, the measurement accuracy of the three-dimensional shape of the measurement object SC may decrease. The information processing device 30 in this embodiment is configured to avoid the measurement accuracy of the three-dimensional shape of the measurement object SC from decreasing even under the influence of the reflection characteristics of the measurement object SC or the surrounding external light. The following description will be centered on the information processing device 30.

[0021] The information processing device 30 is, for example, a personal computer. As shown in Fig. 1, the information processing device 30 has a communication device 300, a storage device 310, and a processing device 320. The communication device 300 is connected to the projector 10 via a cable. The communication device 300 is also connected to the imaging device 20 via a cable. The communication device 300 receives image data transmitted from the imaging device 20. The communication device 300 also transmits image data representing a pattern image to be projected on the measurement target SC to the projector 10 under the control of the processing device 320.

[0022] The storage device 310 is a recording medium readable by the processing device 320. The storage device 310 includes, for example, a nonvolatile memory and a volatile memory. The nonvolatile memory is, for example, a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM), or an Electrically Erasable Programmable Read Only Memory (EEPROM). The volatile memory is, for example, a Random Access Memory (RAM).

[0023] A program 311 to be executed by the processing device 320 is stored in advance in the non-volatile memory of the storage device 310. The volatile memory of the storage device 310 is used by the processing device 320 as a work area when executing the program 311. The program 311 may also be called an "application program," "application software," or an "app." The program 311 is obtained, for example, from a server or the like (not shown) via the communication device 300, and then stored in the storage device 310.

[0024] The processing device 320 includes a processor such as a CPU (Central Processing Unit), that is, a computer. The processing device 320 may be configured with a single computer or multiple computers. The processing device 320 reads the program 311 from a non-volatile memory to a volatile memory when an operation to instruct the start of execution of the program 311 is performed on an input device (not shown). The processing device 320 executes the read program 311. The processing device 320 operating according to the program 311 functions as the identification unit 321, the detection unit 322, the estimation unit 323, the adjustment unit 324, and the measurement unit 325 shown in FIG. 1. The identification unit 321, the detection unit 322, the estimation unit 323, the adjustment unit 324, and the measurement unit 325 shown in FIG. 1 are software modules realized by operating the processing device 320 according to the program 311. The functions of the identification unit 321, the detection unit 322, the estimation unit 323, the adjustment unit 324, and the measurement unit 325 are as follows.

[0025] The specifying unit 321 specifies the correspondence relationship between the projector coordinate system and the camera coordinate system. The projector coordinate system is a coordinate system that indicates a position within an image projected by the projector 10. A specific example of the projector coordinate system is a two-dimensional coordinate system that has the upper left corner of the image projected by the projector 10 as its origin. The camera coordinate system is a coordinate system that indicates a position within an image captured by the imaging device 20. A specific example of the camera coordinate system is a two-dimensional coordinate system that has the upper left corner of the image captured by the imaging device 20 as its origin.

[0026] In this embodiment, the specification unit 321 displays an image G1 of an 8×8 dot pattern shown in FIG. By providing the image data to the projector 10, the dot pattern image G1 is projected. The image G1 is projected from the vector 10 onto the measurement target SC. The areas marked with white dots are areas where light is not projected from the projector 10, and the white areas are areas where light is not projected from the projector 10. The dots projected by the projector 10 Department The dot pattern image G1 is 1 is an example of a first image in the present disclosure.

[0027] The specifying unit 321 also causes the imaging device 20 to capture the measurement object SC on which the dot pattern image G1 is projected. The captured image of the measurement object SC on which the dot pattern image G1 is projected by the imaging device 20 is an example of a first captured image in this disclosure. The specifying unit 321 then specifies the correspondence between the projector coordinate system and the camera coordinate system based on the captured image of the measurement object SC on which the dot pattern image G1 is projected and the dot pattern image G1.

[0028] Assume that an image G2 shown in FIG. 3 is obtained as a captured image of the measurement target SC in a state in which the image G1 shown in FIG. 2 is projected. The identification unit 321 identifies the correspondence between the projector coordinate system and the camera coordinate system by associating the position of each of the 8×8 dots in the image G1 shown in FIG. 2 in the projector coordinate system with the position of the dot in the camera coordinate system corresponding to the dot in the image G2 shown in FIG. 3. For example, a dot DB11 in the image G2 is associated with a dot DA11 located in the first row and first column in the image G1. Also, a dot DB88 in the image G2 is associated with a dot DA88 located in the eighth row and eighth column in the image G1. The dot DA11 is an example of a first feature point in the present disclosure. The dot DA88 is an example of a second feature point in the present disclosure. The dot DB11 is an example of a third feature point in the present disclosure. The dot DB88 is an example of a fourth feature point in the present disclosure.

[0029] For the projected image from the projector 10 to the measurement target SC, 8×8 regions can be set that are centered on each of the 8×8 dots shown in FIG. 2 and do not overlap with each other. In FIG. 2, among these 8×8 regions, an area AA11 centered on the dot DA11 and an area AA88 centered on the dot DA88 are illustrated by dotted lines. Similarly, for the captured image by the imaging device 20, 8×8 regions can be set that are centered on each of the 8×8 dots shown in FIG. 3 and do not overlap with each other. In FIG. 3, among these 8×8 regions, an area AB11 centered on the dot DB11 and an area AB88 centered on the dot DB88 are illustrated by dotted lines. Each of the 8×8 regions set for the projected image is one-to-one associated with each of the 8×8 regions set for the captured image by the correspondence relationship identified by the identification unit 321. Specifically, the area AB11 is associated with the area AA11. Additionally, area AA88 is associated with area AB88. Area AB11 is an example of a first area in the present disclosure. Area AB88 is an example of a second area in the present disclosure. Area AA11 is an example of a third area in the present disclosure. Area AA88 is an example of a fourth area in the present disclosure.

[0030] The detection unit 322 provides the projector 10 with image data representing a monochromatic image, thereby causing the projector 10 to project the monochromatic image onto the measurement target SC. The detection unit 322 then causes the imaging device 20 to capture the measurement target SC in a state in which the monochromatic image is projected. This monochromatic image is an example of a second image in the present disclosure. The captured image of the measurement target SC in a state in which the monochromatic image is projected is an example of a second captured image in the present disclosure. In this embodiment, the detection unit 322 causes the projector 10 to project each of a plurality of monochromatic images, each of which has a uniform luminance within the image and differs from one another, onto the measurement target SC. More specifically, the detection unit 322 causes the projector 10 to project each of white images, each of which has a uniform luminance within the image and has a luminance of 20%, 40%, 60%, 80%, and 100%, onto the measurement target SC. The detection unit 322 causes the imaging device 20 to capture the measurement target SC in a state in which the monochromatic image is projected, for each of the plurality of monochromatic images. Then, the detection section 322 detects the luminance in each of the 8×8 regions AB11 to AB88 in the captured image for each monochrome image.

[0031] The estimation unit 323 estimates a luminance distribution map indicating the distribution of luminance of the projected image that does not saturate the luminance above or below in any of the areas AB11 to AB88 in the captured image, based on the detection result by the detection unit 322. More specifically, for each of the 8×8 areas AA11 to AA88, the estimation unit 323 estimates a luminance range that does not saturate the luminance above or below in the area associated by the correspondence identified by the identification unit 321, based on the luminance detected for each monochrome image by the detection unit 322.

[0032] For example, for the region AB11, it is assumed that the luminance value is saturated upward for a 100% white image and saturated downward for a 40% white image. Here, the luminance value is saturated upward means that the luminance value is the upper limit value, and the luminance value is saturated downward means that the luminance value is the lower limit value. In this case, the estimation unit 323 estimates the luminance range of the region AA11, in which the luminance of the region AB11 is not saturated upward or downward, to be 60% to 80%. The luminance range estimated for the region AA11 is an example of the first luminance range in the present disclosure. In addition, it is assumed that the luminance value is saturated upward for an 80% white image and saturated downward for a 20% white image, in the region AB88. In this case, the estimation unit 323 estimates the luminance range of the region AA88, in which the luminance of the region AB88 is not saturated upward or downward, to be 40% to 60%. The luminance range estimated for the region AA88 is an example of the second luminance range in the present disclosure. In this embodiment, for each of the 8 × 8 regions AA11 to AA88, the estimation unit 323 estimates a luminance range that does not saturate the luminance of the corresponding region by the correspondence relationship identified by the identification unit 321, either above or below, and the distribution of the estimated luminance range becomes a luminance distribution map.

[0033] The adjustment unit 324 performs the following process on each of the four stripe pattern images, each of which is shifted in phase by π / 2, as an adjustment target image. The adjustment unit 324 divides the adjustment target image into 8×8 regions that correspond one-to-one to each of the above-mentioned regions AA11 to AA88. Then, the adjustment unit 324 adjusts the amplitude of the sine wave for each region obtained by dividing the adjustment target image into 8×8 regions, based on the luminance distribution map estimated by the estimation unit 323. For example, for the region corresponding to the region AA11, the adjustment unit 324 adjusts the luminance of each pixel belonging to the region so that it falls within the luminance range estimated for the region AA11. The region corresponding to the region AA11 in each of the four stripe pattern images is an example of a fifth region in the present disclosure. Similarly, for the region corresponding to the region AA88, the adjustment unit 324 adjusts the luminance of each pixel belonging to the region so that it falls within the luminance range estimated for the region AA88. The area corresponding to area AA88 in each of the four stripe pattern images is an example of the sixth area in this disclosure.

[0034] The measurement unit 325 measures the three-dimensional shape of the measurement object SC by performing a phase shift method using an image of a stripe pattern whose luminance has been adjusted for each region by the adjustment unit 324. More specifically, the measurement unit 325 causes the projector 10 to project an image of a stripe pattern whose luminance has been adjusted for each region by the adjustment unit 324. The measurement unit 325 also causes the imaging device 20 to capture the measurement object SC in a state in which the pattern image is projected. Then, the measurement unit 325 calculates the relative phase φ(x, y) at the coordinates (x, y) by the calculation shown in the above-mentioned formula (5). The measurement unit 325 estimates the stripe order to calculate the absolute phase. Then, the measurement unit 325 measures the three-dimensional shape of the measurement object SC by calculating the height at the coordinates (x, y) by the principle of triangulation based on the absolute phase at the coordinates (x, y).

[0035] Moreover, the processing device 320 operating according to the program 311 executes the measurement method of the present disclosure. Fig. 4 is a flowchart showing the flow of the measurement method of the present disclosure. As shown in Fig. 4, the measurement method of the present disclosure includes an identification process SA110, a detection process SA120, an estimation process SA130, an adjustment process SA140, and a measurement process SA150.

[0036] In the identification process SA110, the processing device 320 functions as an identification unit 321. In the identification process SA110, the processing device 320 causes the projector 10 to project an image G1 of a dot pattern shown in Fig. 2 onto the measurement object SC. The processing device 320 also causes the imaging device 20 to capture the measurement object SC in a state in which the image G1 of the dot pattern is projected. Then, the processing device 320 identifies the correspondence between the projector coordinate system and the camera coordinate system based on the captured image of the measurement object SC in a state in which the image G1 of the dot pattern is projected and the image G1 of the dot pattern.

[0037] In the detection process SA120, the processing device 320 functions as a detection unit 322. In the detection process SA120, the processing device 320 provides the projector 10 with image data representing each of a plurality of monochromatic images having different luminance values, thereby causing the projector 10 to project each monochromatic image onto the measurement object SC. Furthermore, the processing device 320 causes the imaging device 20 to capture an image of the measurement object SC in a state in which the monochromatic image is projected, for each monochromatic image. Then, the processing device 320 detects the luminance value in each of a plurality of regions set in the captured image, for each monochromatic image.

[0038] In the estimation process SA130, the processing device 320 functions as an estimation unit 323. In the estimation process SA130, the processing device 320 estimates a luminance distribution map indicating a distribution of luminance values ​​of a projected image in which luminance values ​​are not saturated above or below in any region of the captured image, based on a detection result by the detection unit 322.

[0039] In the adjustment process SA140, the processing device 320 functions as an adjustment unit 324. In the adjustment process SA140, the processing device 320 sets each of the four stripe pattern images with a phase shift of π / 2 as an adjustment target image, and adjusts the amplitude of the sine wave so that the luminance falls within the range indicated by the luminance distribution map estimated in the estimation process SA130 for each region of the adjustment target image.

[0040] In the measurement process SA150, the processing device 320 functions as a measurement unit 325. In the measurement process SA150, the processing device 320 measures the three-dimensional shape of the measurement object SC by performing a phase shift method using an image of a stripe pattern whose luminance has been adjusted for each region in the adjustment process SA140.

[0041] Hereinafter, in a captured image of the measurement target SC in a state where a stripe pattern image is projected, Only the area AB11 has a higher brightness than the other areas due to the reflection characteristics of the area or the influence of external light, etc. The effect of this embodiment will be described by taking the case where the luminance distribution is uniform as an example. In the phase shift method, the overall brightness is suppressed so that the brightness in the region AB11 does not become saturated. Since the image of the stripe pattern is projected onto the measurement target SC, the area other than the area AB11 is Okay. The captured image The signal value decreases. The S / N ratio decreases in areas other than area AB11. Therefore, the calculation accuracy of the relative phase φ(x, y) decreases, and the measurement accuracy of the three-dimensional shape of the measurement target SC decreases. decreases.

[0042] On the other hand, in the measurement method of this embodiment, a stripe pattern image in which only the luminance of area AA11 that corresponds to area AB11 based on the correspondence between the projector coordinate system and the camera coordinate system is suppressed is projected onto the measurement object SC, so that a high signal value is obtained over almost the entire captured image of the measurement object SC, and it is possible to avoid a decrease in the calculation accuracy of the relative phase φ(x, y). Therefore, according to this embodiment, it is possible to avoid a decrease in the measurement accuracy of the three-dimensional shape of the measurement object SC due to the influence of the reflection characteristics or external light, etc.

[0043] As described above, according to this embodiment, a stripe pattern image whose luminance has been adjusted according to the influence of the reflection characteristics of the measurement object SC or the surrounding external light is projected. Therefore, in a captured image of the measurement object SC in a state where the stripe pattern image is projected, the occurrence of whiteout caused by the luminance value saturating upward or blackout caused by the luminance value saturating downward is avoided. In a captured image of the measurement object SC in a state where the stripe pattern image is projected, no whiteout or blackout occurs, so the measurement accuracy of the three-dimensional shape of the measurement object SC is not reduced. In this way, according to the measurement system 1 of this embodiment, it is possible to prevent the measurement accuracy of the three-dimensional shape of the measurement object SC from being reduced due to the reflection characteristics of the measurement object SC or the influence of the surrounding external light.

[0044] In this embodiment, the three-dimensional shape of the measurement object SC is measured by the principle of triangulation using an image of a stripe pattern whose brightness has been adjusted for each region and a captured image of the measurement object SC onto which the image of the stripe pattern is projected. However, the three-dimensional shape of the measurement object SC may be measured using an image obtained by capturing an image of the measurement object SC onto which an image of a stripe pattern whose brightness has been adjusted for each region is projected by the imaging device 20, which is a first camera, and an image obtained by capturing an image by a second camera disposed at a position different from the first camera.

[0045] In the three-dimensional measurement by the phase shift method using the first camera and the second camera, the three-dimensional coordinates are calculated from the parallax value between the coordinates of the feature point in the image captured by the first camera and the coordinates of the feature point in the image captured by the second camera. Therefore, if the variation in the coordinates of the feature point in the image captured by the first camera and the image captured by the second camera is small, it is possible to calculate three-dimensional coordinates with smaller variation, that is, to improve the three-dimensional measurement accuracy. FIG. 5 is a diagram showing an example of the variation in the coordinates of the feature point calculated by the phase shift method based on the conventional uniform luminance distribution. On the other hand, FIG. 6 is a diagram showing an example of the variation in the coordinates of the feature point calculated using an image of a stripe pattern in which the luminance has been adjusted for each region. As is clear from a comparison between FIG. 5 and FIG. 6, by using an image of a stripe pattern in which the luminance has been adjusted for each region, the variation in the coordinates of the feature point is smaller than that in the conventional phase shift method based on the uniform luminance distribution. The RMSE in FIG. 5 is 0.260535 pixels, and the RMSE in FIG. 6 is 0.112996 pixels. Note that RMSE is the root mean square error. In this way, even in 3D measurement using the phase shift method with the first and second cameras, by using an image of a stripe pattern with brightness adjusted for each region, a decrease in measurement accuracy due to the reflection characteristics of the measurement object SC or the influence of surrounding external light, etc. is avoided.

[0046] 2. Variations The above embodiment may be modified as follows. (1) In the above embodiment, a dot pattern image is used as a pattern image for identifying the correspondence between the projector coordinate system and the camera coordinate system. However, a gray code pattern or a binary code pattern image may be used instead of the dot pattern image. Also, in the above embodiment, the three-dimensional shape of the measurement target SC is measured using a stripe pattern image in which the luminance in the image changes sinusoidally. However, a gray code pattern or a binary code pattern image may be used instead of the stripe pattern image. In the gray code pattern or binary code pattern image, the luminance in the image changes periodically, similar to the stripe pattern image. When a gray code pattern or a binary code pattern image is used, in a brightly captured area, the pixels in the white portion may be adjusted to a dark color such as gray.

[0047] (2) In the above embodiment, the luminance distribution map is created using four monochromatic images with different luminances. However, the luminance distribution map may be created using two or three monochromatic images with different luminances, or may be created using five or more monochromatic images with different luminances. One monochromatic image in the aspect in which two monochromatic images with different luminances are used is an example of a first monochromatic image in the present disclosure. The other monochromatic image in the aspect in which two monochromatic images with different luminances are used is an example of a second monochromatic image in the present disclosure. Also, the luminance distribution map may be created using one monochromatic image with uniform luminance within the image. Specifically, the luminance distribution map may be created according to the difference between the luminance of each region in the captured image of the measurement target SC in a state in which a monochromatic image is projected and a reference value predicted as the luminance in the captured image.

[0048] (3) In the above embodiment, the program 311 is stored in the storage device 310. However, the program 311 may be manufactured or distributed as a standalone program. Specific methods for distributing the program 311 include distributing the program 311 by writing it to a computer-readable recording medium such as a flash ROM (Read Only Memory) or the like, or distributing it by downloading it via a telecommunications line such as the Internet.

[0049] (4) In the above embodiment, the identification unit 321, the detection unit 322, the estimation unit 323, the adjustment unit 324, and the measurement unit 325 are software modules. However, the detection unit 322, the estimation unit 323, the adjustment unit 324, and the measurement unit 325 may be partly or entirely hardware. Examples of this hardware include a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). Even if the detection unit 322, the estimation unit 323, the adjustment unit 324, and the measurement unit 325 are partly or entirely hardware, the same effects as those of the above embodiment are achieved. Note that the information processing device 30 in the above embodiment may be a tablet terminal or a smartphone.

[0050] (5) The measuring unit 325 in the above embodiment may be omitted. According to the information processing device having the identifying unit 321, the detecting unit 322, the estimating unit 323, and the adjusting unit 324, the occurrence of whiteout or blackout in the captured image of the projection target onto which the projection image is projected from the projector 10 is avoided. Moreover, a projection system may be configured by combining this information processing device, the projector 10, and the imaging device 20. According to this projection system, the occurrence of whiteout or blackout in the captured image of the projection target onto which the projection image is projected from the projector 10 is avoided. Furthermore, a projection image adjustment method including the identifying process SA110, the detecting process SA120, the estimating process SA130, and the adjusting process SA140 may be provided. According to this adjustment method, the occurrence of whiteout or blackout in the captured image of the projection target onto which the projection image is projected from the projector 10 is avoided.

[0051] (6) In the above embodiment, the adjustment unit 324 divides the image to be adjusted into 8×8 regions that correspond one-to-one to each of the regions AA11 to AA88, and adjusts the luminance of each pixel in each region based on the luminance distribution map. However, in order to prevent discontinuous changes in the luminance of two pixels adjacent to each other across the boundary between the regions, a filter process using a moving average filter or the like may be performed on the image to be adjusted after the luminance of each pixel has been adjusted based on the luminance distribution map.

[0052] 3. Aspects grasped from at least one of the embodiments and each modified example The present disclosure is not limited to the above-mentioned embodiments and modifications, and can be realized in various aspects without departing from the spirit of the present disclosure. For example, the present disclosure can also be realized in the following aspects. The technical features in the above-mentioned embodiments corresponding to the technical features in each aspect described below can be appropriately replaced or combined in order to solve some or all of the problems of the present disclosure or to achieve some or all of the effects of the present disclosure. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted.

[0053] In order to solve the above-mentioned problems, the projection image adjustment method of the present disclosure is an adjustment method of a projection image projected from the projector 10 to a projection target, and includes a specification process SA110, a detection process SA120, an estimation process SA130, and an adjustment process SA140. In the specification process SA110, a correspondence relationship between a projector coordinate system and a camera coordinate system is specified based on a first image projected from the projector 10 to a projection target and a first captured image obtained by capturing an image of the projection target in a state in which the first image is projected by the image capture device 20. The projector coordinate system is a coordinate system indicating a position in the first image. The camera coordinate system is a coordinate system indicating a position in the first captured image. In the detection process SA120, the luminance of a first region in a second captured image obtained by capturing an image of the projection target in a state in which the second image is projected from the projector 10 by the image capture device 20 and the luminance of a second region in the second captured image are detected. Note that a monochrome image is an example of the second image. In the estimation process SA130, for a third region in the second image that is associated with the first region by the above-mentioned correspondence, a first luminance range that does not saturate the luminance of the first region vertically is estimated based on the luminance detected for the first region and the luminance of the third region. In addition, in the estimation process SA130, for a fourth region in the second image that is associated with the second region by the above-mentioned correspondence, a second luminance range that does not saturate the luminance of the second region vertically is estimated based on the luminance detected for the second region and the luminance of the fourth region. In the adjustment process SA140, the luminance of a fifth region in the projected image that corresponds to the third region is adjusted so as to fall within the first luminance range. In addition, in the adjustment process SA140, the luminance of a sixth region in the projected image that corresponds to the fourth region is adjusted so as to fall within the second luminance range. According to the adjustment method of this aspect, when the projection target in a state where the projection image is projected is imaged by the first camera, the occurrence of whiteout or blackout in the captured image is avoided.

[0054] In a more preferred embodiment of the adjustment method, the first and second monochromatic images, which are uniform in luminance within the image and have different luminances, may be projected sequentially by the projector 10 as the second image to estimate the first and second luminance ranges. Specifically, in the estimation process SA130, the first luminance range is estimated based on the luminance of the first region detected when the first monochromatic image is projected, the luminance of the first region detected when the second monochromatic image is projected, and the luminance of the third region. Similarly, in the estimation process SA130, the second luminance range is estimated based on the luminance of the second region detected when the first monochromatic image is projected, the luminance of the second region detected when the second monochromatic image is projected, and the luminance of the fourth region.

[0055] In another preferred embodiment of the adjustment method, the first image may be an image including a first feature point and a second feature point, specifically, an image of a dot pattern. In the identification process SA110 in the adjustment method of this embodiment, the correspondence between the projector coordinate system and the camera coordinate system is identified based on the position of the first feature point in the first image, the position of a third feature point corresponding to the first feature point in the first captured image, and the position of the second feature point in the first image, and the position of a fourth feature point corresponding to the second feature point in the first captured image.

[0056] In another preferred embodiment of the adjustment method, the projection image projected from the projector 10 onto the projection target may be an image in which the luminance changes periodically, specifically, an image of a stripe pattern. According to this embodiment, the occurrence of whiteout or blackout in the fifth region of the captured image of the projection target onto which the projection image in which the luminance changes periodically is projected is avoided. Similarly, the occurrence of whiteout or blackout in the sixth region is also avoided.

[0057] Moreover, one aspect of the measurement method of the present disclosure is a measurement method in which a projection image in which the luminance changes periodically within the image is projected from the projector 10 onto a projection object, and a three-dimensional shape of the projection object is measured based on a captured image obtained by capturing an image of the projection object on which the projection image is projected by a first camera and a captured image obtained by capturing an image of the projection object on which the projection image is projected by a second camera different from the first camera. This measurement method includes the above-mentioned identification process SA110, detection process SA120, estimation process SA130, and adjustment process SA140, and the luminance of the projection image projected from the projector 10 onto the projection object is adjusted by the adjustment process SA140. According to the adjustment method of this aspect, it is possible to avoid a decrease in the measurement accuracy of the three-dimensional shape of the projection object due to the influence of the reflection characteristics of the projection object or the surrounding external light, etc.

[0058] Moreover, one aspect of the measurement method of the present disclosure is a measurement method in which a projection image in which the luminance changes periodically within the image is projected from the projector 10 onto a projection object, and a three-dimensional shape of the projection object is measured based on a captured image obtained by capturing an image of the projection object in a state in which the projection image is being projected by a first camera and the projected image. This measurement method includes the above-mentioned identification process SA110, detection process SA120, estimation process SA130, and adjustment process SA140, and the luminance of the projection image projected from the projector 10 onto the projection object is adjusted by the adjustment process SA140. The adjustment method of this aspect also prevents the measurement accuracy of the three-dimensional shape of the projection object from being reduced due to the influence of the reflection characteristics of the projection object or the surrounding external light, etc.

[0059] In addition, in order to solve the above-mentioned problems, one aspect of the projection system of the present disclosure includes a projector 10, an imaging device 20, and an information processing device 30. The projector 10 projects a projection image onto a projection object. The imaging device 20 captures an image of the projection object. The imaging device 20 is an example of a first camera in the present disclosure. The information processing device 30 has a processing device 320 that executes the above-mentioned identification process SA110, detection process SA120, estimation process SA130, and adjustment process SA140. According to this aspect, the occurrence of whiteout or blackout in the fifth region of the captured image obtained by capturing an image of the projection object in a state in which the projection image is projected by the imaging device 20 is avoided. Similarly, the occurrence of whiteout or blackout in the sixth region is also avoided.

[0060] In addition, in order to solve the above-mentioned problems, one aspect of the information processing device of the present disclosure includes a determination unit 321, a detection unit 322, an estimation unit 323, and an adjustment unit 324. The determination unit 321 executes a determination process SA110. The detection unit 322 executes a detection process SA120. The estimation unit 323 executes an estimation process SA130. The adjustment unit 324 executes an adjustment process SA140. According to this aspect, the occurrence of whiteout or blackout in the fifth region of the captured image obtained by capturing an image of the projection object in a state in which the projection image is projected by the imaging device 20 is avoided. Similarly, the occurrence of whiteout or blackout in the sixth region is also avoided.

[0061] In order to solve the above-mentioned problems, one aspect of the program of the present disclosure causes a computer to execute a specification process SA110, a detection process SA120, an estimation process SA130, and an adjustment process SA140. According to this aspect, the occurrence of whiteout or blackout in the fifth region of the captured image obtained by capturing an image of the projection object with the imaging device 20 while the projection image is being projected is avoided. Similarly, the occurrence of whiteout or blackout in the sixth region is also avoided. [Explanation of symbols]

[0062] 1...measurement system, 10...projector, 20...imaging device, 30...information processing device, 300...communication device, 310...storage device, 320...processing device, 321...identification unit, 322...detection unit, 323...estimation unit, 324...adjustment unit, 325...measurement unit, 311...program, SC...measurement object.

Claims

1. A method for adjusting a projection image projected from a projector onto a projection object, comprising: A first feature point and a second feature point projected from the projector onto the projection object. a first image including the first image and the projection object in a state where the first image is projected, by a first camera; and a projector that indicates a position within the first image based on the first captured image. determining a correspondence relationship between the first coordinate system and a camera coordinate system indicating a position within the first captured image; The projection object in a state where the second image is projected from the projector is captured by the first camera. The luminance of the first region in the second captured image obtained by imaging with the laser and the luminance of the second captured image and detecting a luminance of a second region in the image; Regarding a third region of the second image that is associated with the first region according to the correspondence relationship, A first luminance range in which the luminance of the first region is not saturated above or below is set for the first region. Estimating based on the detected luminance and the luminance of the third region; Regarding a fourth region in the second image that is associated with the second region according to the correspondence relationship, A second luminance range in which the luminance of the second region is not saturated above or below is set for the second region. Estimating based on the detected luminance and the luminance of the fourth region; The luminance of a fifth area of ​​the projected image corresponding to the third area is set within the first luminance range. and adjusting the luminance of a sixth area of ​​the projected image corresponding to the fourth area to the Adjusting the brightness to fall within the second brightness range, The position of the first feature point in the first image and the position of the first feature point in the first captured image a position of a third feature point corresponding to the point, and a position of the second feature point in the first image and the and determining whether the corresponding feature point is a fourth feature point in the first captured image, based on the position of the fourth feature point corresponding to the second feature point. Identify relationships, Adjustment method.

2. The method of claim 1 , wherein the second image is a monochromatic image.

3. A first monochrome image and a second monochrome image having uniform brightness in the image, the brightness of which is different from each other The first monochrome image and the second monochrome image are each projected as a second image onto the projector. Sequentially projecting, The luminance of the first area and the luminance of the second monochrome image detected when the first monochrome image is projected Based on the luminance of the first area and the luminance of the third area detected when an image is projected, and estimating the first luminance range based on the The luminance of the second area and the second monochrome image detected when the first monochrome image is projected Based on the luminance of the second area and the luminance of the fourth area detected when an image is projected, The adjustment method according to claim 2 , wherein the second luminance range is estimated based on the first luminance range.

4. 4. The projection image according to claim 1, wherein the luminance of the projected image changes periodically depending on the position of the image. The adjustment method according to any one of the above.

5. An image in which the brightness periodically changes depending on the position within the image is projected from the projector onto the target. The projection image is projected onto the projection target by a first camera. a captured image of the projection object on which the projection image is projected, and A three-dimensional image of the object to be projected is obtained based on the captured image obtained by capturing an image by a different second camera. In the method for measuring the original shape, A first feature point and a second feature point projected from the projector onto the projection object. and a first image including the first image and the projection object in a state where the first image is projected. A plot indicating a position in the first image is generated based on a first captured image obtained by capturing the first captured image with a camera. a projector coordinate system that indicates a position in the first captured image and a camera coordinate system that indicates a position in the first captured image; death, The projection object in a state where the second image is projected from the projector is captured by the first camera. The luminance of the first region and the luminance of the second region in the second captured image obtained by imaging with the camera are detected. broth, Regarding a third region of the second image that is associated with the first region according to the correspondence relationship, A first luminance range in which the luminance of the first region is not saturated above or below is set for the first region. Estimating based on the detected luminance and the luminance of the third region; Regarding a fourth region in the second image that is associated with the second region according to the correspondence relationship, A second luminance range in which the luminance of the second region is not saturated above or below is set for the second region. Estimating based on the detected luminance and the luminance of the fourth region; The luminance of a fifth area of ​​the projected image corresponding to the third area is set within the first luminance range. and adjusting the luminance of a sixth area of ​​the projected image corresponding to the fourth area to the Adjusting the brightness to fall within the second brightness range, The position of the first feature point in the first image and the position of the first feature point in the first captured image a position of a third feature point corresponding to the point, and a position of the second feature point in the first image and the and determining whether the corresponding feature point is a fourth feature point in the first captured image, based on the position of the fourth feature point corresponding to the second feature point. Identify relationships, Measurement method.

6. An image in which the brightness periodically changes depending on the position within the image is projected from the projector onto the target. The projection image is projected onto the projection target by a first camera. measuring a three-dimensional shape of the projection target based on the captured image and the projected image; In the method, A first image including a first feature point and a second feature point is projected from the projector. a first captured image obtained by capturing an image of the projection object in a state in which the projection object is projected by the first camera; A projector coordinate system indicating a position in the first image and a coordinate system indicating a position in the first captured image are calculated based on the image. Identifying a correspondence with a camera coordinate system that indicates a position in the image; The projection object in a state where the second image is projected from the projector is captured by the first camera. The luminance of the first region and the luminance of the second region in the second captured image obtained by imaging with the camera are detected. broth, Regarding a third region of the second image that is associated with the first region according to the correspondence relationship, A first luminance range in which the luminance of the first region is not saturated above or below is set for the first region. Estimating based on the detected luminance and the luminance of the third region; Regarding a fourth region in the second image that is associated with the second region according to the correspondence relationship, A second luminance range in which the luminance of the second region is not saturated above or below is set for the second region. Estimating based on the detected luminance and the luminance of the fourth region; The luminance of a fifth area of ​​the projected image corresponding to the third area is set within the first luminance range. and adjusting the luminance of a sixth area of ​​the projected image corresponding to the fourth area to the Adjusting the brightness to fall within the second brightness range, The position of the first feature point in the first image and the position of the first feature point in the first captured image a position of a third feature point corresponding to the point, and a position of the second feature point in the first image and the and determining whether the corresponding feature point is a fourth feature point in the first captured image, based on the position of the fourth feature point corresponding to the second feature point. Identify relationships, Measurement method.

7. A projector that projects a projection image onto a projection target; A first camera that captures an image of the projection object; An information processing device, The information processing device includes: A first image including a first feature point and a second feature point is projected from the projector. a first captured image obtained by capturing an image of the projection object in a state in which the projection object is projected by the first camera; A projector coordinate system indicating a position in the first image and a coordinate system indicating a position in the first captured image are calculated based on the image. Identifying a correspondence with a camera coordinate system that indicates a position within the image; The projection object in a state where the second image is projected from the projector is captured by the first camera. The luminance of the first region and the luminance of the second region in the second captured image obtained by imaging with the camera are detected. To put out, Regarding a third region of the second image that is associated with the first region according to the correspondence relationship, A first luminance range in which the luminance of the first region is not saturated above or below is set for the first region. estimating based on the detected luminance and a luminance of the third region; Regarding a fourth region in the second image that is associated with the second region according to the correspondence relationship, A second luminance range in which the luminance of the second region is not saturated above or below is set for the second region. estimating based on the detected luminance and a luminance of the fourth region; The luminance of a fifth area of ​​the projected image corresponding to the third area is set within the first luminance range. and adjusting the luminance of a sixth area of ​​the projected image corresponding to the fourth area to the Adjusting the brightness to fall within the second brightness range; and The position of the first feature point in the first image and the position of the first feature point in the first captured image a position of a third feature point corresponding to the point, and a position of the second feature point in the first image and the and determining whether the corresponding feature point is a fourth feature point in the first captured image, based on the position of the fourth feature point corresponding to the second feature point. Identifying relationships; Projection system.

8. A projector that projects a projection image onto a projection target includes a first feature point and a second feature point. a second image obtained by capturing an image of the projection object with a first camera while the first image is being projected; A projector coordinate system indicating a position in the first image based on the captured image and the first image. specifying a correspondence relationship between the reference system and a camera coordinate system indicating a position within the first captured image; The projection object in a state where the second image is projected from the projector is captured by the first camera. The luminance of the first region and the luminance of the second region in the second captured image obtained by imaging with the camera are detected. To put out, Regarding a third region of the second image that is associated with the first region according to the correspondence relationship, A first luminance range in which the luminance of the first region is not saturated above or below is set for the first region. estimating based on the detected luminance and a luminance of the third region; Regarding a fourth region in the second image that is associated with the second region according to the correspondence relationship, A second luminance range in which the luminance of the second region is not saturated above or below is set for the second region. estimating based on the detected luminance and a luminance of the fourth region; The luminance of a fifth area of ​​the projected image corresponding to the third area is set within the first luminance range. and adjusting the luminance of a sixth area of ​​the projected image corresponding to the fourth area to the Adjusting the brightness to fall within the second brightness range; and The position of the first feature point in the first image and the position of the first feature point in the first captured image a position of a third feature point corresponding to the point, and a position of the second feature point in the first image and the and determining whether the corresponding feature point is a fourth feature point in the first captured image, based on the position of the fourth feature point corresponding to the second feature point. Identifying relationships, An information processing device that executes the above.

9. On the computer, A projector that projects a projection image onto a projection target includes a first feature point and a second feature point. a second image obtained by capturing an image of the projection object with a first camera while the first image is being projected; A projector coordinate system indicating a position in the first image based on the captured image and the first image. specifying a correspondence relationship between the reference system and a camera coordinate system indicating a position within the first captured image; The projection object in a state where the second image is projected from the projector is captured by the first camera. The luminance of the first region and the luminance of the second region in the second captured image obtained by imaging with the camera are detected. To put out, Regarding a third region of the second image that is associated with the first region according to the correspondence relationship, A first luminance range in which the luminance of the first region is not saturated above or below is set for the first region. estimating based on the detected luminance and a luminance of the third region; Regarding a fourth region in the second image that is associated with the second region according to the correspondence relationship, A second luminance range in which the luminance of the second region is not saturated above or below is set for the second region. estimating based on the detected luminance and a luminance of the fourth region; The luminance of a fifth area of ​​the projected image corresponding to the third area is set within the first luminance range. and adjusting the luminance of a sixth area of ​​the projected image corresponding to the fourth area to the Adjusting the brightness to fall within the second brightness range; and The position of the first feature point in the first image and the position of the first feature point in the first captured image a position of a third feature point corresponding to the point, and a position of the second feature point in the first image and the and determining whether the corresponding feature point is a fourth feature point in the first captured image, based on the position of the fourth feature point corresponding to the second feature point. Identifying relationships, A program that executes the following.

Images