Image processing equipment, image processing system and program

Abstract

Image processing equipment (3), comprising: an image acquisition part (322) which captures a multitude of different measurement images which have been captured by recording an object to be measured, onto which light with different projection patterns, the intensity of which changes according to phases determined by the coordinates of a projection image and whose relationships between the phase and the coordinate are different from each other, is sequentially projected; a model part (323) which identifies for each pixel a modeled parameter which approximates a data sequence in which pixel values ​​from the respective measurement images are placed in a recording order; a generating part (324) for reconstructed images, which generates reconstructed images that correspond to the respective measurement images and are reconstructed with approximate values ​​of the respective pixels, which are identified based on the modeled parameter of each pixel; and an image modification part (325) which modifies the pixel values ​​of the measurement images based on statistics of the pixel values ​​of the measurement images and statistics of the pixel values ​​of the corresponding reconstructed images, wherein the image modification part (325) modifies pixel values ​​of the measurement images by subtracting from the pixel value of each pixel of the respective measurement images a difference between a mean value of the pixel values ​​of the pixels contained in the measurement images and a mean value of the pixel values ​​of the pixels contained in the corresponding reconstructed images.

Description

AREA OF TECHNOLOGY

[0001] This invention relates to an image processing device, an image processing system and a program for measuring the geometry of an object to be measured. TECHNICAL BACKGROUND

[0002] A technique for measuring the three-dimensional geometry of an object is known as a 3D geometry measuring machine, which measures the geometry of the object by capturing the object onto which light is projected with a variety of projection patterns (see non-patent literature document 1, "Advanced optical 3D measurement" (Japanese: "Saishin Hikari Sanjigen Sokutei") by Toru Yoshizawa, published by Asakura Shoten in 2006, ISBN 4-254-20129-X). Further non-patent literature includes Rosman, G. et al.: "Sparse Modeling of Shape from Structured Light." In: IEEE 2012 Second Joint 3DIM / 3DPVT Conference: 3D Imaging, Modeling, Processing, Visualization & Transmission, and Leong-Hoi, A. et al.: “High speed implementation of a three-dimensional shape profiler with submillimeter precision using a digital light processing device and a new efficient algorithm for absolute phase retrieval” In: Optical Engineering, Vol. 54, No. 3, 2015.

[0003] US 2010 O 103 169 A1 discloses a method for reconstructing a 3D surface model, comprising the following steps: obtaining a 3D position and the reflection parameters corresponding to an object according to the structured light system; creating a synthesized image according to the 3D position and the reflection parameters; then optimizing the reflection parameters for the synthesized image until the cost functions are less than a predetermined value. The invention presents an optimization algorithm for simultaneously estimating both a 3D shape and the parameters of a surface reflection model of real objects.

[0004] JP 2017-125 801 A describes a method for measuring three-dimensional shape, comprising: sequentially projecting a measurement object at a projection angle with a plurality of structured lights differing in phase input; photographing a measurement image of the measurement object onto which the plurality of structured lights was projected; and calculating a three-dimensional shape of the measurement object using a phase-shift method based on a plurality of photographed measurement images.The signal intensity of each pixel of the measurement image is expressed by an expression that uses an offset amount of the structured light in the measurement image, its amplitude, its phase, its phase shift amount, its number of phase steps and the pixel index, and includes the calculation of the three-dimensional shape: the calculation of the phase based on the signal intensity, the offset amount, the amplitude and the phase shift amount; and the calculation of the three-dimensional shape of the measurement object based on the calculated phase.

[0005] DE 10 2016 105 858 A1 describes a motorized tripod that can be used to hold a three-dimensional (3D) measuring instrument. The disclosed subject matter relates in particular to a coordinate measuring machine with a 3D time-of-flight laser scanner (TOF laser scanner). A 3D laser scanner of this type directs a light beam to a non-cooperating target, such as the diffusely scattering surface of an object. A distance meter in the device measures the distance to the object, and angle encoders measure the rotation angles of two axes in the device. The measured distance and the two measured angles enable a processor in the device to determine the 3D coordinates of the target.

[0006] US 2016 / 0173842A1 discloses a system and method for facilitating keystone correction for a specific projector model with an attached camera. System calibration determines intrinsic and extrinsic parameters of the projector and camera; control points are then identified in a three-dimensional space in front of a screen. The three-dimensional space defines a throw range and maximum pitch and yaw offsets for the projector / screen combination. At each control point, the projector projects a group of structured light elements onto the screen, and the camera captures an image of the projected pattern. These images are used to create three-dimensional lookup tables that establish a relationship between each image and at least one of the following: (i) the pitch and yaw offset angles for the respective control point, and (ii) a focal length and principal point for the respective control point.The given model projectors use the tables to perform the trapezoidal correction. SUMMARY OF THE INVENTION; TASKS TO BE SOLVED BY THE INVENTION

[0007] A measurement image captured by the 3D geometry measuring machine, which records the object to be measured, can be influenced by an external environment, such as fluctuations in the amount of light in the vicinity of the object to be measured, fluctuations in the intensity of the light projected onto the object to be measured, or the like.

[0008] There are concerns that the measurement accuracy at the time of measuring the object being measured will be negatively affected if the measurement image is influenced by the external environment.

[0009] This invention addresses this point, and one object of the invention is to provide a technique for reducing the influence of the external environment on the measurement image. MEANS OF SOLVING THE TASKS

[0010] An image processing device according to the first aspect of this invention comprises: an image acquisition part which acquires a plurality of different measurement images, which are acquired by capturing an object to be measured, onto which light with different projection patterns, the luminous intensity of which changes according to phases determined by the coordinates of a projection image and whose relationships between the phase and the coordinate differ from one another, is sequentially projected; a modeling part which identifies a modeled parameter for each pixel, which approximates a data sequence in which pixel values ​​corresponding to the respective measurement images are placed in a sequence of acquisition;a generation part for reconstructed images, which generates reconstructed images that correspond to the respective measurement images and are reconstructed with approximate values ​​of the respective pixels, identified based on the modeled parameter of each pixel; and an image modification part, which modifies the pixel values ​​of the measurement images based on statistics of the pixel values ​​of the measurement images and statistics of the pixel values ​​of the corresponding reconstructed images. The image modification part modifies the pixel values ​​of the measurement images by subtracting from the pixel value of each pixel of the respective measurement images a difference between a mean value of the pixel values ​​of the pixels contained in the measurement images and a mean value of the pixel values ​​of the pixels contained in the corresponding reconstructed images.

[0011] The modeling part can identify the modeled parameters based on the measurement images whose pixel values ​​have been changed by the image modification part.

[0012] The image processing device may also include a geometry identification part which identifies a geometry of the object to be measured using the measurement images modified by the image modification part.

[0013] The image acquisition unit can capture a multitude of different measurement images to which identifiers have been added that identify the recording sequence of the measurement images.

[0014] The image modification part changes pixel values ​​of the measurement images by subtracting from the pixel value of each pixel of the respective measurement images a difference between an average of the pixel values ​​of the pixels contained in the measurement images and an average of the pixel values ​​of the pixels contained in the corresponding reconstructed images.

[0015] The image processing device may also include a setup control unit or setup regulation unit which controls or regulates a projection device in such a way that it projects the light with the projection patterns sequentially onto the object to be measured.

[0016] The setup control section or setup regulation section can control or regulate the projection device in such a way that it sequentially projects the light with the projection patterns which have sinusoidal patterns, while changing the phases of the sinusoidal patterns.

[0017] The image modification part can change pixel values ​​of the measurement images if the variation amounts of phases of pixels, identified from (i) a phase value contained in the modeled parameter for each pixel of the respective measurement images that have unchanged pixel values ​​and (ii) a phase value contained in the modeled parameter for each pixel of the respective measurement images that contain the changed pixel values, are greater than a specified limit for a number of pixels that is greater than or equal to a specified number.

[0018] The image acquisition part can determine whether the variation magnitudes of the phases, which correspond to a number of pixels greater than a specified number, are equal to or less than the specified variation limit, and the geometry identification part can identify the geometry of the object to be measured if the image modification part determines that the variation magnitudes of phases, which correspond to a number of pixels greater than the specified number, are equal to or less than the specified variation limit.

[0019] A program or computer program product according to the second aspect of this invention, which enables a computer to perform the following functions: acquiring a multitude of different measurement images, which are acquired by capturing a measurement object, which sequentially projects light with different projection patterns, the intensity of which changes according to phases determined by the coordinates of a projection image and whose relationships between the phase and the coordinate differ from one another; identifying, for each pixel, a modeled parameter which approximates a data sequence in which pixel values ​​corresponding to the respective measurement images are placed in a sequence of acquisition;Generating reconstructed images, which are images that correspond to the respective measurement images and are reconstructed with an approximate value for each pixel, identified based on the modeled parameter of each pixel; and modifying the pixel values ​​of the measurement images by subtracting from the pixel value of each pixel of the respective measurement images a difference between a mean value of the pixel values ​​of the pixels contained in the measurement images and a mean value of the pixel values ​​of the pixels contained in the corresponding reconstructed images.

[0020] An image processing system according to the third aspect comprises a projection device which sequentially projects light onto an object to be measured with different projection patterns, the intensity of which changes according to the coordinates of a projection image in specific phases and whose relationships between the phase and the coordinate differ from one another; a recording device which generates a multitude of different measurement images, which are captured by recording the object to be measured onto which the light is projected with the multitude of different projection patterns; and the image processing device which modifies pixel values ​​of the respective measurement images. IMPACT OF THE INVENTION

[0021] According to this invention, the effect achieved is a reduction in the influence of the external environment on the measurement image. BRIEF DESCRIPTION OF THE DRAWINGS Fig. Figure 1 shows an overview of an image processing system according to the embodiment. Fig. Figure 2 shows an overview of the process for reducing the influence of the external environment on a measurement image. Fig. Figure 3 shows a functional configuration of an image processing device according to the invention. Fig. 4A and Fig. Figures 4B each show an approximation function that approximates a data sequence. Fig. Figure 5 shows a process for identifying a modeled parameter of the approximation function that approximates a data sequence. Fig. Figure 6 is a flowchart showing the processing to reduce the influence of the external environment on the measurement image, which is performed by the image processing unit. DETAILED DESCRIPTION OF THE INVENTION< IDEAL DESIGN >

[0022] The overview of an image processing system S according to the embodiment is given with reference to Fig. 1 explained. Fig. Figure 1 shows an overview of an image processing system S according to the embodiment. The image processing system according to the embodiment is an image processing system for measuring the geometry of an object to be measured by capturing the object to be measured, onto which a projection image is projected.

[0023] The image processing system S comprises a projection device 1, a recording device 2, and an image processing device 3. The image processing device 3 is connected to the projection device 1 and the recording device 2 in a manner that enables communication. The projection device 1 is, for example, a liquid crystal projector. The projection device 1 projects light with a variety of different projection patterns (P). a 1, Pa 2, ..., P a N) onto the object to be measured. The projection device 1 projects, for example, light with sinusoidal patterns several times sequentially, while changing the phase of the sinusoidal patterns.

[0024] The recording device 2 is, for example, a digital camera with an image capture element of two million pixels. The recording device 2 generates a large number of different measurement images (P b 1, P b 2, ..., P b N) by sequentially recording the object to be measured, onto the light with the multitude of different projection patterns (P a 1, P a 2, ..., P a N) is projected. The recording device 2 transmits the recorded measurement images to the image processing device 3.

[0025] The measurement images acquired by the recording device 2 can be affected by fluctuations in the amount of light in the environment where the measurement of the object to be measured takes place, fluctuations in the luminous intensity of the light projected by the projection device 1, and the like. In such a case, the image processing device 3 reduces the influence of the external environment by performing image processing on a large number of measurement images.

[0026] The image processing unit 3 is, for example, a computer. The image processing unit 3 acquires the measurement images by controlling or regulating the projection unit 1 for the sequential projection of light with the multitude of projection patterns, and by controlling or regulating the recording unit 2 for the sequential generation of the measurement images by recording the object to be measured onto which the projection image is projected. The image processing unit 3 then reduces the influence of the external environment by performing image processing on the multitude of recorded measurement images.

[0027] The overview of the processing to reduce the influence of the external environment, which the image processing unit 3 performs on the measurement image, is given with reference to Fig. 2 explained. Fig. Figure 2 shows an overview of the processing to reduce the influence of the external environment on the measurement image. First, the image processing unit 3 acquires a large number of measurement images ((1) of the Fig. 2) The image processing unit 3 identifies a modeled parameter of an approximation function which approximates the pixel values ​​of each pixel of the acquired measurement images ((2) the Fig. 2) In the following, modeled parameters of approximation functions, which approximate pixel values ​​of the acquired measurement images, can be referred to as "modeled parameters for the measurement image". The image processing unit 3 generates a large number of reconstructed images (P c 1, P c 2, ..., P c N) using an approximate value for each pixel identified from the modeled parameter ((3) the Fig. 2).

[0028] The image processing unit 3 identifies a defect in each pixel of the measurement images (Pa 1, P d 2, ..., P a N) using the captured measurement images and the generated reconstructed images ((4) the Fig. 2) The image processing unit 3 generates a large number of measurement images, which show the changed pixel values ​​(P b 1', P b 2', ..., P b N') by changing the pixel value of each pixel of the measurement images using the identified error ((5) of the Fig. 2) The image processing unit 3 identifies modeled parameters for the measurement images that have the changed pixel values ​​((6) of the Fig. 2).

[0029] The image processing unit 3 identifies a variation amount between the modeled parameter for the acquired measurement images and the modeled parameter for the measurement images with the changed pixel values ​​((7) of the Fig. 2) The image processing unit 3 then determines whether the identified variation is equal to or less than a variation limit. The variation limit is an influence limit that corresponds to a convergence level acceptable to the image processing unit 3 for identifying the geometry of the object being measured. The influence limit can be determined beforehand through experimentation, taking into account the amount of light from the projection unit 1, the efficiency of the recording unit 2, the accuracy required to measure the geometry of the object, and similar factors.

[0030] If it is determined that the identified variation amount is greater than the variation limit, the image processing unit 3 generates the reconstructed images based on the modeled parameters for the measurement images with the changed pixel values ​​((8-1) of the Fig. 2) The image processing unit 3 identifies the error of each pixel of the respective measurement image with the modified pixel values ​​using (i) the measurement images with the modified pixel values ​​and (ii) the reconstructed images, which are newly generated. The image processing unit 3 regenerates a large number of measurement images whose pixel values ​​are again modified (P b 1'', P b 2'', ... P b N''), by modifying the pixel value of each pixel of the respective measurement images with the changed pixel values ​​using the identified error. The image processing unit 3 identifies a variation amount between the modeled parameters for the measurement images with the changed pixel values ​​(P b 1', P b 2', P b N') and modeled parameters for the measurement images with the again changed pixel values ​​(P b 1'', P b 2'', ... P b N'').

[0031] If it is determined that the amount of variation is greater than the limit of variation, the image processing unit 3 changes the pixel value 3 of each pixel in the respective measurement image. In this way, the image processing unit 3 can reduce the influence of the external environment on the multitude of measurement images. Furthermore, if it is determined that the amount of variation corresponding to all pixels of the measurement images has become, for example, equal to or less than the limit of variation, the image processing unit 3 stops changing the pixel values ​​and identifies the geometry of the object to be measured using the measurement images whose pixel values ​​were last changed ((8-2) of the Fig. 2) Since the image processing device 3 is able to identify the geometry of the object to be measured using the multitude of measurement images, where the influence of the external environment is reduced, the image processing device 3 can improve the measurement accuracy. It should be noted that the image processing device 3 can stop changing the pixel values ​​if it determines that the variation values ​​corresponding to a number of pixels greater than a predetermined number are equal to or less than the variation limit. < Configurations of the image processing device 3 according to the embodiment >

[0032] A functional configuration of the image processing device 3 according to the embodiment is described with reference to Fig. 3 described. Fig. Figure 3 shows the functional configuration of the image processing device 3 according to the embodiment. The image processing device 3 according to the embodiment includes a memory unit 31 and a control unit 32.

[0033] Memory section 31 contains a storage medium comprising read-only memory (ROM), working memory (RAM), and the like. Memory section 31 stores the multitude of measurement images, the modeled parameters, the multitude of reconstructed images, the error of each pixel in the respective measurement images, or the measurement images in which the pixel values ​​have been changed. Furthermore, memory section 31 stores programs to be executed by the control section 32.

[0034] The control unit 32 is a computing source containing processors, such as a central processing unit (CPU), which is not shown. By executing the programs stored in the memory unit 31, the control unit 32 implements the functions of a setup control unit 321, an image acquisition unit 322, a modeling unit 323, a reconstructed image generation unit 324, an image modification unit 325, and a geometry identification unit 326. The setup control unit 321 controls the projection unit 1 and the recording unit 2 by transmitting command information.

[0035] The setup control unit or setup regulation unit 321 controls or regulates the projection device 1 for projecting the projection light onto the object to be measured. In particular, the setup control unit or regulation unit 321 controls or regulates the control device 1 so that it sequentially projects light onto the object to be measured with a multitude of projection patterns, the light intensity of which changes according to the phases determined by the coordinates of a projection image, and whose relationship between the phase and the coordinate differs from one another. The setup control unit or regulation unit 321 controls or regulates the projection device 1 such that, for example, it sequentially projects the light with projection patterns generated by the projection image with the sinusoidal patterns, while the phases of the sinusoidal patterns are changed.For example, the phase differences between sequentially projected images are constant.

[0036] The setup control section or setup regulation section 321 can control or regulate the projection device 1 to project N projection images sequentially onto the object to be measured by changing the phases of the sinusoidal patterns N times. For example, if the phases of the sinusoidal patterns are changed N times, the setup control section or regulation section 321 controls or regulates the projection device 1 to change the phases of the sinusoidal patterns by δ i = 2πi / N (i = 1, 2, ..., N). When projecting the sinusoidal patterns, the setup control section or control section 321 performs the control or regulation such that the projected image with the sinusoidal patterns, which I(x, y) = l0sin(2πxT / M x +δ i) corresponds to the projection onto the object to be measured, where I(x, y) denotes the light intensity at the coordinates (x, y) of the projected image. It should be noted that M x denotes the number of pixels in the x-direction of the projection image, and T denotes the number of sinusoidal patterns contained in the projection image.

[0037] The setup control unit or setup regulation unit 321 controls or regulates the recording device 2 such that it generates the measurement image by recording the object to be measured, onto which the projection light is projected. In particular, the setup control unit or regulation unit 321 controls or regulates the recording device 2 such that it generates a multitude of different measurement images by sequentially recording the object to be measured, onto which light is projected with a multitude of projection patterns. The setup control unit or regulation unit 321 controls or regulates the recording device 2 such that it transmits the multitude of different measurement images to the image acquisition unit 322.The setup control unit or control unit 321 can control or regulate the recording unit 2 so that it transmits the different measurement images to the image acquisition unit 322, to which identifiers have been added that identify the recording sequence of the measurement images.

[0038] The image acquisition unit 322 captures a multitude of different measurement images. For example, the image acquisition unit 322 captures the different measurement images acquired by recording the object to be measured, onto which light is projected with projection patterns whose light intensity changes according to phases determined by the coordinates of the projection image. In particular, the image acquisition unit 322 captures the different measurement images acquired by sequentially recording the object to be measured, onto which light is projected with a projection pattern obtained by adding the multitude of different phase differences to the phases determined by the coordinates. The image acquisition unit 322 transmits the different measurement images it acquires to the modeling unit 323.

[0039] The modeling part 323 identifies, for each pixel, a modeled parameter of the approximation function that approximates the data sequence in which the pixel values ​​of the pixels corresponding to the measurement images are placed in the order of acquisition. The modeling part 323 can also identify modeled parameters based on the measurement images whose pixel values ​​have been modified by the image modification part 325.

[0040] A procedure of modeling part 323, which identifies the modeled parameters of the approximation function that approximates a data sequence, is described with reference to the Fig. 4A and Fig. 4B described. Fig. 4A and Fig. Figures 4B each show an approximation function that approximates a data sequence. Fig. 4A is a schematic diagram showing the measurement images placed in the order they were taken. Fig. Figure 4A shows the horizontal axis and the vertical axis the x-coordinate and y-coordinate, respectively, of the measurement image. The following explanation assumes that the image acquisition unit 322 captures six measurement images (P b 1, P b 2, P b 3, P b 4, P b 5 and P b 6) recorded. In Fig. 4A designates I i the pixel value of the pixel, which is determined by the x-coordinate and the y-coordinate of the i-th measured image.

[0041] Fig. 4B is a schematic diagram showing the data sequences where the pixel values ​​I i The pixels, determined by the coordinates (x, y), are placed in the order they were captured. Fig. 4A indicates the horizontal axis as the acquisition sequence of the measurement images, and the vertical axis indicates the pixel value of the pixel. The pixel value I iThe (x, y) of pixels, determined from the coordinates (x, y) of the i-th measurement image, are plotted with black circles. The function f(x, y), indicated by the dashed line, is the approximation function that approximates the plotted data sequence.

[0042] Each data element in the plotted data sequence contains an error at the time of measurement. Therefore, the value of each data element and the approximate value, which is identified using the approximation function corresponding to the data sequence, do not necessarily coincide. The approximation function can be fitted to the plotted data sequence by identifying the modeled parameter, where the sum of the squares of an error ε i (x, y) becomes minimal, where a difference between each data element and the approximate value corresponding to each data element represents the error at the time of measurement ε. i(x, y) is defined.

[0043] The processing for identifying a modeled parameter by the modeling part 323 is described with reference to Fig. 5 described. Fig. Figure 5 shows the process for identifying the modeled parameter of an approximation function that approximates a data sequence. Similar to Fig. 4B, is Fig. Figure 5 is a schematic diagram showing the data sequences, where the pixel values ​​are placed in the order they were acquired. Fig. 5. The function f, denoted by the solid line, is a reference approximation function. An amplitude V denotes an amplitude of the approximation function, a bias I'(x,y) denotes a midpoint of the amplitude V, and an offset A denotes an offset of the approximation function. The function f(x, y), denoted by the dashed line, is the approximation function of the data sequence in which the pixel values ​​of the pixels of the coordinates (x, y) corresponding to the respective measurement images are placed in the order of acquisition, and deviates from the reference approximation function by the phase φ, the amplitude ΔV, and the offset ΔA. Furthermore, errors ε denote i (i = 1 to 6) each a distance from each data element to the approximate value corresponding to each data element.

[0044] The approximation function is represented by equation (1). Ii(x,y)=I'(x,y)+V(x,y)cos[ϕ(x,y)+k(x,y)+δi]

[0045] Here, the phase φ(x, y) is a variable that denotes the phase shift of the approximation function of the data sequence where the pixel values ​​of the pixels are arranged at the coordinates (x, y), and is also a variable that corresponds to the height of the object to be measured. In equation (1), I'(x,y) denotes a bias voltage and V(x, y) denotes the amplitude of the approximation function. i denotes the magnitude of the phase change, which is assigned to the i-th measurement image. For δ i As shown above, δ i = 2πi / N (i = 1, 2, ..., N) and in this embodiment N = 6. The phase k(x, y) is a variable related to the phase of the sinusoidal patterns contained in the projection pattern. In the following explanation, the sum of the phase φ(x, y) and the phase k(x, y) can be referred to as a phase value. Furthermore, equation (1) is transformed so that it is represented by equation (2). Ii(x,y)=I'(x,y)+V(x,y)cos[ϕ(x,y)+k(x,y)]cos(δi)−V(x,y)sin[ϕ(x,y)+k(x,y)]sin(δi)

[0046] In this case, each element of equation (2) is replaced as in equations (3) to (5). a0(x,y)=I'(x,y) a1(x,y)=V(x,y)cos[ϕ(x,y)+k(x,y)] a2(x,y)=−V(x,y)sin[ϕ(x,y)+k(x,y)]

[0047] Equation (2) is represented by equation (6) using equations (3) to (5). Ii(x,y)=a0(x,y)+a1(x,y)cos(δi)+a2(x,y)sin(δi)

[0048] The pixel value of the coordinates (x, y) is represented by equation (7) by expressing equation (6) for the respective measurement images. I1(x,y)=ao(x,y)+a1(x,y)cos(δ1)+a2(x,y)sin(δ1)I2(x,y)=ao(x,y)+a1(x,y) cos(δ2)+a2(x,y)sin(δ2)⋮IN(x,y)=ao(x,y)+a1(x,y)cos(δN)+a2(x,y)sin(δN)

[0049] Equation (7) is represented using a matrix by equation (8). [I1I2⋮IN]=[1cos(δ1)sin(δ1)1cos(δ2)sin(δ2)⋮⋮⋮1cos(δN)sin(δN)][aoa1a2]

[0050] Suppose that equation (8) I = Xa. In such a case, the vector 'a' which is optimal with respect to minimizing the 2-norm of I-Xa is known as a solution of the least squares method and can be expressed as a = (X T X) -1 X T I can be expressed. Therefore, X T X is represented by equation (9). Furthermore, X T I is represented by equation (10). Unless otherwise stated, in the following equation Σ without a subscript, is the total sum from I = 1 to N. XTX=[N∑cos(δi)∑sin(δi)∑cos(δi)∑cos2(δi)∑cos(δi)sin(δi)∑sin(δi)∑cos(δi)sin(δi)∑sin2(δi)] XTI(x,y,δi)=[∑Ii∑Iicos(δi)∑Iisin(δi)]

[0051] If δ iIf = 2πi / N (i = 1, 2, ..., N) holds, then the non-diagonal matrix in equation (9) is equal to zero, thus the vector 'a' is represented by equation (11). [a0a1a2]=[N000N / 2000N / 2]−1[∑Ii∑Iicos(δi)∑Iisin(δi)]

[0052] Thus, a0, a1 and a2 are represented by equations (12) (13) and (14). a0=1N∑Ii a1=2N∑Iicos(δi) a2=2N∑Iisin(δi)

[0053] Using equations (12) to (14), the modeling part 323 identifies the pixel phase value (the sum of phase φ(x, y) and phase k(x, y)), the amplitude V of the approximation function, and the offset A, which are the modeled parameters. The image pixel value is represented by equation (15), the amplitude V of the approximation function is represented by equation (16), and the offset A is represented by equation (17). The modeling part 323 passes the identified modeled parameters to the image generation part 324. ϕ(x,y)+k(x,y)=arctan(−∑i=1NIisin(δi)∑i=1NIicos(δi)) V(x,y)=2N[∑iIicos(δi)]2+[∑iIisin(δi)]2 A(x,y)=1N∑i=1NIi−V

[0054] The image generation unit 324 identifies an approximate value for each pixel using the modeled parameter for each pixel identified by the modeling unit 323. The image generation unit 324 then generates the reconstructed images, which are reconstructed with the identified approximate value for each pixel according to the respective measurement images. The image generation unit 324 then passes the generated reconstructed images to the image modification unit 325.

[0055] Image Modification Section 325 modifies the pixel values ​​of the multitude of measurement images based on statistics of the pixel values ​​of the respective measurement images and based on statistics of the pixel values ​​of the corresponding reconstructed images. For example, Image Modification Section 325 subtracts from the pixel value of each pixel of the respective measurement images a difference between the mean pixel values ​​of the pixels contained in the measurement images and the mean pixel values ​​of the pixels contained in the corresponding reconstructed images. The processing for modifying the pixel value of each pixel of the measurement images by Image Modification Section 325 is described below.

[0056] First, the image modification section 325 identifies an average of the pixel values ​​of the pixels contained in the i-th measurement image. Then, the image modification section 325 identifies an average of the pixel values ​​of the pixels contained in the reconstructed image corresponding to the i-th measurement image. Then, the image modification section 325 identifies an error ε. i of the i-th measurement image, a difference between the mean pixel values ​​of the pixels contained in the i-th measurement image and the mean pixel values ​​of the pixels contained in the reconstructed image corresponding to the i-th measurement image. The error ε i is represented by equation (18). 1M∑x,y{Ii(x,y)−IiR(x,y)}=εi

[0057] In equation (18) M denotes the total number of pixels contained in the measurement images, and I i R(x, y) denotes the pixel value of pixels in the reconstructed image corresponding to the i-th measurement image. It should be noted that the total number of pixels contained in the reconstructed image is equal to the total number of pixels contained in the measurement image.

[0058] Image modification part 325 changes the pixel value by subtracting the identified error ε. i of the pixel value l i (x,y) of each pixel contained in the i-th measurement image. The changed pixel value is represented by equation (19). Ii(x,y)−εi=Ii'(x,y)

[0059] Similarly, the image modification part 325 changes the pixel values ​​of the pixels contained in the respective captured measurement images.

[0060] The image modification unit 325 sends the respective measurement images with the modified pixel values ​​to the modeling unit 323. The modeling unit 323 identifies a phase value contained in the modeled parameter for each pixel of the respective measurement images with the modified pixel values. The modeling unit 323 then sends the identified phase value to the image modification unit 325.

[0061] Image change part 325 identifies a phase variation based on (i) a phase value contained in the modeled parameter for each pixel of the respective measurement images with unchanged pixel values ​​and (ii) a phase value contained in the modeled parameter for each pixel of the respective measurement images with the changed pixel values. For example, image change part 325 identifies the phase variation as the difference between the phase value contained in the modeled parameter for each pixel of the respective measurement images with unchanged pixel values ​​and the phase value contained in the modeled parameter for each pixel of the respective measurement images with the changed pixel values.

[0062] The image modification section 325 determines whether the phase variation is equal to or less than the specified variation limit. If it is determined that the identified phase variation for a number of pixels greater than or equal to a specified number is greater than the specified variation limit, the image modification section 325 modifies the pixel values ​​of the respective measurement images with the modified pixel values. The specified value is, for example, one. If it is determined that the phase variation is equal to or less than the specified variation limit, the image modification section 325 determines that the phase value has converged and does not modify the pixel values ​​of the respective measurement images, instead passing the measurement images with the modified pixel values ​​to the geometry identification section 326.In this way, the image modification part 325 can reduce the variation amount of the phase so that it is equal to or less than the variation limit which is acceptable for the geometry identification part 326 to identify the geometry of the object to be measured, and thus the image modification part 325 can reduce the influence of the external environment on the measurement image.

[0063] The geometry identification part 326 identifies the geometry of the object to be measured using the multitude of measurement images with the pixel values ​​modified by the image modification part 325. For example, the geometry identification part 326 identifies the geometry of the object to be measured based on the phase value of the modeled parameters for the measurement images with the modified pixel values. In particular, the geometry identification part 326 identifies a phase φ(x, y) related to the height of the object to be measured by subtracting the phase k(x, y) that corresponds in equation (15) to the projection patterns projected onto the object to be measured.Since the geometry identification part 326 is able to identify the geometry of the object to be measured using the multitude of measurement images in which the influence of the external environment is reduced, the geometry identification part 326 as such can improve the measurement accuracy of the object to be measured.

[0064] The processing to reduce the influence of the external environment on the object to be measured, which is carried out by the image processing unit 3, is described below with reference to Fig. 6 described. Fig.Figure 6 is a flowchart illustrating the processing steps performed by the image processing unit 3 to reduce the influence of the external environment on the measurement image. First, the setup control unit 321 projects light with a variety of projection patterns by controlling the projection unit 1 (step S1). The setup control unit 321 then generates the measurement images by controlling the acquisition unit 2 to capture the object to be measured, onto which the light with the variety of projection patterns is projected. The image acquisition unit 322 captures the measurement images generated by the acquisition unit 2 (step S2).

[0065] The modeling part 323 then identifies modeled parameters for the acquired measurement images. The image generation part 324 generates a reconstructed image using the approximate value of each pixel, which has been identified based on the identified modeled parameters (step S3). The image modification part 325 then identifies a difference between (i) the mean pixel values ​​of the pixels contained in the measurement images and (ii) the mean pixel values ​​of the pixels contained in the reconstructed images corresponding to the measurement images as an error (step S4).

[0066] Then, the image modification section 325 changes the pixel values ​​of the pixels contained in the respective measurement images based on the identified error (step S5). Next, the modeling section 323 identifies modeled parameters for the measurement images with the changed pixel values ​​(step S6). The image modification section 325 determines whether a variation in the phase, which is a difference between (i) the phase value of the measurement images with unchanged pixel values ​​and (ii) a phase value of the measurement images with the changed pixel values, is equal to or less than the variation limit (step S7).

[0067] If the image modification part 325 determines that the phase variation is greater than the variation limit (No in step S7), the process returns to step S3, and the image modification part 325 generates the reconstructed images whose pixel values ​​have been modified by the reconstructed image generation part 324, based on the measurement images. If the image modification part 325 determines that the phase variation is equal to or less than the variation limit (Yes in step S7), the geometry identification part 326 identifies the geometry of the object to be measured based on the measurement images with the modified pixel values ​​(step S8). [Effects of the invention]

[0068] As described above, the modeling section 323 of the image processing unit 3 identifies the modeled parameters of each pixel in the respective measurement images, and the reconstructed image generation section 324 generates the reconstructed images based on the modeled parameter. Then, the image modification section 325 modifies the pixel value of each pixel in the respective measurement images based on the measurement images and the corresponding reconstructed images. In this way, the image modification section 325 reduces the influence on the measurement images caused by fluctuations in the amount of light in the external environment, fluctuations in the intensity of the projected light, or similar factors.

[0069] Furthermore, the image modification section 325 can reduce the influence of the external environment on the multitude of measurement images by changing the pixel value of each pixel in the respective measurement images several times. Since the geometry identification section 326 can identify the geometry of the object to be measured using the measurement images where the influence of the external environment is reduced, the geometry identification section 326 can thus improve the measurement accuracy.

[0070] This invention is explained starting from the exemplary embodiments. The technical scope of this invention is not limited to that described in the embodiments above, and it is possible to make various changes and modifications within the scope of the invention. For example, the specific embodiments of the distribution and integration of the device are not limited to the embodiments described above; all or part of them can be configured with any unit, whether functionally or physically separate or integrated. Furthermore, new exemplary embodiments generated by any combination thereof are included in the exemplary embodiments of this invention. Moreover, the effects of the new exemplary embodiments brought about by these combinations are the same as the effects of the original exemplary embodiments. (Variation example 1)

[0071] In the above explanation, the image processing unit 3 has the setup control unit 321 and controls the projection unit 1 and the recording unit 2, but the image processing unit 3 does not necessarily have to have the setup control unit 321. For example, the image processing system S may have a system control unit in addition to the projection unit 1, the recording unit 2, and the image processing unit 3, and may generate a measurement image by having the system control unit control the recording unit 2 and the image processing unit 3. (Variation example 2)

[0072] In the above explanation, the image processing device 3 acquires the multitude of measurement images from the recording device 2, but methods for acquiring the multitude of measurement images are not limited to this. For example, the image processing device 3 may include a communication module and may acquire a multitude of measurement images from an external device via the communication module. In particular, the image processing device 3 is connected to a local area network (LAN) via the communication module and acquires the multitude of measurement images from another external device connected to the LAN.

[0073] The image processing unit 3 can include an image input unit and can acquire a large number of measurement images from a storage medium via the image input unit. In particular, the image input unit includes a card interface, a USB interface, or the like. The image input unit acquires the large number of measurement images from the storage medium connected to this interface. The storage medium is, for example, an SD card or a USB flash drive. [Description of reference symbols] 1 Projection device 2. Receiving facility 3 Image processing equipment 31 Storage section 32 Control section or regulating section 321 Setup control part or setup control part 322 Image capture section 323 Model part 324 Generation unit for reconstructed images 325 Image modification section 326 Geometry identification part

Claims

[1] Image processing device (3), comprising: an image acquisition part (322) which captures a multitude of different measurement images which have been captured by recording an object to be measured, onto which light with different projection patterns, the intensity of which changes according to phases determined by the coordinates of a projection image and whose relationships between the phase and the coordinate are different from each other, is sequentially projected; a model part (323) which identifies for each pixel a modeled parameter which approximates a data sequence in which pixel values ​​from the respective measurement images are placed in a recording order; a generating part (324) for reconstructed images, which generates reconstructed images that correspond to the respective measurement images and are reconstructed with approximate values ​​of the respective pixels, which are identified based on the modeled parameter of each pixel; and an image modification part (325) which modifies the pixel values ​​of the measurement images based on statistics of the pixel values ​​of the measurement images and statistics of the pixel values ​​of the corresponding reconstructed images, wherein the image modification part (325) modifies pixel values ​​of the measurement images by subtracting from the pixel value of each pixel of the respective measurement images a difference between a mean value of the pixel values ​​of the pixels contained in the measurement images and a mean value of the pixel values ​​of the pixels contained in the corresponding reconstructed images. [2] Image processing device (3) according to claim 1, wherein the modeling part (323) identifies the modeled parameter starting from the measurement images whose pixel values ​​are changed by the image modification part (325). [3] Image processing device (3) according to claim 1 or 2, further comprising a geometry identification part (326) which identifies a geometry of the object to be measured using the measurement images modified by the image modification part (325). [4] Image processing device (3) according to one of claims 1 to 3, wherein the image acquisition part (322) acquires the plurality of different measurement images to which identifiers are added which identify the acquisition sequence of the measurement images. [5] Image processing device (3) according to any one of claims 1 to 4, further comprising: a setup control unit or setup control unit (321) which controls or regulates a projection unit (1) such that it sequentially projects the light with the projection patterns onto the object to be measured. [6] Image processing device (3) according to claim 5, wherein the device control part or device control part (321) controls or regulates the projection device (1) such that it sequentially projects the light with the projection patterns with sinusoidal patterns while changing the phases of the sinusoidal patterns. [7] Image processing device (3) according to one of claims 1 to 6, wherein the image modification part (325) changes pixel values ​​of the measurement images when variation amounts of phases of pixels, which are identified starting from (i) a phase value contained in the modeled parameter for each pixel of the respective measurement images that have unchanged pixel values, and (ii) a phase value contained in the modeled parameter for each pixel of the respective measurement images that contain the changed pixel values, are greater than a predetermined limit for a number of pixels that is greater than or equal to a predetermined number. [8] Image processing device (3) according to claim 7, wherein the image modification part (325) determines whether the variation amounts of the phases corresponding to a number of pixels greater than a predetermined number are equal to or less than the predetermined variation limit, and the geometry identification part (326) identifies the geometry of the object to be measured when the image modification part (325) determines that the variation amounts of phases corresponding to a number of pixels greater than a predetermined number are equal to or less than the predetermined variation limit. [9] Computer program product comprising computer-readable instructions which, when loaded onto and executed on a computer, cause the computer to perform the following functions: Capturing a large number of different measurement images, which are captured by recording an object to be measured, onto which light with different projection patterns is sequentially projected, the intensity of which changes according to phases determined by the coordinates of a projection image and whose relationships between the phase and the coordinate differ from each other; Identify, for each pixel, a modeled parameter which approximates a data sequence where pixel values ​​from the respective measurement images are placed in a recording order; Generating reconstructed images, which are images that correspond to the respective measurement images and are reconstructed with an approximate value for each pixel, which has been identified based on the modeled parameter of each pixel; and Changing the pixel values ​​of the measurement images by subtracting from the pixel value of each pixel of the respective measurement images a difference between an average of the pixel values ​​of the pixels contained in the measurement images and an average of the pixel values ​​of the pixels contained in the corresponding reconstructed images. [10] Image processing system, comprising: a projection device (1) which sequentially projects light onto an object to be measured with different projection patterns, the intensity of which changes according to phases determined by the coordinates of a projection image and the relationships between the phase and the coordinate differ from one another; a recording device (2) which generates a multitude of different measurement images, which are acquired by recording the object to be measured, onto which the light is projected with the multitude of different projection patterns; and the image processing device (3) according to one of claims 1 to 8, which changes pixel values ​​of the respective measurement images.

Images