Method for creating a 3D table, a measurement method using a 3D table, and a measuring device thereof.

Abstract

To reduce the memory capacity required to store the entire spatial table necessary for three-dimensional measurement using pattern projection, data compression is required. Furthermore, it is necessary to minimize degradation from the original table data when using the compressed table, enabling highly accurate measurements. [Solution] The phase difference with respect to the phase value at the reference position is calculated, and a full spatial table of phase differences and coordinate values ​​is created. By using this phase difference, the phase difference value of any pixel will be close to the phase difference values ​​of the surrounding pixels, making it possible to compress the full spatial table into a format that is easy to compress. The full spatial table created using this phase difference is then compressed by a predetermined magnification in each of the i, j, and m directions, thereby reducing the data size of the full spatial table.

Description

[Technical Field]

[0001] This invention relates to a method for creating a three-dimensional table, a measurement method using a three-dimensional table, and a measurement apparatus therefor. [Background technology]

[0002] Traditionally, research on non-contact 3D shape measurement using light has been actively pursued. There are several types of 3D shape measurement methods, including the point light scanning method, which measures the shape by irradiating the object with a laser spot, and the light section method, which shortens measurement time by converting the spot light into a line and expanding the area that can be illuminated on the object. These methods measure the 3D shape of an object using the principle of triangulation and enable highly accurate measurements. On the other hand, they have problems such as requiring measurement while scanning and the complexity of the equipment configuration used for shape measurement. On the other hand, many pattern projection methods have been developed that project a two-dimensional pattern onto the object to be measured and measure the object's height from the captured pattern image. This method has the advantage of high spatial resolution, fast measurement speed, and relatively low cost because it can use all of the camera's pixels for measurement. The pattern projection method is a grid projection method that determines the three-dimensional shape of an object by projecting a grid pattern onto the object to be measured and analyzing the phase of the grid pattern projected onto the object. In this method, the phase of the projected grid pattern is shifted, and an image is taken for each shift. The phase value is determined from the change in the brightness value of each pixel in the captured grid image, and the shape of the object is measured by converting the phase value to a coordinate value. By converting from multiple brightness values ​​to a phase value, a resolution greater than the quantization bit count of the camera can be obtained. In addition, the influence of the surface color of the object and ambient light can be reduced, enabling high-precision measurement. Furthermore, unlike point light scanning and light section methods, scanning is not required, making it relatively easy to implement. In pattern projection, a method for rapidly acquiring the three-dimensional coordinates of an object's three-dimensional shape is known, which involves creating a relationship table between phase value and height for each pixel in advance, and then converting the phase value acquired from the object to three-dimensional coordinates by referring to the relationship table during measurement (Patent Documents 1 and 2). This method eliminates the influence of image distortion caused by camera lens distortion by acquiring reference plane information during calibration. Patent Document 2 describes a feature-type whole-space table method in which a table is created in three dimensions using three phase values ​​obtained by using three projection devices for grid pattern projection. While this describes a three-dimensional measurement method using a combination of three or more features, it does not describe a method for performing arbitrary compression and expansion using phase difference. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Patent No. 5854544 [Patent Document 2] Patent No. 6308637 [Overview of the project] [Problems that the invention aims to solve]

[0004] In 3D shape measurement using a full-space table method, the size of the table data increases in proportion to the camera resolution and the number of elements that determine how finely the table is created. For example, if 3D shape measurement is performed with 1000 divisions in the height direction, each pixel will have 1000 elements, requiring 12K bytes of memory per pixel. For a 2000x2000 pixel image, the size of the full-space table will be 12K x 4M = 48GB. Table data is necessary for 3D shape measurement, and this large-capacity table must be stored on the analysis computer. This presented a challenge in integrating the method for creating a full-space table into small personal computers with limited memory capacity. Therefore, data compression is required to reduce the memory capacity needed to store the entire spatial table necessary for three-dimensional measurement using pattern projection. Furthermore, it is necessary to minimize degradation from the original table data when using the compressed table, enabling highly accurate measurements. [Means for solving the problem]

[0005] To solve the above problems, instead of creating a full spatial table of phase values ​​and coordinate values ​​as in the conventional method, the present invention calculates the phase difference with respect to the phase value at the reference position and creates a full spatial table of phase differences and coordinate values. By using this phase difference, the phase difference value of any pixel will be close to the phase difference values ​​of the surrounding pixels, making it possible to compress the full spatial table into a format that is easy to compress. By compressing the full spatial table created using the phase difference using pre-set magnifications in the i, j, and m directions, the data size of the full spatial table is reduced. [Effects of the Invention]

[0006] This invention makes it possible to reduce the memory capacity required to store the entire spatial table necessary for three-dimensional measurement using the pattern projection method. [Brief explanation of the drawing]

[0007] [Figure 1] This diagram shows the phase relationship between the grid and the luminance distribution. [Figure 2] This is a schematic diagram of the reference plane image in the z direction. [Figure 3] This figure shows the relationship between phase and height obtained from reference plane imaging. [Figure 4] This is a diagram illustrating the creation of table elements. [Figure 5] This figure shows the created phase-height relationship table. [Figure 6] This is a diagram showing the relationship between the z and x coordinates obtained from a reference plane image. [Figure 7] This diagram shows the relationship between the distance between the light source and the lattice glass and the projected lattice. [Figure 8]This diagram shows the relationship between the phase value and the z-coordinate. [Figure 9] This figure shows the relationship between two phase values ​​and a region at an arbitrary pixel. [Figure 10] This is a diagram showing the area number search table. [Figure 11] This diagram illustrates the calculation procedure using the area number search table. [Figure 12] This is a diagram illustrating the measurement depth expansion method. [Figure 13] This is a diagram showing a grid image. [Figure 14] This figure shows the phase after phase connection and the phase before phase connection. [Figure 15] This diagram shows the relationship between the phase value or phase difference and the z-direction. [Figure 16] This diagram shows the relationship between two phase-connected phase values ​​obtained as a result of interpolating two different pitches. [Figure 17] This figure shows the relationship between the phase difference, region, and z-direction in the range of 0 to 2π (rad). [Figure 18] This figure shows the relationship between the phase difference and the interpolated region number. [Figure 19] This figure shows the improved area number search table. [Figure 20] This diagram shows the relationship between the phase difference and x. [Figure 21] This diagram shows the relationship between the phase difference and y. [Figure 22] This diagram illustrates the relationship between phase difference and z. [Figure 23] This is a schematic diagram of the measurement procedure using the area number search table. [Figure 24] This is a diagram showing the results of the measurements taken on the doll. [Figure 25] This diagram shows the table compression procedure. [Figure 26] This figure shows the procedure for compressing a table at an arbitrary magnification. [Figure 27] This is a schematic diagram of the unfolding of a compressed table in the I-axis direction. [Figure 28]This is a schematic diagram of coordinate calculation when ri=rj=rm=0. [Figure 29] This is a schematic diagram of coordinate calculation when ri≠0 and rj=rm=0. [Figure 30] This is a schematic diagram of coordinate calculation when ri≠0, rj≠0, and rm=0. [Figure 31] This is a schematic diagram of coordinate calculation when ri≠0, rj≠0, and rm≠0. [Figure 32] This is a diagram showing the configuration of the experimental apparatus. [Figure 33] This figure shows the case where the table is not compressed. [Figure 34] This figure shows the case where compression is doubled in the i-direction. [Figure 35] This figure shows the case where compression is doubled in the j direction. [Modes for carrying out the invention]

[0008] Embodiments of the present invention will be described below with reference to the drawings. Measurement principle First, we will describe the three-dimensional shape measurement techniques used in this invention: the phase shift method, the light source switching phase shift method, and the full-space table method. The following describes the phase shift method used for phase analysis of captured grid images, the creation of tables for the correspondence between z-coordinates and phase values, the creation of tables for the correspondence between xy-coordinates and phase values, and the measurement depth expansion method. Furthermore, the vertical direction of the reference plane used to create the full spatial table is defined as the z-coordinate (height / depth) direction, and the horizontal and vertical directions of the reference plane are defined as the x-direction and y-direction, respectively.

[0009] In this invention, the phase shift method is used as a technique to analyze a projection grid and derive its phase. This technique requires multiple grid images in which the phase changes at equal intervals. Therefore, during measurement, a cosine wave grid pattern is projected, and the projected grid pattern is shifted while capturing the projected grid pattern for each shift. Then, the phase distribution of the projected grid is determined from the brightness change at each pixel in these multiple captured images. Various techniques can be used to determine the phase distribution, such as the phase shift method, the sampling moiré method, and the Fourier transform method. Determining the phase distribution is equivalent to determining the phase value θ(i,j) for each pixel. In the case of a grid image, the phase can be represented by a real number. It can also be represented as a repetition of a 2π period from 0 to 2π. Figure 1(a) shows the brightness distribution of the grid image, and Figure 1(b) shows the distribution of the phase value (θ(i,j)) as a repetition from 0 to 2π.

[0010] Table creation between phase and z-coordinate using a full-space tableization method Photography of the reference plane To perform analysis using the whole-space table method, it is necessary to determine the correspondence between phase and coordinates using a reference plane, as shown in Figure 2(a). The x-axis is defined as the direction orthogonal to the cosine wave-shaped grid pattern projected onto the reference plane. At this time, the camera and projector are positioned such that parallax occurs in the x-axis direction. First, the grid pattern projected from the projector is shifted at equal intervals, and multiple projected grid patterns are photographed for one period. By applying the phase shift method to the grid image obtained for one period in this way, the phase distribution of the reference plane at the initial point z0 is determined. The projector is a device that projects pattern light onto an object using the method described in paragraph "0009". Next, as shown in Figure 2(b), the position of the reference plane is moved by Δz in the z-axis direction, and a grid pattern is projected at each z position to determine the phase distribution of the reference plane from the captured images. At this time, the height of the reference plane is set from the lowest to the highest z iLet i = 0, 1, 2, ..., n-1. Here, n is an integer representing the number of times the reference plane has been moved. This operation is repeated until a predetermined measurement range is reached, and the phase value at each z-coordinate of each pixel in the captured image is determined. In the figure, solid lines indicate areas of high brightness in the projection grid, and dashed lines indicate areas of low brightness. Through the steps outlined so far, we can determine the phase value at each z-coordinate for each pixel of the camera. The relationship between the z-coordinate value and the phase value is shown in Figure 3. Using this, we create a table of phase and height z for all pixels. The elements from 0 to k in Figure 3 are used to create the table. Here, k represents the element whose phase value exceeds the first element after completing one cycle from the 0th element.

[0011] Create a table Next, a conversion table between the z-coordinate and the phase value is created. An overview of the conversion table creation is shown in Figure 4. The elements of the table are arranged so that the spacing Δθ between adjacent elements remains constant. The elements are numbered 0, 1, 2, ..., m-1 from smallest to largest phase value, and each phase value is θ. i Let i = 0, 1, 2, ..., m-1. Here, m is a positive integer representing the total number of elements in the table. There is a relationship between m and Δθ given by equation 1.

[0012]

number

[0013]

Equation

[0014] Table creation between phase and xy - coordinates by the full - space tabulation method Shooting of the reference plane For the x - direction and y - direction, coordinates are calculated from the images obtained by shooting the reference plane. This is shown in Fig. 5(a). However, at this time, nothing is projected onto the reference plane. A lattice image is displayed on the liquid - crystal monitor used as the reference plane and photographed by the camera. Then, while shifting the phase of the displayed lattice image by a fixed interval in the x - and y - directions on the liquid - crystal monitor, shooting is performed in order. This is repeated for one cycle of the displayed lattice to obtain phase - shifted images in the x - and y - directions respectively. The phase - shift method is applied to the obtained images to obtain the phase distribution. And by connecting the phases based on the correspondence between the range photographed by the camera and the range being displayed, a one - to - one correspondence is made between the phase of the displayed lattice and the x, y coordinates. That is, considering the x - coordinate here, the x - coordinate x(i,j) at an arbitrary pixel point (i,j) is the phase value φ x (i,j), the length p of one cycle of the lattice x Can be calculated from the following three equations.

[0015] [[ID=二十九]]

Equation

[0016] Next, move the position of the reference plane by Δz in the z - direction as shown in Fig. 5(b), and analyze the displayed lattice images photographed at each position to obtain the x - coordinates and x - coordinates pixel by pixel. From the above operations,the correspondence relationships between the z - coordinate, x - coordinate, and y - coordinate for each pixel of the camera can be obtained. This is shown in Fig. 6. From this correspondence relationship and the previously obtained correspondence relationship between the phase and the z - coordinate, a table of the phase and the x - coordinate and y - coordinate is created. In this explanation, we used a liquid crystal monitor as an example to describe how to create tables for the x and y directions. However, any reference surface that can provide x and y coordinates, such as a reference surface with a grid pattern or dot pattern, can be used instead of a liquid crystal monitor.

[0017] Measurement depth expansion method The light source switching phase shift method is a type of grating projection method. By placing a grating glass or cylindrical lens array between the light source and the object, the shadow of the grating glass can be projected onto the object. Figure 7 shows how two light sources placed at different distances from the grating glass project different grating patterns with different pitches. The solid lines emanating from each light source represent the center of the bright lines of the projected grating (the position where the phase is 0). The pitch of the grating is inversely proportional to the distance between the light source and the grating. In other words, since light source B is closer to the grating glass than light source A, the pitch of the projected grating by light source B is larger. The distance between light source A and the lattice glass is α. A The distance between light source B and the lattice glass is α. B Let b be the distance between the lattice and z=z0, and P0 be the pitch of the lattice in the projected lattice glass. Then, P is the pitch of the projected lattice pattern at the position z=z0. A and P B This is shown in equations 4 and 5.

[0018]

number

[0019]

number

[0020] The left diagram in Figure 8 shows the relative positions of the projector and camera. Assume that any single pixel of the camera is capturing a line L. In this case, line L will cross several projection grids. Assume that the phase at position z=z0 is θ0. If we observe the phase value along line L from position z=0 toward the camera, as shown in the right diagram in Figure 8, the phase increases as the Z coordinate increases, but returns to 0 when it reaches 2π, and then increases again. As a result, z R1 ,z R2 ,z R3 The phase value is θ0 in this case as well. The relationship between the phase value and the z coordinate is shown in the right diagram of Figure 8. The four solid lines extending diagonally upward in the right diagram of Figure 8 correspond to line L in the left diagram of Figure 8. The region is divided with respect to the Z direction such that Range 1 is the range from the position z=z0 to the next position where the phase is the same, and Range 2 is the range to the next position where the phase is the same. Since each region on line L has the same phase value, it is not possible to distinguish which region the object being measured is located in from the phase of the projected grid. However, by using phase values ​​obtained from grids with different pitches, the region can be identified.

[0021] A region number lookup table is created in advance using two phase values ​​obtained from projection grids with different pitches, and the region is identified by referring to the region number lookup table during measurement. The phase values ​​necessary for creating the region number lookup table can be obtained simultaneously during the calibration of the full-space tableization method. Figure 9 shows a graph plotting two phase values ​​for an arbitrary pixel during calibration. Circles represent region 1, crosses represent region 2, and black circles represent region 3. The plotted point cloud data also contains information about the measurement region, and the region number lookup table shown in Figure 10 can be created by interpolating the blank areas on the graph. During measurement, the phase θ of the target to be measured is plotted in the region number lookup table. A and θ B By substituting this, we can obtain point S on the graph. num The position is determined, and the region number can be obtained. The measurement flow is shown in Figure 11. First, as in 1.3 and 1.4, a table of grid A is created for each measurement area during calibration. The measurement depth expansion method also creates a range number search table for each pixel. During measurement, two phase distributions θ are first measured. a and θ b Substitute this into the area number search table, and find the area number s num We obtain the phase θ of lattice A. a and area number s num By substituting this into the Grid table group, the coordinate z ab Calculate.

[0022] Table compression method using phase difference Creating a correspondence table between phase difference and coordinate values As mentioned above, the phase value repeats in a 2π period, so a region occurs where the phase value changes abruptly at 2π periods. For example, as shown in Figure 1, an abrupt change occurs at the transition between phase 0 and 2π. In the phase distribution of the image taken of the reference plane, an abrupt change occurs at the transition between 0 and 2π for each grid cell in the grid pattern. Therefore, in this invention, instead of phase, the phase difference with respect to the phase value at z=z0 is used. While the phase value at a given reference plane varied from pixel to pixel when using phase, by using the phase difference instead, the phase difference value at a given reference plane becomes close to that of the surrounding pixels. In other words, the coordinate value referenced for any phase difference becomes close to the coordinate value of the surrounding pixels. Consequently, the error can be reduced even when interpolating from surrounding pixels, making it easier to compress the entire spatial table using the phase difference. If we denote the phase difference at pixel (i,j) as Δφ(i,j), the phase value as θ(i,j), and the initial phase value at z=z0 as θ0(i,j), then the phase difference at each pixel can be expressed by equation 6. Furthermore, if the initial phase value θ0(i,j) is greater than the phase value θ(i,j) and the phase difference Δφ(i,j) is negative, we can make the phase difference positive by adding 2π as shown in equation 7.

[0023]

number

[0024]

number

[0025] A method for expanding measurement depth using phase difference. Creating a region number search table using phase difference Next, to expand the measurement depth, we will explain a combination of a table compression method using phase difference and a phase continuation method. In one embodiment of the present invention, projection, phase difference, and phase continuation methods of two types of grid patterns are used to create a single full-space table in the region of 0 to 2kπ [rad], which is beyond the original range of 0 to 2π [rad], by using the projection, phase difference, and phase continuation methods of two types of grid patterns. This makes it possible to increase the measurement area. By substituting the phase difference into the resulting all-space table, multiple candidate coordinate values ​​can be obtained. By comparing these candidate values ​​with the phase difference of one of the grids, the true values ​​are calculated, thereby expanding the measurement depth. The number in the region number search table can be quickly retrieved from the phase differences of the two types of grids. Figure 12 shows the measurement depth expansion method using the phase differences of the two types of grids. The table shown in Figure 12 is the region number search table. Furthermore, in order to quickly obtain x, y, and z coordinate values ​​from the searched number, a table from 0 to 2kπ [rad] is created using a full-space tableization method with phase difference (hereinafter referred to as the "phase difference type full-space table"). When performing shape measurement, the searched number is substituted as k, and the phase difference type full-space table is referenced to obtain the x, y, and z coordinate values.

[0026] Phase connection method using two types of grids The phase coherence method takes a phase distribution that originally repeats in the range of 0 to 2π [rad] and adds a phase value of 2π to the adjacent intervals, using a certain interval as a reference. By adding a phase value of 2kπ to k adjacent intervals, the entire phase distribution can be transformed into a continuous curve. Figure 13 shows grid images G1 and G2 with pitches p1 and p2, respectively. Here, p1 > p2. Figure 14 shows the phase values ​​of grids G1 and G2 before and after phase coherence. The relationship between the phase value or phase difference and the z-direction is shown in Figures 15(c) and 15(d), respectively. Then, the relationship between the phase difference and the z-direction using the phase connection method and the phase difference is shown in Figures 15(e) and 15(f), respectively.

[0027] Create area number search table A region number lookup table is created for each pixel. This region number lookup table contains the measurement range number and is a distribution diagram that represents the two types of phase distributions measured in one image during calibration on a two-dimensional plane with the difference between the phase value and the phase value at the position z=z0 as the axis. To obtain a region number search table, it is necessary to interpolate points on the reference plane, filling in the black areas between two points to form a continuous curve. Therefore, as described above, two points on the reference plane can be interpolated using the phase difference and phase connection method, filling in the portion of the graph. Figure 16 shows the relationship between two phase-connected phase values ​​obtained as a result of interpolating two types of pitches. Figure 17 shows the relationship between phase difference, region, and z direction in the range of 0 to 2π [rad]. The first interval is set as range 1, and then the relationship between phase difference and region can be determined by adding 1 each time. Figure 18 shows the relationship between phase difference and the interpolated region number. However, in environments where the grid pitch or phase shift amount changes, such as in the light source switching phase shift method, external environmental influences can occur. If there is variation in phase difference, misidentification of region numbers may occur. Therefore, an interpolated curve can be used as a reference, and all areas within a 2-pixel radius around it can be assigned the same region number. Figure 19 shows the improved region number search table. The points in the figure are points on the reference plane, and the surrounding areas of the same color represent the same region number. The white areas are unused areas. By referring to this region number lookup table, the region number can be instantly determined from two phase differences. Once the region number is obtained, the phase difference of phase-connected regions can be determined by adding an integer multiple of 2π corresponding to that number to the phase difference in the range of 0 to 2π.

[0028] Calculation of coordinates using area number search table Creating a full-space table using phase difference In conventional methods for creating tables across all space, multiple tables (groups of tables) are created. When measuring the 3D shape, two phase distributions are first substituted into a region number search table to obtain region numbers. Then, the coordinates are calculated by substituting the region numbers into the group of tables. This invention first calculates the phase difference, then performs phase connection to transform the distribution of the phase difference into a continuous distribution from 0 to 2kπ [rad]. When using the whole-space table method, a single table from 0 to 2kπ [rad] is created. Therefore, boundary errors are eliminated. The relationship between the phase difference and x, y, and z is shown in Figures 20, 21, and 22, respectively. By using the phase difference, the whole-space table can be transformed into a single table that is easy to compress. The compressed table significantly reduces the data reading time.

[0029] Calculating coordinates from area number search table Figure 23 shows the measurement flow using the region number search table. First, a table for grid B (phase difference type full space table) is created as in the previous section. When measuring a 3D shape, the phase distributions θa and θb with different pitches are first calculated, and then the phase value of the reference plane is subtracted to obtain the phase difference distribution. Then, the distribution of the region numbers can be obtained by substituting the two phase difference distributions into the region number search table. If the region number is k, the final phase difference θdiff can be obtained by adding 2kπ [rad] to the phase difference θdiff. The calculated phase difference can be substituted into the phase difference type full space table to obtain the respective x, y, and z distributions. Then, by uploading the x, y, and z distributions and color information to OpenGL, a full-color 3D point cloud distribution of the measured object can be obtained.

[0030] As described above, a grid pattern is projected onto a reference plane, the projected grid pattern is photographed by an imaging device, the phase value of the grid pattern at a reference position on the reference plane is determined for each pixel from the captured image, and further, the phase value of the grid pattern at an arbitrary height on the reference plane is determined for each pixel. Here, height refers to the z-coordinate when the reference position is z=0. The reference plane is made capable of being translated in the out-of-plane direction to an arbitrary z-coordinate position. Next, the phase difference between the phase value on the reference plane and the phase value on the plane at the arbitrary height is determined for each pixel, and the attribute value of the point on the reference plane that the pixel photographs is determined. The spatial coordinates (x, y, z) of the point are used as this attribute value, but in some cases, only the z-coordinate may be used. Also, the image captured by the imaging device is composed of a collection of pixels, and each pixel obtains corresponding specific brightness information on the object being photographed through imaging by the imaging lens, so it can be said that the pixel is the point that it photographs.

[0031] Next, element numbers ni, nj, and nφ are created, each consisting of three integer values, based on the two components of the in-screen coordinates representing the position of the image pixels (horizontal in-screen coordinate i and vertical in-screen coordinate j) and the phase difference Δφ. The element defined by this set of element numbers (ni, nj, nφ) stores the attribute values ​​corresponding to that element. This table is called a 3D table because it has a three-dimensional structure based on the element numbers ni, nj, and nφ. Each element stores the attribute values ​​corresponding to that element. Note that the above-mentioned phase difference type full-space table is a table created for each pixel, but the 3D table is an extension of that, a table that is treated three-dimensionally.

[0032] The element number is an integer value generated from the values ​​of i, j, and Δφ by a calculation formula such as that shown in equation 8. Here, the value of Δφ is the same as that of m mentioned above.

[0033] [Number 8] ei = ai × i + bi ej = aj × j + bj eφ = aφ × Δφ + bφ ni = round(ei) nj = round(ej) nφ = round(eφ)

[0034] Here, ai, bi, aj, bj, aφ, and bφ are coefficients determined on a case-by-case basis, and ei, ej, and eφ are values ​​converted to real numbers. Also, round() is a function that rounds to an integer. This calculation formula can be any relationship, not just linear transformations, as long as the element numbers ni, nj, and nφ increase with increasing values ​​of i, j, and Δφ, respectively. It can also be a relationship where the element numbers ni, nj, and nφ increase or decrease independently with increasing values ​​of i, j, and Δφ. In this way, the element numbers (ni, nj, nφ) of the 3D table are constructed in such a way that there are no abrupt changes with increasing values ​​of i, j, and Δφ, and furthermore, there are no points where the increase turns into a decrease.

[0035] In the above calculation formula, element indices ki, kj, and kφ are calculated from the values ​​of i, j, and Δφ, respectively. However, by considering (i, j, Δφ) and (ei, ej, eφ) as vectors and combining them with matrix transformations, it is also possible to calculate a set of real values ​​(ei, ej, eφ) that will become the element indices from a set of horizontal and vertical in-screen coordinates representing the position of the image pixels and the phase difference value (i, j, Δφ), and from there generate a set of element indices (ni, nj, nφ) composed of three integer values. In addition to the linear transformation method using matrices, any method of mapping using an arbitrary calculation formula can also be used, as long as there are no abrupt changes as the values ​​of i, j, and Δφ increase, and there are no points where the increase turns into a decrease.

[0036] In other words, by using the transformation formula for the mapping that obtains (ei,ej,eφ) from (i,j,Δφ), we can generate a set of table element indices (ni,nj,nφ). In addition to spatial coordinates, the attribute values ​​stored in each element of a 3D table can also include various other pieces of information specific to that point, such as the intensity of projected light at that point or values ​​representing reliability, such as the level of noise.

[0037] Furthermore, the element number kφ in the above 3D table can also use phase-connected phase values. Phase connection can be performed by adding integer multiples of 2π. Methods for obtaining these integer multiples include adding or subtracting integer multiples at abrupt changes using the phase values ​​or phase difference values ​​of neighboring pixels of interest, methods that combine spatial coding methods or projection grids of different pitches to obtain integer multiples, and methods that project various patterns or perform rough 3D measurements using multiple cameras and estimate from the obtained coordinate values.

[0038] One method using grid patterns with different pitches involves projecting two types of grid patterns with different pitches onto a reference plane, capturing each projected grid pattern with an imaging device, determining the phase value of the grid pattern at a reference position on the reference plane for each pixel from the captured images, further determining the phase value of the grid pattern at an arbitrary height on the reference plane for each pixel, determining the phase difference between the phase value on the reference plane and the phase value on the plane at the arbitrary height for each pixel, and creating a region number search table for each pixel using these phase differences to obtain the integer multiples of the values ​​mentioned above for phase connection. When performing three-dimensional measurement, two types of grid patterns with different pitches, as described above, are projected onto the object to be measured. Each of the projected grid patterns is then photographed by an imaging device. From each of the captured images, the phase value at the point on the object that corresponds to the pixel is determined. The phase difference is then calculated for each pixel using the phase value of the grid pattern at the reference position on the reference plane. The measurement area is then determined from this phase difference using the previously created area number search table to obtain the integer multiples mentioned above. By adding the value obtained by multiplying this value by 2π, the phase-connected phase difference can be obtained.

[0039] Using the phase difference or phase-connected phase difference obtained in this way, the aforementioned 3D table can be constructed. Furthermore, the phase difference or phase-connected phase difference can be similarly determined for the object to be measured, and shape measurement can be performed using these values ​​and the aforementioned 3D table. As a supplement to the above-mentioned method for compressing tables using phase difference, we will now explain how to reconstruct the 3D table constructed in this manner. By reconstructing, the number of elements in the table can be reduced. Conversely, it can also be increased as needed. The element numbers (ni, nj, nφ) of the 3D table obtained as described above are such that there are no abrupt changes as the values ​​of i, j, and Δφ increase, and there are no points where the increase turns into a decrease. Therefore, the number of table elements is changed by further transforming the table element numbers, and the attribute values ​​stored in the elements are generated by interpolation from the attribute values ​​of surrounding elements, and stored in the new elements. If the attribute values ​​are spatial coordinates, the attribute values ​​themselves also do not have abrupt changes as the values ​​of i, j, and Δφ increase, so the error caused by interpolation can be kept to a minimum.

[0040] However, in the case of three-dimensional measurement using grid projection, a striped error distribution may occur depending on the pattern of the projection grid. This occurs because the positional relationship between the camera and projection device within the apparatus changes slightly due to temperature changes, etc., between the time the 3D table is created and the time the object to be measured is measured. When such a striped error distribution occurs, the interpolation error may also become larger. When reducing the number of table elements, it is necessary to reduce the number of elements to a extent that minimizes the impact of this striped error. To do this, the number of elements in the 3D table is determined and the 3D table is created based on the distribution of attribute values ​​of the elements within the 3D table.

[0041] When measuring an object, the element number (ni, nj, nφ) of the 3D table is determined from the in-screen coordinates (i, j) of the pixel of interest and the phase difference Δφ obtained at that pixel. The attribute value of the pixel of interest is then determined by referring to a pre-constructed 3D table. In this process, a set of real values ​​(ei, ej, eφ) that form the basis of the element number is calculated from the set of in-screen coordinates and the phase difference value (i, j, Δφ), and the element number (ni, nj, nφ) is then determined from this set. However, since the element number itself is an integer value, the attribute value corresponding to (i, j, Δφ) is not stored in that element. Therefore, the attribute value corresponding to (i, j, Δφ) is calculated by interpolation using the attribute values ​​of multiple elements that are close to (ei, ej, eφ). This interpolation process can use not only linear interpolation but also various other interpolation methods. If the attribute value is a spatial coordinate, three-dimensional measurement can be realized. If the attribute value includes values ​​that represent reliability, such as the intensity of the projected light at that point or the level of noise, these values ​​can also be determined by interpolation.

[0042] By doing so, it becomes possible to perform three-dimensional measurements using a 3D table constructed with a reduced number of elements. Furthermore, by increasing the number of elements, it is possible to perform accurate three-dimensional measurements without performing interpolation calculations.

[0043] By constructing a shape measurement device that performs three-dimensional measurement using the method described above, the data size of the entire spatial table can be compressed to a smaller size, thereby solving the aforementioned problems. For example, the shape measurement device consists of a projector for grid projection, a camera for capturing images, and an analysis device for determining the distribution of spatial coordinates using a 3D table. In addition, to construct the 3D table, a reference plane and a device for moving it, or a device for moving the reference plane and the shape measurement device relative to each other can be used.

[0044] Experiment using a region number search table Using the environmental and phase difference information measured in the previous chapter, an analysis was performed using a region number search table. Figure 24 shows the doll's face captured by a color camera, the synthesized color image, the z-coordinate distribution, and the 3D point cloud distribution. To accurately determine region numbers due to variations in phase difference, the region number search table was improved. Using the interpolated curve as a reference, all areas within a 2-pixel radius surrounding it are assigned the same range number. This confirmed that even with some variation in phase difference, the region numbers remained the same.

[0045] Table compression By using phase difference, the entire spatial table can be transformed into a form that is easier to compress. This section describes how to compress this entire spatial table. This section describes a compression method using arbitrary magnifications, where the i, j, and m axes each have different magnifications. Figure 26 shows the procedure for compressing a table using arbitrary magnifications. The compression magnification coefficients for the i, j, and m axes are set to k, respectively. i ,k j ,k m Let the coordinates of the original table be i, j, and m, respectively. Also, let the coordinates of the compressed table be i, j, and m, respectively. zip Today J zip ,m zip Thus, these can be expressed as equations 9, 10, and 11, respectively. Here, the compression ratio coefficient k i ,k j ,k m Let each be an arbitrary natural number.

[0046]

number

[0047]

number

[0048]

number

[0049] Calculation of coordinate values ​​using a compressed table Expand the compressed table to calculate coordinates The compressed table cannot be used directly during measurement because the resolution of the phase distribution obtained by phase analysis of the grid image is different. Therefore, it is necessary to restore the compressed table to its original size. First, the table is expanded along the i-axis. The phase difference to be substituted for the pixel with coordinate value i in the expanded table can be expressed as equation 12. ri=0 is the calculation formula for pixels where the value remains after compression. The remaining two cases are the calculation formulas for the phase difference of pixels where the value is compressed and no longer remains during the compression process, and interpolation from surrounding pixels is necessary. <ri<w zip If the value is -1, the phase difference value to be substituted is generated by performing linear interpolation based on the phase difference values ​​of the pixels before and after the value that remains. ri=w zip In the case of -1, since the only pixel with a value remaining is the immediately preceding pixel, the phase difference to be substituted is generated by performing linear interpolation based on the values ​​of the immediately preceding pixel and the pixel before that.

[0050] Here, the width of the compressed table is w zip , the coordinate values ​​of the compressed table i zip Let i be the coordinate value of the expanded table, and let k be the compression factor of the i-axis. i Let's assume that the coordinate value i of the compressed table is also included. zip The phase difference in the pixel is φ zip (i zip Let φ(i) be the phase difference to be substituted for the pixel of coordinate value i in the expanded table. Furthermore, the expanded coordinate value i is the compression factor k. i The remainder when divided by r i This is shown in equation 13, and the coordinate value i of the compressed table zip is variable r i It can be expressed as equation 14 using the formula shown. Figure 27 shows a schematic diagram of equation 12. By performing similar processing in the j-axis and m-axis directions, the compressed table can be expanded back to its original size. Furthermore, the same procedure is used to expand tables containing x, y, and z coordinate values, not just phase differences.

[0051]

number

[0052]

number

[0053]

number

[0054] Calculating coordinates from a compressed table While it is possible to calculate coordinates by restoring the compressed table to its original size using the method described in paragraph "0049," this does not resolve the problem of the large amount of memory required for analysis. Therefore, it is necessary to use the compressed table directly for measurement without decompressing it. Here, the coordinate values ​​of the respective compression tables for the i, j, and m axes are i zip Today J zip ,m zip Let i, j, and m be the coordinate values ​​of the expanded table, and let k be the compression factor. i ,k j ,k m Let's assume that the pixels (i) of the compressed table are... zip Today J zip ,m zip The height inside is z zip (i zip Today J zip ,m zip Let z(i,j,m) be the height to be substituted for pixel (i,j,m) in the expanded table. Furthermore, the remainder r when the coordinate values ​​of the expanded table are divided by the compression factor is i ,r j ,r mThis is shown in equation 15. The coordinate values ​​of the compressed table are each variable r. i ,r j ,r m It can be expressed as shown in equation 16 using the following formula. The height value z0 is contained in the surrounding pixels, which is present in the pixels before compression and used for interpolation during measurement. zip From z7 zip This will be expressed as shown in equation 17.

[0055]

number

[0056]

number

[0057]

number

[0058] First, let's describe how to find the height z(i,j,m) when ri=0, rj=0, and rm=0. In this case, as shown in equation 18, z(i,j,m) is z zip (i zip Today J zip ,m zip Since this value is equal to ), the value in the corresponding pixel of the compression table can be substituted for the pixel in the image to be output as the measurement result. A schematic diagram of the coordinate calculation procedure at this time is shown in Figure 28.

[0059]

number

[0060] Next, the method for obtaining the height z(i, j, m) when ri≠0, rj = 0, and rm = 0 will be described. In this case, since the corresponding coordinate values remain in the j-axis direction and the m-axis direction after compression, they can be used as they are. However, for the i-axis direction, since the values of the pixels are lost when the table is compressed, it is necessary to interpolate from the surrounding pixels. Two surrounding pixels z0 zip and z1 zip are expressed using Equation (17) respectively, and z(i, j, m) can be calculated as shown in Equation (19). For the cases of ri = 0, rj≠0, rm = 0 and ri = 0, rj = 0, rm≠0, instead of z0 zip , z1 zip , respectively use z0 zip , z2 zip and z0 zip , z4 zip . By using k i , r i instead of k j , r j and k m , r m , it can be obtained in the same way. Also, a schematic diagram of the coordinate calculation procedure at this time is shown in Figure 29.

[0061]

Equation

[0062] Also, the method for obtaining the height z(i, j, m) when ri≠0, rj≠0, and rm = 0 will be described. In this case, since the corresponding coordinate values remain in the m-axis direction after compression, they can be used as they are. However, for the i-axis direction and the j-axis direction, since the values of the pixels are lost when the table is compressed, it is necessary to interpolate from the surrounding pixels. Four surrounding pixels z0 zip , z1 zip , z2 zip , z3 zip are expressed using Equation (17) respectively, and z(i, j, m) can be calculated as shown in Equation (20). First, z0 zip , z1 zip and z2 zip , z3 zipPerform linear interpolation on the i-axis in two sets of two points each, and then perform linear interpolation on the j-axis from the two points z0 and z ( zip , z1 zip , z2 zip , z3 zip thus obtained to find the height z(i, j, k) at an arbitrary pixel (i, j, m) by interpolation from the values of the four surrounding points in the ij plane. In the case where ri≠0 and rj = 0 and rm≠0 and ri = 0 and rj≠0 and rm≠0, z0 zip , z1 zip , z4 zip , z5 zip and z0 zip , z2 zip , z4 zip , z6 zip are used instead of k i , r i , k j , r j and k i , r i , k m , r m and k j , r j , k m , r m in the same way. Also, a schematic diagram of the coordinate calculation procedure at this time is shown in Fig. 30.

[0063]

Number

[0064] Finally, the method for obtaining the height z(i, j, m) in the case where ri≠0 and rj≠0 and rm≠0 will be described. In this case, since the pixel values are missing when the table is compressed in all directions of the i-axis, j-axis, and m-axis, it is necessary to interpolate from the surrounding pixels. The eight surrounding pixels z0 zip , z1 zip , z2 zip , z3 zip , z4 zip , z5 zip , z6 zip , z7 zipIf we express each of these using equation 17, then z(i,j,m) can be calculated as shown in equation 21. First, z0 zip ,z1 zip and z2 zip ,z3 zip and z4 zip ,z5 zip and z6 zip ,z7 zip By performing linear interpolation on the i-axis using four pairs of two points, four points z0, z1, z2, and z3 are obtained. Next, using the four obtained points, two points z4 and z5 are obtained by performing linear interpolation on the j-axis using two pairs of two points: z0, z1 and z2, z3. Then, using the two obtained points z4 and z5, linear interpolation is performed on the m-axis to interpolate from the values ​​of the eight surrounding points in the ijm space, thereby determining the height z(i,j,m) of an arbitrary pixel (i,j,m). A schematic diagram of this coordinate calculation procedure is shown in Figure 31.

[0065]

number

[0066] The method described above is for calculating the z-coordinate value, but the x and y coordinate values ​​can be calculated using the same procedure. By obtaining the x, y, and z coordinate values ​​in this way, we can acquire 3D coordinates.

[0067] <Examples> An apparatus as shown in Figure 32 was constructed, and after calibration, the reference surface was treated as a planar sample to evaluate the measurement accuracy. Measurements were taken by moving the reference surface from z=0.50mm to z=40.50mm in 10.00mm increments, using a position different from the z-coordinate of the reference surface captured during calibration. A feedback stage with a positioning accuracy of 100nm was used for this experiment.

[0068] Figure 33 is (k i ,k j ,k m This is the experimental result for the case where )=(1,1,1), i.e., when the table is not compressed. Figure 34 shows (ki ,k j ,k m In the case where )=(2,1,1), i.e., when the table is compressed twice in the i-direction, Figure 35 shows the experimental results. i ,k j ,k m )=(1,2,1), which is the experimental result when the table is compressed twice in the j-direction.

[0069] When compression was doubled in the j-direction, there was almost no difference compared to the uncompressed case. This confirmed the effectiveness of the compression. On the other hand, when compression was doubled in the i-direction, a large error appeared. Upon examining the distribution of z-coordinate values ​​in the i-direction, it was found that unevenness in the z-coordinate occurred in the i-direction according to the pattern of the projection grid, and this was the cause. In other words, when determining the compression ratio, errors can occur depending on the distribution of attribute values.

Claims

1. Project the grid pattern onto the reference plane, The projected grid pattern is photographed by an imaging device. From the captured image, the phase value of the grid pattern at the reference position on the reference plane is determined for each pixel. Furthermore, the phase value based on the grid pattern at an arbitrary height of the reference plane is determined for each pixel. The phase difference between the phase value on the reference plane and the phase value on the plane of the arbitrary height is determined for each pixel. On that reference plane, determine the attribute values ​​of the points on the reference plane that the pixel is capturing, A method for creating a three-dimensional table in which a set of element numbers consisting of three integer values ​​is created by combining two components of the in-screen coordinates representing the position of the pixels in the aforementioned image and the value of the phase difference, and each element stores the attribute value corresponding to that element.

2. A method for creating a three-dimensional table using phase-connected phase values ​​according to claim 1.

3. A method for creating a three-dimensional table in which values ​​including spatial coordinates as attribute values ​​are stored in the table elements, according to claim 1 or claim 2.

4. A method for performing shape measurement using a three-dimensional table according to claim 1 or claim 2.

5. Two different grid patterns with different pitches are projected onto the reference plane. Each of the projected grid patterns is photographed by an imaging device. From each captured image, the phase value of the grid pattern at the reference position on the reference plane is determined for each pixel. Furthermore, the phase value based on the grid pattern at any height on the reference plane is determined for each pixel. The phase difference between the phase value on the reference plane and the phase value on the plane of the arbitrary height is determined for each pixel. Using the respective phase differences mentioned above, a region number search table is created for each pixel. When performing three-dimensional measurement, the two types of grid patterns with different pitches are projected onto the object to be measured. Each of the projected grid patterns is photographed by an imaging device. From each captured image, the phase value at the point on the object being measured that corresponds to that pixel is determined. Using the phase value of the grid pattern at the reference position on the reference plane, the phase difference is determined for each pixel. Using the aforementioned region number search table, determine the measurement region from the phase difference. Depending on the measurement area, the phase difference is obtained by adding an integer multiple of 2π to the phase difference, A method for performing shape measurement using the three-dimensional table, wherein the phase difference is defined as the phase difference in claim 1 or claim 2.

6. A method for creating a three-dimensional table, according to claim 1 or claim 2, which involves changing the number of elements in the three-dimensional table and reconstructing the three-dimensional table.

7. A method for creating a three-dimensional table according to claim 6, wherein the number of elements in the three-dimensional table is determined by the distribution of attribute values ​​of the elements in the three-dimensional table.

8. A shape measurement method according to claim 1 or claim 2, which performs interpolation processing when referencing a three-dimensional table.

9. A shape measuring device that implements claim 1 or claim 2.

Images