Method for calculating three-dimensional shape information of an object surface, optical system, program for calculating three-dimensional shape information of an object surface, and processing device for the optical system.

Abstract

To provide a method for calculating three-dimensional shape information of an object surface by acquiring three-dimensional shape information of the object surface without splitting a light beam on an illumination side.SOLUTION: According to an embodiment, a method of calculating three-dimensional shape information of an object surface comprises: acquiring an image captured through an anisotropic wavelength selection portion having at least two different regions configured to select a wavelength to be shielded and a wavelength to be passed from reflected light from the object surface illuminated with light; color-mapping light beam directions based on the image; and calculating three-dimensional shape information of the object surface from a geometric optics relational expression between an inclination angle of the object surface and the light beam direction.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] Embodiments of the present invention relate to a method for calculating three-dimensional shape information of an object surface, an optical system, a program for calculating three-dimensional shape information of an object surface, and a processing device for an optical system.

Background Art

[0002] In various industries, non-contact surface measurement of objects is important. In a conventional method, there is a technique in which light rays are spectroscopically analyzed to illuminate an object, images spectroscopically analyzed in each case are acquired by an imaging device, and the three-dimensional shape information of the object surface is acquired by estimating each light ray direction.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Non-Patent Documents

[0004]

Non-Patent Document 1

Non-Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] The problem that the present invention aims to solve is to provide a method for calculating three-dimensional shape information of an object surface, an optical system, a program for calculating three-dimensional shape information of an object surface, and a processing device for the optical system, which acquire three-dimensional shape information of an object surface without spectrally separating the light rays on the illumination side. [Means for solving the problem]

[0006] According to the embodiment, the method for calculating the three-dimensional shape information of an object surface involves imaging through an anisotropic wavelength selection unit having at least two different regions that select wavelengths to block and wavelengths to pass through from the reflected light from the object surface illuminated with light. one To acquire an image, one This includes color mapping of ray directions from an image, and calculating three-dimensional shape information of an object surface from a geometric-optical relationship between the inclination angle of the object surface and the ray direction. [Brief explanation of the drawing]

[0007] [Figure 1] A schematic diagram showing an optical system according to the first embodiment. [Figure 2] A schematic block diagram of the processing unit for an optical system. [Figure 3] A schematic diagram showing the relationship between incident light, object point, reflected light, normal direction, and inclination angle θx. [Figure 4] A schematic flowchart of the processing performed by the processing unit of an optical system. [Figure 5] A schematic diagram showing the relationship between the anisotropic wavelength selection region, incident light, reflected light, and the directional component of light in the x-axis direction, relative to the xyz orthogonal coordinate system. [Figure 6] An example of an image acquired using the optical system according to the first embodiment. [Figure 7] An example of a three-dimensional shape reconstructed from the image shown in Figure 6. [Figure 8] A schematic diagram showing an optical system according to the second embodiment. [Figure 9] A schematic diagram showing an optical system according to the third embodiment. [Modes for carrying out the invention]

[0008] The embodiments are described below with reference to the drawings. The drawings are schematic or conceptual, and the relationship between the thickness and width of each part, the ratio of the sizes of the parts, etc., are not necessarily the same as those of reality. Also, even when representing the same part, the dimensions and ratios may be represented differently in the drawings. In this specification and each drawing, the same reference numerals are used for elements that are the same as those described above with respect to previously shown drawings, and detailed explanations are omitted as appropriate.

[0009] (First Embodiment) The optical system 10 according to this embodiment will be described below with reference to Figures 1 to 4.

[0010] In this specification, light is defined as a type of electromagnetic wave, and includes X-rays, ultraviolet rays, visible light, infrared rays, microwaves, and the like. In other words, any electromagnetic wave that can be described by Maxwell's equations is acceptable. In this embodiment, the light is defined as visible light, and for example, its wavelength is in the range of 400 nm to 750 nm.

[0011] Figure 1 shows a schematic cross-sectional view of the optical system 10 according to this embodiment.

[0012] The optical system 10 according to this embodiment includes an optical device 12 and a processing device 14.

[0013] The optical device 12 includes an illumination unit 22, an imaging unit 24, and a wavelength selection unit (multi-wavelength aperture) 26.

[0014] The illumination unit 22 includes a light source 32, an aperture 34, and an illumination lens 36. The light source 32 can be anything that emits light. Here, for example, it is assumed that the light source 32 uses a white LED. The aperture 34 is a light shielding plate provided with slits. The light source 32 is disposed on the focal plane of the illumination lens 36. In such a configuration, the light emitted from the light source 32 is partially blocked and partially passes through at the aperture 34. Then, the light passing through the aperture 34 becomes substantially parallel light by the illumination lens 36. Therefore, the illumination unit 22 converts the light from the light source 32 into parallel light with the illumination lens 36. However, the parallel light may have a divergence angle. The illumination lens 36 can design a free-form surface by an original optical design method incorporating analytical geometric optics so as to efficiently emit highly parallel light rays.

[0015] The parallel light from the illumination unit 22 is irradiated onto the surface of the object S through the beam splitter 28. The parallel light is irradiated onto the surface (object surface) of the object S along the z-axis. By irradiating the parallel light onto the surface of the object S, the incident direction of light can be aligned at each point on the surface of the object S. That is, the incident direction of light can be aligned over the entire imaging surface.

[0016] The imaging unit 24 is directed toward a site where the parallel light on the surface of the object S is illuminated. The imaging unit 24 includes an imaging optical element 42 and an image sensor (color image sensor) 44. The imaging optical element 42 is, for example, an imaging lens. The imaging optical element 42 has a focal length of f. In FIG. 1, the imaging lens is schematically depicted by a single lens, but it may be a combination lens composed of a plurality of lenses. Alternatively, the imaging optical element 42 may be a concave mirror, a convex mirror, or a combination thereof. That is, the imaging optical element 42 may be any optical element that has the function of collecting a group of light rays emitted from a point on the object S, that is, an object point, to a conjugate image point on the image sensor 44. The imaging of a group of light rays emitted from an object point on the surface of the object S to an image point by the imaging optical element 42 (condensed) is referred to as imaging. Alternatively, it is also said that the object point is moved to the image point (the conjugate point of the object point). Thus, the object point and the image point are associated as a conjugate relationship via the imaging optical element 42. The set surface of the conjugate points to which a group of light rays emitted from a sufficiently distant object point is moved by the imaging optical element 42 is called the focal plane of the imaging optical element 42. Also, a line perpendicular to the focal plane and passing through the center of the imaging optical element 42 is defined as the optical axis L. The point where the optical axis L intersects the focal plane is called the focus.

[0017] Note that a three-dimensional orthogonal coordinate system (xyz orthogonal coordinate system) is defined by taking the direction along the optical axis L as the z-axis, an x-axis orthogonal to the z-axis, and a y-axis orthogonal to the x-axis and the z-axis. Here, the xyz coordinate system is defined for the wavelength selection unit 26, and the origin O is set on the third wavelength selection region 56 described later. The z-axis intersects the wavelength selection unit 26. The x-axis is where the plurality of wavelength selection regions 52, 54, 56 intersect. The y-axis is along the plurality of wavelength selection regions 52, 54, 56. However, this is not limited to this, and a plurality of wavelength selection regions may intersect the y-axis, and at least two of the plurality of wavelength selection regions may intersect the x-axis.

[0018] The wavelength selection unit 26 in this embodiment has a stripe shape parallel to the longitudinal direction of the line sensor 44, which will be described later. The wavelength selection unit 26 is provided between the surface of the object S and the imaging unit 24. The wavelength selection unit 26 has at least two or more wavelength selection regions 52, 54, 56. Two of these wavelength selection regions are designated as the first wavelength selection region 52 and the second wavelength selection region 54. The first wavelength selection region 52 allows light rays having a wavelength spectrum including a first wavelength to pass through. Here, allowing light rays to pass through means directing the light rays from the object point to the image point by transmission or reflection. In this embodiment, the first wavelength selection region 52 is assumed to transmit light rays of the first wavelength. On the other hand, the first wavelength selection region 52 substantially shields light rays of the second wavelength. Here, shielding means not allowing light rays to pass through. In other words, it means not directing light rays from the object point to the image point.

[0019] The second wavelength-selective region 54 allows light rays of the second wavelength to pass through. Therefore, in this embodiment, the second wavelength-selective region 54 transmits light rays of the second wavelength. On the other hand, the second wavelength-selective region 54 substantially blocks light rays of the first wavelength.

[0020] In this embodiment, the first wavelength selection region 52 and the second wavelength selection region 54 each extend, for example, along the y-axis. The first wavelength selection region 52 and the second wavelength selection region 54 intersect with the x-axis.

[0021] For example, the first wavelength is 450 nm blue light (B), and the second wavelength is 650 nm red light (R). However, the wavelengths are not limited to these; any wavelengths can be used.

[0022] The arrangement of the wavelength selection regions 52 and 54 of the wavelength selection unit 26 is anisotropic with respect to the optical axis L of the imaging optical element 42. In other words, when the optical axis L is used as the axis, the overall shape (arrangement) of the wavelength selection regions 52 and 54 depends on the direction of rotation around the optical axis L. For this reason, the wavelength selection unit 26 will also be called an anisotropic multi-wavelength aperture. In this embodiment, the first wavelength selection region 52 and the second wavelength selection region 54 are opposite each other with respect to the optical axis L, and the wavelength selection unit 26 is anisotropic.

[0023] In this embodiment, the wavelength selection unit 26 further includes a third wavelength selection region 56. The third wavelength selection region 56 is located along the x-axis between the first wavelength selection region 52 and the second wavelength selection region 54. The third wavelength selection region 56 is positioned on the optical axis L. The third wavelength selection region 56 allows wavelength spectra containing rays of a third wavelength to pass through, while substantially blocking rays of the first and second wavelengths. The third wavelength selection region 56 also extends along the y-axis. The first wavelength-selective region 52 substantially blocks light rays of the third wavelength. Similarly, the second wavelength-selective region 54 substantially blocks light rays of the third wavelength. As an example, the third wavelength is 550 nm green G light.

[0024] The arrangement of the wavelength selection regions of the wavelength selection unit 26 can be set as appropriate. For example, the wavelength selection regions of the wavelength selection unit 26 are formed along the x-axis in order from shortest to longest transmitted wavelengths, with each region having an appropriate width in the x-axis direction. Conversely, the wavelength selection regions of the wavelength selection unit 26 are formed along the x-axis in order from longest to shortest transmitted wavelengths, with each region having an appropriate width in the x-axis direction. In addition, the arrangement of the wavelength selection regions of the wavelength selection unit 26 can be set as appropriate, such as a region that transmits green G light with a wavelength of 550 nm and blocks everything else, a region that transmits blue B light with a wavelength of 450 nm and blocks everything else, and a region that transmits red R light with a wavelength of 650 nm and blocks everything else, along the x-axis.

[0025] The image sensor 44 has at least one pixel, and each pixel is capable of receiving at least two different wavelengths of light, namely a first wavelength of light and a second wavelength of light. The image sensor 44 according to this embodiment is further capable of receiving a third wavelength of light.

[0026] The plane containing the region where the image sensor 44 is located is defined as the image plane of the imaging optical element 42. The image sensor 44 may be an area sensor or a line sensor. An area sensor has pixels arranged in an area shape within the same plane. A line sensor has pixels arranged in a line shape. Each pixel may also have three color channels: R, G, and B.

[0027] In this embodiment, the image sensor 44 is a line sensor. The longitudinal direction of the image sensor (line sensor) 44 is aligned with the y-axis. The image sensor 44 is assumed to have at least two color channels for each pixel: red R light and blue B light. That is, it is assumed that blue B light with a wavelength of 450 nm and red R light with a wavelength of 650 nm can be received in separate color channels. In this embodiment, each pixel is also assumed to be able to receive green G light with a wavelength of 550 nm in separate color channels.

[0028] The image sensor 44 is connected to the processing unit 14 by wire or wirelessly. Figure 2 is a block diagram showing an example of the processing unit 14 of the optical system 10 according to this embodiment.

[0029] The processing unit 14 includes, for example, a processor 61 (control unit), a ROM (storage unit) 62, a RAM 63, an auxiliary storage device 64 (storage unit), a communication interface 65 (communication unit), and an input unit 66.

[0030] The processor 61 is the central part of the computer that performs calculations and control necessary for the processing of the processing unit 14, and comprehensively controls the entire processing unit 14. Based on programs such as system software, application software, or firmware stored in a storage unit such as ROM 62 or auxiliary storage device 64, the processor 61 executes control to realize various functions of the processing unit 14. The processor 61 includes, for example, a CPU (central processing unit), MPU (micro processing unit), DSP (digital signal processor), ASIC (Application Specific Integrated Circuit), or FPGA (Field Programmable Gate Array). Alternatively, the processor 61 is a combination of several of these. The processing unit 14 may have one processor 61 or multiple processors 61.

[0031] ROM62 corresponds to the main memory of a computer centered around the processor 61. ROM62 is a non-volatile memory used exclusively for reading data. ROM62 stores the program described above. Furthermore, ROM62 stores data or various settings used by the processor 61 in performing various processes.

[0032] RAM63 corresponds to the main memory of a computer centered around the processor 61. RAM63 is memory used for reading and writing data. RAM63 is used as a so-called work area, where data temporarily used by the processor 61 during various processes is stored.

[0033] The auxiliary storage device 64 corresponds to the auxiliary storage device of a computer centered on the processor 61. The auxiliary storage device 64 is, for example, an EEPROM (electrically erasable programmable read-only memory) (registered trademark), an HDD (hard disk drive), or an SSD (solid state drive). The auxiliary storage device 64 may also store the above-mentioned programs. In addition, the auxiliary storage device 64 stores data used by the processor 61 in performing various processes, data generated by processing by the processor 61, or various setting values.

[0034] The program stored in the ROM 62 or auxiliary storage device 64 includes a program for controlling the processing unit 14. For example, a program for the three-dimensional shape of an object surface is preferably stored in the ROM 62 or auxiliary storage device 64.

[0035] The communication interface 65 is an interface for communicating with other devices via a network or the like, either by wire or wireless connection, receiving various information transmitted from other devices, and transmitting various information to other devices. The processing unit 14 acquires image data obtained by the image sensor 44 via the communication interface 65.

[0036] The processing unit 14 preferably includes an input unit 66, such as a keyboard, for inputting, for example, the arrangement and type of the anisotropic wavelength selection unit 26. The input unit 66 may also be capable of receiving various types of information wirelessly from the processor 61 via a communication interface 65.

[0037] The processing unit 14 performs various functions by causing the processor 61 to execute programs stored in the ROM 62 and / or auxiliary storage device 64, etc. It is also preferable that the control program of the processing unit 14 is not stored in the ROM 62 and / or auxiliary storage device 64 of the processing unit 14, but is instead located on an appropriate server or cloud. In this case, the control program is executed while communicating with the processor 61 of the optical system 10, for example, via the communication interface 65. That is, the processing unit 14 according to this embodiment may be part of the optical system 10, or it may be located on a server or cloud of a system in various inspection facilities, separate from the optical system. Therefore, it is also preferable that the three-dimensional shape program of the object surface is located on a server or cloud, rather than being stored in the ROM 62 or auxiliary storage device 64, and that the program is executed while communicating with the processor 61 of the optical system 10, for example, via the communication interface 65. Consequently, the processor 61 (processing unit 14) can execute programs related to three-dimensional shape calculations, which will be described later.

[0038] The processor 61 (processing unit 14) controls the emission of light from the light source 32 of the illumination unit 22, the timing of image data acquisition by the image sensor 44, and the acquisition of image data from the image sensor 44.

[0039] The basic operation of the optical system 10 described above will now be explained.

[0040] Under the control of the processing unit 14, the light source 32 of the illumination unit 22 emits light. The light from the light source 32 becomes substantially parallel light and, via the beam splitter 28, illuminates the surface of object S with substantially parallel light.

[0041] In normal imaging without using the wavelength selector 26, reflected light from object points on the surface of object S is focused directly onto a single image point by imaging, regardless of its direction. Therefore, information regarding the direction of reflected light from object points cannot be obtained from the captured image.

[0042] Here, we assume that an object point lies on the optical axis L intersecting the surface of object S. The object point on the optical axis L is imaged onto the image point of the image sensor 44, which is also on the optical axis. At this time, a light ray propagating parallel to the optical axis L from the object point on the optical axis L passes through the origin O of the wavelength selection unit 26. On the other hand, a light ray propagating obliquely to the optical axis L from the object point passes through a position on the wavelength selection unit 26 that is far from the origin O. Furthermore, depending on the region through which the light passes through the wavelength selection unit 26 (first wavelength selection region 52, second wavelength selection region 54, third wavelength selection region 56), the light becomes one with different wavelength spectra.

[0043] Here, when a light ray is projected onto the xz plane, the angle of inclination of the light ray with respect to the optical axis L is denoted as θx. This is called the directional component of light with respect to the x-direction. The directional component θx of light is determined by the wavelength spectrum obtained when the light passes through the wavelength selection unit 26. In this case, as shown in Figure 2, the light ray takes on different colors depending on the directional component θx. Alternatively, even if the directional component θx of light cannot be precisely determined by the wavelength spectrum, it is possible to determine whether the direction of the light is inclined in the positive or negative direction of the x-axis. In other words, because the wavelength selection unit 26 is anisotropic, it is possible to determine whether the directional component θx of the x-axis of the reflection direction of light is positive or negative. For this reason, the optical system 10 according to this embodiment can color map the direction of light rays from the surface of an object S based on an image captured through the anisotropic wavelength selection unit 26.

[0044] The directional distribution of reflected light from a point on the surface of object S can be represented by a distribution function called BRDF (Bidirectional Reflectance Distribution Function). Generally, BRDF changes depending on the surface properties and shape of object S. For example, if the surface of object S is rough, the reflected light spreads in various directions, resulting in a broad BRDF distribution. In other words, reflected light exists over a wide range of angles. On the other hand, if the surface of object S is mirror-like, the reflected light consists almost entirely of specular reflection, resulting in a narrow BRDF distribution. Thus, BRDF reflects the surface properties and shape of object S. Here, the surface properties and shape of object S can be surface roughness, micron-sized irregularities, surface inclination, strain, etc. In short, anything related to the height distribution of the surface of object S is acceptable. When the surface properties and shape of object S are composed of a fine structure, the typical structural scale can be nanoscale, micronscale, milliscale, or any other scale.

[0045] As shown in the example in Figure 1, the wavelength spectrum of light acquired at the image point of the image sensor 44 changes depending on the BRDF at the object point on the central axis (z-axis) of the xyz coordinate system. For example, if the surface of object S at the object point is flat (the surface is reflective or nearly reflective), the BRDF will have a narrow distribution, and the light reflected at the object point will pass through the third wavelength selection region 56, which is sandwiched between the first wavelength selection region 52 and the second wavelength selection region 54 of the wavelength selection unit. In other words, it will not pass through the first wavelength selection region 52 and the second wavelength selection region 54.

[0046] On the other hand, if defects such as minute defects exist at the object point on the surface of object S, the BRDF will have a broad distribution, and the light reflected at the object point will pass through either the first wavelength selection region 52 or the second wavelength selection region 54. As a result, the wavelength spectrum of the light reaching the image point of the image sensor 44 will differ depending on the BRDF at the object point. If the wavelength spectrum reaching the image point of the image sensor 44 is different, the color (ray direction) acquired by the image sensor 44 will be different. This has the effect of allowing the difference in BRDF to be distinguished by the color (ray direction). Furthermore, if the difference in BRDF can be distinguished, the presence or absence of minute defects at the object point can be determined.

[0047] Furthermore, if the BRDF is anisotropic, light may pass through the first wavelength selection region 52 but not the second wavelength selection region 54. Alternatively, it may pass through the second wavelength selection region 54 but not the first wavelength selection region 52. In these cases, the color (ray direction) acquired by the image sensor 44 will be different. Also, in all cases, the color will be different from that when the BRDF is isotropic. As a result, it is possible to distinguish whether the BRDF is anisotropic or isotropic by its color. Furthermore, it is possible to distinguish between different types of anisotropic BRDFs. Such identification is difficult if the wavelength selection unit 26 is isotropic rather than anisotropic.

[0048] If the BRDF on the surface of object S has a narrow distribution, the optical system 10 can reconstruct the three-dimensional shape, including minute shapes, as described later. In fact, using this embodiment, a method can be constructed to obtain the direction of light reflected from the surface of object S, determine the normal direction of the surface of object S, and calculate three-dimensional shape information.

[0049] The reflected light from the surface of object S often has a large specular reflection component or a large intensity in its vicinity. In particular, as the surface of object S approaches a mirror surface, the reflected light consists almost entirely of the specular reflection component. In other words, the BRDF has a narrow distribution. The direction of the specularly reflected light lies on the plane (incident plane) spanned by the direction of the light incident on a point on the surface of object S (referred to here as the object point) and the normal direction of the surface, as shown in Figure 3. The normal direction is determined such that the angle of incidence and the angle of reflection are equal. Therefore, if the direction of the incident light is known in advance, the normal direction at the object point on the surface of object S can be determined by measuring the direction of the specularly reflected light.

[0050] Here, if the position of a point on the surface of object S is (x, y, h) and the height is h, then the three-dimensional shape of the surface of object S can be determined if the height h is determined as a function of x and y. Here, x and y are positions on the imaging plane. Furthermore, the normal direction of the surface of object S can be expressed as the spatial partial derivative with respect to the height h. Therefore, by using the relationship between the normal direction and the direction of specular reflection, an equation can be derived that expresses the relationship between the height h and the directional components of specular reflection (tilt angles θx and θy). In other words, the following partial differential equation can be derived for height h.

[0051]

number

[0052] Equation (1) is a geometrical optical relation, a partial differential equation obtained by partially differentiating the height h of the surface of object S with respect to the positions x and y on the object's surface. The positions x and y on the object's surface can be mapped one-to-one with the positions on the imaging plane. Therefore, the positions on the imaging plane are expressed here using the same terms as the positions x and y on the object's surface. In other words, by solving equation (1), the height h of the surface of object S can be expressed in terms of the positions x and y on the imaging plane. This has the effect of determining the three-dimensional shape of the surface of object S.

[0053] Furthermore, equation (1) can be calculated by applying the Fast Fourier Transform (FFT). This enables high-speed calculation. In other words, three-dimensional shape information can be calculated instantaneously from two components of the direction of light (slope angles θx and θy). Moreover, if the area around the target region (object point) is flat, three-dimensional shape information can be calculated using only one component of the direction of light (slope angle θx). That is, the height h at the position (x,y,h) of the object point on the surface of object S can be obtained from equation (1) as follows.

[0054]

number

[0055] Here, x = x0 is defined as the surrounding flat area, and the height there is set to 0. Equation (2) can also be reduced to simple arithmetic operations, and since the calculation can be parallelized, it can be calculated quickly. As a result, the height h at the position (x,y,h) of the object point on the surface of object S (height relative to the surrounding flat area) can be instantly determined from one component of the direction of light (tilt angle θx).

[0056] The calculation of the three-dimensional shape using the processing unit 14 is generally performed as shown in Figure 4.

[0057] The processing unit 14 acquires an image at the imaging unit 24 through an anisotropic wavelength selection unit 26 (ST1). The processing unit 14 acquires the image by forming an image on the image sensor 44 with light rays that have passed through the imaging optical element 42.

[0058] The acquired image is color-coded according to the direction of the light rays. The optical system 10 according to this embodiment can acquire the BRDF of the surface of object S, thereby determining whether or not there are minute defects. The processing device 14 calculates the direction component of light (tilt angle) θx corresponding to the hue (ST2).

[0059] Subsequently, the processing unit 14 obtains the height h based on equation (1) or equation (2) (ST3). At this time, the three-dimensional shape of the surface of object S is calculated. In the optical system 10 according to this embodiment, when the BRDF has a narrow distribution, that is, when the surface of object S is close to a mirror surface, the three-dimensional shape information of minute defects on the surface of object S can be measured.

[0060] The processing unit 14 according to this embodiment stores, for example, a program for calculating three-dimensional shape information, which includes capturing image data using the image sensor 44, obtaining the relationship between color and the direction of light rays (tilt angle θx (and θy)) based on the anisotropic wavelength selection unit 26, and calculating equations (1) and (2). In this embodiment, the processing unit 14 can perform the emission of light from the light source 32, image acquisition with the image sensor 44, and three-dimensional shape calculation of the image acquired by the image sensor 44 as a series of processes.

[0061] Therefore, the processor 61 of the processing unit 14 according to this embodiment reads a calculation program for three-dimensional shape information stored in a storage unit such as a ROM 62 or an auxiliary storage device 64 and executes processing to color-map the direction of light rays from an image captured through an anisotropic wavelength selection unit 26, and calculates the three-dimensional shape information of the surface of the object S from the geometric-optical relationship between the inclination angle θx of the surface of the object S and the direction of light rays.

[0062] The object S according to this embodiment may move in the x-axis direction relative to, for example, the imaging unit 24 and the wavelength selection unit 26. In this case, the emission timing of the light source 32 and the image acquisition timing are set as appropriate.

[0063] In Figure 1, the position of the wavelength selector 26 relative to the surface of object S is indicated by height (distance) l. If the x-axis component of the light tilt angle is θx, the x-component of the position where the light passes through the wavelength selector 26 can be expressed as follows.

[0064]

number

[0065] Equation (3) shows that increasing the height l increases x. In other words, even if the tilt angle θx of light is small, increasing the height l increases the absolute value of the position of x passing through the wavelength selection unit 26. This means that even minute tilt angles θx of light can be identified by increasing the height l of the wavelength selection unit 26. As a result, in the optical system 10 according to this embodiment, minute changes in BRDF can be captured by adjusting the height l. Furthermore, the optical system 10 according to this embodiment has the effect of being able to detect even minute defects. In terms of three-dimensional shape measurement, the optical system 10 according to this embodiment also has the effect of being able to measure the shape of even minute defects.

[0066] The height l of the wavelength selection unit 26 can be freely positioned without depending on the imaging optical element 42. That is, for example, if the focal length of the imaging optical element 42 is f, the height l may be made larger than the focal length f of the imaging optical element 42. In the optical system 10 according to this embodiment, the acquisition sensitivity of the BRDF can be increased without depending on the imaging optical element 42. In addition, in the optical system 10 according to this embodiment, the measurement accuracy of the three-dimensional shape can be improved without depending on the imaging optical element 42.

[0067] The optical device 12 according to this embodiment includes a support portion 72 that supports the outer edge of the wavelength selection portion 26, a first adjustment portion 74 that adjusts the distance between the support portion 72 and the surface of the object S to bring them closer or further apart, and a second adjustment portion 76 that allows the support portion 72 to rotate around the axis of the optical axis L.

[0068] The first adjustment unit 74 may be a servo motor or the like, and is preferably controlled by the processing unit 14 wirelessly or via a wire to control the arrangement of the support unit 72, that is, the arrangement of the wavelength selection unit 26 along the optical axis L in the z-axis direction. In this way, the distance between the anisotropic wavelength selection unit 26 and the surface of the object S can be optimized, and minute defects can be efficiently detected.

[0069] The second adjustment unit 76 preferably uses, for example, a servo motor and is controlled wirelessly or by wire by the processing unit 14 to control the arrangement of the support unit 72, that is, the arrangement of the wavelength selection unit 26 around the axis of the optical axis L (around the axis of the origin O). In this way, the second adjustment unit 76 can rotate the wavelength selection unit 26 to a desired angle, for example, around the axis of the optical axis L, relative to the imaging unit 24. If the BRDF has special anisotropy, the processing unit 14 can obtain an accurate BRDF distribution by rotating the wavelength selection unit 26 around the axis of the optical axis L using the second adjustment unit 76 while imaging the surface of the object S with the image sensor 44.

[0070] In this embodiment, since the wavelength selection unit 26 is positioned in front of the imaging unit 24, the optical device 12 can be assembled with any imaging unit (i.e., camera) 24. In other words, the optical device 12 has the advantage of a wide range of camera options.

[0071] In this embodiment, the wavelength selection unit 26 is described in an example in which a first wavelength selection region 52 is located off-center from the origin O, a second wavelength selection region 54 is located off-center from the origin O, and a third wavelength selection region 56 is located at the origin O. For example, the third wavelength selection region 56 may be formed as a light-shielding region so as not to transmit light of any wavelength. In this case, it is also preferable that the first wavelength selection region 52 and the second wavelength selection region 54 are located adjacent to each other with the origin O in between.

[0072] (Examples of application) Figure 5 shows the wavelength selection region (anisotropic wavelength selection area) that selects the transmitted / shielded wavelength spectrum across the Cartesian coordinate system x, and the relationship between the incident light, reflected light, and the tilt angle θx. The wavelength selection area 26 in Figure 5 is depicted as being divided into multiple (seven) wavelength selection regions. A rainbow filter is used in which the wavelengths that pass through are progressively longer from left to right, with the left end being blue, the origin O being green, and the right end being red. In practice, the wavelength selection area 26 may be formed with multiple colors, rather than just two wavelength selection regions (two colors) or three wavelength selection regions (three colors), as shown in Figure 5.

[0073] When the tilt angle θx = 0°, the light captured by the image sensor 44 through the wavelength selection unit 26 is acquired as green G light. When the tilt angle θx = -2°, the light captured by the image sensor 44 through the wavelength selection unit 26 is acquired as blue B light. When the tilt angle θx = +2°, the light captured by the image sensor 44 through the wavelength selection unit 26 is acquired as red R light.

[0074] Figure 6 shows an example of an aluminum plate with convex defects being photographed using the optical system 10 according to this embodiment.

[0075] The image sensor 44 acquires the hue of each pixel when acquiring the image data shown in Figure 6. The processing unit 14 calculates the directional component θx of light relative to a minute ridge from the hue of each pixel. The processing unit 14 calculates the directional component θx of light for all pixels. Then, using the above equation (2), the height h of the surface of object S is determined from the directional component θx of light. By assembling these values, the three-dimensional shape shown in Figure 7 is drawn. In Figure 7, the height h of the reproduced (reconstructed) three-dimensional shape is shown using color contour lines.

[0076] Thus, according to this embodiment, the direction of light rays can be color-mapped from an image captured through an anisotropic wavelength selection unit that selects wavelengths to block and wavelengths to pass through among the reflected light from the surface (object surface) of an object S illuminated by parallel light, depending on the rotation direction around the axis L of the optical axis. Three-dimensional shape information of the surface of the object S can then be calculated from the geometric-optical relationship between the inclination angle θx of the surface of the object S and the direction of light rays.

[0077] (Second embodiment) (Area sensor type) The optical system 10 according to the second embodiment will be described with reference to Figure 8. This embodiment is a modified example of the optical system 10 according to the first embodiment, and the same reference numerals are used as much as possible for components that are the same as or have the same function as those described in the first embodiment, and detailed descriptions are omitted.

[0078] Figure 8 shows a schematic cross-sectional view of the optical system 10 according to this embodiment in the xz plane.

[0079] The basic configuration of the optical device 12 of the optical system 10 in this embodiment is essentially the same as that of the optical device 12 of the optical system 10 in the first embodiment.

[0080] Unlike the optical device 12 described in the first embodiment, the wavelength selection unit (multi-wavelength aperture) 26 is positioned at the focal plane of the imaging optical element 42. That is, the wavelength selection unit 26 can be positioned within the imaging unit 24. The wavelength selection unit 26 is assumed to be anisotropic, as described in the first embodiment. Here, the wavelength selection unit 26 is formed at the origin O as a third wavelength selection region 56 that allows light rays of wavelengths corresponding to green G light to pass through and blocks light rays of other wavelengths. Adjacent to the third wavelength selection region 56 in the -x axis direction, a first wavelength selection region 52 is formed that allows light rays of wavelengths corresponding to blue light to pass through and blocks light rays of other wavelengths. Adjacent to the third wavelength selection region 56 in the +x axis direction, a second wavelength selection region 54 is formed that allows light rays of wavelengths corresponding to red light to pass through and blocks light rays of other wavelengths.

[0081] The image sensor 44 was configured as an area sensor.

[0082] The operation of the optical system 10 described above will now be explained.

[0083] The illumination unit 22 irradiates the surface of object S with substantially parallel light via the beam splitter 28.

[0084] Light rays propagating parallel to the optical axis L from a point on the surface of object S pass through the origin O of the wavelength selection unit 26 (the green region in Figure 8). On the other hand, light rays propagating obliquely from the point to the optical axis L pass through a position (a, b) on the wavelength selection unit 26 that is far from the origin O. Furthermore, depending on the region that passes through the wavelength selection unit 26, the light will have different wavelength spectra. When the light ray is projected onto the xz plane, the directional component of the light is the inclination angle θx with respect to the optical axis L. The directional component θx of the light is obtained by dividing the x-coordinate a of the position that passed through the wavelength selection unit 26 by the focal length f. At this time, the light ray will have different colors depending on the directional component θx. This relationship between the directional component of light and color is independent of the position of the point to the object and is always the same. Therefore, the directional component θx of light can be identified by color in all pixels of the acquired image. In other words, because the wavelength selection unit 26 is positioned at the focal plane of the imaging optical element 42, the directional component θx of light can be identified by color even if the object point is not on the optical axis L.

[0085] As described above, the optical system 10 according to this embodiment has the effect that even when the object point on the surface of the object S is not on the optical axis of the imaging optical element 42, the directional component θx of light can be identified by color.

[0086] Even when using the optical device 12 according to this embodiment, the first adjustment unit 74 and / or the second adjustment unit 76 described in the first embodiment can be used.

[0087] According to this embodiment, it is possible to provide a method for calculating three-dimensional shape information of an object surface, an optical system 10, a program for calculating three-dimensional shape information of the surface of an object S, and a processing device 14 for the optical system 10, which acquire three-dimensional shape information of the surface of an object S without spectrally separating the light rays on the illumination side.

[0088] (Third embodiment) (Line sensor type) The optical system 10 according to the third embodiment will be described with reference to Figure 9. This embodiment is a modified version of the optical system 10 according to the first and second embodiments, and the same reference numerals are used as much as possible for components that are the same as or have the same function as those described in the first and second embodiments, and detailed descriptions are omitted.

[0089] Figure 9 shows a perspective view of the optical system 10 according to this embodiment. The projection of the optical device 12 of this embodiment onto the first cross-section S1 in Figure 9 is basically the same as the optical device 12 of the first embodiment (see Figure 1).

[0090] First, in the optical device 12 of the optical system 10 according to this embodiment, the optical system was configured without using a beam splitter 28.

[0091] The image sensor 44 is a line sensor.

[0092] The first cross-section S1 is defined as a cross-section that includes the optical axis L of the imaging optical element 42 and is perpendicular to the longitudinal direction of the image sensor (line sensor) 44. In the first cross-section S1, the light projected onto this cross-section is parallel light. On the other hand, the second cross-section S2 is defined as a cross-section perpendicular to the first cross-section S1. In the second cross-section S2, the light projected onto this cross-section S2 may be parallel light or not; it may be diffuse light. Here, it is defined as diffuse light.

[0093] The wavelength selection unit 26 comprises multiple (for example, three) wavelength selection regions 52, 54, and 56. Each wavelength selection region 52, 54, and 56 intersects the x-axis and has a long stripe shape along the y-axis. In the first cross-section S1, the three wavelength selection regions 52, 54, and 56 are arranged. In other words, in the first cross-section S1, the projection of the wavelength selection regions 52, 54, and 56 of the wavelength selection unit 26 onto this cross-section S1 changes anisotropically with respect to the optical axis L. On the other hand, in the second cross-section S2, which is orthogonal to the first cross-section S1, the projection of the wavelength selection regions 52, 54, and 56 of the wavelength selection unit 26 onto this cross-section S1 does not change along the y-axis.

[0094] The illumination unit 22 illuminates the surface of object S, forming an illumination field F. The illumination field F of the illumination unit 22 is formed in a linear or rectangular shape on the surface of object S. A first object point within the illumination field F is imaged onto a first image point on the line sensor 44 by the imaging optical element 42. At the first object point, the BRDF becomes the first BRDF. The first ray is contained within the first BRDF.

[0095] In the first cross-section S1, the extent of the distribution of the first BRDF can be identified by the wavelength spectrum of light that has passed through the wavelength selection regions 52, 54, and 56 of the wavelength selection unit 26. When light reaches the image point in the line sensor 44, it is identified as a color corresponding to its wavelength spectrum. This allows the BRDF to be identified by color. Obtaining the BRDF has the effect of identifying the presence or absence of minute defects on the surface of object S. Furthermore, if the BRDF has a narrow distribution, the light tilt angle component θx can be obtained. Obtaining the light tilt angle component θx has the advantage of obtaining minute three-dimensional shape information of the surface of object S.

[0096] This embodiment uses a line sensor as the image sensor 44. The image sensor (line sensor) 44 has the characteristic of being able to acquire an image of the surface of an object S being transported in a predetermined direction at a predetermined speed with high accuracy. Therefore, by using the optical system 10 according to this embodiment, it is possible to inspect the surface of an object S being transported with high accuracy and to acquire three-dimensional shape information of the surface of the object S.

[0097] Furthermore, in this embodiment, by increasing the longitudinal length of the line sensor 44 and the illumination unit 22, it is possible to acquire images of a wide range of object surfaces S. For example, the longitudinal dimensions of the line sensor 44 and the illumination unit 22 can be increased from several hundred mm to several thousand mm. This is also true for the line sensor 44 in the first embodiment.

[0098] In this embodiment, since the wavelength selection unit 26 is positioned in front of the imaging unit 24, this optical system can be assembled for any imaging unit (i.e., camera) 24. In other words, it has the advantage of a wide range of camera options.

[0099] Furthermore, the wavelength selection unit 26 is supported by a support unit 72, and this support unit 72 is capable of rotating the wavelength selection unit 26 by a second adjustment unit 76. In this case, there is an advantage in that a highly accurate BRDF distribution can be obtained by imaging the surface of the object S in accordance with the rotation of the wavelength selection unit 26 in accordance with the rotation of the support unit 72. Alternatively, there is an advantage in that the tilt angle component of light can be obtained not only for the x-axis component θx but also for the y-axis component θy.

[0100] Thus, even when using the line sensor 44, the second adjustment unit 76 can be used.

[0101] In addition, when using this line sensor 44, the distance between the wavelength selection unit 26 and the surface of object S may also be adjusted by the first adjustment unit 74.

[0102] According to this embodiment, it is possible to provide a method for calculating three-dimensional shape information of an object surface, an optical system 10, a program for calculating three-dimensional shape information of the surface of an object S, and a processing device 14 for the optical system 10, which acquire three-dimensional shape information of the surface of an object S without spectrally separating the light rays on the illumination side.

[0103] According to at least one embodiment described above, it is possible to provide a method for calculating three-dimensional shape information of an object surface, an optical system 10, a program for calculating three-dimensional shape information of the surface of an object S, and a processing device 14 for the optical system 10, which acquire three-dimensional shape information of the surface of an object S without spectrally separating the light rays on the illumination side.

[0104] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. The claims of this application as they were at the time of filing are included below. [Note 1] An image is acquired through an anisotropic wavelength selection unit having at least two different regions for selecting wavelengths to block and wavelengths to pass through from the reflected light from the surface of an object illuminated by light. Color mapping of the light ray direction from the aforementioned image, The three-dimensional shape information of the object surface is calculated from the geometric-optical relationship between the inclination angle of the object surface and the direction of the light ray. A method for calculating three-dimensional shape information of an object's surface, including [specific details omitted]. [Note 2] The illumination is parallel light, and the parallel light is used to illuminate the object surface. A method for calculating the three-dimensional shape information of an object surface, including the method described in Appendix 1. [Note 3] The aforementioned geometric-optical relation is a partial differential equation obtained by partially differentiating the height h of the object surface with respect to its position on the imaging surface.

number

number

[0105] 10...Optical system, 12...Optical device, 14...Processing device, 22...Illumination unit, 24...Imaging unit, 26...Wavelength selection unit, 28...Beam splitter, 32...Light source, 34...Aperture, 36...Illumination lens, 42...Imaging optical element, 44...Image sensor (line sensor), 52...First wavelength selection region, 54...Second wavelength selection region, 56...Third wavelength selection region, 61...Processor, 72...Support unit, 74...First adjustment unit, 76...Second adjustment unit.

Claims

1. To acquire a single image captured through an anisotropic wavelength selection unit having at least two different regions for selecting wavelengths to block and wavelengths to pass through from the reflected light from the surface of an object illuminated by light. Color mapping of the direction of light rays from the aforementioned single image, The three-dimensional shape information of the object surface is calculated from the geometric-optical relationship between the inclination angle of the object surface and the direction of the light ray. A method for calculating three-dimensional shape information of an object's surface, including [specific details omitted].

2. The illumination is parallel light, and the parallel light is used to illuminate the object surface. A method for calculating three-dimensional shape information of an object surface according to claim 1, including the method described in claim 1.

3. To acquire an image captured through an anisotropic wavelength selection unit having at least two different regions for selecting wavelengths to block and wavelengths to pass through from the reflected light from the surface of an object illuminated with light, Color mapping of the light ray direction from the aforementioned image, The three-dimensional shape information of the object surface is calculated from the geometric-optical relationship between the inclination angle of the object surface and the direction of the light ray. Includes, The aforementioned geometric-optical relation is a partial differential equation obtained by partially differentiating the height h of the object surface with respect to its position on the imaging surface. [Math 1] And, Here, x, y, and h are the positions of the object points on the object surface. θx is the tilt angle of the x-component of specularly reflected light at the object point, θy is the tilt angle of the y-direction component of the specularly reflected light at the object point. A method for calculating three-dimensional shape information of an object's surface.

4. The aforementioned single image is acquired by forming an image on an image sensor by directing light rays that have passed through an imaging optical element. A method for calculating three-dimensional shape information according to claim 1 or claim 2.

5. To acquire an image captured through an anisotropic wavelength selection unit having at least two different regions for selecting wavelengths to block and wavelengths to pass through from the reflected light from the surface of an object illuminated with light, Color mapping of the light ray direction from the aforementioned image, The three-dimensional shape information of the object surface is calculated from the geometric-optical relationship between the inclination angle of the object surface and the direction of the light ray. Includes, The aforementioned image is acquired by imaging light rays that have passed through an imaging optical element onto an image sensor. A line sensor is used as the image sensor. As the anisotropic wavelength selection unit, a stripe shape parallel to the longitudinal direction of the line sensor is used. A method for calculating three-dimensional shape information of an object's surface.

6. The distance between the anisotropic wavelength selection unit and the object surface is adjusted. The method for calculating three-dimensional shape information according to claim 5.

7. An area sensor is used as the image sensor. The method for calculating three-dimensional shape information according to claim 4.

8. To acquire an image captured through an anisotropic wavelength selection unit having at least two different regions for selecting wavelengths to block and wavelengths to pass through from the reflected light from the surface of an object illuminated with light, Color mapping of the light ray direction from the aforementioned image, The three-dimensional shape information of the object surface is calculated from the geometric-optical relationship between the inclination angle of the object surface and the direction of the light ray. Includes, The anisotropic wavelength selection unit is rotated around the axis of the optical axis. A method for calculating three-dimensional shape information of an object's surface.

9. An image is captured through an anisotropic wavelength selection unit having at least two different regions that select wavelengths to block and wavelengths to pass through from the reflected light from the surface of an object illuminated by light. The direction of light rays is color-mapped from the aforementioned single image, The three-dimensional shape information of the object surface is calculated from the geometric-optical relationship between the inclination angle of the object surface and the direction of the light ray. Equipped with a processor, Optical system.

10. An image is captured through an anisotropic wavelength selection unit having at least two different regions for selecting wavelengths to block and wavelengths to pass through from the reflected light from the surface of an object illuminated with light, From the aforementioned image, the direction of the light rays is color-mapped, The three-dimensional shape information of the object surface is calculated from the geometric-optical relationship between the inclination angle of the object surface and the direction of the light ray. Equipped with a processor, The processor uses a partial differential equation obtained by partially differentiating the height h of the object surface with respect to its position on the imaging surface. [Math 2] The process is performed using the above geometric optical relation as the expression. Here, x, y, and h are the positions of the object points on the object surface. θx is the x-component of specularly reflected light at the object point, θy is the y-direction component of the specularly reflected light at the aforementioned point. Optical system.

11. It has a first adjustment unit that optimizes the distance between the anisotropic wavelength selection unit and the object surface, The optical system according to claim 9 or claim 10.

12. The system includes a support for the anisotropic wavelength selector and a second adjustment unit that allows the anisotropic wavelength selector to rotate around the axis of the optical axis. The optical system according to claim 9 or claim 10.

13. An illumination unit that illuminates the object surface with parallel light, An imaging unit directed towards the portion of the object surface illuminated by the parallel light, An anisotropic wavelength selection unit is provided between the object surface and the imaging unit, which selects wavelengths to block and wavelengths to pass through from the reflected light from the object surface, and which depends on the rotational direction of the optical axis. Having, The optical system according to claim 9 or claim 10.

14. The imaging unit uses a line sensor as a color image sensor. As the anisotropic wavelength selection unit, a stripe shape parallel to the longitudinal direction of the line sensor is used. The optical system according to claim 13.

15. To acquire a single image captured through an anisotropic wavelength selection unit having at least two different regions for selecting wavelengths to block and wavelengths to pass through from the reflected light from the surface of an object illuminated by light. Color mapping of the direction of light rays from the aforementioned single image. The three-dimensional shape information of the object surface is calculated from the geometric-optical relationship between the inclination angle of the object surface and the direction of the light rays. A program that causes a processor to perform calculations to determine the three-dimensional shape information of an object's surface.

16. A storage unit for storing the calculation program for three-dimensional shape information described in claim 15, A processor that reads the calculation program stored in the storage unit from the storage unit and executes it. A processing apparatus for optical systems having the following features.

Images