Surface shape measuring device and surface shape measuring method
Through the dual shooting system linkage and coordinate system transformation technology, the error problems caused by position deviation and vibration of the surface shape measurement device during the scanning process are solved, and high-precision measurement in a space-constrained environment is achieved.
Patent Information
- Application Number
- CN202380030106.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-03-25
- Filing Date
- 2023-03-15
- Publication Date
- 2024-11-26
AI Technical Summary
The existing surface shape measurement devices are susceptible to positional offsets and vibrations during scanning, resulting in measurement errors. Especially when the space is limited when the installation is in the processing machine and in the factory production line, it is difficult to install a vibration deflector and a windshield.
Using the method of linkage of dual shooting systems, the first shooting system scans and shoots the object in a vertical direction, and the second shooting system shoots and detects the displacement simultaneously, corrects the surface shape through coordinate system transformation information, detects the displacement using a single eye or stereo camera, and Accurate positioning by correcting target and feature point tracking.
It effectively reduces measurement errors caused by vibration, improves the accuracy and stability of surface shape measurement, and is suitable for space-constrained environments.
Smart Images

Figure CN119032252A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a surface shape measuring device and a surface shape measuring method. Background Art
[0002] It is known that a scanning measurement method is used to measure the three-dimensional shape (full focus image and surface shape, etc.) of the measurement surface of the measurement object using a scanning measurement device such as a focus variation (FV) microscope, a confocal microscope, a white light interference microscope, and an auto focus (AF) device (see patent documents 1 to 3). Such a measurement device causes a microscope with a camera to scan along a scanning direction while photographing the measurement surface with a camera at a constant interval, and based on the photographed images at each interval, the focus degree (focal position of the microscope) or the height information is calculated for each pixel of each photographed image, thereby measuring the three-dimensional shape of the measurement surface. These measurement devices can obtain the height profile of the measurement object in a planar manner, and therefore become very useful measurement devices when measuring fine three-dimensional shapes and roughness.
[0003] Prior Art Literature
[0004] Patent Literature
[0005] Patent Document 1: Japanese Patent Application Publication No. 2016-99213
[0006] Patent Document 2: Japanese Patent Application Publication No. 2015-84056
[0007] Patent Document 3: Japanese Patent Application Publication No. 2016-90520 Summary of the invention
[0008] Problems to be solved by the invention
[0009] However, in the above-mentioned measuring device, it is necessary to scan the optical system in the height direction, and there is a problem that if the position of the measuring object is shifted or vibrated during scanning, the amount of the position shift becomes a measurement error and is easily noticeable.
[0010] Therefore, it is considered to install the measuring device on a vibration isolation table in order to isolate the vibration from the floor, or to install a windshield around the measuring device in order to isolate the vibration caused by wind and sound.
[0011] However, there is a problem that the space for installing the vibration isolation table and the wind and soundproof cover is greatly restricted, and in particular, the vibration isolation table and the wind and soundproof cover cannot be installed in the measuring device in the processing machine or the factory production line.
[0012] The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a surface shape measuring device and a surface shape measuring method capable of reducing errors caused by the influence of vibrations generated during measurement.
[0013] Solutions to Solve Problems
[0014] A first embodiment relates to a surface shape measuring device that measures the surface shape of a measurement object, wherein the surface shape measuring device comprises: a first imaging system that images the measurement object at predetermined imaging intervals while performing relative scanning in a vertical direction relative to the measurement object; a second imaging system that is separate from the first imaging system and that images the measurement object or a support of the measurement object synchronously with the first imaging system; a computing unit that calculates the surface shape of the measurement object based on a plurality of first imaging images captured by the first imaging system; a storage unit that stores coordinate system transformation information for transforming a second coordinate system of the second imaging system into a first coordinate system of the first imaging system; a displacement detection unit that detects a displacement of the measurement object in imaging by the first imaging system based on a plurality of second imaging images captured by the second imaging system; and a correction unit that corrects the surface shape calculated by the computing unit based on a detection result of the displacement detection unit and the coordinate system transformation information.
[0015] In the surface shape measuring device according to the second aspect, the coordinate system transformation information is a transformation matrix for transforming the second coordinate system into the first coordinate system.
[0016] In the surface shape measurement device according to the third aspect, the surface shape measurement device includes a calibration unit that acquires coordinate system conversion information based on results of imaging the calibration target by the first imaging system and the second imaging system.
[0017] In the surface shape measuring device according to the fourth aspect, the second imaging system includes a single-lens camera, and the displacement detecting unit detects the displacement of the measurement object by a bundle adjustment method.
[0018] In the surface shape measuring device according to the fifth aspect, the second imaging system includes a compound-eye camera, and the displacement detecting unit detects the displacement of the measurement object by a stereo camera method.
[0019] In the surface shape measuring device according to the sixth aspect, when a marker is attached to the object to be measured or a support of the object to be measured, the displacement detecting unit detects the displacement of the object to be measured by tracking the marker.
[0020] In the surface shape measuring device according to the seventh aspect, when the object to be measured or a support for the object to be measured is not marked, the displacement detecting unit detects the displacement of the object to be measured by tracking a feature point set on the object to be measured or a support for the object to be measured.
[0021] In the surface shape measuring device according to the eighth aspect, the first imaging system is a microscope of any one of a white light interference type, a laser confocal type, and a focusing type.
[0022] The surface shape measurement method of the ninth scheme comprises: a first shooting step, which causes the first shooting system to shoot the measuring object at a prescribed shooting interval while performing relative scanning in a vertical direction relative to the measuring object; a second shooting step, which uses a second shooting system separate from the first shooting system to synchronously shoot the measuring object or the support of the measuring object with the first shooting system; an operation step, which calculates the surface shape of the measuring object based on a plurality of first shooting images captured in the first shooting step; a displacement detection step, which detects the displacement of the measuring object in the shooting of the first shooting system based on a plurality of second shooting images captured by the second shooting system; and a correction step, which corrects the surface shape calculated by the operation step based on the detection result of the displacement detection step and the coordinate system transformation information for transforming the second coordinate system of the second shooting system into the first coordinate system of the first shooting image.
[0023] Effects of the Invention
[0024] According to the present invention, it is possible to reduce errors caused by the influence of vibrations generated during measurement. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 This is a schematic diagram of the surface shape measuring device according to the first embodiment.
[0026] Figure 2 It is a diagram for explaining the first imaging system.
[0027] Figure 3 It is a diagram for explaining the second imaging system.
[0028] Figure 4 This is a functional block diagram of the control device in the surface shape measuring device according to the first embodiment.
[0029] Figure 5 This is a diagram for explaining optical flow.
[0030] Figure 6 This is a diagram for explaining the logo.
[0031] Figure 7 This is a flowchart showing an example of a surface shape measuring method.
[0032] Figure 8 It is a diagram for explaining the calibration prepared in advance.
[0033] Fig. 9 It is a schematic diagram of a surface shape measuring device according to a second embodiment. DETAILED DESCRIPTION
[0034] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.
[0035] <First Embodiment>
[0036] Figure 1 1 is a schematic diagram of the surface shape measuring device 1 according to the first embodiment. It should be noted that, among the XYZ directions orthogonal to each other in the figure, the XY direction is the horizontal direction, and the Z direction is the up-down direction (vertical direction).
[0037] like Figure 1 As shown in FIG. 1 , the surface shape measuring device 1 is a measuring device for measuring the surface shape of a measuring object W, and includes a first imaging system 10, a second imaging system 50 which is separate from the first imaging system 10, and a control device 90. Figure 1 In the embodiment, the object W to be measured is placed on the jig 72. The jig 72 is an example of a support for the object W to be measured according to the present invention. The support for the object W to be measured may be any object as long as it can support the object W to be measured, and its size, shape, etc. are not limited.
[0038] The first imaging system 10 images the object W at predetermined imaging intervals while performing relative scanning in a vertical direction with respect to the object W. In the embodiment, the first imaging system 10 is a white light interference microscope.
[0039] The second imaging system 50 images the measurement object W or the jig 72 in synchronization with the first imaging system 10. In the first embodiment, the second imaging system 50 includes two cameras 51 and 52 and is configured as a stereo camera (compound-eye camera).
[0040] The control device 90 is connected to the first imaging system 10 and the second imaging system 50, and comprehensively controls the surface shape measuring device 1 according to input operations performed on the operation unit 91. The display unit 92 displays various information under the control of the control device 90.
[0041] In the surface shape measuring device 1 of the first embodiment, the first imaging system 10 images the measurement object W, and the second imaging system 50 which is separate from the first imaging system 10 images the displacement of the measurement object W from the start of imaging by the first imaging system 10. Furthermore, the surface shape measuring device 1 calculates the surface shape of the measurement object W based on the plurality of images (the first images of the present invention) captured by the first imaging system 10. The surface shape measuring device 1 also corrects the surface shape of the measurement object W calculated as described above, based on the displacement (translation displacement and rotation displacement) detected based on the image (the second image of the present invention) captured by the second imaging system 50.
[0042] Here, in the surface shape measuring device 1, as a preliminary preparation for measurement, calibration is performed to obtain coordinate system transformation information (a coordinate system transformation matrix for transforming the second coordinate system of the second imaging system 50 into the first coordinate system of the first imaging system 10), and the coordinate system transformation information obtained by the calibration is stored in the storage unit 108 described later. Furthermore, when measuring the measurement object W, the surface shape measuring device 1 corrects the surface shape of the measurement object W based on the imaging results obtained by imaging the measurement object W by the first imaging system 10 and the second imaging system 50 and the coordinate system transformation information stored in the storage unit 108. It should be noted that the calibration of the surface shape measuring device 1 will be described later.
[0043] The surface shape measuring device 1 is calibrated in advance, so the relative positions of the first imaging system 10 and the second imaging system 50 are preferably the same during measurement as during calibration. Therefore, the first imaging system 10 and the second imaging system 50 are installed in the same system, such as the same stand.
[0044] Next, each configuration of the first imaging system 10 , the second imaging system 50 , and the control device 90 will be described.
[0045] <First Camera System>
[0046] Figure 2 1 is a diagram for explaining the first imaging system 10. Figure 2 As shown, the first imaging system 10 includes an optical head 12, a driving unit 16, an encoder 18, a stage 70, and a stage driving unit 74 for measuring the three-dimensional shape (surface shape) of the measuring surface of the measuring object W. The stage 70 is arranged below the optical head 12 in the Z direction.
[0047] Optical head 12 Figure 1 As shown, it is composed of a Michelson type white light interference microscope.
[0048] The optical head 12 includes a camera 14 , a light source unit 26 , a beam splitter 28 , an interference objective lens 30 , and an imaging lens 32 .
[0049] The interference objective lens 30, the beam splitter 28, the imaging lens 32, and the camera 14 are arranged in order along the upper side in the Z direction from the measuring object W. The light source unit 26 is arranged at a position facing the beam splitter 28 in the X direction (or the Y direction).
[0050] The light source unit 26 emits a parallel beam of white light (low coherence light with little coherence) as the measurement light L1 toward the beam splitter 28 under the control of the control device 90. The light source unit 26 includes a light source such as a light emitting diode, a semiconductor laser, a halogen lamp, or a high-intensity discharge lamp that can emit the measurement light L1, and a condenser that converts the measurement light L1 emitted from the light source into a parallel beam, although illustration thereof is omitted.
[0051] The beam splitter 28 uses, for example, a half mirror. The beam splitter 28 reflects a portion of the measurement light L1 incident from the light source unit 26 toward the interference objective lens 30 on the lower side in the Z direction. In addition, the beam splitter 28 transmits a portion of the combined light L3, which is described later, incident from the interference objective lens 30 toward the upper side in the Z direction, and emits the combined light L3 toward the imaging lens 32.
[0052] The interference objective lens 30 is a Michelson type, and includes an objective lens 30A, a beam splitter 30B, and a reference surface 30C. The beam splitter 30B and the objective lens 30A are arranged in order along the upper side in the Z direction from the measurement object W. In addition, the reference surface 30C is arranged at a position opposite to the beam splitter 30B in the X direction (or the Y direction).
[0053] The objective lens 30A has a focusing function, and focuses the measurement light L1 incident from the beam splitter 28 onto the measurement target object W through the beam splitter 30B.
[0054] The beam splitter 30B is, for example, a half mirror. The beam splitter 30B splits a portion of the measurement light L1 incident from the objective lens 30A into the reference light L2, transmits the remaining measurement light L1 to be emitted toward the measurement object W, and reflects the reference light L2 toward the reference surface 30C. The measurement light L1 that has transmitted the beam splitter 30B is irradiated onto the measurement object W, and then reflected by the measurement object W and returns to the beam splitter 30B.
[0055] The reference surface 30C uses, for example, a reflector to reflect the reference light L2 incident from the beam splitter 30B toward the beam splitter 30B. The position of the reference surface 30C in the X direction can be manually adjusted by a position adjustment mechanism (not shown). Thus, the optical path length of the reference light L2 between the beam splitter 30B and the reference surface 30C can be adjusted. The reference optical path length is adjusted to be consistent with (including approximately consistent with) the optical path length of the measurement light L1 between the beam splitter 30B and the measurement object W.
[0056] The beam splitter 30B generates a combined light L3 of the measurement light L1 returned from the measurement object W and the reference light L2 returned from the reference surface 30C, and emits the combined light L3 toward the objective lens 30A on the upper side in the Z direction. The combined light L3 passes through the objective lens 30A and the beam splitter 28 and enters the imaging lens 32. In the case of a white light interference microscope, the combined light L3 becomes interference light including interference fringes.
[0057] The imaging lens 32 forms an image of the combined light L3 incident from the beam splitter 28 on an imaging plane (not shown) of the camera 14 . Specifically, the imaging lens 32 forms an image of a point on the focal plane of the objective lens 30A as an image point on the imaging plane of the camera 14 .
[0058] The camera 14 has a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) type imaging element, which is not shown in the figure. The camera 14 captures a plurality of images of the combined light L3 formed on the imaging surface by the imaging lens 32 as an image of the measuring object W while scanning by the driving unit 16 .
[0059] The drive unit 16 is composed of a well-known linear motor or motor drive mechanism. The drive unit 16 holds the optical head 12 so that it can freely scan the measurement object W in the vertical scanning direction (the optical axis direction of the optical head 12), that is, the Z direction. Under the control of the control device 90, the drive unit 16 moves the optical head 12 relative to the measurement object W within the range of the set scanning speed and scanning direction.
[0060] It should be noted that the driving unit 16 only needs to be able to scan the optical head 12 relative to the measuring object W in the scanning direction. For example, the driving unit 16 may scan the stage 70 supporting the measuring object W in the scanning direction.
[0061] The workbench 70 has a workbench surface for supporting the measuring object W. The workbench surface is composed of a flat surface substantially parallel to the X direction and the Y direction. The workbench driving unit 74 is composed of a well-known linear motor or motor driving mechanism, and under the control of the control device 90, the workbench 70 is relatively horizontally moved relative to the optical head 12 in a plane perpendicular to the scanning direction (X direction and Y direction).
[0062] The encoder 18 is a position detection sensor that detects the scanning direction position of the optical head 12 relative to the measurement object W, and uses, for example, an optical linear encoder (also called a scale). The optical linear encoder is composed of, for example, a linear scale having slits formed at a constant interval, and a light receiving element and a light emitting element that are arranged opposite to each other with the linear scale interposed therebetween.
[0063] <Second Camera System>
[0064] Figure 3 is a diagram for explaining the second imaging system 50. Figure 3As shown, the second imaging system 50 includes two monocular cameras 51 and 52, which constitute a stereo camera. The cameras 51 and 52 include a CCD type or COMS type imaging element and a lens system. The cameras 51 and 52 of the second imaging system 50 photograph the measuring object W or the fixture 72 from a plurality of different directions while the first imaging system 10 photographs the measuring object W. The second imaging system 50 outputs the photographed stereo image SG to the control device 90 as a second photographed image. The stereo image SG is composed of a first stereo image SG1 photographed by the camera 51 and a second stereo image SG2 photographed by the camera 52. The stereo image SG includes binocular parallax, so the three-dimensional coordinates in the stereo space can be obtained by processing the control device 90 based on the stereo image SG.
[0065] As described above, the first imaging system 10 includes the camera 14, and the second imaging system 50 includes the cameras 51 and 52. However, the camera 14 and the cameras 51 and 52 have different characteristics for the purposes of the first imaging system 10 and the second imaging system 50. The first imaging system 10 measures the fine shape and roughness of the measurement object W, so the camera 14 needs to have a high resolution capable of distinguishing space. On the contrary, the camera 14 with a high resolution has a narrow field of view, and it is difficult to measure the displacement (translation displacement and rotation displacement) of the measurement object W in the imaging of the first imaging system 10.
[0066] The second imaging system 50 measures the displacement (translation displacement and rotation displacement) of the measurement object W during imaging by the first imaging system 10, so the cameras 51 and 52 do not need such high resolution as required for the camera 14 to be able to distinguish space. Since high resolution is not required, the cameras 51 and 52 have a wide field of view and can detect the displacement of the measurement object W during imaging by the first imaging system 10.
[0067] Therefore, the first imaging system 10 and the second imaging system 50 use cameras with different characteristics according to their purposes.
[0068] <Control device>
[0069] The control device 90 has a computing circuit composed of various processors and memories. The various processors include CPU (Central Processing Unit), GPU (Graphics Processing Unit), ASIC (Application Specific Integrated Circuit), and programmable logic devices [such as SPLD (Simple Programmable Logic Devices), CPLD (Complex Programmable Logic Device), and FPGA (Field Programmable Gate Arrays)]. It should be noted that the various functions of the control device 90 can be implemented by one processor or by multiple processors of the same or different types.
[0070] Figure 4 FIG. 1 is a functional block diagram of the control device 90 in the surface shape measuring device 1 of the first embodiment. Figure 4 As shown, the first imaging system 10 , the second imaging system 50 , and the operation unit 91 are connected to the control device 90 .
[0071] The control device 90 includes a first shooting system control unit 100, a computing unit 102, a second shooting system control unit 104, a displacement detection unit 106, a storage unit 108, a control unit 110, a correction unit 112, a calibration unit 114, and a measurement unit 116. The control unit 110 realizes the respective functions and performs processing by executing a control program (not shown) read from the storage unit 108. The control unit 110 controls the overall processing of the control device 90. In addition, the storage unit 108 stores various programs, measurement results, etc., and stores coordinate system transformation information described later. The coordinate system transformation information is a coordinate system transformation matrix for transforming from the second coordinate system to the first coordinate system. It should be noted that the coordinate system transformation information is not particularly limited as long as it can perform a coordinate system transformation from the second coordinate system to the first coordinate system, and can also be in a form other than a matrix. For example, it can also be a mathematical formula, a parameter expression, etc.
[0072] The first imaging system control unit 100 controls the camera 14, the drive unit 16, the light source unit 26, and the stage drive unit 74 of the first imaging system 10, and acquires a plurality of first captured images of the measuring object W. Specifically, after the first imaging system control unit 100 starts the emission of the measuring light L1 from the light source unit 26, it controls the drive unit 16 to scan the optical head 12 in the Z direction. In addition, while the drive unit 16 is scanning the optical head 12 in the Z direction, the first imaging system control unit 100 causes the camera 14 to capture the measuring object W at every predetermined capturing interval based on the detection result of the Z direction position of the optical head 12 detected by the encoder 18, and repeatedly performs the output of the first captured images to the control device 90.
[0073] The operation unit 102 detects the brightness value of each pixel of the first captured image in which interference fringes occur. Furthermore, the operation unit 102 compares the envelope of the brightness value based on the brightness value (interference signal) of each pixel at the same coordinate of each first captured image (the imaging element of the camera 14). The operation unit 102 determines the Z-direction position where the envelope becomes the maximum for each pixel at the same coordinate of the first captured image, thereby calculating the height information of the measurement object W for each pixel at the same coordinate, and calculates the surface shape (three-dimensional shape) of the measurement object W.
[0074] The second imaging system control unit 104 controls the cameras 51 and 52 of the second imaging system 50 to image the measurement object W in synchronization with the first imaging system 10 , and outputs the stereoscopic image SG as a second captured image to the control device 90 .
[0075] The displacement detection unit 106 obtains the coordinates before displacement (initial position) and after displacement (current position) of the focus point set on the measurement object W based on the second captured image of the second capturing system 50, and calculates (detects) a displacement matrix representing the displacement (translational displacement and rotational displacement) of the current position relative to the initial position.
[0076] Here, as setting modes of the focus point set on the measurement object W, there are (A) a mode in which no mark is attached to the measurement object W, and (B) a mode in which a mark is attached to the measurement object W. Each mode will be described below.
[0077] (A) Form in which no mark is attached to the measurement object W
[0078] When the first imaging system 10 captures an image of the measurement object W, if the measurement object W has a feature point that can be tracked with sufficient accuracy, the feature point is set as a focus point.
[0079] In a state where the measurement object W is not marked, the displacement of the measurement object W can be detected based on the second captured image captured by the second imaging system 50 by the optical flow method.
[0080] Figure 5 is a diagram used to illustrate optical flow. Figure 5 As shown, the optical flow refers to the displacement vector for each feature point on the measurement object W. In the optical flow method, a two-dimensional vector field representing the displacement vector can be obtained based on the assumption that the brightness of the object on the screen does not change between consecutive shooting frames and the surface of the measurement object W viewed by adjacent pixels moves in the same direction.
[0081] It should be noted that, when it is difficult to set feature points on the object W to be measured, feature points may be set on the jig 72 on which the object W to be measured is placed, and the displacement of the jig 72 may be detected by the optical flow method. In this case, the displacement of the jig 72 detected can be regarded as equivalent to the displacement of the object W to be measured.
[0082] (B) Form of attaching a mark to the measurement object W
[0083] When the first imaging system 10 images the object W, if the object W does not have a feature point that can be tracked with sufficient accuracy, a marker is attached to the object W. By tracking the marker attached to the object W as a focus point, the relative displacement of the object W can be detected.
[0084] Figure 6 is a diagram showing an example of a sign. Figure 6 In FIG. 6A , a two-dimensional barcode 75 is attached as a mark to the measuring object W. As the two-dimensional barcode 75 , for example, a QR code (registered trademark) 76 shown on the right side of the figure can be used. In addition, a Data Matrix (registered trademark) 77 shown on the left side of the figure can be used instead of the QR code 76.
[0085] exist Figure 6 In 6B, the two-dimensional barcode 75 is attached to the fixture 72 as a mark. In the case where the object W to be measured is small and a mark cannot be attached, the two-dimensional barcode 75 may be attached to the fixture 72, thereby detecting the displacement of the fixture 72 as the displacement of the object W to be measured. In the case where the two-dimensional barcode 75 is attached to the fixture 72, the two-dimensional barcode 75 is also attached to the object W to be measured ( Figure 6 6A) can also use QR code 76 or Data Matrix 77.
[0086] Although the two-dimensional barcode 75 is illustrated as an example of the mark, a simple circle or other graphic may be used as the mark. As long as the mark recognized at the start of measurement can be tracked, the two-dimensional barcode 75 or the graphic may be used.
[0087] When any of the above two forms (A) and (B) is selected as the form of setting the focus point on the measurement object W, the displacement detection unit 106 obtains the coordinates of the focus point set by the stereo camera method before and after the displacement (that is, the initial position and current position of the focus point) based on the shooting result of the second shooting system 50, and calculates the displacement matrix representing the displacement of the measurement object W in the shooting of the camera 14 of the first shooting system 10. It should be noted that this displacement matrix is calculated based on the shooting result of the second shooting system 50 and is determined based on the second coordinate system, and is hereinafter referred to as the "second coordinate system displacement matrix".
[0088] Here, a method of deriving the second coordinate system displacement matrix obtained by the displacement detection unit 106 will be described.
[0089] When the coordinates (initial positions) before the displacement of the zero frame of each focus point on the measurement object W obtained by using the cameras 51 and 52 of the second imaging system 50 are set as P i (0) Set the coordinates after displacement (current position) when measuring the nth frame as P i (n), according to the captured second image, P i (0) can be expressed by the following formula (1), P i (n) can be expressed by the following equation (2): The subscript i is the position number of the focus point, and the n in the parentheses indicates the frame number.
[0090] P i (0) = [X i (0), Y i (0), Z i (0), 1] T ···(1)
[0091] P i (n) = [X i (n), Y i (n), Z i (n), 1] T ···(2)
[0092] When the second coordinate system displacement matrix expressing the displacement (translation displacement and rotation displacement) of the measurement object W from the 0th frame to the nth frame is represented by M B (n), P i (0) and P i (n) can be expressed by the following formula (3).
[0093] P i (n) = M B (n)P i (0)···(3)
[0094] Therefore, for the four or more points of interest, P is investigated in advance as the coordinates before and after the displacement. i (0) and P i The change of (n) makes the following equation (4) hold.
[0095] [P0(n), P1(n), …, P M (n)]=M B (n)[P0(0), P1(0), …, P M (0)]···(4)
[0096] Moreover, regarding M B (n) By solving equation (4), the second coordinate system displacement matrix M representing the displacement (translation displacement and rotation displacement) of the measurement object W can be obtained: B (n).
[0097] Using the above-described derivation method, the displacement detection unit 106 obtains the coordinates (initial position and current position) before and after the displacement of the four or more focus points set at the measurement object W using the cameras 51 and 52 of the second imaging system 50, and calculates (detects) the second coordinate system displacement matrix M representing the amount of displacement (translation displacement and rotation displacement) of the measurement object W based on the obtained results. B It should be noted that the second coordinate system displacement matrix M calculated by the displacement detection unit 106 is B (n) is obtained based on the second coordinate system of the second imaging system 50 .
[0098] The correction unit 112 uses the second coordinate system displacement matrix M calculated by the displacement detection unit 106 B (n) to correct the surface shape of the measurement object W calculated by the calculation unit 102.
[0099] The correction unit 112 obtains the coordinate system transformation matrix M stored in the storage unit 108 ct (Coordinate system transformation information). Coordinate system transformation matrix M ct is a matrix for transforming the second coordinate system of the second photographing system 50 into the first coordinate system of the first photographing system 10. It should be noted that the coordinate system transformation matrix M ct The coordinate system transformation matrix M is obtained by calibration prepared in advance and stored in the storage unit 108 when measuring the object W. ct , see below.
[0100] The correction unit 112 sets the corrected coordinate group as P i When the coordinate group P of the surface of the object W is measured, i (n), the second coordinate system displacement matrix M detected by the displacement detection unit 106 B (n), and the coordinate system transformation matrix M ct , the corrected coordinate group P is calculated by the following formula (5): i Here, the coordinate group P i (n) is a coordinate group of the surface of the measurement object W calculated by the calculation unit 102 based on the image of the N-th frame captured by the first imaging system 10 .
[0101] P i =M ct M B (n)P i (n)···(5)
[0102] The corrected coordinate group P calculated by the correction unit 112 is i This is a coordinate group obtained by correcting errors due to displacement (translational displacement and rotational displacement) generated in the measurement object W during imaging by the first imaging system 10 .
[0103] <Surface shape measurement method>
[0104] Next, a surface shape measuring method performed using the surface shape measuring device 1 configured as described above will be described. Figure 7 This is a flowchart showing an example of a surface shape measuring method. Figure 7 The surface shape measuring method shown is roughly divided into a preliminary preparation step (step S1 to step S7) and a measurement step (step S8 to step S14). Each step will be described below.
[0105] <Preliminary preparation process>
[0106] In the surface shape measurement method of the present embodiment, a preliminary preparation step is performed before the measurement step. In the preliminary preparation step, a method for acquiring coordinate system transformation information (coordinate system transformation matrix M) is performed in order to transform the second coordinate system of the second imaging system 50 into the first coordinate system of the first imaging system 10. ct ) correction work.
[0107] Figure 8 This is a diagram for explaining the calibration work performed in the preliminary preparation process. Figure 8As shown in FIG. 1 , when the calibration operation is performed, a calibration target 80 is set instead of the measurement object W (step S1: calibration target setting process). In this example, four hemispheres 82 are set on the surface of the calibration target 80 facing upward in the Z direction (the surface to be measured facing the first imaging system 10), and a QR code 84 is attached to the side surface of the calibration target 80 (the surface facing the second imaging system 50). The four hemispheres 82 are used for calibration of the first imaging system 10, and the QR code 84 is used for calibration of the second imaging system 50.
[0108] Next, the calibration target 80 is measured in the same manner as when measuring the measurement object W (step S2: calibration target measurement process). Specifically, the calibration unit 114 controls the first imaging system control unit 100 to perform relative scanning in the vertical direction relative to the calibration target 80 while the first imaging system 10 captures the calibration target 80 at a predetermined imaging interval, and controls the second imaging system control unit 104 to capture the calibration target 80 by the second imaging system 50 in synchronization with the first imaging system 10.
[0109] Next, the calibration unit 114 determines whether the number of measurements on the calibration target 80 is the second time (step 3: calibration target judgment process). When the number of measurements on the calibration target 80 is the first time (in the case of "No" in step S3), the processing from step S2 to step S3 is repeated. On the other hand, when the number of measurements on the calibration target 80 is the second time (in the case of "Yes" in step S3), proceed to the next step S4. It should be noted that in this example, the measurement of the calibration target 80 is repeated twice, but it can also be repeated three times or more. The user can arbitrarily set the number of measurements in the calibration target judgment process (set to more than 2 times).
[0110] It should be noted that while step S2 is being performed (ie, while measurement is being performed on calibration target 80 ), both translational displacement and rotational displacement occur in calibration target 80 .
[0111] Next, the calibration unit 114 controls the computing unit 102 to calculate and store four calibration reference positions (three-dimensional coordinates) pre-set on the calibration target 80 based on the imaging results of the first imaging system 10 (step S4: first imaging system calibration reference position calculation process). In this example, as described above, four hemispheres 82 are formed on the calibration target 80, and the calibration unit 114 obtains the center coordinate positions of the four hemispheres 82 as four calibration reference positions ci(n) for each number of measurements. It should be noted that the subscript i of "ci(n)" represents the number of the calibration reference position (1 to 4), and the n in the brackets represents the number of measurements (1 to 2).
[0112] Here, the three-dimensional coordinates of each calibration reference position obtained in the first measurement are set as c i (0) When the three-dimensional coordinates of each calibration reference position obtained in the second measurement are denoted by ci(1), the following equation (6) holds.
[0113] [c0(1),c1(1),c2(1),c3(1)]=M A [c0(0), c1(0), c2(0), c3(0)]···(6)
[0114] M A is a displacement matrix (hereinafter referred to as “first coordinate system displacement matrix”) that expresses the displacement (translation displacement and rotation displacement) in the first coordinate system inherent to the first imaging system 10. That is, the first coordinate system displacement matrix M A It is a matrix indicating the correlation between the coordinate values of the first imaging system 10 obtained in the first measurement and the coordinate values of the first imaging system 10 obtained in the second measurement when the calibration target 80 is displaced between the first measurement and the second measurement.
[0115] The correction unit 114 is about M A By solving equation (6) determined based on the three-dimensional coordinates of each calibration reference position obtained in the first and second measurements as described above, M is obtained as the first coordinate system displacement matrix representing the displacement (translation displacement and rotation displacement) in the first coordinate system. A .
[0116] Next, the calibration unit 114 controls the displacement detection unit 106 to calculate and store a plurality of calibration reference positions (three-dimensional coordinates) preset for the calibration target 80 based on the imaging result of the second imaging system 50 (step S5 : second imaging system calibration reference position calculation step).
[0117] In this example, as described above, the QR code 84 is formed on the side of the calibration target 80, and the calibration unit 114 obtains four calibration reference positions d determined by the QR code 84 for each number of measurements as calibration reference positions. i (n). It should be noted that “d i The subscript i in "(n)" represents the number of the calibration reference position (1 to 4), and the n in the parentheses represents the number of measurements (1 to 2).
[0118] Here, the three-dimensional coordinates of each calibration reference position obtained in the first measurement are set as d i (0) The three-dimensional coordinates of each calibration reference position obtained in the second measurement are set as d i In the case of (1), the following formula (7) holds.
[0119] [d0(1), d1(1), d2(1), d3(1)]=M B [d0(0), d1(0), d2(0), d3(0)]···(7)
[0120] M B is a second coordinate system displacement matrix expressing the displacement (translation displacement and rotation displacement) in the second coordinate system inherent to the second imaging system 50. That is, the second coordinate system displacement matrix M B It is a matrix indicating the correlation between the coordinate values of the second imaging system 50 obtained in the first measurement and the coordinate values of the second imaging system 50 obtained in the second measurement when the calibration target 80 is displaced between the first measurement and the second measurement.
[0121] The correction unit 114 is about M B By solving equation (7) determined based on the three-dimensional coordinates of each calibration reference position obtained in the first and second measurements as described above, M is obtained as a second coordinate system displacement matrix representing the displacement (translation displacement and rotation displacement) within the second imaging system 50. B .
[0122] The displacement matrix M of the first coordinate system is obtained in this way. A and the second coordinate system displacement matrix M B is a displacement matrix when the same calibration target 80 is observed in different coordinate systems, and thus the following equation (8) holds.
[0123] M A =M ct M B ···(8)
[0124] Here, M ct is a transformation matrix for transforming the second coordinate system of the second imaging system 50 into the first coordinate system of the first imaging system 10. That is, by using the coordinate system transformation matrix M ct As described later (see equation (9)), the displacement (translational displacement and rotational displacement) of the measurement object W obtained by the second imaging system 50 can be transformed from the second coordinate system to the first coordinate system.
[0125] The correction unit 114 obtains the coordinate system transformation matrix M according to equation (8): ct , and store the result in the storage unit 108 (step S6: transformation matrix calculation process, step S7: transformation matrix storage process).
[0126] The above-mentioned preliminary preparation process is completed. It should be noted that after the coordinate system transformation matrix M is obtained by the correction unit 114 ct, and the result is stored in the storage unit 108, it is not necessary to perform the preparatory process. That is, when the relative position of the first imaging system 10 and the second imaging system 50 has not changed since the last calibration operation (for example, when the measurement process described later is continued, etc.), the coordinate system transformation matrix M ct Since the value is kept constant, the above-mentioned calibration operation can be omitted, and it is not necessary to perform the calibration operation every time measurement is performed.
[0127] <Measurement process>
[0128] like Figure 1 As shown in FIG. 1 , the object W to be measured is set at the measurement position of the surface shape measuring device 1 (step S8: object to be measured setting process). It should be noted that the object W to be measured may be in a form where no mark is attached to the object W to be measured or in a form where a mark is attached to the object W to be measured. In addition, the object W to be measured may be placed on the fixture 72 or may not be placed on the fixture 72.
[0129] Next, the set measurement object W is measured (step S9: measurement object measurement process). Specifically, the measurement unit 116 controls the first imaging system control unit 100 to perform relative scanning in the vertical direction relative to the measurement object W while the first imaging system 10 captures the measurement object W at a predetermined imaging interval, and controls the second imaging system control unit 104 to capture the measurement object W by the second imaging system 50 in synchronization with the first imaging system 10.
[0130] Next, the computing unit 102 calculates the surface shape of the object W (step S10: surface shape calculation process). Specifically, the computing unit 102 measures the three-dimensional shape (surface shape) of the measurement surface of the object W based on the imaging result of the first imaging system 10, and calculates the coordinate group P of the surface of the object W. i (n).
[0131] Next, the displacement detection unit 106 calculates the second coordinate system displacement matrix M based on the imaging result of the second imaging system 50. B (n) (Step S11: Second coordinate system displacement matrix calculation step) Specifically, the displacement detection unit 106 calculates the second coordinate system displacement matrix M based on equations (1) to (4): B (n).
[0132] Next, the correction unit 112 obtains the coordinate system transformation matrix M from the storage unit 108. ct (Step S12: Coordinate system transformation matrix acquisition step) ct It is acquired in the preliminary preparation process and stored in the storage unit 108 .
[0133] Next, the correction unit 112 uses the coordinate system transformation matrix M ct and the second coordinate system displacement matrix M B (n), to calculate the first coordinate system displacement matrix M A (n) (Step S13: First Coordinate System Displacement Matrix Calculation Step) Specifically, the correction unit 112 calculates the first coordinate system displacement matrix M according to the following equation (9): A (n).
[0134] M A =M ct M B ···(9)
[0135] Next, the correction unit 112 uses the first coordinate system displacement matrix M A (n) to correct the surface shape of the object to be measured (step S14: correction process). Specifically, the correction unit 112 corrects the surface shape of the object to be measured based on the coordinate group P i (n) and the first coordinate system displacement matrix M A (n) The corrected coordinate group P is calculated by the following formula (10): i As long as the corrected coordinate group P can be calculated i The processing ends.
[0136] P i =M A (n)P i (n)···(10)
[0137] In this embodiment, the surface shape of the measuring object W calculated based on the first image captured by the first shooting system 10 is corrected based on the displacement of the measuring object W detected based on the second image captured by the second shooting system 50 that is separate from the first shooting system 10, and the coordinate system transformation information used to transform the second coordinate system of the second shooting system 50 into the first coordinate system of the first shooting system 10, thereby enabling the surface shape to be measured with high precision.
[0138] <Second Embodiment>
[0139] The second embodiment will be described with reference to the drawings. It should be noted that the same reference numerals are given to parts having the same functions as those of the first embodiment described above, and detailed description of the parts will be omitted, and differences from the other embodiments will be mainly described.
[0140] Fig. 91 is a schematic diagram of a surface shape measuring device 2 for measuring the surface shape of a measuring object W. It should be noted that the structure of the second imaging system 50 is different between the surface shape measuring device 2 of the second embodiment and the surface shape measuring device 1 of the first embodiment. The second imaging system 50 of the second embodiment is composed of a camera 53 (monocular camera), which captures an image G as a second captured image according to a capturing instruction and outputs it to the control device 90. The second embodiment is particularly useful in the case where only a structure with a single camera can be selected due to space limitations. In addition, the processing of the displacement detection unit 106 is different between the first embodiment and the second embodiment.
[0141] The first imaging system 10 and the computing unit 102 of the second embodiment are the same as those of the first embodiment. In the second embodiment, as in the first embodiment, as the form of setting the focus point on the measurement object W, a form in which a mark is attached to the measurement object W and a form in which a mark is not attached to the measurement object W can be selected.
[0142] In the second embodiment, the displacement detection unit 106 detects displacement (translational displacement and rotational displacement) through the following steps, for example.
[0143] When the coordinates (initial positions) before the displacement of the zero frame of each focus point on the measurement object W obtained by using the camera 53 of the second imaging system 50 are denoted as p i (0), and the coordinates after displacement (current position) measured at the nth frame are set as p i (n), according to the second captured image captured by the camera 53 of the second capturing system 50, p i (0) can be expressed by the following formula (11), p i (n) can be expressed by the following equation (12): The subscript i is the position number of the focus point, and the n in the parenthesis indicates the frame number.
[0144] p i (0) = [x i (0), y i (0), 1] T ···(11)
[0145] p i (n) = [x i (n), y i (n), 1] T ···(12)
[0146] Get the coordinate p before displacement i (0) and the coordinates p after displacement i(n) as the two-dimensional coordinates in the image of the image (second captured image) captured by one camera 53. Therefore, unlike the first embodiment, the following processing is required.
[0147] First, consider that camera 53 captures point P in three-dimensional space i (n) and project it into a point p on the two-dimensional image i (n) When P in three-dimensional space i (n) Projection into two dimensions p i When the matrix of (n) is set to A[R(n)|t(n)], i (n) and p i (n)Then the following formula (13) holds.
[0148] p i (n)=A[R(n)|t(n)]P i (n)···(13)
[0149] Here, A is called an internal parameter or camera matrix, which is a 3×3 matrix that depends on the focal length of the lens and the number of pixels of the imaging element. R(n) is a 3×3 matrix representing rotational displacement, and t(n) is a 3×1 vector representing translational displacement. Here, [R(n)|t(n)] becomes a 3×4 rotation-translation matrix that represents rotational displacement and translational displacement in three-dimensional space.
[0150] The camera matrix A is generally expressed by the following equation (14), which represents the mapping from three-dimensional space to two-dimensional space.
[0151]
[0152] When R(n) is expressed using θ or the like, it is generalized to be the product of rotation matrices around the axes as in the following equation (15).
[0153] R(n)=R x (α)R y (β)R z (γ)···(15)
[0154] When R x (α), R y (β) and R z (γ) When the respective angles are θ, they can be expressed by the following equation (16).
[0155]
[0156] In addition, t(n) is a 3×1 vector indicating a translation displacement, and thus can be expressed by the following equation (17).
[0157] t(n)=[x,y,z] T ···(17)
[0158] Therefore, equation (13) becomes a matrix that maps points in three-dimensional space to two-dimensional space.
[0159] Next, a mode in which a mark is attached to the measurement object W and a mode in which a mark is not attached to the measurement object W will be described respectively.
[0160] First, the form of attaching a marker to the measurement object W is described. The camera matrix A can be obtained in advance by a known method called camera calibration. Therefore, when using a marker, the three-dimensional space P i (n) and two-dimensional p i (n) becomes a known value. As a result, in equation (13), the unknowns become R(n) and t(n), and the following equation (18) holds.
[0161] A -1 [p0(n), p1(n), …, p I (n)]=[R(n)|t(n)][P0(n), P1(n),…,P I (n)]···(18)
[0162] Furthermore, by solving equation (18) with respect to R(n)|t(n)], the rotation and translation matrix [R(n)|t(n)] can be obtained.
[0163] Next, a description will be given of a mode in which no mark is attached to the measurement object W. In the mode in which no mark is used, in equation (13), the unknowns include P in addition to R(n) and t(n). i (n) is an unknown number, so equation (13) cannot be solved directly. As a result, the rotation and translation matrix [R(n)|t(n)] cannot be directly obtained.
[0164] Therefore, by applying the bundle adjustment method, we can obtain the relative R(n), t(n) and P from n = 0. i (n) It should be noted that the bundle adjustment method exists as a known technology (Iwamoto Yuki, Sugaya Yasuyuki, Kanaya Kenichi. "Installation and evaluation of bundle adjustment method for three-dimensional restoration." Computer Vision and Imaging Media (CVIM) 2011.19(2011): 1-8.).
[0165] In any of the forms in which the measured object W is marked and in which the measured object W is not marked, the rotation and translation matrix [R(n)|t(n)] can be obtained. Then, using the rotation and translation matrix [R(n)|t(n)] obtained by the above method, the displacement detection unit 106 calculates (detects) the second coordinate system displacement matrix M by the following formula (19): B (n) is used as the displacement (translational displacement and rotational displacement) of the measurement object W.
[0166] M B (n) = [R(n) | t(n); 0 | 1] -1 ···(19)
[0167] As in the first embodiment, the correction unit 112 uses the second coordinate system displacement matrix M calculated by the displacement detection unit 106 B (n), and the coordinate system transformation matrix M stored in the storage unit 108 ct , to correct the surface shape of the object W calculated by the calculation unit 102. Specifically, the correction unit 112 calculates the coordinate group P of the surface of the object W by the following equation (20): i (n) Calculate the corrected coordinate group P i .
[0168] P i =M ct M B (n)P i (n)···(20)
[0169] In the second embodiment, the preparatory process is also carried out in the same manner as in the first embodiment. Fig. 9 The surface shape measuring device 2 is shown.
[0170] In the second embodiment in which the second shooting system 50 is composed of one camera 53, similarly to the first embodiment, the surface shape of the measuring object W calculated based on the first shooting image captured by the first shooting system 10 is corrected based on the displacement of the measuring object W detected based on the second shooting image captured by the second shooting system 50 which is separate from the first shooting system 10, and the coordinate system transformation information for transforming the second coordinate system of the second shooting system 50 into the first coordinate system of the first shooting system 10, thereby enabling the surface shape to be measured with high precision.
[0171] It should be noted that the optical head 12 of the first shooting system 10 is described as a Michelson type white light interference microscope, but it can also be a Mirau type white light interference microscope or a Linnik type white light interference microscope. In addition, the optical head 12 can also be a microscope of any type, such as a laser confocal type or a focusing type.
[0172] Description of reference numerals:
[0173] 1…surface shape measuring device, 2…surface shape measuring device, 10…first imaging system, 12…optical head, 14…camera, 16…driving unit, 18…encoder, 26…light source unit, 28…beam splitter, 30…interference objective lens, 30A…objective lens, 30B…beam splitter, 30C…reference surface, 32…imaging lens, 50…second imaging system, 51, 52, 53…camera, 70…workbench, 72…fixture, 74…workbench driving unit, 75…two-dimensional barcode, 76…QR code, 77…Data Matrix, 80…calibration target, 82…hemisphere, 84…QR code, 90…control device, 91…operation unit, 92…display unit, 100…first imaging system control unit, 102…calculation unit, 104…second imaging system control unit, 106…displacement detection unit, 108…storage unit, 110…control unit, 112…correction unit, 114…calibration unit, 116…measurement unit, W…measurement object.
Claims
1. A surface shape measuring device for measuring the surface shape of a measurement object, wherein: The surface shape measuring device comprises: a first imaging system for imaging the object to be measured at predetermined imaging intervals while performing relative scanning in a vertical direction with respect to the object to be measured; a second imaging system which is separate from the first imaging system and which images the object to be measured or a support for the object to be measured synchronously with the first imaging system; a computing unit configured to calculate a surface shape of the object to be measured based on a plurality of first captured images captured by the first imaging system; a storage unit storing coordinate system transformation information for transforming a second coordinate system of the second photographing system into a first coordinate system of the first photographing system; a displacement detection unit configured to detect a displacement of the measurement object in the imaging by the first imaging system based on a plurality of second imaging images captured by the second imaging system; as well as A correction unit corrects the surface shape calculated by the calculation unit based on the detection result of the displacement detection unit and the coordinate system conversion information.
2. The surface shape measuring device according to claim 1, wherein: The coordinate system transformation information is a transformation matrix for transforming the second coordinate system into the first coordinate system.
3. The surface shape measuring device according to claim 1 or 2, wherein: The surface shape measuring device includes a calibration unit that acquires the coordinate system conversion information based on a result of imaging of a calibration target by the first imaging system and the second imaging system.
4. The surface shape measuring device according to any one of claims 1 to 3, wherein: The second imaging system includes a single-lens camera, and the displacement detection unit detects the displacement of the measurement object by a bundle adjustment method.
5. The surface shape measuring device according to any one of claims 1 to 3, wherein: The second imaging system includes a compound-eye camera, and the displacement detection unit detects the displacement of the measurement object using a stereo camera method.
6. The surface shape measuring device according to any one of claims 1 to 5, wherein: When a marker is attached to the object to be measured or a support of the object to be measured, the displacement detection unit detects the displacement of the object to be measured by tracking the marker.
7. The surface shape measuring device according to any one of claims 1 to 5, wherein: When the object to be measured or a support for the object to be measured is not provided with a mark, the displacement detection unit detects the displacement of the object to be measured by tracking a feature point set on the object to be measured or a support for the object to be measured.
8. The surface shape measuring device according to any one of claims 1 to 7, wherein: The first imaging system is a microscope of any one of a white light interference method, a laser confocal method and a focusing method.
9. A method for measuring a surface shape, wherein: The surface shape measuring method comprises: a first imaging step of causing the first imaging system to perform relative scanning in a vertical direction relative to the object to be measured while imaging the object to be measured at predetermined imaging intervals; a second photographing step of photographing the object to be measured or a support for the object to be measured synchronously with the first photographing system using a second photographing system that is separate from the first photographing system; a calculation step of calculating a surface shape of the object to be measured based on the plurality of first captured images captured in the first capturing step; a displacement detecting step of detecting a displacement of the measuring object in the imaging of the first imaging system based on a plurality of second imaging images captured by the second imaging system; as well as A correction step of correcting the surface shape calculated in the calculation step based on the detection result of the displacement detection step and coordinate system conversion information for converting the second coordinate system of the second imaging system into the first coordinate system of the first imaging system.
