3D Shape Data Generation Device

Abstract

To suppress an error between measurements when a plurality of workpieces having the same shape are measured.SOLUTION: A three-dimensional shape data generation apparatus 1 includes: a storage section 304 that stores a plurality of measurement files in which each of measurement conditions of a workpiece and each of alignment images are associated with each other; a display control section 305 that causes a display section to display a live image of the workpiece and an alignment image associated with one measurement file from among the plurality of measurement files stored in the storage section 304; a reception section 301b that receives a measurement start instruction of the workpiece; and a measurement control section 130 that controls a structured illumination section 110 and an imaging section 120 on the basis of the measurement condition associated with the one measurement file in response to the measurement start instruction.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to a three-dimensional shape data generation device that generates three-dimensional shape data of a workpiece.

Background Art

[0002] Conventionally, a three-dimensional shape data generation device that generates three-dimensional shape data of a workpiece placed on a stage has been known. The three-dimensional shape data generation device disclosed in Patent Document 1 irradiates structured light having a predetermined pattern onto a workpiece placed on a stage, receives the structured light reflected by the workpiece with an imaging unit to generate pattern image data of the workpiece, and is configured to generate three-dimensional shape data of the workpiece based on the generated pattern image data.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, when operating the three-dimensional shape data generation device, for example, when measuring the shape data of the entire circumference of a workpiece, it is necessary to reposition the workpiece in different poses and image it multiple times so that there are no blind spots. Then, when it is desired to measure a plurality of workpieces of the same shape, it is necessary to execute the same measurement procedure for each workpiece.

[0005] One method for measuring a workpiece is to attach alignment markers to it. However, when measuring multiple workpieces of the same shape, or when the same person is performing the measurement but there is a long gap since the last measurement, variations in the number of alignment markers and their placement on the workpiece can occur, making it difficult to obtain stable measurement results. Furthermore, differences in the workpiece's orientation on the stage can easily lead to measurement errors stemming from the measurement principle.

[0006] While alignment markers are sometimes not used when measuring workpieces, this increases the likelihood of rework due to incorrect workpiece orientation during measurement. Furthermore, similar to the case where alignment markers are used, the differing orientations on the stage can easily lead to measurement errors stemming from the measurement principle.

[0007] Furthermore, when creating a single 3D shape data by combining multiple image data obtained from multiple measurements of a workpiece in different orientations, it is necessary to align the image data from different orientations, but performing the same alignment for each of multiple workpieces is difficult. Moreover, when imaging a workpiece from an oblique direction, all six degrees of freedom must be determined to align the relative position of the workpiece and the imaging unit, and performing this for each of multiple workpieces individually is extremely difficult.

[0008] This disclosure is made in view of the above points, and its purpose is to suppress measurement errors when measuring multiple workpieces of the same shape. [Means for solving the problem]

[0009] To achieve the above objective, this embodiment may be based on a three-dimensional shape data generation device that includes an imaging unit with a field of view that receives structured light irradiated by a structured illumination unit and reflected by the workpiece, and generates pattern image data of the workpiece, and generates three-dimensional shape data of the workpiece based on the pattern image data generated by the imaging unit. The three-dimensional shape data generation device can store multiple measurement files in a storage unit, which associate the measurement conditions of the workpiece with alignment images used to position the workpiece in a predetermined location before it is imaged by the imaging unit. From among the multiple measurement files stored in this storage unit, it is possible to display the alignment image associated with one measurement file and a live image of the workpiece on a display unit, and it is also possible to receive a measurement start instruction for the workpiece, and in response to the received measurement start instruction, it is possible to control the structured illumination unit and the imaging unit based on the measurement conditions associated with one measurement file.

[0010] With this configuration, for example, a measurement file can be stored in the memory unit that associates the measurement conditions applied to the workpiece during a previous measurement with the alignment image of the workpiece acquired during that previous measurement. Therefore, when measuring a workpiece of the same shape as one measured in the past, the measurement file can be read, and the alignment image and a live image of the workpiece to be measured can be displayed on the display unit. This allows the user to accurately position the workpiece to be measured while looking at the display unit, thus reducing the likelihood of measurement errors. It also reduces the likelihood of incorrect orientation of the workpiece to be measured, thus suppressing rework. Note that the same shape does not need to be strictly identical; for example, slight differences in shape due to manufacturing errors or differences within tolerance ranges are also included in the definition of the same shape.

[0011] Furthermore, the reception unit may be capable of receiving the setting of measurement conditions for the master workpiece and the instruction to start measurement. In this case, the imaging unit generates the alignment image in accordance with the instruction to start measurement received by the reception unit, and the storage unit can store a measurement file that associates the measurement conditions received by the reception unit with the alignment image generated by the imaging unit. As a result, since the measurement of the master workpiece and the measurement of a workpiece other than the master workpiece are performed by the same 3D shape data generation device, the same measurement conditions can be reproduced, and errors between measurements can be further suppressed.

[0012] Furthermore, the display control unit can also display a alignment image associated with the first measurement file and a live image of the workpiece on the display unit. In this case, the reception unit receives a first measurement start instruction for the workpiece, and the measurement control unit identifies the measurement conditions associated with the first measurement file in response to the first measurement start instruction for the workpiece, and controls the structured illumination unit and the imaging unit based on the identified measurement conditions. The three-dimensional shape data generation unit generates first three-dimensional shape data of the workpiece based on the pattern image data generated by the imaging unit, with the structured illumination unit and the imaging unit controlled based on the measurement conditions identified by the measurement control unit.

[0013] Furthermore, the display control unit can also display a alignment image associated with the second measurement file and a live image of the workpiece on the display unit. In this case, the reception unit receives a second measurement start instruction for the workpiece, and the measurement control unit identifies the measurement conditions associated with the second measurement file in response to the second measurement start instruction for the workpiece, and controls the structured illumination unit and the imaging unit based on the identified measurement conditions. The three-dimensional shape data generation unit generates second three-dimensional shape data of the workpiece based on the pattern image data generated by the imaging unit, with the structured illumination unit and the imaging unit controlled based on the measurement conditions identified by the measurement control unit. By combining the first three-dimensional shape data of the workpiece and the second three-dimensional shape data of the workpiece based on the alignment information, composite three-dimensional shape data of the workpiece can be generated.

[0014] In other words, by using alignment images, the positional deviation of the workpiece relative to the master workpiece can be suppressed, and the alignment information of the master workpiece can be used to align the first 3D shape data of the workpiece with the second 3D shape data of the workpiece. This eliminates the need to set the alignment for each measurement, and allows for the reproduction of measurement results with higher consistency. [Effects of the Invention]

[0015] As explained above, the 3D shape data generation device according to this disclosure displays both a positioning image acquired during a previous measurement and a live image of the workpiece to be measured on the display unit, allowing the user to accurately place the workpiece in a predetermined position. This makes it possible to suppress measurement errors when measuring multiple workpieces of the same shape. [Brief explanation of the drawing]

[0016] [Figure 1] This figure shows the overall configuration of a three-dimensional shape data generation device according to Embodiment 1 of the present invention. [Figure 2] This is a block diagram of a 3D shape data generation device. [Figure 3] This figure shows the process of repositioning the workpiece in a different orientation and then taking an image. [Figure 4] This flowchart shows an example of the procedure for the initial measurement process of a workpiece. [Figure 5] This figure shows an example of a user interface screen for measurement settings. [Figure 6] This flowchart shows an example of the data synthesis process. [Figure 7] This figure shows an example of a user interface screen for data synthesis. [Figure 8] This is a diagram showing the data structure. [Figure 9] This flowchart shows an example of the procedure for processing when measurement results are reproduced. [Figure 10]It is a diagram showing an example of a user interface screen for data selection. [Figure 11] It is a diagram showing an example of a user interface screen for measurement. [Figure 12] It is a diagram corresponding to FIG. 11 in which the alignment image is displayed in an overlay manner. [Figure 13] It is a flowchart showing an example of the procedure of data synthesis processing during measurement reproduction. [Figure 14] It is a diagram corresponding to FIG. 7 during measurement reproduction. [Figure 15] It is a diagram corresponding to FIG. 10 when alignment markers are provided. [Figure 16] It is a diagram showing the overall configuration of the three-dimensional shape data generation device according to Embodiment 2 of the present invention.

Mode for Carrying Out the Invention

[0017] Hereinafter, embodiments of the present invention will be described in detail based on the drawings. Note that the following description of the preferred embodiments is merely illustrative in nature and is not intended to limit the present invention, its applications, or its uses.

[0018] (Embodiment 1) FIG. 1 is a diagram showing the overall configuration of the three-dimensional shape data generation device 1 according to Embodiment 1 of the present invention. The three-dimensional shape data generation device 1 is a system that generates three-dimensional shape data of a workpiece (measurement object) W. For example, it is possible to convert the mesh data of the workpiece W obtained by measuring the shape of the workpiece W into CAD data and output it.

[0019] Although not particularly limited, the three-dimensional shape data generation device 1 is used, for example, when acquiring CAD data of an existing product and performing next-generation model development or shape analysis on CAD / CAE, reflecting the shape of a model or mock in product design, designing a mating product based on the shape of the mating component, or performing an improved design based on the shape of a prototype. Therefore, examples of the workpiece W include existing products, models, mocks, prototypes, and the like.

[0020] Furthermore, the 3D shape data generation device 1 can also convert the mesh data of the workpiece W into surface data and output it. By converting the mesh data of the workpiece W into surface data and outputting it, it can support the user's reverse engineering process and reverse engineering work, so the 3D shape data generation device 1 can also be called a reverse engineering support device.

[0021] In the following description, when measuring the shape of workpiece W, coordinate information of the workpiece W surface is obtained by irradiating workpiece W with structured light for measurement and obtaining coordinate information based on the structured light reflected from the surface of workpiece W. For example, a measurement method using triangulation with a fringe projection image obtained from the structured light reflected from the surface of workpiece W can be applied. However, in this invention, the principle and configuration for obtaining the coordinate information of workpiece W are not limited to this, and other methods can also be applied.

[0022] The 3D shape data generation device 1 comprises a measuring unit 100, a base unit 200, a controller 300, a display unit 400, and an operation unit 500. The measuring unit 100 and the controller 300 may be connected by a communication cable or the like, or by a wireless connection.

[0023] As shown in Figure 2, the measurement unit 100 includes a structured illumination unit 110 and an imaging unit 120, as well as a housing 100A to which the structured illumination unit 110 and the imaging unit 120 are mounted. Furthermore, the measurement unit 100 also includes a measurement control unit 130 that controls the structured illumination unit 110 and the imaging unit 120. The measurement control unit 130 may be located in the housing 100A or on the controller 300 side.

[0024] The housing 100A is separate from the controller 300 and is supported by a support unit 600. The support unit 600 is portable and comprises a base unit 601, an extendable unit 602 fixed to the base unit 601, and an angle adjustment unit 603 provided on the upper part of the extendable unit 602, allowing the user to freely set the installation position. The measuring unit 100 is detachably attached to the angle adjustment unit 603. The height of the measuring unit 100 can be adjusted by extending or retracting the extendable unit 602 in the vertical direction. Furthermore, the angle adjustment unit 603 is configured to allow adjustment of, for example, rotation around the horizontal axis, rotation around the vertical axis, and rotation around the inclination axis. This allows the installation angle of the measuring unit 100 relative to the horizontal plane and the installation angle relative to the vertical plane to be adjusted arbitrarily.

[0025] The support section 600 is not limited to the configuration described above, and may be composed of, for example, a tripod, a flexible arm that can be freely bent and maintain a bent shape, a bracket, or a combination of these. The measuring section 100 can also be used by attaching it to, for example, a 6-degree-of-freedom arm of an industrial robot. Furthermore, the measuring section 100 can be used by the user holding it by hand, in which case the support section 600 is unnecessary. In other words, the support section 600 may be a component included in the 3D shape data generation device 1, or it may be a component not included in the 3D shape data generation device 1.

[0026] When a user takes the housing 100A and measures the workpiece W, the measuring unit 100 can be brought to the manufacturing site of the workpiece W for measurement. In this case, the user can measure the shape of the workpiece W by moving the measuring unit 100 to any position and taking images at any time. This can be called manual measurement.

[0027] Furthermore, by supporting the measuring unit 100 with the support unit 600 and placing the workpiece W on the automatically rotating base unit 200 (described later), the shape of a wide area of ​​the workpiece W can be measured by rotating the workpiece W on the base unit 200 and taking images at predetermined timings. This can be called semi-automatic measurement. Note that the workpiece W can also be measured by placing it on a surface plate or the like, for example, without placing it on the base unit 200.

[0028] Furthermore, by attaching the measuring unit 100 to the arm of an industrial robot and moving it, the shape of a wide range of the workpiece W can be measured without the user's intervention. This can be called fully automatic measurement. The present invention is applicable to all manual, semi-automatic, and fully automatic measurements.

[0029] As shown in Figure 2, the measurement unit 100 includes a structured illumination unit 110 that irradiates structured light for measurement onto the workpiece W, and an imaging unit 120 having a field of view that receives the structured light irradiated by the structured illumination unit 110 and reflected by the workpiece W, and generates pattern image data of the workpiece W. The measurement unit 100 may include a plurality of structured illumination units 110. For example, there may be a first structured illumination unit capable of irradiating first structured light onto the workpiece W from a first direction, and a second structured illumination unit capable of irradiating second structured light onto the workpiece W from a second direction different from the first direction. The measurement unit 100 may also include a plurality of imaging units 120.

[0030] Although not shown in the figures, it is also possible to have three or more structured illumination units 110, or to move the structured illumination unit 110 and the base unit 200 relative to each other, so that even while using a common structured illumination unit 110, the direction of illumination of the structured light can be different and projected onto the workpiece W. In addition to providing multiple structured illumination units 110 and receiving the light with a common imaging unit 120, it is also possible to provide multiple imaging units 120 for a common structured illumination unit 110 and configure them to receive the light. Furthermore, the illumination angle of the structured light projected by the structured illumination unit 110 with respect to the Z direction may be fixed or variable.

[0031] The structured illumination unit 110 includes a measurement light source 111, a pattern generation unit 112, and a plurality of lenses 113. The measurement light source 111 can be a light source that emits monochromatic light, such as a halogen lamp that emits white light, a blue LED (light-emitting diode) that emits blue light, or an organic EL. The light emitted from the measurement light source 111 is focused and then incident on the pattern generation unit 112.

[0032] The pattern generation unit 112 reflects the light emitted from the measurement light source 111 so that structured light is irradiated onto the workpiece W. The measurement light incident on the pattern generation unit 112 is converted into a preset pattern and a preset intensity (brightness) and emitted. The structured light emitted by the pattern generation unit 112 is converted by multiple lenses 113 into light with a diameter larger than the observation and measurement field of view of the imaging unit 120, and then irradiated onto the workpiece W.

[0033] The pattern generation unit 112 is a component that can switch between an irradiation state in which structured light is irradiated onto the workpiece W and a non-irradiation state in which structured light is not irradiated onto the workpiece W. For example, a DMD (Digital Micromirror Device) can be used for such a pattern generation unit 112. A pattern generation unit 112 using a DMD can be controlled by the measurement control unit 130 to switch between a reflection state in which structured light is reflected towards the optical path as the irradiation state and a light-shielding state in which structured light is blocked as the non-irradiation state.

[0034] A DMD is an element in which numerous micromirrors (tiny mirror surfaces) are arranged on a plane. Each micromirror can be individually switched ON or OFF by the measurement control unit 130, so by combining the ON and OFF states of numerous micromirrors, it is possible to generate light with a desired projection pattern as structured light for measurement. This makes it possible to generate the pattern necessary for triangulation and measure the shape of the workpiece W. In this way, the DMD functions as part of the optical system that illuminates the workpiece W with a periodic projection pattern for measurement during measurement. Furthermore, the DMD has excellent response speed and offers the advantage of being able to operate at a higher speed than shutters and the like.

[0035] In the above example, an example using a DMD for the pattern generation unit 112 was described, but the present invention is not limited to a DMD for the pattern generation unit 112, and other materials can be used. For example, an LCOS (Liquid Crystal on Silicon: reflective liquid crystal element) may be used as the pattern generation unit 112. Alternatively, a transmissive material may be used instead of a reflective material to adjust the amount of structured light transmitted. In this case, the pattern generation unit 112 is placed on the optical path and switches between an illumination state that transmits light and a light-blocking state that blocks light. For example, an LCD (liquid crystal display) can be used for such a pattern generation unit 112. Alternatively, the pattern generation unit 112 may be configured using a projection method using multiple line LEDs, a projection method using multiple optical paths, an optical scanner method composed of a laser and a galvanometer mirror, an AFI (Accordion fringe interferometry) method that uses interference fringes generated by superimposing beams divided by a beam splitter, or a projection method using a physical grid composed of a piezo stage and a high-resolution encoder and a moving mechanism. Furthermore, the pattern generation unit 112 can also irradiate uniform light without generating a pattern.

[0036] The imaging unit 120 includes an image sensor 121 and a plurality of lenses 122. Structured light reflected from the workpiece W is incident on the lenses 122, focused, and imaged, and then received by the image sensor 121. The imaging unit 120 may include a high-magnification imaging unit equipped with a high-magnification lens 122 and a low-magnification imaging unit equipped with a low-magnification lens 122. The lenses 122 may also be zoom lenses or the like with adjustable magnification, or the imaging unit 120 may have adjustable magnification. The magnification at the time of imaging is associated with the image data, making it possible to identify the magnification at which the image data was captured.

[0037] The image sensor 121 is composed of an image sensor such as a CCD (charge-coupled device) or a CMOS (complementary metal-oxide-semiconductor) sensor. Each pixel of the image sensor 121 outputs an analog electrical signal corresponding to the amount of light received (hereinafter referred to as the "light-receiving signal") to the A / D converter described later. Because color image sensors require each pixel to correspond to the reception of red, green, and blue light, their measurement resolution is lower compared to monochrome image sensors, and sensitivity is reduced because each pixel needs to have a color filter. For this reason, in this embodiment, a monochrome CCD is used as the image sensor 121. However, a color image sensor may also be used as the image sensor 121.

[0038] The imaging unit 120 is equipped with an A / D converter (analog-to-digital converter), a FIFO (First In First Out) memory, a CPU, and other components (not shown). The light signal output from the image sensor 121 is sampled at a constant sampling period by the A / D converter and converted into a digital signal. The digital signals output from the A / D converter are sequentially stored in the FIFO memory. The digital signals stored in the FIFO memory are sequentially output to the CPU as pixel data, and the CPU generates pattern image data.

[0039] For example, based on the light-receiving signal output from the image sensor 121, pattern image data representing the three-dimensional shape of the workpiece W contained within the field of view of the image sensor 121 at a specific position is generated. The pattern image data is the image itself acquired by the image sensor 121, and for example, when measuring the shape of the workpiece W using a phase-shift method, multiple images will constitute the pattern image data. The pattern image data may also be point cloud data, which is a collection of points having three-dimensional position information, and the pattern image data of the workpiece W can be acquired using this point cloud data. Point cloud data is data represented by a collection of multiple points having three-dimensional coordinates. The generated pattern image data is transferred to the controller 300.

[0040] Furthermore, when the structured illumination unit 110 is illuminating with uniform light without illuminating with structured light, the imaging unit 120 captures images of the workpiece W illuminated with uniform light. In this case, the imaging unit 120 can capture live images. Live images are images that are updated in real time at a predetermined short frame rate (fps) and are viewed by the user as a video.

[0041] The operation unit 500 may include, for example, a pointing device such as a keyboard 501 or a mouse 502. A joystick may also be used as the pointing device. Furthermore, the operation unit 500 may include a touch panel or the like that senses user touch operations. The operation unit 500 is connected to the arithmetic unit 301 within the controller 300, and the arithmetic unit 301 can detect what operations are performed by the operation unit 500.

[0042] The base unit 200 comprises a base plate 201, a stage 202 that forms a mounting surface on which the workpiece W is placed, and a rotation mechanism 203. The base unit 200 may also include a clamping mechanism for clamping the workpiece W on the stage 202. The rotation mechanism 203 is provided between the base plate 201 and the stage 202 and is a mechanism that rotates the stage 202 about a vertical axis (the Z-axis shown in Figure 1) relative to the base plate 201. Therefore, the stage 202 is a rotating stage, and by rotating it with the workpiece W placed on it, it is possible to switch the relative positional relationship of the workpiece W with respect to the imaging unit 120. The direction of rotation about the Z-axis is defined as the θ direction and is indicated by the arrow θ. The base unit 200 may also include a tilt stage having a mechanism that can rotate about an axis parallel to the mounting surface.

[0043] The rotation mechanism 203 has a motor or the like controlled by the measurement control unit 130, which will be described later, and is capable of rotating the stage 202 by a desired rotation angle and then holding it in a stopped state. The base portion 200 is not an essential component of the present invention and is provided as needed. The base portion 200 may also be controlled by the controller 300.

[0044] Although not shown in the figures, the base portion 200 may be equipped with a translation mechanism that moves the stage 202 horizontally in the X and Y directions, which are mutually orthogonal. The translation mechanism also has a motor controlled by the measurement control unit 130 and the controller 300, and is capable of moving the stage 202 by a desired amount in the X and Y directions and then holding it in a stopped state. The present invention is also applicable even if the stage 202 is a fixed stage.

[0045] The controller 300 includes an arithmetic unit 301, a working memory 302, a ROM (read-only memory) 303, a storage unit 304, and a display control unit 305, etc. A PC (personal computer) can be used for the controller 300, but it may also consist of a dedicated computer only, or a combination of a PC and a dedicated computer.

[0046] The ROM 303 of the controller 300 stores, for example, a system program. The working memory 302 of the controller 300 consists of, for example, RAM (Random Access Memory) and is used for processing various data. The storage unit 304 consists of, for example, a solid-state drive or a hard disk drive. The storage unit 304 stores a program for generating three-dimensional shape data. The storage unit 304 is also used to store various data such as pixel data (pattern image data) provided by the measurement control unit 130, setting information, measurement conditions for the workpiece W, and alignment images. Measurement conditions include, for example, the settings of the structured illumination unit 110 (pattern frequency, pattern type), the magnification of the imaging unit 120, the measurement field of view (single field of view or wide field of view), the measurement position, the rotational orientation, the exposure conditions (exposure time, gain, brightness of illumination), and the resolution setting (low-resolution measurement, standard measurement, high-resolution measurement).

[0047] The alignment image is an image used to align the workpiece W to a predetermined position before it is captured by the imaging unit 120. The file that associates the measurement conditions of the workpiece W with the alignment image is the measurement file. The storage unit 304 can also store multiple such measurement files.

[0048] The display control unit 305 controls the display unit 400 and displays on the display unit 400 a positioning image associated with one of the multiple measurement files stored in the storage unit 304, and a live image of the workpiece W. The live image of the workpiece W is a live image captured by the imaging unit 120 when uniform light is shone on the workpiece W currently placed on the stage 202 of the base unit 200, and the workpiece W illuminated by uniform light is captured.

[0049] The arithmetic unit 301 consists of control circuits and control elements that process given signals and data, perform various calculations, and output calculation results. In this specification, the arithmetic unit 301 refers to the elements and circuits that perform calculations, and is used to mean not limited to processors such as CPUs, MPUs, GPUs, and TPUs for general-purpose PCs, regardless of their name, but also including processors such as FPGAs, ASICs, and LSIs, microcontrollers, and chipsets such as SoCs.

[0050] The arithmetic unit 301 performs various processing on the pattern image data generated by the imaging unit 120 using the working memory 302. The arithmetic unit 301 comprises the 3D shape data generation unit 301a, the reception unit 301b, and the synthesis unit 301c, etc. The 3D shape data generation unit 301a, the reception unit 301b, and the synthesis unit 301c may consist solely of the hardware of the arithmetic unit 301, or they may consist of a combination of hardware and software. For example, by executing a 3D shape data generation program, the arithmetic unit 301 can realize the functions of the 3D shape data generation unit 301a, the reception unit 301b, and the synthesis unit 301c.

[0051] Details of the 3D shape data generation unit 301a, the reception unit 301b, and the synthesis unit 301c will be described later, but a brief overview will be given here. The 3D shape data generation unit 301a is the part that generates 3D shape data of the workpiece W based on the pattern image data generated by the imaging unit 120.

[0052] The reception unit 301b is the part that receives the instruction to start measuring the workpiece W. For example, if a user interface with a measurement start button is displayed on the display unit 400, and the operation of the measurement start button by the user is detected, the reception unit 301b receives the instruction to start measuring the workpiece W. The reception unit 301b can also receive multiple measurement start instructions.

[0053] The synthesis unit 301c is the part that generates composite three-dimensional shape data of the workpiece W by combining multiple three-dimensional shape data based on alignment information.

[0054] The measurement control unit 130 is connected to the controller 300 and is controlled by the arithmetic unit 301 of the controller 300. The measurement control unit 130 receives a measurement start instruction from the reception unit 301b. When the measurement control unit 130 receives a measurement start instruction from the reception unit 301b, it controls the structured illumination unit 110 and the imaging unit 120 in accordance with the measurement start instruction, based on the measurement conditions associated with a single measurement file.

[0055] The display unit 400 is composed of, for example, a liquid crystal display device or an organic FL display device. The display unit 400 is connected to the display control unit 305 of the controller 300 and is controlled by the display control unit 305. The display unit 400 displays, for example, images captured by the imaging unit 120, various user interface screens, setting screens, input screens, and images based on the three-dimensional shape data of the workpiece W.

[0056] The details of the 3D shape data generation device 1 will be explained below with reference to flowcharts and user interface screens. When operating the 3D shape data generation device 1, for example, it may be necessary to measure the shape data of the entire circumference of a workpiece W. In that case, first, during the initial measurement, it is necessary to reposition the workpiece W in a different orientation to eliminate blind spots, as shown in Figure 3, and take multiple images with the imaging unit 120. Subsequently, if it is necessary to measure multiple workpieces W of the same shape, it is necessary to perform the same measurement procedure for each workpiece W. However, by using the 3D shape data generation device 1, it is possible to align the workpiece to be measured to a predetermined position by referring to the alignment image on the display unit 400.

[0057] (During the initial measurement) First, an example of the initial measurement procedure will be explained based on Figure 4. In step SA1 of the flowchart shown in Figure 4, the three-dimensional shape data of the workpiece to be measured for the first time is acquired. Here, mesh data is acquired as the three-dimensional shape data. The workpiece W measured for the first time is also called the master workpiece and is distinguished from the workpieces measured in subsequent operations.

[0058] Specifically, the masterwork W is illuminated by the structured illumination unit 110, and the masterwork W is imaged by the imaging unit 120 to generate pattern image data of the masterwork W. Then, the three-dimensional shape data generation unit 301a generates three-dimensional shape data of the masterwork W based on the pattern image data generated by the imaging unit 120. The mesh data generated here as three-dimensional shape data includes multiple polygons and can also be called polygon data. A polygon is data composed of information that identifies multiple points and information that shows a polygonal surface formed by connecting those points. For example, it can consist of information that identifies three points and information that shows a triangular surface formed by connecting those three points. Mesh data and polygon data can also be defined as data represented by a collection of multiple polygons.

[0059] Figure 5 shows the user interface screen 700 for measurement settings that the display control unit 305 displays on the display unit 400 when measuring the master work W. The user interface screen 700 for measurement settings includes a measurement method selection area 701, a fully automatic selection area 702, a measurement mode selection area 703, a brightness setting area 704, a measurement field of view selection area 705, a rotation link setting area 706, and a schematic diagram display area 707.

[0060] In the measurement method selection area 701, it is possible to select either "One Shot," which takes only one image, or "Linking," which takes multiple images of the master work W in different orientations and links the captured images together. In the fully automatic selection area 702, it is possible to select either "Auto," which does not require user operation during measurement, or "Manual," which requires user operation. In the measurement mode selection area 703, it is possible to select one of three modes: "High Speed," which enables high-speed measurement, "Standard," which enables measurement at a standard speed, or "High Definition," which enables high-definition measurement. In the brightness setting area 704, it is possible to set the brightness of the image acquired by the imaging unit 120. In addition to "Auto," which automatically sets the brightness by the 3D shape data generation device 1, it is also possible for the user to manually set the brightness. The brightness of the image acquired by the imaging unit 120 is related to the exposure time of the imaging unit 120, so the exposure time setting of the imaging unit 120 can be accepted as a measurement condition by the reception unit 301b. Furthermore, in the measurement field of view selection area 705, it is possible to select between a relatively narrow "single field of view" and a relatively wide "wide field of view." In addition, in this embodiment, there are multiple projection patterns of structured light emitted from the structured illumination unit 110, and the user can select one projection pattern from among the multiple projection patterns. In other words, the reception unit 301b can also accept the setting of the projection pattern of structured light emitted from the structured illumination unit 110 as a measurement condition.

[0061] In the rotation linkage setting area 706, the operation settings of the rotation mechanism 203 of the base portion 200 can be configured, allowing the setting of the rotation angle, step angle, and rotation direction (clockwise, counterclockwise) of the stage 202 by the rotation mechanism 203. For example, if the rotation angle is set to 360 degrees and the step angle to 60 degrees, the stage 202 will rotate 60 degrees and stop, repeating this operation six times to complete one rotation. By rotating the stage 202 in this way by a predetermined step angle, the relative positional relationship of the workpiece W with respect to the imaging unit 120 can be switched, as can the relative imaging angle of the workpiece W with respect to the imaging unit 120. The imaging angle is determined by the user operating the operation unit 500 to specify the rotation angle of the stage 202. In other words, the reception unit 301b can receive a specification of the relative imaging angle of the workpiece W with respect to the imaging unit 120.

[0062] While the stage 202 is stopped, illumination by the structured illumination unit 110 and imaging by the imaging unit 120 are performed. The schematic diagram display area 707 displays a schematic diagram of the mounting surface of the stage 202, and it is also possible to display, for example, a stop position marker corresponding to the step angle.

[0063] The measurement conditions set in areas 701 to 707 of the measurement setting user interface screen 700 during the initial measurement are the measurement conditions for the master workpiece. The measurement conditions set in areas 701 to 707 of the measurement setting user interface screen 700 are received by the reception unit 301b. The measurement setting user interface screen 700 is also provided with a measurement execution button 708. When the user operates the measurement execution button 708, that operation is received by the reception unit 301b as an instruction to start measurement of the master workpiece.

[0064] Step SA1 allows for the execution of multiple sequences. A sequence is a series of measurement processes that can be executed solely by the 3D shape data generation device 1 without user intervention. As illustrated in Figure 3, after the user places the master workpiece W on the stage 202 in the first orientation, the user sets the measurement conditions in areas 701 to 707 of the measurement setting user interface screen 700, and then operates the measurement execution button 708. The 3D shape data generation device 1 then executes the first sequence. The setting of measurement conditions before the execution of the first sequence is the setting of the first measurement conditions. The measurement start instruction received by the reception unit 301b before the execution of the first sequence is the first measurement start instruction.

[0065] In the first sequence, the measurement control unit 130 controls the structured illumination unit 110 and the imaging unit 120 based on the first measurement conditions. The imaging unit 120 generates a first alignment image in response to the first measurement start instruction received by the reception unit 301b, and generates pattern image data by receiving structured light reflected from the masterwork W. Subsequently, the 3D shape data generation unit 301a generates first 3D shape data (point cloud data or mesh data) of the masterwork W based on the pattern image data generated by the imaging unit 120.

[0066] At this time, if "Link" is selected in the measurement method selection area 701, the measurement control unit 130 controls the rotation mechanism 203 of the base unit 200 based on the setting in the rotation link setting area 706, and each time the stage 202 stops, the measurement control unit 130 performs illumination by the structured illumination unit 110 and imaging by the imaging unit 120.

[0067] The measurement conditions include setting the projection pattern of structured light, setting the exposure time of the imaging unit 120, and setting the relative imaging angle of the masterwork W with respect to the imaging unit 120. If the imaging angle setting is received by the reception unit 301b, the measurement control unit 130 drives the stage 202 based on the imaging angle received by the reception unit 301b and positions the masterwork W to achieve that imaging angle.

[0068] Once the first sequence is complete, the user places the masterwork W on the stage 202 in the second orientation shown in Figure 3, sets the measurement conditions in areas 701 to 707 of the measurement setting user interface screen 700, and then operates the measurement execution button 708. The 3D shape data generation device 1 then executes the second sequence. The measurement conditions set before the execution of the second sequence are the second measurement conditions. The measurement start instruction received by the reception unit 301b before the execution of the second sequence is the second measurement start instruction.

[0069] In the second sequence, the measurement control unit 130 controls the structured illumination unit 110 and the imaging unit 120 based on the second measurement conditions. The imaging unit 120 generates a second alignment image in response to the second measurement start instruction received by the reception unit 301b, and generates pattern image data by receiving structured light reflected from the masterwork W. Subsequently, the three-dimensional shape data generation unit 301a generates second three-dimensional shape data of the masterwork W based on the pattern image data generated by the imaging unit 120. If "Link" is selected in the measurement method selection area 701, the rotation mechanism 203 of the base unit 200 is controlled, and illumination by the structured illumination unit 110 and imaging by the imaging unit 120 are performed, similar to the first sequence.

[0070] After the second sequence is completed, the user places the masterwork W on the stage 202 in the third orientation shown in Figure 3, sets the measurement conditions in areas 701 to 707 of the user interface screen 700 for measurement settings, and then operates the measurement execution button 708. The 3D shape data generation device 1 then executes the third sequence. In the third sequence, the imaging unit 120 generates a third alignment image in response to the measurement start instruction received by the reception unit 301b. Subsequently, similar to the first sequence, pattern image data is generated and 3D shape data is generated. The fourth sequence, fifth sequence, and so on can be executed in the same manner. Note that the measurement may be completed with only the first sequence.

[0071] When each sequence is completed, the arithmetic unit 301 stores the user-set measurement conditions, the alignment image generated by the imaging unit 120, and the three-dimensional shape data in the storage unit 304. That is, the storage unit 304 stores a measurement file that associates the measurement conditions received by the reception unit 301b with the alignment image and the three-dimensional shape data. More specifically, as shown in the data structure of the first sequence of the masterwork in Figure 8, in the first sequence, the first three-dimensional shape data A is acquired, the first measurement conditions A for acquiring this first three-dimensional shape data A are set, and the first alignment image A is acquired by imaging the masterwork W in the first orientation. The first measurement file A, which associates the first three-dimensional shape data A of the masterwork W with the first measurement conditions A and the first alignment image A, is stored in the storage unit 304 as the first measurement file A. The first measurement conditions A and the first alignment image A are the measurement reproduction data A of the first sequence. Furthermore, if the measurement conditions include the imaging angle, the memory unit 304 stores a measurement file that includes the imaging angle.

[0072] In the second sequence, second three-dimensional shape data is acquired, and second measurement conditions are set for acquiring this second three-dimensional shape data. Additionally, a second alignment image is acquired by imaging the masterwork W in a second orientation. A measurement file associating the second three-dimensional shape data of the masterwork W, the second measurement conditions, and the second alignment image is stored in the storage unit 304 as the second measurement file. More specifically, as shown in the data structure of the second sequence of the masterwork in Figure 8, in the second sequence, second three-dimensional shape data B is acquired, second measurement conditions B are set for acquiring this second three-dimensional shape data B, and a second alignment image B is acquired by imaging the masterwork W in a second orientation. A measurement file associating the second three-dimensional shape data B of the masterwork W, the second measurement conditions B, and the second alignment image B is stored in the storage unit 304 as the second measurement file B. The second measurement conditions B and the second alignment image B are the measurement reproduction data B of the second sequence. The same applies to the third sequence, but the explanation will be omitted.

[0073] After acquiring the 3D shape data in step SA1 shown in Figure 4, the acquired 3D shape data is displayed on the display unit 400 by the display control unit 305. This allows the user to verify the 3D shape data. In step SA2, the user's determination of whether or not there are blind spots in the 3D shape data acquired in step SA1 is accepted. If there are blind spots, the process returns to step SA1, the posture of the masterwork W is changed to a different posture than that used during the previous imaging, and the process returns to step SA1. If this is repeated, for example, multiple times until there are no blind spots, the process proceeds to step SA3, and the data synthesis process is executed.

[0074] The procedure for data synthesis is shown in Figure 6. In step SB1, alignment elements are created based on the user's selection. Specifically, the display control unit 305 displays the data synthesis user interface screen 710 shown in Figure 7 on the display unit 400. The data synthesis user interface screen 710 includes a first display area 711 that displays the shape based on the original 3D shape data (e.g., the first 3D shape data), a second display area 712 that displays the shape based on the added 3D shape data (e.g., the second 3D shape data), and a third display area 713 that displays the shape based on the synthesized 3D shape data. Note that the first to third display areas 711 to 713 display masterwork W with shapes different from the masterwork W shown in Figures 1 and 3, but all masterwork W are examples, and their shapes can be any shape.

[0075] Furthermore, the data synthesis user interface screen 710 is provided with a procedure display area 714 that shows the alignment procedure and an element setting area 715 for setting alignment elements. In the element setting area 715, as shown in the first display area 711 and the second display area 712, the reception unit 301b receives the specification of geometric elements such as planes "Plane A" and "Plane B" and cylinders "Area C" based on the user's operation of the operation unit 500. The set geometric elements are used when identifying alignment information and are information generated by the reception unit 301b receiving the specification of corresponding faces in the first three-dimensional shape data of the master work W and the second three-dimensional shape data of the master work W. "Plane A", "Plane B", and "Area C" in the first display area 711 are faces corresponding to "Plane A", "Plane B", and "Area C" in the second display area 712, respectively. Note that the types of geometric elements are not limited to planes and cylinders. Furthermore, the alignment information only requires information on at least one geometric element, and it is sufficient if at least one geometric element can be specified in both the first display area 711 and the second display area 712. Alternatively, instead of using geometric element information, the relative positional relationship between the three-dimensional shape data when the shapes of the workpieces W are aligned may be used as the alignment information. This is the processing content of step SB1.

[0076] In step SB2, the synthesis unit 301c performs alignment of the geometric elements created in step SB1. Specifically, the synthesis unit 301c identifies alignment information for aligning the first three-dimensional shape data of the masterwork W acquired in the first sequence with the second three-dimensional shape data of the masterwork W acquired in the second sequence. For example, if the first three-dimensional shape data is the data for the front side of the masterwork W and the second three-dimensional shape data is the data for the back side of the masterwork W, then alignment of the front and back sides can be performed.

[0077] As an example of identifying alignment information, the synthesis unit 301c calculates a transformation matrix to match the position and orientation of the first 3D shape data of the masterwork W with the position and orientation of the second 3D shape data of the masterwork W, based on the designation of corresponding faces in the first 3D shape data of the masterwork W and the second 3D shape data of the masterwork W received by the reception unit 301b. The synthesis unit 301c can then identify the calculated transformation matrix as alignment information.

[0078] The synthesis unit 301c identifies alignment information and then generates composite three-dimensional shape data of the masterwork W by combining the first three-dimensional shape data of the masterwork W and the second three-dimensional shape data of the masterwork W based on the identified alignment information.

[0079] Next, the process proceeds to step SB3, where the user is prompted to indicate whether or not to perform alignment with the additional elements. If alignment with the additional elements is selected, the process returns to step SB1, adds the geometric elements, and proceeds to step SB2. Step SB4 performs precise alignment between the original 3D shape data and the added 3D shape data. Finally, in step SB5, the original 3D shape data and the added 3D shape data are combined and re-meshed.

[0080] Once step SB5 is completed, the process proceeds to step SA4 shown in Figure 4, where it is determined whether the synthesis of all 3D shape data has been completed. If the synthesis of all 3D shape data has not been completed, the process returns to step SA3 to perform the data synthesis process; however, if the synthesis of all 3D shape data has been completed, the process terminates.

[0081] The arithmetic unit 301 also stores the alignment information and the composite three-dimensional shape data in the storage unit 304. When storing these, the storage unit 304 stores a composite data file that associates the first measurement file, the second measurement file, the alignment information, and the composite three-dimensional shape data of the master work W.

[0082] More specifically, as shown in the composite data structure of the masterwork in Figure 8, the composite data file AB is associated with the composite reproduction data file AB and the composite three-dimensional shape data AB. The composite reproduction data file AB is associated with the measurement reproduction data A of the first sequence, the measurement reproduction data B of the second sequence, and alignment information for aligning the second three-dimensional shape data B with the first three-dimensional shape data A. The storage unit 304 can store multiple composite data files.

[0083] The composite data file may also have analysis conditions associated with it. These analysis conditions are the conditions for extracting multiple geometric elements from the generated composite solid shape data, and for measuring dimensions between the extracted geometric elements, as well as the conditions for measuring tolerances. These analysis conditions can also be included in the composite data file and stored in the storage unit 304.

[0084] (When measurement is reproducible) Next, the procedure for measuring a workpiece W of the same shape after measuring a master workpiece W and generating three-dimensional shape data of that workpiece W will be described. In step SC1 shown in Figure 9, the storage unit 304 accepts the selection of one composite data file from among a plurality of composite data files stored in the storage unit 304 and reads the accepted composite data file. In step SC2, the alignment images contained in the composite data file read in step SC1 are displayed on the display unit 400. Specifically, the display control unit 305 displays a data selection user interface screen 720 on the display unit 400 as shown in Figure 10. The data selection user interface screen 720 is provided with an alignment image display area 721 where the alignment images contained in the composite data file are displayed. In this example, the display unit 400 displays information indicating the measurement file contained in the composite data file read in step SC1. The information indicating the measurement file is, for example, an alignment image, and the alignment image associated with the first measurement file can be displayed on the display unit 400 as information indicating the first measurement file, and the alignment image associated with the second measurement file can be displayed as information indicating the second measurement file. In the example shown in Figure 10, the three alignment images contained in each of the three measurement files associated with a single composite data file are displayed in the alignment image display area 721 as information for each of the three measurement files. The number of alignment images displayed in the alignment image display area 721 is not particularly limited. The alignment images displayed in the alignment image display area 721 can be thumbnail images that are smaller than the images actually captured by the imaging unit 120.

[0085] In the explanation of steps SC1 and SC2, the case of selecting a composite data file and displaying the alignment images contained in that composite data file was described. However, if there is no composite data file, step SC1 may be omitted, and in step SC2, multiple alignment images stored in the storage unit 304 may be displayed in the alignment image display area 721. In step SC3, it is determined whether or not a single alignment image has been selected from among the alignment images displayed in the alignment image display area 721. This selection operation can be performed by the user by operating the operation unit 500. In the example shown in Figure 10, the case where the upper left alignment image has been selected is shown, and a selection frame 721a indicating which image has been selected is displayed. When the OK button 722 is operated by the user, the composite data file containing the alignment images enclosed in the selection frame 721a will be selected. Therefore, the alignment image selection operation is the operation of selecting one measurement file from among multiple measurement files stored in the storage unit 304 and associated with the composite data file read in step SC1, and this selection operation is received by the reception unit 301b.

[0086] In step SC4, the first measurement conditions included in the measurement file selected in step SC3 are read from the storage unit 304, and the read first measurement conditions are restored, i.e., applied. The user may change the applied measurement conditions. In step SC5, the display control unit 305 displays on the display unit 400 the alignment image included in the composite data file selected in step SC3 and associated with the first measurement file, and a live image of the workpiece W on the stage 202. Figure 11 shows the measurement user interface screen 730 that the display control unit 305 displays on the display unit 400. The measurement user interface screen 730 is provided with a live image display area 731 where the live image currently being captured by the imaging unit 120 is displayed. At this stage, the workpiece W is not placed on the stage 202, so only the stage 202 is displayed in the live image display area 731 in Figure 11.

[0087] Figure 12 shows an example in which a alignment image 733 is overlaid (superimposed) on the live image display area 731 of the measurement user interface screen 730. The alignment image 733 is a guide image used to align the workpiece W, before it is captured by the imaging unit 120, to the position (predetermined position) where the master workpiece W was captured. The alignment image 733 has a predetermined transparency set so that the live image is displayed through the alignment image 733. The transparency should be such that the stage 202 and workpiece W can be seen through the alignment image 733.

[0088] The measurement user interface screen 730 displays the alignment image 733, and the measurement reproduction window 732 is also displayed. The measurement reproduction window 732 displays instructions to the user to superimpose the actual workpiece W onto the superimposed workpiece image, i.e., the alignment image 733.

[0089] The user adjusts the position and orientation of the workpiece W so that it overlaps with the alignment image 733 while viewing the alignment image 733. The user also adjusts the position and orientation of the measuring unit 100 so that it overlaps with the alignment image 733 while viewing the alignment image 733. At this time, the user may change the direction of movement of the workpiece W on the stage 202 so that the direction of movement of the workpiece W on the live image displayed on the display unit 400 is the same. Whether or not to change the direction of movement can be set by the user.

[0090] The measurement user interface screen 730 is provided with a measurement execution button 708, similar to that in Figure 5. After the user has finished aligning the workpiece W, the user operates the measurement execution button 708. When the user operates the measurement execution button 708, the operation is received by the reception unit 301b as a first measurement start instruction for the workpiece W currently placed on the stage 202.

[0091] Upon receiving the first measurement start instruction, the process proceeds to step SC6. In step SC6, the measurement control unit 130 identifies a first measurement condition A associated with the first measurement file A shown in Figure 8 in response to the first measurement start instruction for the workpiece W, and controls the structured illumination unit 110 and the imaging unit 120 to generate a first pattern image data based on the identified first measurement condition A. The three-dimensional shape data generation unit 301a controls the structured illumination unit 110 and the imaging unit 120 based on the first measurement condition A identified by the measurement control unit 130, and generates first three-dimensional shape data A' of the workpiece W based on the pattern image data generated by the imaging unit 120. At this time, the imaging unit 120 also images the workpiece W on the stage 202 to generate a first alignment image A'. The first measurement condition A, the first three-dimensional shape data A', and the first alignment image A' are associated and stored in the storage unit 304.

[0092] After step SC6, the process proceeds to step SC7 to determine whether all three-dimensional shape data has been acquired. Here, "all three-dimensional shape data has been acquired" means that, if the measurement conditions include rotation linking, the acquisition of three-dimensional shape data has been completed at all set angular positions. If the measurement conditions do not include rotation linking, this step may be omitted. If not all three-dimensional shape data has been acquired, the process returns to step SC6; if all three-dimensional shape data has been acquired, the process proceeds to step SC8. In step SC8, it is determined whether all sequences have been completed. If there is a second sequence and only the first sequence has been completed, the process returns to step SC4, reads the first measurement conditions included in the composite data file selected in step SC3 from the storage unit 304, and restores, i.e., applies, the read second measurement conditions.

[0093] In step SC5, the alignment image associated with the second measurement file and the live image of the workpiece W on the stage 202 are displayed in the live image display area 731 of the measurement user interface screen 730 shown in Figure 12.

[0094] When a user operates the measurement execution button 708 on the measurement user interface screen 730, the operation is received by the reception unit 301b as a second measurement start instruction for the workpiece W currently placed on the stage 202. Upon receiving the second measurement start instruction, the process proceeds to step SC6. In step SC6, the measurement control unit 130 identifies a second measurement condition B associated with the second measurement file B shown in Figure 8, in response to the second measurement start instruction for the workpiece W, and controls the structured illumination unit 110 and the imaging unit 120 to generate a second pattern image data based on the identified second measurement condition B. The three-dimensional shape data generation unit 301a controls the structured illumination unit 110 and the imaging unit 120 based on the second measurement condition B identified by the measurement control unit 130, and generates the second three-dimensional shape data B' of the workpiece W based on the pattern image data generated by the imaging unit 120. At this time, the imaging unit 120 also images the workpiece W on the stage 202 to generate a second alignment image B'. The second measurement condition B, the second three-dimensional shape data B', and the second alignment image B' are associated and stored in the storage unit 304.

[0095] Once all sequences are completed as described above, the process proceeds to step SC9, where data synthesis is performed. Note that measurement reproduction is not required for all sequences; for example, measurement reproduction using alignment images may be performed for only one of the first or second sequences, while a normal measurement without alignment images is performed for the other. For instance, a normal measurement without alignment images may be performed for the front side of workpiece W, while measurement reproduction using alignment images may be performed for the back side of workpiece W.

[0096] The data synthesis process is shown in Figure 13. In step SD1, the initial posture is reproduced. That is, the synthesis unit 301c reads and restores the transformation matrix, which is the alignment information calculated by the synthesis unit 301c during the measurement of the master work W, from the synthesis data file. Figure 14 shows the user interface screen 710 for data synthesis during measurement reproduction. The first display area 711 displays the first 3D shape data measured in the measurement reproduction, and the second display area 712 displays the second 3D shape data measured in the measurement reproduction. The synthesis unit 301c reads the transformation matrix contained in one of the synthesis data files, and based on the read transformation matrix, automatically aligns the first 3D shape data of the work W and the second 3D shape data of the work W to generate synthesized 3D shape data. That is, the synthesis unit 301c performs a coordinate transformation on at least one of the first 3D shape data of the work W and the second 3D shape data of the work W based on the transformation matrix to generate synthesized 3D shape data. The synthesized 3D shape data, which is the 3D shape data after alignment, is displayed in the third display area 713.

[0097] In step SD2, the user input is received regarding whether or not to perform alignment with additional geometric elements. If alignment with additional geometric elements is not performed, the process proceeds to step SD3, where the compositing unit 301c performs precise alignment. Then, in step SD4, the compositing unit 301c performs the compositing process of the first and second 3D shape data, converts it into remeshed data, and updates the composite 3D shape data generated in step SD1.

[0098] On the other hand, if the process proceeds to step SD5, the user creates additional alignment elements using the data synthesis user interface screen 710 shown in Figure 7. In step SD6, the synthesis unit 301c performs alignment of the geometric elements added in step SD5 and updates the synthesized solid shape data generated in step SD1. If it is no longer necessary to add alignment elements, the process proceeds to step SD3.

[0099] As shown in Figure 8, the composite data structure of the workpiece W includes a first 3D shape data A', a second 3D shape data B', alignment information (information for aligning the second 3D shape data B' with the first 3D shape data A'), and the final composite 3D shape data A' and B', which are stored in the storage unit 304.

[0100] In this example, we described the case where the master workpiece is the workpiece used during the initial measurement. However, the master workpiece does not have to be the workpiece used during the initial measurement. For example, when measuring workpiece W for the third time, the workpiece W measured during the second measurement can be used as the master workpiece.

[0101] (Display of measurement conditions) Using the data selection user interface screen 720 shown in Figure 10, it is also possible to select a single measurement data to display the measurement conditions included in the measurement data. The data selection user interface screen 720 displays a measurement condition display button 723. When the operation unit 500 accepts the selection of one alignment image from among multiple alignment images displayed in the alignment image display area 721 and receives an input from the operation of the measurement condition display button 723, the display control unit 305 can display a measurement condition display user interface screen on the display unit 400 that is similar to the measurement setting user interface screen 700 shown in Figure 5. This measurement condition display user interface screen may omit the measurement execution button 708 from the measurement setting user interface screen 700 shown in Figure 5. By displaying the measurement conditions on the display unit 400 in this way, the user can check the measurement conditions when measuring the workpiece W without actually performing the measurement.

[0102] (Analysis reproduction using analysis conditions) As explained in Figure 8, analysis condition data can be associated with the masterwork composite data file. By applying the analysis conditions associated with the masterwork composite data file to the composite three-dimensional shape data of workpiece W generated in step SD4, the analysis of the masterwork can be reproduced even for workpiece W measured for the second time or later. Specifically, the analysis unit 301d included in the calculation unit 301 identifies the analysis condition data included in the masterwork composite data file. Then, the analysis unit 301d identifies the target of analysis and performs the analysis of the composite three-dimensional shape data of workpiece W based on the identified analysis condition data. If the analysis condition is the distance between plane A and plane B, the analysis unit 301d identifies the planes corresponding to plane A and plane B from the composite three-dimensional shape data of workpiece W and calculates the distance between the identified planes.

[0103] (Alignment marker) Figure 15 shows the case where alignment markers M are applied to the surface of the masterwork W. The user applies the alignment markers M to the surface of the masterwork W, and then, as described above, the first and second sequences are executed on the masterwork W to generate the first and second 3D shape data. The compositing unit 301c uses the alignment markers M to match the position and orientation of the first 3D shape data with the position and orientation of the second 3D shape data, and generates composite 3D shape data of the masterwork W.

[0104] When aligning the first and second 3D shape data using a master workpiece W that has alignment markers M attached, the workpiece W used for measurement reproduction can reproduce the position and orientation for each sequence based on the master workpiece W, so alignment markers M are not required on the workpiece W used for measurement reproduction.

[0105] (Display format and alignment method for alignment images) In the above embodiment, the alignment image is displayed on the display unit 400 as a transparent image, but the display format of the alignment image is not limited to this. For example, the alignment image does not have to be a transparent image, and may be displayed on the display unit 400 as an overhead image. Alternatively, the alignment image may be projected onto a screen (display unit). Furthermore, the display unit 400 may be a head-mounted display, in which case the user can wear the head-mounted display and have the alignment image and live image of the workpiece W displayed on the head-mounted display, and perform the alignment of the workpiece W while viewing the images.

[0106] Furthermore, in the above embodiment, the user aligns the workpiece W so that it overlaps with the alignment image. However, the user is not limited to this, and the alignment image and the live image may be displayed in separate areas of the display unit 400, and the position and orientation of the workpiece W may be adjusted on the stage 202 so that it is placed in the same position as the alignment image while viewing the alignment image. Alternatively, the workpiece W may be held using a 6-degree-of-freedom arm, and its position and orientation may be adjusted.

[0107] The magnification of the imaging unit 120 may differ for each sequence. For example, even if the first sequence is at high magnification and the second sequence is at low magnification, the synthesis unit 301c can perform data synthesis, and the user can align the workpiece W based on the alignment image.

[0108] (Automatic alignment function) The 3D shape data generation device 1 may have an automatic alignment function. When a user aligns the workpiece W while viewing the alignment image, it can be difficult to place the workpiece W in a position that perfectly matches the alignment image. In particular, when aligning the workpiece W by making the alignment image transparent, if the workpiece W is even slightly misaligned from the alignment image, the image will become blurred. In this case, the calculation device 301 can calculate the misalignment between the alignment image and the workpiece image on the live image, and automatically adjust the position of the workpiece W by controlling the movable stage 202 to eliminate this misalignment.

[0109] (Embodiment 2) Figure 16 shows the overall configuration of the three-dimensional shape data generation device 1 according to Embodiment 2 of the present invention. The three-dimensional shape data generation device 1 according to Embodiment 2 differs from that of Embodiment 1 in that the measuring unit 100 and the base unit 200 are integrated. Hereinafter, the same reference numerals are used for parts that are the same as in Embodiment 1 and their descriptions are omitted, while the different parts will be described in detail.

[0110] Specifically, a support section 250 for supporting the measuring section 100 is provided on the rear side of the base section 200, extending upward. The measuring section 100 is fixed to the upper part of this support section 250. The measuring section 100 is equipped with a structured illumination section 110 and an imaging section 120 such that the optical axis is directed toward the stage 202.

[0111] Even with a three-dimensional shape data generation device 1 like that in Embodiment 2, three-dimensional shape data can be generated in the same way as in Embodiment 1.

[0112] The embodiments described above are merely illustrative in all respects and should not be interpreted restrictively. Furthermore, any modifications or changes that fall within the equivalent scope of the claims are all within the scope of the present invention. [Industrial applicability]

[0113] As described above, the 3D shape data generation device relating to this disclosure can be used to generate 3D shape data of a workpiece. [Explanation of symbols]

[0114] 1. 3D Shape Data Generation Device 110 Structured lighting section 120 Imaging Unit 130 Measurement Control Unit 202 stages 301a 3D Shape Data Generation Unit 301b Reception Desk 301c Synthesis Department 304 Storage section 305 Display Control Unit 400 Display Double job

Claims

1. A structured illumination unit that irradiates the workpiece with structured light for measurement, An imaging unit having a field of view that receives structured light irradiated by the structured illumination unit and reflected by the workpiece, and generates pattern image data of the workpiece, A three-dimensional shape data generation apparatus comprising: a three-dimensional shape data generation unit that generates three-dimensional shape data of a workpiece based on pattern image data generated by the imaging unit, A storage unit that stores multiple measurement files, each of which associates the measurement conditions of the workpiece with an alignment image used to position the workpiece in a predetermined location before it is captured by the imaging unit. A display control unit that displays the alignment image associated with one measurement file and a live image of the workpiece on the display unit, selected from a plurality of measurement files stored in the storage unit. A reception unit that receives instructions to start measuring the workpiece and sets the measurement conditions for the master workpiece, The system includes a measurement control unit that controls the structured illumination unit and the imaging unit based on measurement conditions associated with the measurement file in response to the measurement start instruction received by the reception unit, The reception unit receives the setting of the first measurement conditions and the instruction to start the first measurement. The measurement control unit controls the structured illumination unit and the imaging unit based on the first measurement conditions. The imaging unit generates a first alignment image in response to the first measurement start instruction received by the reception unit, and generates pattern image data by receiving structured light reflected from the master workpiece. The three-dimensional shape data generation unit generates first three-dimensional shape data of the masterwork based on the pattern image data generated by the imaging unit. The reception unit further receives the setting of the second measurement conditions and the instruction to start the second measurement. The measurement control unit controls the structured illumination unit and the imaging unit based on the second measurement conditions. The imaging unit generates a second alignment image in response to the second measurement start instruction received by the reception unit, and generates pattern image data by receiving structured light reflected from the master workpiece. The three-dimensional shape data generation unit generates second three-dimensional shape data of the masterwork based on the pattern image data generated by the imaging unit. The aforementioned storage unit is A first measurement file that associates the first three-dimensional shape data of the masterwork, the first measurement conditions, and the first alignment image, A three-dimensional shape data generation device that stores a second measurement file which associates the second three-dimensional shape data of the masterwork, the second measurement conditions, and the second alignment image.

2. A three-dimensional shape data generation apparatus according to claim 1, The system further includes a synthesis unit that identifies alignment information for aligning the first three-dimensional shape data and the second three-dimensional shape data of the masterwork, and generates a composite three-dimensional shape data of the masterwork by combining the first three-dimensional shape data and the second three-dimensional shape data of the masterwork based on the identified alignment information. The storage unit is a three-dimensional shape data generation device that stores a composite data file which associates the first measurement file, the second measurement file, the alignment information, and the composite three-dimensional shape data of the master work.

3. A three-dimensional shape data generation apparatus according to claim 2, The receiving unit receives the selection of one composite data file from among the multiple composite data files stored in the storage unit. The display control unit causes the display unit to display a live image of the workpiece and a positioning image included in a composite data file received by the reception unit and associated with the first measurement file. The reception unit receives the first instruction to start measuring the workpiece. The measurement control unit, in response to a first measurement start instruction for the workpiece, identifies measurement conditions associated with the first measurement file, and controls the structured illumination unit and the imaging unit based on the identified measurement conditions. The three-dimensional shape data generation unit controls the structured illumination unit and the imaging unit based on the measurement conditions specified by the measurement control unit, and generates first three-dimensional shape data of the workpiece based on the pattern image data generated by the imaging unit. The display control unit causes the display unit to display a alignment image and a live image of the workpiece, which are included in a composite data file received by the reception unit and associated with the second measurement file. The reception unit receives the second instruction to start measuring the workpiece. The measurement control unit, in response to a second measurement start instruction for the workpiece, identifies measurement conditions associated with the second measurement file, and controls the structured illumination unit and the imaging unit based on the identified measurement conditions. The three-dimensional shape data generation unit controls the structured illumination unit and the imaging unit based on the measurement conditions specified by the measurement control unit, and generates second three-dimensional shape data of the workpiece based on the pattern image data generated by the imaging unit. The synthesis unit is a three-dimensional shape data generation device that generates a composite three-dimensional shape data of a workpiece by synthesizing a first three-dimensional shape data of a workpiece and a second three-dimensional shape data of a workpiece based on the alignment information contained in the first synthesis data file.

4. A three-dimensional shape data generation apparatus according to claim 3, The alignment information is generated by the receiving unit receiving the specification of corresponding faces in the first three-dimensional shape data and the second three-dimensional shape data of the masterwork, in a three-dimensional shape data generation device.

5. A three-dimensional shape data generation apparatus according to claim 4, The synthesis unit calculates a transformation matrix for aligning the position and orientation of the first three-dimensional shape data of the masterwork with the position and orientation of the second three-dimensional shape data of the masterwork, based on the designation of corresponding faces in the first three-dimensional shape data of the masterwork and the second three-dimensional shape data of the masterwork received by the reception unit, and identifies the calculated transformation matrix as the alignment information, in a three-dimensional shape data generation device.

6. A three-dimensional shape data generation apparatus according to claim 3, When the display control unit receives the selection of the first composite data file from the reception unit, it causes the display unit to display information indicating the first measurement file and information indicating the second measurement file contained in the first composite data file. The receiving unit is a three-dimensional shape data generation device that accepts the selection of one measurement file from among the information indicating a first measurement file and the information indicating a second measurement file displayed on the display unit.

7. A three-dimensional shape data generation apparatus according to claim 6, The information indicating the first measurement file is an alignment image associated with the first measurement file, The information indicating the second measurement file is a positional image associated with the second measurement file, in a three-dimensional shape data generation device.

8. A three-dimensional shape data generation apparatus according to claim 1, The display control unit is a three-dimensional shape data generation device that causes a measurement condition display user interface screen, which displays the measurement conditions associated with the one measurement file, to be displayed on the display unit.

9. A three-dimensional shape data generation apparatus according to claim 3, The system further includes an analysis unit that performs analysis of the composite three-dimensional shape data of the aforementioned masterwork. The aforementioned composite data file is further associated with masterwork analysis conditions. The analysis unit is a three-dimensional shape data generation device that performs analysis of the composite three-dimensional shape data of the workpiece based on the master workpiece analysis conditions associated with the composite data file.

10. A structured illumination unit that irradiates the workpiece with structured light for measurement, An imaging unit having a field of view that receives structured light irradiated by the structured illumination unit and reflected by the workpiece, and generates pattern image data of the workpiece, A three-dimensional shape data generation apparatus comprising: a three-dimensional shape data generation unit that generates three-dimensional shape data of a workpiece based on pattern image data generated by the imaging unit, A storage unit that stores multiple measurement files, each of which associates the measurement conditions of a master workpiece with a positioning image used to align the workpiece to a predetermined position before it is captured by the imaging unit. A display control unit that displays the alignment image associated with one measurement file and a live image of the workpiece on the display unit, selected from a plurality of measurement files stored in the storage unit. A reception unit that receives instructions to start measuring the workpiece, The system includes a measurement control unit that controls the structured illumination unit and the imaging unit based on measurement conditions associated with the measurement file in response to the measurement start instruction received by the reception unit, The reception unit receives the setting of measurement conditions for the masterwork, which includes setting the projection pattern of structured light emitted from the structured illumination unit and setting the exposure time of the imaging unit, as well as the instruction to start the measurement of the masterwork. The imaging unit is a three-dimensional shape data generation device that generates the alignment image in response to the instruction to start measurement of the master work.

11. A three-dimensional shape data generation apparatus according to claim 10, The system further includes a rotating stage on which the master workpiece is placed and which switches the relative positional relationship of the master workpiece with respect to the imaging unit, The reception unit receives the specification of the relative imaging angle of the master work with respect to the imaging unit as a measurement condition associated with the first measurement file. The measurement control unit drives the rotating stage based on the imaging angle received by the reception unit. The storage unit stores a measurement file in which the imaging angle is included as a measurement condition, and is a three-dimensional shape data generation device.

12. A three-dimensional shape data generation apparatus according to claim 11, The measurement control unit is a three-dimensional shape data generation device that controls the rotating stage based on the imaging angle specified as a measurement condition associated with the measurement file, in response to a measurement start instruction received by the reception unit.

13. A three-dimensional shape data generation apparatus according to claim 10, The display control unit is a three-dimensional shape data generation device that causes a measurement condition display user interface screen, which displays the measurement conditions associated with the one measurement file, to be displayed on the display unit.

14. A three-dimensional shape data generation apparatus according to claim 10, The system includes a rotating stage that switches the relative imaging angle of the workpiece with respect to the imaging unit, The measurement control unit controls the rotating stage and the imaging unit to generate pattern image data of the master work at multiple imaging angles. The memory unit is a three-dimensional shape data generation device that stores multiple imaging angles, including the rotation of the rotating stage, as measurement conditions for the master work.

15. In the three-dimensional shape data generation apparatus according to claim 10, The reception unit receives the setting of the first measurement conditions and the instruction to start the first measurement. The measurement control unit controls the structured illumination unit and the imaging unit based on the first measurement conditions. The imaging unit generates a first alignment image in response to the first measurement start instruction received by the reception unit, and generates pattern image data by receiving structured light reflected from the master workpiece. The three-dimensional shape data generation unit generates first three-dimensional shape data of the masterwork based on the pattern image data generated by the imaging unit. The reception unit further receives the setting of the second measurement conditions and the instruction to start the second measurement. The measurement control unit controls the structured illumination unit and the imaging unit based on the second measurement conditions. The imaging unit generates a second alignment image in response to the second measurement start instruction received by the reception unit, and generates pattern image data by receiving structured light reflected from the master workpiece. The three-dimensional shape data generation unit generates second three-dimensional shape data of the masterwork based on the pattern image data generated by the imaging unit. The aforementioned storage unit is A first measurement file that associates the first three-dimensional shape data of the masterwork, the first measurement conditions, and the first alignment image, A three-dimensional shape data generation device that stores a second measurement file which associates the second three-dimensional shape data of the masterwork, the second measurement conditions, and the second alignment image.

Images