3D Shape Data Generation Device

Abstract

To acquire texture information on a measurement object without using a separate camera such as a color camera while suppressing deterioration in measurement accuracy due to ambient light.SOLUTION: A 3D shape data generating device 1 comprises a bandpass filter 122 having as a transmission band: a wavelength band of a predetermined width that includes the peak wavelength of structured light; a wavelength band with a predetermined width including the peak wavelength of first illumination light; a wavelength band with a predetermined width including the peak wavelength of second illumination light; and a wavelength band with a predetermined width including the peak wavelength of third illumination light. An imaging unit 120 generates pattern image data by receiving the structured light that has passed through the bandpass filter 122, and generates texture image data by receiving observation illumination light. Composite data is generated by combining three-dimensional shape data generated based on the pattern image data and texture data generated based on the texture image data. The composite data is displayed on a display unit 400.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to a three-dimensional shape data generation device that generates three-dimensional shape data of a measurement object.

Background Art

[0002] Conventionally, a three-dimensional shape data generation device that generates three-dimensional shape data of a measurement object has been known. The three-dimensional shape data generation device disclosed in Patent Document 1 includes a projection unit that irradiates a measurement object with pattern light of a certain illuminance, and an auxiliary illumination unit that illuminates the measurement object irradiated with the pattern light. In Patent Document 1, the brightness of the surface of the measurement object including the component of the pattern light and the component of the illumination by the auxiliary illumination unit is detected, and the illumination intensity by the auxiliary illumination unit is controlled so that the detected brightness becomes constant.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, in the three-dimensional shape data generation device of Patent Document 1, the auxiliary illumination unit is controlled so that the brightness of the surface of the measurement object becomes constant. However, even if the detected brightness is constant, the intensity of light (measurement light) of the wavelength used for measurement may vary due to ambient light or the like. If the intensity of the measurement light varies, there is a risk of deterioration in measurement accuracy. In particular, when trying to achieve a wide viewing angle, the deterioration in measurement accuracy due to ambient light becomes an even more prominent problem.

[0005] Another possibility is to place a bandpass filter in front of the image sensor that selectively transmits the wavelength band of the measurement light. However, in this case, light of wavelengths other than the measurement light band will not be transmitted, making it difficult to acquire texture information of the object being measured. To acquire texture information, a separate color camera would be necessary, but color cameras have the problem of lower resolution compared to monochrome cameras.

[0006] Furthermore, methods include setting up the object to be measured in a darkroom to block out ambient light, or using RGB lighting with light-shielding materials such as curtains. In this case, the same camera can be used for both shape measurement and texture information acquisition, but preparing the darkroom is time-consuming, and it is anticipated that light-shielding materials may not be sufficient depending on the shape of the object to be measured.

[0007] This disclosure is made in view of the above points, and its purpose is to enable the acquisition of texture information of the object to be measured without using a separate camera such as a color camera, while suppressing the deterioration of measurement accuracy due to ambient light. [Means for solving the problem]

[0008] To achieve the above objective, the three-dimensional shape data generation apparatus according to this embodiment includes a structured illumination unit that irradiates the object to be measured with structured light for measurement, an observation illumination unit that irradiates the object to be measured with a first illumination light, a second illumination light, and a third illumination light, each having different wavelengths, as observation illumination light, and a bandpass filter. The bandpass filter has a transmission band consisting of a predetermined wavelength band including the peak wavelength of the structured light, a predetermined wavelength band including the peak wavelength of the first illumination light, a predetermined wavelength band including the peak wavelength of the second illumination light, and a predetermined wavelength band including the peak wavelength of the third illumination light. The imaging unit generates pattern image data showing an image of the object to be measured by receiving structured light irradiated from the structured illumination unit, reflected by the object to be measured, and transmitted through the bandpass filter, and generates texture image data of the object to be measured by receiving observation illumination light irradiated from the observation illumination unit, reflected by the object to be measured, and transmitted through the bandpass filter. Based on the pattern image data generated by the imaging unit, the three-dimensional shape data generation unit generates three-dimensional shape data of the object to be measured. Based on the texture image data generated by the imaging unit, the texture data generation unit generates texture data indicating the surface state of the object to be measured. The three-dimensional shape data generated by the three-dimensional shape data generation unit and the texture data generated by the texture data generation unit are combined by the synthesis unit to generate synthesized data, and the generated synthesized data can be displayed on the display unit by the display control unit.

[0009] With this configuration, structured light reflected from the object being measured passes through a bandpass filter and is received by the imaging unit, thus enabling the generation of pattern image data with suppressed interference from ambient light, and avoiding deterioration of the measurement accuracy of the 3D shape data. On the other hand, when the object being measured is illuminated with the first, second, and third illumination lights at different timings, each illumination light passes through a bandpass filter and is received by the imaging unit. By using illumination lights corresponding to, for example, RGB for the first, second, and third illumination lights, it is possible to use, for example, a monochrome high-resolution camera as the imaging unit to improve the measurement accuracy of the 3D shape data, while simultaneously generating texture image data of the object being measured using the imaging unit. Furthermore, by passing the structured light for measurement and the illumination light for observation through the bandpass filter, the signal-to-noise ratio is also improved, further enhancing the measurement accuracy.

[0010] In another embodiment, the imaging unit may be a monochrome camera for receiving the structured light and the illumination light for observation. In this case, the observation illumination unit irradiates the object to be measured with the first illumination light, the second illumination light, and the third illumination light in a time-division manner, and the texture data generation unit can generate a color two-dimensional texture image based on a plurality of texture image data obtained by capturing the first illumination light, the second illumination light, and the third illumination light irradiated from the observation illumination unit in a time-division manner with the imaging unit.

[0011] With this configuration, pattern image data and texture image data can be generated using the same monochrome camera, eliminating the need for mapping between pattern image data and texture image data, thereby simplifying the processing.

[0012] Furthermore, in other embodiments of the bandpass filter, the transmission bandwidth of the structured light may be wider than the transmission bandwidth of the illumination light. That is, the wavelength of the structured light (measurement light) irradiated from the structured illumination unit may vary due to temperature changes, etc. By setting the transmission bandwidth of the structured light for measurement to be wider than that of other wavelengths, the bandpass filter reliably transmits the structured light, so that three-dimensional shape data can be generated without causing a decrease in measurement accuracy.

[0013] Furthermore, the wavelength of the structured light may be set to a wavelength in the blue band, which allows for the irradiation of a projection pattern with high resolution. This can further improve measurement accuracy.

[0014] In another embodiment, the wavelength band of the first illumination light and the wavelength band of the structured light may overlap. The bandpass filter can be configured such that the width of the transmission band of the first illumination light is wider than the width of the wavelength band of the second illumination light and the width of the third wavelength band. The structured light may also be light other than visible light, such as infrared or ultraviolet light, in which case the wavelength band of the illumination light and the wavelength band of the structured light do not overlap.

[0015] Furthermore, the bandpass filter can be placed between the optical system of the imaging unit and the photodetector. In other words, if the bandpass filter is placed in front of the optical system (the surface into which structured light and illumination light are incident), shading may worsen due to the angular characteristics, especially when using a wide-angle optical system. However, by placing the bandpass filter between the optical system of the imaging unit and the photodetector, such deterioration of shading can be avoided. [Effects of the Invention]

[0016] As explained above, by providing a bandpass filter with a transmission band containing the peak wavelength of structured light and transmission bands containing the peak wavelengths of the first to third illumination lights for observation, it is possible to acquire texture information of the object being measured without using a separate camera such as a color camera, while suppressing the deterioration of measurement accuracy due to ambient light.

Brief Description of Drawings

[0017] [Figure 1] This is a diagram showing the overall configuration of a three-dimensional shape data generation device according to Embodiment 1 of the present invention. [Figure 2] This is a block diagram of a three-dimensional shape data generation device. [Figure 3] This is a perspective view of a camera module constituting an imaging unit. [Figure 4] This is an exploded perspective view of the camera module. [Figure 5] This is a graph showing the transmittance of a band-pass filter. [Figure 6] This is a flowchart showing an example of the processing procedure of a three-dimensional shape data generation device. [Figure 7] This is a diagram showing the overall configuration of a three-dimensional shape data generation device according to Embodiment 2 of the present invention.

Modes for Carrying Out the Invention

[0018] Hereinafter, embodiments of the present invention will be described in detail based on the drawings. It should be noted that the following description of the preferred embodiments is merely illustrative in nature and is not intended to limit the present invention, its applications, or its uses.

[0019] (Embodiment 1) FIG. 1 is a diagram showing the overall configuration of a three-dimensional shape data generation device 1 according to Embodiment 1 of the present invention. The three-dimensional shape data generation device 1 is a system that generates three-dimensional shape data of a workpiece (measurement object) W. For example, it is possible to convert the mesh data of the workpiece W obtained by measuring the shape of the workpiece W into CAD data and output it.

[0020] Although not particularly limited, the three-dimensional shape data generation device 1 is used, for example, to obtain CAD data of an existing product and perform next-generation product development and shape analysis on CAD / CAE, to reflect the shape of a model or mock in product design, to design a product to be mated based on the shape of the mating component, or to perform an improved design based on the shape of a prototype. Therefore, examples of the workpiece W include an existing product, a model, a mock, a prototype, etc.

[0021] In addition, the three-dimensional shape data generation device 1 can also convert the mesh data of the workpiece W into surface data and output it. By converting and outputting the mesh data of the workpiece W into surface data, the reverse engineering process and reverse engineering work of the user can be supported, so the three-dimensional shape data generation device 1 can also be called a reverse engineering support device.

[0022] In the following description, when measuring the shape of the workpiece W, in order to obtain the coordinate information on the surface of the workpiece W, structured light for measurement is irradiated onto the workpiece W, and the coordinate information is obtained based on the structured light reflected from the surface of the workpiece W. For example, a measurement method based on triangulation using a fringe projection image obtained from the structured light reflected from the surface of the workpiece W can be applied. However, in the present invention, the principle and configuration for obtaining the coordinate information of the workpiece W are not limited to this, and other methods can also be applied.

[0023] The three-dimensional shape data generation device 1 includes a measurement unit 100, a pedestal unit 200, a controller 300, a display unit 400, and an operation unit 500. As shown in FIG. 2, the measurement unit 100 includes a structured illumination unit 110, an observation illumination unit 115, and an imaging unit 120, and also includes a housing 100A to which the structured illumination unit 110, the observation illumination unit 115, and the imaging unit 120 are attached. Further, the measurement unit 100 also includes a measurement control unit 130 that controls the structured illumination unit 110, the observation illumination unit 115, and the imaging unit 120. The measurement control unit 130 may be provided in the housing 100A or on the controller 300 side.

[0024] The housing 100A is separate from the controller 300 and is supported by a support unit 600. The support unit 600 is portable and comprises a base unit 601, an extendable unit 602 fixed to the base unit 601, and an angle adjustment unit 603 provided on the upper part of the extendable unit 602, allowing the user to freely set the installation position. The measuring unit 100 is detachably attached to the angle adjustment unit 603. The height of the measuring unit 100 can be adjusted by extending or retracting the extendable unit 602 in the vertical direction. Furthermore, the angle adjustment unit 603 is configured to allow adjustment of, for example, rotation around the horizontal axis, rotation around the vertical axis, and rotation around the inclination axis. This allows the installation angle of the measuring unit 100 relative to the horizontal plane and the installation angle relative to the vertical plane to be adjusted arbitrarily.

[0025] The support section 600 is not limited to the configuration described above, and may be composed of, for example, a tripod, a flexible arm that can be freely bent and maintain a bent shape, a bracket, or a combination of these. The measuring section 100 can also be used by attaching it to, for example, a 6-degree-of-freedom arm of an industrial robot. Furthermore, the measuring section 100 can be used by the user holding it by hand, in which case the support section 600 is unnecessary. In other words, the support section 600 may be a component included in the 3D shape data generation device 1, or it may be a component not included in the 3D shape data generation device 1.

[0026] When a user takes the housing 100A and measures the workpiece W, the measuring unit 100 can be brought to the manufacturing site of the workpiece W for measurement. In this case, the user can measure the shape of the workpiece W by moving the measuring unit 100 to any position and taking images at any time. This can be called manual measurement.

[0027] Furthermore, by supporting the measuring unit 100 with the support unit 600 and placing the workpiece W on the automatically rotating base unit 200 (described later), the shape of a wide area of ​​the workpiece W can be measured by rotating the workpiece W on the base unit 200 and taking images at predetermined timings. This can be called semi-automatic measurement. Note that the workpiece W can also be measured by placing it on a surface plate or the like, for example, without placing it on the base unit 200.

[0028] Furthermore, by attaching the measuring unit 100 to the arm of an industrial robot and moving it, the shape of a wide range of the workpiece W can be measured without the user's intervention. This can be called fully automatic measurement. The present invention is applicable to all manual, semi-automatic, and fully automatic measurements.

[0029] The operation unit 500 may include, for example, a pointing device such as a keyboard 501 or a mouse 502. A joystick may also be used as the pointing device. Furthermore, the operation unit 500 may include a touch panel or the like that senses user touch operations. The operation unit 500 is connected to the arithmetic unit 301 within the controller 300, and the arithmetic unit 301 can detect what operations are performed by the operation unit 500.

[0030] The base unit 200 comprises a base plate 201, a stage 202 that forms a mounting surface on which the workpiece W is placed, and a rotation mechanism 203. The base unit 200 may also include a clamping mechanism for clamping the workpiece W on the stage 202. The rotation mechanism 203 is provided between the base plate 201 and the stage 202 and is a mechanism that rotates the stage 202 about a vertical axis (the Z-axis shown in Figure 1) relative to the base plate 201. Therefore, the stage 202 is a rotating stage, and by rotating it with the workpiece W placed on it, it is possible to switch the relative positional relationship of the workpiece W with respect to the imaging unit 120. The direction of rotation about the Z-axis is defined as the θ direction and is indicated by the arrow θ. The base unit 200 may also include a tilt stage having a mechanism that can rotate about an axis parallel to the mounting surface.

[0031] The rotation mechanism 203 has a motor or the like controlled by the measurement control unit 130, which will be described later, and is capable of rotating the stage 202 by a desired rotation angle and then holding it in a stopped state. The base portion 200 is not an essential component of the present invention and is provided as needed. The base portion 200 may also be controlled by the controller 300.

[0032] Although not shown in the figures, the base portion 200 may be equipped with a translation mechanism that moves the stage 202 horizontally in the X and Y directions, which are mutually orthogonal. The translation mechanism also has a motor controlled by the measurement control unit 130 and the controller 300, and is capable of moving the stage 202 by a desired amount in the X and Y directions and then holding it in a stopped state. The present invention is also applicable even if the stage 202 is a fixed stage.

[0033] As shown in Figure 2, the structured illumination unit 110 is the part that irradiates the workpiece W with structured light for measurement. The measurement unit 100 may include a plurality of structured illumination units 110. For example, there may be a first structured illumination unit capable of irradiating the workpiece W with first structured light from a first direction, and a second structured illumination unit capable of irradiating the workpiece W with second structured light from a second direction different from the first direction.

[0034] Although not shown in the figures, it is also possible to have three or more structured illumination units 110, or to move the structured illumination unit 110 and the base unit 200 relative to each other, so that even while using a common structured illumination unit 110, the direction of illumination of the structured light can be different and projected onto the workpiece W. In addition to providing multiple structured illumination units 110 and receiving the light with a common imaging unit 120, it is also possible to provide multiple imaging units 120 for a common structured illumination unit 110 and configure them to receive the light. Furthermore, the illumination angle of the structured light projected by the structured illumination unit 110 with respect to the Z direction may be fixed or variable.

[0035] The structured illumination unit 110 includes a measurement light source 111, a pattern generation unit 112, and a plurality of lenses 113. The measurement light source 111 can be a light source that emits monochromatic light, such as a halogen lamp that emits white light, a blue LED (light-emitting diode) that emits blue light, or an organic EL. In this example, the measurement light source 111 is composed of a blue LED that emits blue light, but light in a wavelength range other than visible light may also be used as structured light for measurement, for example, the measurement light source 111 may be composed of a light-emitting element that emits infrared or ultraviolet light. The light emitted from the measurement light source 111 is focused and then incident on the pattern generation unit 112.

[0036] The pattern generation unit 112 reflects the light emitted from the measurement light source 111 so that structured light is irradiated onto the workpiece W. The measurement light incident on the pattern generation unit 112 is converted into a preset pattern and a preset intensity (brightness) and emitted. The structured light emitted by the pattern generation unit 112 is converted by multiple lenses 113 into light with a diameter larger than the observation and measurement field of view of the imaging unit 120, and then irradiated onto the workpiece W.

[0037] The pattern generation unit 112 is a component that can switch between an irradiation state in which structured light is irradiated onto the workpiece W and a non-irradiation state in which structured light is not irradiated onto the workpiece W. For example, a DMD (Digital Micromirror Device) can be used for such a pattern generation unit 112. A pattern generation unit 112 using a DMD can be controlled by the measurement control unit 130 to switch between a reflection state in which structured light is reflected towards the optical path as the irradiation state and a light-shielding state in which structured light is blocked as the non-irradiation state.

[0038] The pattern generation unit 112 is an element in which a large number of micromirrors (tiny mirror surfaces) are arranged on a plane (in the X and Y directions). Each micromirror can be individually switched ON or OFF by the measurement control unit 130, so by combining the ON and OFF states of a large number of micromirrors, it is possible to generate light with a desired projection pattern as structured light for measurement. This makes it possible to generate the pattern necessary for triangulation and measure the shape of the workpiece W. In this way, the pattern generation unit 112 functions as part of the optical system that irradiates the workpiece W with a periodic projection pattern for measurement during measurement. Furthermore, the DMD has excellent response speed and offers the advantage of being able to operate at a higher speed compared to shutters and the like.

[0039] By controlling the pattern generation unit 112, multiple types of structured light having a light-dark period can be generated. Specifically, the pattern generation unit 112 can generate structured light such as Gray code pattern light or multi-slit pattern light. By controlling the pattern generation unit 112, multiple Gray code pattern lights can be generated, and multiple multi-slit pattern lights with different phases can be generated by changing the phase of the multi-slit pattern light. Changing the phase of multi-slit pattern light is called phase shifting.

[0040] The structured illumination unit 110 can, when measuring the shape of the workpiece W, control the pattern generation unit 112 to irradiate the workpiece W with Gray code pattern light of different brightness-dark periods multiple times, and can also irradiate the workpiece W with multi-slit pattern light multiple times while phase-shifting it. Furthermore, the structured illumination unit 110 may be configured to generate patterns such as random dots and irradiate the workpiece W with them. In this case, shape measurement can be performed using the random dot method, and imaging is only required once.

[0041] In the above example, an example using a DMD for the pattern generation unit 112 was described, but the present invention is not limited to a DMD for the pattern generation unit 112, and other materials can be used. For example, an LCOS (Liquid Crystal on Silicon: reflective liquid crystal element) may be used as the pattern generation unit 112. Alternatively, a transmissive material may be used instead of a reflective material to adjust the amount of structured light transmitted. In this case, the pattern generation unit 112 is placed on the optical path and switches between an illumination state that transmits light and a light-blocking state that blocks light. For example, an LCD (liquid crystal display) can be used for such a pattern generation unit 112. Alternatively, the pattern generation unit 112 may be configured using a projection method using multiple line LEDs, a projection method using multiple optical paths, an optical scanner method composed of a laser and a galvanometer mirror, an AFI (Accordion fringe interferometry) method that uses interference fringes generated by superimposing beams divided by a beam splitter, or a projection method using a physical grid composed of a piezo stage and a high-resolution encoder and a moving mechanism. Furthermore, the pattern generation unit 112 can also irradiate uniform light without generating a pattern.

[0042] The observation illumination unit 115 is the part that illuminates the workpiece W with red light, green light, and blue light, each with a different wavelength, as illumination light for observation. That is, the observation illumination unit 115 is equipped with a red light source 116, a green light source 117, and a blue light source 118. The red light source 116, green light source 117, and blue light source 118 are composed of LEDs, for example, and the measurement control unit 130 controls their on / off state and brightness when on individually. In addition to LEDs, other light sources such as semiconductor lasers (LDs), halogen lights, and HIDs can also be used as appropriate for the light sources of the observation illumination unit 115. In this example, since the measurement light source 111 of the structured illumination unit 110 is composed of a blue LED that emits blue light, the wavelength band of the structured light and the wavelength band of the blue light emitted from the blue light source 118 of the observation illumination unit 115 overlap. The wavelength bands of the structured light and the blue light used for observation may completely overlap or partially overlap, but it is preferable that at least the peak wavelength of the structured light overlaps with the wavelength band of the blue light used for observation. Furthermore, if the measurement light source 111 of the structured illumination unit 110 emits infrared or ultraviolet light, the wavelength bands of the structured light and the wavelength bands of each color in the observation illumination unit 115 will not overlap.

[0043] The measurement control unit 130 controls the observation illumination unit 115 so that only the red light source 116 is lit while the green light source 117 and blue light source 118 are turned off to generate red illumination light. Alternatively, only the green light source 117 is lit while the red light source 116 and blue light source 118 are turned off to generate green illumination light. Similarly, only the blue light source 118 is lit while the green light source 117 and red light source 116 are turned off to generate blue illumination light. In this way, the observation illumination unit 115 is capable of irradiating the workpiece W with red, green, and blue light in a time-division manner. Furthermore, by lit all of the red, green, and blue light sources 116, white illumination light can also be generated.

[0044] Furthermore, the configuration of the observation illumination unit 115 is not limited to the configuration described above. For example, it may be individual illumination units, each having a red light source 116, a green light source 117, and a blue light source 118.

[0045] The imaging unit 120 includes a light-receiving element 121, a bandpass filter 122, and a plurality of lenses 123, and is composed of a camera module CM as shown in Figure 3. The lenses 123 are components that constitute an optical system into which structured light and illumination light are incident. The light-receiving element 121 is a component that receives light emitted from the lenses 123 and outputs an electrical signal corresponding to the amount of light received. The light-receiving element 121 is composed of an image sensor such as a CCD (charge-coupled device) or CMOS (complementary metal-oxide-semiconductor). From each pixel of the light-receiving element 121, an analog electrical signal corresponding to the amount of light received (hereinafter referred to as the "receiving signal") is output to an A / D converter, which will be described later. In this embodiment, a monochrome camera with higher resolution than a color camera is used as the light-receiving element 121.

[0046] As shown in Figure 4, the camera module CM includes a holder 125 to which the light-receiving element 121 is fixed. The holder 125 has a lens fixing hole 125a to which the lens 123 is fixed. The light-receiving element 121 is positioned at the back of the lens fixing hole 125a, with its light-receiving surface facing the lens fixing hole 125a.

[0047] A mounting seat 125b for fixing the bandpass filter 122 in a predetermined position is provided on the inner peripheral edge of the lens fixing hole 125a, projecting radially inward from the lens fixing hole 125a. The bandpass filter 122 is disc-shaped, and its inner surface (the surface facing the light-receiving element 121) abuts against the mounting seat 125b. An annular retainer 126 is interposed between the front surface of the bandpass filter 122 and the lens 123. When the lens 123 is fixed in the lens fixing hole 125a, the lens 123 presses the retainer 126 against the bandpass filter 122, and the bandpass filter 122, pressed by the retainer 126, is pressed against the mounting seat 125b and fixed in a predetermined position.

[0048] In this way, the bandpass filter 122 is positioned between the light-emitting surface of the lens 123 and the light-receiving surface of the photodetector 121. If the bandpass filter 122 is placed in front of the lens 123 (the surface into which structured light and illumination light enter), shading may worsen due to the angular characteristics, especially when using a wide-angle lens 123. However, by placing the bandpass filter 122 between the lens 123 and the photodetector 121, such deterioration of shading can be avoided. Note that if a lens 123 with an angle of view in which shading is not a problem is used, the bandpass filter 122 may be placed in front of the lens 123.

[0049] Furthermore, if the bandpass filter 122 is too close to the light-emitting surface of the lens 123, it becomes more susceptible to surface accuracy issues. On the other hand, if the bandpass filter 122 is too close to the photodetector 121, it becomes more susceptible to dust and scratches. To minimize both of these effects, in this example, the bandpass filter 122 is positioned in the center between the light-emitting surface of the lens 123 and the light-receiving surface of the photodetector 121. The bandpass filter 122 may also be positioned closer to the light-emitting surface of the lens 123, or closer to the light-receiving surface of the photodetector 121.

[0050] The structured light for measurement, emitted from the structured illumination unit 110 and reflected by the workpiece W, is incident on the lens 123, focused and imaged, then passes through the bandpass filter 122 and is sequentially received by the photodetector 121 to sequentially generate multiple pattern image data representing the image of the workpiece W. In addition, each illumination light for observation, emitted from the observation illumination unit 114 and reflected by the workpiece W, is incident on the lens 123, focused and imaged, then passes through the bandpass filter 122 and is received by the photodetector 121 to generate texture image data of the workpiece W.

[0051] The imaging unit 120 may include a high-magnification imaging unit equipped with a high-magnification lens 123 and a low-magnification imaging unit equipped with a low-magnification lens 123. Furthermore, the lens 123 may be a zoom lens or the like with adjustable magnification, or the imaging unit 120 itself may have adjustable magnification. The magnification at the time of imaging is associated with the image data, making it possible to identify the magnification at which the image data was captured.

[0052] The imaging unit 120 is equipped with an A / D converter (analog-to-digital converter), a FIFO (First In First Out) memory, a CPU, and other components (not shown). The light-receiving signal output from the light-receiving element 121 is sampled at a constant sampling period by the A / D converter and converted into a digital signal. The digital signals output from the A / D converter are sequentially stored in the FIFO memory. The digital signals stored in the FIFO memory are sequentially output to the CPU as pixel data, and the CPU generates pattern image data.

[0053] Based on the light-receiving signal output from the photodetector 121, pattern image data representing the three-dimensional shape of the workpiece W contained within the field of view of the photodetector 121 at a specific position is generated. The pattern image data is the image itself acquired by the photodetector 121. For example, when measuring the shape of the workpiece W using the phase-shift method, multiple images will constitute the pattern image data.

[0054] When the structured illumination unit 110 irradiates the workpiece W with multi-slit pattern light while shifting its phase, the imaging unit 120 sequentially receives the multi-slit pattern light reflected from the workpiece W each time the multi-slit pattern light irradiates the workpiece W. As a result, the imaging unit 120 sequentially generates multiple pattern image data showing the image of the workpiece W when irradiated with the multi-slit pattern light.

[0055] Furthermore, when the structured illumination unit 110 irradiates the workpiece W with Gray code pattern light, the imaging unit 120 sequentially receives the Gray code pattern light reflected from the workpiece W each time the Gray code pattern light is irradiated onto the workpiece W. As a result, the imaging unit 120 sequentially generates multiple pattern image data showing the image of the workpiece W when irradiated with Gray code pattern light.

[0056] The pattern image data may also be point cloud data, which is a collection of points having three-dimensional positional information. Pattern image data of the workpiece W can be obtained using this point cloud data. Point cloud data is data represented by a collection of multiple points having three-dimensional coordinates. The generated pattern image data is transferred to the controller 300.

[0057] The bandpass filter 122 has transmission bands of a predetermined width including the peak wavelength of the structured light, a predetermined width including the peak wavelength of the red light (illumination light), a predetermined width including the peak wavelength of the green light (illumination light), and a predetermined width including the peak wavelength of the blue light (illumination light). The predetermined width including the peak wavelength of the blue light is called the blue wavelength band, the predetermined width including the peak wavelength of the red light is called the red wavelength band, and the predetermined width including the peak wavelength of the green light is called the green wavelength band. In this example, since the structured light is blue light, the red wavelength band, the green wavelength band, and the blue wavelength band are the transmission bands of the bandpass filter 122. If the structured light is infrared or ultraviolet light, the bandpass filter 122 will have transmission bands of a predetermined width including the peak wavelength of infrared light and a predetermined width including the peak wavelength of ultraviolet light.

[0058] The graph shown in Figure 5 has the wavelength of light (nm) on the horizontal axis and the transmittance on the vertical axis. The wavelengths of blue light from structured light and illumination light are shown as "blue light," the wavelengths of green light from illumination light are shown as "green light," and the wavelengths of red light from illumination light are shown as "red light." The wavelengths of light emitted from fluorescent lamps and sunlight are also shown in the same figure. The transmission characteristics of the bandpass filter 122 are shown by the dashed lines, with the blue wavelength band, green wavelength band, and red wavelength band being the transmission bands, and there are regions with relatively low light transmittance between each wavelength. In other words, the bandpass filter 122 is configured to selectively transmit the blue wavelength band, green wavelength band, and red wavelength band.

[0059] For example, the transmittance of blue light in the blue wavelength band can be set to, for example, 90% or higher, the transmittance of green light in the green wavelength band can be set to, for example, 45% or higher, and the transmittance of red light in the red wavelength band can be set to, for example, 80% or higher. When set in this way, the transmittance of light emitted from, for example, a fluorescent lamp will be 50% or less, and the transmittance of sunlight will be 40% or less. This suppresses the effects of ambient light.

[0060] The transmittance at wavelengths shorter than the blue wavelength band is almost zero. Furthermore, the transmittance at wavelengths between the blue and green wavelength bands is also almost zero, as is the transmittance at wavelengths longer than the red wavelength band. Note that the blue wavelength band of the illumination light corresponds to the wavelength band of the first illumination light of the present invention, the green wavelength band of the illumination light corresponds to the wavelength band of the second illumination light of the present invention, and the red wavelength band of the illumination light corresponds to the wavelength band of the third illumination light of the present invention.

[0061] The bandpass filter 122 is configured such that the transmission bandwidth of the structured light is wider than the transmission bandwidth of the illumination light. Specifically, the transmission bandwidth of blue light is wider than the transmission bandwidth of the green and red wavelengths. By widening the transmission bandwidth of the structured light, even when a greater amount of light is required depending on the material of the workpiece W, the loss of light intensity by the bandpass filter 122 can be minimized, thereby improving measurement accuracy. In addition, the wavelength of the structured light emitted from the structured illumination unit 110 may vary due to temperature changes, etc. By setting the transmission bandwidth of the structured light for measurement to be wider than that of other wavelengths, the bandpass filter 122 reliably transmits the structured light, so that 3D shape data can be generated without causing a decrease in measurement accuracy.

[0062] During the manufacturing of the bandpass filter 122, multiple films are deposited on a material such as a colorless, transparent glass plate. This results in a bandpass filter 122 with multiple transmission bands as described above. However, if multiple films are deposited on only one side of the material, the surface accuracy after deposition may deteriorate due to the stress caused. To minimize this effect, a stress-relieving film is formed on the other side. The stress-relieving film can also be used in conjunction with an AR coating, which is generally deposited to prevent light loss. By forming a stress-relieving film, the deterioration of the surface accuracy of the bandpass filter 122 is suppressed, and consequently, the measurement accuracy can be improved. The stress-relieving film can be provided as needed and is not essential.

[0063] The controller 300 includes an arithmetic unit 301, a working memory 302, a ROM (read-only memory) 303, a storage unit 304, and a display control unit 305 that controls the display unit 400. The controller 300 can use a PC (personal computer), but it may also consist of a dedicated computer only, or a combination of a PC and a dedicated computer.

[0064] The ROM 303 of the controller 300 stores, for example, a system program. The working memory 302 of the controller 300 consists of, for example, RAM (Random Access Memory) and is used for processing various data. The storage unit 304 consists of, for example, a solid-state drive or a hard disk drive. The storage unit 304 stores a program for generating three-dimensional shape data. The storage unit 304 is also used to store various data such as pixel data (pattern image data, texture image data) provided by the measurement control unit 130 and measurement conditions for the workpiece W. Measurement conditions include, for example, the settings of the structured illumination unit 110 (pattern frequency, pattern type), the magnification of the imaging unit 120, the measurement field of view (single field of view or wide field of view), the measurement position, the rotational orientation, the exposure conditions (exposure time, gain, brightness of illumination), and the resolution setting (low-resolution measurement, standard measurement, high-resolution measurement).

[0065] The arithmetic unit 301 consists of control circuits and control elements that process given signals and data, perform various calculations, and output calculation results. In this specification, the arithmetic unit 301 refers to the elements and circuits that perform calculations, and is used to mean not limited to processors such as CPUs, MPUs, GPUs, and TPUs for general-purpose PCs, regardless of their name, but also including processors such as FPGAs, ASICs, and LSIs, microcontrollers, and chipsets such as SoCs.

[0066] The arithmetic unit 301 performs various processing on the pattern image data generated by the imaging unit 120 using the working memory 302. The arithmetic unit 301 comprises the 3D shape data generation unit 301a, the texture data generation unit 301b, and the synthesis unit 301c, etc. The 3D shape data generation unit 301a, the texture data generation unit 301b, and the synthesis unit 301c may consist solely of the hardware of the arithmetic unit 301, or they may consist of a combination of hardware and software. For example, by executing a 3D shape data generation program, the arithmetic unit 301 enables the realization of each function of the 3D shape data generation unit 301a, the texture data generation unit 301b, and the synthesis unit 301c. The 3D shape data generation program may be stored on various storage media such as CD-ROMs or DVD-ROMs, or it may be stored on a server and provided in a form that can be downloaded by the user.

[0067] The 3D shape data generation unit 301a is the part that generates 3D shape data of the workpiece W based on multiple pattern image data generated by the imaging unit 120. For example, the 3D shape data generation unit 301a calculates the relative phase of the workpiece W based on the pattern image data of the multi-slit pattern light and calculates the absolute phase of the workpiece W based on the pattern image data of the Gray code pattern light. Subsequently, the 3D shape data generation unit 301a generates phase images, point cloud data, mesh data, etc., i.e., performs 3D image processing and drawing processing based on the relative phase and absolute phase of the workpiece W.

[0068] The texture data generation unit 301b is the part that generates texture data indicating the surface state of the workpiece W based on multiple texture image data generated by the imaging unit 120. Specifically, the observation illumination unit 115 illuminates the workpiece W with red light, green light, and blue light in a time-division manner. The imaging unit 120 receives the reflected light from the workpiece W when red light is irradiated onto it and generates texture image data for the red light illumination. The imaging unit 120 also receives the reflected light from the workpiece W when green light is irradiated onto it and generates texture image data for the green light illumination. Furthermore, the imaging unit 120 receives the reflected light from the workpiece W when blue light is irradiated onto it and generates texture image data for the green light illumination. In this way, the imaging unit 120 generates at least three texture image data by time-division imaging the workpiece W irradiated with red light, green light, and blue light, respectively. The texture data generation unit 301b generates a color two-dimensional texture image based on the multiple texture image data generated in time division.

[0069] The synthesis unit 301c generates composite data by combining the 3D shape data generated by the 3D shape data generation unit 301a and the texture data generated by the texture data generation unit 301b. This makes it possible to obtain a 3D shape image in which the texture can be recognized. In this example, since the same monochrome camera can be used to generate both pattern image data and texture image data, mapping between the pattern image data and texture image data becomes unnecessary, thereby simplifying the processing.

[0070] Figure 6 is a flowchart showing an example of the processing procedure of a 3D shape data generation device. When a texture image is required, the user operates the operation unit 500 to execute the process shown in this flowchart. For example, when the user operates the measurement start button (not shown) using the operation unit 500, step SA1 is executed, and the measurement control unit 130 controls the structured illumination unit 110 to irradiate structured light. In step SA2, the measurement control unit 130 controls the imaging unit 120, and the structured light reflected from the workpiece W is received by the imaging unit 120. At this time, only the light that has passed through the bandpass filter 122 is received by the photodetector 121, so that pattern image data with the influence of ambient light suppressed is generated. Steps SA1 and SA2 are repeated multiple times depending on the type of structured light, etc.

[0071] Next, the process proceeds to step SA3, where the measurement control unit 130 controls the observation illumination unit 115 to irradiate with red light. In step SA4, the measurement control unit 130 controls the imaging unit 120, which receives the light reflected from the workpiece W. At this time, since red light is one of the transmission bands of the bandpass filter 122, it is received by the photodetector 121 and texture image data of the time when red light was irradiated is generated.

[0072] In step SA5, the measurement control unit 130 controls the observation illumination unit 115 to emit green light. In step SA6, the measurement control unit 130 controls the imaging unit 120 to receive the light reflected from the workpiece W. At this time, since green light is one of the transmission bands of the bandpass filter 122, it is received by the photodetector 121 and texture image data of the green light illumination is generated.

[0073] In step SA7, the measurement control unit 130 controls the observation illumination unit 115 to irradiate with blue light. In step SA8, the measurement control unit 130 controls the imaging unit 120 to receive the light reflected from the workpiece W. At this time, since blue light is one of the transmission bands of the bandpass filter 122, it is received by the photodetector 121 and texture image data of the blue light irradiation is generated. Note that the order of steps SA3, SA4, steps SA5, SA6, and steps SA7, SA8 may be changed.

[0074] In step SA9, the synthesis unit 301c generates composite data by combining the three-dimensional shape data generated by the three-dimensional shape data generation unit 301a and the texture data generated by the texture data generation unit 301b.

[0075] In step SA10, the display control unit 305 displays the composite data generated in step SA9 on the display unit 400. This allows the user to view the composite data, which is a combination of 3D shape data and texture data, as an image.

[0076] (Embodiment 2) Figure 7 shows the overall configuration of the three-dimensional shape data generation device 1 according to Embodiment 2 of the present invention. The three-dimensional shape data generation device 1 according to Embodiment 2 differs from that of Embodiment 1 in that the measuring unit 100 and the base unit 200 are integrated. Hereinafter, the same reference numerals are used for parts that are the same as in Embodiment 1 and their descriptions are omitted, while the different parts will be described in detail.

[0077] Specifically, a support section 250 for supporting the measuring section 100 is provided on the rear side of the base section 200, extending upward. The measuring section 100 is fixed to the upper part of this support section 250. The measuring section 100 is equipped with a structured illumination section 110 and an imaging section 120 such that the optical axis is directed toward the stage 202.

[0078] Even with a three-dimensional shape data generation device 1 like that in Embodiment 2, three-dimensional shape data can be generated in the same way as in Embodiment 1.

[0079] The embodiments described above are merely illustrative in all respects and should not be interpreted restrictively. Furthermore, any modifications or changes that fall within the equivalent scope of the claims are all within the scope of the present invention. [Industrial applicability]

[0080] As described above, the 3D shape data generation device relating to this disclosure can be used to generate 3D shape data of a workpiece. [Explanation of Symbols]

[0081] 1. 3D Shape Data Generation Device 110 Structured lighting section 115 Observation lighting unit 120 Imaging Unit 122 Bandpass Filter 130 Imaging control unit 301a 3D Shape Data Generation Unit 301b Texture Data Generation Unit 301c Synthesis Department 305 Display Control Unit 400 Display

Claims

1. A structured illumination unit that irradiates the object to be measured with structured light for measurement, An observation illumination unit that irradiates the object to be measured with a first illumination light, a second illumination light, and a third illumination light, each having different wavelengths, as observation illumination light, A bandpass filter having a transmission band comprising a predetermined wavelength band including the peak wavelength of the structured light, a predetermined wavelength band including the peak wavelength of the first illumination light, a predetermined wavelength band including the peak wavelength of the second illumination light, and a predetermined wavelength band including the peak wavelength of the third illumination light, An imaging unit generates pattern image data showing an image of the object to be measured by receiving structured light that is irradiated from the structured illumination unit, reflected by the object to be measured, and transmitted through the bandpass filter, and also generates texture image data of the object to be measured by receiving observation illumination light that is irradiated from the observation illumination unit, reflected by the object to be measured, and transmitted through the bandpass filter, A three-dimensional shape data generation unit generates three-dimensional shape data of an object to be measured based on pattern image data generated by the imaging unit, A texture data generation unit generates texture data indicating the surface state of an object to be measured based on the texture image data generated by the imaging unit, A synthesis unit generates composite data by combining the three-dimensional shape data generated by the three-dimensional shape data generation unit and the texture data generated by the texture data generation unit. A three-dimensional shape data generation apparatus comprising: a display control unit that displays the composite data generated in the composite unit on a display unit; and a display control unit that displays the composite data generated in the composite unit on a display unit.

2. In the three-dimensional shape data generation apparatus according to claim 1, The imaging unit has a monochrome camera for receiving the structured light and the illumination light for observation. The observation illumination unit irradiates the object to be measured with the first illumination light, the second illumination light, and the third illumination light in a time-division manner. The three-dimensional shape data generation apparatus is characterized in that the texture data generation unit generates a color two-dimensional texture image based on a plurality of texture image data obtained by capturing the first illumination light, the second illumination light, and the third illumination light irradiated from the observation illumination unit in a time-division manner with the imaging unit.

3. A three-dimensional shape data generation apparatus according to claim 1, The three-dimensional shape data generation apparatus is characterized in that the bandpass filter is configured such that the width of the transmission band of the structured light is wider than the width of the transmission bands of the second illumination light and the third illumination light.

4. A three-dimensional shape data generation apparatus according to claim 1, A three-dimensional shape data generation device characterized in that the wavelength of the structured light is in the blue wavelength band.

5. A three-dimensional shape data generation apparatus according to claim 1, The wavelength band of the first illumination light and the wavelength band of the structured light overlap, The three-dimensional shape data generation apparatus is characterized in that the bandpass filter is configured such that the width of the transmission band of the first illumination light is wider than the width of the wavelength band of the second illumination light and the width of the wavelength band of the third illumination light.

6. A three-dimensional shape data generation apparatus according to claim 5, The wavelength band of the first illumination light is the blue band. The wavelength band of the second illumination light is the green band. The wavelength band of the third illumination light is the red band. The three-dimensional shape data generation apparatus is characterized in that the bandpass filter is configured such that the width of the transmission band for wavelengths in the blue band is wider than the width of the transmission bands for wavelengths in the green band and the red band.

7. A three-dimensional shape data generation apparatus according to claim 1, The imaging unit comprises an optical system into which the structured light and the illumination light are incident, and a photodetector that receives the light emitted from the optical system and outputs an electrical signal corresponding to the amount of light received. The three-dimensional shape data generation apparatus is characterized in that the bandpass filter is placed between the optical system and the light-receiving element.

Images