Three-Dimensional Shape Measurement Device, Three-Dimensional Shape Measurement Method, and Storage Medium

Abstract

The three-dimensional shape measurement device includes: a projection unit that projects pattern light onto a measurement object from a plurality of directions with different angles around the vertical axis with respect to the measurement object; an imaging unit that images the measurement object; and a measurement unit that measures the three-dimensional shape of the measurement object based on the images captured by the imaging unit. The imaging unit acquires a plurality of pattern projection images with different reflection forms, the plurality of pattern projection images being a plurality of images obtained by projecting pattern light from the plurality of directions, the reflection form being the reflection form including the intensity of the pattern light at a portion of the measurement object having an inclination. The measurement unit measures the direction of the inclination based on the differences in a specified feature amount of each pixel constituting the plurality of pattern projection images between the images.

Description

Technical Field

[0001] The present invention relates to a three-dimensional shape measuring device, a three-dimensional shape measurement method, and a program. Background Art

[0002] Conventionally, in the technical field of inspecting the solder joint state of components mounted on a printed circuit board, etc., a three-dimensional shape measurement method based on the so-called phase shift method is known. The phase shift method is a method of obtaining (taking) a plurality of images of a state in which pattern light is projected onto an object surface while changing the phase, and restoring the three-dimensional shape of the object surface by analyzing the deformation of the pattern in the plurality of images. When measuring a printed circuit board using such a phase shift method, there is a problem that light irradiated onto a highly specular member such as solder undergoes specular reflection, which has an adverse effect on the measurement accuracy.

[0003] In response to this, for example, Patent Document 1 discloses a substrate inspection device that combines the above-described phase shift method with three-dimensional shape measurement of a specular object based on the so-called color highlight method. In addition, the color highlight method is a method in which light of a plurality of colors (wavelengths) is irradiated onto a substrate at different incident angles, and the substrate is photographed in a state where a color characteristic corresponding to its normal direction (the color of the light source in the specular reflection direction when observed from the camera) appears on the solder surface, thereby capturing the three-dimensional shape of the solder surface as two-dimensional hue information. Thus, by performing measurement based on color highlight on the specular part of the printed circuit board and measurement based on the phase shift method on a diffuse reflection object such as resin, the accuracy of three-dimensional shape measurement can be improved.

[0004] Prior Art Documents

[0005] Patent Documents

[0006] Patent Document 1: Japanese Patent Application Laid-Open No. 2016-11857 Summary of the Invention

[0007] Problems to be Solved by the Invention

[0008] However, according to the inspection device described in the above Patent Document 1, there is also a problem that it is difficult to determine the "direction" of the inclination of the surface of the measurement object. That is, in the case of inspecting a printed circuit board, it is difficult to accurately determine whether the inclination of the solder surface is relative to the electrode deposition (so-called wetting bulge) or relative to the electrode becoming lower (so-called non-wetting state).

[0009] The present invention has been made in view of the above actual situation, and an object thereof is to provide a technique for accurately determining the direction of the inclination included in a measurement object in three-dimensional shape measurement.

[0010] Means for Solving the Problems

[0011] To achieve the above object, the present invention adopts the following structure.

[0012] The three-dimensional shape measurement device of the present invention is characterized in that the three-dimensional shape measurement device includes: a projection unit that projects pattern light onto a measurement object from a plurality of directions with different angles around a vertical axis with respect to the measurement object; a photographing unit that photographs the measurement object; and a measurement unit that measures the three-dimensional shape of the measurement object based on an image photographed by the photographing unit. The photographing unit obtains a plurality of pattern projection images with different reflection forms, the plurality of pattern projection images are a plurality of images obtained by projecting pattern light from the plurality of directions, the reflection form is a reflection form including the intensity of the pattern light at a portion of the measurement object having an inclination, and the measurement unit measures the direction of the inclination based on the difference in a specified feature amount of each pixel constituting the plurality of pattern projection images between the images.

[0013] In addition, the "measurement" mentioned here also includes the case of measurement by calculation (the same applies hereinafter). In addition, the "pattern" refers to, for example, a stripe pattern in which the change in brightness shows periodicity and a pattern capable of changing the phase over time. In addition, the "inclination" mentioned here not only refers to the inclination defined by a straight line with respect to the horizontal plane, but also includes the inclination in a broad sense defined by a curve. In addition, the inclined plane including this inclination is also understood to mean an inclined plane in a broad sense including not only a plane but also a curved surface. Hereinafter, the same meaning is understood in this specification.

[0014] In addition, the "plurality of pattern projection images with different reflection forms including the intensity of the pattern light" does not refer to a plurality of images with only different phases of the same pattern irradiated from the same direction, but represents a plurality of images with different reflection forms of the pattern light due to irradiating the pattern light from a plurality of different directions.

[0015] With such a structure, it is possible to measure the direction of the inclination included in the measurement object, and thus it is possible to accurately measure the three-dimensional shape of the inclined portion. In addition, as the specified feature amount, for example, the brightness of each pixel can be set.

[0016] In addition, it may be that the projection unit projects the pattern light onto the measurement object from at least a plurality of different positions opposed to each other with the measurement object as the center on a circumference centered on the measurement object, thereby projecting the pattern light from the plurality of directions. With such a configuration, it is possible to suppress a portion where a shadow not irradiated with the pattern light is generated in the measurement object.

[0017] Alternatively, the projection unit may be configured to be rotatable in the circumferential direction centered on the measurement object, and project pattern light onto the measurement object from a plurality of different positions on a circumference centered on the measurement object, thereby projecting pattern light from the plurality of directions.

[0018] With such a structure, it is possible to irradiate the measurement object with pattern light from a desired position on the circumference, and irradiate the pattern light from the optimal direction regardless of the orientation and shape of the specified part of the measurement object. Alternatively, a plurality of the projection units may be arranged at a plurality of different positions on a circumference centered on the measurement object, and the plurality of projection units project pattern light onto the measurement object respectively, thereby projecting pattern light from the plurality of directions.

[0019] Alternatively, the three-dimensional shape measurement device may further include an illumination unit that irradiates the measurement object with illumination light of different wavelengths from a plurality of different angles between the vertical direction and the horizontal direction, the imaging unit further obtains an illumination light irradiation image of the measurement object irradiated with the illumination light, and the measurement unit further measures the degree of the inclination based on the reflection form including either the intensity or the wavelength of the illumination light in the illumination light irradiation image.

[0020] If such a structure is adopted, three-dimensional shape measurement based on the color highlight method can be performed, and the degree of inclination included in the measurement object can be measured with high precision. In addition, by combining three-dimensional shape measurement based on the color highlight method and the phase shift method, the measurement accuracy can be improved.

[0021] Alternatively, the three-dimensional shape measurement device may further include an image display unit that displays a special image created based on the illumination light irradiation image obtained by the imaging unit and the plurality of pattern projection images. The special image is an image that discriminatively displays the presence or absence and degree of the inclination obtained from the illumination light irradiation image and the direction of the inclination obtained from the difference in feature amounts of the respective pixels constituting the plurality of pattern projection images in a distinguishable manner using different colors and / or patterns.

[0022] With such a structure, the user can recognize the three-dimensional shape of the measurement object measured by the three-dimensional measurement device by confirming the two-dimensional image. In addition, by comparing the image with the actual measurement object, the settings of the three-dimensional measurement device can also be adjusted.

[0023] In addition, the method for measuring a three-dimensional shape of the present invention includes the following steps: a projection step of projecting pattern light onto a measurement object from a plurality of directions having different angles with respect to the vertical axis of the measurement object; a photographing step of photographing the measurement object; and a measurement step of measuring the three-dimensional shape of the measurement object based on the images photographed in the photographing step. In the photographing step, a plurality of pattern projection images having different reflection forms are obtained. The plurality of pattern projection images are a plurality of images obtained by projecting pattern light from the plurality of directions. The reflection form is the reflection form including the intensity of the pattern light at a portion of the measurement object having an inclination. In the measurement step, the direction of the inclination is measured based on the difference between the specified feature amounts of the respective pixels constituting the plurality of pattern projection images among the images.

[0024] Alternatively, in the projection step, the pattern light may be projected onto the measurement object from at least a plurality of different positions opposed to each other with the measurement object as the center, thereby projecting the pattern light from the plurality of directions.

[0025] Alternatively, the three-dimensional shape measurement method may further include an illumination step of irradiating the measurement object with illumination light having different wavelengths from a plurality of different angles between the vertical direction and the horizontal direction. In the photographing step, an illumination light irradiation image of the measurement object irradiated with the illumination light is also obtained. In the measurement step, the degree of the inclination is measured based on the reflection form including any one of the intensity and wavelength of the illumination light in the illumination light irradiation image.

[0026] Alternatively, the three-dimensional shape measurement method may further include an image display step of displaying a special image produced based on the illumination light irradiation image and the plurality of pattern projection images obtained in the photographing step. The special image is an image in which the presence or absence and degree of the inclination obtained from the illumination light irradiation image and the direction of the inclination obtained based on the difference in the feature amounts of the respective pixels constituting the plurality of pattern projection images are distinguished and displayed in a recognizable manner by different colors and / or patterns.

[0027] In addition, the present invention can also be understood as a program for causing a three-dimensional shape measurement device to execute the above method, and a computer-readable recording medium non-temporarily recording such a program.

[0028] In addition, the above-described respective structures and processes can be combined with each other as long as there is no technical contradiction to constitute the present invention.

[0029] Effect of the Invention

[0030] According to the present invention, a technique capable of accurately determining the direction of inclination included in a measurement object in three-dimensional shape measurement can be provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Figure 1 is a schematic diagram showing the structure of a three-dimensional shape measurement device as an application example of the present invention.

[0032] Figure 2 Figure 2 is a flowchart showing the flow of three-dimensional shape measurement processing of the three-dimensional shape measurement device as an application example of the present invention.

[0033] Figure 3 Figure 3 is a schematic diagram showing the hardware structure of the substrate inspection device according to Embodiment 1.

[0034] Figure 4 Figure 4 is a block diagram illustrating the functions of the information processing device according to Embodiment 1.

[0035] Figure 5 Figure 5 is a plan view illustrating the structure of the lighting device according to Embodiment 1.

[0036] Figure 6 Figure 6 The (A) of [] is a side view illustrating the welded portion of the substrate as a measurement object. Figure 6 The (B) of [] is a diagram illustrating the color highlight image of the welded portion of the substrate. Figure 6 The (C) of [] is a diagram illustrating the color highlight corrected image of the welded portion of the substrate.

[0037] Figure 7 Figure 7 is a flowchart showing the flow of substrate inspection processing of the substrate inspection device according to Embodiment 1. DETAILED DESCRIPTION OF THE INVENTION

[0038] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0039] <Application Example>

[0040] (Structure of the Application Example)

[0041] The present invention can be applied, for example, to Figure 1 a three-dimensional shape measurement device as shown. Figure 1 is a schematic diagram showing the structure of the three-dimensional shape measurement device 9 of this application example. The three-dimensional shape measurement device 9 is a device for measuring the three-dimensional shape of the measurement object O, as Figure 1 ​​​​​​​​​​​​​​As shown, as the main structure, there are projectors 91a and 91b as projection units, a camera 92 as a photographing unit, and an information processing device 93 (such as a computer) as a measurement unit. The measurement object O includes a three-dimensional part OP with an inclination.

[0042] The projectors 91a and 91b are units that project patterns onto the measurement object. Here, the pattern refers to, for example, a stripe pattern in which the change in brightness shows periodicity and the phase can change over time. In this application example, the pattern projected from the projector 91a is set as pattern a, and the pattern projected from the projector 91b is set as pattern b. The projectors 91a and 91b are respectively arranged at a certain inclination angle with respect to the measurement object O.

[0043] The camera 92 is a unit that photographs the measurement object O in a state where the pattern has been projected and outputs a digital image. In addition, hereinafter, the image captured by the photographing unit will also be referred to as an observation image. The camera 92 is configured to have an optical system and an image sensor, for example. As Figure 1 shown, the camera 92 is arranged to photograph the measurement object O from directly above the measurement object O. In addition, the projector 91a and the projector 91b are arranged at positions opposite to each other in the circumferential direction with the camera 92 as the center.

[0044] The information processing device 93 has functions such as controlling the projectors 91a, 91b, the camera 92, and the conveying mechanism, processing the images taken in from the camera 92, and three-dimensional shape measurement, and corresponds to the measurement unit in the present invention. The information processing device 93 can be composed of a computer equipped with a CPU (Central Processing Unit), a RAM (Random Access Memory), a non-volatile storage device (such as a hard disk drive, a flash memory, etc.), an input device (such as a keyboard, a mouse, a touch panel, etc.), and a display device (such as a liquid crystal display, etc.).

[0045] When measuring the three-dimensional shape of the measurement object O in the three-dimensional shape measurement device 9 having the above structure, while changing the phase of the pattern projected from each projector onto the measurement object O, the camera 92 takes multiple images, and the information processing device 93 processes the captured images by, for example, the phase shift method, etc., thereby measuring the three-dimensional shape of the measurement object O.

[0046] (Function of the information processing device)

[0047] Next, the functions related to the three-dimensional shape measurement of the information processing device 93 will be described. The information processing device 93 includes an image acquisition unit 931, a height data calculation unit 932, a feature quantity extraction unit 933, a feature quantity comparison unit 934, a composite data production unit 935, and a three-dimensional shape measurement unit 936 as functional modules related to three-dimensional shape measurement.

[0048] The image acquisition unit 931 has the function of taking in a plurality of observation images used in three-dimensional shape measurement from the camera 92. For example, four images with a phase difference of 1 / 4π for the patterns projected onto the measurement object O are respectively obtained through the projection pattern of the projector 91a and the projection pattern of the projector 91b. In this application example, the projection pattern of the projector 91a is called pattern a, and the observation image obtained by photographing it is called observation image a. In addition, the projection pattern of the projector 91b is called pattern b, and the observation image obtained by photographing it is called observation image b.

[0049] The height data calculation unit 932 is a function for calculating the height data of the measurement object O based on the acquired plurality of observation images. For example, based on the two-dimensional phase difference of the pixels representing the position of a point on the surface of the measurement object O among the four acquired observation images, the height of this point is obtained in the observation image a and the observation image b respectively. In this application example, the height data calculated from the observation image a is set as height data a, and the height data calculated from the observation image b is set as height data b for explanation.

[0050] The feature quantity extraction unit 933 extracts the feature quantity (such as the luminance value) possessed by each pixel of the acquired observation image from each acquired observation image. In this application example, the feature quantity data extracted from the observation image a is set as feature quantity data a, and the feature quantity data extracted from the observation image b is set as feature quantity data b for explanation.

[0051] The feature quantity comparison unit 934 compares the feature quantity data a extracted by the feature quantity extraction unit 933 with the feature quantity data b. More specifically, in each observation image, the feature quantity values of the pixels representing the same part of the measurement object O are compared with each other, and it is determined which image of the observation image a and the observation image b the pixel with the larger value belongs to. Here, when the feature quantity is luminance, it is determined which side of the image where more light is incident on the camera 92, that is, the brighter side. Moreover, when there is an inclined part in the measurement object O, the pixels of the image on the side where the pattern light is irradiated from the direction opposite to the inclination become brighter, so the direction of the inclination can be determined from here. The feature quantity comparison unit 934 generates data related to the determination of the direction of the inclination (hereinafter referred to as inclination direction data) in this way.

[0052] The synthetic data creation unit 935 creates synthetic data for three-dimensional shape measurement based on the height data a, b, and the tilt direction data. Specifically, synthetic height data is generated by connecting the contours of the height data a and b, taking the average, etc. in a prescribed method, and this is corrected based on the tilt direction data to create contour data of the three-dimensional shape. Thereby, contour data for three-dimensional shape measurement that determines the orientation of the tilted part in the measurement object O can be obtained.

[0053] The three-dimensional shape measurement unit 936 measures the three-dimensional shape of the measurement object O based on the contour data created by the synthetic data creation unit 935.

[0054] (Flow of three-dimensional shape measurement processing)

[0055] Next, with reference to Figure 2 , the steps of three-dimensional shape measurement in this application example will be described. First, the information processing device 93 controls the projector 91a to project the pattern a onto the measurement object O from a first direction on the circumference centered on the measurement object O (step S901). Next, the information processing device 93 controls the camera 92 to photograph the measurement object O in a state where it is irradiated with the pattern a light from the first direction, and obtains the observation image a (step S902).

[0056] Next, the information processing device 93 controls the projector 91b to project the pattern b onto the measurement object O from a second direction opposite to the first direction across the measurement object O (step S903). Next, the information processing device 93 controls the camera 92 to photograph the measurement object O in a state where the pattern b is projected from the second direction, and obtains the observation image b (step S904).

[0057] Next, the information processing device 93 calculates the height data a and b respectively from the obtained observation images a and b (step S905), and extracts the feature amounts possessed by the pixels of each image (step S906). Subsequently, the information processing device 93 compares the extracted feature amounts, and uses the pixels with larger feature amounts to create tilt direction data (step S907). Then, based on the obtained height data a, b, and the tilt direction, synthetic data for three-dimensional shape measurement of the measurement object O is generated (step S908), and the three-dimensional shape of the measurement object O including the tilted part OP is measured based on this synthetic data (step S909), and a series of processing is ended. In addition, the measurement result can also be displayed on a display device (not shown). Also, the extraction of the feature amount does not necessarily need to be performed based on the observation images a and b used to calculate the height data a and b, and image data for extracting the feature amount can also be obtained separately.

[0058] With the structure of the three-dimensional shape measurement device 9 in the application example as described above, when there is an inclined part OP in the measurement object O, it is possible to measure the three-dimensional shape with high precision based on the three-dimensional shape contour data that determines the direction of the inclination.

[0059] <Embodiment 1>

[0060] Next, a substrate inspection device 1, which is another example of a mode for implementing the present invention, will be described. However, unless otherwise specified, the dimensions, materials, shapes, relative arrangements, etc. of the constituent components described in this embodiment do not mean that the scope of the present invention is limited thereto.

[0061] (Hardware Structure of Substrate Inspection Device)

[0062] Refer to Figure 3 , and the overall structure of the substrate inspection device according to the embodiment of the present invention will be described. Figure 3 is a schematic diagram showing the hardware structure of the substrate inspection device. This substrate inspection device 1 is used for inspecting the appearance of substrates on a surface mounting production line (such as inspecting the solder joint state after reflow, etc.).

[0063] The substrate inspection device 1 mainly includes a workbench 10, a measurement unit 11, a control device 12, an information processing device 13, and a display device 14. The measurement unit 11 includes a camera 110, a lighting device 111, and a pattern projection device (projector) 112.

[0064] The workbench 10 is a mechanism for holding the substrate K and aligning the component KB and the solder KH, which are the inspection objects, with the measurement position of the camera 110. As Figure 3 shown, when the X-axis and the Y-axis are taken parallel to the workbench 10 and the Z-axis is taken perpendicular to the workbench 10, the workbench 10 can at least perform translation in two axes, the X-direction and the Y-direction. The camera 110 is arranged with its optical axis parallel to the Z-axis and photographs the substrate K on the workbench 10 from directly above. The image data captured by the camera 110 is input into the information processing device 13.

[0065] The lighting device 111 (111R, 111G, 111B) is a lighting unit that irradiates the substrate K with illumination light of different colors (wavelengths). Figure 3FIG. 0 is a diagram schematically showing the XZ cross section of the illumination device 111. In fact, the illumination device 111 has an annular or dome shape so as to be able to illuminate light of the same color from all directions (all directions around the Z axis). The projectors 112a and 112b are pattern projection units that project pattern light with a specified pattern onto the substrate K. The projector 112 projects the pattern light through an opening provided in the middle of the illumination device 111. In the present embodiment, two projectors 112 are arranged at diagonal positions across the substrate K, but more projectors may also be provided. Both the illumination device 111 and the projector 112 are illumination systems used when photographing the substrate K with the camera 110. The illumination device 111 is used to measure the surface shape of specular objects such as solder, and the projector 112 is used to measure the surface shape of diffuse reflection objects such as components.

[0066] The control device 12 is a control unit that controls the operation of the substrate inspection device 1, and is responsible for the movement control of the workbench 10, the lighting and dimming control of the illumination device 111, the lighting control or pattern change of the projectors 112a and 112b, the photographing control of the camera 110, etc.

[0067] The information processing device 13 is a device having the following functions: using the image data taken in from the camera 110, obtaining various measurement values related to the component KB and the solder KH, or inspecting the solder bonding state of the component KB. The display device 14 is a device that displays the measurement values and inspection results obtained by the information processing device 13. The information processing device 13 can be constituted by a general computer having a CPU, a RAM, a non-volatile storage device, and an input device, for example. In addition, Figure 3 in FIG., the control device 12, the information processing device 13, and the display device 14 are shown in different frames, but they can be constituted by separate devices or a single device.

[0068] (Functional Structure)

[0069] Figure 4 FIG. 15 is a block diagram showing the structure of functional modules related to the inspection process provided by the information processing device 13. These functional modules are realized by the CPU of the information processing device 13 reading and executing a program stored in the auxiliary storage device. However, all or part of the functions may also be constituted by circuits such as ASIC and FPGA.

[0070] The image acquisition unit 131 is a functional module that takes in image data from the camera 110. The solder shape measurement unit 132 is a functional module that restores the three-dimensional shape of the specular object part such as solder based on the two-dimensional image data, and the component shape measurement unit 133 is a functional module that restores the three-dimensional shape of the diffuse reflection object part such as components based on the two-dimensional image data. The image data and restoration algorithms used in each restoration process will be described later.

[0071] The inspection unit 134 is a functional module that measures various indices related to the shapes of the solder KH and the component KB based on the three-dimensional shape data obtained by the solder shape measurement unit 132 and the component shape measurement unit 133, and uses these measured values to inspect the state of the solder joint. The inspection program storage unit 135 is a functional module that stores an inspection program defining items, conditions, etc. in the inspection by the inspection unit 134. In the inspection program, for example, the position and size of the pads to be inspected, the size of the component, the types of measured indices, the determination reference values (thresholds and value ranges for determining qualified and unqualified products) for each index, etc. are defined. The output processing unit 136 is a functional module that externally outputs the measured values, inspection results, three-dimensional shapes of the component KB and the solder KH, etc. obtained by the inspection unit 134 to the display device 14, etc.

[0072] Hereinafter, after separately explaining the method for restoring the three-dimensional shape of the solder KH (specular object) and the method for restoring the three-dimensional shape of the component KB (diffuse reflection object), the inspection processing flow of the information processing device 13 will be described.

[0073] (Measurement of the three-dimensional shape of the solder)

[0074] When measuring the three-dimensional shape of the solder KH, an image obtained by the so-called color specular method is used. The color specular method is a method in which light of multiple colors (i.e., wavelengths) is irradiated onto the substrate at different incident angles, and the solder surface is photographed in a state where the color characteristics corresponding to its normal direction (i.e., the color of the light source in the specular reflection direction as observed from the camera) appear, thereby capturing the three-dimensional shape of the solder surface as two-dimensional hue information. It is possible to extract only the regions presenting the light source colors of R, G, and B from the image, and based on the shapes, widths, and orders of the R, G, and B regions, the three-dimensional shape of the solder is restored. In addition, since the restoration of the three-dimensional shape can use a known method, a detailed description thereof is omitted here.

[0075] First, referring to Figure 5 , the structure of the lighting device 111 used in the color specular method will be described. Figure 5 is a schematic plan view schematically showing the arrangement relationship of the light sources 111R, 111G, and 111B of the lighting device 111. The lighting device 111 has a structure in which three circular ring-shaped light sources, namely, a red light source 111R, a green light source 111G, and a blue light source 111B, are arranged in concentric circles with the optical axis of the camera 110 as the center. The light sources 111R, 111G, and 111B are adjusted in elevation angle and orientation such that the incident angles with respect to the substrate K increase in the order of red light, green light, and blue light. Such a lighting device 111 can be formed, for example, by arranging LEDs of R, G, and B colors in a circular ring shape outside a dome-shaped diffuser plate.

[0076] When the substrate K is photographed by the camera 110 while the lighting device 111 is lit, color characteristics corresponding to the normal direction (tilt angle) appear in the solder KH, which is a mirror object. For example, when the tilt of the solder KH becomes gentler as it moves away from the component electrode, a change in hue such as B→G→R appears in the area of the solder KH. The shape, width, appearance order, etc. of the regions of each of the R, G, and B colors vary depending on the surface shape of the solder KH.

[0077] Refer to Figure 6 to describe the image of the solder part that can be obtained when the substrate is photographed by the camera 110 while the lighting device 111 of this embodiment is lit. Figure 6 (A) of is a view of the electrode part of the component KB on the substrate K and the solder part (hereinafter referred to as the soldering part) joined thereto as viewed from the side. As Figure 6 shown in (A) of, the solder KH in this case is not sufficiently fused to the electrode extending from the component KB, and the contact area is small (i.e., in a non-wetting state).

[0078] Figure 6 (B) of is an image of the soldering part (hereinafter referred to as the color highlight image) when the lighting device 111 is lit and photographed by the camera 110. In Figure 6 (B) of, the regions of each of R, G, and B are shown. From the image shown in Figure 6 (B), it can be speculated that there is an inclination (a defined inclined plane) at the joint between the solder and the electrode, but it is impossible to determine in which direction this inclined plane faces (i.e., whether it is a wetting bulge or a non-wetting state). In addition, on the surface of the component KB main body and the electrode, diffuse reflection is dominant, so instead of the light source colors such as R, G, and B, the color of the object itself when illuminated with white light appears.

[0079] (Three-dimensional shape measurement of components)

[0080] On the other hand, in the measurement of the three-dimensional shape of the component KB, which is a diffuse reflection object, the phase shift method is used. The phase shift method is one of the methods for restoring the three-dimensional shape of an object surface by analyzing the deformation of a pattern when the pattern light is projected onto the object surface. Specifically, using projectors 112a and 112b, while projecting a prescribed pattern (for example, a striped pattern with brightness varying in a sine wave shape) onto the substrate, the camera 110 is used for shooting. Then, the deformation of the pattern corresponding to the unevenness appears on the surface of the substrate K. By repeating this process multiple times while changing the phase of the brightness variation of the pattern light, a plurality of images with different brightness characteristics (hereinafter referred to as pattern analysis images) can be obtained. Since the brightness of the same pixel in each image should vary with the same period as the variation of the striped pattern, by applying a sine wave to the variation of the brightness of each pixel, the phase of each pixel can be known. Then, by obtaining the phase difference of the phase with respect to a prescribed reference position (such as the surface of the workbench, the surface of the substrate, etc.), the distance (i.e., height) from the reference position can be calculated. In addition, the pattern projected by the projector 112a is called pattern a, the pattern projected by the projector 112b is called pattern b, and the images obtained by shooting each pattern are called observation image a and observation image b.

[0081] Here, in the present embodiment, the information processing device 13 acquires the observation image a and the observation image b, and the projectors 112a and 112b are arranged at diagonal positions with the substrate K therebetween. Therefore, the observation image a and the observation image b are images obtained by projecting the patterns from exactly opposite directions. Thus, similar to the case of the application example, by extracting the feature quantity data from the observation image a and the observation image b and comparing them, the tilt direction determination data for determining the tilt direction of the part with a tilt such as the welded part can be obtained. In addition, since the tilt direction determination data can be generated by the same method as the application example, detailed description thereof is omitted.

[0082] Moreover, by correcting the contour data obtained from the above-described color highlight image using the tilt direction determination data, the direction of the inclined surface of the joint portion between the solder KH and the electrode can be discriminated.

[0083] Figure 6 (C) in shows an example of a special image (hereinafter referred to as a color highlight correction image) created based on the synthetic data created in this way. In the color highlight correction image, the tilt in the basic direction (the direction in which more faces face) is displayed in the same red (R), green (G), and blue (B) as the original color highlight image, and the tilt in the direction opposite to the basic direction is represented by magenta (M), yellow (Y), and cyan (C). In addition, the selection of colors here is completely arbitrary and is not limited to these six colors. However, by using similar colors although they are in different color systems, it is possible to intuitively grasp the cases where the degrees of tilt are the same and the cases where the tilt directions are different.

[0084] If such a color highlight-corrected image can be displayed, for example, on the display device 14, the user can observe the color highlight-corrected image and easily grasp the state of the soldered portion.

[0085] (Flow of inspection process)

[0086] Next, Figure 7 is used to explain the flow of the inspection process performed in the substrate inspection apparatus 1. Figure 7 is a flowchart showing the flow of the inspection process.

[0087] First, the control device 12 controls the workbench 10 according to the inspection program to move the component KB and the solder KH to be inspected to the measurement position (the field of view of the camera 110) (step S101). Then, the control device 12 turns on the illumination device 111 (step S102), and performs the imaging process of the camera 110 in a state where red light, green light, and blue light are irradiated (step S103). The obtained image data (color highlight image) is taken into the information processing device 13 by the image acquisition unit 131.

[0088] Next, the control device 12 projects pattern light from the projector 112a (step S104), and performs imaging using the camera 110 (step S105). Further, pattern light is projected from the projector 112b (step S106), and imaging is performed by the camera 110 (step S107). In the case of using the phase shift method, the processes of steps S104 to S107 are performed multiple times while changing the phase of the pattern light. The data of the obtained multiple images is taken into the information processing device 13 by the image acquisition unit 131. In addition, in the present embodiment, the imaging based on the illumination device 111 is performed first, but the imaging based on the projector 112 may be performed first. In addition, when there are other inspection objects outside the field of view of the camera 110, the processes of steps S101 to S107 may be repeated.

[0089] Thereafter, it becomes the process performed by the information processing device 13. The component shape measurement unit 133 restores the three-dimensional shape of the component KB by the phase shift method based on the observed images a and b obtained in steps S105 and S107 (step S108). The data of the restored three-dimensional shape is saved, for example, in the form of image data (referred to as a height map) that represents the height (Z position) of each pixel by a pixel value.

[0090] The solder shape measurement unit 132 restores the three-dimensional shape of the solder KH (and the electrodes of the component KB) based on the color highlight image obtained in step S103 (step S109). Further, the solder shape measurement unit 132 extracts luminance values from the observed images a and b obtained from the projected pattern light, and generates tilt direction determination data (step S110). The data of the restored three-dimensional shape is saved, for example, in the form of a height map that represents the height (Z position) of each pixel in the solder KH region with pixel values.

[0091] Then, by synthesizing these height maps and the tilt direction determination data, a global height map representing the height information of both the solder KH as a specular object and the component KB as a diffuse reflection object can be obtained. Here, correction based on the determined direction of the inclined plane is performed on the global height map.

[0092] Then, the inspection unit 134 inspects the substrate K based on the global height map and the threshold value of the inspection program (step S111). When the inspection is completed, the display device 14 displays a color highlight composite image that visually represents the inspection result and the composite data created in step S108 (step S112), and a series of processes is ended.

[0093] According to the substrate inspection device of the present embodiment described above, since the three-dimensional shapes of the solder as a specular object and the component electrodes as diffuse reflection objects are restored by methods suitable for them respectively, high-precision three-dimensional shape data can be obtained for both the solder and the component electrodes. In addition, when restoring the three-dimensional shape of the solder, by performing correction based on the tilt direction determination data obtained by projecting patterns from multiple directions, the orientation of the inclined plane at the location with an inclination is determined and the three-dimensional shape is restored, so that the shape of the inclined plane of the solder can be restored with high precision.

[0094] <Other>

[0095] The above-described embodiment merely illustratively explains the present invention, and the present invention is not limited to the above specific embodiments. The present invention can be variously modified within the scope of its technical idea. For example, in the above-described Embodiment 1, the three-dimensional shape of the specular object is measured based on the color highlight image, and the three-dimensional shape of the diffuse reflection object is measured based on the pattern projection image, but it is not necessarily so. It is also possible to synthesize the data after creating two contour data for measuring the shape of the entire measurement object based on each image. In addition, in Embodiment 1, the projector is fixed, but it can also be configured to be able to rotate the projector around the vertical axis. In addition, in such a case, the number of projectors can be set to one.

[0096] In addition, in the above-described embodiment, the color high-light correction image can identify the degree and direction of inclination by color differentiation, but is not limited to color. For example, it can also be an image that can be identified and displayed according to differences in patterns such as hatching, or an image that combines differences in color and differences in patterns.

[0097] <Supplementary Note>

[0098] One aspect of the present invention is a three-dimensional shape measurement device, characterized by comprising: a projection unit (91a, 91b) that projects pattern light onto a measurement object (O) from a plurality of directions having different angles around a vertical axis with respect to the measurement object; a photographing unit (92) that photographs the measurement object; and a measurement unit (93) that measures the three-dimensional shape of the measurement object based on an image photographed by the photographing unit.

[0099] The photographing unit acquires a plurality of pattern projection images having different reflection forms, the plurality of pattern projection images being a plurality of images obtained by projecting pattern light from the plurality of directions, and the reflection form being a reflection form including the intensity of the pattern light in a portion of the measurement object having an inclination.

[0100] The measurement unit measures the direction of the inclination based on the difference in a specified feature amount among respective pixels constituting the plurality of pattern projection images among the respective images.

[0101] In addition, another aspect of the present invention is a method for measuring a three-dimensional shape, characterized by comprising: a projection step (S901, S903) of projecting pattern light onto a measurement object from a plurality of directions having different angles around a vertical axis with respect to the measurement object; a photographing step (S902, S904) of photographing the measurement object; and a measurement step (S909) of measuring the three-dimensional shape of the measurement object based on an image photographed by the photographing unit. In the photographing step, a plurality of pattern projection images having different reflection forms are acquired, the plurality of pattern projection images being a plurality of images obtained by projecting pattern light from the plurality of directions, and the reflection form being a reflection form including the intensity of the pattern light in a portion of the measurement object having an inclination. In the measurement step, the direction of the inclination is measured based on the difference in a specified feature amount among respective pixels constituting the plurality of pattern projection images among the respective images.

[0102] Reference Numeral Explanation

[0103] 1: Substrate inspection device; 9: Three-dimensional shape measurement device; 10: Workbench; 11: Inspection unit; 110, 92: Cameras; 111, 91: Lighting devices; 12: Control device; 13, 93: Information processing devices; 14: Display device; K: Substrate; O: Measurement object.

Claims

1. A three-dimensional shape measurement device, characterized in that, The three-dimensional shape measurement device has: a projection unit that projects pattern light onto a measurement object from a plurality of directions with different angles around a vertical axis with respect to the measurement object; a photographing unit that photographs the measurement object; and a measurement unit that measures the three-dimensional shape of the measurement object based on an image photographed by the photographing unit, wherein the photographing unit obtains a plurality of pattern projection images with different reflection forms, the plurality of pattern projection images being a plurality of images obtained by projecting pattern light from the plurality of directions, and the reflection form being a reflection form including the intensity of the pattern light at an inclined mirror surface portion in the measurement object; the measurement unit sets, based on the difference in luminance of each pixel constituting the plurality of pattern projection images between the images, the direction in which the inclined slope faces to be opposite to the direction in which the pattern light projected in the pattern projection image with the maximum luminance of the pixels representing the same part of the measurement object between the pattern projection images is irradiated, and measures the three-dimensional shape.

2. The three-dimensional shape measurement device according to claim 1, wherein the projection unit projects the pattern light onto the measurement object from at least a plurality of different positions opposed to each other with the measurement object as the center on a circumference centered on the measurement object, thereby projecting the pattern light from the plurality of directions.

3. The three-dimensional shape measurement device according to claim 1 or 2, wherein the projection unit is configured to be rotatable in the circumferential direction centered on the measurement object, and projects pattern light onto the measurement object from a plurality of different positions on a circumference centered on the measurement object, thereby projecting the pattern light from the plurality of directions.

4. The three-dimensional shape measurement device according to claim 1 or 2, wherein a plurality of the projection units are arranged at a plurality of different positions on a circumference centered on the measurement object, and the plurality of projection units respectively project pattern light onto the measurement object, thereby projecting the pattern light from the plurality of directions.

5. The three-dimensional shape measurement device according to claim 1 or 2, wherein the three-dimensional shape measurement device further has an illumination unit that irradiates the measurement object with illumination light of different wavelengths from a plurality of different angles between the vertical direction and the horizontal direction, the photographing unit further obtains an illumination light irradiation image of the measurement object irradiated with the illumination light, and the measurement unit further measures the degree of inclination based on the reflection form including any one of the intensity and wavelength of the illumination light in the illumination light irradiation image.

6. The three-dimensional shape measurement device according to claim 5, wherein the three-dimensional shape measurement device further has an image display unit that displays a special image produced based on the illumination light irradiation image and the plurality of pattern projection images obtained by the photographing unit. The special image is an image in which the presence or absence and degree of the inclination obtained from the image irradiated with the illumination light and the direction of the inclination obtained from the difference in the feature amounts of the respective pixels constituting the plurality of pattern projection images are distinguished and displayed in a distinguishable manner by different colors and / or patterns.

7. A three-dimensional shape measurement method, characterized in that, The three-dimensional shape measurement method includes the following steps: A projection step of projecting pattern light onto a measurement object from a plurality of directions having different angles around a vertical axis with respect to the measurement object; A photographing step of photographing the measurement object; and A measurement step of measuring the three-dimensional shape of the measurement object based on the image photographed in the photographing step. In the photographing step, a plurality of pattern projection images having different reflection forms are obtained. The plurality of pattern projection images are a plurality of images obtained by projecting pattern light from the plurality of directions. The reflection form is the reflection form including the intensity of the pattern light at a specular part having an inclination in the measurement object. In the measurement step, based on the difference in the brightness of the respective pixels constituting the plurality of pattern projection images between the images, the inclined slope direction is set to be opposite to the direction in which the pattern light projected in the pattern projection image having the maximum brightness of the pixels representing the same part of the measurement object among the pattern projection images is irradiated, and the three-dimensional shape is measured.

8. The three-dimensional shape measurement method according to claim 7, wherein In the projection step, the pattern light is projected onto the measurement object from at least a plurality of different positions opposed to each other with the measurement object as the center, whereby the pattern light is projected from the plurality of directions.

9. The three-dimensional shape measurement method according to claim 7 or 8, wherein The three-dimensional shape measurement method further includes an illumination step of irradiating the measurement object with illumination light of different wavelengths from a plurality of different angles between the vertical direction and the horizontal direction. In the photographing step, an illumination light irradiation image of the measurement object irradiated with the illumination light is also obtained. In the measurement step, the degree of the inclination is also measured according to the reflection form including any one of the intensity and wavelength of the illumination light in the illumination light irradiation image.

10. The three-dimensional shape measurement method according to claim 9, wherein The three-dimensional shape measurement method further includes an image display step of displaying a special image produced based on the illumination light irradiation image and the plurality of pattern projection images obtained in the photographing step. The special image is an image in which the presence or absence and degree of the inclination obtained from the illumination light irradiation image and the direction of the inclination obtained from the difference in the feature amounts of the respective pixels constituting the plurality of pattern projection images are distinguished and displayed in a distinguishable manner by different colors and / or patterns.

11. A storage medium that non-temporarily stores a program for causing a three-dimensional shape measurement device to execute each step of the three-dimensional shape measurement method according to any one of claims 7 to 10.

Images