Shape inspection apparatus and shape inspection method for inspection-target object
The shape inspection device addresses the limitations of conventional methods by using a movable camera and adjustable illumination to capture multi-angle images of bonding wires, enabling efficient and cost-effective detection of three-dimensional shapes and defects, including wire contacts.
Patent Information
- Application Number
- PCT/JP2023/018731
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-19
- Publication Date
- 2024-11-28
AI Technical Summary
Conventional shape inspection devices for bonding wires cannot accurately determine the three-dimensional shape, particularly the depth information, and fail to detect contacts between wires or between wires and structures, due to their high cost and time-consuming nature.
A shape inspection device equipped with a movable camera, adjustable illumination, and a controller that captures images from multiple angles, using a commercially available camera and illumination device to measure the three-dimensional shape of bonding wires, enabling detection of wire bends and intersections at high speed and low cost.
The device effectively inspects the three-dimensional shape of bonding wires, including contacts, with high speed and low cost, providing accurate defect detection before product shipping.
Smart Images

Figure JP2023018731_28112024_PF_FP_ABST
Abstract
Description
Apparatus and method for inspecting shape of object to be inspected
[0001] The present application relates to a shape inspection device and a shape inspection method for an object to be inspected.
[0002] In conventional inspection equipment for the shape of bonding wires, which are the objects to be inspected, the wire shape is inspected by capturing an image of the wire from above with a 2D camera and comparing it with data on non-defective products. However, this method does not provide information on the shape of the wire in the depth direction, and it is not possible to determine loop shapes, contact between crossing wires, or contact between the wire and a structure.
[0003] On the other hand, there are devices and sensors that can measure three-dimensional shapes. Methods for measuring three-dimensional shapes include stereo cameras and those that use laser light, but both of these methods have problems such as expensive equipment and time-consuming measurements.
[0004] In the following Patent Document 1, the quality of the three-dimensional shape of the bonding wire is determined by irradiating the bonding wire with laser slit light of different wavelengths and comparing the binary pattern of each reflected light with non-defective product data.
[0005] JP-A-7-037955 JP-A-4-250347
[0006] Because bonding wires have a three-dimensional structure, capturing an image from just one direction with a 2D (planar) camera does not provide information about the depth direction, which is perpendicular to the plane. Capturing images of the wire from multiple directions with the 2D camera makes it possible to measure the three-dimensional shape. However, in most cases, a large number of bonding wires are placed on a single product, and capturing images of all the wires from multiple directions takes an enormous amount of time. Furthermore, there are cases where the target wire cannot be captured because it is blocked by other wires or structures.
[0007] In Patent Document 1, a slit laser beam is projected onto the bonding wire, and the binary pattern of each reflected beam is compared with non-defective data to determine the quality of the three-dimensional shape of the bonding wire. However, the absolute value of the wire position is not measured, and therefore it is not possible to determine whether crossing wires are in contact with each other or with a structure.
[0008] The present application discloses a technology for solving the above-mentioned problems, and aims to provide a wire shape inspection device that can measure the three-dimensional shape of the bonding wire being inspected using a camera that can recognize two-dimensional shapes, and can inspect a large number of wires used in a product for defects such as bent wires, contact between crossing wires, or contact between wires and structures, quickly and inexpensively before the product is shipped.
[0009] a controller that controls the operation of the adjustment mechanism and also controls the operations of changing the position of the light source of the lighting device, turning on the light source, changing the spectral distribution of the light source, and imaging the object under test by the camera; and a computer that outputs operation commands to change the positions of the camera and the light source, turn on the light source, change the spectral distribution of the light source, and image the object under test by the camera, respectively. The bonding wire shape inspection device disclosed in the present application comprises: a camera that is movable relative to the object under test and images the object under test; an adjustment mechanism that adjusts the distance between the camera and the object under test; a lighting device that has a light source that irradiates light onto the object under test and the position of the light source is changeable; a controller that controls the operation of the adjustment mechanism and also controls the operations of changing the position of the light source of the lighting device, turning on the light source, changing the spectral distribution of the light source, and image the object under test by the camera; and a computer that outputs operation commands to change the positions of the camera and the light source, turn on the light source, change the spectral distribution of the light source, and image the object under test by the camera, respectively. The device is characterized in that the position of the camera is moved and the irradiation position of the light from the light source relative to the object under test is changed to image the object under test, and the computer estimates the shape of the object under test based on the image and compares it with the shape of a reference object under test that has been measured in advance, thereby determining the three-dimensional shape of the object under test.
[0010] According to the bonding wire shape inspection device disclosed in the present application, the three-dimensional shape of the bonding wire being inspected is measured using a camera that can recognize two-dimensional shapes, and a wire shape inspection device can be provided that can inspect defects in a large number of wires used in products, such as bent wires, contact between crossing wires, or contact between wires and structures, quickly and inexpensively before the product is shipped.
[0011] 1 is a diagram showing an example of the configuration of an apparatus for inspecting the shape of an object under inspection according to a first embodiment. FIG. 2 is a diagram showing the state of reflected light reflected from an object under inspection. FIG. 3 is a diagram for explaining the optical path of specularly reflected light when illumination light is irradiated onto an object under inspection with a normal shape. FIG. 4 is a diagram for explaining the optical path of specularly reflected light when illumination light is irradiated onto an object under inspection with a defective shape. FIG. 5 is a flowchart for explaining an overview of determining whether the shape of an object under inspection is good or bad. FIG. 6 is a diagram for explaining, with a specific example, determining whether the shape of an object under inspection is good or bad. FIG. 7 is a flowchart for explaining a method for measuring the planar shape of an object under inspection. FIG. 8 is a diagram for explaining the details of the method for measuring the planar shape of an object under inspection. FIG. 9 is a flowchart for explaining a method for measuring the angular shape of an object under inspection. FIG. 10 is a diagram for explaining the angular shape of an object under inspection measured by angular shape measurement of the object under inspection. FIG. 11 is a diagram for explaining the planar shape and angular shape by measurement points in shape measurement of an object under inspection. FIG. 12 is a flowchart for explaining a method for measuring the height of a vertex of an object under inspection. FIG. 13 is a diagram for explaining the measured height of a vertex of an object under inspection. FIG. 14 is a diagram for explaining a method for inspecting the shape of an object under inspection according to a second embodiment. FIG. 15 is a diagram showing an example of the configuration of an apparatus for inspecting the shape of an object under inspection according to a third embodiment. Fig. 1 is a diagram showing the arrangement of an object to be inspected and a shape inspection device when an object to be inspected having a height difference is inspected by the shape inspection device for object to be inspected according to embodiment 1. Fig. 2 is a diagram showing the arrangement of an object to be inspected and a shape inspection device when an object to be inspected having a height difference is inspected by the shape inspection device for object to be inspected according to embodiment 4. Fig. 3 is a diagram for explaining hardware used in a controller and a computer used in the shape inspection devices for object to be inspected according to embodiments 1 to 4.
[0012] 1 is a diagram showing the configuration of an apparatus 1 for inspecting the shape of an object to be inspected in accordance with embodiment 1. A typical example of an object to be inspected is a bonding wire. The shape inspection device 1 for an object to be inspected, which is an apparatus capable of three-dimensionally inspecting the shape of the object to be inspected, comprises a camera 2, a lens 3 attached to the camera 2, a height adjustment mechanism 4 (hereinafter simply referred to as the adjustment mechanism) for adjusting the distance between the camera 2 and the bonding wire 10, an illumination device 5 (dome-shaped in Figure 1) that can control the irradiation angle of the illumination light, a stage 6 on which the object to be inspected is placed and which moves in a direction perpendicular to the optical axis of the lens (left and right directions indicated by the dotted arrow in Figure 1), a controller 20 that controls the operation of the adjustment mechanism 4 and also controls the position change operation of the illumination device 5 (including the change operation of the angular position described in detail below), the operation of changing the intensity of the light source of the illumination device 5 including the lighting operation, the operation of changing the spectral distribution, and the operation of having the camera 2 capture an image of the object to be inspected, and a computer 21 that outputs command signals to control the controller 20 and calculates and grasps the shape of the bonding wire 10 of the object to be inspected from the image acquired by the camera from the camera 2 via the controller 20, and determines whether the product is good or bad.
[0013] Here, the bonding wire shape inspection device 1 in this embodiment 1 is characterized in that when performing three-dimensional measurement of a wire, it is possible to measure the shape of the wire in three dimensions at high speed using a camera that can recognize two-dimensional images at a lower cost, without using an expensive camera such as a stereo camera.
[0014] Here, a commercially available industrial camera is used as the camera 2, and one selected according to various conditions such as the diameter of the bonding wire 10, the required pixel resolution, and the frame rate is used.
[0015] The lens 3 is selected according to various conditions, such as the diameter of the bonding wire 10, the required pixel resolution, and the field of view. For example, if a normal lens is used when multiple bonding wires 10 of the device under test exist in the same field of view, measurement errors will occur, especially at the edges of the field of view, due to the influence of the angle of view. Therefore, when multiple bonding wires 10 of the device under test exist in the same field of view, a telecentric lens is used.
[0016] The height adjustment mechanism 4 is used to adjust the distance between the camera 2 and the bonding wire 10 and capture a focused image. The height adjustment mechanism 4 may be provided on the stage 6 side instead of the camera side. The height adjustment mechanism 4 is necessary when the height of the wire does not fit within the depth of field of the camera 2 and lens 3.
[0017] The lighting device 5 can emit illumination light from any angle. For example, a dome-shaped device equipped with multiple light-emitting elements, each of which can be individually controlled to be turned on and off, can be used. However, the lighting device 5 is not limited to this, and can also be equipped with a movable mechanism to control the position and angle of the lighting device.
[0018] In the configuration without the moving mechanism, the three-dimensional shape of the bonding wire can be measured by taking multiple images of the bonding wire on the same plane while changing the irradiation angle of the lighting device, which enables high-speed inspection.
[0019] Next, a method for determining whether the shape of a bonding wire is good or bad using the bonding wire shape inspection device 1 shown in Fig. 1 will be described below. First, the functions of the lighting device and camera, which are the basic components of this inspection device, will be described using Fig. 2.
[0020] FIG. 2 is a diagram showing the state of light reflected from the bonding wire 10 when the bonding wire 10 is irradiated with illumination light from the light source 7, which is an example of the illumination device 5. As shown in FIG.
[0021] 2, there are two types of reflected light reflected from the bonding wire 10: specular reflected light, which is light reflected from area A that appears bright to camera 2, and diffuse reflected light, which is light reflected from areas other than area A that are difficult to see from camera 2. In other words, when an object is illuminated with illumination light from a light source, it generally produces specular reflected light and diffuse reflected light. In this case, if the object is made of metal, the specular reflected light is strong and the diffuse reflected light is weak. Because the bonding wire is made of metal, when illumination light is illuminated on the bonding wire, the specular reflected light is stronger than the diffuse reflected light.
[0022] In the first embodiment, when the bonding wire 10 is irradiated with illumination light, the entire bonding wire glows uniformly due to diffuse reflected light (see the dashed arrow in FIG. 2). On the other hand, from a region R of the bonding wire that forms a specific angle with the illumination light (see the portion indicated by the dotted line in FIG. 2), specular reflected light is returned to the camera 2 (see the thick solid arrow in FIG. 2), and the bonding wire appears brighter than the regions other than the region R of the bonding wire.
[0023] Next, an outline of a method for determining whether the three-dimensional shape of a bonding wire is good or bad using the above-mentioned lighting device (light source) and camera configuration, taking a typical shape of a bonding wire as an example, will be explained using Figures 3A and 3B.
[0024] 3A is a conceptual diagram showing the optical path of specularly reflected light when illumination light is irradiated onto a bonding wire 10n having a normal shape, and FIG. 3B is a conceptual diagram showing the optical path of specularly reflected light when illumination light is irradiated onto a bonding wire 10a having a defective shape. In these figures, the angular position of the illumination device is the same in both figures, i.e., angular position L.
[0025] 3A shows that when illumination light is irradiated onto a bonding wire 10n having a normal shape, specularly reflected light from a brightly-appearing region R reaches the direction of the camera 2. On the other hand, FIG. 3B shows that when illumination light is irradiated onto a bonding wire 10a having a defective shape, specularly reflected light does not reach the direction of the camera 2. In other words, it can be seen that the bonding wire shape inspection device 1 shown in FIG. 1 can determine whether the bonding wire shape is good or bad.
[0026] Therefore, the following describes in more detail a specific method for determining whether the three-dimensional shape of a bonding wire is acceptable. In particular, a measurement method that applies image measurement or the like is described using a flowchart. Although a detailed description is omitted, the measurement method described in detail below can also be applied to detecting bending of a bonding wire, contact between intersecting bonding wires, or contact between a wire and a structure.
[0027] FIG. 4 is a flowchart outlining the procedure of a measurement method for three-dimensionally determining whether the bonding wire shape is good or bad.
[0028] First, in step S10, the planar shape of the bonding wire is measured. This measurement involves projecting the shape of the bonding wire onto (x, y) coordinates on plane P, where P is a plane perpendicular to the optical axis of camera 2, and then projecting multiple points on the bonding wire onto plane P and measuring multiple (x, y) coordinates corresponding to each of the multiple points. A more detailed measurement method will be described later.
[0029] Next, in step S11, the angular shape of the bonding wire is measured. Here, this angle θ (hereinafter also referred to as the angular shape θ) is defined as the angle at which the bonding wire, which is the object to be measured, is placed with respect to the plane P. A more detailed explanation of this angular shape measurement will be given later.
[0030] Next, in step S12, the apex height of the bonding wire is measured. Here, the apex height zt is the apex of the bonding wire 10, i.e., the point where the height in the z direction is maximum. This apex height measurement will be explained in more detail later.
[0031] In the above-mentioned measurement of the apex height of the bonding wire, the apex of the bonding wire 10, i.e., the point where the height in the z direction is maximum, is measured based on the focus information of the bonding wire (here, for example, information in which the most focused image is determined from the difference with adjacent pixels and this is set as the point of maximum height in the z direction). This allows for more accurate measurement of the three-dimensional shape of the bonding wire 10.
[0032] The three measurements described above, i.e., planar shape measurement, angular shape measurement, and apex height measurement, are performed at multiple points on the bonding wire 10, and the three-dimensional shape of the bonding wire is estimated at each point based on the three types of data measured above in step S13. The quality of the three-dimensional shape of the bonding wire is determined by comparing this estimated shape with values at multiple points of master data (reference three-dimensional shape data) of the bonding wire obtained by measuring it in advance (step S14).
[0033] Next, a specific example of the method for comparing with the master data will be described with reference to Fig. 5. In Fig. 5, the dotted line shows the two-dimensional shape of the reference bonding wire 10s, while the solid line shows the two-dimensional shape of the measured bonding wire 10m.
[0034] For the five points on the bonding wire 10m shown by the solid line, the data obtained by measurement are, from left to right, p1 = p((x1, y1), z1, θ1), p2 = p((x2, y2), z2, θ2), p3 = p((x3, y3), z3, θ3), p4 = p((x4, y4), z4, θ4), p5 = p((x5, y5), z5, θ5). Here, (xm, ym) (where m is an integer from 1 to 5) is the data obtained by planar shape measurement, θm (where m is an integer from 1 to 5) is the data obtained by angular shape measurement, and zm (where m is an integer from 1 to 5) is the data obtained by vertex height measurement.
[0035] The differences between the data of the five points on the bonding wire 10m and five points on the reference bonding wire 10s that correspond to the five points on the measurement positions (these points will be denoted as p01, p02, ..., p05) are calculated (these differences will be Δp1, Δp2, Δp3, Δp4, and Δp5, respectively, as shown in Figure 5) (for example, Δp1 = |p1 - p01|), (Δpm) 2 (where m is an integer from 1 to 5), each value from m=1 to m=5 is calculated and added, and the square root of this sum is calculated. Next, this square root is compared with a preset threshold value ε. In this case, if the square root is equal to or smaller than the threshold value ε, the three-dimensional shape of the bonding wire is determined to be good, and if the square root is larger than the threshold value ε, the three-dimensional shape of the bonding wire is determined to be bad. The above is a specific example of step S14.
[0036] Next, the details of the wire planar shape measurement in step S10 will be described with reference to Figures 6 and 7. Here, Figure 6 is a flowchart for explaining the details of the wire planar shape measurement in step S10, and Figure 7 shows a collection of coordinate points on the (x, y) plane of the image of the bonding wire 10 captured by the camera 2.
[0037] 6, in step S100, lighting is turned on by the computer 21 via the controller 20 to turn on the lighting device 5. In the next step S101, imaging is performed by issuing an imaging command from the computer 21 to the camera 2 to capture an image of the bonding wire 10. Then, from the captured image, the image of the bonding wire 10 is identified, and the position coordinates of multiple points on the (x, y) plane are measured.
[0038] In the next step S102, measuring a reference point on a plane, a corner of a structure near the bonding wire 10 of the device under test, for example, the bonding pad 11 located on the left side of the figure, is set as the reference point (xo, yo), and the position coordinates of this reference point on the (x, y) plane are measured. Here, the state in which the bonding pad 11 is located near the bonding wire 10 refers to the positional relationship between the end point of the bonding wire 10 and the bonding pad 11 when the end point of the bonding wire 10 is connected to the bonding pad 11.
[0039] In the next step S103, the start point is measured on the plane, and a search is made to determine whether the start point (xs, ys) of the bonding wire 10 of the device under test is in the correct position in relative coordinates from the reference point. Whether the start point (xs, ys) of the bonding wire 10 is in the correct position is determined using a threshold value th1 and the following equation (1).
[0040] In the above explanation, the position (xs, ys) of the end of the bonding wire 10 is taken as the starting point, but a point other than the position of the end of the wire, for example, point (x1, y1), may also be used as the starting point. Furthermore, the normal position means, for example, that the end point (xs, ys) of the bonding wire 10 is located within the bonding pad 11 shown in a rectangular frame in Figure 7. Note that step S103 is a step required to identify whether the object in the image is a bonding wire of the device under test.
[0041] In the planar shape measurement in step S104, the wire shape (a group of planar coordinates (x, y) of the wire) is measured sequentially at multiple points along the trajectory of the bonding wire 10 from the start point (xs, ys) of the bonding wire 10 of the device under test to another end point (xe, ye) of the bonding wire 10. For example, in FIG. 7, points (x1, y1), (x2, y2), ..., point (x5, y5) are measured. Then, as shown in FIG. 7, the shape of the bonding wire may be recorded as a set of the start point (xs, ys), intermediate points (x1, y1), (x2, y2), ..., (xn, yn), and end point (xe, ye), or the outline information of the wire may be recorded as is. Here, the number of intermediate points and the intervals between them are determined taking into consideration the length of the bonding wire 10, the pixel resolution of the camera 2, etc. The point (xe, ye) is a point in the bonding pad 12 that is different from the starting point (xs, ys) of the bonding pad 11, and is a point to which another end point of the bonding wire 10 is connected. The arrows x and y at the bottom left indicate the directions of the plane coordinates (x, y) in FIG.
[0042] However, if there are multiple wires in the image and the starting point (xs, ys) of the bonding wire 10 of the device under test is not in the correct position, it is not possible to identify which wire is the bonding wire 10 of the device under test.
[0043] If the bonding wire 10 of such a test object cannot be identified, the steps after step S103 (step S104) do not need to be performed for that wire, or the subsequent steps may be continued for the wire that is thought to have a high probability of being the bonding wire 10 of the test object.
[0044] For example, for wires that are likely to be the bonding wire 10 of the device under test, a threshold value th2 is used to determine the wire using the following equation (2): Among the wires that are likely to be the bonding wire 10 of the device under test, the wire that satisfies equation (2) and has the smallest value on the left side of equation (2) is selected and the subsequent steps are continued.
[0045] Next, the wire angle measurement method (step S11 in FIG. 4) will be described in detail below with reference to FIGS. 8, 9, and 10. Here, the angle of the bonding wire 10 with respect to a predetermined plane (here, a plane perpendicular to the optical axis of the lens 3) is measured as the angle of the bonding wire 10 in the depth direction when viewed from the camera 2.
[0046] First, Fig. 8 will be described. Fig. 8 is a diagram for explaining the principle of the above-mentioned wire angle measurement method. In Fig. 8, the bonding wire 10 specularly reflects illumination light from the light source 7, and the image is captured by the camera 2, thereby making it possible to measure the angle of the bonding wire 10 in the depth direction relative to the camera 2.
[0047] As shown in Figure 8, it is assumed that the bonding wire is placed at an angle θ with respect to a plane P perpendicular to the optical axis of the lens 3 attached to the camera 2 (this axis can also be said to be the optical axis of the camera 2; therefore, hereinafter, this axis will also be referred to as the optical axis of the camera 2). In this case, the angle of incidence and the angle of reflection of the illumination light with respect to the bonding wire are defined as φ, and the angle formed by these angles of incidence and reflection is defined as α. As shown in the figure, this angle α also represents the setting position of the light source with respect to the optical axis of the camera 2, and therefore, hereinafter, this α will also be referred to as the angular position of the illumination device.
[0048] 8, that is, when light emitted from a light source is reflected specularly onto a camera placed directly above the bonding wire, the relationship θ+φ=90° holds between the angle θ of the bonding wire and φ, which is the angle of incidence and reflection of the illumination light. Therefore, in such a case, it can be seen that the relationship α=2θ holds as a result.
[0049] From the above, it can be seen that when a certain point on the bonding wire glows brightly due to direct reflection of the illumination light, the angle θ representing the shape of that point on the bonding wire can be measured from the angular position α relative to the lens optical axis of the illumination device (light source).
[0050] Next, a method for measuring the wire angle shape at multiple points on the bonding wire 10 will be described using the fundamental wire angle measurement method described for the bonding wire 10. That is, using the wire angle shape measurement method described in step S11 (measurement of angle θ in FIG. 8), the angle θ at each measurement point on the bonding wire 10 is measured while changing the irradiation angle of the lighting device with respect to the bonding wire 10. Below, details of this method for measuring the wire angle shape at each measurement point on the bonding wire 10 will be described with reference to the flowchart in FIG.
[0051] 9 is a flowchart showing the details of the wire angle shape measurement in step S11 (see FIG. 4). In step S110, the lighting is turned on by the computer 21 via the controller 20, and the lighting device 5 is turned on. In the next step S111, the lighting angle is changed by changing the angle α of the lighting light relative to the optical axis of the camera within the range of αs≦α≦αe. Here, αs and αe may be determined according to the possible angles of the bonding wire 10 (the range of change in θ). The amount of change in the angle from αs to αe is determined according to the required measurement accuracy.
[0052] However, in order to capture the specular reflection light of a bonding wire that is perpendicular to the above-mentioned plane P (in this case, φ = 0° and θ = 90°) with a camera, the illumination light must be irradiated from α = 180°, i.e., directly below camera 2, and therefore, due to physical constraints, the angle of the bonding wire cannot be measured with this configuration.
[0053] In the next step S112, the camera height is adjusted to match the angle of the lighting device so that the camera height is focused on the bonding wire. In this case, since the shape of the bonding wire 10 is the same for good products, it is known which part of the wire will reflect specularly reflected light when illumination light is irradiated onto the bonding wire 10 at a certain angle θ. Therefore, the camera height is adjusted to a position where the focus is on the part of the bonding wire 10 that appears bright when specularly reflected light enters the camera. In the next step S113, imaging is performed by issuing an imaging command from the computer 21 to the camera 2, causing it to capture an image including the wire. In the next step S114, the angular shape θ of the bonding wire at each of multiple points is sequentially measured from the captured images.
[0054] 10, the case where illumination light is irradiated onto the bonding wire 10 at an angle α will be described below. If the value measured by wire angle measurement at measurement point (xm, ym) (where m is an integer from 1 to 5) measured by wire planar shape measurement in step S10 is θm (where m is an integer from 1 to 5), each measurement point p2m (where m is an integer from 1 to 5) shown in FIG. 10 can be expressed as p2m = p2((xm, ym), θm) (where m is an integer from 1 to 5). Then, assume that specularly reflected light from a location corresponding to the planar coordinates (xm, ym) of measurement point p2m is incident on camera 2, and a bright-looking image is captured.
[0055] In this case, the angle θ3 of the bonding wire 10 at measurement point p23 can be calculated from the angle α with respect to the optical axis of the illumination light as θ3 = 0.5α. The arrow (pair) shown in the lower left of Figure 10 indicates the direction of the coordinate system (x, z) in this figure. Here, x is the x direction of the above-mentioned plane coordinate system, and z is the height direction of the bonding wire 10 (the depth direction as seen from camera 2, which is also the optical axis direction) (the same applies to the following figures). In this way, in steps S111 to S114, the wire is imaged while changing the irradiation angle of the illumination light within the above-mentioned range of αs ≦ α ≦ αe, and the angular shape θ is measured each time.
[0056] Next, the details of the wire apex height measurement method (step S12 in FIG. 4) will be described below with reference to FIGS. 11 and 12. In the wire apex height measurement in step S12, the apex of the bonding wire 10, i.e., the point where the height in the z direction is maximum, is measured. This allows the three-dimensional shape of the bonding wire 10 to be measured more accurately.
[0057] 11 is a flowchart showing the details of measuring the wire apex height of the bonding wire 10. First, in step S120, an illumination device is turned on to capture an image of the apex of the bonding wire 10. The illumination light may be irradiated at an angle of α=90° so that the apex of the bonding wire 10 is strongly reflected, or may be irradiated at an angle so that the apex portion is uniformly illuminated using diffuse reflected light.
[0058] In the next step S121, the camera height is changed, and the camera position is adjusted to a height where the apex of the bonding wire 10 is in focus. The position of the apex of the bonding wire 10 as viewed on the (x, y) coordinate plane is assumed to be either a point where the θ measured at each measurement point in the wire angle shape measurement in step S11 is θ = 0, or a position where θa < 0 < θb holds for two adjacent measurement points (assuming the angle measurement values at these measurement points are θa and θb). Here, the upper limit hs and lower limit he of the camera height are appropriately determined depending on the range in which the apex of the bonding wire 10 can exist. The amount of change when changing the height from hs to he (the amount of change per position change) is determined depending on the required measurement accuracy.
[0059] In the next step S122, the computer 21 issues an image capture command to the camera 2 to capture an image of the bonding wire. Steps S121 to S122 are repeated to capture images of the apex of the bonding wire 10 at different heights, and a set of captured images is obtained. In the next step S123, the apex height zt of the bonding wire 10 is determined from the captured images.
[0060] Now, by repeating steps S121 to S122, multiple sets (I1, h1), (I2, h2), ..., (In, hn) of captured image I and camera height h are obtained, and from this set, the image that is most in focus as the vertex of bonding wire 10 is determined.
[0061] When making this determination, image processing (e.g., convolution integral) is performed using image I and, for example, an 8-neighbor Laplacian filter F, which is one of the image processing filters for image I. Here, when the elements of image I and Laplacian filter F are expressed as in the following equation (3), the result of the image processing is expressed, for example, as in equation (4) using element G of the convolution integral. That is, image processing expressed by the elements of the convolution integral is performed, and the image is judged based on the obtained results.
[0062] Here, the Laplacian filter F generally emphasizes areas in an image where the difference in pixel values between adjacent pixels is large through second-order differentiation (see, for example, Patent Document 2), so the brightness value of the bonding wire in image q increases as the image becomes more in focus. Then, a portion of image q containing a point assumed to be the vertex of the bonding wire is cut out, and the sum S of the pixel values in the cut-out range is calculated. The vertex height zt of the bonding wire 10 can be measured from the camera height h and the focal length of lens 3 when the sum S of pixel values is maximized.
[0063] The wire planar shape measurement in step S10, the wire angular shape measurement in step S11, and the wire apex height measurement in step S12 may be performed simultaneously. For example, the wire planar shape measurement, the wire angular shape measurement, and the wire apex height measurement may be performed simultaneously using a single image obtained by irradiating an object to be measured with illumination light at a certain angle α and capturing the image.
[0064] In the next step S13 of estimating the wire shape, the wire height at other measurement points is estimated based on the apex height zt of the bonding wire 10 measured in the wire apex height measurement in step S12 and the angle θ at each measurement point of the bonding wire 10 measured in the wire angle shape measurement in step S11.
[0065] 12, the measured vertex height zt of the bonding wire 10 corresponds to measurement point p3 = p((x3, y3), z3, θ3), (where z3 = zt). The same applies to other measurement points p1, p2, p4, and p5 (for example, p2 = p((x2, y2), z2, θ2)). Here, the relative distance Lxy on the (x, y) coordinate plane between measurement point (x3, y3) (where z = z3 = zt, θ = θ3) and measurement point (x2, y2) (where z = z2, θ = θ2) can be expressed by the following equation (5).
[0066] At this time, the bonding wire height z2 at the adjacent measurement point can be expressed by the following equation (6) using the above Lxy and θ3.
[0067] Using these, the bonding wire height can be estimated for all measurement points. Note that L1, L2, L3, and L4 represent the distance between p1 and p2, the distance between p2 and p3, the distance between p3 and p4, and the distance between p4 and p5 on the (x, z) coordinate system, respectively.
[0068] In step S14 (see FIG. 4), the quality of the bonding wire shape is judged based on the estimated data of the wire shape measured in the previous step. As described above, the judgement is made by comparing the estimated data of the measured wire shape with the wire shape data of a good product (normal shape).
[0069] Further, other judgment conditions may be added as necessary. For example, when wires cross each other, if the difference in wire height z at the crossing point is smaller than a certain threshold, the wires may be in contact with each other and the product may be judged as defective. Furthermore, when there is a risk of contact between the wire and a structure, the position data of the structure may be compared with the wire shape data, and whether or not there is contact may be judged based on the difference between the data and a threshold.
[0070] Embodiment 2 The inspection device for an object under inspection in embodiment 2 includes, in addition to the illumination device 5, an illumination device that projects a grid of equally spaced parallel lines, i.e., a grid projection illumination device 50. By using this grid projection illumination device 50 to measure the wire angle shape in step S11, inspection can be performed at higher speed.
[0071] 13 is a diagram showing a method for measuring the wire angular shape using a grating projection lighting device 50. It is assumed that a bonding wire 10 having an angle shape θ is irradiated with illumination light from the grating projection lighting device 50 at an irradiation angle φ with respect to the optical axis, an image of the grating pattern projected onto the bonding wire 10 is captured by a camera 2, and it is confirmed that the grating spacing is L1.
[0072] In this case, the above-mentioned irradiation angle φ can be expressed by the following equation (7) using the angular shape θ of the bonding wire 10, the spacing D1 between two adjacent lattices (spacing on the plane P), and the spacing D2 of the lattices created when θ = 0, i.e., the illumination light is irradiated from a direction perpendicular to the reference plane P.
[0073] Using the above formula (7), the angle θ of the bonding wire 10 can be measured from the interval L1 of the grid pattern projected onto the bonding wire 10. This measurement method makes it possible to measure the angle θ of the bonding wire 10 with higher accuracy. Since the steps other than the wire angle shape measurement in step S11 are the same as those in the first embodiment, detailed description thereof will be omitted here.
[0074] Embodiment 3 The inspection device in embodiment 3 has the same configuration as the inspection device in embodiment 1 (configuration in which the illumination device 5 can irradiate illumination light from any angle), but is also configured to be able to irradiate illumination light having a spectral distribution and a predetermined center wavelength or peak wavelength (illumination light having a predetermined spectral distribution) from different angles simultaneously (see FIG. 14 ).
[0075] With this configuration, for example, by irradiating illumination light with a central wavelength λ1 from an angle α1 and simultaneously irradiating illumination light with a central wavelength λ2 from an angle α2 and capturing images with camera 2 (not shown), the number of times images are captured in measuring the wire angle shape in step S11 can be reduced, enabling faster inspection.
[0076] Furthermore, if the hyperspectral camera 2a is used to irradiate illumination light (illumination light having different spectral distributions) with different center wavelengths or peak wavelengths (in FIG. 14, the illumination light has five different center wavelengths or peak wavelengths, λa, λb, λc, λd, and λe, and has spectrally distributed characteristics) from more angles simultaneously, the number of times images are taken can be similarly significantly reduced (see FIG. 14).
[0077] 14, the solid arrows indicate illumination light irradiated onto the bonding wire 10 from a dome-shaped illumination device, and the dotted arrows indicate specular light reflected from the bonding wire 10 toward the hyperspectral camera 2a by the irradiated light. Furthermore, the process is the same as that of the first embodiment except for the wire angle shape measurement in step S11.
[0078] Fourth Embodiment In the inspection device of the first embodiment, the camera 2 and the lighting device 5 or the stage 6 may be provided with a movable mechanism, so that the positional relationship between the bonding wire 10, the camera 2, and the lighting device 5 can be freely changed (see, for example, FIGS. 15A and 15B). An inspection device with such a configuration is the fourth embodiment.
[0079] With the above-described configuration, in the wire angle shape measurement in step S11, it is possible to measure the three-dimensional shape of the complex bonding wire 10. For example, when the bonding wire 10 is present between structures with different heights as shown in Fig. 15, if the inspection is performed by imaging from directly above, it is difficult to measure the angle of the nearly vertical part of the wire, and therefore only the very apex of the bonding wire can be measured (see the measurable range Ra in Fig. 15A).
[0080] However, by adjusting the positional relationship between the bonding wire 10, the camera 2, and the lighting device 5, it becomes possible to measure the angle of a point that cannot be measured from directly above (see the measurable range Ra in FIG. 15B), so accurate angle measurement can be performed even when measuring a bonding wire with a complex shape. Note that, since the steps other than the wire angle shape measurement in step S11 are the same as those in embodiment 1, a description thereof will be omitted here.
[0081] The controller 20 described above, like the computer 21, is configured with a processor 100 and a storage device 101, an example of hardware of which is shown in FIG. 16 . The storage device 101 includes a volatile storage device such as a random access memory (not shown) and a non-volatile auxiliary storage device such as a flash memory. Alternatively, a hard disk auxiliary storage device may be included instead of the flash memory. The processor 100 executes a program input from the storage device 101. In this case, the program is input to the processor 100 from the auxiliary storage device via the volatile storage device. The processor 100 may output data such as calculation results to the volatile storage device of the storage device 101, or may store the data in the auxiliary storage device via the volatile storage device.
[0082] Although various exemplary embodiments and examples are described herein, the various features, aspects, and functions described in one or more embodiments are not limited to the application of a particular embodiment, but may be applied to the embodiments individually or in various combinations. Therefore, countless variations not illustrated are contemplated within the scope of the technology disclosed herein. For example, this includes modifying, adding, or omitting at least one component, or extracting at least one component and combining it with components of other embodiments. Specifically, while the number of measurement points for three-dimensional measurement of the object to be measured is described as five, the number of measurement points, measurement range, etc. can be changed depending on the measurement target or measurement purpose. For example, when measuring whether or not intersecting wires are in contact with each other, it is possible to limit the area to the area where contact is possible and measure only that area, thereby shortening the measurement time.
[0083] DESCRIPTION OF SYMBOLS 1 Shape inspection device for inspected object, 2 Camera, 2a Hyperspectral camera, 3 Lens, 4 Height adjustment mechanism (adjustment mechanism), 5 Illumination device, 6 Stage, 7 Light source, 10, 10a, 10m, 10n, 10s Bonding wire, 11, 12 Bonding pad, 20 Controller, 21 Computer, 50 Grating projection illumination device, 100 Processor, 101 Storage device
Claims
1. An apparatus for inspecting a shape of an object to be inspected, comprising: a camera that is movable relative to the object to be inspected and captures an image of the object to be inspected; an adjustment mechanism for adjusting the distance between the camera and the object to be inspected; an illumination device having a light source that irradiates light onto the object to be inspected and the position of the light source can be changed; a controller that controls the operation of the adjustment mechanism and also controls the operation of changing the position of the light source of the illumination device, turning the light source on, changing the spectral distribution of the light source, and the operation of the camera to capture an image of the object to be inspected; and a computer that outputs operational commands to the object to be inspected, respectively, to change the positions of the camera and the light source, turn the light source on, change the spectral distribution of the light source, and have the camera capture an image of the object to be inspected; wherein the apparatus moves the position of the camera and changes the position at which the light source irradiates the object to be inspected, captures an image of the object to be inspected, and determines the three-dimensional shape of the object to be inspected by estimating the shape of the object to be inspected by the computer based on the captured image and comparing it with a shape of a reference object to be inspected that has been measured in advance.
2. The shape inspection device for an object to be inspected according to claim 1, further comprising a grating projection illumination device for projecting a grating made up of equally spaced parallel lines.
3. The shape inspection device for an object to be inspected according to claim 1, characterized in that the camera is a hyperspectral camera, the illumination device has a plurality of light sources which irradiate the object to be inspected from different angular positions and irradiate light having different spectral distributions, and the illumination device emits light simultaneously from a plurality of angular positions onto the object to be inspected.
4. The device for inspecting the shape of an object to be inspected according to claim 1, characterized in that the object to be inspected is mounted on a movable stage.
5. A method for inspecting the shape of an object under inspection using the device for inspecting the shape of an object under inspection described in claim 1, comprising: determining a three-dimensional shape of the object under inspection using the device for inspecting the shape of the object under inspection using the device for inspecting the shape of the object under inspection, and discriminating the shape of the object under inspection, from: a planar position of the object under inspection when the object under inspection is projected onto a plane perpendicular to the optical axis of the camera while changing the position of the lighting device relative to the object under inspection, and an angular shape of the object under inspection determined from the angular position of the lighting device when specularly reflected light from the object under inspection is incident on the camera; and a vertex height of the object under inspection determined from an image captured at a focused position when the camera is moved in the direction of the optical axis of the camera relative to the object under inspection.
6. A method for inspecting the shape of an object to be inspected using the shape inspection device for an object to be inspected described in claim 2, characterized in that the method measures the three-dimensional shape of the object to be inspected and determines whether the shape of the object to be inspected is good or bad based on the grating spacing measured by the camera when the grating projection lighting device irradiates the object to be inspected and the angular shape of the object to be inspected.
7. A method for inspecting the shape of an object to be inspected using the device for inspecting the shape of the object to be inspected according to claim 3, characterized in that illumination light is irradiated onto the object to be inspected from a plurality of positions simultaneously.
8. A method for inspecting the shape of an object to be inspected using the shape inspection device for the object to be inspected according to claim 4, characterized in that the object to be inspected is measured by changing the positional relationship between the object to be inspected and the camera and the lighting device.
Citation Information
Patent Citations
Visual inspection device and visual inspection method
JP2012053015A
Wire shape inspecting device and wire shape inspecting method
WO2019124508A1
