Appearance inspection device

Abstract

This invention provides a highly reproducible and accurate appearance inspection technology. An appearance inspection device (10) includes: a camera (32) consisting of an imaging optical system and an image sensing element for acquiring an image (18) for appearance inspection; a mechanism (31) for changing the positional relationship between the camera and the object under test (30) by a moving distance equal to the dividing distance when the field of view of the camera (32) is divided into M×N (2≦M×N) equal parts; and an image processing and detection unit (23) for processing and inspecting the image (18) obtained from the camera; wherein, by means of the mechanism (31), each time the moving distance is made, at least one image is acquired by the camera (32), and M×N images of all the tested areas of the object under test (30) are acquired, and the image processing and detection unit (23) performs inspection processing on the acquired image (18).

Description

Technical Field

[0001] This invention relates to an appearance inspection technique for various products that utilizes image processing technology. Prior Technology

[0002] Appearance inspection technology, which uses cameras as devices for acquiring images to perform appearance inspection processing on various products, is becoming increasingly widespread. A camera consists of an imaging optical system that forms an optical image of the object under test, and an image sensing element (generally a complementary metal-oxide-semiconductor image sensor; CMOS image sensor) that converts the light intensity of the image into an electronic image through photoelectric conversion. The size of the image obtained in one photograph is called the field of view.

[0003] While the field of view can be changed through lens magnification, in inspection, the size of the object being inspected—equivalent to a pixel on the image sensor (i.e., pixel resolution)—is the primary factor determining inspection performance. Therefore, the magnification of the imaging optical system must be set to achieve the desired pixel resolution, meaning the field of view cannot be freely set. Consequently, when inspecting objects larger than the field of view, the camera or object is moved, and multiple fields of view are used to inspect the entire object. For example, if the camera's field of view is 11×11mm and the object is 20×20mm, four fields of view can be used to inspect the entire object. This type of inspection will be referred to as multi-field inspection.

[0004] Specifically, bump detection in the semiconductor packaging process (later steps) is described below. IC (integrated circuit) wafers must be encapsulated in resin to prevent degradation and for heat treatment. Packaging involves mounting the wafer on a multilayer wiring board, connecting the electrodes on the multilayer wiring board to the electrodes on the wafer, and then encapsulating it in resin. However, while wire bonding was traditionally used for electrode connections, flip-chip packaging, which primarily uses a two-dimensional array of tiny, spherical electrodes (hereinafter referred to as "bumps") for connection, has become increasingly popular in recent years.

[0005] Since flip-chip packaging involves direct electrode overlap, consistent bump size is crucial. Inconsistent bump size can lead to poor connections. Therefore, the bump morphology must be accurately inspected before connection (see, for example, Patent Document 1).

[0006] In the visual inspection of bumps, one step in the online inspection process during manufacturing is to photograph the bumps arranged in an array from above using a camera. In recent years, with the widespread adoption of comprehensive inspection, inspection speed has become extremely important. Current bumps range in diameter from 10 to 80 μm, and on each multilayer wiring board, bumps are arranged with a spacing of 20 to 130 μm, numbering in the hundreds to tens of thousands. Furthermore, all inspections of these bumps on each multilayer wiring board must be performed at a high speed of one to tens of seconds.

[0007] In recent years, with the development of packaging technology using the chiplet concept and the mounting of a large number of chips on multilayer wiring boards, the size of multilayer wiring boards has become larger. On the other hand, due to the miniaturization of circuits, the diameter of bumps has become smaller, requiring fine pixel resolution during appearance inspection, resulting in a smaller field of view. As a result, "multi-field inspection" with a large number of fields of view has become necessary. [Previous Technical Documents] [Patent Literature]

[0008] [Patent Document 1] Japanese Patent No. 4984332 Summary of the Invention

[0009] [The problem that the invention aims to solve] One of the problems with using images for detection is the aberrations inherent in the imaging optical system. Due to these aberrations, image quality is uneven within the field of view; for example, the aberration state differs between the center and periphery of the field of view, resulting in variations in image quality. If image quality differs, the detection results may also differ. For instance, in detecting the height of a bump, even for the same bump, the height value at the center of the field of view may differ from the height value at the periphery (Topic 1).

[0010] Furthermore, as the measured object becomes increasingly miniaturized, requiring higher precision detection, the pixel configuration and the object's position can affect the detection results. For example, a spherical bump, being relatively close to a mirror surface, will only receive reflected light from its apex when illuminated from directly above. Therefore, as the bump diameter decreases, the area where reflected light returns also decreases, and the size of the captured apex will be smaller than the camera's pixel size. This results in situations where reflected light from the bump's apex is captured by a single pixel versus multiple adjacent pixels. Consequently, depending on the situation, the bump's morphological information will differ, leading to a decrease in the accuracy and reproducibility of the detection results (Problem 2).

[0011] This invention was made in consideration of such circumstances, and its purpose is to provide a highly reproducible and accurate appearance inspection technology. [Technical means to solve the problem]

[0012] The appearance inspection apparatus of the present invention comprises: a camera, which is composed of an imaging optical system and an image sensing element, for acquiring images for appearance inspection; a mechanism, which changes the positional relationship between the camera and the object under test by moving a distance that is almost equal to the division distance when the field of view of the camera is divided into M×N (2≦M×N) equal parts; and an image processing and detection unit, which processes and performs inspection on the images acquired from the camera; wherein, by means of the aforementioned mechanism, each time the camera moves by the aforementioned moving distance, at least one image is acquired by the aforementioned mechanism, and M×N images of all the measured areas of the object under test are acquired, and the aforementioned image processing and detection unit performs inspection processing on the acquired images. [Effects of the Invention]

[0013] According to the present invention, a highly accurate appearance inspection technology can be provided. Simple Explanation of the Diagram

[0014] [Figure 1] is a block diagram of the appearance inspection device according to the first embodiment of the present invention. [Figure 2] (A) is a top perspective view of a multilayer wiring board with bumps arranged in an array; (B) is a side perspective view of a multilayer wiring board on which a chip is mounted through the bumps. [Figure 3] is an explanatory diagram of the operation of the appearance inspection device according to embodiments (A) to (F). [Figure 4] is an explanatory diagram of the operation of the appearance inspection device in embodiments (G) to (L). [Figure 5] (A) is a side view showing the reflected light from the vertex into the camera in bumps of different sizes; (B) is the same as above, a top view. [Figure 6] (A) shows the image of the bump when reflected light from the vertex is detected by one pixel; (B) same as above, shows the image of the bump when it is detected by two pixels; (C) same as above, shows the image of the bump when it is detected by four pixels. [Figure 7] shows the algorithm for appearance detection. Implementation

[0015] (First Embodiment) Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. For specific explanation, bump detection will be used as an example. FIG1 is a block diagram of the appearance inspection device 10 according to the first embodiment of the present invention. FIG2(A) is a top perspective view of a multilayer wiring board (test object) 30 in which bumps 14 are arranged in an array. FIG2(B) is a side perspective view of an IC package 37 in which a chip 36 is mounted on the multilayer wiring board 30 through the bumps 14.

[0016] As shown in Figure 2, the electrodes of the multilayer wiring substrate 30 (hereinafter referred to as "substrate 30") of the IC package 37 and the electrodes of the IC chip 36 are electrically bonded by bumps 14. This method of electrical connection using bumps 14 is called flip-chip packaging. In this embodiment, the bumps 14 arranged on the substrate 30 are inspected. Then, the chip 36 is mounted, and to prevent deterioration and for heat treatment, it is encapsulated in resin 35 to complete the IC package 37. Furthermore, although the test object of the appearance inspection device 10 in this embodiment is an example of a substrate 30 with bumps 14, it is not particularly limited to this.

[0017] As shown in Figure 1, a plurality of substrates 30 (3 × 5 = 15 in the figure) with bumps 14 arranged on them (Figure 2) are housed on a tray 38. The tray 38, which houses the plurality of substrates 30, is placed on an inspection table 39 and inspected by an appearance inspection device 10. Then, the inspected tray 38 is transferred to the next step, while the tray 38 containing the uninspected substrates 30 is placed back on the inspection table 39. Furthermore, the substrate 30 with bumps 14 arranged on it is only one example of a multilayer wiring board and is not particularly limited to it.

[0018] Figures 3(A) to (F) and 4(G) to (L) are explanatory diagrams of the operation of the appearance inspection device 10 for protrusions 14 according to one embodiment. The description continues with reference to Figure 1. The field of view 12 of the camera 32 is divided into four equal parts of 2×2, each part being called a segment 11. The device includes: a mechanism 31 that sequentially moves the field of view 12 in a stepped manner relative to the test area 14a where a group of protrusions 14 are located, with a movement distance equal to the division distance of the segments 11; and an image processing and detection unit 23 that performs image processing and detection on the image 18 of the field of view 12 obtained from the camera 32.

[0019] Furthermore, in the above description, although the size of segment 11 is four equal parts of the field of view 12 of camera 32, this does not need to be very precise. When performing multi-field detection, a certain degree of overlap is usually set between adjacent fields of view. For example, taking a bump as an example, if the bump is divided and photographed by adjacent fields of view, the measurement process becomes complicated. Therefore, in order to avoid being photographed by any field of view, it is necessary to overlap the photographs in a way that allows the area above the distance between the bumps to be photographed in any field of view. Excluding such areas, as long as it is divided into four equal parts, it does not need to be strictly one-quarter of the field of view. The fact that the dimensions of the field of view 12 and the four segments 11 are not consistent in Figures 3 and 4 illustrates this situation.

[0020] Furthermore, the 4-equal division is only one example; the generalized result is an M×N equal division (2≦M×N). Therefore, the field of view 12 is moved sequentially in a step-like manner with a movement distance equal to the segmentation spacing of section 11, so that all bumps are photographed M×N times. This allows the image information from the M×N images to be used to determine the morphological information 24 of the bump 14, thus improving the accuracy of the determination.

[0021] The mechanism 31 moves the camera 32 sequentially in a stepped manner with a movement distance equal to the dividing distance of the section 11. In addition, as a variation of the mechanism 31, although omitted in the figure, it can also be used to move the detection table 39 on which the carrier plate 38 is placed in a stepped manner.

[0022] The operation of the bump appearance detection device 10 first involves positioning the field of view 12 of the camera 32, as shown in FIG3(A), so that it covers a segment 11 located in the corner of the test area 14a where a group of bumps 14 exist, and acquiring an image 18 in the image processing and detection unit 23 (FIG. 1).

[0023] Next, as shown in Figure 3(B), the field of view 12 is moved one step along the arrangement direction of the protrusions 14 with a movement distance equal to the segmentation spacing, and the image 18 is acquired in the image processing and detection unit 23. Then, the sequential movement of the field of view 12 and the acquisition of the image 18 are repeated along the arrangement direction of the protrusions 14 until the opposite end, as shown in Figure 3(C). Then, it is moved one step in the vertical direction, as shown in Figure 3(D), and the sequential movement of the field of view 12 and the acquisition of the image 18 are repeated in the opposite direction until the opposite end, as shown in Figure 3(E). The same applies below, from Figure 3(F) to Figures 4(G) to (L), the sequential movement and acquisition of the image 18 are repeated in a step-like manner with a movement distance equal to the segmentation spacing of the segment 11.

[0024] In this way, the image processing and detection unit 23 acquires M×N (4 in the figure) images 18 for each of the plurality of segments 11. Moreover, each of the same segment 11 captured by the plurality of images 18 is taken in a different region of the M×N divided field of view 12.

[0025] Furthermore, the image 18 is typically affected by aberrations in the imaging optical system. Generally, aberrations tend to increase and image quality decreases as the image moves from the center to the periphery of the field of view. Moreover, the effects of these aberrations are not necessarily point-symmetrical due to manufacturing errors such as eccentricity. On the other hand, it is generally accepted that bumps 14 captured in different regions of the field of view 12 are located at positions with different aberration conditions in the image 18, resulting in differences in accuracy. Therefore, among M×N images 18 (4 in the figure), some will be detected in the center of the field of view 12, while others will be detected in the periphery. Furthermore, by statistically processing these detection values, the accuracy of determining the shape information 24 of the bump 14 can be made consistent.

[0026] Figure 5(A) is a side view showing the reflected light 26 (26x, 26y) from vertex 27 incident on camera 32 through bumps 14 (14x, 14y) of different sizes. Figure 5(B) is the same as above, but is a top view. Because the spherical bump 14 is relatively close to the mirror surface, if illumination is provided from directly above, the reflected light 26 (26x, 26y) incident on camera 32 will be limited to a portion of vertex 27 (27x, 27y). Therefore, as the diameter of bump 14 decreases, the area where the reflected light 26 returns to vertex 27 also decreases.

[0027] Figure 6(A) shows an image 18 of the bump 14 when the reflected light 26 from vertex 27 is detected by one pixel 22; (B) same as above, showing an image 18 of the bump 14 when it is detected by two pixels 22; (C) same as above, showing an image 18 of the bump 14 when it is detected by four pixels 22. Thus, when the bump 14 becomes smaller, and the size of the captured vertex 27 is smaller than the size of the camera's pixel 22, the vertex 27 can be captured by one pixel, or by a plurality of adjacent pixels.

[0028] The movement distance of the field of view 12 is the size of the segment 11 (= the segmentation distance). However, by setting the movement distance to an integer multiple of the pixel resolution and adding 1 / M (for vertical movement distance) or 1 / N (for horizontal movement distance) of the pixel resolution, if the movement amount is accurate, then M×N images 18 can respectively obtain the image after the vertical offset of 1 / M and the horizontal offset of 1 / N of the positional relationship between the bump and the pixel 22.

[0029] That is, the value obtained by multiplying the segmentation distance of section 11 by the magnification of the imaging optical system and dividing it by the size of the pixel 22 of the camera 32 (pixel size), taking its approximate integer value, and adding 1 / M times the pixel size in the vertical direction and 1 / N times in the horizontal direction, and dividing such values ​​by the magnification of the imaging optical system, so that the obtained values ​​become the vertical and horizontal dimensions of the movement distance.

[0030] By employing such pixel shifting, the improvement in pixel resolution can be confirmed, mitigating the impact of differences in the positional relationship between pixels and the object on the measured values, as shown in Figure 6. This, in turn, enhances the accuracy and reproducibility of the detection results.

[0031] The image processing and detection unit 23 (Fig. 1) determines the morphological information 24 of the bump 14 of interest based on the four (M×N) vertex pixels 22a acquired in sequence. The morphological information 24 specifically includes the position, height, outer diameter of the bump 14, shape anomalies of other bumps 14, and the height and surface foreign matter of the substrate 30.

[0032] The measured values ​​of the morphological information 24 are obtained from four (M×N) images 18 representing the morphology of each bump 14, based on the brightness values ​​of the vertex pixel 22a and / or its surrounding pixels 22. A total of four (M×N) values ​​are then obtained. Statistical processing is performed on these multiple measured values, and one representative value is output for each bump 14. Furthermore, the measured values ​​and their representative values ​​representing the morphological information 24 of the bump 14 are not limited to a single item; multiple items can be determined.

[0033] Figure 7 shows the operational formula for the algorithm of appearance inspection. The representative value V of morphological information 24 is obtained from a complex number of measurement values ​​vᵢ according to a complex number of algorithms (3 types) illustrated below. However, the operation of the representative value V is not limited to these algorithms.

[0034] The first algorithm is the "simple averaging". The "simple averaging" is based on equation (1) in Figure 7, which averages M×N measurement values ​​vᵢ (i=1 to M×N) to calculate the representative value V. Through the effect of averaging (law of large numbers, central limit theorem), the reliability is improved compared to selecting one measurement value vᵢ as the representative value V.

[0035] The second algorithm is a "reliability-based weighted average". This "reliability-based weighted average" is based on equation (2) in Figure 7, using the reliability value tᵢ of pixel 22 after deriving M×N measurement values ​​vᵢ (i=1 to M×N) as a weighting coefficient, multiplied by the corresponding measurement value vᵢ. Then, a weighted average is performed to calculate the representative value V. By using this "reliability-based weighted average", the influence of pixel segmentation shown in Figures 6(B) and (C) can be suppressed, and the reliability of the representative value V can be improved.

[0036] The reliability value here is a measure of the reliability of the detected / measured values ​​of each pixel, and it varies depending on the appearance inspection method. For example, the brightness value of a pixel represents the amount of light as a signal, so it can be said that reliability is expressed by whether the signal is well obtained, and it is an example of a reliability value.

[0037] The third algorithm is a "weighted average based on distance from the center". This "weighted average based on distance from the center" is based on equation (3) in Figure 7, using the distance Lᵢ (i=1 to M×N) of pixel 22 from the center of the field of view 12 after deriving M×N measurement values ​​vᵢ (i=1 to M×N) as a weighting coefficient, multiplied by the corresponding measurement value vᵢ. Then, a weighted average is performed to calculate the representative value V. Through this "weighted average based on distance from the center", by emphasizing the values ​​measured near the center of the field of view, the influence of aberrations can be suppressed, and the reliability of the representative value V can be improved.

[0038] According to the present invention, the influence of aberrations in the imaging optical system (first issue) and the influence of pixel segmentation (second issue) in appearance inspection using images can be reduced, thereby improving the reliability of inspection. On the other hand, there is a clear negative factor of increased inspection time. As M×N increases, the reliability of inspection is higher, but the inspection time also increases. However, by equipping the appearance inspection device with the functions of the present invention, users can select the most suitable reliability / speed, which is considered a very significant advantage.

[0039] 10: Appearance inspection device 11: Section 12: Vision 14: Bumps 14a: A group of bumps (test area) 15: Adjustment Department 18: Images 22: pixels 22a: Vertex pixel 23: Image Processing and Inspection Department 24: Morphological Information 26: Reflected light 27: Vertex 30: Multilayer wiring board (substrate, test object) 31: Department of Organization 32: Camera 35: Package 36: IC chips 37: IC package 38: Carrier disk 39: Testing station

Claims

1. An appearance inspection apparatus comprising: a camera, comprising an imaging optical system and an image sensing element, for acquiring an image for appearance inspection; a mechanism for changing the positional relationship between the camera and the object under test by a moving distance equal to the segmentation distance when the field of view of the camera is divided into M×N (2≦M×N) equal parts; and an image processing and inspection unit for processing the image obtained from the camera and performing inspection; wherein, By means of the aforementioned mechanism, each time the movement is performed at the aforementioned moving interval, at least one image is acquired by the aforementioned camera, and M×N images of all the tested areas of the test object are acquired, and the acquired images are detected and processed by the aforementioned image processing and detection unit.

2. The appearance inspection device as described in claim 1 is configured to: multiply the aforementioned segmentation distance by the magnification of the imaging optical system, divide the value by the pixel size of the aforementioned camera, take the obtained value as an approximate integer, and increase the value by 1 / M times the aforementioned pixel size in the vertical direction and by 1 / N times in the horizontal direction, and divide the values ​​by the magnification of the imaging optical system, so that the obtained value becomes the vertical and horizontal dimensions of the aforementioned movement distance.

Images