Image processing device, image processing method, and inspection device

Abstract

To provide an image processing device, an image processing method, and an inspection device that facilitate positioning with respect to an inspection object in which pads are arranged at a high density.SOLUTION: In an inspection device 1 for inspecting an internal wiring circuit formed on an inspection object (substrate), multiple pad rows each including multiple pads arranged along an X direction on the inspection object are arranged in a Y direction to form multiple rows. An image processing unit 81, which sets a positioning reference point with respect to the inspection object, includes a pad area extraction unit 814 configured to execute pad area extraction processing for extracting a substantially rectangular pad area surrounding multiple pads from an image of an inspection object 100, a virtual straight line setting unit 815 configured to execute virtual straight line setting processing for setting a virtual first straight line parallel to the side extending in the X direction and setting a virtual second straight line parallel to the side extending in the Y direction of the pad area, and a reference point setting unit 816 configured to execute reference point setting processing for setting an intersection of the first straight line and the second straight line as a reference point.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an image processing device and an image processing method for setting a reference point for positioning by image processing, and an inspection device using the same. [Background technology]

[0002] Conventionally, a substrate inspection device has been known that includes a CCD camera that detects the position of a mark on a substrate having the mark, a calculation unit that calculates the amount of misalignment and tilt of a circuit pattern from the positions of at least two marks detected by the CCD camera, and a control unit that operates a positioning mechanism according to the amount of misalignment and tilt (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-200314 Summary of the Invention [Problem to be solved by the invention]

[0004] In order to provide marks on a board, it is necessary to secure space on the board for the marks. However, in recent years, pad arrangements have become denser, making it difficult to secure space for providing marks on components with pads. Therefore, there is a need to be able to position a board without providing positioning marks on the board.

[0005] An object of the present invention is to provide an image processing device, an image processing method, and an inspection device that can facilitate positioning of an object to be inspected in which pads are arranged at high density. [Means for solving the problem]

[0006] An image processing device according to one example of the present invention is an image processing device that sets a positioning reference point for an object to be inspected, in which pad rows in which a plurality of pads are lined up along a first direction are formed in multiple rows in a second direction that intersects the first direction, and is equipped with a pad area extraction unit that performs a pad area extraction process to extract an approximately rectangular pad area surrounding the plurality of pads from an image of the object to be inspected, a virtual line setting unit that performs a virtual line setting process to set a virtual first line parallel to a side of the pad area extending in the first direction and to set a virtual second line parallel to a side extending in the second direction, and a reference point setting unit that performs a reference point setting process to set the intersection of the first line and the second line as the reference point.

[0007] Moreover, an inspection apparatus according to an embodiment of the present invention positions the inspection object using the reference point set by the image processing apparatus described above.

[0008] Moreover, an image processing method according to one example of the present invention is an image processing method for setting a positioning reference point for an object to be inspected, in which pad rows, in which a plurality of pads are lined up along a first direction, are formed in multiple rows in a second direction intersecting the first direction, and includes a pad area extraction process for extracting an approximately rectangular pad area surrounding the multiple pad rows from an image of the object to be inspected, a virtual line setting process for setting a virtual first line parallel to a side of the pad area extending in the first direction and a virtual second line parallel to a side extending in the second direction, and a reference point setting process for setting the intersection of the first line and the second line as the reference point. [Effects of the Invention]

[0009] The image processing device, image processing method, and inspection device configured as described above can facilitate positioning of an object to be inspected in which pads are arranged at a high density. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is an explanatory diagram conceptually showing a schematic configuration of an inspection device 1 equipped with an image processing device according to one embodiment of the present invention. [Figure 2] 2 is a block diagram showing an example of mainly the electrical configuration of the inspection device 1 shown in FIG. 1. FIG. [Figure 3] FIG. 2 is an explanatory diagram showing an enlarged view of the inspection object 100 shown in FIG. [Figure 4] 10 is a flowchart showing an example of noise reduction processing, binarization processing, narrowing-down processing, selection processing 1 and 2, and pad region extraction processing (A1) to (A4). [Figure 5] 10 is a flowchart showing an example of pad area extraction processing (A5), (A6), virtual line setting processing (B1), (B2), and reference point setting processing. [Figure 6] 10 is an explanatory diagram for explaining a case where illumination light is irradiated parallel to the optical axis of the imaging unit 41 and an image is captured by the imaging unit 41. FIG. [Figure 7] 10 is an explanatory diagram showing an example of an image G1 captured by the imaging unit 41 when illumination light is irradiated parallel to the optical axis of the imaging unit 41. FIG. [Figure 8] 10 is an explanatory diagram for explaining a case where an image is captured by the imaging unit 41 when illumination light is obliquely irradiated onto the pad BP. FIG. [Figure 9] 10 is an explanatory diagram showing an example of an image G1 captured by the imaging unit 41 when illumination light is obliquely irradiated onto the pad BP. FIG. [Figure 10] FIG. 10 is an explanatory diagram showing an example of an image of a pad BP selected by selection processes 1 and 2. [Figure 11] FIG. 10 is an explanatory diagram showing an example of an image of a pad BP in an image G2. [Figure 12] FIG. 10 is an explanatory diagram for explaining the pad region extraction process (A1). [Figure 13] FIG. 10 is an explanatory diagram showing an example of a presence check area CA. [Figure 14] FIG. 10 is an explanatory diagram showing an example of grouped pad areas BA. [Figure 15] 10 is a flowchart showing an example of a virtual straight line setting process (B1). [Figure 16] FIG. 10 is an explanatory diagram for explaining a method for selecting peripheral pads. [Figure 17]FIG. 10 is an explanatory diagram for explaining step S22. [Figure 18] FIG. 10 is an explanatory diagram for explaining the expansion area EA in detail. [Figure 19] FIG. 10 is an explanatory diagram showing an example of a pad candidate; [Figure 20] 10 is a flowchart showing an example of a virtual straight line setting process (B2). [Figure 21] FIG. 10 is an explanatory diagram showing an example of a linear region LA expanded in step S31. [Figure 22] FIG. 10 is an explanatory diagram showing an example of a pad candidate selected as a boundary pad. [Figure 23] 10 is an explanatory diagram showing an example of a first straight line VL1, a second straight line VL2, and a reference point RP. FIG. [Figure 24] 1 is an explanatory diagram showing an example of a state in which a reference point RP is set on an inspection object 100. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In each drawing, components denoted by the same reference numerals are the same components, and their description will be omitted. The inspection device 1 shown in FIGS. 1 and 2 is a device for inspecting an internal wiring circuit formed in an inspection object 100, which is an example of an inspection object.

[0012] The inspection object 100 may be, for example, an interposer, a semiconductor wafer, a semiconductor package, a semiconductor substrate, a film carrier, a printed wiring board, a flexible substrate, a ceramic multilayer wiring board, an electrode plate for a display such as a liquid crystal display, or the like.

[0013] 3, a plurality of pads BP to be inspected are arranged in a pad row L in the X direction (first direction) on the inspection object 100. The pads BP may be, for example, three-dimensionally protruding bumps or planar electrodes.

[0014] In addition to the pad BP to be inspected, pads, patterns, etc. that are not to be inspected but are different in size, shape, etc. from the pad BP may be formed on the inspection object 100. In the following explanation, as an example, a case where the pad BP is a bump will be explained.

[0015] Furthermore, the pad row L only needs to include at least three pads BP, and the pad row L only needs to have at least three rows. In other words, the number of pads BP needs to be 3×3=9 or more. If the pad row L includes three or more pads BP and the pad row L has three or more rows, it becomes possible to set a first straight line VL1 and a second straight line VL2, which will be described later, based on three or more pads BP each.

[0016] The inspection device 1 shown in FIG. 1 generally comprises an inspection mechanism 4, a fixing device 6, and an inspection unit 8. The fixing device 6 is configured to fix an inspection target 100 to be inspected at a predetermined position. The fixing device 6 may be configured to transport the inspection target 100 to the inspection position by sliding. The inspection mechanism 4 comprises an inspection jig 3 and an imaging unit 41. The imaging unit 41 is attached directly or indirectly to the inspection jig 3.

[0017] The inspection mechanism 4 supports the inspection jig 3 and the imaging unit 41 above the fixing device 6. The inspection object 100 held by the fixing device 6 is movable in three mutually orthogonal axis directions of X, Y, and Z, and is rotatable around the Z axis, by a driving mechanism 801. In FIG. 1, an example is shown in which the up-down direction is the Z axis.

[0018] The inspection mechanism 4 is provided with an inspection jig 3 detachably mounted thereon for inspecting the internal wiring circuit formed in the inspection object 100 .

[0019] The imaging unit 41 is a camera configured using, for example, a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) imaging element. The imaging unit 41 captures an image of the surface of the inspection object 100. The image captured by the imaging unit 41 is output to the control unit 80 of the inspection unit 8.

[0020] The imaging unit 41 is attached to the inspection jig 3. Therefore, based on the image captured by the imaging unit 41, the relative positional relationship between the inspection jig 3 and the inspection object 100, and the inclination of the inspection jig 3 with respect to the inspection object 100 (the rotation angle around the Z axis) can be determined.

[0021] The inspection jig 3 is equipped with a plurality of probes P. The rear end of each probe P is electrically connected to the inspection unit 8. The plurality of probes P are arranged in the same manner as the plurality of pads BP. Therefore, by appropriately positioning the inspection jig 3 and the inspection object 100, it is possible to bring each probe P into contact with each pad BP.

[0022] 2, the inspection unit 8 includes, for example, a control unit 80, a drive mechanism 801, a measurement unit 802, and a scanner unit 803. The drive mechanism 801 is configured using, for example, a motor, a gear mechanism, etc. The drive mechanism 801 moves the inspection object 100 in three axial directions of X, Y, and Z, and rotates it around the Z axis, in response to a control signal from the control unit 80.

[0023] The driving mechanism 801 is not limited to one that moves and rotates the inspection object 100. The driving mechanism 801 may, for example, move and rotate the inspection jig 3 and the imaging unit 41, thereby relatively moving and rotating the inspection jig 3 and the inspection object 100. Alternatively, the driving mechanism 801 may move and rotate both the inspection jig 3 and the inspection object 100.

[0024] The scanner unit 803 is a switching circuit configured using switching elements such as semiconductor switches, relay switches, etc. The scanner unit 803 electrically connects a probe selected from the plurality of probes P to the measurement unit 802 in response to a control signal from the control unit 80.

[0025] The measurement unit 802 includes, for example, a power supply circuit, a voltmeter, an ammeter, etc. The measurement unit 802 supplies a current between a pair of probes selected by the scanner unit 803, measures the current flowing between the probes and the voltage generated between the probes, and outputs the measured values ​​to the control unit 80.

[0026] The control unit 80 is a so-called microcomputer that is configured with, for example, a CPU (Central Processing Unit) that executes predetermined arithmetic processing, a RAM (Random Access Memory) that temporarily stores data, a non-volatile storage device that stores predetermined control programs, etc., and peripheral circuits for these.

[0027] The control unit 80 executes, for example, a predetermined control program, thereby functioning as an image processing unit 81 (image processing device) and an inspection processing unit 82. The image processing unit 81 includes a binarization unit 811, a selection unit 812, a noise reduction unit 813, a pad area extraction unit 814, a virtual straight line setting unit 815, and a reference point setting unit 816.

[0028] The image processing unit 81 corresponds to an example of an image processing device. Note that the image processing unit 81 is not limited to being provided in the inspection device 1. The image processing unit 81 may be configured as an independent image processing device, or may be incorporated in a device other than the inspection device.

[0029] The binarization unit 811 performs binarization processing to binarize the image of the inspection object 100 captured by the imaging unit 41. The selection unit 812 performs selection processing to select images of multiple pads BP by pattern matching. The noise reduction unit 813 performs noise reduction processing to reduce noise near the center of the image of the pad BP captured by the imaging unit 41.

[0030] The pad area extraction unit 814 executes a pad area extraction process to extract a substantially rectangular pad area BA surrounding a plurality of pad rows L from the image of the inspection object 100 captured by the imaging unit 41. The virtual line setting unit 815 executes a virtual line setting process to set a virtual first line VL1 along a side of the pad area BA extending in the X direction (first direction), and to set a virtual second line VL2 along a side extending in the Y direction (second direction).

[0031] The reference point setting unit 816 executes a reference point setting process to set the intersection of the first line VL1 and the second line VL2 as the reference point RP.

[0032] Based on the reference point RP set by the reference point setting unit 816, the inspection processing unit 82 calculates the relative positional relationship and tilt between the inspection jig 3 and the object under test 100 from the position of the reference point RP in the image of the object under test 100 captured by the imaging unit 41. Then, the inspection processing unit 82 moves the object under test 100 by the driving mechanism 801 so that the probe P is brought into proper contact with each pad BP of the object under test 100, and positions the inspection jig 3 with respect to the object under test 100.

[0033] The inspection processing unit 82 positions the inspection object 100 relative to the inspection jig 3, and with each probe P in contact with each pad BP of the inspection object 100, causes the measuring unit 802 and the scanner unit 803 to measure the presence or absence of conductivity between the pads BP of the inspection object 100, resistance values, etc. via each probe P. The inspection processing unit 82 inspects the inspection object 100 based on the measurement results of the measuring unit 802.

[0034] Next, the operation of the image processing unit 81 that uses the image processing method according to one embodiment of the present invention will be described. First, with reference to Fig. 4, the noise reduction unit 813 reduces noise near the center of the image of the pad BP in the image G1 of the inspection object 100 captured by the imaging unit 41 (step S1: noise reduction process).

[0035] It should be noted that the image processing unit 81 does not necessarily have to process the entire image captured by the imaging unit 41. For example, from the entire image area captured by the imaging unit 41, an area where the pads BP to be inspected are expected to be located may be partially extracted as the image G1 to be processed.

[0036] Furthermore, it is not necessary to perform the noise reduction process (step S1). Furthermore, the noise reduction process (step S1) may be performed after the binarization process (step S2) and before the narrowing-down process (step S3). Furthermore, the noise reduction process (step S1) may be performed after the narrowing-down process (step S3) and before the selection processes 1 and 2 (steps S4 and S5). Furthermore, the noise reduction process (step S1) may be performed after the selection processes 1 and 2 (steps S4 and S5) and before the pad area extraction process (steps S6 to S12). Furthermore, the noise reduction process (step S1) may be performed after the pad area extraction process (steps S6 to S12) and before the virtual straight line setting process (steps S13 and S14). Alternatively, steps S4 and S5 may be performed after the virtual straight line setting process (steps S13 and S14) and before the reference point setting process (step S15).

[0037] The image G1 captured by the imaging unit 41 shows, by means of shading, the intensity of the reflected light that hits and is reflected from the inspection object 100. Therefore, even if each pad BP formed on the inspection object 100 actually has a circular shape as shown in Fig. 3, in the image G1 captured by the imaging unit 41, the outline of the pad BP does not necessarily have a neat circular shape as shown in Fig. 3. In Fig. 6 and Fig. 8 described later, the illumination light is indicated by a solid arrow, and the reflected light is indicated by a dashed arrow.

[0038] 6, the pad BP protrudes in a substantially dome-like shape from the substrate surface of the inspection object 100. Therefore, for example, when illumination light is applied parallel to the optical axis of the imaging unit 41, i.e., perpendicular to the substrate surface of the inspection object 100, the light reflected from the substrate surface is received as is by the imaging unit 41, while the light that strikes the side surface of the pad BP is reflected obliquely due to the inclination of the slope of the pad BP. As a result, in the image G1 captured by the imaging unit 41, as shown in FIG. 7, the reflected light from the substrate surface of the inspection object 100 is strong, and the image of the substrate surface appears white.

[0039] On the other hand, the light that strikes the side surface of the pad BP is not received by the imaging unit 41, or only a portion of the light that is diffusely reflected is received by the imaging unit 41. As a result, the image of the side surface of the pad BP appears black or gray. In FIG. 7, the black or gray portions of the image are indicated by hatching. Furthermore, near the top of the pad BP, the illumination light strikes the surface of the pad BP approximately perpendicularly, so the illumination light is strongly reflected from the top of the pad BP, and the image near the top of the pad BP appears white in image G1. For ease of explanation, the following description will be given using an example in which the image of the pad BP is black or gray and the image around the pad BP is white, as shown in FIG. 7.

[0040] As shown in Figure 7, when the image near the top of a black pad BP appears white due to reflected light, the white portion of the image near the top becomes noise. Therefore, an image of a pad BP with such noise may reduce the accuracy of recognizing it as a circular pad BP in the sorting process described below. Therefore, in step S1, by reducing the noise caused by strong reflection near the center of the image of the pad BP, the recognition accuracy of the pad BP in the sorting process described below and the calculation accuracy of the center coordinates of the pad BP in the virtual straight line setting process can be improved.

[0041] As a method for reducing noise, various image processing methods for noise reduction can be used. For example, a method can be used in which white areas caused by strong reflection are extracted from image G1, the extracted white areas are binarized, the binarized white areas are expanded, the expanded white areas are filled with a predetermined gray color, and then a smoothing filter is applied to the entire image G1.

[0042] In addition, if the image G1 is an image in which the substrate surface is black and the pad BP is white or gray, as shown in Figure 9, the noise reduction processing of step S1 does not need to be performed, and the image processing unit 81 does not need to be equipped with the noise reduction unit 813.

[0043] Next, the binarization unit 811 generates an image G2 by binarizing the image G1 whose noise has been reduced in step S1 (step S2: binarization process).

[0044] For example, if the image shows a pad BP in black or gray and the area around the pad BP in white, then by binarizing the image G1 with an appropriate threshold, the gray area of ​​the pad BP can be made black, thereby creating a clear contrast with the white substrate surface around the pad BP. On the other hand, if the image shows a pad BP in white or gray and the area around the pad BP in black, then by binarizing the image G1 with an appropriate threshold, then it is possible to create a clear contrast with the black substrate surface around the pad BP.

[0045] By making the contrast between the pad BP and the substrate surface surrounding the pad BP clearer, the recognition accuracy of the pad BP in the sorting process described below and the calculation accuracy of the center coordinates of the pad BP in the virtual straight line setting process are improved, and ultimately the setting accuracy of the reference point in the reference point setting process described below is improved.

[0046] It is not necessary to execute the binarization process (step S2). Alternatively, the binarization process (step S2) may be executed after the narrowing-down process (step S3) and before the selection processes 1 and 2 (steps S4 and S5). Alternatively, the binarization process (step S2) may be executed after the selection processes 1 and 2 (steps S4 and S5) and before the pad region extraction process (steps S6 to S12). Alternatively, the binarization process (step S2) may be executed after the pad region extraction process (steps S6 to S12) and before the virtual line setting process (steps S13 and S14). Alternatively, the binarization process (step S2) may be executed after the virtual line setting process (steps S13 and S14) and before the reference point setting process (step S15).

[0047] Next, as a pre-processing step for the selection process, the selection unit 812 narrows down the search area in which to search for the image of the pad BP for pattern matching (step S3: narrowing down process). To narrow down the search area, various image processes such as binarization of the pad BP image and morphology process can be used. This narrows down the area in which pattern matching is performed, making it possible to prevent erroneous recognition in pattern matching and shorten the time required for the selection process. Note that step S3 may be omitted.

[0048] Next, the selection unit 812 selects images of a plurality of pads BP from within the area of ​​the image G2 narrowed down in step S3 by pattern matching (step S4: selection process 1).

[0049] The surface of the pad BP may be uneven or deformed. Also, if the pad BP has been in contact with a probe P in the past, there may be probe marks on the surface of the pad BP. In such cases, the image of the pad BP in image G2 may not be circular, and may have a distorted shape, as shown in FIG. 11, for example.

[0050] Therefore, the selection unit 812 selects an image of the pad BP that has a good image shape by pattern matching with a preset reference image of the pad BP, for example, a circular image. The image of the pad BP selected by the selection process is recognized by the image processing unit 81 as the image of the pad BP to be used for setting the reference point.

[0051] Next, the selection unit 812 selects images that appear to be pads from the images of the pads BP selected in the selection process 1 of step S4, using the pad radius, roundness, and area value (step S5: selection process 2).

[0052] In the sorting by pattern matching in step S4, there is a risk that pads that are not to be inspected and that are different in size from the pads BP may be mistakenly selected. Therefore, by performing sorting process 2 in step S5 in addition to sorting process 1 in step S4 as the sorting process, it is possible to improve the sorting accuracy of the pads BP.

[0053] 10, the selected pads BP are indicated by hatching. According to the selection processes 1 and 2 in steps S4 and S5, pads BP that are not suitable for setting a reference point due to their shape being deformed, pads that are not to be inspected, patterns, and other dirt and noise can be excluded from the pads BP used for setting a reference point. As a result, the accuracy of calculating the center coordinates of the pad BP in the virtual straight line setting process described later is improved, and ultimately the accuracy of setting a reference point in the reference point setting process described later is improved.

[0054] It is not necessary to execute the selection processes 1 and 2 in steps S4 and S5 as the selection process, and the selection process may be only the selection process 1 in step S4. Also, steps S4 and S5 do not have to be executed. Also, the selection process (steps S4 and S5) may be executed after the pad area extraction process (steps S6 to S12) and before the virtual straight line setting process (steps S13 and S14). Alternatively, the selection process (steps S4 and S5) may be executed after the virtual straight line setting process (steps S13 and S14) and before the reference point setting process (step S15).

[0055] Next, the pad area extraction unit 814 sets one or more substantially rectangular pad areas BA surrounding the images of the pads BP selected in the selection process from the image G2 (step S6: pad area extraction process (A1)). For example, the pad area extraction unit 814 executes a closing process and a filling process on the images of the pads BP selected in the selection process, thereby setting a pad area BA surrounded by a boundary line as shown in FIG.

[0056] Next, the pad area extraction unit 814 sets a part of the area in the image G2 as a presence check area CA (step S7: pad area extraction process (A2)). In the example shown in Fig. 13, a rectangular presence check area CA is set near the center of the image G2. In Fig. 13, two pad areas BA are indicated by dashed lines.

[0057] Next, the pad area extraction section 814 finds the geometric center CP of each of one or more pad areas BA (step S8: pad area extraction process (A3)).

[0058] Next, among one or more pad areas BA, a pad area BA whose geometric center CP is located within the presence check area CA is selected as a processing target (step S9: pad area extraction process (A4)). In the example shown in Figure 13, the geometric centers CP of two pad areas BA are both located within the presence check area CA, so two pad areas BA are selected as processing targets.

[0059] According to step S9, by setting the existence check area CA in advance so as to include the geometric center CP of the pad area BA to be processed, it becomes easy to select the intended pad area BA as the processing object.

[0060] 5, the pad area extraction unit 814 excludes from the processing target, among one or more pad areas BA, a pad area BA whose geometric center CP is located outside the existence check area CA (step S11: pad area extraction process (A5)). As a result, even if an unintended pad area BA is selected as the processing target in step S9, the unintended pad area BA can be excluded from the processing target.

[0061] Next, the pad area extraction unit 814 groups the multiple pad areas BA to be processed into a single pad area BA (step S12: pad area extraction process (A6)). As shown in Fig. 14, the multiple pad areas BA are grouped into a single pad area BA including an interval portion. As a method for grouping the multiple pad areas BA into a single pad area BA, for example, morphology processing or a method for grouping by approximating with a preset rectangular shape can be used.

[0062] Before grouping, each pad area BA may be subjected to a process to remove noise such as dirt that is not a pad. Various image processing methods can be used to remove noise from each pad area BA. For example, a method can be used in which noise attached to the pad area BA in the X or Y direction is removed by repeating a closing process, an opening process in the X direction, and an opening process in the Y direction.

[0063] In addition, the pad area extraction unit 814 is not necessarily limited to performing steps S7 to S12 as a pad area extraction process as long as it can extract an approximately rectangular pad area BA surrounding multiple pad rows L from the image of the inspection object 100 captured by the imaging unit 41.

[0064] Next, the imaginary line setting unit 815 selects from among the pads BP located at least near the outer edge of the grouped pad area BA (step S13: imaginary line setting process (B1)).

[0065] 15, first, the virtual line setting unit 815 selects, as peripheral pads, images of pads BP that contact the boundary line of the grouped pad area BA from the images of pads BP selected in steps S4 and S5 (step S21). As a result, multiple pads BP located on the outer edge of the grouped pad area BA are selected, as shown in FIG.

[0066] Referring to FIG. 17, next, the virtual line setting unit 815 sets an area surrounding the peripheral pads as a peripheral area AA, and sets an expanded area EA by expanding the peripheral area AA in the X direction by two rows of pads (step S22).

[0067] 18, the X direction is the direction along the pad row L, and the pads BP are lined up in a linear row in the X direction. However, between adjacent pad rows L, the positions of the pads BP are shifted in the X direction. Therefore, the peripheral area AA is expanded in the X direction to a width that can accommodate two pads BP.

[0068] The pads BP are not limited to being positioned with a shift in the X direction between adjacent pad rows L, and the peripheral area AA does not have to be expanded in the X direction by the amount of two pad rows. The virtual straight line setting unit 815 may expand the peripheral area AA in the Y direction, or in both the X and Y directions.

[0069] Furthermore, the number of rows in which the peripheral area AA is expanded is not limited to two, and any number of rows may be expanded. Although the example in which the expansion area EA is set to include the peripheral pads has been shown, the expansion area EA does not have to include the peripheral pads. The expansion area EA may also be set in the pad area BA that is more inward than the peripheral pads.

[0070] As a result, in the virtual straight line setting process (B1), it is possible to select from among a plurality of rows of pads BP for at least one of the rows of pads aligned in the X direction and the rows of pads aligned in the Y direction.

[0071] Next, the virtual line setting unit 815 applies pattern matching to the image of the pad BP in the expansion area EA against a preset reference image of the pad BP, and obtains pad candidates for determining the virtual line (step S23).

[0072] Next, the virtual straight line setting unit 815 narrows down the pad candidates obtained in step S23 by filtering them by brightness value (step S24). As a result, pad candidates such as those shown in Fig. 19 are obtained. Even if a stain or the like having a shape similar to that of a pad BP is erroneously recognized by the pattern matching in step S23, further filtering by brightness value makes it easy to eliminate the erroneous recognition due to pattern matching.

[0073] Next, the virtual straight line setting unit 815 narrows down the pad candidates again by pattern matching the images of the pad BP of the narrowed down pad candidates using parameters different from those in step S23 (step S25). This makes it easy to reduce the risk that an incorrect image of the pad BP will be included in the pad candidates due to erroneous recognition.

[0074] Next, the virtual straight line setting unit 815 performs morphology processing on the image of the pad BP of the pad candidate narrowed down in step S25 (step S26). This makes it easy to reduce the risk that, for example, a stain or the like of a size similar to that of the pad BP will be included in the pad candidate due to erroneous recognition.

[0075] In the virtual line setting process (B1), it is preferable that the final number of pad candidates lined up in the X direction and the final number of pad candidates lined up in the Y direction are both 3 or more. This means that in the virtual line setting process (B2), the number of pads used to set the first line VL1 is 3 or more, and the number of pads used to set the second line VL2 is 3 or more. As a result, it becomes easier to improve the setting accuracy of the first line VL1 and the second line VL2.

[0076] Next, the imaginary line setting unit 815 sets a first line VL1 and a second line VL2 based on the image of the pad BP that has been selected in steps S21 to S26 and is set as a pad candidate (step S14: imaginary line setting process (B2)).

[0077] 20, the virtual line setting unit 815 sets and expands four straight line areas LA connecting the four vertices PP (see FIG. 14) of the pad area BA (step S31: FIG. 21). In the virtual line setting process (B1), if a pad candidate is selected from the expansion area EA set in the pad area BA inside the peripheral pads, the width of the four straight line areas LA is expanded so that the width overlaps with the area where the pad candidate is located.

[0078] Next, the virtual line setting unit 815 selects, as boundary pads, pad candidates that intersect with the four straight line regions LA from among the pad candidates selected in the virtual line setting process (B1) in step S13 (step S32). Fig. 22 shows only images of pads BP selected as boundary pads based on the straight line regions LA shown in Fig. 21.

[0079] Next, the virtual line setting unit 815 calculates the center coordinate HP of each boundary pad by elliptical approximation of the images of the pads BP selected as boundary pads in step S32 (step S33). By calculating the center coordinate HP using elliptical approximation, it becomes easy to calculate the center coordinate HP of each boundary pad even if the image of the pad BP is slightly deformed from a perfect circle.

[0080] Next, the virtual line setting unit 815 obtains a pair of first lines VL1 and a pair of second lines VL2 by calculating virtual lines from the center coordinates HP of the plurality of boundary pads (step S34: FIG. 23). As an example of a method for calculating virtual lines from the center coordinates HP of the plurality of boundary pads, for example, a line passing through each center coordinate HP may be used as the virtual line, or the virtual line may be calculated by linearly approximating each center coordinate HP, or the virtual line may be calculated by other methods.

[0081] The virtual line setting unit 815 only needs to be able to determine the first line VL1 and the second line VL2 extending along the boundary pad, and the method for determining the first line VL1 and the second line VL2 is not limited. Furthermore, the virtual line setting unit 815 does not necessarily need to execute steps S13 and S14 as long as it can set the virtual first line VL1 along a side of the pad area BA extending in the X direction and set the virtual second line VL2 along a side of the pad area BA extending in the Y direction, as the virtual line setting process. Furthermore, the first line VL1 and the second line VL2 do not necessarily need to be along the side of the pad area BA. It is sufficient that the first line VL1 extends parallel to a side of the pad area BA extending in the X direction, and the second line VL2 extends parallel to a side of the pad area BA extending in the Y direction.

[0082] Next, the virtual line setting unit 815 sets the intersections of the first line VL1 and the second line VL2 as reference points RP (step S15: reference point setting process), as shown in Fig. 23. According to the example shown in Fig. 23, four reference points RP can be set at four intersections where the pair of first lines VL1 and the pair of second lines VL2 intersect, as shown in Fig. 24.

[0083] As a result, as described above, the inspection processing unit 82 is able to position the inspection jig 3 relative to the object to be inspected 100 based on the reference point RP set by the reference point setting unit 816, and inspect the object to be inspected 100.

[0084] As described above, by processing steps S1 to S15, a reference point RP can be set based on an image of the pad BP to be inspected for the inspection object 100 having a high density pad arrangement, thereby facilitating positioning of the inspection object having a high density pad arrangement.

[0085] That is, an image processing device according to one example of the present invention is an image processing device that sets a positioning reference point for an object to be inspected in which pad rows, in which a plurality of pads are lined up along a first direction, are formed in multiple rows in a second direction that intersects the first direction, and is equipped with a pad area extraction unit that executes a pad area extraction process to extract an approximately rectangular pad area surrounding the plurality of pads from an image of the object to be inspected, a virtual line setting unit that executes a virtual line setting process to set a virtual first line parallel to a side of the pad area extending in the first direction and to set a virtual second line parallel to a side extending in the second direction, and a reference point setting unit that executes a reference point setting process to set the intersection of the first line and the second line as the reference point.

[0086] Moreover, an inspection apparatus according to an embodiment of the present invention positions the inspection object using the reference point set by the image processing apparatus described above.

[0087] Moreover, an image processing method according to one example of the present invention is an image processing method for setting a positioning reference point for an object to be inspected, in which pad rows, in which a plurality of pads are lined up along a first direction, are formed in multiple rows in a second direction intersecting the first direction, and includes a pad area extraction process for extracting an approximately rectangular pad area surrounding the multiple pad rows from an image of the object to be inspected, a virtual line setting process for setting a virtual first line parallel to a side of the pad area extending in the first direction and a virtual second line parallel to a side extending in the second direction, and a reference point setting process for setting the intersection of the first line and the second line as the reference point.

[0088] According to these configurations, the reference points for positioning can be set based on images of the pads provided on the substrate, making it easy to position a substrate that does not have a positioning mark.

[0089] Furthermore, it is preferable that the virtual straight line setting process includes (B1) a step of selecting from among a plurality of pads located at least near the outer edge of the pad area, and (B2) a step of setting the first straight line and the second straight line based on the pads selected in the (B1) step.

[0090] According to this configuration, a first line parallel to a side of the pad area extending in a first direction and a second line parallel to a side of the pad area extending in a second direction are set based on an image of a pad selected from a plurality of pads located near the outer edge of the pad area, thereby making it easy to set the first line and the second line.

[0091] In addition, in the virtual line setting process, it is preferable that the number of pads used to set the first line is three or more, and the number of pads used to set the second line is three or more.

[0092] According to this configuration, the first line and the second line can be set from three or more pad images each, which makes it easy to improve the setting accuracy of the first line and the second line.

[0093] Furthermore, the pad area may be one or more, and the pad area extraction process may include the steps of (A1) extracting one or more of the pad areas from the image of the substrate, (A2) setting a portion of the image as an existence check area, (A3) determining the geometric center of each of the one or more pad areas, and (A4) selecting, from the one or more pad areas, a pad area whose geometric center is located within the existence check area as a processing target, and it is preferable that the virtual straight line setting process sets the first straight line and the second straight line for the pad area selected as the processing target.

[0094] According to this configuration, the pad region to be processed can be easily extracted from one or more pad regions provided on the substrate.

[0095] Furthermore, the pad area extraction process preferably includes the step (A5) of excluding from the processing target, among the one or more pad areas, a pad area whose geometric center is located outside the presence check area.

[0096] According to this configuration, pad areas located outside the presence check area can be excluded from the processing target, making it easy to reduce the amount of processing.

[0097] It is also preferable that the device further includes a binarization unit that performs a binarization process to binarize the image of the substrate, and that at least one of the pad area extraction process, the virtual straight line setting process, and the reference point setting process is performed based on the binarized image.

[0098] According to this configuration, by binarizing the image of the board, the contrast of the image can be increased, making it easier to perform at least one of the image processing steps of pad area extraction processing, virtual straight line setting processing, and reference point setting processing.

[0099] It is also preferable that the device further includes a noise reduction unit that performs noise reduction processing to reduce noise near the center of the image of the pad, and that at least one of the pad area extraction processing, the virtual straight line setting processing, and the reference point setting processing is performed based on the image of the pad with the noise reduced.

[0100] This configuration reduces noise caused by illumination light reflected near the top of the pad, making it easier to perform at least one of the image processes: pad area extraction processing, virtual straight line setting processing, and reference point setting processing.

[0101] It is also preferable that the device further includes a selection unit that performs a selection process to select images of the multiple pads by pattern matching, and that at least one of the pad area extraction process, the virtual straight line setting process, and the reference point setting process is performed based on the images of the pads selected in the selection process.

[0102] According to this configuration, at least one of the pad area extraction process, virtual straight line setting process, and reference point setting process is performed based on the image of the pad selected by pattern matching, making it easy to improve the processing accuracy of at least one of the pad area extraction process, virtual straight line setting process, and reference point setting process.

[0103] In addition, it is preferable that in the step (B2), the first line and the second line are set by determining a virtual line based on the pads selected in the step (B1).

[0104] This configuration makes it easy to improve the accuracy of setting the first straight line and the second straight line.

[0105] In addition, in the step (B1), it is preferable to select at least one of the pad row and the row of pads aligned in the second direction from among the pads in a plurality of rows.

[0106] With this configuration, at least one of the first straight line and the second straight line can be set based on pads selected from multiple rows of pads. Therefore, even if there are pads that are difficult to image recognize, it is easy to set at least one of the first straight line and the second straight line from the remaining pads. [Explanation of symbols]

[0107] 1. Inspection equipment 3 Inspection jig 4. Inspection Organization 6 Board fixing device 8. Inspection Department 41 Imaging unit 80 Control Unit 81 Image processing unit (image processing device) 82 Inspection processing section 100 boards 801 Drive mechanism 802 Measuring part 803 Scanner unit 811 Binarization section 812 Sorting Department 813 Noise reduction section 814 Pad area extraction unit 815 Virtual straight line setting section 816 Reference point setting section AA surrounding area BA pad area BP Pad CA existence check area CP geometric center EA Expansion Area G1, G2 images HP center coordinates L Pad Row LA linear area P probe PP Vertex RP reference point VL1 first straight line VL2 Second straight line

Claims

1. An image processing device that sets a reference point for positioning an object to be inspected, in which a plurality of pad rows, each of which is formed by lining up a plurality of pads along a first direction, are arranged in a second direction intersecting the first direction, a pad area extraction unit that executes a pad area extraction process to extract a substantially rectangular pad area surrounding the plurality of pads from the image of the inspection object; a virtual line setting unit that executes a virtual line setting process to set a virtual first line parallel to a side extending in the first direction and to set a virtual second line parallel to a side extending in the second direction, based on a plurality of the pads located near an outer edge of the pad area; a reference point setting unit that executes a reference point setting process to set an intersection point between the first line and the second line as the reference point, The virtual straight line setting unit selects from among a plurality of pads located at least near the outer edge of the pad area, and sets the first straight line and the second straight line based on the selected pads.

2. 2. The image processing device according to claim 1, wherein in the virtual line setting process, the number of pads used to set the first line is three or more, and the number of pads used to set the second line is three or more.

3. The pad area may be one or more. The pad area extraction process includes: (A1) extracting one or more pad regions from an image of the inspection object; (A2) setting a partial area within the image as a presence check area; (A3) determining the geometric center of each of the one or more pad regions; (A4) selecting, as a processing target, a pad area whose geometric center is located within the presence check area from the one or more pad areas; 3. The image processing apparatus according to claim 1, wherein the virtual line setting process sets the first line and the second line for the pad area selected as the processing target.

4. The pad area extraction process includes: (A5) The image processing apparatus according to claim 3, further comprising a step of excluding from the processing target, of the one or more pad areas, a pad area whose geometric center is located outside the presence check area.

5. a binarization unit that performs binarization processing to binarize the image of the inspection object, 5. The image processing device according to claim 1, wherein at least one of the pad area extraction process, the virtual straight line setting process, and the reference point setting process is executed based on the binarized image.

6. a noise reduction unit that performs noise reduction processing to reduce noise near the center of the image of the pad; An image processing device according to any one of claims 1 to 5, wherein at least one of the pad area extraction process, the virtual straight line setting process, and the reference point setting process is performed based on an image of the pad in which the noise has been reduced.

7. a sorting unit that performs a sorting process to sort the images of the plurality of pads by pattern matching; An image processing device according to any one of claims 1 to 5, wherein at least one of the pad area extraction process, the virtual straight line setting process, and the reference point setting process is performed based on an image of the pad selected in the selection process.

8. An inspection device that positions the object to be inspected using the reference point set by the image processing device according to any one of claims 1 to 7.

9. 1. An image processing method for setting a reference point for positioning an object to be inspected, in which a plurality of pad rows, each of which is a row of pads arranged along a first direction, are arranged in a second direction intersecting the first direction, the method comprising: a pad area extraction process for extracting a substantially rectangular pad area surrounding the plurality of pad rows from the image of the inspection object; a virtual line setting process for setting a virtual first line parallel to a side extending in the first direction and a virtual second line parallel to a side extending in the second direction based on a plurality of the pads located near an outer edge of the pad area; a reference point setting process for setting an intersection point between the first straight line and the second straight line as the reference point; The image processing method includes selecting from a plurality of pads located at least near the outer edge of the pad area, and setting the first line and the second line based on the selected pads.

Images