Visual inspection method, inspection range designation method, and program

Abstract

To provide a visual inspection method with which it is possible to carry out solder failure and defect by an image measuring instrument.SOLUTION: A visual inspection method detects solder bumps formed on pads of an inspection object on the basis of the image of the inspection object. The visual inspection method includes: an inspection range designation step for designating an inspection range in the image of the inspection object; a pad region detection step for detecting a pad region included in the inspection range; a bump region detection step for detecting a bump region included in the inspection range; and a defect determination step for determining the presence or absence of a defect on the basis of the detected pad region and / or bump region.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to an appearance inspection method and a program for inspecting the appearance of a measurement object based on an image obtained by capturing an image of the measurement object. [Background technology]

[0002] A vision measuring machine is a device that captures an image of a measurement target (hereinafter referred to as a "workpiece"), analyzes the image, extracts a point cloud of edges contained in the image, and evaluates the distance, inclination, diameter, width, etc. of geometric shapes such as lines, circles, polygons, etc. that are approximated from the extracted point cloud of edges. In addition to evaluating such geometric shapes, modern vision measuring machines are equipped with algorithms that detect defects such as dirt on the workpiece, foreign matter inside the hole shape, minute chips, deformations, burrs, dirt, etc. on the workpiece, enabling image-based defect inspection (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-071106 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, the demand for solder defect inspection on printed circuit boards and other devices has increased due to factors such as rising demand for semiconductors, but an algorithm for inspecting solder defects using an image measuring machine has not yet been developed.

[0005] In view of the above problems, the present invention aims to provide an appearance inspection method that can perform solder defect inspection using an image measuring device, and a program for realizing such an appearance inspection method. [Means for solving the problem]

[0006] In order to solve the above problems, the visual inspection method of the present invention is a visual inspection method for inspecting solder bumps formed on pads of an object to be inspected based on an image of the object to be inspected, and is characterized by comprising an inspection range designation step for designating an inspection range in the image of the object to be inspected, a pad area detection step for detecting pad areas included in the inspection range, a bump area detection step for detecting bump areas included in the inspection range, and a defect determination step for determining the presence or absence of defects based on the detected pad areas and / or bump areas.

[0007] In the present invention, in the inspection range designation step, multiple inspection ranges may be designated in the image of the object to be inspected, and the pad area detection step, bump area detection step, and defect determination step may be performed for each of the designated multiple inspection ranges.

[0008] In the present invention, the inspection range designation step may designate the inspection range based on predetermined inspection range information. The inspection range information may include at least information indicating the position of the inspection range. The inspection range information may further include information indicating the size of the inspection range. The inspection range information may further include information regarding the determination conditions used for determination in the defect determination step.

[0009] The visual inspection method according to the present invention may further include a step of extracting a chip image of the inspection range from the image of the object to be inspected. In this case, the pad area detection step may include a step of binarizing the chip image, a step of detecting edges in the binarized chip image, and a step of acquiring the outermost edge of the detected edges as the outline of the pad area. Also, the bump area detection step may include a step of inverting the binarized brightness values ​​of the inside of the outline of the pad area for the binarized chip image, a step of detecting edges in the chip image in which the brightness values ​​of the inside of the outline of the pad area are inverted, and a step of acquiring the outermost edge of the detected edges as the outline of the bump area.

[0010] In the present invention, the defects whose presence or absence is determined in the defect determination step may include one or more of missing bumps, out-of-design range, and misalignment.

[0011] A program according to the present invention is characterized by causing a computer to execute any one of the above-described visual inspection methods.

[0012] To solve the above problems, an inspection area designation method according to the present invention inspects solder bumps formed on pads of an object to be inspected based on an image of the object to be inspected, and is characterized by comprising: a bump area detection step of detecting one or more bump areas included in the image; and an inspection area designation step of designating an inspection area for each of the detected one or more bump areas.

[0013] The inspection range designation method according to the present invention may further include an inspection range information saving step of saving inspection range information including at least information indicating the position of the inspection range for the inspection range designated in the inspection range designation step.

[0014] A program according to the present invention is characterized by causing a computer to execute any one of the above-described inspection range designation methods. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a perspective view showing an example of the configuration of an image measuring instrument 1. [Figure 2] FIG. 2 is a schematic diagram showing the configuration of an imaging unit 120 together with a stage 100. [Figure 3] FIG. 2 is a block diagram showing the configuration of a position acquisition means 110. [Figure 4] FIG. 2 is a block diagram showing the configuration of a computer main body 141. [Figure 5] FIG. 10 is a diagram illustrating an example of a screen display. [Figure 6] FIG. 10 is a diagram showing an example of an appearance inspection screen. [Figure 7] 10 is a flowchart illustrating an example of a procedure for visual inspection. [Figure 8] 10 is a flowchart illustrating a subroutine of an appearance inspection. [Figure 9] 10 is a flowchart illustrating a subroutine of an appearance inspection. [Figure 10] FIG. 10 is a diagram showing the transition of images in the visual inspection process. [Figure 11] 10 is a flowchart illustrating a procedure for automatically specifying an inspection range. [Figure 12] 10 is a flowchart illustrating a subroutine for automatically specifying an inspection range. [Figure 13] FIG. 10 is a diagram showing image transitions during processing for automatically specifying an inspection range. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Fig. 1 is a perspective view showing an example of the configuration of a vision measuring instrument 1. The vision measuring instrument 1 includes a stage 100, a position acquisition means 110, an imaging unit 120, a remote box 130, and a computer system 140.

[0017] The stage 100 is placed so that its upper surface is horizontal, and a workpiece (object to be measured or inspected) W is placed on this upper surface. At least the portion of the upper surface of the stage 100 on which the workpiece W is placed is made of a light-transmitting material such as glass. The stage 100 is driven by an X-axis drive motor and a Y-axis drive motor (not shown), and is movable in the X-axis and Y-axis directions parallel to the horizontal surface. Drive control signals for the drive motors of each axis are given to the drive motors of each axis from a remote box 130 or a computer system 140 (described later).

[0018] FIG. 2 is a schematic diagram showing the configuration of the imaging unit 120 together with the stage 100. The imaging unit 120 includes an optical system 122, an imaging means 124, and a light source 126. The optical system 122 is, for example, a telecentric optical system formed by combining multiple lenses and apertures. In a telecentric optical system, the chief ray is considered to be parallel, so the dimensions of the captured image do not depend on the position in the Z-axis direction (height direction). This makes it suitable for measuring workpieces W that have undulations (e.g., steps, holes, etc.). When capturing an image of the workpiece W, the light source 126 irradiates light onto at least the portion of the workpiece W to be imaged under the control of the computer system 140. In this embodiment, the light source 126 includes a light source 126a for epi-illumination that irradiates the workpiece W from above (i.e., the imaging means 124 side) via the optical system 122, and a light source 126b for transmitted illumination that irradiates the workpiece W from below (i.e., the back side of the stage 100). The imaging means 124 is, for example, a two-dimensional image sensor such as a CCD or CMOS. An image of the workpiece W is formed on the light receiving surface of the imaging means 124 by the optical system 122. The imaging means 124 captures the formed image and outputs image data in a predetermined format. This image data includes information about the pixels that make up the image, as well as at least an index indicating the order in which the images were captured. The imaging unit 120 transmits the image signal output by the imaging means 124 to the computer system 140. The computer system 140 and the imaging unit 120 are connected via a general-purpose communication standard such as USB (Universal Serial Bus). In addition, the imaging unit 120 outputs a trigger signal to the latch means 118 at the timing when it completes capturing one image (one frame).

[0019] The imaging unit 120 is driven by a Z-axis drive motor (not shown) and is movable in the Z-axis direction (i.e., the direction perpendicular to the upper surface of the stage 100). Focus adjustment is performed by adjusting the position of the imaging unit 120 in the Z-axis direction. A drive control signal for the Z-axis drive motor is provided from a remote box 130 or a computer system 140, which will be described later.

[0020] 3 is a block diagram showing the configuration of the position acquisition means 110. The position acquisition means 110 includes an X-axis encoder 112, a Y-axis encoder 114, a Z-axis encoder 116, and a latch means 118.

[0021] The X-axis encoder 112 measures and outputs the position coordinate of the stage 100 in the X-axis direction. The Y-axis encoder 114 measures and outputs the position coordinate of the stage 100 in the Y-axis direction. The Z-axis encoder 116 measures and outputs the position coordinate of the imaging unit 120 in the Z-axis direction. Each encoder is equipped with a scale with graduations and a scale reading unit that reads the graduations on the scale. The scale is attached to the movable part of the stage 100 or the imaging unit 120 along each axis. Meanwhile, the scale reading unit is located on the non-moving part.

[0022] The latch means 118 includes a counter 118a and a buffer 118b. When a trigger signal (e.g., a pulse signal) is supplied from the outside, the counter 118a increments its count value by 1. The value of the counter 118a is reset appropriately based on instructions from the computer system 140. The buffer 118b has storage areas with multiple addresses, and when a trigger signal is supplied, it latches and stores the output value of the encoder for each axis in a storage area with an address corresponding to the count value of the counter 118a. The trigger signal may be supplied, for example, from the imaging means 124 at the timing when capturing of one image is completed. The position coordinates of each axis held by the latch means 118 are associated with address values ​​(i.e., count values) and are appropriately input into the computer system 140. The computer system 140 and the latch means 118 are connected via a general-purpose communication standard such as USB (Universal Serial Bus). The image data and the position coordinates are each separately input into the computer system 140, but since the image data is assigned an index indicating the order of capture and the position coordinates are assigned a count value indicating the order of capture, even if they are input into the computer system 140 asynchronously, they can be associated after input.

[0023] Returning to FIG. 1 , the remote box 130 is an operation means for setting the positions of the stage 100 and the imaging unit 120, and transmits drive control signals to the X-axis drive motor, Y-axis drive motor, and Z-axis drive motor via wired or wireless communication in response to operations by the operator. The remote box 130 includes a joystick 132 and a jog shuttle 134. The joystick 132 is an operation input means for setting the position of the stage 100, and the remote box 130 transmits drive control signals for moving the stage 100 in the X-axis and Y-axis directions in response to the tilt direction of the joystick 132. The jog shuttle 134 is an operation input means for setting the position of the imaging unit 120 in the Z-axis direction, and the remote box 130 transmits drive control signals for moving the imaging unit 120 in the Z-axis direction in response to the rotation direction, rotation amount, rotation speed, etc. of the jog shuttle 134.

[0024] The computer system 140 includes a computer main body 141, a keyboard 142, a mouse 143, and a display 144. Fig. 4 is a block diagram showing the configuration of the computer main body 141. The computer main body 141 includes a CPU 40 which serves as the center of control, a storage unit 41, a work memory 42, interfaces (indicated as "IF" in Fig. 4) 43 and 44, and a display control unit 45 which controls the display on the display 144.

[0025] Operator instruction information entered from a keyboard 142 or a mouse 143 is input to the CPU 40 via an interface 43. The interface 44 is connected to the imaging unit 120 and the stage 100, and supplies various control signals from the CPU 40 to the imaging unit 120 and the stage 100, and receives various status information and measurement results from the imaging unit 120 and the stage 100 and inputs them to the CPU 40.

[0026] The display control unit 45 displays the image captured by the imaging unit 120 on the display 144. In addition to the image captured by the imaging unit 120, the display control unit 45 also displays on the display 144 an interface for inputting control instructions to the image measuring instrument 1, an interface for tools for analyzing the captured image, and the like.

[0027] The work memory 42 provides a working area for various processes of the CPU 40. The storage unit 41 is configured with, for example, a hard disk drive, RAM, etc., and stores programs executed by the CPU 40, image data captured by the imaging unit 120, etc.

[0028] The CPU 40 controls the imaging unit 120, the X-axis drive motor, the Y-axis drive motor, the Z-axis drive motor, etc. based on various input information via each interface, instructions from the operator, a measurement definition program (part program) stored in the memory unit 41, etc., and performs various processes such as setting the movement path of the imaging unit 120, adjusting the movement speed and exposure time, adjusting the light intensity of the light source 126, capturing a two-dimensional image using the imaging unit 120, image stitching processing for stitching together multiple partial images, and analyzing the entire image obtained by capturing the image.

[0029] Measurements performed using the above-described image measuring instrument 1 will be described below.

[0030] [Basic Image Measurement] First, the operator operates the joystick 132 or controls the computer system 140 to move the stage 100 so that the workpiece W is within the imaging field of view. Then, the Z-axis position of the imaging unit 120 is adjusted so that the workpiece W is in focus. After the focus is set on the workpiece W, the imaging means 124 captures an image for measurement. At this time, the coordinates of the stage 100 output from the X-axis encoder 112 and the Y-axis encoder 114 along with the captured image are input into the computer system 140 and stored in the memory unit 41. Specifically, the imaging means 124 outputs a pulse that serves as a trigger signal to the latch means 118 when it completes capturing one image. The latch means 118 latches and holds the position coordinates of each axis at the rising transition of the pulse (i.e., approximately simultaneously with the completion of capturing the image). The computer system 140 captures the image signal from the imaging means 124 and also captures the position coordinates at the time the image was captured from the latch means 118, and stores them in association with each other.

[0031] The computer system 140 displays the obtained measurement image on the display 144 together with an interface of a measurement tool for analyzing the image. Fig. 5 is a diagram showing an example of a screen display. This screen display is displayed on the display 144 by a program (measurement application software) executed by the CPU 40 of the computer system 140.

[0032] As shown in Fig. 5, a main window MW is displayed on the display 144 when the program is executed. Furthermore, multiple windows (first window W1 to eighth window W8) are displayed within the main window MW. Menus and icons for various operations and settings are also displayed above the main window MW. Note that, although an example in which eight windows are displayed is shown in this embodiment, windows other than eight may be displayed as needed, and windows may be divided, merged, or omitted depending on their purpose. Furthermore, the layout of each window can be freely changed by the operator's operation.

[0033] The first window W1 displays the image WG of the workpiece W captured by the imaging unit 120. The operator can adjust the position of the image WG of the workpiece W displayed in the first window W1 by, for example, operating the mouse 143 or the joystick 132 of the remote box 130. The operator can also enlarge or reduce the image WG of the workpiece W by, for example, selecting an icon with the mouse 143.

[0034] The second window W2 displays icons of measurement tools that can be selected by the operator. The measurement tool icons are provided corresponding to the designation methods for designating measurement points from the image WG of the workpiece W. Specific examples of measurement tools include a straight line edge detection tool and a circular edge detection tool.

[0035] The third window W3 displays icons of functions that can be selected by the operator. A function icon is provided for each measurement method. For example, there are methods for measuring the coordinates of a point, measuring the length of a line, measuring a circle, measuring an ellipse, measuring a square hole, measuring a slot, measuring pitch, and measuring the tolerance between two lines. In accordance with the operator's selection, the computer system 140 measures dimensions such as the length of a line, the distance between lines, and the diameter of a circle, and evaluates deviations (irregularities) from an ideal geometric shape, such as straightness, roundness, and parallelism.

[0036] The fourth window W4 displays guidance showing the operating procedures for the measurement.

[0037] The fifth window W5 displays various sliders for controlling the lighting irradiated onto the workpiece W from the imaging unit 120. The operator can apply desired lighting to the workpiece W by operating these sliders.

[0038] The sixth window W6 displays the X and Y coordinate values ​​of the stage 100. The X and Y coordinate values ​​displayed in the sixth window W6 are the coordinates of the stage 100 in the X and Y directions relative to a predetermined origin.

[0039] The seventh window W7 displays the tolerance determination result, i.e., when a measurement method capable of determining tolerance is selected, the seventh window W7 displays the result.

[0040] The eighth window W8 displays the measurement results. That is, when a measurement method that obtains measurement results by a predetermined calculation is selected, the eighth window W8 displays the measurement results. Note that the details of the display of the tolerance judgment results in the seventh window W7 and the measurement results in the eighth window W8 are omitted from the illustration.

[0041] [Visual inspection for solder defects] In the image measuring instrument 1 of this embodiment, the program (measurement application software) executed by the CPU 40 of the computer system 140 provides the function of performing an appearance inspection focusing on soldering defects (hereinafter simply referred to as appearance inspection) in addition to the basic image measurement described above. In the following description, unless a specific processing entity is mentioned, it should be understood that the program executed by the CPU 40 of the computer system 140 is the entity.

[0042] In this embodiment, the soldering defect includes the following three types of defects. (1) Missing bump: A defect in which no bump candidate exists within the inspection range. (2) Outside the design range: A defect in which a bump candidate within the inspection range is outside the range of the specified design value. (3) Misalignment: A defect in which the distance from the bump candidate to the pad (the shortest distance between the outer edges) is less than a threshold.

[0043] The range of design values ​​used to determine whether or not a value is outside the design range, and the threshold value used to determine misalignment, are configured so that the user can change them as desired on the screen of the measurement application software.

[0044] 6 shows an example of a screen for visual inspection (hereinafter referred to as visual inspection screen). The visual inspection screen is composed of an image pane P1, a filmstrip pane P2, a measurement result display pane P3, and a control pane P4.

[0045] The image pane P1 is an area for displaying the image to be inspected. The image displayed in the image pane P1 is subjected to image processing and defect judgment under conditions set in the control pane P4.

[0046] By dragging the mouse over the image pane P1, you can specify a rectangular area whose diagonal is the line segment connecting the start and end points of the drag as the inspection area IR for which defect judgment will be performed on the displayed image. When you perform this dragging operation, an inspection area tool showing the outline of the specified rectangular area is displayed in the image pane P1, superimposed on the image. The inspection area tool can be selected by clicking it, and its size can be changed by dragging the handle that appears when selected. The area tool can also be deleted by pressing the DEL key on the keyboard while it is selected.

[0047] The filmstrip pane P2 is an area that displays images loaded for visual inspection in thumbnail format. When you double-click one of the images displayed in the filmstrip pane P2, that image is displayed in the image pane P1 and becomes the subject of image processing and defect judgment. In the initial state immediately after loading images, a specific image (for example, the first image when sorted by name or saved date and time) is selected in the filmstrip pane P2 and displayed in the image pane P1.

[0048] If an image processed and judged in the image pane P1 has a defect, the image in which the defect was found is marked with hatching H in the film strip pane P2, making it easy to distinguish from images in which no defect was found.

[0049] The measurement result display pane P3 is an area that displays a list of defect information if defects are found in the image displayed in the image pane P1. If there are multiple defects, information on all of the defects is displayed in a list format.

[0050] The control pane P4 is an area where a user interface is displayed for setting conditions for image processing and defect judgment to be performed on the image displayed in the image pane P1. The user interface provided in the control pane P4 allows the setting of image processing parameters (e.g., threshold value for binarization, whether or not to invert brightness values, etc.), defect judgment parameters (e.g., design values ​​for bumps, allowable minimum distance to pads, etc.), and the size of the inspection area tool. The user interface may be provided as a form for directly inputting numerical values, or as GUI (Graphical User Interface) controls such as slider bars and switches. Also provided in the control pane P4 are a button B1 for inputting a command to perform a visual inspection, a button B2 for inputting a command to automatically specify the inspection area, etc.

[0051] Next, the procedure for visual inspection will be explained with reference to the flowcharts shown in Figures 7 to 9 and the image transition diagram shown in Figure 10, taking as an example a case where the inspection range IR is specified by user operation on the screen of the measurement application software.

[0052] Prior to starting the visual inspection, the user selects a menu for performing the visual inspection in the program (measurement application software). In response, a visual inspection screen is displayed on the display 144. The program then prompts the user to specify one or more images to be subject to the visual inspection. When the user specifies an image file in response, the image file is loaded, and all the loaded images are displayed in the filmstrip pane P2, with the first image (FIG. 10(a)) being displayed in the image pane P1.

[0053] The image displayed in the image pane P1 is the subject of the visual inspection, and if the user wishes to visually inspect an image other than the first image, the user can double-click the desired image in the filmstrip pane P2 to have the desired image displayed in the image pane P1. In this way, the visual inspection begins with the image to be inspected displayed in the image pane P1.

[0054] When the visual inspection is started, the program first accepts various conditions set by the user (step S01). Specifically, the program accepts settings for image processing parameters (such as binarization thresholds), defect determination parameters (such as allowable defect width, height, and area), and the allowable number of defects. These settings can be changed at any time.

[0055] Next, the inspection range IR is specified by operating the mouse 143 or the joystick 132 of the remote box 130 on the image WG of the workpiece W displayed in the image pane P1 (step S02). Any method for specifying the target area may be used, but for example, the mouse 143 may be dragged on the image pane P1, and the inspection range IR may be defined as a rectangular area whose diagonal is the line segment connecting the start and end points of the drag. Within the specified inspection range IR, an inspection range tool is displayed as a rectangular frame in the image pane P1, superimposed on the image WG of the workpiece.

[0056] When the user specifies the inspection range IR, it is advisable to specify the inspection range IR so that it surrounds the pad PD where the bump BP is to be formed. For example, if the pad PD does not fit within the image WG, such as a pad PD that crosses the image WG horizontally, it is advisable to specify the periphery of the position on the pad PD where the bump BP is to be placed as the inspection range IR. Since the outer edge of the pad PD is necessary to determine misalignment, it is advisable to specify the inspection range IR so that at least a part of the outer edge of the pad PD is included in the inspection range IR.

[0057] Any number of inspection ranges IR can be specified for one image WG. The user specifies inspection ranges IR for the number of bumps BP that require inspection. The computer system 140 may store information (position and range) of the inspection ranges IR specified as described above in the memory unit 41. A method for using the information of the inspection ranges IR stored in the memory unit 41 in subsequent inspections will be described later.

[0058] Subsequently, when a command to execute a visual inspection is input via the user interface (step S03), a soldering defect judgment is performed for each inspection range IR specified as described above, according to the procedure described below.

[0059] First, an inspection range IR where a determination is to be made is cut out from the image WG and set as a rectangular small piece image SG (step S04; FIG. 10(b)). Then, the outline of the pad PD is obtained in the small piece image SG (step S05).

[0060] The outline of the pad PD in step S05 can be obtained, for example, by the subroutine (steps S11 to S12) shown in FIG. 8. That is, the cut-out small piece image SG is binarized (step S11; FIG. 10(c)). The threshold for this binarization is set so that the pad PD is black (value 0) and the bump BP and areas where neither the bump BP nor the pad PD are provided are white (value 1). When binarizing, processing such as noise removal may be performed as necessary. Next, edges (boundaries between black areas and white areas) are detected for the small piece image SG binarized in step S11, and the outermost edge among them is obtained as the outline (outer edge) of the pad PD (step S12; FIG. 10(d)).

[0061] Returning to FIG. 7, following step S05, the contour of the bump BP is obtained in the small piece image SG (step S06).

[0062] The contour of the bump BP in step S06 can be obtained, for example, by the subroutine (steps S13 to S14) shown in Fig. 9. That is, the binarized values ​​(i.e., black and white) of only the inside of the contour of the pad PD obtained in step S12 are inverted (step S13). As a result, as shown in Fig. 10(e), the edges that form the boundary between the inside and outside of the pad PD disappear from the small piece image SG. Next, for the small piece image SG that has been processed in step S13, edges are detected again, and the outermost contour among them is obtained and used as the contour (outer edge) of the bump BP (step S14; Fig. 10(f)).

[0063] Returning to FIG. 7, based on the outline of the pad PD and the outline of the bump BP acquired in steps S05 and S06, feature quantities related to the pad PD and the bump BP (for example, the area and aspect ratio of the bump BP, the distance between the pad PD and the bump BP, etc.) are calculated, and a soldering defect is determined (step S07). Specifically, if the outline of the bump BP is not acquired in step S05, it is determined to be a "missing bump." Furthermore, if the area surrounded by the outline of the bump BP acquired in step S06 is outside the range of the set design values, it is determined to be "outside the design range." Furthermore, if the distance (shortest distance) between the outline of the pad PD acquired in step S04 and the outline of the bump BP acquired in step S06 is equal to or less than a set threshold, it is determined to be a "misalignment." If neither of these conditions is met, it is determined that there is no soldering defect.

[0064] The results of the judgment are then displayed on the display 144 (step S08). Anything judged to be a defect is displayed in the measurement result pane P3, and may also be displayed in the image pane P1, overlaid on the workpiece image WG. For example, the manner in which defects are displayed in the image pane P1 is arbitrary. For example, the color of the outline of an inspection range IR judged to have a defect may be changed, or the outline of a bump judged to have a defect may be filled in with a conspicuous color (e.g., red), or other methods may be used to display the detected defects so that the user can easily recognize them. The example in FIG. 6 shows an example in which the outline of an inspection range IR judged to have a defect is displayed in a thick color.

[0065] If there are any uninspected inspection areas IR remaining (step S09; Yes), the process returns to step S04, and steps S04 to S08 are performed on all inspection areas IR specified in step S02. If there are no uninspected inspection areas IR (step S09; No), the appearance inspection ends.

[0066] In this way, inspection for solder defects can be performed using an image measuring device, making it possible to measure and inspect soldered parts using a single system.

[0067] [Saving and reusing inspection range information] In the above example, the inspection range IR is specified by user operation on the screen of the measurement application software. However, the inspection range IR may also be specified based on pre-saved inspection range information. The inspection range information includes at least the position of each inspection range IR (e.g., center position coordinates) for a desired number of inspection ranges IR, and is stored in the memory unit 41. The position of each inspection range IR included in the inspection range information may be expressed as absolute coordinates in the coordinate system of the image measuring device 1, but is preferably expressed as relative coordinates based on a reference position within the workpiece W. The inspection range information may also include the size of each inspection range IR (e.g., the height and width of a rectangle). The inspection range information may store information about the inspection range IR specified by user operation on the screen of the measurement application software, or may store information about the inspection range IR specified by automatic specification, as described below. Alternatively, the inspection range information may be edited by the user and saved.

[0068] To apply the inspection range information to the visual inspection, in step S02 of the visual inspection procedure described above, instead of specifying individual inspection ranges IR by user operation on the screen of the measurement application software, the inspection range information stored in the storage unit 41 is read and applied to the image to be inspected. If the inspection range information contains only information about the position of each inspection range IR but not about its size, it is preferable to specify an inspection range IR of a predetermined size (for example, a size set by the user in the measurement application software). This makes it possible to specify multiple inspection ranges IR at once, without having to specify each individual inspection range IR manually, which is effective when repeating visual inspections of workpieces W with the same specifications. Once the inspection ranges IR have been specified, the presence or absence of defects can be determined for each specified inspection range IR in the same manner as in the visual inspection procedure described above.

[0069] In the inspection range information, the conditions for the visual inspection may be further linked to each inspection range IR. For example, the range of design values ​​and threshold values ​​used for judgment may be linked to each inspection range IR, and in the visual inspection, the presence or absence of defects may be judged using the range of design values ​​and threshold values ​​specified in the inspection range information. In addition, the type of judgment to be applied may be linked to each inspection range IR (for example, judging only "missing bump"). In this way, the presence or absence of defects can be judged for multiple inspection ranges IR under mutually different conditions.

[0070] [Automatic specification of inspection range] In the above example, the inspection range IR was specified by user operation on the screen of the measurement application software, but the inspection range IR may also be automatically generated for the image WG of the workpiece W by image processing (automatic specification of the inspection range). This automatic specification of the inspection range may be performed on a workpiece (reference workpiece) for which it has been confirmed that all bumps are properly provided, prior to determining the presence or absence of defects (and determining the pass / fail of the workpiece W) through visual inspection. The inspection range IR automatically specified for the reference workpiece may then be stored in the memory unit 41 as inspection range information, and then, when determining the presence or absence of defects (and the pass / fail of the workpiece W) for each workpiece W, the inspection range information may be read out and used to specify the inspection range IR.

[0071] The procedure for automatically specifying an inspection range using a reference workpiece will be described below with reference to the flowcharts shown in FIGS. 11 and 12 and the image transition diagram shown in FIG.

[0072] The automatic setting of the inspection range begins with the image of the reference workpiece (FIG. 13(a)) being displayed in the image pane P1 on the visual inspection screen shown in FIG. 6. When processing begins, the program first accepts settings of various conditions from the user (step S101). Specifically, settings of image processing parameters (such as binarization thresholds), defect judgment parameters (such as allowable defect width, height, and area), and the allowable number of defects are accepted.

[0073] When an execution command for automatic designation is input (step S102), the contours of all bumps BP appearing in the image of the reference workpiece are acquired and recorded in a bump contour list (step S103). The contours of the bumps BP appearing in the image of the reference workpiece can be acquired, for example, by the subroutine (steps S111 to S118) shown in FIG. 12. That is, the image of the reference workpiece is binarized (step S111; FIG. 13(b)), edges are detected for the image binarized in step S111, and the outermost contour is identified among them (step S112; FIG. 13(c)). The contour (outer edge) of the pad PD is identified at this time. Next, the binarized value (i.e., black and white) for the inside of the outermost contour identified in step S112 is inverted (step S113). As a result, the edge that forms the boundary between the inside and outside of the pad PD disappears, as shown in FIG. 13(d). Next, edges are detected again for the image of the reference workpiece that has been processed in step S113, and the outermost contour among them is obtained (step S114; FIG. 13(e)).

[0074] Then, the feature values ​​(e.g., the contour dimensions, the size of the smallest circumscribing rectangle, etc.) are calculated for one of the acquired contours that has not yet been determined (step S115). Furthermore, it is determined based on the calculated feature values ​​whether the acquired contour is the contour of a bump BP, and if it is determined that it is the contour of a bump BP (step S116; Yes), that contour is recorded in the bump contour list (step S117). On the other hand, if it is determined that it is not the contour of a bump BP (step S116; No), that contour is not recorded in the bump contour list, and the process proceeds to step S118. If there are any contours that have not yet been determined as being the contour of a bump BP (step S118; Yes), the process returns to step S115, and a determination is made as to whether all contours acquired in step S114 are the contour of a bump BP. When there are no more undetermined contours (step S118; No), the bump contour list will contain the contours of all bump BPs that appear in the reference workpiece image.

[0075] Returning to FIG. 11, an inspection range IR is specified for all contours recorded in the bump contour list (step S104), and the automatic specification process is completed. At this time, an inspection range tool is displayed for each automatically specified inspection range IR in the image pane P1, superimposed on the workpiece image WG (FIG. 13(f)). The method of specifying the inspection range IR for each contour is arbitrary. For example, the inspection range IR may be a rectangle of a predetermined size centered on the reference position (e.g., center or center of gravity) of each contour. Alternatively, the inspection range IR may be a rectangle that surrounds each contour at a predetermined distance.

[0076] In this way, an inspection range IR can be automatically specified for the many bumps BP contained in the image of the reference workpiece. If the inspection range information for the automatically specified inspection range is saved, that inspection range information can be repeatedly applied to specify the inspection range for the image of each workpiece W to be inspected. Note that in the above example, the process of automatically specifying the inspection range was applied to the image of the reference workpiece, but the inspection range may also be automatically specified for the image of each workpiece W to be inspected.

[0077] According to the visual inspection described above, inspection for solder defects can be performed using the image measuring device 1, and measurement and inspection of soldered parts can be performed in one system. In addition, workpieces with many soldered parts to be inspected can be inspected efficiently.

[0078] [Modifications of the embodiment] The present invention is not limited to the above-described embodiments, and modifications and improvements within the scope of achieving the object of the present invention are included in the present invention. For example, the method for acquiring the contours of the pads PD and bumps BP is not limited to the method disclosed in the above-described embodiments (i.e., the subroutines shown in Figures 8 and 9), and any method may be adopted.

[0079] Furthermore, any embodiment or specific example thereof to which a person skilled in the art appropriately adds, deletes, or modifies components is also included within the scope of the present invention as long as it includes the gist of the present invention. [Explanation of symbols]

[0080] 1. Image measuring device 40 CPU 41 Storage section 42 Working Memory 43, 44 Interface 45 Display control unit 100 stages 110 Location acquisition means 112 X-axis encoder 114 Y-axis encoder 116 Z-axis encoder 118 Latching means 118a Counter 118b buffer 120 Imaging unit 122 Optical system 124 Imaging means 126, 126a, 126b light source 130 Remote Box 132 Joystick 134 Jog Shuttle 140 Computer Systems 141 Computer main body 142 keyboards 143 Mouse 144 displays P1 Image Pane P2 Filmstrip Pane P3 Measurement result display pane P4 Control Pane double work BP Bump PD Pad IR Inspection Range

Claims

1. 1. A visual inspection method for inspecting solder bumps formed on pads of an object to be inspected based on an image of the object to be inspected, comprising: an inspection range designation step of designating an inspection range in the image of the inspection object; a pad area detection step of detecting a pad area included in the inspection range; a bump area detection step of detecting a bump area included in the inspection range; a defect determination step of determining whether or not there is a defect based on the detected pad region and / or bump region; A visual inspection method comprising:

2. In the inspection area designation step, a plurality of inspection areas are designated in the image of the inspection object; 2. The visual inspection method according to claim 1, wherein the pad area detection step, the bump area detection step, and the defect determination step are performed for each of the plurality of designated inspection ranges.

3. The inspection range designation step designates an inspection range based on predetermined inspection range information, 3. The visual inspection method according to claim 1, wherein the inspection range information includes at least information indicating the position of the inspection range.

4. The visual inspection method according to claim 3 , wherein the inspection range information further includes information indicating a size of the inspection range.

5. 4. The visual inspection method according to claim 3, wherein the inspection range information further includes information regarding a determination condition used for determination in the defect determination step.

6. further comprising a step of extracting a small piece image of the inspection range from the image of the inspection object; The pad area detection step includes: binarizing the particle image; detecting edges in the binarized fragment image; acquiring the outermost edge of the detected edges as the outline of the pad area; and The bump region detection step includes: Inverting the binarized luminance values ​​of the inside of the outline of the pad area for the binarized small piece image; detecting an edge in the piece image in which the brightness values ​​inside the outline of the pad area are inverted; acquiring the outermost edge of the detected edges as the contour of the bump region; 2. The visual inspection method according to claim 1, further comprising:

7. 3. The visual inspection method according to claim 1, wherein the defects to be determined in the defect determination step include one or more of missing bumps, out-of-design range defects, and misalignment defects.

8. A program for causing a computer to execute the visual inspection method according to claim 1 or 2.

9. 1. A method for specifying an inspection area in a visual inspection for inspecting solder bumps formed on pads of an object to be inspected based on an image of the object to be inspected, comprising: a bump region detection step of detecting one or more bump regions included in the image; an inspection range designation step of designating an inspection range for each of the one or more detected bump regions; An inspection range designation method comprising:

10. 10. The inspection range designation method according to claim 9, further comprising an inspection range information saving step of saving inspection range information including at least information indicating the position of the inspection range for the inspection range designated in the inspection range designation step.

11. A program for causing a computer to execute the inspection range designation method according to claim 9 or 10.

Images