Appearance inspection method, inspection area specifying method, and program

Abstract

Provided is an appearance inspection method capable of inspecting solder defects using an image measuring device. The appearance inspection method inspects a solder bump formed on a pad of an inspection object based on an image of the inspection object. The appearance inspection method includes: an inspection area specifying step for specifying an inspection area in an image of an inspection object; a pad region detection step for detecting a pad region included in the inspection region; a raised region detection step for detecting a raised region included in the inspection region; and a defect determination step for determining the presence or absence of a defect on the basis of the detected pad region and / or bump region.

Description

Technical Field

[0001] The present invention relates to an appearance inspection method and a program for inspecting the appearance of a measurement object based on an image obtained by capturing the measurement object. Background Art

[0002] Vision measuring devices capture images of a measurement object (hereinafter referred to as a "workpiece"), analyze the images, extract a point cloud of edges contained in the images, and evaluate the distance, inclination, diameter, width, and other parameters of geometric shapes (such as lines, circles, and polygons) approximated from the extracted edge point cloud. In addition to evaluating geometric shapes, recent vision measuring devices are also implemented with algorithms that detect defects such as contamination on the workpiece, foreign matter inside hole shapes, minute chips, deformation, burrs, and contamination, enabling image-based defect inspection (see, for example, JP2020-071106). Summary of the Invention

[0003] Problems to be solved by the present invention

[0004] In recent years, due to the rising demand for semiconductors, there has been an increasing demand for inspecting solder defects on printed circuit boards, etc. However, algorithms for inspecting solder defects using image measurement devices have not yet been implemented.

[0005] In view of the above-mentioned problems, an object of the present invention is to provide a visual inspection method capable of inspecting solder defects using a vision measuring device, and a program for realizing such a visual inspection method.

[0006] Means used to solve problems

[0007] A visual inspection method according to one aspect of the present invention inspects solder bumps formed on a pad of an inspection object based on an image of the inspection object. The visual inspection method includes: an inspection area designation step for designating an inspection area in the image of the inspection object; a pad area detection step for detecting a pad area included in the inspection area; a bump area detection step for detecting a bump area included in the inspection area; and a defect determination step for determining the presence or absence of a defect based on the detected pad area and / or bump area. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a perspective view showing an example of the configuration of the vision measuring device 1 .

[0009] Figure 2 is a schematic diagram showing the configuration of the image capturing unit 120 and the platform 100 .

[0010] Figure 3 is a block diagram showing the configuration of the position acquisition unit 110 .

[0011] Figure 4 141 is a block diagram showing the configuration of the computer main body 141 .

[0012] Figure 5 An example of a view of a screen display is shown.

[0013] Figure 6 An example of an appearance inspection view is shown.

[0014] Figure 7 is a flowchart showing the procedure of appearance inspection.

[0015] Figure 8 is a flowchart showing an example of a subroutine of appearance inspection.

[0016] Figure 9 is a flowchart showing an example of a subroutine of appearance inspection.

[0017] Figure 10 The figure shows the image transition during the appearance inspection process.

[0018] Figure 11 is a flowchart illustrating an example of a process of automatic designation of an inspection area.

[0019] Figure 12 : is a flowchart showing an example of a subroutine for automatic designation of an inspection area.

[0020] Figure 13 An image transition diagram during the process of automatically specifying an inspection region is shown. DETAILED DESCRIPTION

[0021] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Figure 1 1 is a perspective view showing the internal structure of the vision measuring device 1. The vision measuring device includes a platform 100, a position acquisition unit 110, an image capturing unit 120, a remote control box 130, and a computer system 140.

[0022] The platform 100 is arranged with its upper surface horizontal, and a workpiece W (an object to be measured or inspected) is placed on its upper surface. At least a portion of the top surface of the platform 100, on which the workpiece W is placed, is formed of a light-transmitting material such as glass. The platform 100 is driven by an X-axis drive motor and a Y-axis drive motor (not shown in the drawings) and is movable in the X-axis and Y-axis directions parallel to a horizontal plane. Drive control signals for the drive motors of each axis are supplied to the drive motors of each axis from the remote control box 130 and the computer system 140, described later.

[0023] Figure 21 is a schematic diagram illustrating the configuration of the image capture unit 120 and the stage 100. The image capture unit 120 includes an optical system 122, an image sensor 124, and a light source 126. The optical system 122 is composed, for example, of a telecentric optical system combining multiple lenses and an aperture. In a telecentric optical system, the principal rays can be considered parallel, so the dimensions of the captured image do not depend on the position in the Z-axis direction (height direction). For this reason, telecentric optical systems are suitable for measuring workpieces W with irregularities (e.g., steps or holes). When capturing an image of the workpiece W, the light source 126, under the control of the computer system 140, illuminates at least the portion of the workpiece W to be imaged. In this embodiment, the light source 126a for epi-illumination illuminates the workpiece W from above (i.e., toward the image sensor 124) via the optical system 122, and the light source 126b for transillumination illuminates the workpiece W from below (i.e., toward the back surface of the stage 100). The image sensor 124 is a two-dimensional image sensor such as a CCD or CMOS. An image of the workpiece W is formed by the optical system 122 on the light-receiving surface of the image sensor 124. The image sensor 124 captures the formed image and outputs image data in a predetermined format. This image data includes information about the pixels that make up the image and an index indicating the order in which the images were captured. The image capture unit 120 transmits the image signal output by the image sensor 124 to the computer system 140. The computer system 140 and the image capture unit 120 are connected using a common communication standard such as USB (Universal Serial Bus). Furthermore, the image capture unit 120 outputs a trigger signal to the latch unit 118 upon completion of capturing one image (one frame).

[0024] The image capture unit 120 is driven by a Z-axis drive motor (not shown in the drawings) and is movable in the Z-axis direction (i.e., a direction perpendicular to the top surface of the platform 100). Focus adjustment is performed by adjusting the Z-axis position of the image capture unit 120. A drive control signal for the Z-axis drive motor is supplied from the remote control box 130 and the computer system 140, which will be described later.

[0025] Figure 3 2 is a block diagram showing the configuration of the position acquisition unit 110. The position acquisition unit 110 has an X-axis encoder 112, a Y-axis encoder 114, a Z-axis encoder 116, and a latch unit 118.

[0026] The X-axis encoder 112 measures and outputs the position coordinates of the platform 100 in the X-axis direction. The Y-axis encoder 114 measures and outputs the position coordinates of the platform 100 in the Y-axis direction. The Z-axis encoder 116 measures and outputs the position coordinates of the image capture unit 120 in the Z-axis direction. Each encoder is equipped with a scale and a scale reader for reading the scale. The scale can be attached to the movable part of the platform 100 and the image capture unit 120 along each axis. On the other hand, the scale reader is placed on the immovable part.

[0027] The latch unit 118 includes a counter 118a and a buffer 118b. Counter 118a increments its count value by 1 when supplied with an external trigger signal (e.g., a pulse signal). The value of counter 118a is reset as appropriate based on instructions from the computer system 140. Buffer 118b has storage areas with multiple addresses. When a trigger signal is supplied, the encoder output value for each axis is latched and stored in the storage area at the address corresponding to the count value of counter 118a. The trigger signal can be supplied from the image sensor 124, for example, at the end of capturing an image. The position coordinates of each axis held by the latch unit 118 are associated with the address value (i.e., the count value) and are appropriately retrieved into the computer system 140. The computer system 140 and the latch unit 118 are connected using a common communication standard such as USB (Universal Serial Bus). The image data and position coordinates are imported into the computer system 140 separately, but the image data is indexed to indicate the capture order, and the position coordinates are associated with the count values ​​to indicate the capture order, so that even if they are imported into the computer system 140 asynchronously, they can be associated after importing.

[0028] return Figure 1 The remote control box 130 is an operating component for setting the position of the platform 100 and the image capture unit 120. In response to an operator's operation, it transmits drive control signals to the X-axis drive motor, the Y-axis drive motor, and the Z-axis drive motor via wired or wireless communication. The remote control box 130 includes a joystick 132 and a jog shuttle 134. The joystick 132 is an input device for setting the position of the platform 100. The remote control box 130 transmits drive control signals to move the platform 100 in the X- and Y-axis directions based on the tilt direction of the joystick 132. The jog shuttle 134 is an input device for setting the Z-axis position of the image capture unit 120. The remote control box 130 transmits drive control signals to move the image capture unit 120 in the Z-axis direction based on the rotation direction, amount, and speed of the jog shuttle 134.

[0029] The computer system 140 includes a computer body 141 , a keyboard 142 , a mouse 143 , and a display 144 . Figure 41 is a block diagram showing the configuration of a computer main body 141. The computer main body 141 includes a CPU 40 serving as a control center, a storage unit 41, a work memory 42, and interfaces 43 and 44 (in Figure 4 ), and a display control unit 45 that controls the field of view on the display 144.

[0030] Operator instruction information input from the keyboard 142 or the mouse 143 is input to the CPU 40 via the interface 43. The interface 44 is connected to the image capture unit 120 and the platform 100, provides various control signals from the CPU 40 to the image capture unit 120 and the platform 100, receives various status information and measurement results from the image capture unit 120 and the platform 100, and inputs them to the CPU 40.

[0031] The display control unit 45 causes the image captured by the image capture unit 120 to be displayed on the display 144. In addition, the display control unit 45 causes the display 144 to display the image captured by the image capture unit 120, an interface for inputting control instructions to the vision measuring device 1, and an interface for a tool for analyzing the captured image.

[0032] The work memory 42 provides a work area for various types of processing by the CPU 40. The storage unit 41 is configured by, for example, a hard disk drive, a RAM, etc., and stores programs to be executed by the CPU 40, image data captured by the image capture unit 120, and other data.

[0033] Based on various types of information input via various interfaces, operator instructions, measurement definition programs (partial programs) stored in the storage unit 41, and the like, the CPU 40 performs various types of processing, including: control of the image capture unit 120, the X-axis drive motor, the Y-axis drive motor, and the Z-axis drive motor, etc., setting of the movement path of the image capture unit 120 and adjustment of the movement speed and exposure time, adjustment of the light intensity of the light source 126, image capture of a two-dimensional image by the image capture unit 120, image stitching processing of pasting multiple partial images together, and analysis of the overall image obtained by image capture, etc.

[0034] Hereinafter, measurement performed by using the vision measuring device 1 is described.

[0035] (Basic image measurement)

[0036] First, the operator moves the stage 100 by operating the joystick 132 or through control by the computer system 140, so that the workpiece W enters the imaging field of view. The Z-axis position of the image capture unit 120 is then adjusted to bring the workpiece W into focus. After the workpiece W is in focus, an image for measurement is captured using the image sensor 124. At this point, the coordinates of the stage 100 output by the X-axis encoder 112 and the Y-axis encoder 114 are captured by the computer system 140 along with the captured image and stored in the storage unit 41. Specifically, when the image capture unit 124 completes capturing an image, it outputs a pulse as a trigger signal to the latch unit 118. The latch unit 118 latches and holds the position coordinates of each axis at the timing of the rising transition of the pulse (i.e., approximately simultaneously with the completion of image capture). The computer system 140 acquires the image signal from the image capture unit 124 and the position coordinates at the time of image capture from the latch unit 118, storing them in association with each other.

[0037] The computer system 140 displays the obtained image for measurement and an interface of a measurement tool for analyzing the image on a display 144 . Figure 5 An example of a view of a screen display is shown. This screen display is shown on the display 144 by a program (measurement application software) executed on the CPU 40 of the computer system 140 .

[0038] like Figure 5 As shown, when a program is executed, a main window MW is displayed on display 144. Multiple windows (windows W1 to W8) are also displayed within main window MW. Menu icons, various operations, and settings are also displayed at the top of main window MW. While this embodiment shows an example of eight windows, it is possible to display more than eight windows as needed, or to divide, integrate, or omit windows according to their purpose. The layout of each window can also be freely changed by the operator.

[0039] The first window W1 displays an image WG of the workpiece W captured by the image capture unit 120. The operator can adjust the position of the image WG of the workpiece W displayed in the first window W1 by, for example, operating the mouse 143 or joystick 132 of the remote control box 130. Furthermore, the operator can also enlarge or reduce the image WG of the workpiece W by, for example, selecting an icon using the mouse 143.

[0040] The second window W2 displays icons of measurement tools that can be selected by the operator. The icons for the measurement tools are provided to correspond to methods of specifying measurement points from the image WG of the workpiece W. Specific examples of the measurement tools include a straight edge detection tool, a circular edge detection tool, and the like.

[0041] The third window W3 displays icons for functions that can be selected by the operator. These icons are provided for each measurement method. For example, there are methods for measuring the coordinates of a single point, measuring the length of a line, measuring a circle, measuring an ellipse, measuring a square hole, measuring a slotted hole, measuring a distance, and measuring the tolerance between two lines. Based on the operator's selections, the computer system 140 performs measurements of dimensions such as the length of a line, the distance between lines, and the diameter of a circle, as well as assesses deviations (errors) from ideal geometrical shapes, such as straightness, circularity, and parallelism.

[0042] In the fourth window W4 , guidance showing an operation procedure for measurement is displayed.

[0043] In the fifth window W5 , various sliders for controlling illumination from the image capturing unit 120 to the workpiece W are displayed. The operator can operate the sliders to irradiate the workpiece W with desired illumination.

[0044] The sixth window W6 displays the XY coordinate values ​​of the platform 100. The XY coordinate values ​​displayed in the sixth window W6 are the X-axis coordinates and the Y-axis coordinates of the platform 100 relative to a predetermined coordinate origin.

[0045] The seventh window W7 displays the tolerance judgment result. That is, when a measurement method capable of performing tolerance judgment is selected, the judgment result is displayed in the seventh window W7.

[0046] The eighth window W8 displays the measurement results. Specifically, when a measurement method that obtains measurement results through a predetermined calculation is selected, the measurement results are displayed in the eighth window W8. Details of the tolerance determination results for the seventh window W7 and the measurement results for the eighth window W8 are omitted in the accompanying drawings.

[0047] (Visual inspection of solder defects)

[0048] In the vision measuring device 1 of this embodiment, the program (measurement application software) executed by the CPU 40 of the computer system 140 provides, in addition to the basic image measurement functions described above, a function for visual inspection (hereinafter referred to as visual inspection) focused on solder defects. In the following description, unless a specific object is mentioned, it should be understood that the object is the program executed by the CPU 40 of the computer system 140.

[0049] In this system, solder defects include the following three types of defects.

[0050] (1) Missing bulge: This is a defect where there is no bulge candidate within the inspection area.

[0051] (2) Out of design range: This is a defect where the candidate protrusions in the inspection area are outside the specified design value range.

[0052] (3) Misalignment: This is a defect where the distance from the bump candidate to the pad (the shortest distance between the outer edges) is less than the threshold.

[0053] The range of the design value for judging whether a value is out of range and the threshold value for judging whether there is a misalignment can be changed by the user on the screen of the measurement application software.

[0054] Figure 6 This shows an example of a screen for appearance inspection (hereinafter referred to as appearance inspection view). The appearance inspection view consists of an image pane P1, a filmstrip pane P2, a measurement result display pane P3, and a control pane P4.

[0055] The image pane P1 is an area where an image for visual inspection is displayed. Image processing and defect determination are performed on the image displayed in the image pane P1 under the conditions set in the control pane P4.

[0056] By dragging the mouse over the image pane P1, you can designate a rectangular area, whose diagonal lines connect the start and end points of the drag, as an inspection region IR on the displayed image for defect detection. While dragging, the Inspection Region tool, which outlines the designated rectangular area, is displayed overlaid on the image in the image pane P1. The Inspection Region tool can be selected by clicking it, and its size can be changed by dragging the handles that appear when it is selected. The Region tool can also be deleted by pressing the DEL key on the keyboard while the tool is selected.

[0057] The Filmstrip P2 displays images loaded for visual inspection in thumbnail format. Double-clicking an image displayed in Filmstrip P2 displays that image in Image Pane P1 and becomes the target of image processing and defect detection. In the initial state immediately after image loading, a specific image (for example, the image that appears first when sorting by name or by saved date and time) is selected in Filmstrip P2 and displayed in Image Pane P1.

[0058] If the image processed and judged in the image pane P1 has a defect, a hatching H is added to the image in the film pane P2 where the defect is found, so that it can be easily distinguished from an image where no defect is found.

[0059] The measurement result display pane P3 is an area where a list of defect information is displayed when a defect is found in the image displayed in the image pane P1. If there are multiple defects, information about all defects is displayed in a list format.

[0060] Control pane P4 displays a user interface for setting conditions for image processing and defect detection performed on the image displayed in image pane P1. The user interface provided in control pane P4 allows you to set parameters for image processing (e.g., the threshold for binary conversion, whether to invert brightness values, etc.) and defect detection (e.g., the design value for bumps, the minimum allowable distance from pads, the size of the inspection area tool, etc.). In addition to allowing direct input of numerical values, the user interface can be provided as a graphical user interface (GUI) control, such as a slider or switch. Control pane P4 also includes a button B1 for inputting a command to execute a visual inspection and a button B2 for inputting a command to automatically designate an inspection area.

[0061] Next, refer to Figures 7 to 9 The flowchart shown and Figure 10 The image transition diagram shown explains the process of performing appearance inspection using an example in which the inspection area IR is specified by user operation on the screen of the measurement application software.

[0062] Before starting the visual inspection, the user selects a menu for performing the visual inspection in the program (measurement application software). In response, the display 144 shows the visual inspection view. The program then prompts the user to specify one or more images to be inspected. When the user specifies an image file in response, the image file is loaded and all loaded images are displayed in the filmstrip P2, with the first image ( Figure 10 (A) is displayed in image pane P1.

[0063] The image displayed in image pane P1 undergoes a visual inspection. If the user wishes to perform a visual inspection on another image other than the first image, the user can double-click the desired image in filmstrip pane P2 to display the desired image in image pane P1. This displays the image to be inspected in image pane P1, and the visual inspection begins.

[0064] When starting a visual inspection, the program first accepts various user-defined settings (step S01). Specifically, the program accepts settings for image processing parameters (such as the binarization threshold), defect judgment parameters (such as the allowable defect width, height, and area), and the allowable number of defects. These settings can be changed at any given time.

[0065] Next, by operating the mouse 143 or joystick 132 of the remote control box 130, an inspection region IR is designated on the image WG of the workpiece W displayed in the image pane P1 (step S02). The method for designating the target region is arbitrary, but for example, by dragging the mouse 143 on the image pane P1, a rectangular region with a line connecting the start and end points of the drag as its diagonal line can be designated as the inspection region IR. To designate the inspection region IR, an inspection region tool is displayed as a rectangular frame overlaid on the image of the workpiece WG in the image pane P1.

[0066] When the user specifies an inspection region IR, the inspection region IR can be specified to surround the pad PD where the bump BP is to be formed. For example, if the pad PD is not completely within the image WG, such as if the pad PD horizontally passes through the image WG, the inspection region IR can be specified around the location of the pad PD where the bump BP is to be formed. Because the outer edge of the pad PD is necessary for determining misalignment, the inspection region IR should be specified so that at least a portion of the outer edge of the pad PD is included in the inspection region IR.

[0067] The number of inspection regions IR that can be specified for a single image WG is arbitrary. The user can also specify the number of inspection regions IR corresponding to the number of protrusions BP to be inspected. The computer system 140 can store the information (position and range) of the inspection regions IR specified as described above in the storage unit 41. A method for using the information regarding the inspection regions IR stored in the storage unit 41 in subsequent inspections will be described later.

[0068] Then, when a command to execute the appearance inspection is input via the user interface (step S03 ), judgment of solder defects is performed for each inspection region IR designated as described above according to the procedure described below.

[0069] First, the inspection area IR to be judged is cut out from the image WG to create a rectangular small block image SG (step S04; Figure 10 Then, the outline of the pad PD is obtained in the small block image SG (step S05).

[0070] The outline of the pad PD in step S05 can be determined by, for example Figure 8 That is, the cut-out small block image SG is binarized (step S11; Figure 10(c) in the figure). The threshold value of the binarization is set so that the pad PD is black (value 0), and the bump BP and the area where neither the bump BP nor the pad PD is provided are white (value 1). When the binarization is performed, noise removal and other processing can be performed as needed. Next, for the small block image SG after the binarization in step S11, the edge (the boundary between the black and white areas) is detected, and the outermost edge is obtained as the outline (outer edge) of the pad PD (step S12; Figure 10 (d)).

[0071] return Figure 7 , following step S05 , the contour of the bulge BP is obtained in the small block image SG (step S06 ).

[0072] The contour of the protrusion BP in step S06 can be determined by, for example Figure 9 That is, the binary values ​​(ie, black and white) within the outline of the pad PD obtained in step S12 are inverted (step S13). Figure 10 As shown in (e) in FIG. 1 , the edge forming the boundary between the inner and outer sides of the pad PD disappears from the small block image SG. Next, the edges of the small block image SG processed in step S13 are detected again, and the outermost contour is obtained as the contour (outer edge) of the bump BP (step S14; Figure 10 (f)).

[0073] return Figure 7 Based on the contours of the pad PD and the bump BP obtained in steps S05 and S06, characteristic values ​​(e.g., the area and aspect ratio of the bump BP, the distance between the pad PD and the bump BP, etc.) are calculated for each pad PD and the bump BP, and a solder defect is determined (step S07). Specifically, if the bump BP contour is not obtained in step S05, it is determined to be "missing bump." If the area enclosed by the bump BP contour obtained in step S06 is outside the set design value range, it is determined to be "out of design range." If the distance (shortest distance) between the contours of the pad PD obtained in step S04 and the bump BP contour obtained in step S06 is less than a set threshold, it is determined to be "misaligned." If these criteria are not met, it is assumed that there is no solder defect.

[0074] The results of the determination are then displayed on the display 144 (step S08). Defects identified by the determination are displayed in the measurement result display pane P3 and superimposed on the workpiece image WG in the image pane P1. The method for displaying defects in the image pane P1 is arbitrary. For example, the outline color of an inspection area IR determined to have defects can be changed, or the protrusions determined to be defects can be displayed by filling the outline with a conspicuous color (e.g., red), making it easier for the user to identify the discovered defects. Figure 6 An example is shown in which the outline of the inspection region IR determined to have a defect is displayed with a bold line.

[0075] If there are uninspected inspection areas IR (step S09; Yes), the process returns to step S04 and steps S04 to S08 are performed on all inspection areas IR specified in step S02. If there are no uninspected inspection areas IR (step S09; No), the appearance inspection ends.

[0076] In this way, solder defects can be inspected using the image measuring device, and measurement and inspection of the soldered portion can be performed using a single system.

[0077] (Preservation and reuse of inspection area information)

[0078] In the above example, the inspection region IR is specified by a user operation on the measurement application software screen, but the inspection region IR can be specified based on pre-stored inspection region information. The inspection region information includes the positions of the individual inspection regions IR (e.g., the position coordinates of the center) for at least the desired number of inspection regions IR and is stored in the storage unit 41. The position of each inspection region IR included in the inspection region information can be an absolute coordinate in the coordinate system of the image measuring device 1, but it is preferable to use relative coordinates based on a reference position within the workpiece W. The inspection region information may also include the size of each inspection region IR (e.g., the height and width of a rectangle). The inspection region information can be stored as information for the inspection region IR specified by the user on the screen of the measurement application software, or can be stored as information for the inspection region IR specified by the automatic designation described later. Alternatively, these can be stored after being edited by the user.

[0079] In order to apply the inspection area information to the appearance inspection, instead of specifying each inspection area IR by user operation on the screen of the measurement application software as in step S02 of the above-mentioned appearance inspection process, the inspection area information stored in the storage unit 41 is read out and applied to the image to be inspected. If the inspection area information includes only the position information of each inspection area IR but not the size information, an inspection area IR of a predetermined size (for example, a size set by the user in the measurement application software) can be specified. Thus, the user can specify multiple inspection areas IR at one time without having to manually specify each inspection area IR. This is effective in the case of repeatedly performing appearance inspections on workpieces W of the same specifications. Once the inspection area IR is specified, it can be determined whether there are any defects in each specified inspection area IR in the same manner as in the above-mentioned appearance inspection process.

[0080] In the inspection region information, visual inspection conditions can be further associated with each inspection region IR. For example, a design value or threshold range for determination can be defined for each inspection region IR. During the visual inspection, the design value or threshold range defined in the inspection region information can be used to determine the presence of defects. The type of determination applied to each individual inspection region IR can also be defined (for example, only determining that a protrusion is missing). This allows for defect determination in multiple inspection regions IR under different conditions.

[0081] (Automatic designation of inspection area)

[0082] In the above example, the inspection region IR is specified by the user on the measurement application screen. However, image processing can also be used to automatically generate the inspection region IR for the image WG of the workpiece W (automatic inspection region specification). Automatic inspection region specification can be performed on a workpiece (reference workpiece) for which all protrusions have been confirmed to be correctly formed, before determining the presence of defects (and the acceptability of the workpiece W) during visual inspection. The inspection region IR automatically specified for the reference workpiece can be stored as inspection region information in the storage unit 41. This information can then be read out and used to specify the inspection region IR when determining the presence of defects (and the acceptability) of each workpiece W.

[0083] In the following, reference Figure 11 and Figure 12 The flowchart shown and Figure 13 The image migration diagram shown illustrates the process of automatic designation of inspection areas using reference artifacts.

[0084] Automatic designation of inspection areas Figure 6 The image of the reference workpiece shown in the image pane P1 on the appearance inspection view is shown. Figure 13Once processing begins, the program first accepts user input for various conditions (step S101). Specifically, the program accepts settings for image processing parameters (such as the binarization threshold), defect determination parameters (such as the allowable defect width, height, and area), and the allowable number of defects.

[0085] When the command to execute automatic designation is input (step S102), the contours of all bulges BP appearing in the image of the reference workpiece are obtained and recorded in a bulge contour list (step S103). The contours of bulges BP in the image of the reference workpiece can be obtained, for example, by Figure 12 That is, the image of the reference workpiece is binarized (steps S111, S118). Figure 13 (b)), detecting the edge of the image after the binarization processing in step S111, identifying the outermost contour (step S112, Figure 13 (c)). At this time, the outline (outer edge) of the pad PD is identified. Then, the binary values ​​(i.e., black and white) inside the outermost outline identified in step S112 are inverted (step S113). As a result, the edge forming the boundary between the inner and outer sides of the pad PD disappears, as shown in FIG. Figure 13 Next, the image of the reference workpiece that has undergone the processing of step S113 is reprocessed to detect edges and obtain the outermost contour (step S114, Figure 13 (e)).

[0086] Next, for one of the acquired contours that has not yet been determined, feature values ​​(e.g., contour dimensions, minimum enclosing rectangle size, etc.) are calculated (step S115). Furthermore, based on the calculated feature values, it is determined whether the acquired contour is a convex BP contour. If it is determined to be a convex BP contour (step S116: Yes), the contour is recorded in the convex contour list (step S117). On the other hand, if it is determined not to be a convex BP contour (step S116: No), the contour is not recorded in the convex contour list, and the process proceeds to step S118. If there are still contours that have not been determined to be convex BP contours (step S118: Yes), the process returns to step S115, and all contours acquired in step S114 are determined to be convex BP contours. If there are no more contours to be determined (step S118: No), the convex contour list will contain all contours of convex BP that appear in the image of the reference workpiece.

[0087] return Figure 11, the inspection region IR is specified for all the contours recorded in the raised contour list (step S104), and the automatic specification process ends. At this time, in the image pane P1, the inspection region tool is displayed for each of the automatically specified inspection regions IR, which is superimposed on the image of the workpiece WG ( Figure 13 (f)). The method used to specify the inspection region IR for each contour is arbitrary. For example, a rectangle of a predetermined size centered on a reference position (e.g., center or center of gravity) of each contour can be used as the inspection region IR. Alternatively, the inspection region IR can be a rectangle surrounding each contour at a predetermined distance.

[0088] In this way, it is possible to automatically designate inspection regions IR for the numerous protrusions BP included in the image of the reference workpiece. By storing inspection region information for the automatically designated inspection regions, the same inspection region information can be repeatedly applied to designating inspection regions for images of individual workpieces W to be inspected. In the above example, automatic designation of inspection regions was applied to the image of the reference workpiece, but automatic designation of inspection regions can also be applied to each image of an individual workpiece to be inspected.

[0089] According to the appearance inspection described above, solder defects can be inspected using the vision measuring device 1, and solder portion measurement and inspection can be performed in a single system. In addition, workpieces having a large number of solder portions to be inspected can be inspected efficiently.

[0090] (Modification of the embodiment)

[0091] The present invention is not limited to the examples of the above-mentioned embodiments, and any modifications, improvements, etc. are included in the present invention within the scope of achieving the purpose of the present invention. For example, the method of obtaining the profiles of the pad PD and the bump BP is not limited to the method disclosed in the above-mentioned embodiment (i.e., Figure 8 and Figure 9 subroutine shown), and any method can be used.

[0092] Furthermore, inventions in which those skilled in the art appropriately add, delete, or change the designs of the above-described embodiments or specific examples thereof are also included in the scope of the present invention as long as they have the gist of the present invention.

[0093] Regarding the embodiments including the above examples, the following appendix is ​​further disclosed.

[0094] (Appendix 1) A visual inspection method for inspecting a solder bump formed on a pad of an inspection object based on an image of the inspection object, comprising:

[0095] an inspection region designating step for designating an inspection region in an image of an inspection object;

[0096] a pad area detection step for detecting a pad area included in the inspection area;

[0097] a convex region detecting step for detecting a convex region included in the inspection region;

[0098] A defect judging step for judging the presence or absence of a defect based on the inspected pad area and / or bump area.

[0099] (Appendix 2) The appearance inspection method according to Appendix 1, wherein, in the inspection area designation step, a plurality of inspection areas are designated in the image of the inspection object,

[0100] The pad area detection step, the bump area detection step and the defect judgment step are performed for each of the plurality of designated inspection areas.

[0101] (Appendix 3) The appearance inspection method according to Appendix 1 or 2, wherein, in the inspection area designation step, the inspection area is designated based on predetermined inspection area information,

[0102] The inspection area information includes at least information indicating the position of the inspection area.

[0103] (Appendix 4) The appearance inspection method according to Appendix 3, wherein the inspection area information further includes information indicating a size of the inspection area.

[0104] (Appendix 5) The appearance inspection method according to Appendix 3, wherein the inspection area information further includes information related to a judgment standard used in the defect judgment step.

[0105] (Appendix 6) The appearance inspection method according to Appendix 1, further comprising the step of cutting out a small image of the inspection area from the image of the inspection object, and

[0106] The pad area detection step includes:

[0107] The step of binarizing the small image blocks;

[0108] The step of detecting edges in the binarized small patch image; and

[0109] a step of obtaining the outermost edge of the detected edges as the outline of the pad area, and

[0110] The convex area detection step includes:

[0111] A step of inverting the binary brightness value within the outline of the pad area for the binary small block image;

[0112] a step of detecting edges in a small block image whose brightness value has been inverted within the contour of the pad area; and

[0113] The step of obtaining the outermost edge among the detected edges as the contour of the convex region.

[0114] (Appendix 7) The appearance inspection method according to Appendix 1 or 2, wherein the defects judged in the defect judgment step include one or more of missing protrusions, exceeding the design range, and misalignment.

[0115] (Appendix 8) A program for causing a computer to execute the appearance inspection method according to Appendix 1 or 2, and a non-transitory recording medium having the program recorded thereon.

[0116] (Appendix 9) An inspection area designation method for use in a visual inspection method for inspecting solder bumps formed on pads of an inspection object based on an image of the inspection object, the inspection area designation method comprising:

[0117] a convex region detection step for detecting one or more convex regions included in an image; and

[0118] The inspection region designating step is for designating an inspection region for each of the one or more detected convex regions.

[0119] (Appendix 10) The inspection area designation method according to Appendix 9 further includes an inspection area information storage step of storing inspection area information including at least information indicating a position of the inspection area designated in the inspection area designation step.

[0120] (Appendix 11) A program for causing a computer to execute the inspection area designation method according to Appendix 9 or 10, and a non-transitory recording medium having the program recorded thereon.

Claims

1. A visual inspection method for inspecting solder bumps formed on pads of an inspection object based on an image of the inspection object, the visual inspection method comprising: an inspection area designating step for designating an inspection area in the image of the inspection object; a pad area detection step for detecting a pad area included in the inspection area; a convex region detecting step for detecting a convex region included in the inspection region; A defect judging step for judging the presence or absence of a defect based on the inspected pad area and / or bump area.

2. The appearance inspection method according to claim 1, wherein: In the inspection area designation step, a plurality of inspection areas are designated in the image of the inspection object. The pad area detection step, the bump area detection step, and the defect judgment step are performed for each of a plurality of designated inspection areas.

3. The appearance inspection method according to claim 1 or 2, wherein: In the inspection area specifying step, the inspection area is specified based on predetermined inspection area information, The inspection area information includes at least information indicating a position of the inspection area.

4. The appearance inspection method according to claim 3, wherein: The inspection region information further includes information indicating a size of the inspection region.

5. The appearance inspection method according to claim 3, wherein: The inspection area information also includes information related to a judgment standard used in the defect judgment step.

6. The appearance inspection method according to claim 1, further comprising the step of cutting out a small image of the inspection area from the image of the inspection object, and in, The pad area detection step includes: A step of binarizing the small image block; The step of detecting edges in the binarized small patch image; and a step of obtaining the outermost edge of the detected edges as the outline of the pad area, and The convex area detection step includes: A step of inverting the binary brightness value within the outline of the pad area for the binary small block image; a step of detecting an edge in the small block image in which the brightness value within the outline of the pad area has been inverted; and The step of obtaining the outermost edge among the detected edges as the contour of the convex area.

7. The appearance inspection method according to claim 1 or 2, wherein: The defects judged in the defect judging step include one or more of missing protrusions, exceeding a design range, and misalignment. 8 . A non-transitory recording medium having recorded thereon a program for causing a computer to execute the appearance inspection method according to claim 1 .

9. An inspection area designation method for use in a visual inspection method for inspecting solder bumps formed on pads of an inspection object based on an image of the inspection object, the inspection area designation method comprising: a raised area detection step for detecting one or more raised areas included in the image; as well as The inspection region designating step is for designating an inspection region for each of the one or more detected convex regions.

10. The inspection area specifying method according to claim 9, further comprising an inspection area information storing step of storing inspection area information including at least information indicating a position of the inspection area specified in the inspection area specifying step. 11 . A non-transitory recording medium having recorded thereon a program for causing a computer to execute the inspection area designation method according to claim 9 or 10.

Images