APPEARANCE TEST PROCEDURE, TEST AREA DETERMINATION PROCEDURE AND PROGRAM

Abstract

An appearance inspection method that can be used to inspect solder defects using an image measurement device is provided. An appearance inspection method inspects solder bumps formed on the pads of the inspection target based on an image of an inspection target. The appearance inspection method includes: an inspection region determining step for determining the inspection region in the image of the inspection target; a field region detecting step for detecting a field region included in the inspection region; a bump region detecting step for detecting a bump region included in the inspection region; and a defect judging step for judging the presence or absence of the defect based on the detected field region and / or bump region.

Description

BACKGROUND OF THE INVENTION FIELD OF THE INVENTION

[0001] The present invention relates to an appearance inspection method and program for inspecting the appearance of a measurement target based on an image obtained by detecting the measurement target. STATE OF THE ART

[0002] An image measuring device is a device that acquires an image of the measurement target (hereinafter referred to as a "workpiece"), analyzes the image, extracts the point cloud of the edges contained in the image, and evaluates the distance, inclination, diameter, width, or the like of geometric shapes such as lines, circles, and polygons, or the like, approximated from the extracted edge point cloud. In addition to evaluating geometric shapes, recent image measuring devices are also implemented with algorithms that detect defects such as impurities on the workpiece, foreign objects within hole shapes, minute chips, deformation, and burrs, and implements defect inspection based on the image (see, for example, JP2020-071106). SUMMARY OF THE INVENTION TASK TO BE SOLVED BY THE INVENTION

[0003] In recent years, due to the increasing demand for semiconductors, there has been a growing demand for inspecting solder defects in printed circuit boards, etc. However, no algorithm for inspecting solder defects using the image measurement device has been realized.

[0004] In view of the above problems, the object of the present invention is to provide an appearance inspection method that can perform a soldering defect inspection using an image measuring device and a program that realizes such an appearance inspection method. SOLUTION TO THE TASK

[0005] An appearance inspection method according to one aspect of the present invention inspects solder bumps formed on patches of the inspection target based on an image of the inspection target. The appearance inspection method includes: an inspection region determining step for determining the inspection region in the image of the inspection target; a patch region detecting step for detecting a patch region included in the inspection region; a bump region detecting step for detecting a bump region included in the inspection region; and a defect judging step for judging the presence or absence of the defect based on the detected patch region and / or bump region. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a perspective view showing an example of the configuration of the image measuring device 1. Fig. 2 is a schematic diagram showing a configuration of the image pickup unit 120 together with the stage 100. Fig. 3 is a block diagram showing a configuration of the position detecting unit 110. Fig. 4 is a block diagram showing a configuration of a computer main body 141. Fig. Figure 5 shows an example of a screen display view. Fig. 6 shows an example of an appearance check view. Fig. Figure 7 is a flowchart showing the process of appearance testing. Fig. 8 is a flowchart showing an example of an appearance check subroutine. Fig. Figure 9 is a flowchart showing an example of an appearance check subroutine. Fig. Figure 10 shows an image transition diagram in the appearance inspection process. Fig. 11 is a flowchart showing an example of the process of automatically determining the inspection area. Fig. 12 is a flowchart showing an example of the subroutine of automatically determining the inspection area. Fig. 13 shows an image transition diagram in the process of automatically determining the inspection area. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0006] Embodiments of the present invention will now be described with reference to the drawings. Fig. Figure 1 is a perspective view showing the internal structure of an image measuring device 1. The image measuring device includes a stage 100, a position detection unit 110, an image pickup unit 120, a remote box 130, and a computer system 140.

[0007] The stage 100 is arranged so that its upper surface is horizontal, and a workpiece (target of measurement or inspection) W is placed on the upper surface. At least the part of the upper surface of the stage 100 where the workpiece W is placed is made of a material that transmits light, such as glass. The stage 100 is driven by an X-axis drive motor and a Y-axis drive motor, which are not shown in the drawings, and can move in the X-axis direction and Y-axis direction parallel to the horizontal plane. The drive control signals for the drive motors of each axis are supplied to the drive motors of each axis from the remote box 130 and the computer system 140, which will be described later.

[0008] Fig. 2 is a schematic diagram showing a configuration of the image pickup unit 120 together with the stage 100. The image pickup unit 120 includes an optical system 122, an image sensor 124, and a light source 126. The optical system 122 is composed, for example, of a telecentric optical system that combines a plurality of lenses and a diaphragm. In a telecentric optical system, the principal rays can be considered as parallel light, so that the dimensions in the captured image do not depend on the position in the Z-axis direction (height direction). For this reason, the telecentric optical system is suitable for measuring the workpiece W having undulations (e.g., steps or holes). The light source 126 radiates light onto at least the part of the workpiece W to be imaged under the control of the computer system 140 when the image of the workpiece W is captured.In this embodiment, there is an epi-illumination light source 126a that emits light from above (i.e., toward the image sensor 124) via the optical system 122 toward the workpiece W, and a transmission light source 126b that emits light from below (i.e., toward the back of the stage 100) toward the workpiece W. The image of the workpiece W is formed on the light-receiving surface of the image sensor 124 by the optical system 122. The image sensor 124 captures the formed image and outputs image data in a predetermined format. This image data includes information about the pixels constituting the image and an index indicating the order of image capture. The image capture unit 120 transmits the image signals output by the image sensor 124 to the computer system 140.The computer system 140 and the image capture unit 120 are connected using a universal communication standard such as USB (Universal Serial Bus). Furthermore, the image capture unit 120 outputs a trigger signal to the locking unit 118 at the time of completing the capture of an image (a frame).

[0009] The image pickup unit 120 is driven by a Z-axis drive motor, not shown in the drawings, and is capable of moving in the Z-axis direction (i.e., a direction perpendicular to the upper surface of the stage 100). Focus adjustment is performed by adjusting the Z-axis position of the image pickup unit 120. The drive control signal for the Z-axis drive motor is supplied from the remote box 130 and the computer system 140, which will be described later.

[0010] Fig. Figure 3 is a block diagram showing a configuration of the position detection unit 110. The position detection unit 110 includes an X-axis encoder 112, a Y-axis encoder 114, a Z-axis encoder 116, and the locking unit 118.

[0011] The X-axis encoder 112 measures and outputs the position coordinate in the X-axis direction of the stage 100. The Y-axis encoder 114 measures and outputs the position coordinate in the Y-axis direction of the stage 100. The Z-axis encoder 116 measures and outputs the position coordinate in the Z-axis direction of the image pickup unit 120. Each encoder is equipped with a graduated scale and a scale reader that reads the scale. The scale can be attached to the movable parts of the stage 100 and the image pickup unit 120 along each axis. On the other hand, the scale readers are placed on the non-movable parts.

[0012] The locking unit 118 includes a counter 118a and a buffer 118b. The counter 118a increments the count value by 1 when an external trigger signal (e.g., pulse signal) is supplied. The value of the counter 118a is reset as needed based on the instructions of the computer system 140. The buffer 118b has a storage area for multiple addresses, and at the time the trigger signal is supplied, the output value of the encoder of each axis is latched and stored in the storage area of ​​the address corresponding to the count value of the counter 118a. The trigger signal can be supplied, for example, from the image sensor 124 at the time the acquisition of an image is completed. The position coordinates of each axis held by the locking unit 118 are assigned address values ​​(i.e., count values) and are recorded into the computer system 140 as needed.Computer system 140 and locking unit 118 are connected using a universal communication standard such as USB (Universal Serial Bus). The image data and position coordinates are imported into computer system 140 separately, but the image data is indexed to indicate the order in which it was acquired, and the position coordinates are assigned a count value to indicate the order in which it was acquired, so that even if they were imported into computer system 140 asynchronously, they can be assigned after import.

[0013] With further reference to Fig. 1, the remote box 130 is an actuator for adjusting the position of the stage 100 and the image pickup unit 120, and transmits drive control signals to the X-axis drive motor, the Y-axis drive motor, and the Z-axis drive motor via wired or wireless communication in response to operator operation. The remote box 130 includes a joystick 132 and a jog shuttle 134. The joystick 132 is an input device for adjusting the position of the stage 100, and the remote box 130 sends drive control signals to move the stage 100 in the X-axis and Y-axis directions according to the tilt direction of the joystick 132.The jog shuttle 134 is an input device for setting the Z-axis direction position of the image pickup unit 120, and the remote box 130 transmits drive control signals to move the image pickup unit 120 in the Z-axis direction according to the rotation direction, rotation amount, and rotation speed of the jog shuttle 134.

[0014] The computer system 140 includes a computer body 141, a keyboard 142, a mouse 143, and a display 144. Fig. Fig. 4 is a block diagram showing a configuration of a computer main body 141. The computer body 141 includes a CPU 40 serving as a control center, a storage unit 41, a memory 42, interfaces 43 and 44 (in Fig. 4 as “IF”) and a display control unit 45 which controls the view on the display 144.

[0015] Operator instruction information input from the keyboard 142 or the mouse 143 is input to the CPU 40 via the interface 43. The interface 44 is connected to the image pickup unit 120 and the stage 100, carries various control signals from the CPU 40 to the image pickup unit 120 and the stage 100, receives various status information and measurement results from the image pickup unit 120 and the stage 100, and inputs them to the CPU 40.

[0016] The display control unit 45 causes the image captured by the image acquisition unit 120 to be displayed on the display 144. Furthermore, the display control unit 45 causes the display 144 to display the images captured by the image acquisition unit 120, as well as the interface for inputting control instructions to the image measuring device 1 and the interface for the tool for analyzing the captured images.

[0017] The RAM 42 provides a work area for various processing of the CPU 40. The storage unit 41 is configured, for example, by a hard disk drive, a RAM, and the like, and stores programs to be executed by the CPU 40, the image data acquired by the image acquisition unit 120, and other data.

[0018] Based on various types of information input through the respective interfaces, the operator instructions, the measurement definition program (part program) stored in the storage unit 41, and the like, the CPU 40 performs various types of processing, including: controlling the image pickup unit 120, the X-axis drive motor, the Y-axis drive motor, and the Z-axis drive motor, etc., setting the movement path of the image pickup unit 120 and setting the movement speed and exposure time, adjusting the light intensity of the light source 126, image pickup of two-dimensional images by the image pickup unit 120, image stitching processing that stitches together multiple partial images, and analyzing the entire image obtained by image pickup, etc.

[0019] The measurement performed using the image measuring device 1 is explained below. (Basic image measurement)

[0020] First, the operator moves the stage 100 so that the workpiece W enters the imaging field of view by operating the joystick 132 or by controlling the computer system 140. Then, the Z-axis position of the image pickup unit 120 is adjusted so that the workpiece W is in focus. After the workpiece W is in focus, an image for measurement is captured using the image sensor 124. At this time, the coordinates of the stage 100 output by the X-axis encoder 112 and the Y-axis encoder 114 are captured by the computer system 140 along with the captured image and stored in the storage unit 41. Specifically, a pulse is output as a trigger signal to the lock unit 118 at the time the image pickup unit 124 completes capturing an image.The locking unit 118 temporarily stores and holds the position coordinates of each axis at the time of the rising transition of the pulse (i.e., almost simultaneously with the completion of image acquisition). The computer system 140 acquires image signals from the image acquisition unit 124 and acquires the position coordinates when the image is acquired by the locking unit 118, and stores them in association with each other.

[0021] The computer system 140 displays the obtained images for measurement on the display 144 along with the interface of the measuring tool for analyzing the image. Fig. Figure 5 shows an example of a screen display view. This screen display is shown on the display 144 by a program (measurement application software) executing on the CPU 40 of the computer system 140.

[0022] As in Fig. As shown in Figure 5, when the program is executed, the main window MW is displayed on the display 144. Furthermore, a plurality of windows (windows W1 to W8) are displayed within the main window MW. Icons for menus, various operations, and settings are also displayed on top of the main window MW. In this embodiment, an example is shown where eight windows are displayed, but it is also possible to display more than eight windows as needed, or to divide, integrate, or omit windows according to their purpose. The layout of each window can also be freely changed by operator operation.

[0023] The first window W1 displays the image WG of the workpiece W captured by the image capture unit 120. The operator can adjust the position of the image WG of the workpiece W displayed in the first window W1, for example, by operating the mouse 143 or the joystick 132 of the remote box 130. Furthermore, the operator can also expand or reduce the image WG of the workpiece W, for example, by selecting an icon with the mouse 143.

[0024] The second window W2 displays icons of the measuring tools that can be selected by the operator. The measuring tool icons are provided to correspond to the method for designating the measuring points from the image WG of the workpiece W.

[0025] The third window W3 displays icons of functions that can be selected by the operator. The function icons are provided for each measurement method. For example, there are methods for measuring the coordinates of a single point, measuring the length of a straight line, measuring a circle, measuring an ellipse, measuring a square hole, measuring a long hole, measuring the distance, and measuring the tolerance between two lines. The computer system 140 performs measurements of dimensions such as the length of a straight line, the distance between straight lines, and the diameter of a circle, and evaluates deviations (errors) from ideal geometric shapes such as straightness, roundness, and parallelism according to the operator's selection.

[0026] The fourth window W4 displays the instructions that show the operating procedure for the measurement.

[0027] In the fifth window W5, various sliders for controlling the illumination from the image acquisition unit 120 to the workpiece W are displayed. The operator can operate this slider to radiate the desired illumination onto the workpiece W.

[0028] The sixth window W6 displays the XY coordinate values ​​of stage 100. The XY coordinate values ​​displayed in the sixth window W6 are the X-axis coordinate and the Y-axis coordinate of stage 100 relative to a specified coordinate origin.

[0029] The seventh window W7 displays a tolerance assessment result. When a measurement method capable of performing tolerance assessment is selected, the assessment result is displayed in the seventh window W7.

[0030] The eighth window W8 displays a measurement result. When a measurement method that obtains a measurement result through a predefined calculation is selected, the measurement result is displayed in the eighth window W8. The details of the tolerance judgment results for the seventh window W7 and the measurement results for the eighth window W8 are omitted from the drawing. (Appearance test for soldering defects)

[0031] In the image measurement device 1 of the present embodiment, the program (measurement application software) executed by the CPU 40 of the computer system 140 provides, in addition to the basic image measurement described above, a function for performing an appearance inspection focused on soldering defects (hereinafter referred to simply as an appearance inspection). In the following description, unless specifically referenced to the subject of processing, the subject is understood to be the program executed by the CPU 40 of the computer system 140.

[0032] In this system, soldering defects include the following three types of defects. (1) Missing bumps: This is a defect where no bump candidates are present within the test area. (2) Outside the specified range: This is a defect in which the candidate bump is outside the specified value range within the inspection area. (3) Misalignment: This is a defect where the distance from the bump candidate to the field (the shortest distance between the outer edges) is smaller than the threshold.

[0033] The range of design values ​​used to judge whether a value is out of range or not and the threshold values ​​used to judge whether misalignment occurs or not can be changed by the user on the measurement application software screen.

[0034] Fig. Figure 6 shows an example of an appearance inspection screen (hereinafter referred to as an appearance inspection view). The appearance inspection view consists of an image panel P1, a film strip panel P2, a measurement result display window P3, and a control window P4.

[0035] Image field P1 is the area that displays the image for appearance inspection. Image processing and defect assessment are performed on the image displayed in this image field P1 under the conditions set in the control window P4.

[0036] When a dragging operation is performed using the mouse on the image field P1, a rectangular area with a line connecting the dragging start point and end point as its diagonal line can be designated as the inspection area IR for defect judgment on the displayed image. When this dragging operation is performed, an inspection area tool showing the outline of the designated rectangular area is displayed superimposed on the image in the image field P1. The inspection area tool can be selected by clicking, and the size can be changed by dragging the handle that appears when selected. The area tool can also be deleted by pressing the DEL key on the keyboard while the tool is in a selected state.

[0037] The filmstrip area P2 is an area that displays loaded images in thumbnail format for appearance inspection. Double-clicking any image displayed in the filmstrip area P2 displays the image in the image area P1 and becomes the subject of image processing and defect assessment. In the initial state immediately after images are loaded, a predefined image (for example, the image that appears first when sorted by name or by date and time of saving) in the filmstrip area P2 becomes the selected state and is displayed in the image area P1.

[0038] If the image processed and assessed in the image field P1 has a defect, a hatching H is added to the image in the filmstrip field P2 in which the defect is found so that it can be easily distinguished from images in which the defect is not found.

[0039] The measurement result display window P3 is an area that displays a list of defect information when a defect is found in the image displayed in the image field P1. If there are multiple defects, information about all defects is displayed in a list format.

[0040] The control window P4 is the area where the user interface for setting the conditions for image processing and defect assessment performed on the image displayed in the image field P1 is displayed. Using the user interface provided in the control window P4, it is possible to set parameters for image processing (e.g., threshold values ​​for binary conversion, whether brightness values ​​should be inverted or not, etc.) and parameters for defect assessment (e.g., design values ​​for bumps, minimum allowable distance to fields, size of the inspection area tool, etc.). The user interface can be provided as a graphical user interface (GUI) control, such as a slider bar or a switch, in addition to a way that allows direct input of numeric values.In addition, a button B1 for entering a command to perform an appearance inspection, a button B2 for entering a command to perform automatic determination of the inspection area, etc. are also provided in the control window P4.

[0041] Next, the procedure for performing an appearance inspection will be described using the example of determining the inspection area IR by user operation on the screen of the measurement application software with reference to the Fig. 7 to Fig. 9 flowcharts and the one in Fig. The image transition diagram shown in Figure 10 is explained.

[0042] Before starting the appearance test, the user selects the menu for performing the appearance test in the program (measurement application software). In response, the display 144 shows the appearance test view. The program then prompts the user to specify one or more images to be tested. If the user specifies image files in response, the image files are loaded and all loaded images are displayed in the filmstrip field P2, while the first image ((a) in Fig. 10) is displayed in the image field P1.

[0043] The image displayed in image field P1 is subject to appearance inspection. If the user wishes to perform the appearance inspection on an image other than the first image, the user can double-click the desired image in filmstrip field P2 to display the desired image in image field P1. This displays the image to be inspected in image field P1 and starts the appearance inspection.

[0044] When the appearance inspection is started, the program first accepts various condition settings from the user (step S01). Specifically, the program accepts the setting of image processing parameters (binarization threshold, etc.), defect assessment parameters (allowable defect width, height, area, etc.), allowable number of defects, etc. These settings can be changed at any time.

[0045] Then, the inspection area IR is designated on the image WG of the workpiece W displayed in the image field P1 by operating the mouse 143 or the joystick 132 of the remote box 130 (step S02). The method for specifying the target area is arbitrary, but, for example, by dragging the mouse 143 on the image field P1, a rectangular area with a line connecting the drag start point and end point as its diagonal line can be designated as the inspection area IR. For the designated inspection area IR, the inspection area tool is displayed as a rectangular frame superimposed on the image of the workpiece WG in the image field P1.

[0046] When the user specifies the inspection region IR, the inspection region IR can be specified to surround the field PD where the bump BP is to be formed. For example, if the field PD is not completely within the image WG, such as a field PD that crosses the image WG horizontally, an inspection region IR can be specified around the position in the field PD where the bump BP is to be formed. Since the outer edge of the field PD is necessary for assessing misalignment, the inspection region IR should be specified so that at least part of the outer edge of the field PD is included in the inspection region IR.

[0047] The number of inspection areas IR determined for a single image WG is arbitrary. The user can determine inspection areas IR for the number of bumps BP that need to be inspected. The computer system 140 can store the information (position and area) of the inspection areas IR determined as described above in the storage unit 41. The method for using the information about the inspection area IR stored in the storage unit 41 in a subsequent inspection will be described later.

[0048] Then, when the command to execute the appearance inspection is input via the user interface (step S03), the judgment of soldering defects is made for each inspection area IR determined as described above according to the procedure explained below.

[0049] First, the test area IR to be assessed is cut out from the image WG to produce a rectangular miniature image SG (step S04; (b) in Fig. 10). Then, the contour of the field PD in the small piece image SG is obtained (step S05).

[0050] The contour of the field PD in step S05 can be determined, for example, by the Fig. 8 (steps S11-S12). That is, the cut-out small piece image SG is binarized (step S11; (c) in Fig. 10). The threshold value for this binarization is set so that the field PD is black (value 0) and the bump BP and the area where neither the bump BP nor the field PD is provided are white (value 1). When performing binarization, noise removal and other processing can be performed as needed. Next, for the binarized small-piece image SG, the edges (the boundaries between the black and white regions) are detected in step S11, and the outermost edge is obtained as the contour (outer edge) of the field PD (step S12; (d) in Fig. 10).

[0051] With further reference to Fig. 7 After step S05, the contour of the hump BP is obtained in the miniature image SG (step S06).

[0052] The contour of the hump BP in step S06 can be determined, for example, by the Fig. 9 (steps S13-S14). That is, the binarized values ​​(ie, black and white) only within the contour of the field PD obtained in step S12 are inverted (step S13). As a result, as shown in (e) in Fig. 10, the edges forming the boundary between the inside and outside of the field PD are extracted from the small piece image SG. Next, the edges of the small piece SG that was subjected to the processing in step S13 are detected again, and the outermost contour is obtained among them, which is used as the contour (outer edge) of the bump BP (step S14; (f) in Fig. 10).

[0053] With further reference to Fig. 7, based on the contour of the patch PD and the contour of the bump BP obtained in steps S05 to S06, the feature values ​​(e.g., the area and aspect ratio of the bump BP, the distance between the patch PD and the bump BP, etc.) for the patch PD and the bump BP are calculated, and the soldering defect is judged (step S07). Specifically, if the contour of the bump BP is not obtained in step S05, it is judged to be a "missing bump." If the area surrounded by the contour of the bump BP obtained in step S06 is outside the set value range, it is judged to be "out of the predetermined range." When the distance (shortest distance) between the contour of the field PD obtained in step S04 and the contour of the bump BP obtained in step S06 is smaller than the set threshold, it is judged to be “misalignment”.If none of the above criteria is met, it is assumed that there is no soldering defect.

[0054] Then, the judgment result is displayed on the display 144 (step S08). Those judged to be defects by the judgment can be displayed in the measurement result display window P3 and superimposed on the workpiece image WG in the image window P1. The manner of displaying the defect in the image field P1 is arbitrary. For example, for the inspection area IR judged to have a defect, the outline color can be changed, or the bump judged to be defective is displayed by filling the inside of the outline with a conspicuous color (e.g., red) so that the user can easily recognize the detected defect. Fig. 6 shows an example of displaying the contour of the inspection area IR judged to have a defect in a bold line.

[0055] If there are IR inspection areas that have not been inspected (step S09; Yes), the process returns to step S04, and steps S04 to S08 are performed for all IR inspection areas determined in step S02. If there are no IR inspection areas that have not been inspected (step S09; No), the appearance inspection is terminated.

[0056] In this way, it is possible to use an image measuring device to inspect soldering defects, and it is possible to perform measurement and inspection of soldered parts using a single system. (Storage and reuse of test area information)

[0057] In the above example, the inspection area IR was determined by user operation on the measurement application software screen, but the inspection area IR can be determined based on pre-stored inspection area information. The inspection area information includes the location of each inspection area IR (e.g., the location coordinates of the center) for at least the desired number of inspection areas IR and is stored in the storage unit 41. The position of each inspection area IR included in the inspection area information can be absolute coordinates in the coordinate system of the image measuring device 1, but it is preferable to use relative coordinates based on the reference position within the workpiece W. The inspection area information can also include the size of each inspection area IR (e.g., the height and width of a rectangle).The inspection area information can be saved as the information for the inspection area IR specified by the user on the measurement application software screen, or it can be saved as the information for the inspection area IR determined by the automatic determination described below. Alternatively, it can be saved after being edited by the user.

[0058] To apply the inspection area information to the appearance inspection, instead of specifying each inspection area IR by user operation on the screen of the measurement application software as in step S02 of the appearance inspection flow described above, the inspection area information stored in the storage unit 41 is read out and applied to the image to be inspected. When the inspection area information includes only the position information for each inspection area IR and does not include size information, the inspection area IR of a predetermined size (e.g., the size set by the user in the measurement application software) can be specified. This allows the user to specify multiple inspection areas IR at once without having to manually specify each inspection area IR. It is effective when repeated appearance inspections of workpieces W with the same specification are performed.Once the inspection areas IR have been determined, it is possible to judge whether or not there are any defects in each of the determined inspection areas IR in the same manner as the appearance inspection procedure described above.

[0059] In the inspection area information, the conditions for the appearance inspection can be further assigned to each inspection area IR. For example, for each inspection area IR, the range of design values ​​or threshold values ​​used for judgment can be defined, and during an appearance inspection, the presence or absence of defects can be judged using the range of design values ​​or threshold values ​​defined in the inspection area information. It is also possible to define the type of judgment to be applied to each individual inspection area IR (for example, only "missing bump" should be judged). In this way, it is possible to judge whether or not there are defects in a plurality of inspection areas IR under different conditions. (Automatic determination of the test area)

[0060] In the above example, the inspection area IR was specified by the user on the measurement application software screen, but it is also possible to automatically generate the inspection area IR for the image WG of the workpiece W using image processing (automatic inspection area determination). Automatic inspection area determination can be performed on a workpiece for which all protrusions have been confirmed to be correctly formed (reference workpiece) before judging whether there are defects in the appearance inspection (and judging the acceptability of the workpiece W).The inspection area IR automatically determined for the reference workpiece may be stored in the storage unit 41 as inspection area information, and when judging whether or not each workpiece W has a defect (and judging the acceptability of the workpiece W), the inspection area information may be read out and used to determine the inspection area IR.

[0061] The following describes a procedure for automatically determining the test area using the reference workpiece with reference to the Fig. 11 and Fig. 12 and the flow chart shown in Fig. The image transition diagram shown in Figure 13 is explained.

[0062] The automatic determination of the test area is carried out using the image of the reference workpiece (in (a) in Fig. 13) is started, which is in the image field P1 on the Fig. 6 is displayed. Once processing begins, the program first accepts the user's settings for various conditions (step S101). Specifically, the program accepts the setting of image processing parameters (binarization threshold, etc.), defect assessment parameters (allowable defect width, height, area, etc.), allowable number of defects, etc.

[0063] When a command to execute automatic determination is input (step S102), the contours of all bumps BP appearing in the image of the reference workpiece are obtained and recorded in the bump contour list (step S103). The contours of the bumps BP in the image of the reference workpiece can be determined, for example, by the Fig. 12 (steps S111 to S118). That is, the image of the reference workpiece is binarized (step S111; (b) in Fig. 13), the edges of the image binarized in step S111 are detected and the outermost contour is identified (step S112; (c) in Fig. 13). The contour (outer edge) of the field PD is identified at this time. Then, the binary value (i.e., black and white) within the outermost contour identified in step S112 is inverted (step S113). As a result, the edge forming the boundary between the inside and outside of the field PD disappears, as shown in (d) in Fig. 13. Next, the image of the reference workpiece subjected to the processing in step S113 is reprocessed to detect edges, and the outermost contour is obtained (step S114; (e) in Fig. 13).

[0064] Then, for one of the obtained contours that has not yet been judged, the feature values ​​(e.g., the dimensions of the contour, the size of the smallest bounding box, etc.) are calculated (step S115). Further, based on the calculated feature values, it is judged whether the acquired contour is the contour of a bump BP or not, and if it is judged to be the contour of a bump BP (step S116; Yes), the contour is recorded in the bump contour list (step S117). On the other hand, if it is judged not to be the contour of the bump BP (step S116; No), the contour is not recorded in the bump contour list, and the processing advances to step S118. If there are still contours that have not been judged to be bump BP contours (step S118; Yes), the process returns to step S115, and the judgment of whether they are bump BP contours or not is performed for all the contours obtained in step S114.If there are no more contours to be judged (step S118; No), the bump contour list will contain the contours of all bumps BP that appear in the image of the reference workpiece.

[0065] With further reference to Fig. 11, the inspection area IR is determined for all contours recorded in the bump contour list (step S104), and the automatic determination process ends. At this time, the inspection area tool for each of the automatically specified inspection areas IR is displayed in the image field P1, superimposed on the image of the workpiece WG ((f) in Fig. 13). The method for determining the inspection area IR for each contour is arbitrary. For example, a rectangle of a specified size centered on the reference position (e.g., the center or centroid) of each contour can be used as the inspection area IR. Alternatively, the inspection area IR can be a rectangle surrounding each contour at a specified distance.

[0066] In this way, it is possible to automatically determine the inspection area IR for the multiple protrusions BP contained in the image of the reference workpiece. If the inspection area information for the automatically determined inspection area is saved, the same inspection area information can be repeatedly applied to the inspection area determination for the images of individual workpieces W to be inspected. In the above example, the automatic inspection area determination was applied to the image of the reference workpiece, but it is also possible to apply the automatic inspection area determination to the image of each individual workpiece to be inspected.

[0067] According to the appearance inspection described above, the image measuring device 1 can be used to inspect soldering defects, and measurement and inspection of soldering portions can be performed in a single system. Furthermore, workpieces with a large number of soldering portions to be inspected can be efficiently inspected. (Variation of the embodiment)

[0068] The present invention is not limited to the examples of the above embodiments, and any modification, improvement, and the like are included in the present invention insofar as the object of the present invention can be achieved. For example, the method for obtaining the contours of field PD and bump BP is not limited to the method disclosed in the above embodiments (ie, the method disclosed in Fig. 8 and Fig. 9) and any method can be used.

[0069] Furthermore, an invention in which a person skilled in the art appropriately adds, deletes, or changes the design of the above-described embodiment or a concrete example thereof is also included in the scope of the present invention as long as it has the gist of the present invention.

[0070] With respect to the embodiments including the above examples, the following appendices are further disclosed.

[0071] (Appendix 1) Appearance inspection method that inspects solder bumps formed on patches of the inspection target based on an image of an inspection target, comprising: an inspection area determining step for determining the inspection area in the image of the inspection target; a field area detecting step for detecting a field area included in the test area; a bump region detecting step of detecting a bump region included in the test region; a defect judgment step for judging the presence or absence of the defect based on the detected field area and / or bump area.

[0072] (Annex 2) Appearance inspection method according to Annex 1, whereby in the inspection area determination step several inspection areas are determined in the image of the inspection target, and wherein the field region detecting step, the bump region detecting step, and the defect judging step are performed for each of a plurality of specific inspection regions.

[0073] (Annex 3) Appearance testing procedure according to Annex 1 or 2, whereby in the test area determination step the test area is determined on the basis of specified test area information, and the test area information shall include at least information indicating the location of the test area.

[0074] (Annex 4) Appearance testing procedure according to Annex 3, where the test area information further includes information indicating the size of the test area.

[0075] (Annex 5) Appearance inspection method according to Annex 3, wherein the inspection area information further includes information related to the assessment criteria used in the defect assessment step.

[0076] (Annex 6) Appearance inspection method according to Annex 1, further comprising a step of cutting out a miniature image of the inspection area from the image of the inspection target, and wherein the field area detecting step comprises: a step to binarize the small-piece image; a step for detecting edges in the binarized small-piece image; and a step of obtaining the outermost edge of the detected edges as the contour of the field area, and wherein the bump region detecting step comprises: a step of inverting the binarized brightness values ​​within the contour of the field area for the binarized small-scale image; a step for detecting edges in the small-scale image in which the brightness values ​​within the contour of the field area were inverted; and a step of obtaining the outermost edge of the detected edges as the contour of the bump region.

[0077] (Annex 7) Appearance inspection method according to Annex 1 or 2, where the defect to be assessed in the defect assessment step includes missing protuberances and / or protuberances outside the specified range and / or misalignment.

[0078] (Annex 8) A program for causing a computer to carry out the appearance testing procedure set out in Annex 1 or 2, and a non-volatile storage medium on which the program is stored.

[0079] (Annex 9) Inspection area determination method used in an appearance inspection method that inspects solder bumps formed on patches of an inspection target based on an image of the inspection target, comprising: a bump region detecting step for detecting one or more bump regions included in the image; and a test area determining step for determining a test area for each of the one or more detected bump areas.

[0080] (Annex 10) The test area determining method according to Annex 9, further comprising a test area information storing step that stores test area information including at least information indicating the location of the test area determined in the test area determining step.

[0081] (Annex 11) A program for causing a computer to carry out the test area determination procedure according to Annex 9 or 10 and a non-volatile storage medium on which the program is stored. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] JP 2020-071106

[0002]

Claims

[1] An appearance inspection method that inspects solder bumps formed on patches of an inspection target based on an image of an inspection target, comprising: an inspection area determining step for determining the inspection area in the image of the inspection target; a field area detecting step for detecting a field area included in the test area; a bump region detecting step of detecting a bump region included in the test region; a defect judgment step for judging the presence or absence of the defect based on the detected field area and / or bump area. [2] The appearance inspection method according to claim 1, wherein in the inspection area determining step, a plurality of inspection areas are determined in the image of the inspection target, and wherein the field area detecting step, the bump area detecting step and the defect judging step are performed for each of a plurality of determined inspection areas. [3] The appearance inspection method according to claim 1 or 2, wherein in the inspection area determining step, the inspection area is determined based on predetermined inspection area information, and the inspection area information includes at least information indicating the location of the inspection area. [4] The appearance inspection method according to claim 3, wherein the inspection area information further includes information indicating the size of the inspection area. [5] The appearance inspection method according to claim 3, wherein the inspection area information further includes information related to the judgment criteria used in the defect judgment step. [6] The appearance inspection method according to claim 1, further comprising a step of cutting out a small piece image of the inspection area from the image of the inspection target, and wherein the field area detecting step comprises: a step to binarize the small-piece image; a step for detecting edges in the binarized small-piece image; and a step of obtaining the outermost edge of the detected edges as the contour of the field area, and wherein the bump region detecting step comprises: a step of inverting the binarized brightness values ​​within the contour of the field area for the binarized small-scale image; a step for detecting edges in the small-scale image in which the brightness values ​​within the contour of the field area were inverted; and a step of obtaining the outermost edge of the detected edges as the contour of the bump region. [7] The appearance inspection method according to claim 1 or 2, wherein the defect to be judged in the defect judging step includes missing bumps and / or outside the predetermined range and / or misalignment. [8] A non-volatile storage medium having stored thereon a program for causing a computer to execute the appearance checking method according to claim 1 or 2. [9] An inspection area determining method used in an appearance inspection method that inspects solder bumps formed on patches of an inspection target based on an image of an inspection target, comprising: a bump region detecting step for detecting one or more bump regions included in the image; and a test area determining step for determining a test area for each of the one or more detected bump areas. [10] The inspection area determining method according to claim 9, further comprising an inspection area information storing step that stores inspection area information including at least information indicating the location of the inspection area determined in the inspection area determining step. [11] A non-volatile storage medium having stored thereon a program for causing a computer to execute the test area determining method according to claim 9 or 10.

Images