A method and system for detecting appearance of gold wire bonding process

Abstract

The present invention relates to the field of machine vision inspection technology, and in particular to a method and system for inspecting the appearance of a gold wire bonding process. The present invention uses different light source modes to obtain a reference image of the gold wire and multiple background images. In the reference image, a dividing line is set based on the contrast between the gold wire and the background object. The gold wire in the reference image is divided into multiple gold wire segments based on the dividing line. For each gold wire segment, the gold wire segment with the highest contrast is selected from the background image and the reference image. The selected gold wire segments are spliced ​​to obtain a gold wire to be tested that meets the contrast requirements. Furthermore, according to the set inspection requirements, the gold wire to be tested is inspected for defects such as spacing, bending, crossing, and broken wires. The present invention achieves fully automatic machine vision inspection of optical communication products through flexible light source configuration and inspection methods. The present invention has high inspection adaptability and strong anti-interference characteristics, good inspection stability and product detection rate, and realizes automated inspection of the gold wire bonding process.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a method and system for detecting the appearance of a gold wire bonding process. Background Art

[0002] Photodetectors, core components of optical communication receiver products, ship in the tens of millions annually. To accommodate large-scale production, mainstream processes have become fully automated. However, auxiliary processes, such as inspection, still rely on manual visual inspection, which is not only inefficient but also prone to missed detections due to labor variability and visual fatigue. Machine vision is well-established in defect detection in the semiconductor and PCB (Printed Circuit Board) industries. However, direct application of these technologies to optical communication products suffers from high false positive and missed detection rates, and poor adaptability of product inspection procedures. This is due to the significant process and material variability of optical communication products. For example, within a single product, component heights vary widely, multiple candidate components exist, and material color tolerances vary widely. Complex background textures can interfere with the product, resulting in poor gold wire contrast. Wire arcs vary in variety, and bending inflection points fluctuate significantly. These variations are precisely where machine vision inspection equipment and methods are most sensitive, necessitating customized inspection systems and methods tailored to the specific characteristics of optical communication products.

[0003] In view of this, overcoming the defects of the prior art is an urgent problem to be solved in this technical field. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method and system for appearance inspection of gold wire bonding process, so as to solve the problem that when the existing machine vision inspection method is directly applied to defect detection of optical communication products, the contrast of the obtained gold wire is poor due to the complex background texture of the product and the large interference, resulting in high false alarm rate and missed alarm rate when inspecting the gold wire.

[0005] In a first aspect, the present invention provides a method for appearance inspection of a gold wire bonding process, comprising:

[0006] Acquire a reference image of the gold wire using the first light source mode, and switch the light source mode of the composite light source 2 according to the type of the background object to acquire at least one background image, wherein both the reference image and the background image include the gold wire and the background object;

[0007] In the reference image, a cutting line is set according to the contrast between the gold wire and the background object;

[0008] Dividing the gold wire in the reference image into a plurality of gold wire segments according to the dividing line, selecting a gold wire segment with the highest contrast from the background image and the reference image for each gold wire segment, and splicing the selected gold wire segments to obtain a gold wire to be tested that meets the contrast requirement;

[0009] Perform defect detection on the gold wire to be tested.

[0010] Furthermore, the background objects include a lead frame 3, a chip 4, and a capacitor 5; the composite light source 2 includes a coaxial light 24, an upper ring light 23, a middle ring light 22, and a lower ring light 21, and the upper ring light 23, the middle ring light 22, and the lower ring light 21 are stacked in sequence;

[0011] The step of acquiring a reference image of the gold wire by using the first light source mode and switching the light source mode of the composite light source 2 according to the type of the background object to acquire at least one background image includes:

[0012] A reference image of the gold wire is obtained by combining the middle ring light 22 and the lower ring light 21 to improve the contrast between the gold wire, the solder joint and the background object;

[0013] When the background object is the chip 4, the coaxial light 24 is used to obtain the background image to improve the contrast between the gold wire and the chip 4;

[0014] When the background object is the capacitor 5, the background image is obtained by combining the coaxial light 24 and the upper ring light 23 to improve the contrast between the gold wire and the capacitor 5;

[0015] When the background object is the lead frame 3 , a background image is acquired by combining the coaxial light 24 and the lower ring light 21 to improve the contrast between the gold wire and the lead frame 3 .

[0016] Furthermore, in the reference image, setting the dividing line according to the contrast between the gold line and the background object includes:

[0017] Analyzing the reference image to obtain a contrast ratio between the gold wire and the background object;

[0018] The position where the contrast is less than the set contrast threshold is used as the dividing line.

[0019] Furthermore, performing defect detection on the gold wire to be tested includes:

[0020] Performing a rough positioning of the gold wire to be tested according to the cutting line to obtain an initial gold wire area;

[0021] The initial gold wire area is accurately located by grayscale threshold and contrast threshold to obtain a target gold wire area, and the skeleton of the target gold wire area is extracted to generate a gold wire center line;

[0022] The defects of the gold wire are judged by the geometric characteristics and grayscale information of the gold wire centerline.

[0023] Furthermore, performing rough positioning of the gold wire to be tested according to the cutting line to obtain an initial gold wire area includes:

[0024] Obtaining the first end solder joint and the tail end solder joint of the gold wire to be tested;

[0025] Obtain the target point before each cutting line in sequence, and connect the target point to the tail end weld point to obtain a transition line; wherein, for the cutting line closest to the head end weld point, the target point before the cutting line is the head end weld point, and the target points corresponding to other cutting lines are the positioning points located before the cutting line;

[0026] Obtain an intersection point between the transition line and the dividing line, use the intersection point as a normal line of the transition line, and select at least one transition point on the normal line;

[0027] Connecting the transition point to the target point and the tail end welding point to establish a first image area and a second image area, respectively, calculating the grayscale g(A1) of the first image area and the grayscale g(A2) of the second image area, and calculating the average grayscale gm of the transition point based on the grayscale g(A1) and the grayscale g(A2);

[0028] The transition point where the average grayscale gm meets the preset conditions is used as the positioning point of the gold wire to be tested;

[0029] Connect the first end solder joint, all the positioning points and the tail end solder joint in sequence to form the initial gold wire area.

[0030] Furthermore, the step of connecting the transition point to the target point and the tail welding point to establish a first image area and a second image area, respectively calculating a grayscale g(A1) of the first image area and a grayscale g(A2) of the second image area, and calculating an average grayscale gm of the transition point based on the grayscale g(A1) and the grayscale g(A2) includes:

[0031] The distance between the transition point and the positioning point is used as the first major axis, and the distance between the transition point and the tail end welding point is used as the second major axis;

[0032] A first image region is created by taking the first major axis as the length of the rectangular region and the width of the gold line as the width of the rectangular region; a second image region is created by taking the second major axis as the length of the rectangular region and the width of the gold line as the width of the rectangular region;

[0033] Calculate the grayscale g(A1) of the first image area and the grayscale g(A2) of the second image area respectively, calculate the area A1 of the first image area and the area A2 of the second image area respectively, obtain the grayscale sum of the grayscale g(A1) and the grayscale g(A2), obtain the area sum of the area A1 and the area A2, and take the ratio between the grayscale sum and the area sum as the average grayscale gm of the transition point.

[0034] Furthermore, the method of using the transition point where the average grayscale gm satisfies a preset condition as the positioning point of the gold wire to be tested includes:

[0035] When the average grayscale value gm of the transition point is greater than or equal to the grayscale threshold gt, the average grayscale value gm-1 of the previous transition point and the average grayscale value gm+1 of the next transition point are obtained;

[0036] If the difference between the average gray value gm and the average gray value gm-1 and the difference between the average gray value gm and the average gray value gm+1 are both greater than the set contrast threshold gc, the transition point is used as the positioning point of the gold wire to be tested.

[0037] Furthermore, the method of accurately locating the initial gold wire area by using a grayscale threshold and a contrast threshold to obtain a target gold wire area, and extracting a skeleton of the target gold wire area to generate a gold wire center line includes:

[0038] A search area is generated along the initial gold wire area. Based on the different segmented images of the gold wire to be tested, a bright area or a dark area is selected as the target area. The skeleton of the gold wire to be tested is determined in the target area. After processing the skeleton, the center line of the gold wire is precisely located. Furthermore, the method of judging the defects of the gold wire based on the geometric characteristics and grayscale information of the gold wire center line includes:

[0039] The gold wires to be tested are grouped, and the intersection of the center lines of the gold wires in each group is calculated in turn. If the intersection does not meet the preset requirements, it is determined that the gold wires to be tested have an abnormal intersection;

[0040] According to the grouping of the gold wires to be tested, the distance between the center lines of the gold wires in each group is calculated in sequence. If the distance does not meet a preset value, it is determined that the spacing between the gold wires to be tested is abnormal;

[0041] Generate an image area of ​​the gold wire to be tested along the center line of the gold wire, and then generate two tolerance lines parallel to the straight line between the first and tail solder joints of the gold wire to be tested. If the image area intersects with the tolerance lines, it is determined that the gold wire to be tested is abnormally bent;

[0042] Calculating the centerline length of the gold wire, and if the length does not meet the preset requirement, determining that the gold wire to be tested is broken;

[0043] Calculating the centerline length of the gold wire to be tested within the head end solder joint and the tail end solder joint, and if the length does not meet the preset requirements, determining that the gold wire to be tested is broken;

[0044] Calculate the intersection of the annular area near the beginning and end and the gold wire area, extract the gold wire outline in the area, and determine the grayscale and features of the outline. If it does not meet the preset requirements, it is determined to be a gold wire break anomaly.

[0045] In a second aspect, the present invention further provides an appearance inspection system for a gold wire bonding process, the appearance inspection system being used to perform the appearance inspection method of the first aspect, the appearance inspection system comprising a visual inspection module and a controller, the visual inspection module comprising a camera 1 and a composite light source 2, the composite light source 2 comprising a coaxial light 24, an upper ring light 23, a middle ring light 22, and a lower ring light 21, the upper ring light 23, the middle ring light 22, and the lower ring light 21 being stacked in sequence;

[0046] The composite light source 2 is used to provide different light source modes;

[0047] The controller is used to switch the light source mode of the composite light source 2 according to the type of the detection object;

[0048] The camera 1 is used to obtain a reference image of the gold wire in a first light source mode and to obtain at least one background image in a light source mode corresponding to the background object;

[0049] The controller is used to set a dividing line in the reference image according to the contrast between the gold wire and the background object; and is also used to divide the gold wire in the reference image into multiple gold wire segments according to the dividing line. For each gold wire segment, the gold wire segment with the highest contrast is selected from the background image and the reference image, and the selected gold wire segments are spliced ​​to obtain a gold wire to be tested that meets the contrast requirements, so as to perform defect detection on the gold wire to be tested.

[0050] An embodiment of the present invention uses different light source modes to obtain a reference image of a gold wire and multiple background images. In the reference image, a dividing line is set based on the contrast between the gold wire and the background object. The gold wire in the reference image is then divided into multiple gold wire segments based on the dividing line. For each gold wire segment, the gold wire segment with the highest contrast is selected from the background image and the reference image. The selected gold wire segments are then spliced ​​together to obtain a gold wire to be tested that meets the contrast requirements. The gold wire to be tested is then inspected for defects such as spacing, bends, intersections, and broken wires according to the set inspection requirements. The present invention achieves fully automatic machine vision inspection of optical communication products through flexible light source configuration and inspection methods. The system has high inspection adaptability and strong anti-interference properties, as well as good inspection stability and product detection rate, thus enabling automated inspection of the gold wire bonding process. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0052] Figure 11 is a flow chart of a method for appearance inspection of a gold wire bonding process provided by an embodiment of the present invention;

[0053] Figure 2 Schematic diagram of the structure of a product to be tested in a method for appearance inspection of a gold wire bonding process provided by an embodiment of the present invention; Figure 3 1 is a schematic structural diagram of a gold wire bonding process appearance inspection system provided by an embodiment of the present invention;

[0054] Figure 4 1 is a schematic diagram of a specific flow chart of step 101 in a method for appearance inspection of a gold wire bonding process provided by an embodiment of the present invention;

[0055] Figure 5 Schematic diagram of gold wire cutting in a method for appearance inspection of a gold wire bonding process provided by an embodiment of the present invention;

[0056] Figure 6 1 is a schematic diagram of a specific flow chart of step 104 in a method for appearance inspection of a gold wire bonding process provided by an embodiment of the present invention;

[0057] Figure 7 1 is a schematic diagram of a specific flow chart of step 1041 in a method for appearance inspection of a gold wire bonding process provided by an embodiment of the present invention;

[0058] Figure 8 Schematic diagram of gold wire thickness positioning in a method for appearance inspection of a gold wire bonding process provided by an embodiment of the present invention;

[0059] Figure 9 This is a specific flow chart of step 10414 in the appearance inspection method provided by an embodiment of the present invention.

[0060] The accompanying drawings are marked as follows: camera 1; composite light source 2; lower ring light 21; middle ring light 22; upper ring light 23; coaxial light 24; lead frame 3; chip 4; capacitor 5. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0062] In the description of the present invention, the terms "inside", "outside", "longitudinal", "lateral", "upper", "lower", "top", "bottom", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and do not require that the present invention must be constructed and operated in a specific orientation. Therefore, they should not be understood as limitations on the present invention.

[0063] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0064] Embodiment 1:

[0065] See Figure 1 , Embodiment 1 of the present invention provides a method for appearance inspection of a gold wire bonding process:

[0066] Step 101: Acquire a reference image of the gold wire using a first light source mode, and switch the light source mode of the composite light source 2 according to the type of the background object to acquire at least one background image, wherein both the reference image and the background image contain the gold wire and the background object.

[0067] Step 102: In the reference image, a dividing line is set according to the contrast between the gold wire and the background object.

[0068] Step 103: Divide the gold wire in the reference image into multiple gold wire segments according to the dividing line. For each gold wire segment, select the gold wire segment with the highest contrast from the background image and the reference image, and splice the selected gold wire segments to obtain a gold wire to be tested that meets the contrast requirement.

[0069] Step 104: performing defect detection on the gold wire to be tested.

[0070] In this embodiment, a reference image of the gold wire and multiple background images are obtained using different light source modes. In the reference image, a dividing line is set based on the contrast between the gold wire and the background object. The gold wire in the reference image is then divided into multiple gold wire segments based on the dividing line. For each gold wire segment, the gold wire segment with the highest contrast is selected from the background image and the reference image. The selected gold wire segments are then spliced ​​to obtain a gold wire to be tested that meets the contrast requirements. The gold wire to be tested is then inspected for defects such as spacing, bends, intersections, and broken wires according to the set inspection requirements. This invention achieves fully automated machine vision inspection of optical communication products through flexible light source configuration and inspection methods. It has high inspection adaptability and strong anti-interference properties, as well as good inspection stability and product detection rate, thus achieving automated inspection of the gold wire bonding process.

[0071] In order to obtain a reference image and background image with good contrast, it is necessary to take targeted lighting photos according to different background objects, such as Figure 2 As shown, the background objects include a lead frame 3, a chip 4 and a capacitor 5; Figure 3 The composite light source 2 includes a coaxial light 24, an upper ring light 23, a middle ring light 22 and a lower ring light 21, and the upper ring light 23, the middle ring light 22 and the lower ring light 21 are stacked in sequence.

[0072] Considering the factors such as the large variety of product components, the variable curvature of the gold wire and the complex texture of the background object, Figure 4 , step 101 specifically includes:

[0073] Step 1011: Acquire a reference image of the gold wire using a combination of the middle ring light 22 and the lower ring light 21 to improve the contrast between the gold wire, the solder joint, and the background object.

[0074] Step 1012 : When the background object is the chip 4 , a background image is acquired using the coaxial light 24 to increase the contrast between the gold wire and the chip 4 .

[0075] Step 1013 : When the background object is the capacitor 5 , a background image is acquired by combining the coaxial light 24 and the upper ring light 23 to improve the contrast between the gold wire and the capacitor 5 .

[0076] Step 1014 : When the background object is the lead frame 3 , a background image is acquired by combining the coaxial light 24 and the lower ring light 21 to improve the contrast between the gold wire and the lead frame 3 .

[0077] In an optional embodiment, the lower ring light 21 is red light and is positioned at the bottom of the composite light source 2, 1 mm above the product to be tested, with an incident angle of 20°. This lower ring light can project the reflected light from the gold wire into the lens, facilitating imaging by the camera 1. The middle ring light 22 is blue light and is positioned above the lower ring light 21, with an incident angle of 45°. The upper ring light 23 is white light and is positioned above the middle ring light 22, with an incident angle of 60°. The coaxial light 24 is white light and is positioned above the upper ring light 23. The aforementioned height and angle values ​​are merely preferred values; in other embodiments, these values ​​can be adjusted adaptively based on the product to be tested.

[0078] In step 1011, the gold wire is illuminated by a combination of middle ring light 22 and lower ring light 21, which can form a reference image of the gold wire with good contrast. In step 1012, the background object is chip 4. Since chip 4 is a mirror surface and generally has texture inside, the coaxial light 24 is used to illuminate the chip 4, which can clearly show the texture of chip 4. In step 1013, the background object is capacitor 5. Since capacitor 5 has both mirror surfaces and rough surfaces but no texture, the coaxial light 24 and upper ring light 23 are used to illuminate the capacitor 5, which can form a background image with clear edge imaging and obvious contrast with the other background objects. In step 1014, the background object is lead frame 3. Since lead frame 3 has many textures, the coaxial light 24 and lower ring light 21 are used to illuminate the lead frame 3 with uniform texture. In addition, the brightness of the composite light source 2 is adjustable and can be adjusted according to actual conditions. By using the above-mentioned method of lighting and photographing different background objects, images with better contrast can be obtained, which facilitates later machine vision inspection.

[0079] It should be noted that when the coaxial light 24 is involved in lighting, the gold line in the captured background image will appear as black with a lower grayscale value; when the coaxial light 24 is not involved in lighting, that is, when various types of coaxial light 24 are combined for lighting, the gold line in the captured reference image will appear as white with a higher grayscale value.

[0080] In order to obtain a gold wire image with good contrast, setting a dividing line according to the contrast between the gold wire and the background object in the reference image includes:

[0081] The reference image is analyzed to obtain the contrast between the gold line and the background object; and the position where the contrast is less than a set contrast threshold is used as the dividing line.

[0082] Since the gold wire has a certain curvature and span, the change in curvature causes the light to not be reflected into the camera 1, and the gold wire cannot be illuminated in its entirety. The span of the gold wire causes a lot of background interference with the gold wire, which makes it difficult to obtain a complete gold wire image with good contrast in one picture. Therefore, it is necessary to divide the reference picture into multiple segments based on the contrast difference of the gold wire path. For each segment, select the image with good gold wire contrast in the reference picture and the background picture for splicing to obtain a complete gold wire image with good contrast. The segmentation line can be manually specified based on the reference image and background image, or it can be selected by the program. For example Figure 5 As shown, the gold wire is divided into four segments (segment 11, segment 12, segment 14, and segment 15) by the cutting line. Segment 11 and segment 15 are closer to the chip 4 or the capacitor 5, and the background on segment 12 and segment 14 contains glue.

[0083] For example, when the contrast of the gold wire segment in the reference image is good, this gold wire segment can be selected as the reference image; when the gold wire segment is above the chip 4, since the chip 4 will cause the light to form a mirror reflection, when the coaxial light 24 is used for illumination, the chip 4 is bright and the gold wire is dark. Therefore, in the background image formed in step 1012, the gold wire color is darker, black, and the grayscale value is lower, and the chip 4 color is brighter, white, and the grayscale value is higher, so that the gold wire has a good contrast. Therefore, the gold wire segment passing above the chip 4 can be selected as the background image formed in step 1012. Background picture; when there is glue on the gold wire background, when the upper ring light 23 is combined with the coaxial light 24 to illuminate, the glue will be brightened and the gold wire will become dark. The gold wire color is darker, black, and the grayscale value is lower. The background color is brighter, white, and the grayscale value is higher. Therefore, the gold wire segment with glue in the background can choose the background picture formed in step 1013; that is, each gold wire segment uses the part with better contrast in the background picture and the reference picture, and then splices them to obtain a gold wire image with better contrast. In this gold wire image, part of the gold wire is displayed in white and part of the gold wire is displayed in black.

[0084] In this embodiment, the appearance inspection method further includes solder joint positioning and defect detection. Prior to performing defect detection on the gold wire under test, the solder joint under test is positioned and defect detected. Specifically, the component under test is first matched, and then a solder joint whose solder joint reference is the component under test is obtained. The solder joint under test is then searched for based on its coordinates relative to the component under test to obtain the solder joint under test. Once the solder joint under test is obtained, a determination is made as to whether the solder joint under test has a defect based on its diameter and offset distance.

[0085] When searching for solder joints, the solder joints are searched in the form of a rectangular search box. When the rectangular search box finds an area where a solder joint may exist, the area is subjected to grayscale feature calculation, threshold segmentation, morphological processing, and fitting into a circumscribed circle. The area and roundness of the fitted circumscribed circle are then used to determine whether it is a solder joint.

[0086] When matching the components to be tested, the components to be tested are matched according to a preset matching method. Shape matching is used for components with patterns, such as chip 4; edge matching with scaling is used for components without patterns due to dimensional tolerance issues, such as capacitor 5.

[0087] Component information, solder joint information and gold wire information are all stored in the visual inspection template. The visual inspection template is independently archived according to the product number and can be used for the appearance inspection method to detect different products.

[0088] The component information stores the coordinates and matching methods of the components for matching the components to be tested; the solder point information stores the solder point reference and the component to which the solder point belongs. The solder point reference includes the component that the solder point follows and the coordinates of the solder point relative to the component, which is used to search for the solder point to be tested based on the component to be tested; the solder point information also stores the solder point diameter and the allowable range of solder point offset, which is used to determine whether the solder point to be tested has defects; the gold wire information includes gold wire pairing information, which contains the head end solder point and tail end solder point information of the gold wire; the gold wire information also includes gold wire grouping information, which is convenient for grouping the gold wires for defect judgment; the gold wire information also includes gold wire spacing, gold wire bending range, and gold wire crossing information, which is used to determine whether the gold wire has defects.

[0089] In this embodiment, combined with Figure 6 The specific process of performing defect detection on the gold wire to be tested includes:

[0090] Step 1041: performing rough positioning of the gold wire to be tested according to the dividing line to obtain an initial gold wire area.

[0091] Step 1042: accurately locate the initial gold wire area through grayscale threshold and contrast threshold to obtain the target gold wire area, and extract the skeleton of the target gold wire area to generate the gold wire center line.

[0092] Step 1043: Determine the defects of the gold wire based on the geometric features and grayscale information of the gold wire centerline.

[0093] Combine Figure 7 Step 1041 specifically includes the following process:

[0094] Step 0411: Obtain the first end solder joint and the tail end solder joint of the gold wire to be tested.

[0095] Step 0412: Obtain the target point before each cutting line in turn, and connect the target point with the tail end weld point to obtain a transition line; among which, for the cutting line closest to the head end weld point, the target point before the cutting line is the head end weld point, and the target points corresponding to other cutting lines are the positioning points located before the cutting line.

[0096] Step 0413: Obtain the intersection of the transition line and the dividing line, use the intersection as the normal of the transition line, and select at least one transition point on the normal.

[0097] Step 0414: Connect the transition point to the target point and the tail welding point respectively to establish the first image area and the second image area respectively, calculate the grayscale g(A1) of the first image area and the grayscale g(A2) of the second image area respectively, and calculate the average grayscale gm of the transition point based on the grayscale g(A1) and the grayscale g(A2).

[0098] Step 0415: The transition point where the average grayscale gm meets the preset conditions is used as the positioning point of the gold wire to be tested.

[0099] Step 0416: Connect the first end solder joint, all the positioning points and the tail end solder joint in sequence to form the initial gold wire area.

[0100] Specific, combined Figure 8 , there are three dividing lines A, B and C in the figure, P1 is the head end welding point, P2 is the tail end welding point; first connect P1 and P2 to get the transition line P2P1; the dividing line corresponding to P1 is the dividing line A, take the intersection point Pm1 of the transition line P2P1 and the dividing line A, make the normal of the transition line P2P1 through Pm1, and take Pm1 as the transition point; connect Pm1 with P1 and P2 respectively to establish the first image area and the second image area; calculate the grayscale g(A1) of the first image area and the grayscale g(A2) of the second image area respectively, calculate the average grayscale gm of the transition point Pm1 according to the grayscale g(A1) and the grayscale g(A2); and calculate the average grayscale gm of the average grayscale gm The grayscale value and the coordinates of the transition point are recorded in the grayscale value set, and it is determined whether the position of the current transition point Pm1 meets the preset conditions. The transition point that meets the preset conditions is used as the positioning point Pc1. Otherwise, the transition point Pm1 is moved along the normal on one side of the transition line, one pixel unit at a time, and the symmetrical point Pm1' of the transition point Pm1 with respect to the transition line is calculated at the same time, and the larger average grayscale value of Pm1 or Pm1' is taken for comparison each time until the position of the current transition point Pm1 or Pm1' meets the preset conditions; if the positioning point is not found after moving a predetermined number of times, the transition point corresponding to the largest average grayscale value in the grayscale value set is taken as the positioning point.

[0101] After finding Pc1, connect Pc1 and P2 to obtain the transition line P2Pc1; take the intersection Pm2 of the transition line P2Pc1 and the dividing line B, make the normal of the transition line P2Pc1 through Pc1, and take Pm2 as the transition point; connect Pm2 with P1 and P2 respectively to establish the first image area and the second image area; Pm2 moves along the normal, and judges whether the average grayscale of the grayscale g(A1) of the first image area and the grayscale g(A2) of the second image area meet the requirements, until Pc2 is obtained; the method of calculating Pc3 is the same as the above-mentioned method of calculating Pc1 and Pc2, and will not be repeated.

[0102] Combine Figure 9 Step 0414 specifically includes the following process:

[0103] Step 4141: The distance between the transition point and the positioning point is used as the first long axis, and the distance between the transition point and the tail end welding point is used as the second long axis.

[0104] Step 4142: Create a first image area with the first major axis as the length of the rectangular area and the width of the gold line as the width of the rectangular area; create a second image area with the second major axis as the length of the rectangular area and the width of the gold line as the width of the rectangular area.

[0105] Step 4143: Calculate the grayscale g(A1) of the first image area and the grayscale g(A2) of the second image area respectively, calculate the area A1 of the first image area and the area A2 of the second image area respectively, obtain the grayscale sum of the grayscale g(A1) and the grayscale g(A2), obtain the area sum of the area A1 and the area A2, and use the ratio between the grayscale sum and the area sum as the average grayscale gm of the transition point.

[0106] Combine Figure 8 , after connecting Pm1 with P1 and P2 respectively, straight lines P1Pm1 and P2Pm1 are obtained; a rectangular area with P1Pm1 as the long side and the width of the gold wire to be measured is established as the first image area, and a rectangular area with P1Pm2 as the long side and the width of the gold wire to be measured is established as the second image area;

[0107] The average grayscale gm of the transition point is calculated by the grayscale g(A1) of the first image area and the grayscale g(A2) of the second image area, the area A1 of the first image area and the area A2 of the second image area. The specific formula for calculating the average grayscale gm of the transition point is:

[0108] gm=(g(A1)+g(A2))÷(A1+A2)

[0109] In step 0415, the process of using the transition point where the average grayscale gm meets the preset conditions as the positioning point of the gold wire to be tested includes:

[0110] When the average grayscale value gm of a transition point is greater than or equal to the grayscale threshold gt, the average grayscale value gm-1 of the previous transition point and the average grayscale value gm+1 of the next transition point are obtained. If the difference between the average grayscale value gm and the average grayscale value gm-1, and the difference between the average grayscale value gm and the average grayscale value gm+1, are both greater than the set contrast threshold gc, the transition point is used as the positioning point of the gold wire to be tested. The grayscale threshold gt is not a fixed value; it is determined by the image of the gold wire to be tested and can be set by the program or manually.

[0111] In step 1042, the initial gold wire area is accurately located by using a grayscale threshold and a contrast threshold to obtain a target gold wire area, and the skeleton of the target gold wire area is extracted to generate a gold wire center line, including:

[0112] A search area is generated along the initial gold wire region. Based on the different segmented gold wire images and the color variations of the gold wire in the images, a bright or dark area is selected as the target area. The skeleton of the gold wire to be tested is determined within the target area. After processing the skeleton, the gold wire centerline is precisely located. Specifically, because the gold wire centerline may be above or below the initial gold wire region, it is necessary to increase the width of the initial gold wire region to a certain range, with the initial gold wire region as the center, to generate an expanded search area. The gold wire centerline is then precisely located within the expanded search area.

[0113] Because when coaxial white light is used, the gold wire appears black with a lower grayscale value, and the background appears white with a higher grayscale value; when annular light combination is used, the gold wire appears white with a higher grayscale value, and the background appears black with a lower grayscale value; the image of the gold wire to be tested is composed of multiple images, part of the gold wire appears black, and part of the gold wire appears white. Therefore, when accurately positioning the gold wire, it is necessary to select a bright area or a dark area as the target area according to the change of the gold wire color in the image of the gold wire to be tested, and search for the gold wire skeleton. The skeleton is the center line outline of the gold wire. Specifically, when the gold wire color is darker, black, the dark area is selected as the target area. When the gold wire color is brighter, white, the bright area is selected as the target area. In this way, the image outline of the center line of the gold wire can be obtained in the image of the gold wire to be tested.

[0114] After determining the target area, due to the large difference in contrast between the gold wire image area and the background image, the closer the part of the target area is to the gold wire center line, the greater the contrast difference between it and the background image; by comparing the contrast between the target area image and the background image, the image contour of the gold wire center line in the target area can be obtained; after obtaining the image contour of the gold wire center line, the image contour of the gold wire center line is screened and smoothed to obtain the precisely positioned gold wire center line.

[0115] In this embodiment, the defects of the gold wire are determined by the geometric characteristics and grayscale information of the gold wire centerline. The defect types include poor gold wire crossing abnormality, gold wire spacing abnormality, gold wire bending abnormality, gold wire breakage, gold wire missing, etc. The detection method of each defect type is as follows:

[0116] The gold wires to be tested are grouped, and the intersection of the center lines of the gold wires in each group is calculated in turn. If the intersection state does not meet the preset requirements, it is determined that the gold wires to be tested have an abnormal intersection.

[0117] According to the grouping of the gold wires to be tested, the distance between the center lines of the gold wires in each group is calculated in sequence. If the distance does not meet the preset value, it is determined that the spacing between the gold wires to be tested is abnormal.

[0118] An image area of ​​the gold wire to be tested is generated along the center line of the gold wire, and then two tolerance lines parallel to the straight line between the head end solder joint and the tail end solder joint of the gold wire to be tested are generated. If the image area intersects with the tolerance line, it is determined that the gold wire to be tested is abnormally bent.

[0119] The center line length of the gold wire is calculated. If the length does not meet the preset requirement, it is determined that the gold wire to be tested is broken.

[0120] The centerline length of the gold wire under test within the head end solder joint and the tail end solder joint is calculated. If the length does not meet the preset requirements, the gold wire under test is determined to be broken. If the centerline length of the gold wire within the head end solder joint and the tail end solder joint meets the requirements, the length of the middle section of the gold wire under test is calculated. If the length of the middle section of the gold wire under test does not meet the preset requirements, the gold wire under test is determined to be broken.

[0121] Calculate the intersection of the annular area near the beginning and end and the gold wire area, extract the gold wire outline in the area, and determine the grayscale and features of the outline. If it does not meet the preset requirements, it is determined that the gold wire is broken.

[0122] When the center line of the gold wire to be tested cannot be extracted, it is determined that the gold wire to be tested is missing and the gold wire is insufficient.

[0123] Specifically, the distance and intersection relationship between the center lines of the gold wires to be tested can be calculated according to the center lines of the gold wires extracted from the gold wires to be tested, and then it can be judged whether the spacing and intersection relationship between the gold wires to be tested meet the requirements. The advantage of grouping the gold wires to be tested is that it is only necessary to judge the intersection relationship and spacing of the gold wire center lines within the group, and there is no need to compare one gold wire to be tested with all other gold wires to be tested, thereby improving the efficiency of defect judgment.

[0124] When judging whether the bending state of the gold wire to be tested is abnormal, the image contour of the gold wire to be tested is generated according to the center line of the gold wire, and two tolerance lines are set. If the image contour of the gold wire to be tested touches the tolerance line, it is considered that the bending state of the gold wire to be tested is abnormal.

[0125] The wire's centerline length is the primary criterion for determining whether it's broken. If the centerline length doesn't meet the required length, the wire is considered broken. Breakage detection is performed separately at the wire's leading and trailing solder joints, as wires are more susceptible to breakage at the solder joints and center. While separate determinations increase the complexity of wire breakage detection, they improve defect detection accuracy.

[0126] When judging whether there are too few gold wires, although the gold wire to be tested can be roughly positioned, the fine positioning of the gold wire cannot be completed. That is, the center line of the gold wire cannot be extracted between the first and tail solder points. In this case, the gold wire can be directly judged as being too few.

[0127] Example 2:

[0128] This embodiment 2 provides an appearance inspection system for a gold wire bonding process, the appearance inspection system is used to execute the appearance inspection method of embodiment 1, and the appearance inspection system includes a visual inspection module and a controller (not shown in the figure), such as Figure 3 As shown, the visual detection module includes a camera 1 and a composite light source 2, and the composite light source 2 includes a coaxial light 24, an upper ring light 23, a middle ring light 22 and a lower ring light 21, and the upper ring light 23, the middle ring light 22 and the lower ring light 21 are stacked in sequence.

[0129] The composite light source 2 is used to provide different light source modes; the controller is used to switch the light source mode of the composite light source 2 according to the type of the detection object; the camera 1 is used to obtain a reference image of the gold wire in the first light source mode, and to obtain at least one background image in the light source mode corresponding to the background object.

[0130] The controller is used to set a dividing line in the reference image according to the contrast between the gold wire and the background object; and is also used to divide the gold wire in the reference image into multiple gold wire segments according to the dividing line. For each gold wire segment, the gold wire segment with the highest contrast is selected from the background image and the reference image, and the selected gold wire segments are spliced ​​to obtain a gold wire to be tested that meets the contrast requirements, so as to perform defect detection on the gold wire to be tested.

[0131] Camera 1 uses a black and white CCD and needs to detect a minimum defect of 1.5μm. Camera 1 has 15 million pixels, a lens depth of field of no less than 0.4mm, and a magnification of 1x. The lower ring light 21 is red and positioned at the bottom of the composite light source 2, 1mm above the product to be tested, with an incident angle of 20°. This lower ring light projects reflected light from the gold wire into the lens, facilitating imaging by camera 1. The middle ring light 22 is blue and positioned above the lower ring light 21, with an incident angle of 45°. The upper ring light 23 is white and positioned above the middle ring light 22, with an incident angle of 60°. The coaxial light 24 is white and positioned above the upper ring light 23. The coaxial light 24, upper ring light 23, middle ring light 22, and lower ring light 21 in the composite light source 2 can be combined in any manner, with adjustable brightness and incident angle, allowing for flexible use of the appropriate light source combination to achieve high image contrast based on the inspection object.

[0132] The combination of middle ring light 22 and lower ring light 21 can create a good contrast baseline image of the gold wire. When the background object is chip 4, since chip 4 is a mirror surface and generally has texture inside, the coaxial light 24 can be used to illuminate the chip 4's texture. When the background object is capacitor 5, since capacitor 5 has both a mirror surface and a rough surface but no texture, the combination of coaxial light 24 and upper ring light 23 can create a background image with a clear image of the capacitor 5's boundary and a clear contrast with the rest of the background. When the background object is lead frame 3, since lead frame 3 has many textures, the combination of coaxial light 24 and lower ring light 21 can even out the texture of lead frame 3. The brightness of the composite light source 2 is adjustable and can be adjusted according to actual conditions. By using different light source combinations for different background objects, images with better contrast can be obtained, facilitating subsequent machine vision inspection.

[0133] The appearance inspection system further includes an automatic feeding module, an automatic sorting module and an automatic discharging module. The controller controls the motion mechanisms in the automatic feeding module, the automatic sorting module and the automatic discharging module to perform fully automatic feeding and discharging.

[0134] The specific implementation of the appearance detection method is described in detail in Example 1 and will not be repeated here.

[0135] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for appearance inspection of a gold wire bonding process, characterized in that: include: Acquire a reference image of the gold line using a first light source mode, and switch the light source mode of the composite light source according to the type of the background object to acquire at least one background image, wherein both the reference image and the background image include the gold line and the background object; In the reference image, a cutting line is set according to the contrast between the gold wire and the background object; Dividing the gold wire in the reference image into a plurality of gold wire segments according to the dividing line, selecting a gold wire segment with the highest contrast from the background image and the reference image for each gold wire segment, and splicing the selected gold wire segments to obtain a gold wire to be tested that meets the contrast requirement; Perform defect detection on the gold wire to be tested.

2. The appearance inspection method according to claim 1, characterized in that: The background objects include a lead frame, a chip and a capacitor; the composite light source includes a coaxial light, an upper ring light, a middle ring light and a lower ring light, and the upper ring light, the middle ring light and the lower ring light are stacked in sequence; The step of acquiring a reference image of the gold wire by using the first light source mode and switching the light source mode of the composite light source according to the type of the background object to acquire at least one background image comprises: A baseline image of the gold wire was obtained using a combination of middle and lower ring lights to improve the contrast between the gold wire, solder joints, and background objects. When the background object is a chip, coaxial light is used to obtain the background image to improve the contrast between the gold wire and the chip; When the background object is a capacitor, a combination of coaxial light and upper ring light is used to obtain the background image to improve the contrast between the gold wire and the capacitor; When the background object is a lead frame, a combination of coaxial light and lower ring light is used to obtain the background image to improve the contrast between the gold wire and the lead frame.

3. The appearance inspection method according to claim 1, characterized in that: The step of setting the dividing line according to the contrast between the gold line and the background object in the reference image includes: Analyzing the reference image to obtain a contrast ratio between the gold wire and the background object; The position where the contrast is less than the set contrast threshold is used as the dividing line.

4. The appearance inspection method according to claim 1, wherein: Performing defect detection on the gold wire to be tested includes: Performing a rough positioning of the gold wire to be tested according to the cutting line to obtain an initial gold wire area; The initial gold wire area is accurately located by grayscale threshold and contrast threshold to obtain a target gold wire area, and the skeleton of the target gold wire area is extracted to generate a gold wire center line; The defects of the gold wire are judged by the geometric characteristics and grayscale information of the gold wire centerline.

5. The appearance inspection method according to claim 4, characterized in that: The method of roughly positioning the gold wire to be tested according to the cutting line to obtain the initial gold wire area includes: Obtaining the first end solder joint and the tail end solder joint of the gold wire to be tested; Obtain the target point before each cutting line in sequence, and connect the target point to the tail end weld point to obtain a transition line; wherein, for the cutting line closest to the head end weld point, the target point before the cutting line is the head end weld point, and the target points corresponding to other cutting lines are the positioning points located before the cutting line; Obtain an intersection point between the transition line and the dividing line, use the intersection point as a normal line of the transition line, and select at least one transition point on the normal line; Connecting the transition point to the target point and the tail end welding point to establish a first image area and a second image area, respectively, calculating the grayscale g(A1) of the first image area and the grayscale g(A2) of the second image area, and calculating the average grayscale gm of the transition point based on the grayscale g(A1) and the grayscale g(A2); The transition point where the average grayscale gm meets the preset conditions is used as the positioning point of the gold wire to be tested; Connect the first end solder joint, all the positioning points and the tail end solder joint in sequence to form the initial gold wire area.

6. The appearance inspection method according to claim 5, characterized in that: Connecting the transition point to the target point and the tail end welding point to respectively establish a first image area and a second image area, respectively calculating the grayscale g(A1) of the first image area and the grayscale g(A2) of the second image area, and calculating the average grayscale gm of the transition point based on the grayscale g(A1) and the grayscale g(A2) includes: The distance between the transition point and the positioning point is used as the first major axis, and the distance between the transition point and the tail end welding point is used as the second major axis; A first image region is created by taking the first major axis as the length of the rectangular region and the width of the gold line as the width of the rectangular region; a second image region is created by taking the second major axis as the length of the rectangular region and the width of the gold line as the width of the rectangular region; Calculate the grayscale g(A1) of the first image area and the grayscale g(A2) of the second image area respectively, calculate the area A1 of the first image area and the area A2 of the second image area respectively, obtain the grayscale sum of the grayscale g(A1) and the grayscale g(A2), obtain the area sum of the area A1 and the area A2, and take the ratio between the grayscale sum and the area sum as the average grayscale gm of the transition point.

7. The appearance inspection method according to claim 5, characterized in that: The method of using the transition point where the average grayscale gm meets the preset conditions as the positioning point of the gold wire to be tested includes: When the average grayscale value gm of the transition point is greater than or equal to the grayscale threshold gt, the average grayscale value gm-1 of the previous transition point and the average grayscale value gm+1 of the next transition point are obtained; If the difference between the average gray value gm and the average gray value gm-1 and the difference between the average gray value gm and the average gray value gm+1 are both greater than the set contrast threshold gc, the transition point is used as the positioning point of the gold wire to be tested.

8. The appearance inspection method according to claim 4, characterized in that: The method of accurately locating the initial gold wire area by using a grayscale threshold and a contrast threshold to obtain a target gold wire area, and extracting a skeleton of the target gold wire area to generate a gold wire center line includes: A search area is generated along the initial gold wire area. Based on the different segmented gold wire images and the color change of the gold wire in the gold wire images, a bright area or a dark area is selected as the target area. The skeleton of the gold wire to be tested is determined in the target area. After processing the skeleton, the center line of the gold wire with precise positioning is obtained.

9. The appearance inspection method according to claim 4, characterized in that: The method of judging the defects of the gold wire by using the geometric features and grayscale information of the center line of the gold wire includes: The gold wires to be tested are grouped, and the intersection of the center lines of the gold wires in each group is calculated in turn. If the intersection does not meet the preset requirements, it is determined that the gold wires to be tested have an abnormal intersection; According to the grouping of the gold wires to be tested, the distance between the center lines of the gold wires in each group is calculated in sequence. If the distance does not meet a preset value, it is determined that the spacing between the gold wires to be tested is abnormal; Generate an image area of ​​the gold wire to be tested along the center line of the gold wire, and then generate two tolerance lines parallel to the straight line between the first and tail solder joints of the gold wire to be tested. If the image area intersects with the tolerance lines, it is determined that the gold wire to be tested is abnormally bent; Calculating the centerline length of the gold wire, and if the length does not meet the preset requirement, determining that the gold wire to be tested is broken; Calculating the centerline length of the gold wire to be tested within the head end solder joint and the tail end solder joint, and if the length does not meet the preset requirements, determining that the gold wire to be tested is broken; Calculate the intersection of the annular area near the beginning and end and the gold wire area, extract the gold wire outline in the area corresponding to the intersection, and determine the grayscale and features of the outline. If it does not meet the preset requirements, it is determined to be a gold wire break anomaly.

10. An appearance inspection system for a gold wire bonding process, the appearance inspection system being used to execute the appearance inspection method according to any one of claims 1 to 9, characterized in that: The appearance inspection system includes a visual inspection module and a controller. The visual inspection module includes a camera and a composite light source. The composite light source includes a coaxial light, an upper ring light, a middle ring light, and a lower ring light. The upper ring light, the middle ring light, and the lower ring light are stacked in sequence. The composite light source is used to provide different light source modes; The controller is used to switch the light source mode of the composite light source according to the type of the detection object; The camera is used to obtain a reference image of the gold wire in a first light source mode and to obtain at least one background image in a light source mode corresponding to the background object; The controller is used to set a cutting line in the reference image according to the contrast between the gold wire and the background object; It is also used to divide the gold wire in the reference image into multiple gold wire segments according to the dividing line. For each gold wire segment, the gold wire segment with the highest contrast is selected from the background image and the reference image, and the selected gold wire segments are spliced ​​to obtain the gold wire to be tested that meets the contrast requirements, so as to perform defect detection on the gold wire to be tested.

Images