Chip soldering quality detection method and system

Abstract

The application is suitable for the chip welding technical field, and particularly relates to a chip welding leg welding quality detection method and system. The method obtains pixel point classification on a to-be-detected image, determines predicted pixel points belonging to the welding leg category, extracts pixel values of all the predicted pixel points, calculates pixel mean values of the pixel values of all the predicted pixel points, obtains a pixel mean value range according to a range adjustment parameter and the pixel mean values, uses the pixel mean value range to filter the to-be-detected image to obtain target pixel points, performs connected domain clustering on all the target pixel points to obtain N connected domain clusters, detects the number distribution of the pixel points in each connected domain cluster, and determines, according to the number distribution of each connected domain cluster, that the corresponding connected domain cluster has welding abnormalities. The above process only performs simple classification of pixel points on the image through artificial intelligence, does not need to participate in abnormality detection, and can realize detection of abnormal welding such as welding leg shape and position based on analysis of the pixel points of the welding leg category.

Description

TECHNICAL FIELD

[0001] The present application is suitable for the chip welding technical field, and in particular relates to a chip soldering quality detection method and system. BACKGROUND

[0002] At present, chips support many computing scenarios, and chip soldering becomes the main technology to ensure the service life of chips. Defects in soldering can cause the chip to be used unsatisfactorily or reduce the service life. Therefore, improving the soldering detection accuracy of chip pins is of the utmost importance. With the rapid development of artificial intelligence technology, the method of using intelligent image processing has gradually replaced the manual detection method to greatly improve the quality detection efficiency of the produced chips and greatly reduce the human resource cost. Product quality detection can include chip pin quality detection, such as pin offset, too large or too small soldering points.

[0003] The image processing method usually adopts a template matching method for product quality detection, that is, a corresponding template image needs to be prepared in advance for each model of product, and the preparation work is relatively cumbersome. When the template image is lacking, the detection accuracy of the image processing method will greatly decrease, that is, the generalization of the image processing method is poor. The artificial intelligence model usually adopts a deep learning model to realize image segmentation or defect classification of product defects. However, if the deep learning model wants to achieve high accuracy, it needs to be applied to the product. Since the model cannot be directly migrated and used, it needs to be retrained based on local data. Therefore, a large number of labeled samples need to be prepared for model training, and the finer the label is, the better. This results in excessive labeling cost. Therefore, how to accurately realize chip soldering quality detection with small preparation cost in image processing has become a problem to be solved. SUMMARY

[0004] In view of this, the present application embodiment provides a chip soldering quality detection method and system to solve the problem of how to accurately realize chip soldering quality detection with small preparation cost in image processing.

[0005] In a first aspect, the present application embodiment provides a chip soldering quality detection method, which comprises:

[0006] An image to be detected collected on a soldering surface of a soldered chip is acquired, pixel points of the image to be detected are classified, and predicted pixel points belonging to a pin class are determined;

[0007] Pixel values of all predicted pixel points are extracted, a pixel mean value of the pixel values of all predicted pixel points is calculated, a pixel mean value range is obtained according to a range adjustment parameter and the pixel mean value;

[0008] screen all pixel points in the to-be-detected image using the pixel mean value range to obtain target pixel points, and perform connected domain clustering on all the target pixel points to obtain N connected domain clusters, where N is an integer greater than zero;

[0009] detect the number distribution of the pixel points in each connected domain cluster, and if it is detected that the number of pixel points in any connected domain cluster does not satisfy a first preset condition and / or the distribution of pixel points in any two connected domain clusters does not satisfy a second preset condition, it is determined that the corresponding connected domain cluster has a welding abnormality.

[0010] In a second aspect, an embodiment of the present application provides a chip solder leg welding quality detection system, which comprises:

[0011] an image classification module, configured to acquire a to-be-detected image collected on a welding surface of a welded chip, and perform pixel point classification on the to-be-detected image to determine predicted pixel points belonging to a solder leg category;

[0012] a pixel value analysis module, configured to extract pixel values of all the predicted pixel points, calculate a pixel mean value of the pixel values of all the predicted pixel points, and obtain a pixel mean value range according to a range adjustment parameter and the pixel mean value;

[0013] a target domain determination module, configured to screen all pixel points in the to-be-detected image using the pixel mean value range to obtain target pixel points, and perform connected domain clustering on all the target pixel points to obtain N connected domain clusters, where N is an integer greater than zero;

[0014] an abnormality detection module, configured to detect the number distribution of the pixel points in each connected domain cluster, and if it is detected that the number of pixel points in any connected domain cluster does not satisfy a first preset condition and / or the distribution of pixel points in any two connected domain clusters does not satisfy a second preset condition, it is determined that the corresponding connected domain cluster has a welding abnormality.

[0015] In addition, an embodiment of the present application provides a computer device, which comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the chip solder leg welding quality detection method according to the first aspect when executing the computer program. In addition, an embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executable on a processor to implement the chip solder leg welding quality detection method according to the first aspect.

[0016] The beneficial effects of the embodiments of the present application compared with the prior art are that: the present application obtains a to-be-detected image collected from a welding surface of a welded chip, classifies pixel points of the to-be-detected image, determines predicted pixel points belonging to a soldering leg type, extracts pixel values of all predicted pixel points, calculates a pixel mean value of the pixel values of all predicted pixel points, obtains a pixel mean value range according to a range adjustment parameter and the pixel mean value, filters all pixel points in the to-be-detected image using the pixel mean value range, obtains target pixel points, and performs connected domain clustering on all target pixel points to obtain N connected domain clusters. The number distribution of pixel points in each connected domain cluster is detected. If the number of pixel points in any connected domain cluster does not meet a first preset condition, and / or the distribution of pixel points in any two connected domain clusters does not meet a second preset condition, it is determined that the corresponding connected domain cluster has a welding abnormality. In the above process, only simple classification of pixel points of the image by artificial intelligence is required, without the need to participate in the detection of abnormalities. Based on the pixel points of the soldering leg type, the detection of the welding abnormality can be realized through automatic comparison and analysis, and the detection of the abnormal welding of the soldering leg shape, position and the like according to the number distribution and the like can be accurately realized. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1 is a flowchart of a chip soldering leg welding quality detection method provided by the first embodiment of the present application;

[0019] Figure 2 is a collection image of a welding surface after chip soldering legs are welded, provided by the first embodiment of the present application;

[0020] Figure 3 is a flowchart of a chip soldering leg welding quality detection method provided by the second embodiment of the present application;

[0021] Figure 4 is a structural schematic diagram of a chip soldering leg welding quality detection system provided by the third embodiment of the present application;

[0022] Figure 5 is a structural schematic diagram of a computer device provided by the fourth embodiment of the present application. DETAILED DESCRIPTION

[0023] In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular architectures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known methods, devices, circuits, and

[0024] It will be understood that the terms "comprises" and / or "comprising," when used in this specification, include the presence of one or more features, integers, steps, operations, elements, and / or components but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0025] It will be understood that the term "and / or," when used in this specification, includes the possibility of both there being a combination of features, integers, steps, operations, elements, and / or components and that there can be one or the other alone, in any and all permutations of these possible combinations.

[0026] As used in this specification and claims, the terms "if" and "when" can be interpreted to mean "upon determination" or "in response to a determination" or "upon detection" or "in response to a detection," depending on the context. Similarly, the phrase "if determined" or "if detected [a described condition or event]" can be interpreted to mean "upon determination" or "in response to a determination" or "upon detection" or "in response to a detection," depending on the context.

[0027] In addition, the terms "first," "second," "third," etc. are used herein, merely for purposes of description, and are not intended to indicate or create an order or sequence unless specifically so stated.

[0028] Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," and the like in various places throughout this specification are not necessarily referring to the same embodiment, unless otherwise specified. The terms "comprise," "comprising," "include," "including," "contain," "containing," and the like are used synonymously to mean "including but not limited to."

[0029] The embodiments of the present application can acquire and process related data based on artificial intelligence technology. Artificial intelligence is the theory, method, technology and application system for using digital computers or computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain optimal results.

[0030] It should be understood that the size of the serial number of each step in the following embodiments does not mean the order of execution. The execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0031] In order to illustrate the technical solutions of the present application, the following will be illustrated by specific embodiments.

[0032] Referring to Figure 1 is a flowchart of a chip solder leg welding quality detection method provided by an embodiment of the present application. The method is applied to a computer device. The computer device can be connected to a corresponding image acquisition device to acquire an image of a welding surface of a chip, or connected to a corresponding database to acquire a welding surface image of a chip stored in the database. As Figure 1 indicated, the chip solder leg welding quality detection method can include the following steps:

[0033] Step S101, acquiring a to-be-detected image acquired from a welding surface of a welded chip, classifying pixels of the to-be-detected image, and determining predicted pixels belonging to a solder leg category.

[0034] In the present application, the solder leg of the chip is imaged after welding from the corresponding welding surface, that is, the image on the welding surface is acquired to obtain the to-be-detected image. As Figure 2 indicated, the acquired image of the welding surface after welding of the chip provided by the embodiment one of the present application is shown in Figure 2 It can be seen from the figure that each black circle represents a welding point, and it can also be seen that the area of the black circle surrounded by the gray circle is smaller than that of other black circles. In certain cases, it can be considered that the welding point corresponding to the surrounded black circle is abnormal.

[0035] Classifying pixels of the detection image is to identify the pixels representing the solder leg from the image, that is, to define that the pixels are only of the solder leg category and other categories, and to extract the pixels of the solder leg category as predicted pixels. As Figure 2 indicated, the pixels with black color can be extracted as the predicted pixels of the solder leg category.

[0036] The above pixel classification can be classified by using a trained image segmentation model, or by using a threshold value.

[0037] Optionally, pixel points in the to-be-detected image are classified to determine predicted pixel points belonging to the solder leg category, including:

[0038] The to-be-detected image is subjected to grayscale processing to obtain a corresponding grayscale image.

[0039] The trained image segmentation model is used to perform category segmentation on each pixel point in the grayscale image to obtain the category of the corresponding pixel point, and the pixel point with the category of the solder leg category is determined as the predicted pixel point.

[0040] The trained image segmentation model is essentially a semantic segmentation model, the input of the semantic segmentation model is the grayscale image after the to-be-detected image is subjected to grayscale processing, and the output of the semantic segmentation model is a semantic segmentation image. In the semantic segmentation image, the pixel value of each pixel point is determined according to the category of the pixel point. In this embodiment, the semantic segmentation image contains two pixel values, 0 and 1. The pixel point with the pixel value of 1 represents the pixel point belonging to the solder leg category, and the pixel point with the pixel value of 0 represents the pixel point not belonging to the solder leg category.

[0041] In an embodiment, the architecture of the semantic segmentation model is an encoder-decoder architecture, and the size of the output semantic segmentation image is consistent with that of the to-be-detected image.

[0042] In this embodiment, the grayscale value of the solder leg pixel point is significantly different from that of the component, and the solder leg pixel point is easy to distinguish, that is, the accuracy of semantic segmentation is high. Moreover, the above step is only used to extract the predicted pixel point, and does not require fine division of the solder leg region. Therefore, the trained semantic segmentation model can be directly used, and the local data set is not used for fine-tuning again to reduce the labeling cost.

[0043] In step S102, the pixel values of all the predicted pixel points are extracted, the pixel mean value of the pixel values of all the predicted pixel points is calculated, and the pixel mean value range is obtained according to the range adjustment parameter and the pixel mean value.

[0044] In this application, all the extracted predicted similar points are pixel points representing the solder leg category. Since all the pixel values of the predicted pixel points have similar characteristics, the average value of the pixel values of all the predicted pixel points (i.e., the pixel mean value) is taken as the basis for repeatedly screening the pixel points of the solder leg category. A pixel mean value range is constructed according to a range adjustment parameter and the pixel mean value, which is used to determine all the pixel points in the pixel mean value range and perform subsequent processing.

[0045] Optionally, the pixel mean value range is obtained according to the range adjustment parameter and the pixel mean value, including:

[0046] The first difference value between the pixel mean value and the maximum value of the pixel values of all the predicted pixel points is calculated, and the second difference value between the pixel mean value and the minimum value of the pixel values of all the predicted pixel points is calculated.

[0047] The maximum value of the first difference value and the second difference value is taken as an initial range adjustment parameter, the initial range adjustment parameter is optimized to obtain an optimized range adjustment parameter, and a range formed by adding or subtracting the optimized range adjustment parameter from the pixel mean value is taken as a pixel mean value range.

[0048] The pixel gray value at the corresponding position in the to-be-detected image is extracted according to the position of the predicted pixel point representing the weld leg class in the semantic segmentation map, the mean value of all pixel gray values is calculated to obtain a gray mean value, the initial value of the adaptive coefficient can be determined according to the maximum and minimum values of the pixel gray values, the absolute value of the difference between the maximum value and the gray mean value is calculated to obtain a first difference value, the absolute value of the difference between the minimum value and the gray mean value is calculated to obtain a second difference value, the larger one of the first difference value and the second difference value is taken as the initial value, the gray mean value is denoted as μ, and the adaptive coefficient is denoted as α, so that the gray range is [μ-α, μ+α], the pixel points with gray values belonging to the gray range in the to-be-detected image are extracted to obtain a plurality of initial weld leg pixel points.

[0049] Optionally, the initial range adjustment parameter is optimized to obtain an optimized range adjustment parameter, including:

[0050] The range formed by adding or subtracting the initial range adjustment parameter from the pixel mean value is taken as an initial pixel mean value range, and all pixel points in the to-be-detected image are screened using the initial pixel mean value range to obtain initial pixel points.

[0051] All initial pixel points are clustered to obtain an initial clustering result, the number of pixel points included in each clustering set in the initial clustering result is counted, the first variance of the number of pixel points of all clustering sets is calculated, the initial range adjustment parameter is optimized with the first variance being minimized as the target, and an optimized range adjustment parameter is obtained.

[0052] The initial range adjustment parameter can be optimized, and the purpose is to make the number of pixel points in each clustering result consistent when the screened pixel points are clustered, which helps to improve the accuracy of the range adjustment parameter.

[0053] Optionally, all initial pixel points are clustered to obtain an initial clustering result, including:

[0054] All initial pixel points are clustered based on the initial clustering parameter to obtain at least one clustering set;

[0055] The second variance of the number of pixel points in each clustering set is calculated, and the initial clustering parameter is optimized with the second variance being minimized as the target to obtain an optimized initial clustering parameter.

[0056] Clustering all initial pixel points based on the optimized initial clustering parameter to obtain an initial clustering result.

[0057] The DBSCAN clustering method is used to cluster all initial pixel points, and the DBSCAN clustering method is a clustering process based on clustering parameters. Since the size of the weld toe is usually fixed, even if the pixel points are mistakenly considered as weld toe pixel points, the size of each weld toe area is still approximate. By adjusting the clustering parameters, the variance of the number of pixel points in each clustering set obtained by clustering is minimized, and the number of pixel points in a single clustering set should be as small as possible under the condition of meeting the preset condition, for example, the preset condition can be that the number of pixel points in a single clustering set is greater than a preset number threshold, and the preset number threshold can be 10. This setting is to avoid two or more weld toes being clustered into the same clustering set.

[0058] Optionally, clustering all initial pixel points based on the initial clustering parameter to obtain at least one clustering set, including:

[0059] Clustering all initial pixel points M times based on the initial clustering parameter to obtain clustering sets corresponding to M times of clustering, M being an integer greater than 1;

[0060] Detecting whether the clustering sets corresponding to M times of clustering are consistent. If not, adjust the initial clustering parameter to obtain an adjusted initial clustering parameter;

[0061] Taking the adjusted initial clustering parameter as the initial clustering parameter, returning to execute the step of clustering all initial pixel points M times based on the initial clustering parameter to obtain clustering sets corresponding to M times of clustering, until the clustering sets corresponding to M times of clustering are consistent;

[0062] Correspondingly, the initial clustering parameter is optimized to minimize the second variance to obtain an optimized initial clustering parameter, including:

[0063] If the second variance is not the minimum, return to execute the step of adjusting the initial clustering parameter to obtain an adjusted initial clustering parameter, until the second variance is the minimum, and the corresponding initial clustering parameter is the optimized initial clustering parameter.

[0064] Wherein, considering that the same weld toe may be clustered into multiple clustering sets, a multiple clustering method is used to determine such a situation, that is, after adjusting the clustering parameter each time, multiple clustering is performed, for example, three times. If the multiple clustering results are consistent, it means that there is no situation of the same weld toe being clustered into multiple clustering sets. If the multiple clustering results are inconsistent, it means that there is a situation of the same weld toe being clustered into multiple clustering sets, and the clustering parameter needs to be adjusted.

[0065] For example, the single weld leg covers 20 pixel points, and the positions of the pixel points are relatively close to each other. Due to clustering, the pixel points in each cluster set are as few as possible, and are divided into two cluster sets with 10 pixel points. In multiple clustering, due to the relatively close positions of the pixel points, the division result changes, thereby determining that the pixel points belonging to the same weld leg are clustered into multiple cluster sets.

[0066] In step S103, the pixel mean range is used to screen all pixel points in the to-be-detected image to obtain target pixel points, and all target pixel points are subjected to connected domain clustering to obtain N connected domain clusters.

[0067] In this application, the target pixel points obtained by range screening are subjected to connected domain clustering analysis, thereby determining connected pixel points, the connected pixel points are clustered into one connected domain cluster, one connected domain cluster corresponds to one weld leg, and N is an integer greater than zero.

[0068] Optionally, all target pixel points are subjected to connected domain clustering to obtain N connected domain clusters, including:

[0069] For any target pixel point, the eight neighborhoods of the target pixel point are searched with the target pixel point as the origin, and it is detected whether the search result of the corresponding neighborhood is a target pixel point;

[0070] If it is detected that the search result of the corresponding neighborhood is not a target pixel point, the search towards the corresponding neighborhood is terminated. If it is detected that the search result of the corresponding neighborhood is a target pixel point, the target pixel point is taken as the origin, and the step of searching the eight neighborhoods of the origin is executed, until all target pixel points are traversed, a group of connected target pixel points are determined as one connected domain cluster, and N connected domain clusters are obtained.

[0071] In the process of connected domain analysis, for any target pixel point, the connected domain of the target pixel point is initialized to contain only the target pixel point itself, the eight neighborhoods are searched, if there is another target pixel point in the eight neighborhoods, the other target pixel point is added to the connected domain of the target pixel point, and the step of searching the eight neighborhoods of each target pixel point in the connected domain is executed, if there is another target pixel point in the eight neighborhoods, the other target pixel point is added to the connected domain of the target pixel point, and the iteration is repeated until there is no other target pixel point in the eight neighborhoods of each weld leg pixel point in the connected domain.

[0072] In step S104, the number distribution of the pixel points in each connected domain cluster is detected. If it is detected that the number of the pixel points in any connected domain cluster does not satisfy a first preset condition, and / or the distribution of the pixel points in any two connected domain clusters does not satisfy a second preset condition, it is determined that the corresponding connected domain cluster has welding abnormality.

[0073] In the present application, any connected domain cluster contains the pixel point distribution of the corresponding number of pixel points. For the number, in general, the number of pixel points in all connected domain clusters should be the same or similar. If there is a large difference, it means that the welding of the welding leg is problematic. Figure 2 As shown in the figure, the size of the welding point surrounded by the circle is obviously smaller than the surrounding, which can indicate that the welding is abnormal.

[0074] For distribution, in general, the connected domain cluster should be uniformly distributed. For example, the interval width between each welding leg and the relative position between the welding legs are fixed. If there is a deviation problem between the welding legs, it can be determined that the welding is abnormal.

[0075] Optionally, the number distribution of the pixel points in each connected domain cluster is detected. If it is detected that the number of pixel points in any connected domain cluster does not meet the first preset condition, and / or the distribution of pixel points in any two connected domain clusters does not meet the second preset condition, it is determined that the corresponding connected domain cluster has a welding abnormality, including:

[0076] The number distribution of the pixel points in each connected domain cluster is detected. According to the number of pixel points in all connected domain clusters, the mean and standard deviation representing the number of pixel points in the connected domain cluster are determined.

[0077] According to the mean and standard deviation, a number threshold range is determined. The number threshold range is the first preset condition. If it is detected that the number of pixel points in any connected domain cluster does not belong to the target threshold range, it is determined that the corresponding connected domain cluster has a welding abnormality.

[0078] Wherein, the number of pixel points in each connected domain is counted. In theory, the number of pixel points in each connected domain should be the same. However, due to the fact that the connected domain contains pixel points, lack of welding and multiple welding of the welding point will cause the number of pixel points in the connected domain to be different. Even in the case where a single connected domain contains two welding legs and a welding point, etc. At this time, the outlier detection method can be used for detection. The mean μ and standard deviation σ of the number of pixel points in the connected domain are calculated. The connected domain corresponding to the number in the region less than μ+3σ in the Gaussian distribution is considered to be a lack of welding leg, and the connected domain corresponding to the number in the region greater than μ+3σ is considered to be a multiple welding leg.

[0079] In an embodiment, the characteristics of the side-symmetrical distribution of the chip welding leg can be used to obtain the chip welding leg set on both sides of the same chip. According to the inter-class comparison method, the detection result is verified.

[0080] The embodiment of the application obtains a to-be-detected image collected on a welding surface of a welded chip, classifies pixel points of the to-be-detected image, determines predicted pixel points belonging to a soldering leg type, extracts pixel values of all predicted pixel points, calculates a pixel mean value of the pixel values of all predicted pixel points, obtains a pixel mean value range according to a range adjustment parameter and the pixel mean value, screens all pixel points in the to-be-detected image using the pixel mean value range, obtains target pixel points, performs connected domain clustering on all target pixel points, obtains N connected domain clusters, detects a quantity distribution of pixel points in each connected domain cluster, and if it is detected that the quantity of pixel points in any connected domain cluster does not satisfy a first preset condition and / or the distribution of pixel points in any two connected domain clusters does not satisfy a second preset condition, it is determined that a corresponding connected domain cluster has a welding abnormality. In the above process, only simple classification of pixel points of an image by artificial intelligence is required, and no detection of abnormalities is required. Based on pixel points of a soldering leg type, detection of welding abnormalities can be realized through automatic comparison and analysis, and accurate detection of abnormal welding of a soldering leg shape, position and the like according to a quantity distribution and the like can be realized.

[0081] Referring to Figure 3 is a flowchart of a chip soldering leg welding quality detection method provided by the embodiment two of the application. As Figure 3 indicated, the chip soldering leg welding quality detection method can include the following steps:

[0082] In step S301, a to-be-detected image collected on a welding surface of a welded chip is obtained, pixel points of the to-be-detected image are classified, and predicted pixel points belonging to a soldering leg type are determined.

[0083] In step S302, pixel values of all predicted pixel points are extracted, a pixel mean value of the pixel values of all predicted pixel points is calculated, and a pixel mean value range is obtained according to a range adjustment parameter and the pixel mean value.

[0084] In step S303, all pixel points in the to-be-detected image are screened using the pixel mean value range, target pixel points are obtained, all target pixel points are subjected to connected domain clustering, and N connected domain clusters are obtained.

[0085] The content of the above steps S301 to S303 is the same as that of the steps S101 to S103, and can be referred to the above steps S101 to S103, which will not be described here.

[0086] In step S304, a quantity distribution of pixel points in each connected domain cluster is detected, and according to the distribution of pixel points in all connected domain clusters, a center pixel point of each connected domain cluster and a near-neighbor center pixel point of each center pixel point are determined.

[0087] In the present application, the position of each pixel point in each connected domain cluster is recorded, so as to calculate the center pixel point representing the center of each connected domain cluster, and the near neighbor center pixel point of the center pixel point is determined according to the positional relationship, that is, the center pixel point corresponding to the near neighbor connected domain cluster of the connected domain cluster of the center pixel point.

[0088] In the present application, the K nearest neighbor connected domain clusters of the connected domain cluster are determined according to the distance between the center pixel points, K can be 5, and the pixel points in the connected domain and the connected domain are matched by using the two-part matching, a plurality of groups of pixel point pairs are obtained, the distance between the pixel points in each group is calculated, and the matching result is obtained by using the distance variance corresponding to each group of pixel point pairs as a constraint.

[0089] In step S305, the translation relationship between the center pixel point and the near neighbor center pixel point is determined for any center pixel point and any near neighbor center pixel point.

[0090] In the present application, the translation relationship can be determined according to the positional relationship between the center pixel points, that is, the corresponding translation mode. For example, the pixel point (3, 5) to the pixel point (4, 6) is translated by 2 pixel points to the right and up.

[0091] In the present application, if there are multiple near neighbors, the most common translation mode between all pixel point pairs in the matching result can be determined by voting, and the target translation mode is determined. The target translation mode is used to translate the connected domain as a whole, and the difference value image is obtained by subtracting the corresponding adjacent connected domain. The absolute value of the difference value image is taken and added, and if the addition result is less than a preset value, the preset value can be 10, it is considered that there is a translation association, and K adjacent connected domains having the translation association with the connected domain are searched out. It can be understood that K adjacent connected domains are obtained according to the center point distance, if any one does not satisfy the translation association, it is removed, and one is added according to the center point distance, until K adjacent connected domains having the translation association are obtained.

[0092] In step S306, all pixel points in the corresponding two connected domain clusters are aligned and translated according to the translation relationship, and the intersection over union of all pixel points in the two connected domain clusters after alignment and translation is calculated, and the preset intersection over union threshold is used as the second preset condition.

[0093] In the present application, one of the two connected domain clusters is translated using the translation relationship, that is, all the pixel points in the connected domain cluster are translated, so as to align the positions of the two connected domain clusters in the image. Then, the number of intersecting pixel points and the number of all pixel points in the union of the two connected domain clusters are calculated, and the ratio of the two is taken as the intersection-union ratio. The higher the intersection-union ratio, the more similar the shape and area of the two connected domain clusters, which means that the probability of welding abnormality in the two connected domain clusters is small. If all the connected domain clusters are compared in the above manner, the connected domain cluster with welding abnormality can be determined.

[0094] Specifically, the K adjacent connected domains and the connected domain to be processed have translation correlation with each other, and the reference connected domains are selected as a set. The two connected domain clusters with the closest translation distance in the set are selected as the reference connected domains, and the translation distance is set as d. The translation distance between any two reference connected domains is taken modulo d. If the modulo result of more than a preset proportion is greater than a preset value, the two reference connected domains with the second closest translation distance are selected, and the translation distance is updated as d. The process is repeated until the modulo result of more than a preset proportion is less than or equal to a preset value. The preset value can be set as 2. When the modulo result of more than a preset proportion is less than or equal to a preset value, d is determined as the reference value. Any one of the two reference connected domains with the current reference value is taken as the basic connected domain, and the multiple of the reference value is used for translation.

[0095] In step S307, if the intersection-union ratio of all pixel points in any two connected domain clusters is less than the intersection-union ratio threshold, it is determined that the corresponding connected domain cluster has welding abnormality.

[0096] In the present application, the intersection-union ratio threshold can be 0.8. The closer the intersection-union ratio is to 1, the higher the similarity of the two connected domain clusters. The closer the intersection-union ratio is to 0, the lower the similarity of the two connected domain clusters, that is, the higher the probability of abnormality.

[0097] The embodiment of the present application analyzes the translation relationship of the connected domain clusters to determine whether the two groups of connected domain clusters have welding abnormality. The shape and area of the connected domain clusters are compared in a simple manner to ensure accuracy while improving detection efficiency.

[0098] Corresponding to the chip leg welding quality detection method of the above embodiment, Figure 4 The structure block diagram of the chip leg welding quality detection system provided by the third embodiment of the present application is shown. For the convenience of description, only the parts related to the embodiments of the present application are shown.

[0099] Referring to Figure 4 The chip leg welding quality detection system comprises:

[0100] The image classification module 41 is configured to acquire a to-be-detected image collected on a welding surface of a welded chip, perform pixel point classification on the to-be-detected image, and determine predicted pixel points belonging to the welding leg category;

[0101] The pixel value analysis module 42 is configured to extract pixel values of all the predicted pixel points, calculate a pixel mean value of the pixel values of all the predicted pixel points, and obtain a pixel mean value range according to a range adjustment parameter and the pixel mean value.

[0102] The target domain determination module 43 is configured to filter all the pixel points in the to-be-detected image using the pixel mean value range to obtain target pixel points, and perform connected domain clustering on all the target pixel points to obtain N connected domain clusters, where N is an integer greater than zero.

[0103] The abnormality detection module 44 is configured to detect the number distribution of the pixel points in each connected domain cluster, and determine that a corresponding connected domain cluster has a welding abnormality if the number of the pixel points in any connected domain cluster does not satisfy a first preset condition and / or the distribution of the pixel points in any two connected domain clusters does not satisfy a second preset condition.

[0104] Optionally, the image classification module 41 comprises:

[0105] The grayscale processing unit is configured to perform grayscale processing on the to-be-detected image to obtain a corresponding grayscale image.

[0106] The image classification unit is configured to perform category segmentation on each pixel point in the grayscale image using a trained image segmentation model to obtain the category of the corresponding pixel point, and determine the pixel points with the category of the welding leg as the predicted pixel points.

[0107] Optionally, the pixel value analysis module 42 comprises:

[0108] The difference calculation unit is configured to calculate a first difference between the pixel mean value and a maximum value of the pixel values of all the predicted pixel points, and calculate a second difference between the pixel mean value and a minimum value of the pixel values of all the predicted pixel points.

[0109] The mean value range determination unit is configured to take the maximum value of the first difference and the second difference as an initial range adjustment parameter, optimize the initial range adjustment parameter to obtain an optimized range adjustment parameter, and take the range formed by the pixel mean value plus or minus the optimized range adjustment parameter as the pixel mean value range.

[0110] Optionally, the mean value range determination unit comprises:

[0111] The pixel point screening subunit is configured to take the range formed by the pixel mean value plus or minus the initial range adjustment parameter as an initial pixel mean value range, filter all the pixel points in the to-be-detected image using the initial pixel mean value range to obtain initial pixel points.

[0112] The optimization parameter subunit is configured to cluster all the initial pixel points to obtain an initial clustering result, count the number of pixel points included in each clustering set in the initial clustering result, calculate a first variance of the number of pixel points of all the clustering sets, and optimize the initial range adjustment parameter to obtain an optimized range adjustment parameter.

[0113] Optionally, the optimization parameter subunit comprises:

[0114] The sub-clustering subunit is configured to cluster all the initial pixel points based on the initial clustering parameter to obtain at least one clustering set.

[0115] The optimization subunit is configured to calculate a second variance of the number of pixel points in each clustering set, optimize the initial clustering parameter to obtain an optimized initial clustering parameter, with the second variance being minimized as the target.

[0116] The initial clustering subunit is configured to cluster all the initial pixel points based on the optimized initial clustering parameter to obtain an initial clustering result.

[0117] Optionally, the sub-clustering subunit is specifically configured to:

[0118] cluster all the initial pixel points based on the initial clustering parameter for M times to obtain clustering sets corresponding to the M times of clustering, M being an integer greater than 1.

[0119] detect whether the clustering sets corresponding to the M times of clustering are consistent, and if not, adjust the initial clustering parameter to obtain an adjusted initial clustering parameter.

[0120] use the adjusted initial clustering parameter as the initial clustering parameter, and return to execute the step of clustering all the initial pixel points based on the initial clustering parameter for M times to obtain clustering sets corresponding to the M times of clustering, until the clustering sets corresponding to the M times of clustering are consistent.

[0121] Correspondingly, the optimization subunit is specifically configured to:

[0122] if the second variance is not the minimum, return to execute the step of adjusting the initial clustering parameter to obtain an adjusted initial clustering parameter, until the second variance is the minimum, and the corresponding initial clustering parameter is the optimized initial clustering parameter.

[0123] Optionally, the target domain determination module 43 comprises:

[0124] The target searching subunit is configured to, for any target pixel point, search an eight-neighbor domain of the target pixel point taken as an origin, and detect whether a search result of the corresponding neighbor domain is a target pixel point.

[0125] The connected domain determination unit is configured to: if it is detected that the search result of the corresponding neighborhood is not a target pixel point, terminate the search of the corresponding neighborhood; if it is detected that the search result of the corresponding neighborhood is a target pixel point, take the target pixel point as an origin, and return to execute the step of searching the eight neighborhoods of the origin until all target pixel points are traversed, determine a group of connected target pixel points as a connected domain cluster, and obtain N connected domain clusters.

[0126] Optionally, the anomaly detection module 44 comprises:

[0127] The number detection unit is configured to detect the number distribution of the pixel points in each connected domain cluster, and determine the mean value and the standard deviation of the number of the pixel points in the connected domain cluster according to the number of the pixel points in all connected domain clusters.

[0128] The first anomaly determination unit is configured to determine a number threshold range according to the mean value and the standard deviation, take the number threshold range as a first preset condition, and determine that the corresponding connected domain cluster has a welding anomaly if it is detected that the number of the pixel points in any connected domain cluster does not belong to the target threshold range.

[0129] Optionally, the anomaly detection module 44 comprises:

[0130] The distribution detection unit is configured to detect the number distribution of the pixel points in each connected domain cluster, and determine the center pixel point of each connected domain cluster and the near-neighbor center pixel points of each center pixel point according to the distribution of the pixel points in all connected domain clusters.

[0131] The translation relationship analysis unit is configured to determine the translation relationship between the center pixel point and the near-neighbor center pixel point for any near-neighbor center pixel point of any center pixel point.

[0132] The intersection-over-union ratio calculation unit is configured to align and translate all pixel points in the corresponding two connected domain clusters according to the translation relationship, calculate the intersection-over-union ratio of all pixel points in the two connected domain clusters after the alignment and translation, and take a preset intersection-over-union ratio threshold as a second preset condition.

[0133] The second anomaly determination unit is configured to determine that the corresponding connected domain cluster has a welding anomaly if it is detected that the intersection-over-union ratio of all pixel points in any two connected domain clusters is less than the intersection-over-union ratio threshold.

[0134] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of the present application, and the specific functions and technical effects brought by the same can be referred to the method embodiments part, which will not be described here.

[0135] Figure 5 A structural schematic diagram of a computer device provided in Embodiment Four of the present application is shown in FIG. 4. Figure 5As shown, the computer device of this embodiment includes: at least one processor ( Figure 5 The diagram shows only one chip solder joint quality inspection method embodiment, a memory, and a computer program stored in the memory that can run on at least one processor. When the processor executes the computer program, it implements the steps in the embodiments of the chip solder joint quality inspection method described above.

[0136] This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 5 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. Computer devices may include more or fewer components than shown in the illustration, or combinations of certain components, or different components, such as network interfaces, displays, and input devices.

[0137] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0138] Memory includes readable storage media, internal memory, etc., wherein internal memory can be the RAM of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of a computer device, or in other embodiments, it can be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal storage units and external storage devices of a computer device. Memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. Memory can also be used to temporarily store data that has been output or will be output.

[0139] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the above device can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here. If the integrated unit is realized in the form of software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, all or part of the processes in the above embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of the above method embodiments can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium at least includes any entity or device that can carry computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, according to legislation and patent practice, computer readable medium cannot be electrical carrier signal and telecommunication signal.

[0140] The above embodiment methods can also be completed by a computer program product, which can be run on a computer device to make the computer device execute the steps of the above method embodiments.

[0141] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0142] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0143] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / computer device and method can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely schematic. The division of the modules or units is merely a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0144] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0145] The above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method of detecting the quality of a die bond, characterized by, The chip solder leg welding quality detection method comprises: Obtaining a to-be-detected image collected from a welded surface of a welded chip, classifying pixels of the to-be-detected image, and determining predicted pixels belonging to a solder leg category; Extracting pixel values of all predicted pixels, calculating a pixel mean value of the pixel values of all predicted pixels, obtaining a pixel mean value range according to a range adjustment parameter and the pixel mean value, and using the pixel mean value range to screen all pixels in the to-be-detected image to obtain target pixels. Clustering all target pixels to obtain N connected domain clusters, where N is an integer greater than zero. Detecting the number distribution of pixels in each connected domain cluster, and determining that a corresponding connected domain cluster has a welding abnormality if the number of pixels in any connected domain cluster does not meet a first preset condition and / or the distribution of pixels in any two connected domain clusters does not meet a second preset condition.

2. The chip-heel solder quality inspection method according to claim 1, characterized by, Classifying pixels of the to-be-detected image, and determining predicted pixels belonging to a solder leg category, comprises: Performing grayscale processing on the to-be-detected image to obtain a corresponding grayscale image; Using a trained image segmentation model to perform category segmentation on each pixel in the grayscale image to obtain the category of the corresponding pixel, and determining pixels with a solder leg category as predicted pixels.

3. The chip-heel solder quality inspection method according to claim 1, characterized by, Obtaining a pixel mean value range according to a range adjustment parameter and the pixel mean value, comprises: Calculating a first difference value between the pixel mean value and the maximum value of the pixel values of all predicted pixels, and calculating a second difference value between the pixel mean value and the minimum value of the pixel values of all predicted pixels; Taking the maximum value of the first difference value and the second difference value as an initial range adjustment parameter, optimizing the initial range adjustment parameter to obtain an optimized range adjustment parameter, and taking the range formed by the pixel mean value plus or minus the optimized range adjustment parameter as the pixel mean value range.

4. The chip-heel solder quality inspection method according to claim 3, characterized by, Optimizing the initial range adjustment parameter to obtain an optimized range adjustment parameter, comprises: Taking the range formed by the pixel mean value plus or minus the initial range adjustment parameter as an initial pixel mean value range, using the initial pixel mean value range to screen all pixels in the to-be-detected image to obtain initial pixels, clustering all initial pixels to obtain an initial clustering result, counting the number of pixels included in each clustering set in the initial clustering result, calculating a first variance of the number of pixels of all clustering sets, and optimizing the initial range adjustment parameter to obtain an optimized range adjustment parameter with the first variance being minimized as the target. Clustering all initial pixels to obtain an initial clustering result, comprises:

5. The chip-heel solder quality inspection method according to claim 4, characterized by, Clustering all initial pixels based on initial clustering parameters to obtain at least one clustering set; Calculating a second variance of the number of pixels in each clustering set, optimizing the initial clustering parameters to obtain optimized initial clustering parameters with the second variance being minimized as the target, and clustering all initial pixels based on the optimized initial clustering parameters to obtain an initial clustering result. Clustering all initial pixels based on initial clustering parameters to obtain at least one clustering set; ​ 6. The chip-heel solder quality inspection method according to claim 5, characterized by, Clustering all initial pixel points based on the initial clustering parameters to obtain at least one clustering set, comprising: Clustering all initial pixel points M times based on the initial clustering parameters to obtain M clustering sets corresponding to M clustering, M being an integer greater than 1; Detecting whether the M clustering sets corresponding to M clustering are consistent, if not, adjusting the initial clustering parameters to obtain adjusted initial clustering parameters; Taking the adjusted initial clustering parameters as the initial clustering parameters, returning to execute the step of clustering all initial pixel points M times based on the initial clustering parameters to obtain M clustering sets corresponding to M clustering until the M clustering sets corresponding to M clustering are consistent; Correspondingly, optimizing the initial clustering parameters with the second minimum variance as the target to obtain optimized initial clustering parameters, comprising: If the second variance is not minimum, returning to execute the step of adjusting the initial clustering parameters to obtain adjusted initial clustering parameters until the second variance is minimum, and obtaining the corresponding initial clustering parameters as the optimized initial clustering parameters.

7. The chip-heel solder quality inspection method according to claim 1, characterized by, Clustering all target pixel points to obtain N connected domain clusters, comprising: For any target pixel point, searching the eight neighborhoods of the target pixel point as the origin, detecting whether the search result of the corresponding neighborhood is a target pixel point; If it is detected that the search result of the corresponding neighborhood is not a target pixel point, terminating the search of the corresponding neighborhood, if it is detected that the search result of the corresponding neighborhood is a target pixel point, taking the target pixel point as the origin, returning to execute the step of searching the eight neighborhoods of the origin until all target pixel points are traversed, determining a group of connected target pixel points as a connected domain cluster to obtain N connected domain clusters.

8. The chip-heel solder quality inspection method according to any one of claims 1 to 7, characterized by, Detecting the number distribution of pixel points in each connected domain cluster, if it is detected that the number of pixel points in any connected domain cluster does not satisfy the first preset condition, and / or the distribution of pixel points in any two connected domain clusters does not satisfy the second preset condition, determining that the corresponding connected domain cluster has welding abnormality, comprising: Detecting the number distribution of pixel points in each connected domain cluster, determining the mean value and the standard deviation representing the number of pixel points in the connected domain cluster according to the number of pixel points in all connected domain clusters; According to the mean value and the standard deviation, determining a number threshold range, taking the number threshold range as the first preset condition, if it is detected that the number of pixel points in any connected domain cluster does not belong to the number threshold range, determining that the corresponding connected domain cluster has welding abnormality.

9. The chip-heel solder quality inspection method according to any one of claims 1 to 7, characterized by, Detecting the number distribution of pixel points in each connected domain cluster, if it is detected that the number of pixel points in any connected domain cluster does not satisfy the first preset condition, and / or the distribution of pixel points in any two connected domain clusters does not satisfy the second preset condition, determining that the corresponding connected domain cluster has welding abnormality, comprising: Detecting the number distribution of pixel points in each connected domain cluster, determining the center pixel point of each connected domain cluster and the near neighbor center pixel point of each center pixel point according to the distribution of pixel points in all connected domain clusters; For any near-neighbor central pixel point of any central pixel point, a translation relationship between the central pixel point and the near-neighbor central pixel point is determined; All pixel points in the corresponding two connected domain clusters are aligned and translated according to the translation relationship, and an intersection-over-union of all pixel points in the two connected domain clusters after alignment and translation is calculated, with a preset intersection-over-union threshold as a second preset condition; If the intersection-over-union of all pixel points in any two connected domain clusters is less than the intersection-over-union threshold, it is determined that the corresponding connected domain cluster has a welding abnormality.

10. A chip-heel solder quality inspection system characterized by comprising: The chip leg welding quality detection system comprises: An image classification module is configured to acquire a to-be-detected image collected on a welding surface of a welded chip, classify pixel points of the to-be-detected image, and determine predicted pixel points belonging to a leg category; A pixel value analysis module is configured to extract pixel values of all predicted pixel points, calculate a pixel mean value of the pixel values of all predicted pixel points, and obtain a pixel mean value range according to a range adjustment parameter and the pixel mean value. A target domain determination module is configured to filter all pixel points in the to-be-detected image using the pixel mean value range to obtain target pixel points, and perform connected domain clustering on all target pixel points to obtain N connected domain clusters, where N is an integer greater than zero. An abnormality detection module is configured to detect a number distribution of pixel points in each connected domain cluster, and if it is detected that the number of pixel points in any connected domain cluster does not satisfy a first preset condition and / or the distribution of pixel points in any two connected domain clusters does not satisfy a second preset condition, it is determined that the corresponding connected domain cluster has a welding abnormality.

Images