A machine vision-based SMT patch production quality detection method and system

Abstract

The application belongs to the technical field of industrial detection, and discloses a kind of SMT patch production quality detection method and system based on machine vision;Including: collecting machine vision image data and production process parameters of SMT patch production station and executing data cleaning to obtain the data set to be detected;Printed boundary correction is executed, and printing inspection result is generated to update image data;Image coordinate mapping relationship is established and registration evaluation is executed;Execute device position correction and output correction detection area set;Execute mounting posture detection and output mounting posture determination result;Execute regional image processing and output fine pin process data;Fusion determination is carried out to output welding quality determination result;Integrate mounting posture determination result and welding quality determination result to construct patch process quality detection log;The traceability, consistency and automation processing efficiency of patch production quality detection result are improved.

Description

Technical Field

[0001] This application relates to the field of industrial inspection technology, and more specifically, to a machine vision-based method and system for SMT (Surface Mount Technology) assembly production quality inspection. Background Technology

[0002] As the electronics manufacturing industry continues to demand higher-density assembly, more reliable soldering, and faster line changeover capabilities for a wider variety of products, surface mount technology (SMT) production has become one of the core processes in electronic product assembly. Therefore, using machine vision to inspect the entire SMT production process has become an important technical requirement for digital inspection and intelligent quality management in electronics manufacturing. However, traditional inspection methods still have many limitations and cannot meet the actual requirements in terms of inspection accuracy and efficiency.

[0003] In actual production scenarios, when the process reaches the solder paste printing and inspection stage, the solder paste edges are easily affected by trace flux seepage, localized high-brightness reflection, and optical halo interference. Traditional inspection methods often lack the ability to stably peel off false boundary areas, easily misidentifying reflective expansion areas as the actual solder paste coverage area, thus masking defects such as under-printing or missing printing. Secondly, in the component placement inspection stage, traditional inspection methods usually rely on standard markers to complete the entire board registration and directly generate the device inspection area. However, when the markers are contaminated, partially obscured, or slightly warped on the board surface, small registration errors can be easily introduced, causing the inspection area of ​​small-sized devices to shift, resulting in the device center or pad area falling out of the inspection window. Furthermore, for... For symmetrical surface mount devices, traditional inspection methods often rely on contour direction or the angle of the circumscribed rectangle for attitude recognition, making it difficult to reliably distinguish subtle attitude anomalies caused by slight rotation. Furthermore, in the inspection of fine-pin devices after reflow soldering, traditional methods typically employ uniform scaling or noise reduction strategies to process images, which can easily cause pin endpoints and solder transition boundaries to be swallowed up by quantization. For minor solder joint defects with approximately normal area but interrupted axial grayscale continuity, traditional inspection methods often rely on solder joint area or contour thresholds for judgment, lacking the ability to jointly analyze the continuity features of the pin direction and the geometric features of the solder joint, resulting in significant missed or false positives for solder joint defects. Consequently, the recognition accuracy and defect judgment accuracy are significantly insufficient.

[0004] In view of this, this application proposes a machine vision-based SMT assembly production quality inspection method and system to solve the above problems. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art and to achieve the above objectives, this application provides the following technical solution: a machine vision-based SMT (Surface Mount Technology) assembly production quality inspection method, comprising:

[0006] S1. Collect machine vision image data and production process parameters of the SMT assembly production station, and obtain the dataset to be inspected by performing data cleaning, including image data and process data;

[0007] S2. Identify the solder paste printing area based on the image data, perform printing boundary correction on the solder paste printing area, and generate printing inspection results to update the image data;

[0008] S3. Extract board-level reference information from image data, establish image coordinate mapping relationship based on the board-level reference information and perform registration evaluation to obtain registration quality index; identify the device detection area set based on the image coordinate mapping relationship and perform device position correction, and output the corrected detection area set;

[0009] S4. Based on the set of corrected detection regions, perform mounting posture detection on the patch corresponding to the image data, determine the relative relationship between the patch body features and the surrounding reference features, and output the mounting posture determination result;

[0010] S5. Combine process data and image data to identify fine-pin devices after reflow soldering; perform regionalized image processing on the detection area corresponding to the fine-pin device, and output the fine-pin feature image and the candidate feature set of pin endpoints to form fine-pin process data;

[0011] S6. Construct a continuous characterization coefficient for the direction of the fine lead based on the fine lead process data; obtain solder joint feature data, perform a fusion judgment based on the continuous characterization coefficient and the solder joint feature data, and output the soldering quality judgment result;

[0012] S7. Integrate the placement posture judgment results and the soldering quality judgment results to construct the placement process quality inspection log; send the placement process quality inspection log to the preset quality inspection terminal for storage.

[0013] Furthermore, methods for performing print boundary correction include:

[0014] Identify the image corresponding to each SMT assembly production station in the image data and extract the pixel grayscale matrix. Perform connected component processing on the pixel grayscale matrix to identify the solder paste printing area.

[0015] Construct a preset neighborhood window to traverse the solder paste printing area, and calculate the grayscale variance of the corresponding solder paste printing area within the preset neighborhood window;

[0016] Obtain the coordinates of the outer contour points of each solder paste printing area, and traverse along the centroid direction pointing to the solder paste printing area, starting from the coordinates of each outer contour pixel point; extract the gray value of each pixel point on the traversal path and the gray value variance of the corresponding preset neighborhood window;

[0017] If the gray value of the corresponding pixel is greater than the preset highlight threshold and the gray variance is less than the preset roughness threshold, then the pixel is removed from the solder paste printing area; if the gray variance of the pixel is not less than the preset roughness threshold, then the traversal stops and the pixel is marked as a real boundary pixel.

[0018] Connect all real boundary pixels to form a closed contour to obtain the corrected solder paste printing area, and calculate the area of ​​the corrected solder paste printing area.

[0019] The specific coordinates of the corrected solder paste printing area and the area of ​​the corrected solder paste printing area are integrated as printing inspection results to mark and update the image data.

[0020] Furthermore, methods for establishing image coordinate mapping relationships and performing registration evaluation include:

[0021] Obtain the target design data that belongs to the target content of patch production from the process data, identify the theoretical pixel coordinates of the standard marker points and draw the corresponding standard marker point image template;

[0022] Traverse the image data corresponding to the SMT assembly production station and perform normalized cross-correlation calculation with the standard marker point image template to obtain the cross-correlation coefficient; when the cross-correlation coefficient has an extreme value, extract the corresponding pixel coordinates, which are the actual marker points, and use the extreme value as the matching score.

[0023] An affine transformation matrix is ​​constructed based on the pixel coordinates of the standard marker points and the actual marker points, and this affine transformation matrix is ​​used as the image coordinate mapping relationship.

[0024] Substitute the pixel coordinates of the standard marker points into the affine transformation matrix to calculate the reprojection coordinates; calculate the Euclidean distance between each reprojection coordinate and the corresponding pixel coordinates of the actual marker point, which is used as the reprojection residual; calculate the mean of the reprojection residuals and the mean of the matching score of all standard marker points, and perform a weighted summation of the mean of the reprojection residuals and the mean of the matching score to output the registration quality index.

[0025] Furthermore, the methods for performing device position correction include:

[0026] Extract the theoretical center coordinates and theoretical size parameters of the corresponding patch from the target design data; substitute the theoretical center coordinates into the affine transformation matrix for mapping, and output the initial mapped center coordinates of the patch;

[0027] Using the initial mapping center coordinates as the center, construct the detection window according to the theoretical size parameters to obtain the initial detection area of ​​the corresponding patch;

[0028] Set the base outward pixel width and the maximum outward pixel width and calculate the width difference. Calculate the width change based on the width difference and the registration quality index. Sum the width change with the base outward pixel width to obtain the search expansion amount of the corresponding patch.

[0029] Each boundary of the initial detection region is translated outward by the length of the search expansion to construct a search window corresponding to the initial detection region; the gray-level gradient magnitude of each search window is calculated, and pixels with gray-level gradient magnitudes greater than a preset edge gradient threshold are selected to form an edge pixel set.

[0030] The edge pixel set is fitted to obtain at least two parallel edge lines corresponding to the patch; the coordinates of the intersection of the perpendicular bisectors of the at least two parallel edge lines are calculated and used as the actual reference center coordinates of the corresponding patch.

[0031] Calculate the coordinate difference vector between the actual reference center coordinates and the initial mapping center coordinates, and translate all boundary coordinates of all initial detection areas by the distance corresponding to the vector magnitude in the direction of the coordinate difference vector to obtain the set of corrected detection areas.

[0032] Furthermore, the methods for performing placement posture detection include:

[0033] Extract the search window of the corresponding patch in the set of correction detection areas and perform binarization processing; filter the pixels whose gray values ​​are in the first preset threshold range to form the body pixel set, and the pixels whose gray values ​​are in the second preset threshold range to form the pad pixel set.

[0034] Construct the minimum bounding rectangle of the body pixel set, extract the major axis direction vector of the minimum bounding rectangle as the body principal axis vector of the corresponding patch, and calculate the geometric center of the body pixel set.

[0035] A local coordinate system is established with the geometric center as the origin. The set of pad pixels is divided into a first pixel subset and a second pixel subset located on opposite sides of the origin. The absolute difference in the number of pixels between the first pixel subset and the second pixel subset is calculated as the area asymmetry. The centroid coordinates of the first pixel subset and the second pixel subset are identified and a reference vector is constructed by connecting them.

[0036] The angle between the main axis vector of the body and the reference vector is calculated as the rotation angle. If the area asymmetry is greater than the preset area tolerance threshold, or the rotation angle is greater than the preset angle tolerance threshold, the posture label of the corresponding patch is constructed as the mounting posture determination result.

[0037] Furthermore, methods for performing regionalized image processing include:

[0038] Obtain the center-to-center distance between adjacent pins of the corresponding patch in the process data. If the center-to-center distance between adjacent pins is less than the preset distance threshold, mark the corresponding patch as a fine-pin device and extract the detection area belonging to the fine-pin device in the image data.

[0039] Obtain the extension direction vector of each pin in the fine-pin device, perform gray-scale difference calculation on the corresponding detection area according to the direction of the extension direction vector, and construct the axial gradient sequence of the corresponding pin.

[0040] Extract the extreme pixel coordinates of the gradient magnitude extreme values ​​in the axial gradient sequence corresponding to each pin; extract adjacent gradient values ​​with the extreme pixel coordinates as the center and perform parabolic fitting; obtain the coordinates of the vertex of the fitted parabola as candidate coordinates of the pin endpoints.

[0041] The detection area is marked as a fine pin feature image, and the set of candidate pin endpoint coordinates of all pins in the fine pin device is taken as the pin endpoint candidate feature set.

[0042] Furthermore, the specific method for calculating the continuous characterization coefficient is as follows:

[0043] Starting from the candidate coordinates of the pin endpoint, extract the grayscale sequence according to the direction of the extension vector, and calculate the absolute value of the grayscale difference between adjacent pixels in the grayscale sequence. Count the number of pixels whose absolute grayscale difference is greater than the preset breakage threshold, and calculate the grayscale variance of the grayscale sequence. Combine the length of the grayscale sequence, the number of pixels, and the grayscale variance to calculate the continuity characterization coefficient.

[0044] Furthermore, the methods for performing fusion determination include:

[0045] Identify the solder connected region corresponding to each pin in the solder joint feature data, and calculate the area and perimeter of the solder connected region;

[0046] If the area of ​​the solder connected region is not less than the standard area threshold, a continuity determination is triggered; the continuity characterization coefficient of the corresponding pin is identified. If the continuity characterization coefficient is less than the safety threshold, the pin is determined to have a soldering defect, and a soldering defect label is constructed; if the continuity characterization coefficient is not less than the safety threshold and the contour perimeter is within the preset perimeter tolerance range, the pin is determined to be soldered qualified, and a qualified label is constructed; the soldering defect label and the qualified label are integrated to generate a soldering quality determination result.

[0047] Furthermore, methods for constructing a surface mount technology (SMT) process quality inspection log include:

[0048] The placement posture judgment result and the welding quality judgment result are associated and spliced ​​according to the position information of the corresponding patch, and converted into a log display format to form a patch process quality inspection log.

[0049] A machine vision-based SMT (Surface Mount Technology) assembly production quality inspection system, which implements a machine vision-based SMT assembly production quality inspection method, including:

[0050] The data acquisition module collects machine vision image data and production process parameters from the SMT assembly production station, and obtains the dataset to be inspected by performing data cleaning, including image data and process data.

[0051] The boundary correction module identifies the solder paste printing area based on image data, performs printing boundary correction on the solder paste printing area, and generates printing inspection results to update the image data.

[0052] The position calibration module extracts board-level reference information from the image data, establishes an image coordinate mapping relationship based on the board-level reference information, performs registration evaluation to obtain registration quality indicators, identifies the set of device detection areas based on the image coordinate mapping relationship, performs device position correction, and outputs the corrected detection area set.

[0053] The pose determination module performs mounting pose detection on the patch corresponding to the image data based on the set of correction detection regions, determines the relative relationship between the patch body and surrounding features, and outputs the mounting pose determination result.

[0054] The pin processing module combines process data and image data to identify fine-pin devices after reflow soldering; it performs regional image processing on the detection area corresponding to the fine-pin device and outputs fine-pin feature images and pin endpoint candidate feature sets to form fine-pin process data.

[0055] The welding judgment module constructs a continuous characterization coefficient for the direction of the fine lead based on the fine lead process data; acquires solder joint feature data; performs a fusion judgment based on the continuous characterization coefficient and the solder joint feature data; and outputs the welding quality judgment result.

[0056] The log generation module integrates the placement posture judgment results and the soldering quality judgment results to construct the surface mount process quality inspection log; the surface mount process quality inspection log is sent to a preset quality inspection terminal for storage; the various modules are connected to each other via wired and / or wireless means.

[0057] The technical effects and advantages of the machine vision-based SMT assembly production quality inspection method and system are as follows:

[0058] By collecting machine vision image data and process parameters from different workstations in the SMT assembly process, a machine vision-based SMT assembly production quality inspection method was realized. Compared with existing technologies, by performing boundary correction and region reconstruction on the solder paste printing area, the accuracy of identifying the true coverage of solder paste was improved, while enhancing the detection capability of printing defects in scenarios such as flux penetration and false boundary interference. Through standard marker point matching, image coordinate mapping, and device position correction, the detection area offset and positioning error caused by marker point contamination and slight board surface deformation were reduced. Through joint analysis of the component body features and pad reference features, the ability to identify slight rotation and abnormal posture of symmetrical component components was improved. By performing regional image processing and endpoint extraction on fine-pin components, the perception capability of pin endpoint detail loss and cold solder joint defects was enhanced. The traceability, consistency, and automation efficiency of SMT assembly production quality inspection results were improved. Attached Figure Description

[0059] Figure 1 This is a schematic diagram of a machine vision-based SMT assembly production quality inspection method according to this application;

[0060] Figure 2 This is a schematic diagram of a machine vision-based SMT assembly production quality inspection system according to this application. Detailed Implementation

[0061] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0062] Example 1

[0063] Please see Figure 1 As shown in this embodiment, a machine vision-based SMT assembly production quality inspection method includes:

[0064] S1. Collect machine vision image data and production process parameters of the SMT assembly production station, and obtain the dataset to be inspected by performing data cleaning, including image data and process data;

[0065] S2. Identify the solder paste printing area based on the image data, perform printing boundary correction on the solder paste printing area, and generate printing inspection results to update the image data;

[0066] S3. Extract board-level reference information from image data, establish image coordinate mapping relationship based on the board-level reference information and perform registration evaluation to obtain registration quality index; identify the device detection area set based on the image coordinate mapping relationship and perform device position correction, and output the corrected detection area set;

[0067] S4. Based on the set of corrected detection regions, perform mounting posture detection on the patch corresponding to the image data, determine the relative relationship between the patch body features and the surrounding reference features, and output the mounting posture determination result;

[0068] S5. Combine process data and image data to identify fine-pin devices after reflow soldering; perform regionalized image processing on the detection area corresponding to the fine-pin device, and output the fine-pin feature image and the candidate feature set of pin endpoints to form fine-pin process data;

[0069] S6. Construct a continuous characterization coefficient for the direction of the fine lead based on the fine lead process data; obtain solder joint feature data, perform a fusion judgment based on the continuous characterization coefficient and the solder joint feature data, and output the soldering quality judgment result;

[0070] S7. Integrate the placement posture judgment results and the soldering quality judgment results to construct the placement process quality inspection log; send the placement process quality inspection log to the preset quality inspection terminal for storage.

[0071] In this embodiment, the machine vision image data of the SMT assembly production station includes images of the solder paste printing station, the assembly station, and the reflow soldering station, which reflect the assembly production process. The production process parameters include the product batch number, component reference number, component package type, and soldering operation parameters of the corresponding station. Data cleaning is achieved by performing image size unification processing and filtering denoising on the image data, and performing missing value filling on the process data, to obtain a higher quality dataset of inspection data corresponding to the same assembly production object.

[0072] Methods for performing print boundary correction include:

[0073] The image corresponding to each SMT assembly station in the image data is identified and the pixel grayscale matrix is ​​extracted. The connected component processing of the pixel grayscale matrix is ​​then performed to identify the solder paste printing area.

[0074] In this embodiment, the image corresponding to each SMT production station refers to the image belonging to the solder paste printing process during the current printing boundary correction process. The original color image is converted into a grayscale image, and then the grayscale image is expanded according to rows and columns of pixels to obtain a pixel grayscale matrix. According to the requirements of solder paste brightness characteristics, the pixels in the pixel grayscale matrix that meet the requirements are gathered together as the solder paste printing area.

[0075] Construct a preset neighborhood window to traverse the solder paste printing area, and calculate the grayscale variance of the corresponding solder paste printing area within the preset neighborhood window.

[0076] The window size is set according to the theoretical range of general solder paste printing, which is small enough to allow for multiple complete scans of each solder paste printing area. For example, the window size can be set to 5*5 pixels or 7*7 pixels, and for scenarios with finer pitch, it can be set to 3*3 pixels. The formula for calculating the grayscale variance is as follows: ;in, This represents the grayscale variance of the corresponding preset neighborhood window; This indicates the number of pixels in the corresponding preset neighborhood window; Indicates the first element within the corresponding preset neighborhood window. The grayscale value of each pixel; This represents the average grayscale value within the corresponding preset neighborhood window; the grayscale variance of each preset neighborhood window is calculated to reflect whether the grayscale is smooth within that local area.

[0077] Obtain the coordinates of the outer contour points of each solder paste printing area, and traverse along the centroid direction pointing to the solder paste printing area, starting from the coordinates of each outer contour pixel point.

[0078] This involves using the coordinates of the outermost contour point of each solder paste printing area as the starting point, emitting rays in the direction pointing to the centroid coordinates of that solder paste printing area, and then traversing along the ray from each starting point using a preset neighborhood window.

[0079] Extract the grayscale value of each pixel along the traversal path and the grayscale variance of the corresponding preset neighborhood window.

[0080] The grayscale value of each pixel and the grayscale variance of the corresponding preset neighborhood window are extracted along the traversal path of each ray. Since the window size is small enough, the grayscale variance of the entire preset neighborhood window can be used to represent the grayscale variance of the corresponding pixel.

[0081] If the gray value of the corresponding pixel is greater than the preset highlight threshold and the gray variance is less than the preset roughness threshold, then the pixel is removed from the solder paste printing area.

[0082] The SMT (Surface Mount Technology) quality inspection specifications include preset highlight thresholds and preset roughness thresholds. The preset highlight threshold distinguishes between the bright, smooth areas formed by flux seepage and the actual coverage areas formed by solder paste particles. For example, under standard white light illumination, the grayscale value of the solder paste particle area is typically between 150 and 220, while the grayscale value of the bright areas formed by flux seepage is typically greater than 230. Therefore, the preset highlight threshold can be set to 230. Regarding the preset roughness threshold, for example, for a 5x5 pixel window, flux seepage... The grayscale variance of the region is generally below 50, while the grayscale variance of the solder paste particle region is generally above 80. Therefore, the preset roughness threshold is set to 80. If there are pixels with large grayscale values ​​and small grayscale variances on the traversal path, it means that the pixel has high brightness and small texture fluctuations. It may belong to high-brightness smooth pixels formed by flux overflow, surface specular reflection, or edge bright bands, and does not belong to the real coverage boundary formed by solder paste particles. Therefore, the pixel needs to be removed from the corresponding solder paste printing area to prevent the false boundary from being mistakenly identified as the solder paste coverage area.

[0083] If the grayscale variance of a pixel is not less than a preset roughness threshold, the traversal stops, and the pixel is marked as a true boundary pixel.

[0084] If the grayscale variance of a pixel reached on the traversal path is greater than or equal to the corresponding roughness threshold, it means that the local area has been transformed from smooth highlight to a real boundary of solder paste with obvious grain undulations. Therefore, the traversal is stopped and the pixel is retained as a real boundary pixel.

[0085] Connect all real boundary pixels to form a closed contour to obtain the corrected solder paste printing area, and calculate the area of ​​the corrected solder paste printing area.

[0086] The process involves connecting real boundary pixels sequentially in a clockwise or counterclockwise order to form a closed contour, creating a correction solder paste printing area after eliminating reflective interference; the area of ​​this region is obtained by counting the number of pixels in this correction solder paste printing area.

[0087] The specific coordinates of the corrected solder paste printing area and the area of ​​the corrected solder paste printing area are integrated as printing inspection results to mark and update the image data.

[0088] The process involves identifying the specific coordinates of each correction solder paste printing area and marking that area in the original image to update the original image data. At the same time, the area of ​​that area is also written into the dataset to be inspected as the attribute information of that area, so that subsequent operations can directly apply it to the processed image data.

[0089] Methods for establishing image coordinate mapping relationships and performing registration evaluation include:

[0090] Obtain the target design data that belongs to the target content of patch production from the process data, identify the theoretical pixel coordinates of the standard marker points and draw the corresponding standard marker point image template.

[0091] The target design data refers to the information extracted from the process data related to various target indicators in SMT (Surface Mount Technology) production, corresponding to the parameter types of the SMT production process targets. The standard marker points refer to the conversion of the theoretical coordinates of the marker points in the original design file into template coordinates in the image coordinate system, and the drawing of a local image template centered on each standard marker point. The size of this template needs to be large enough to completely traverse the image data for subsequent position search and coordinate mapping in the image data. It should be noted that the local image templates extracted here are the board-level reference information, which is the standard marker point image template.

[0092] Traverse the image data corresponding to the SMT assembly production station and perform normalized cross-correlation calculation with the standard marked point image template to obtain the cross-correlation coefficient.

[0093] This process involves traversing the image data of the corresponding SMT assembly station at the next stage of solder paste printing. Following a preset step size based on historical coordinate mapping experience, the matching window moves position by position within each image plane. At each position, a sub-image of the same size as the standard marker point image template is extracted for matching. Normalized cross-correlation calculations are then performed between the sub-image and the standard marker point image template. For example, in scenarios where the marker point spacing is no less than 200 pixels, the traversal step size can be set to 1 pixel to ensure that no candidate matching positions are missed.

[0094] The calculation formula is as follows: ;in, This represents the cross-correlation coefficient between the corresponding sub-image to be matched and the standard marked point image template; This represents a pixel grayscale vector whose grayscale values ​​are expanded in a certain order according to the corresponding standard marker point image template. This represents the average grayscale value of the corresponding standard marker point image template; Represents the pixel grayscale vector of the subimage to be matched; This represents the mean gray level of the sub-image to be matched; normalized cross-correlation calculation is performed to eliminate the influence of different image brightness, overall gray level fluctuations, or local exposure differences on the matching results.

[0095] When the cross-correlation coefficient reaches an extreme value, the corresponding pixel coordinates are extracted, which are the actual marker points, and the extreme value is used as the matching score.

[0096] Since the normalized cross-correlation coefficient is used to represent the degree of gray-scale structural similarity between the standard marker point image template and the local region of the image to be inspected, the normalized cross-correlation coefficient reaches its maximum value when the local region of the image to be inspected and the standard marker point image template achieve optimal spatial overlap. Therefore, the coordinates of this pixel are determined as the actual marker point coordinates, and this maximum value is used as the matching score. The matching score is used to represent the degree of similarity between the sub-image to be matched and the standard marker point image template. The higher the matching score, the closer the marker point in the actual image is to the theoretical template. Conversely, the lower the matching score, the more likely the marker point is to be contaminated, occluded, warped, or subject to reflection interference.

[0097] An affine transformation matrix is ​​constructed based on the pixel coordinates of the standard marker points and the actual marker points, and this affine transformation matrix is ​​used as the image coordinate mapping relationship.

[0098] Based on the correspondence between the pixel coordinates of standard marker points and actual marker points, the least squares method is used to calculate the linear transformation parameters of the image under translation, rotation, or local scaling conditions, and these linear transformation parameters are combined into an affine transformation matrix for coordinate mapping. This matrix realizes the transformation from the design coordinates to the coordinates of the actual image, ensuring that subsequent operations can be uniformly mapped according to the same coordinate transformation relationship, thereby reducing the detection deviation caused by the inconsistency of coordinate systems between different devices.

[0099] The reprojection coordinates are obtained by substituting the pixel coordinates of the standard marker points into the affine transformation matrix.

[0100] The coordinates of the theoretical marker points are mapped to the actual image coordinate system through an affine transformation matrix to obtain the reprojection coordinates of each standard marker point in the corresponding image. These coordinates are used to reflect the image position where the theoretical position should fall under the current coordinate mapping relationship.

[0101] Calculate the Euclidean distance between each reprojected coordinate and the corresponding pixel coordinate of the actual marked point, and use it as the reprojection residual.

[0102] The spatial deviation between the theoretical mapping position and the real image position is quantified by calculating the Euclidean distance between each reprojection coordinate and the corresponding pixel coordinate of the actual marker point. The larger the Euclidean distance, the greater the deviation between the theoretical coordinate and the actual coordinate. The calculated Euclidean distance is used as the reprojection residual to represent the geometric error that still exists in the local marker point after the transformation, which intuitively reflects the influence of factors such as slight warping of the board surface, local occlusion, or shooting angle deviation on the coordinate mapping relationship.

[0103] Calculate the mean reprojection residual and the mean matching sub-residual of all standard marker points, perform a weighted sum of the mean reprojection residual and the mean matching sub-residual, and output the registration quality index.

[0104] The process involves calculating the average error level and average similarity level of the overall image registration by averaging the geometric deviations and template matching results of all standard marker points. Weighted summation of the reprojection residual mean and the matching score mean involves setting corresponding preset weights for the error and similarity terms based on historical registration quality index calculation experience, mapping them together into a single numerical index. This ensures that the registration quality index reflects both the offset of marker points in spatial coordinates and the matching degree of marker points in image morphology. Regarding the preset weights, in this embodiment, the sum of the weights of the reprojection residual mean and the matching score mean is 1. For example, when the general quality of production line marker points is good, the weight of the matching score mean can be set to 0.4, and the weight of the reprojection residual mean can be set to 0.6 to emphasize geometric accuracy. When the production line marker points have a high risk of contamination, both weights can be set to 0.5 to balance the evaluation of the two indicators.

[0105] Methods for calibrating the position of actuators include:

[0106] Extract the theoretical center coordinates and theoretical dimension parameters of the corresponding patch from the target design data.

[0107] The theoretical center coordinates and theoretical dimension parameters are the center point coordinates and corresponding length and width dimensions of each patch device under test, read from the device design file under the original design theoretical conditions.

[0108] Substitute the theoretical center coordinates into the affine transformation matrix for mapping, and output the initial mapped center coordinates of the patch.

[0109] The theoretical center position is converted into the actual corresponding position in the current image using an affine transformation matrix, thus obtaining the initial mapping center coordinates of the patch. These initial mapping center coordinates reflect the preliminary predicted position of the patch device in the image under global registration conditions.

[0110] Using the initial mapping center coordinates as the center, a detection window is constructed according to the theoretical size parameters to obtain the initial detection area of ​​the corresponding patch.

[0111] The initial mapping center coordinates are used as the geometric center, and a rectangular or approximately rectangular region of the corresponding size is cropped from the image based on its theoretical packaging size to obtain the initial detection region of the corresponding patch.

[0112] Set the base outward pixel width and the maximum outward pixel width and calculate the width difference. Calculate the width change based on the width difference and the registration quality index.

[0113] The theoretical size of the patch device is used to define the basic and maximum outward pixel widths for local search, serving as the minimum and maximum expansion ranges to define the boundaries of the detection area. The specific formula for calculating the width change is as follows: ;in, Indicates the amount of width change; Indicates the maximum outward pixel width; Indicates the base outward pixel width; This represents the registration quality index after normalization. When the registration quality index is high, the width change approaches zero, and the local search window only retains the basic extended search range. When the registration quality index is low, the width change increases, allowing the local search window to move closer to the maximum extended search range, ensuring that the actual device boundary can still be completely covered even when there may be local occlusion or slight warping of the board surface.

[0114] The search expansion amount of the corresponding patch is obtained by summing the width change amount with the base outward pixel width.

[0115] This involves superimposing the base outward pixel width with the dynamic width change to form the actual expansion scale for subsequent local edge search. This search expansion can adaptively change the size of the local search window according to the registration quality index, so as to cover the real device boundary position even when there are small errors in the initial detection area, thereby improving the edge recovery capability and position correction capability of small-sized devices.

[0116] Each boundary of the initial detection region is shifted outward by the length of the search expansion to construct a search window corresponding to the initial detection region.

[0117] By synchronously expanding the perimeter of the initial detection area by the same number of pixels as the search expansion, a larger local search range is formed that surrounds the original detection area. This range is used to re-find the actual pad edges and the actual center position of the surface mount device within the local area, thus avoiding the initial mapping center from deviating from the actual device position due to global registration errors.

[0118] Calculate the grayscale gradient magnitude of each search window, and select pixels whose grayscale gradient magnitude is greater than the preset edge gradient threshold to form an edge pixel set.

[0119] The preset edge gradient threshold is set based on the grayscale change of theoretical edge pixels. For example, under standard lighting conditions, the grayscale gradient amplitude of the pad edge area is usually higher than 50, so the preset edge gradient threshold can be set to 50. By calculating the grayscale gradient amplitude of each pixel in the search window and comparing it with the corresponding threshold, if the grayscale gradient amplitude is large, it means that the pixel has a strong edge response. Therefore, these pixels are integrated into an edge pixel set.

[0120] The edge pixel set is fitted to obtain at least two parallel edge lines corresponding to the patch.

[0121] In this embodiment, a straight line fitting algorithm is used to fit discrete edge pixels into an approximately continuous geometric boundary. Based on the geometric arrangement of the device pads in the image, two parallel edge lines that can reflect the left and right or top and bottom boundaries of the device are extracted from the candidate edges. These parallel edge lines are used to determine the actual mounting direction and actual center position of the device.

[0122] Calculate the coordinates of the intersection of the perpendicular bisectors of the at least two mutually parallel edge lines, and use them as the actual reference center coordinates of the corresponding patch.

[0123] This involves drawing and extending the perpendicular bisectors of two parallel boundaries, using the intersection point as the center position, ensuring that the distance from this center position to both sides of the boundary is approximately equal; and using the coordinates of this intersection point as the actual reference center coordinates of the corresponding patch.

[0124] Calculate the coordinate difference vector between the actual reference center coordinates and the initial mapping center coordinates, and translate all boundary coordinates of all initial detection areas by the distance corresponding to the vector magnitude in the direction of the coordinate difference vector to obtain the set of corrected detection areas.

[0125] The coordinate difference vector is formed by subtracting the horizontal and vertical coordinates of the actual reference center coordinates from the initial mapped center coordinates. This vector represents the direction and magnitude of the offset of the current device's actual position in the image relative to the initial predicted position. Therefore, translating the corresponding vector magnitude by the direction pointed to by this coordinate difference vector performs position compensation on the entire initial detection area, enabling the detection window to re-cover the actual area where the device is located and the corresponding pad range.

[0126] The methods for performing placement posture detection include:

[0127] Extract the search window for the corresponding patch from the set of calibration detection regions and perform binarization processing.

[0128] The process involves converting the local original grayscale image in the search window corresponding to each patch into a binary image according to a certain grayscale segmentation threshold. This is used to reduce the influence of factors such as illumination fluctuations and background textures on subsequent contour segmentation, and to perform preliminary separation of the patch device body, pads, and background at the pixel level.

[0129] Pixels whose grayscale values ​​fall within a first preset threshold range are selected to form the body pixel set, and pixels whose grayscale values ​​fall within a second preset threshold range are selected to form the pad pixel set.

[0130] The first preset threshold range and the second preset threshold range are set according to the distribution of theoretical grayscale values ​​in the relevant images of the surface mount device. The first preset threshold range is used to limit the target grayscale range of the surface mount device body in the binary image, and the second preset threshold range is used to limit the target grayscale range of the pads in the binary image. For example, the grayscale value of the black packaged device body under standard lighting conditions is usually between 20 and 80, so the first preset threshold range can be set to 20 to 80; the grayscale value of the metal pads under standard lighting conditions is usually between 180 and 255, so the second preset threshold range can be set to 180 to 255.

[0131] Construct the minimum bounding rectangle of the body pixel set, extract the major axis direction vector of the minimum bounding rectangle as the body principal axis vector of the corresponding patch, and calculate the geometric center of the body pixel set.

[0132] The process involves constructing the minimum bounding rectangle of the region formed by all pixels in the ontology pixel set, quantizing the extension direction of this set in the image into the ontology principal axis vector of the corresponding patch, and using geometric knowledge to calculate the geometric center of the region corresponding to this ontology pixel set.

[0133] A local coordinate system is established with the geometric center as the origin, and the set of pad pixels is divided into a first pixel subset and a second pixel subset located on opposite sides of the origin.

[0134] Using the geometric center of the surface mount device as a local reference point, the pad pixel set is re-divided according to its spatial position relative to the origin, resulting in a first pixel subset and a second pixel subset located on opposite sides of the origin. This operation is used to extract the pixel distribution of the pads at both ends of the surface mount device in the local coordinate system.

[0135] The absolute difference in the number of pixels between the first and second pixel subsets is calculated as the area asymmetry.

[0136] This method involves counting the number of pixels in the first and second pixel subsets respectively, taking the absolute value of the difference, and using this value as the area asymmetry. This quantifies the difference in pixel coverage between the pads on both sides of the surface mount device into a numerical index.

[0137] Identify the centroid coordinates of the first and second pixel subsets, and construct a reference vector by connecting them.

[0138] This involves determining the spatial center position of the pixel distribution set of the pads at both ends of the surface mount device, and constructing a reference vector by connecting the two centroid coordinates. This vector represents the theoretical mounting direction or reference mounting direction of the surface mount device on the pads.

[0139] The angle between the principal axis vector of the body and the reference datum vector is calculated as the rotation angle.

[0140] The formula for calculating the rotation angle is as follows: ;in This represents the angle between the principal axis vector of the entity and the reference vector. Represents the principal axis vector of the entity; This represents the reference vector.

[0141] If the area asymmetry is greater than the preset area tolerance threshold, or the rotation angle is greater than the preset angle tolerance threshold, a corresponding patch posture label is constructed as the mounting posture determination result.

[0142] The system sets preset area tolerance thresholds and preset angle tolerance thresholds based on the distribution of area asymmetry and rotation angles in historical surface mount device (SMT) posture anomalies. For example, for certain types of packaged devices, the difference in the number of pixels between the pads at both ends under normal mounting conditions is usually no more than 10 pixels, so the preset area tolerance threshold can be set to 10. The allowable rotation deviation for this type of packaged device is usually no more than 3°, so the preset angle tolerance threshold can be set to 3°. If the area asymmetry or rotation angle is large, it indicates that there is at least one abnormality in the coverage relationship of the corresponding surface mount device on the pads on both sides or in the mounting direction of the device body. Based on the experience of abnormal situations and the specific values ​​of the area asymmetry or rotation angle, the system determines which abnormal posture situation the surface mount device belongs to, constructs a label for the abnormal posture situation, outputs and records it.

[0143] Methods for performing regionalized image processing include:

[0144] Obtain the center-to-center distance between adjacent pins of the corresponding patch in the process data. If the center-to-center distance between adjacent pins is less than a preset distance threshold, mark the corresponding patch as a fine-pin device and extract the detection area belonging to the fine-pin device from the image data.

[0145] The system sets a preset spacing threshold based on SMT assembly production specifications. It determines whether a component is a fine-pitch device by judging whether the center-to-center distance between adjacent pins is less than the corresponding spacing threshold. For example, a device with a pin pitch of less than 0.5mm is usually considered a fine-pitch device, so the preset spacing threshold can be set to 0.5mm. The detection area of ​​the fine-pitch device is obtained by identifying the image area in the set of calibration monitoring areas that corresponds to the characteristics of the fine-pitch device.

[0146] Obtain the extension direction vector of each pin in the fine-pin device, perform gray-scale difference calculation on the corresponding detection area according to the direction of the extension direction vector, and construct the axial gradient sequence of the corresponding pin.

[0147] Based on the package layout, pin arrangement direction, and device body orientation of the fine-pin device, the main direction of each pin extending outward from the device body is determined and converted into a vector form, namely the extension direction vector. The pixel grayscale values ​​in the corresponding detection area are read pixel by pixel according to the direction of the extension direction vector of each pin, and the grayscale change between adjacent pixels is calculated. These grayscale changes are sorted according to their spatial position to form the axial gradient sequence of the corresponding pin.

[0148] Extract the extreme pixel coordinates corresponding to the extreme values ​​of the gradient magnitude in the axial gradient sequence for each pin.

[0149] The process involves identifying the location with the most significant gradient change in the axial gradient sequence and recording the corresponding pixel coordinates as candidate extreme value pixel coordinates. The extreme values ​​of gradient magnitude generally correspond to locations where the solder continuity changes significantly, such as the solder boundary at the pin end or the starting boundary of the pad. Therefore, the pixel coordinates corresponding to the extreme values ​​of gradient magnitude are extracted.

[0150] The adjacent gradient values ​​are extracted with the extreme pixel coordinates as the center to perform parabolic fitting, and the coordinates of the vertex of the fitted parabola are used as candidate coordinates of the pin endpoint.

[0151] In this embodiment, a local quadratic curve fitting algorithm is used to select several adjacent sampling points near the gradient extremum point, and construct a fitted parabola based on the gradient change trend of these sampling points. Then, the position with the strongest local gradient change, that is, the vertex coordinate of the fitted parabola, is further refined into sub-pixel level endpoint positions, so that the extremum point coordinates that could only be located at the discrete pixel level can be continuously corrected.

[0152] The detection area is marked as a fine pin feature image, and the set of candidate pin endpoint coordinates of all pins in the fine pin device is taken as the pin endpoint candidate feature set.

[0153] The local image corresponding to the detection area after regionalization is used as the fine pin feature image. The candidate coordinates of the endpoints of each pin are uniformly summarized to form a structured coordinate set as the pin endpoint candidate feature set. The geometric features such as the length and spacing of the corresponding pin can be calculated through this pin endpoint candidate feature set.

[0154] The specific method for calculating the continuous characterization coefficient is as follows:

[0155] Using the candidate coordinates of the pin endpoint as the starting point, extract the grayscale sequence according to the direction of the extension direction vector, and at the same time calculate the absolute value of the grayscale difference between adjacent pixels in the grayscale sequence.

[0156] The fine pin feature image in the fine pin process data is sampled point by point along the longitudinal axis of each pin, and the brightness change amplitude between adjacent sampling points is recorded to form a grayscale sequence that reflects the grayscale continuity change of the pin from the device body to the pad.

[0157] The number of pixels whose absolute grayscale difference is greater than a preset breakage threshold is counted, and the grayscale variance of the grayscale sequence is calculated.

[0158] The system sets a preset breakage threshold based on SMT assembly production specifications. For example, under normal soldering conditions, the grayscale difference between adjacent pixels is generally no more than 15, while at the location of poor soldering or soldering breakage, the grayscale difference between adjacent pixels often exceeds 30. Therefore, the preset breakage threshold is set to 30. The system counts the number of pixels whose absolute grayscale difference is higher than the preset breakage threshold. This counts the locations of obvious brightness abrupt changes along the pin direction as continuity interruption points. At the same time, the grayscale variance is calculated to represent the degree of grayscale fluctuation in the entire sequence.

[0159] The continuous characterization coefficient is calculated by combining the length of the grayscale sequence, the number of pixels, and the grayscale variance.

[0160] The formula for calculating the continuous characterization coefficient is as follows: ;in Indicates continuous characterization coefficients; This represents a standardized grayscale sequence length obtained by normalization based on the original grayscale sequence length. This value has a unified comparison benchmark between different pins, so the continuous characterization coefficient will not be directly high or low due to the physical length of the pins of different devices. The normalization process refers to calculating the ratio of the original grayscale sequence length to the theoretical reference length of the corresponding pin. This indicates the number of pixels whose absolute grayscale difference is greater than a preset breakage threshold. Indicates the variance of gray levels; It represents a very small positive number that avoids a denominator of 0.

[0161] It should be noted that the continuity coefficient is a dimensionless evaluation index. This coefficient is only used to evaluate the continuity between pixels. All physical quantities involved in the calculation have been normalized to eliminate dimensional differences. A larger continuity coefficient indicates that the pin soldering continuity is good, the grayscale change is smooth, there are fewer abrupt changes, and the variance is small.

[0162] In this embodiment, the solder joint feature data includes, for example, the area, perimeter, boundary coordinates, centroid coordinates, length and width dimensions, and coverage relationship information of the solder connected region corresponding to each pin. By using the candidate coordinates of each pin endpoint and the corresponding pin extension direction vector as references, the local solder joint region corresponding to each pin is extracted from the feature image of the thin pin, and grayscale segmentation and connected region extraction are performed on the local solder joint region to identify the solder connected region corresponding to the pin. The area is obtained by counting the number of pixels inside the solder connected region, and the contour pixels are extracted along the outer boundary of the solder connected region and the boundary length is accumulated to obtain the contour perimeter. At the same time, the geometric center, boundary distribution range, and degree of overlap with the theoretical solder pad region of the solder connected region are calculated to obtain the solder joint feature data of the corresponding pin.

[0163] The methods for performing fusion determination include:

[0164] Identify the solder connectivity region corresponding to each pin in the solder joint feature data, and calculate the area and perimeter of the solder connectivity region.

[0165] This involves extracting connected components from the solder area corresponding to each pin and calculating the pixel coverage area and circumscribed contour length of the connected component. The area is used to reflect whether the solder coverage meets the basic soldering requirements, and the contour perimeter is used to reflect whether the solder boundary is continuous or has abnormal conditions such as local broken lines.

[0166] If the area of ​​the solder connected region is not less than the standard area threshold, then a continuity determination is triggered.

[0167] The standard area threshold is set based on the soldering specifications. For example, for QFP packaged devices, the minimum effective soldering area of ​​each pin is usually not less than 60% of the theoretical area of ​​the pad. If the theoretical area of ​​the pad is 0.2 square millimeters, then the minimum effective soldering area is 0.12 square millimeters. Therefore, this value is used as the standard area threshold. If the area of ​​the soldered connected region is small, it indicates that there may be insufficient solder coverage. Only when the area is higher than the corresponding threshold can the next continuity judgment be performed.

[0168] Identify the continuous characterization coefficient of the corresponding pin. If the continuous characterization coefficient is less than the safety threshold, it is determined that the pin has a welding defect, and a welding defect label is constructed.

[0169] The safety threshold is set based on historical experience. For example, through statistical analysis of historical data, the continuity characteristic coefficient of a normally soldered pin is usually higher than 0.4, while the continuity characteristic coefficient of a poorly soldered pin is usually lower than 0.2. Therefore, the safety threshold can be set to 0.3 to provide a suitable judgment boundary between the two types of pins. If the continuity characteristic coefficient is lower than the safety threshold, it indicates that there is a significant abrupt change or continuous interruption in the grayscale change of the pin along the axial direction. There may be a discontinuous solder wetting process or insufficient soldering transition. Therefore, the pin is judged as a soldering defect and a soldering defect label is output. It should be noted that the reasons for the continuity characteristic coefficient being lower than the safety threshold are not limited to poor soldering. They may also include non-soldering factors such as pin surface oxidation and foreign matter contamination. Therefore, the soldering defect label in this embodiment refers to the risk of abnormal soldering continuity. The specific defect needs to be confirmed in combination with the actual production situation.

[0170] If the continuous characterization coefficient is not less than the safety threshold and the contour perimeter is within the preset perimeter tolerance range, then the pin welding is deemed qualified and a qualified label is constructed.

[0171] If the continuity coefficient is not less than the safety threshold and the outline perimeter is within the preset perimeter tolerance range set based on the welding specifications, it indicates that the grayscale continuity of the pin axis meets the requirements. At the same time, the shape of the solder connected region is sufficient, indicating that the pin has neither obvious cold solder joint interruption nor abnormal expansion or contraction of the solder boundary. Therefore, the pin is judged as welded and a qualified label is output. The preset perimeter tolerance range is set based on the allowable range of the outline perimeter of the normal shape of the solder joint. It is used to identify abnormal expansion, contraction or edge burrs at the solder joint boundary. For example, the theoretical value of the normal outline perimeter of a certain type of device is about 40 pixels, and the allowable deviation range is 20%. Therefore, the preset perimeter tolerance range can be set to 32 to 48 pixels.

[0172] Integrate welding defect labels and pass labels to generate welding quality assessment results.

[0173] Methods for constructing surface mount technology (SMT) process quality inspection logs include:

[0174] The placement posture judgment result and the welding quality judgment result are associated and spliced ​​according to the position information of the corresponding patch, and converted into a log display format to form a patch process quality inspection log.

[0175] The posture judgment result obtained by the same surface mount device during the surface mount inspection stage and the welding quality judgment result obtained during the post-welding inspection stage are associated one-to-one according to the device tag number, image coordinates and workstation number of the surface mount device. The associated result is encoded into a structured log record, so that the quality information of the same surface mount device at different inspection stages can be presented synchronously in the same log.

[0176] This embodiment realizes a machine vision-based SMT assembly production quality inspection method by collecting machine vision image data and process parameters from different workstations during the SMT assembly process. Compared with existing technologies, by performing boundary correction and region reconstruction on the solder paste printing area, the accuracy of identifying the true coverage of solder paste is improved, while enhancing the detection capability of printing defects in scenarios such as flux penetration and false boundary interference. By matching standard marker points, image coordinate mapping, and device position correction, the detection area offset and positioning error caused by marker point contamination and slight board surface deformation are reduced. By jointly analyzing the features of the component body and the pad reference features, the ability to identify slight rotation and abnormal posture of symmetrical component components is improved. By performing regional image processing and endpoint extraction on fine-pin components, the perception capability of pin endpoint detail loss and cold solder joint defects is enhanced. The traceability, consistency, and automation efficiency of SMT assembly production quality inspection results are improved.

[0177] Example 2

[0178] Please see Figure 2 As shown, parts not described in detail in this embodiment are described in Embodiment 1. A machine vision-based SMT assembly production quality inspection system is provided, including:

[0179] The data acquisition module collects machine vision image data and production process parameters from the SMT assembly production station, and obtains the dataset to be inspected by performing data cleaning, including image data and process data.

[0180] The boundary correction module identifies the solder paste printing area based on image data, performs printing boundary correction on the solder paste printing area, and generates printing inspection results to update the image data.

[0181] The position calibration module extracts board-level reference information from the image data, establishes an image coordinate mapping relationship based on the board-level reference information, performs registration evaluation to obtain registration quality indicators, identifies the set of device detection areas based on the image coordinate mapping relationship, performs device position correction, and outputs the corrected detection area set.

[0182] The pose determination module performs mounting pose detection on the patch corresponding to the image data based on the set of correction detection regions, determines the relative relationship between the patch body and surrounding features, and outputs the mounting pose determination result.

[0183] The pin processing module combines process data and image data to identify fine-pin devices after reflow soldering; it performs regional image processing on the detection area corresponding to the fine-pin device and outputs fine-pin feature images and pin endpoint candidate feature sets to form fine-pin process data.

[0184] The welding judgment module constructs a continuous characterization coefficient for the direction of the fine lead based on the fine lead process data; acquires solder joint feature data; performs a fusion judgment based on the continuous characterization coefficient and the solder joint feature data; and outputs the welding quality judgment result.

[0185] The log generation module integrates the placement posture judgment results and the soldering quality judgment results to construct the surface mount process quality inspection log; the surface mount process quality inspection log is sent to a preset quality inspection terminal for storage; the various modules are connected to each other via wired and / or wireless means.

[0186] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0187] All formulas in this manual are dimensionless and calculated numerically. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0188] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A machine vision-based method for SMT (Surface Mount Technology) assembly production quality inspection, characterized in that, include: S1. Collect machine vision image data and production process parameters of the SMT assembly production station, and obtain the dataset to be inspected by performing data cleaning, including image data and process data; S2. Identify the solder paste printing area based on the image data, perform printing boundary correction on the solder paste printing area, and generate printing inspection results to update the image data; S3. Extract board-level reference information from image data, establish image coordinate mapping relationship based on the board-level reference information and perform registration evaluation to obtain registration quality index; identify the device detection area set based on the image coordinate mapping relationship and perform device position correction, and output the corrected detection area set; S4. Based on the set of corrected detection regions, perform mounting posture detection on the patch corresponding to the image data, determine the relative relationship between the patch body features and the surrounding reference features, and output the mounting posture determination result; S5. Combine process data and image data to identify fine-pin devices after reflow soldering; perform regionalized image processing on the detection area corresponding to the fine-pin device, and output the fine-pin feature image and the candidate feature set of pin endpoints to form fine-pin process data; S6. Construct a continuous characterization coefficient for the direction of the fine lead based on the fine lead process data; obtain solder joint feature data, perform a fusion judgment based on the continuous characterization coefficient and the solder joint feature data, and output the soldering quality judgment result; S7. Integrate the placement posture judgment results and the soldering quality judgment results to construct the placement process quality inspection log; send the placement process quality inspection log to the preset quality inspection terminal for storage.

2. The machine vision-based SMT assembly production quality inspection method according to claim 1, characterized in that, The methods for performing print boundary correction include: Identify the image corresponding to each SMT assembly production station in the image data and extract the pixel grayscale matrix. Perform connected component processing on the pixel grayscale matrix to identify the solder paste printing area. Construct a preset neighborhood window to traverse the solder paste printing area, and calculate the grayscale variance of the corresponding solder paste printing area within the preset neighborhood window; Obtain the coordinates of the outer contour points of each solder paste printing area, and traverse along the centroid direction pointing to the solder paste printing area, starting from the coordinates of each outer contour pixel point; extract the gray value of each pixel point on the traversal path and the gray value variance of the corresponding preset neighborhood window; If the gray value of the corresponding pixel is greater than the preset highlight threshold and the gray variance is less than the preset roughness threshold, then the pixel is removed from the solder paste printing area; if the gray variance of the pixel is not less than the preset roughness threshold, then the traversal stops and the pixel is marked as a real boundary pixel. Connect all real boundary pixels to form a closed contour to obtain the corrected solder paste printing area, and calculate the area of ​​the corrected solder paste printing area. The specific coordinates of the corrected solder paste printing area and the area of ​​the corrected solder paste printing area are integrated as printing inspection results to mark and update the image data.

3. The SMT assembly production quality inspection method based on machine vision according to claim 2, characterized in that, The methods for establishing image coordinate mapping relationships and performing registration evaluation include: Obtain the target design data that belongs to the target content of patch production from the process data, identify the theoretical pixel coordinates of the standard marker points and draw the corresponding standard marker point image template; Traverse the image data corresponding to the SMT assembly production station and perform normalized cross-correlation calculation with the standard marker point image template to obtain the cross-correlation coefficient; when the cross-correlation coefficient has an extreme value, extract the corresponding pixel coordinates, which are the actual marker points, and use the extreme value as the matching score. An affine transformation matrix is ​​constructed based on the pixel coordinates of the standard marker points and the actual marker points, and this affine transformation matrix is ​​used as the image coordinate mapping relationship. Substitute the pixel coordinates of the standard marker points into the affine transformation matrix to calculate the reprojection coordinates; calculate the Euclidean distance between each reprojection coordinate and the corresponding pixel coordinates of the actual marker point, which is used as the reprojection residual; calculate the mean of the reprojection residuals and the mean of the matching score of all standard marker points, and perform a weighted summation of the mean of the reprojection residuals and the mean of the matching score to output the registration quality index.

4. The SMT assembly production quality inspection method based on machine vision according to claim 3, characterized in that, The methods for correcting the position of the actuator include: Extract the theoretical center coordinates and theoretical size parameters of the corresponding patch from the target design data; substitute the theoretical center coordinates into the affine transformation matrix for mapping, and output the initial mapped center coordinates of the patch; Using the initial mapping center coordinates as the center, construct the detection window according to the theoretical size parameters to obtain the initial detection area of ​​the corresponding patch; Set the base outward pixel width and the maximum outward pixel width and calculate the width difference. Calculate the width change based on the width difference and the registration quality index. Sum the width change with the base outward pixel width to obtain the search expansion amount of the corresponding patch. Each boundary of the initial detection region is translated outward by the length of the search expansion to construct a search window corresponding to the initial detection region; the gray-level gradient magnitude of each search window is calculated, and pixels with gray-level gradient magnitudes greater than a preset edge gradient threshold are selected to form an edge pixel set. The edge pixel set is fitted to obtain at least two parallel edge lines corresponding to the patch; the coordinates of the intersection of the perpendicular bisectors of the at least two parallel edge lines are calculated and used as the actual reference center coordinates of the corresponding patch. Calculate the coordinate difference vector between the actual reference center coordinates and the initial mapping center coordinates, and translate all boundary coordinates of all initial detection areas by the distance corresponding to the vector magnitude in the direction of the coordinate difference vector to obtain the set of corrected detection areas.

5. The SMT assembly production quality inspection method based on machine vision according to claim 4, characterized in that, The methods for performing mounting posture detection include: Extract the search window of the corresponding patch in the set of correction detection areas and perform binarization processing; filter the pixels whose gray values ​​are in the first preset threshold range to form the body pixel set, and the pixels whose gray values ​​are in the second preset threshold range to form the pad pixel set. Construct the minimum bounding rectangle of the body pixel set, extract the major axis direction vector of the minimum bounding rectangle as the body principal axis vector of the corresponding patch, and calculate the geometric center of the body pixel set. A local coordinate system is established with the geometric center as the origin. The set of pad pixels is divided into a first pixel subset and a second pixel subset located on opposite sides of the origin. The absolute difference in the number of pixels between the first pixel subset and the second pixel subset is calculated as the area asymmetry. The centroid coordinates of the first pixel subset and the second pixel subset are identified and a reference vector is constructed by connecting them. The angle between the main axis vector of the body and the reference vector is calculated as the rotation angle. If the area asymmetry is greater than the preset area tolerance threshold, or the rotation angle is greater than the preset angle tolerance threshold, the posture label of the corresponding patch is constructed as the mounting posture determination result.

6. The SMT assembly production quality inspection method based on machine vision according to claim 5, characterized in that, The methods for performing regionalized image processing include: Obtain the center-to-center distance between adjacent pins of the corresponding patch in the process data. If the center-to-center distance between adjacent pins is less than the preset distance threshold, mark the corresponding patch as a fine-pin device and extract the detection area belonging to the fine-pin device in the image data. Obtain the extension direction vector of each pin in the fine-pin device, perform gray-scale difference calculation on the corresponding detection area according to the direction of the extension direction vector, and construct the axial gradient sequence of the corresponding pin. Extract the extreme pixel coordinates of the gradient magnitude extreme values ​​in the axial gradient sequence corresponding to each pin; extract adjacent gradient values ​​with the extreme pixel coordinates as the center and perform parabolic fitting; obtain the coordinates of the vertex of the fitted parabola as candidate coordinates of the pin endpoints. The detection area is marked as a fine pin feature image, and the set of candidate pin endpoint coordinates of all pins in the fine pin device is taken as the pin endpoint candidate feature set.

7. The machine vision-based SMT assembly production quality inspection method according to claim 6, characterized in that, The specific method for calculating the continuous characterization coefficient is as follows: Starting from the candidate coordinates of the pin endpoint, extract the grayscale sequence according to the direction of the extension vector, and calculate the absolute value of the grayscale difference between adjacent pixels in the grayscale sequence. Count the number of pixels whose absolute grayscale difference is greater than the preset breakage threshold, and calculate the grayscale variance of the grayscale sequence. The continuous characterization coefficient is calculated by combining the length of the grayscale sequence, the number of pixels, and the grayscale variance.

8. The SMT assembly production quality inspection method based on machine vision according to claim 7, characterized in that, The methods for performing fusion determination include: Identify the solder connected region corresponding to each pin in the solder joint feature data, and calculate the area and perimeter of the solder connected region; If the area of ​​the solder connected region is not less than the standard area threshold, a continuity determination is triggered; the continuity characterization coefficient of the corresponding pin is identified. If the continuity characterization coefficient is less than the safety threshold, the pin is determined to have a soldering defect, and a soldering defect label is constructed; if the continuity characterization coefficient is not less than the safety threshold and the contour perimeter is within the preset perimeter tolerance range, the pin is determined to be soldered qualified, and a qualified label is constructed; the soldering defect label and the qualified label are integrated to generate a soldering quality determination result.

9. A machine vision-based SMT assembly production quality inspection method according to claim 8, characterized in that, The methods for constructing the patch panel process quality inspection log include: The placement posture judgment result and the welding quality judgment result are associated and spliced ​​according to the position information of the corresponding patch, and converted into a log display format to form a patch process quality inspection log.

10. A machine vision-based SMT assembly production quality inspection system, used to implement the machine vision-based SMT assembly production quality inspection method according to any one of claims 1-9, characterized in that, include: The data acquisition module collects machine vision image data and production process parameters from the SMT assembly production station, and obtains the dataset to be inspected by performing data cleaning, including image data and process data. The boundary correction module identifies the solder paste printing area based on image data, performs printing boundary correction on the solder paste printing area, and generates printing inspection results to update the image data. The position calibration module extracts board-level reference information from the image data, establishes an image coordinate mapping relationship based on the board-level reference information, performs registration evaluation to obtain registration quality indicators, identifies the set of device detection areas based on the image coordinate mapping relationship, performs device position correction, and outputs the corrected detection area set. The pose determination module performs mounting pose detection on the patch corresponding to the image data based on the set of correction detection regions, determines the relative relationship between the patch body and surrounding features, and outputs the mounting pose determination result. The pin processing module combines process data and image data to identify fine-pin devices after reflow soldering; it performs regional image processing on the detection area corresponding to the fine-pin device and outputs fine-pin feature images and pin endpoint candidate feature sets to form fine-pin process data. The welding judgment module constructs a continuous characterization coefficient for the direction of the fine lead based on the fine lead process data; acquires solder joint feature data; performs a fusion judgment based on the continuous characterization coefficient and the solder joint feature data; and outputs the welding quality judgment result. The log generation module integrates the placement posture judgment results and the soldering quality judgment results to construct the surface mount process quality inspection log; the surface mount process quality inspection log is sent to a preset quality inspection terminal for storage; the various modules are connected to each other via wired and / or wireless means.

Images