PCB defect detection method and system based on AI vision

Abstract

The application is suitable for the technical field of PCB detection, and particularly relates to a PCB defect detection method and system based on AI vision, which comprises the following steps: obtaining original image data of a target PCB area shot by a first vision device, and obtaining design parameters of the target PCB area to provide a benchmark for subsequent analysis; determining positioning deviation and feature deviation of the first vision device in shooting the target PCB area according to the original image data and the design parameters of the target PCB area, so as to realize quantitative positioning and feature deviation; performing deviation compensation processing on the original image data according to the positioning deviation and the feature deviation, so as to obtain corrected image data of the target PCB area, and effectively avoid problems such as line shape distortion and size distortion caused by shooting deviation; and determining a defect detection result of the target PCB area based on the corrected image data, so as to improve the stability and reliability of intelligent detection.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of PCB detection, and particularly relates to a PCB defect detection method and system based on AI vision. BACKGROUND

[0002] With the rapid development of the electronic manufacturing industry, printed circuit board (PCB) detection technology has become a key link to ensure product quality. With the development of artificial intelligence, especially deep learning technology, PCB defect detection through image input into a machine vision model has attracted widespread attention. This technology not only improves detection speed and accuracy, but also reduces the need for human intervention.

[0003] However, due to factors such as installation deviation of vision equipment, mechanical motion precision limitation, and lens optical characteristics, the lines and pads of the PCB in the image may have shape distortion, size distortion, and other image itself deviation problems. During detection, the image data itself deviation will lead to inaccurate defect detection, especially in high-density and high-precision PCB detection scenarios, where the line spacing is extremely small and the component layout is compact. Small image deviations will be amplified, making the detection process require frequent manual review and calibration, significantly reducing detection efficiency and increasing production costs. SUMMARY

[0004] The embodiments of the application provide a PCB defect detection method and system based on AI vision, which can solve the problem of inaccurate defect detection caused by image data itself deviation in the PCB defect detection process.

[0005] In a first aspect, the embodiments of the application provide a PCB defect detection method based on AI vision, comprising:

[0006] Obtaining original image data of a target PCB region photographed by a first vision equipment;

[0007] Obtaining design parameters of the target PCB region;

[0008] Determining a positioning deviation and a feature deviation of the first vision equipment in the target PCB region according to the original image data and the design parameters of the target PCB region;

[0009] Performing deviation compensation processing on the original image data according to the positioning deviation and the feature deviation to obtain corrected image data of the target PCB region;

[0010] Determining a defect detection result of the target PCB region based on the corrected image data.

[0011] The above technical solutions in the embodiments of the application have at least the following technical effects:

[0012] The AI vision-based PCB defect detection method provided by the embodiments of the present application obtains original image data of a target PCB region captured by a first vision device, and obtains design parameters of the target PCB region to provide a reference for subsequent analysis. According to the original image data and the design parameters of the target PCB region, positioning deviation and feature deviation of the first vision device in capturing the target PCB region are determined to realize quantitative positioning and feature deviation. According to the positioning deviation and the feature deviation, the original image data is subjected to deviation compensation processing to obtain corrected image data of the target PCB region, effectively avoiding problems such as line shape distortion and size distortion caused by shooting deviation. The corrected image data is used to determine a defect detection result of the target PCB region, thereby improving the stability and reliability of intelligent detection.

[0013] In a second aspect, the embodiments of the present application provide an AI vision-based PCB defect detection system, comprising:

[0014] An acquisition unit is configured to acquire original image data of a target PCB region captured by a first vision device.

[0015] A parameter unit is configured to acquire design parameters of the target PCB region.

[0016] A deviation unit is configured to determine, according to the original image data and the design parameters of the target PCB region, positioning deviation and feature deviation of the first vision device in capturing the target PCB region.

[0017] A correction unit is configured to perform deviation compensation processing on the original image data according to the feature deviation and the positioning deviation to obtain corrected image data of the target PCB region.

[0018] A determination unit is configured to determine a defect detection result of the target PCB region based on the corrected image data.

[0019] In a third aspect, the embodiments of the present application provide an electronic device, which comprises a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, the method according to any one of the above aspects is implemented.

[0020] In a fourth aspect, the embodiments of the present application provide a computer program product, which, when running on an electronic device, causes the electronic device to perform the method according to any one of the above aspects.

[0021] It can be understood that the beneficial effects of the above-mentioned second to fourth aspects can be referred to the related description in the above aspects, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0023] Figure 1 is a flowchart of the AI vision-based PCB defect detection method provided by an embodiment of the present application;

[0024] Figure 2 is a running schematic diagram of the AI vision-based PCB defect detection method provided by an embodiment of the present application;

[0025] Figure 3 is a structural schematic diagram of the AI vision-based PCB defect detection system provided by an embodiment of the present application;

[0026] Figure 4 is a structural schematic diagram of the electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0027] In the following description, specific details are set forth in order to provide a thorough understanding of embodiments of the present application. However, persons skilled in the art will understand that embodiments of the present application can be practiced without these specific details. In other instances, well-known systems, structures, circuits, and methods have not been described in detail in order to avoid obscuring the description of embodiments of the present application.

[0028] It should be understood that when used in the specification and the appended claims of the present application, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0029] It should also be understood that the term "and / or" used in the specification and the appended claims of the present application means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0030] As used in the specification and in the appended claims, the term “if’ can be interpreted as meaning “when” or “upon” or “in response to determining” or “in response to detecting” depending on the context. Similarly, the phrase “if it is determined” or “if a described condition or event is detected” can be interpreted as meaning “upon determining” or “in response to determining” or “upon detecting the described condition or event” or “in response to detecting the described condition or event” depending on the context.

[0031] In addition, in the description of the present application and the appended claims, the terms “first”, “second”, “third”, etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0032] Reference in the specification to “one embodiment” or “some embodiments” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrases “in one embodiment”, “in some embodiments”, “in other embodiments”, “in additional embodiments”, and so on, in various places in the specification are not necessarily all referring to the same embodiment, unless otherwise specifically stated. Rather, the phrase means “one or more but not all embodiments”, unless otherwise specifically stated.

[0033] Due to factors such as visual equipment installation deviation, mechanical motion precision limitation, and lens optical characteristics, the lines and pads of the PCB in the image may have shape distortion, size distortion, and other image itself deviation problems. During detection, the image itself deviation will cause inaccurate defect detection, especially in high-density and high-precision PCB detection scenarios, the line spacing is extremely small, and the component layout is compact. A small image deviation will be amplified, so that the detection process needs to be frequently manually reviewed and calibrated, which greatly reduces the detection efficiency and increases the production cost.

[0034] To solve the above problems, the embodiment of the present application provides a PCB defect detection method and system based on AI vision. In the method, the original image data of the target PCB area photographed by the first vision device is obtained, and the design parameters of the target PCB area are obtained to provide a reference for subsequent analysis. According to the original image data and the design parameters of the target PCB area, the positioning deviation and the feature deviation of the first vision device in the target PCB area are determined to realize quantitative positioning and feature deviation. According to the positioning deviation and the feature deviation, the original image data is compensated to obtain the corrected image data of the target PCB area, which effectively avoids the problems of line shape distortion and size distortion caused by shooting deviation. Based on the corrected image data, the defect detection result of the target PCB area is determined to improve the stability and reliability of intelligent detection.

[0035] The PCB defect detection method based on AI vision provided by the embodiment of the present application can be applied to an electronic device, and at this time, the electronic device is the execution subject of the PCB defect detection method based on AI vision provided by the embodiment of the present application. The specific type of the electronic device is not limited in the embodiment of the present application.

[0036] For example, the electronic device can be an ultra-mobile personal computer (UMPC), a netbook, a desktop computer, a computer, a laptop computer, a communication device, a computing device, a satellite wireless device, etc.

[0037] In order to better understand the PCB defect detection method based on AI vision provided by the embodiment of the present application, the specific implementation process of the PCB defect detection method based on AI vision provided by the embodiment of the present application is exemplarily introduced as follows.

[0038] Figure 1 The schematic flowchart of the PCB defect detection method based on AI vision provided by the embodiment of the present application is shown, and the PCB defect detection method based on AI vision includes:

[0039] S100, obtaining original image data of a target PCB area photographed by a first vision device.

[0040] It can be understood that the first visual device can be a camera system (such as a high-resolution industrial CCD / CMOS camera) installed on an industrial production line, which is responsible for collecting images of the target PCB area. During this process, the camera captures the complete field of view image of the target PCB area according to the set exposure parameters, shooting angle, focal length and other conditions. The image acquisition of the original image data can be automatically realized through the cv2.VideoCapture() function of OpenCV or the camera SDK interface. The original image data is generally output in RAW, BMP or JPEG format and stored in local storage or uploaded to a server. The original image data is used for subsequent image calibration and defect detection analysis, has high fidelity but may contain noise, distortion and other information, and therefore needs to be calibrated subsequently.

[0041] In one possible implementation, S100, the original image data of the target PCB area photographed by the first visual device is acquired, including:

[0042] S110, the initial image data and the first shooting angle of the target PCB area photographed by the first visual device, and the standard image data and the second shooting angle of the target PCB area photographed by the second visual device are acquired.

[0043] It can be understood that the accurate image data pair can be obtained through the image cross-referencing of multiple visual devices. The initial image data refers to the image obtained by the current first visual device, and the standard image data is the high-quality reference image obtained by the second visual device with known calibration. During image acquisition, the shooting angle corresponding to each device can be recorded. The shooting angle can be the rotation angle and the pitch angle of the camera relative to the PCB area, which can be represented by Euler angles or rotation matrices. The first shooting angle is the angle information of the target PCB area photographed by the first visual device, and the second shooting angle is the angle information of the target PCB area photographed by the second visual device, which is used for subsequent angle difference calculation and image calibration to ensure the geometric consistency between images.

[0044] S120, performing feature matching processing on the initial image data and the second shooting angle according to the standard image data to obtain the first calibration parameter.

[0045] It can be understood that the key structural features in the initial image data and the standard image data can be extracted, described and paired by an image processing algorithm, and error compensation can be combined with the second shooting angle to calculate the first calibration parameter. The first calibration parameter is the calibration parameter for aligning the initial image data and the standard image data. The first calibration parameter can be an affine transformation matrix, a perspective transformation matrix or a homogeneous projection matrix, which is a real matrix of 3x3 or 2x3, used for subsequent image coordinate conversion and geometric correction. Feature point detection algorithms such as SIFT, SURF or ORB can be used to extract key points and their descriptors in two images, and then BruteForce or FLANN is used for feature point matching, and RANSAC algorithm is used to remove abnormal matching pairs, and functions such as cv2.findHomography or cv2.estimateAffinePartial2D of OpenCV are used to generate the first calibration parameter matrix, which is used for angle offset compensation and coordinate uniform processing of the original image, thereby enhancing the consistency and feature alignment accuracy between images.

[0046] Exemplarily, S120, according to the standard image data, the initial image data and the second shooting angle are processed for feature matching, and the first calibration parameter is obtained, including:

[0047] S121, extracting key feature points in the standard image data to generate a first feature vector set.

[0048] It can be understood that the key feature points in the standard image data refer to important image region points that can represent the geometric profile of the PCB structure, such as pad centers, device corner points, circuit line intersection points, etc. The key feature points have high repeatability and high distinguishability, and are suitable for image matching and structure recognition. The first feature vector set is a vector group generated by vectorizing the key feature points in the standard image data using an image descriptor algorithm, which is used for image feature matching, transformation estimation and difference analysis. A multi-scale space can be constructed based on the SIFT algorithm to find scale extreme points as key feature points, and a 128-dimensional feature vector is generated using a local gradient direction histogram; or a binary first feature vector set is generated using the ORB algorithm based on FAST corner detection and BRIEF descriptor, which will be used as a reference data for subsequent feature matching with the initial image, for determining the transformation relationship between images and error calculation.

[0049] S122, extracting corresponding feature points in the initial image data to generate a second feature vector set.

[0050] It can be understood that the corresponding feature points in the initial image data are image feature position points corresponding to the structure in the standard image data extracted from the initial image data photographed by the first visual device, which can be obtained by a feature point detection algorithm. The second feature vector set is a local image pattern description of the image feature points, which is used for similarity matching with the first feature vector set and calculating the spatial mapping relationship. The same algorithm as S121 (such as SIFT, ORB) can be used for feature extraction and descriptor calculation of the initial image, ensuring homogeneity and matching basis with the standard image data feature vector. The key points and descriptors can be obtained by the cv2.SIFT_create() or cv2.ORB_create() function in OpenCV to generate the second feature vector set, which provides basic data for subsequent matching, correction and deviation compensation.

[0051] S123, calculating a feature matching degree based on the similarity between the first feature vector set and the second feature vector set.

[0052] It can be understood that the feature matching degree is the overall similarity between the first feature vector set and the second feature vector set, which is used to measure the consistency degree on the geometric structure, which can be quantified in the form of the proportion of the number of matched feature points to the total number of points, or expressed by using a weighted matching score. The point pair relationship between the two feature sets can be established by using a brute force matching algorithm or a fast approximate matching algorithm such as FLANN, then the Euclidean distance or Hamming distance between the descriptors of each point pair is calculated, and the unstable point pairs are filtered out by Lowe's ratio test or distance threshold, and the number of valid matching pairs and the total number of detected points are counted to form the feature matching degree, which is used to determine whether the image has sufficient similarity to perform geometric transformation or angle correction operation.

[0053] S124, when the feature matching degree is greater than a preset threshold, calculating an angle offset parameter according to the difference between the first shooting angle and the second shooting angle to obtain a first calibration parameter.

[0054] It can be understood that the angle offset parameter refers to the difference between the first shooting angle and the second shooting angle in the spatial pose, which can be represented as the difference of Euler angles or the difference of rotation matrices in three directions. Under the condition that the feature matching degree meets the preset threshold, the angle offset parameter is used to assist in generating or correcting the image transformation matrix, so that the initial image can be completely aligned with the standard image data through affine transformation or perspective transformation. The first calibration parameter is the image mapping matrix finally formed under the action of the angle offset parameter, which is used for image space coordinate unification and structure correction. The difference matrix ΔR=R2·R1 of the first shooting angle and the second shooting angle can be solved by the extrinsic matrix (R1, R2) of the first shooting angle and the second shooting angle. -1And based on the matching point pairs, the rotation difference is least square fitted to form an angle compensation model applicable to image coordinate transformation, thereby outputting a calibration parameter matrix satisfying geometric consistency, i.e., the first calibration parameter, for subsequent image correction.

[0055] S130, according to the first calibration parameter, the initial image data and the first shooting angle, the original image data of the target PCB region is obtained.

[0056] It can be understood that the initial image data can be mapped into a standard reference coordinate system through image correction operation, thereby obtaining the target PCB region original image data with standardized geometric characteristics. The initial image data can be pixel re-projected according to the calibration parameter using OpenCV function cv2.warpAffine or cv2.warpPerspective, to obtain the original image data of the target PCB region, to realize complete unification of angle information, to ensure that the image can be used for subsequent design parameter comparison and defect detection processing.

[0057] S200, obtaining the design parameters of the target PCB region.

[0058] It can be understood that the design parameters of the target PCB region refer to the standardized reference data related to the structure layout, device arrangement, circuit wiring and key feature point coordinates on the target PCB board. These design parameters are usually derived from PCB design software (such as Altium Designer, Cadence Allegro or Mentor Graphics), including Gerber files, ODB++ format or database files containing structure level information. The design parameters are used as reference templates in the detection process, for structure comparison, position offset analysis and defect identification with actual image data. A preset analysis program can be used to parse the design file into a structure parameter table or a feature point coordinate table, and to register with the image processing result, to determine the corresponding position of each image feature point in the design parameters by means of coordinate mapping relationship, as the basis for subsequent deviation judgment and correction compensation. The accurate acquisition of design parameters is directly related to the image recognition accuracy and the tolerance control ability of detection algorithm, and is the key link between image processing and CAD drawings.

[0059] S300, according to the original image data of the target PCB region and the design parameters, determining the positioning deviation and feature deviation of the first vision device in the target PCB region.

[0060] It can be understood that the positioning deviation and the feature deviation refer to the difference between the original image data and the design parameters in the spatial position and the structural form. The positioning deviation mainly describes the translation, rotation or scaling error of the overall structure in the image relative to the design reference coordinates. The feature deviation describes the detailed deformation or displacement of the specific key points, edges or device contours in the image relative to the standard position. Determining these deviations helps to evaluate whether the current image acquisition effect is qualified, or whether there are mechanical errors, image distortion or process deformation problems. By aligning the image feature point set with the design parameter feature point set, the overall offset can be estimated using affine transformation or least mean square fitting method, and the displacement vector of each local feature point can be further calculated, so as to construct a complete deviation analysis model for guiding subsequent image compensation, equipment positioning correction or structural integrity evaluation. The deviation judgment result can also be used as an important basis for quality control and statistical analysis.

[0061] In one possible implementation, S300, according to the original image data and the design parameters of the target PCB area, the positioning deviation of the first visual device in the target PCB area is determined, including:

[0062] S310, using a preset feature deviation degree model to process the original image data and the design parameters, determining the positioning deviation degree of the first visual device in the target PCB area.

[0063] It can be understood that the feature deviation degree model refers to a model for quantifying the degree of deviation of the structure in the image from the design reference, which is used to map the structural consistency between the original image data and the design parameters into a quantifiable positioning deviation degree index. The feature deviation degree model can be generated based on image feature point matching distance, structural similarity index (such as SSIM, Hausdorff distance) or neural network-based image comparison output result. A model function f(I, D) = L can be constructed, where I is the image feature set, D is the design parameter set, and L is the output positioning deviation degree value. For example, a support vector regression model SVR based on RBF kernel, or a deep neural network (such as CNN + regression layer) prediction deviation score, while traditional methods such as calculating the average offset distance of each key point in the image in the x and y directions as the deviation degree index and setting a qualified threshold for judgment can also be used. The feature deviation degree model is the key to converting the structural deviation of the image data into a standard numerical expression, which supports subsequent offset correction and structural deformation recognition of the PCB image.

[0064] S320, based on the positioning deviation degree and the original image data, determining the positioning deviation of the first visual device in the target PCB area.

[0065] It can be understood that the positioning deviation is a specific spatial displacement determined after further analysis of the positioning deviation degree, reflecting the spatial coordinate difference between the image collected by the first visual device and the standard design coordinate system in the x and y planes. The determination of the positioning deviation is by analyzing the offset of each key reference point in the original image relative to the standard coordinate in the design parameter, and combining the image registration transformation matrix to back-propagate the overall position error vector. The pads or positioning holes with clear structure and clear boundary in the image can be selected as the reference points, the Euclidean distance change between them and the design coordinate is calculated, and then the least square method or ICP algorithm is used to model the overall offset to generate the vector (Vx, Vy) of the positioning deviation. The positioning deviation is directly input for coordinate compensation and subsequent image correction, and is an important intermediate variable for quantifying image acquisition error and repairing.

[0066] S330, according to the positioning deviation and the design parameter, obtaining a feature deviation of the first visual device in the target PCB area.

[0067] It can be understood that the feature deviation refers to the micro deformation of the key structure points in the image relative to the standard feature points in the design parameter on the basis of the determination of the positioning deviation. The feature deviation not only includes the overall position offset, but also includes the structural morphological changes such as rotation, scaling, nonlinear deformation, etc. Its determination method is to first perform preliminary coordinate alignment through the positioning deviation, and then calculate the structure offset degree based on the residual vector between the standard and actual feature points. The feature point coordinates in the design parameter can be transformed through coordinate mapping, and compared with the actual feature point coordinates extracted from the image. The Euclidean distance or Manhattan distance is used to calculate the feature offset, and a matrix of the feature deviation is generated to describe the overall deformation trend. The feature deviation can be used to judge the problems such as pad stretching, device misplacement and line offset, and is the core reference for high-precision defect judgment and image error compensation. It is used for subsequent accurate compensation of the positioning deviation and the feature deviation to effectively solve the problems such as line distortion and size deviation caused by shooting errors.

[0068] Exemplarily, S330, according to the positioning deviation and the design parameter, obtaining a feature deviation of the first visual device in the target PCB area, comprising:

[0069] S331, determining the standard feature point coordinates of the target PCB area according to the design parameter.

[0070] It can be understood that the standard feature point coordinates are position data of reference points with unique physical meaning extracted from design parameters in a two-dimensional design coordinate system, which can include pad centers, device corner points, line nodes, etc. These points are in fixed positions in the PCB design file and should remain stable and consistent in the actual production process, and are the key point set for comparison analysis and accuracy verification. The corresponding coordinate points can be extracted by analyzing the structure layer or pad layer information in the Gerber or ODB++ file, or directly derived from the PCB design drawing in DXF coordinate format, and converted into pixel coordinate system or physical coordinate system consistent with the image acquisition coordinates, providing a standard reference for subsequent image feature deviation calculation.

[0071] S332, generating a coordinate transformation matrix according to the positioning deviation.

[0072] It can be understood that the coordinate transformation matrix is a two-dimensional coordinate mapping function calculated according to the image positioning deviation, which can be an affine transformation matrix or a perspective transformation matrix. Its function is to transform the standard feature point coordinates in the design parameters into the positions in the current image coordinates, thereby realizing consistent comparison across coordinate systems. The transformation matrix can be generated based on the matched point pairs using the estimateAffine2D or getPerspectiveTransform function in OpenCV, and the design parameter coordinate set is applied as input to calculate the new coordinate position using the matrix, forming the reference point position after correction for comparison.

[0073] S333, transforming the standard feature point coordinates using the coordinate transformation matrix to obtain transformed standard feature point coordinates.

[0074] It can be understood that the transformed standard feature point coordinates are new coordinate values obtained by mapping the standard feature point coordinates in the original design parameters through the aforementioned coordinate transformation matrix, representing the positions of these structure points in the current image acquisition coordinate system, which are prediction values for difference analysis with the feature points detected in the actual image. Each design coordinate point can be transformed by an affine matrix to output its mapping result in the image coordinate system, obtaining the transformed standard feature point coordinates, and forming a point pair relationship with the actual image feature points for subsequent deviation calculation and error statistics.

[0075] S334, extracting the actual detected feature point coordinates from the original image data.

[0076] It can be understood that the actually detected feature point coordinates refer to the pixel positions of the structural points detected in the original image data through the image feature extraction algorithm. These points can be output by feature detection algorithms such as ORB and SIFT, and have the same physical meaning as the design standard feature points, but may have problems such as position drift and blur in the image, so accurate extraction is a key prerequisite for deviation evaluation and image quality judgment. Each feature point coordinate can be extracted through the keypoint output interface of the detection algorithm and stored in an array or table form for subsequent comparison with the standard coordinates.

[0077] S335, calculate the difference between the actually detected feature point coordinates and the transformed standard feature point coordinates, and determine the difference as the feature deviation.

[0078] It can be understood that the difference refers to the distance difference between the transformed standard feature point coordinates and the feature point coordinates detected in the actual image, which can be calculated using the Euclidean distance formula d = √[(x1-x2)²+(y1-y2)²]. The difference of all feature points is calculated, and the mean or maximum value is obtained to obtain the overall feature deviation index. Feature deviation is used to determine whether the image meets the design requirements or whether further image compensation and offset correction processing is required. The difference can be calculated in batches using matrix operations, and a deviation tolerance range is set for subsequent determination of whether to trigger the compensation mechanism or the defect judgment process.

[0079] In some embodiments, the method further comprises:

[0080] S610, obtaining sample image data and standard positioning data of a sample PCB area photographed by a sample first vision device; wherein the standard positioning data is the positioning data of the standard image data photographed by the gold standard vision device.

[0081] It can be understood that the sample first vision device refers to a photographing device with the same or equivalent configuration as the target first vision device, which is used to collect sample PCB area images to train the model or generate the reference. The sample image data refers to the image data set collected using the vision device, which can include multiple actual image samples of different types and different deviation levels. The standard positioning data refers to the image or position coordinate result collected and output by the gold standard vision device (such as a high-precision calibration camera or a laser interferometer), which is used as the true reference value of the sample image to calculate the deviation, train the model, or label the label. The image acquisition station can be set up so that the sample PCB is photographed by the first vision device and the gold standard device in turn, and the images are time-synchronized and matched by identification. The accurate spatial position of the same structure point is determined through image coordinate pairs or external positioning devices to construct a high-precision comparison basis between the image and the reference, thereby supporting subsequent error modeling and training data set construction, which is a key link for improving the model generalization ability and detection performance.

[0082] S620, feature point extraction and matching processing is performed on the sample image data to obtain predicted positioning data of the sample PCB region.

[0083] It can be understood that the feature point extraction and matching processing refers to detecting and describing structural feature points between the sample image data and the standard image data, and establishing a pairing relationship of the feature points in the two images, so as to obtain a spatial transformation relationship and a structural consistency degree between the images. The predicted positioning data is spatial position data calculated based on the feature correspondence relationship between the sample image data and the standard image data, and represents the prediction ability of the system for the standard structure position in an unknown input image. The SIFT, ORB and other algorithms in OpenCV can be called to extract key points in the sample image and the standard image data, and the matching point pairs are calculated through the descriptor matching, the RANSAC algorithm is used to filter out the abnormal point pairs, a transformation matrix is generated, and the transformation matrix is applied to map the standard image data coordinates to the sample image coordinates, so as to obtain the predicted positioning points and form the estimated positioning output result of the visual model, thereby providing a benchmark for subsequent error calculation and model optimization.

[0084] Exemplarily, S620, feature point extraction and matching processing is performed on the sample image data to obtain predicted positioning data of the sample PCB region, comprising:

[0085] S621, applying a feature point detection algorithm to the sample image data to extract feature points and generate feature descriptors.

[0086] It can be understood that the feature descriptor is a vector expression form generated by encoding the pixel structure around the feature point, and usually has rotation and scale invariance, and can be used for feature matching between different images. Based on the ORB algorithm, the FAST corner detector is used to select key points, and then the BRIEF descriptor is used to generate a binary vector feature descriptor; or SIFT is used to construct a Gaussian difference pyramid to search for extreme points and calculate a 128-dimensional feature descriptor. The feature descriptor is the core of image structure representation, which is used to support the solution of spatial correspondence relationship between images, and is the input basis for deviation evaluation and model training.

[0087] S622, feature point detection is performed on the standard image data of the sample PCB region to extract standard feature points and generate standard feature descriptors.

[0088] It can be understood that the standard feature descriptor is a structural expression vector of each feature point in the standard image data, which will be matched with the descriptor in the sample image and a mapping relationship will be established. The same feature extraction algorithm (such as SIFT or ORB) can be used to ensure the consistency of the descriptor, and KD tree, LSH or brute force search is used for matching, so that the structural information in the standard image data can be mapped to the sample image, realizing the structural position prediction and error labeling.

[0089] S623, match the feature descriptor with the standard feature descriptor, and screen the feature point pairs with a matching degree higher than a preset threshold.

[0090] It can be understood that the feature descriptor can be matched with the standard feature descriptor, and the high-quality point pairs with a distance difference less than a set tolerance or screened out by a scale test (such as Lowe's ratio test) are screened out. The high-quality point pairs are considered to have a high correspondence in the two images, and can be used for accurate solving of the transformation relationship or training of the model data labeling. The sample descriptor and the standard descriptor can be matched one by one, the matching error or the ratio is calculated, and the matching results with an error less than a set threshold (such as 0.75) are reserved as effective point pairs. Further, the outliers are removed by RANSAC or LMedS, so as to improve the matching accuracy and robustness, and provide high-credibility input for the transformation matrix calculation.

[0091] S624, based on the feature point pairs, a transformation matrix is calculated, the sample image data is mapped to the coordinate system of the standard image data, and the predicted positioning data of the sample PCB area is obtained.

[0092] It can be understood that the transformation matrix is a geometric mapping matrix calculated by the high-quality feature point pairs between the sample image and the standard image data, which is used to map the structure of one image to the coordinate system of another image, realizing image alignment and structure prediction. The OpenCV function cv2.findHomography() can be used to calculate the homography matrix based on the matching point pairs, or the estimateAffinePartial2D() can be used to calculate the affine transformation parameters, which are then applied to the sample image structure point mapping processing. The sample image data is mapped to the coordinate system of the standard image data, and the predicted positioning data of the sample PCB area is obtained. The generated predicted positioning data will be used as the model training target or error evaluation benchmark.

[0093] S630, according to the difference between the standard positioning data and the predicted positioning data, the sample positioning deviation of the sample PCB area photographed by the sample first visual device is determined.

[0094] It can be understood that the sample positioning deviation refers to the structural position difference between the predicted positioning data and the standard positioning data in the sample PCB image captured by the sample first visual device, is an index for quantifying the model prediction accuracy or the sample image error level, and can be in the form of average error distance, maximum offset, or root mean square error of all corresponding feature points. The predicted positioning point coordinates and the standard reference point coordinates can be corresponded one by one, the Euclidean distance difference is calculated, and the overall error index is counted as the label value of the training sample and input into the bias modeling process, which is an important training input to form the structural regression model.

[0095] S640, the initial machine learning model is trained with the sample image data and the design parameters of the sample PCB region as expected input and the sample positioning deviation as expected output to obtain a feature bias degree model.

[0096] It can be understood that the initial machine learning model refers to a model for fitting the mapping relationship between the sample image structural bias and the image features or design parameters, which can adopt forms such as regression type neural network, support vector machine, or gradient boosting tree. The feature bias degree model is a bias prediction function obtained by training the initial machine learning model based on the sample image and its corresponding design parameters as input and the sample positioning deviation as supervised output, which can map the image features to spatial error estimation values. The feature coordinates, angles, brightness, texture features, etc. extracted in the image can be encoded into an input vector, and then the sample positioning deviation is taken as the target to train the model through a machine learning framework (such as TensorFlow, PyTorch, LightGBM), and after loss convergence, a prediction model for actual image bias estimation is output to provide accurate estimation support for subsequent image quality judgment and error compensation.

[0097] S400, according to the positioning bias and the feature bias, performing bias compensation processing on the original image data to obtain corrected image data of the target PCB region.

[0098] It can be understood that the bias compensation processing refers to performing geometric correction and structural calibration on the original image data according to the positioning bias and the feature bias obtained in the foregoing steps, so that the image is regressed to the design structure alignment state, and the corrected image is the corrected image data, which can be used for subsequent high-precision detection processing. The image overall coordinates and key structure point positions can be transformed respectively by constructing a coordinate transformation matrix and a feature compensation matrix, and the image is corrected by applying image remapping functions such as warpAffine or warpPerspective in OpenCV to obtain a corrected image with a structure and position close to the standard reference state, which is a core part of the image preprocessing process.

[0099] In a possible implementation, S400, the original image data is subjected to bias compensation processing according to the positioning bias and the feature bias, to obtain modified image data of the target PCB region, including:

[0100] S410, a coordinate transformation matrix is generated according to the positioning bias; wherein the coordinate transformation matrix is used to correct the position offset in the original image data.

[0101] It can be understood that the coordinate transformation matrix is a two-dimensional affine or perspective transformation matrix generated according to the positioning bias, which is mainly used to correct the overall position offset in the image, so that the image structure is aligned with the standard coordinates in space. The position offset can include translation, rotation and scaling of the image in the x and y axis directions, and a translation matrix can be calculated according to the positioning offset vector, or an affine transformation matrix can be calculated using known reference points, and image repositioning is realized through transformation operation.

[0102] S420, a feature compensation matrix is generated according to the feature bias; wherein the feature compensation matrix is used to correct the feature deformation in the original image data.

[0103] It can be understood that the feature compensation matrix is a structure-level geometric repair parameter matrix generated according to the feature bias, which is used to correct the distortion, stretching or misplacement of local features such as devices or lines in the image. The feature compensation matrix acts on the key point region, and the structure of the abnormal deformation region in the image is reconstructed through local nonlinear interpolation, deformation model or elastic transformation. The feature compensation matrix can be constructed by using TPS (Thin Plate Spline) deformation model or fitting local displacement field based on feature vector difference, and the feature compensation matrix is applied to the local region of the image for fine correction.

[0104] S430, the feature compensation matrix and the coordinate transformation matrix are fused to obtain a comprehensive compensation matrix.

[0105] It can be understood that the comprehensive compensation matrix is a result matrix obtained by weighted fusion or serial combination of the coordinate transformation matrix and the feature compensation matrix, which is used to uniformly execute image position and structure dual compensation, and is an important step to improve image restoration quality. The feature compensation matrix and the coordinate transformation matrix can be synthesized by matrix multiplication or interpolation reconstruction algorithm, and a multi-level compensation process can be constructed by combining the image affine and nonlinear transformation modules, so that the entire image is completely close to the design parameters in the spatial structure.

[0106] S440, the original image data is subjected to bias compensation processing using the comprehensive compensation matrix, to obtain modified image data of the target PCB region.

[0107] It can be understood that the bias compensation processing refers to applying the foregoing comprehensive compensation matrix to the original image data, completing the coordinate and structure double correction of the image through image pixel level remapping and interpolation operation, and outputting the corrected image data meeting the design standard. The remap or warp function in OpenCV can be called to transform the image pixel coordinates, and the image definition is maintained through bilinear interpolation or cubic convolution interpolation, so as to output the image version with minimum error, that is, the corrected image data, for subsequent defect detection processing. Through the accurate compensation of positioning deviation and feature deviation, the problems of line distortion and size deviation caused by shooting errors are effectively solved, and the accuracy of visual detection in high-precision and high-demand PCB detection is effectively improved.

[0108] S500, determining a defect detection result of the target PCB area based on the corrected image data.

[0109] It can be understood that the defect detection result refers to the detection report output after executing image recognition and defect positioning algorithm on the corrected image data, which can include defect type (such as broken line, short circuit, virtual welding), defect position (pixel coordinate or physical coordinate), defect severity and other information, and is used to judge whether the PCB product is qualified. A pre-trained deep learning model (such as YOLO, Mask R-CNN) can be called to perform target detection and semantic segmentation processing on the image, or traditional algorithms based on template matching, edge detection, morphological analysis, etc. can be used to extract abnormal areas, and combined with design parameters to screen defect areas, and finally output structured detection data or image annotation information as the defect detection result for subsequent sorting, report generation or manual review.

[0110] In one possible implementation, S500, determining a defect detection result of the target PCB area based on the corrected image data, includes:

[0111] S510, processing the corrected image data using a preset PCB defect detection model to obtain the defect detection result of the target PCB area; wherein the PCB defect detection model is a machine learning model obtained by training.

[0112] It can be understood that the PCB defect detection model refers to a pre-trained machine learning or deep learning model for identifying common defects in printed circuit board images, which can identify defect types such as broken lines, short circuits, pad missing, solder shift, burrs, and foreign matter according to input correction image data, and locate the specific location area in the image, and then generate a structured defect detection result. The PCB defect detection model can be composed of a convolutional neural network (CNN) structure, which can include YOLO, Faster R-CNN, RetinaNet, EfficientDet, etc. The target detection framework, its training process is based on a large-scale labeled PCB defect image set, the model input is the correction image, and the output is multiple defect candidate boxes and corresponding confidence and class labels. The correction image can be input to the PCB defect detection model, the model first performs multi-scale convolution feature extraction on the image, then integrates the features through the feature pyramid network (FPN), and finally the detection head performs boundary box regression and class prediction. The output detection result includes the bounding coordinates, class, and confidence score of each defect, effectively solving the problems of line distortion and size deviation caused by shooting errors, improving detection accuracy and reliability, and enhancing the ability to detect high-precision and high-quality PCBs.

[0113] Corresponding to the AI vision-based PCB defect detection method of the above embodiment, the embodiment of the application also provides an AI vision-based PCB defect detection system. Each unit of the system can implement each step of the AI vision-based PCB defect detection method. Figure 3 The structure block diagram of the AI vision-based PCB defect detection system provided by the embodiment of the application is shown, and only the part related to the embodiment of the application is shown for ease of description.

[0114] Referring to Figure 3 The AI vision-based PCB defect detection system includes:

[0115] An acquisition unit is configured to acquire original image data of a target PCB region captured by a first vision device;

[0116] A parameter unit is configured to acquire design parameters of the target PCB region;

[0117] A deviation unit is configured to determine a positioning deviation and a feature deviation of the first vision device in capturing the target PCB region according to the original image data and the design parameters of the target PCB region;

[0118] A correction unit is configured to perform deviation compensation processing on the original image data according to the feature deviation and the positioning deviation to obtain correction image data of the target PCB region;

[0119] The determining unit is used to determine the defect detection result of the target PCB area based on the corrected image data.

[0120] It should be noted that the information interaction and execution process between the above systems / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0121] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit module can exist physically separately, or two or more unit modules can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0122] This application also provides an electronic device. Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 As shown, the electronic device 6 of this embodiment includes: at least one processor 60 ( Figure 4 Only one is shown in the image), at least one memory 61 ( Figure 4 (Only one is shown in the image) and a computer program 62 stored in the at least one memory 61 and executable on the at least one processor 60. When the processor 60 executes the computer program 62, it causes the electronic device 6 to implement the steps in any of the above embodiments of the AI ​​vision-based PCB defect detection method, or causes the electronic device 6 to implement the functions of each unit in the above system embodiments.

[0123] For example, the computer program 62 may be divided into one or more units, which are stored in the memory 61 and executed by the processor 60 to complete this application. The one or more units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 62 in the electronic device 6.

[0124] The electronic device 6 can be a desktop computer, a notebook computer, a palm computer, a cloud server, or the like. The electronic device can include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art can understand that Figure 4 The electronic device 6 is only an example and is not limited to the electronic device 6, and can include more or fewer components than shown, or combine certain components, or include different components, such as an input / output device, a network access device, a bus, and the like.

[0125] The processor 60 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0126] The memory 61 can be an internal storage unit of the electronic device 6, such as a hard disk or a memory of the electronic device 6 in some embodiments. The memory 61 can also be an external storage device of the electronic device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, or the like, in other embodiments. Further, the memory 61 can include both an internal storage unit and an external storage device of the electronic device 6. The memory 61 is used to store an operating system, an application program, a boot loader, data, and other programs, such as program codes of the computer program, and the like. The memory 61 can also be used to temporarily store data that has been output or will be output.

[0127] The embodiments of the present application also provide a computer readable storage medium storing a computer program. The computer program is executed by a processor to implement the steps in any of the method embodiments.

[0128] The embodiments of the present application provide a computer program product. When the computer program product is run on an electronic device, the electronic device implements the steps in any of the method embodiments.

[0129] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the present application can implement all or part of the processes in the above-mentioned embodiment methods through a computer program to instruct related hardware to complete, and the computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium at least includes any entity or device capable of carrying the computer program code to the electronic device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, according to legislation and patent practice, the computer readable medium can not be an electrical carrier signal and a telecommunication signal.

[0130] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0131] Those skilled in the art can appreciate that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0132] In the embodiments provided by the present application, it should be understood that the disclosed AI vision-based PCB defect detection system / electronic device and method can be implemented in other ways. For example, the above-described AI vision-based PCB defect detection system / electronic device embodiments are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, another division mode can be used. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0133] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0134] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A PCB defect detection method based on AI vision, characterized in that, include: Acquire raw image data of the target PCB area captured by the first vision device; Obtain the design parameters of the target PCB area; Based on the original image data and design parameters of the target PCB area, the positioning deviation and feature deviation captured by the first vision device in the target PCB area are determined; wherein, the positioning deviation is used to describe the translation, rotation or scaling error of the overall structure in the image relative to the design reference coordinates; the feature deviation is used to describe the detail deformation or displacement of specific key points, edges or device contours in the image relative to the standard position; Based on the positioning deviation and the feature deviation, the original image data is subjected to deviation compensation processing to obtain the corrected image data of the target PCB area; Determine the defect detection result of the target PCB area based on the corrected image data; The step of determining the positioning deviation and feature deviation of the first vision device in the target PCB area based on the original image data and design parameters of the target PCB area includes: The original image data and the design parameters are processed using a preset feature deviation model to determine the positioning deviation of the first vision device in the target PCB area. Based on the positioning deviation and the original image data, the positioning deviation captured by the first vision device in the target PCB area is determined; Based on the positioning deviation and the design parameters, the feature deviation captured by the first vision device in the target PCB area is obtained.

2. The PCB defect detection method based on AI vision as described in claim 1, characterized in that, The acquisition of the original image data of the target PCB area captured by the first vision device includes: Acquire initial image data and a first shooting angle captured by the first vision device in the target PCB area, and standard image data and a second shooting angle captured by the second vision device in the target PCB area; Based on the standard image data, feature matching processing is performed on the initial image data and the second shooting angle to obtain the first calibration parameters; Based on the first calibration parameters, the initial image data, and the first shooting angle, the original image data of the target PCB area is obtained.

3. The PCB defect detection method based on AI vision as described in claim 2, characterized in that, The step of performing feature matching processing on the initial image data and the second shooting angle based on the standard image data to obtain the first calibration parameters includes: Extract key feature points from the standard image data to generate a first feature vector set; Extract the corresponding feature points from the initial image data to generate a second feature vector set; The feature matching degree is calculated based on the similarity between the first feature vector set and the second feature vector set; When the feature matching degree is greater than a preset threshold, the angle offset parameter is calculated based on the difference between the first shooting angle and the second shooting angle to obtain the first calibration parameter.

4. The PCB defect detection method based on AI vision as described in claim 1, characterized in that, The method further includes: Acquire sample image data and standard positioning data of the sample PCB area captured by the first visual device; wherein, the standard positioning data is the positioning data of the standard image data captured by the gold standard visual device; Feature point extraction and matching are performed on the sample image data to obtain the predicted location data of the sample PCB area; Based on the difference between the standard positioning data and the predicted positioning data, the sample positioning deviation of the sample PCB area captured by the first visual device is determined. Using the sample image data and the design parameters of the sample PCB area as expected inputs, and the sample positioning deviation as expected outputs, the initial machine learning model is trained to obtain the feature deviation model.

5. The PCB defect detection method based on AI vision as described in claim 4, characterized in that, The step of extracting and matching feature points from the sample image data to obtain the predicted location data of the sample PCB area includes: The feature point detection algorithm is applied to the sample image data to extract feature points and generate feature descriptors; Feature point detection is performed on the standard image data of the sample PCB area to extract standard feature points and generate standard feature descriptors; The feature descriptor is matched with the standard feature descriptor, and feature point pairs with a matching degree higher than a preset threshold are selected. Based on the feature point pairs, a transformation matrix is ​​calculated, and the sample image data is mapped to the coordinate system of standard image data to obtain the predicted positioning data of the sample PCB area.

6. The PCB defect detection method based on AI vision as described in claim 1, characterized in that, The step of obtaining the feature deviation captured by the first vision device in the target PCB area based on the positioning deviation and the design parameters includes: Determine the standard feature point coordinates of the target PCB area based on the design parameters; Generate a coordinate transformation matrix based on the positioning deviation; The coordinates of the standard feature points are transformed using the coordinate transformation matrix to obtain the transformed coordinates of the standard feature points. Extract the coordinates of the actually detected feature points from the original image data; Calculate the degree of difference between the actual detected feature point coordinates and the transformed standard feature point coordinates, and determine the degree of difference as the feature deviation.

7. The PCB defect detection method based on AI vision as described in claim 1, characterized in that, The step of performing deviation compensation processing on the original image data based on the positioning deviation and the feature deviation to obtain corrected image data of the target PCB area includes: A coordinate transformation matrix is ​​generated based on the positioning deviation; wherein the coordinate transformation matrix is ​​used to correct the positional offset in the original image data; A feature compensation matrix is ​​generated based on the feature deviation; wherein the feature compensation matrix is ​​used to correct feature deformations in the original image data; The feature compensation matrix and the coordinate transformation matrix are fused to obtain the comprehensive compensation matrix; The original image data is subjected to deviation compensation processing using the comprehensive compensation matrix to obtain the corrected image data of the target PCB area.

8. The PCB defect detection method based on AI vision as described in claim 1, characterized in that, Determining the defect detection result of the target PCB area based on the corrected image data includes: The modified image data is processed using a preset PCB defect detection model to obtain the defect detection result of the target PCB area; The PCB defect detection model is a machine learning model obtained through training.

9. A PCB defect detection system based on AI vision, characterized in that, For implementing the method of any one of claims 1 to 8, the AI ​​vision-based PCB defect detection system comprises: The acquisition unit is used to acquire raw image data of the target PCB area captured by the first vision device. The parameter unit is used to obtain the design parameters of the target PCB area. The deviation unit is used to determine the positioning deviation and feature deviation of the first vision device in the target PCB area based on the original image data and design parameters of the target PCB area. The correction unit is used to perform deviation compensation processing on the original image data according to the feature deviation and the positioning deviation to obtain the corrected image data of the target PCB area; The determining unit is used to determine the defect detection result of the target PCB area based on the corrected image data.

Images