Foreign matter detection method and device for circuit board on board feeding equipment

Abstract

The invention provides a foreign matter detection method and device for a circuit board on board feeding equipment. The method comprises the following steps: acquiring sensor data for detecting the board loading state of a circuit board on board loading equipment and image data of the circuit board; respectively preprocessing the sensor data and the image data to obtain preprocessed sensor data and preprocessed image data; detecting the circuit board for the first time according to the preprocessed sensor data to obtain a first detection result; detecting the circuit board for the second time according to the first detection result and the preprocessed image data to obtain a second detection result; and obtaining a target foreign matter detection result according to the first detection result and the second detection result. According to the invention, the efficiency and accuracy of foreign matter detection of the circuit board on the board feeding equipment can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of circuit boards, and also to a method and device for detecting foreign matter on a circuit board on a board loading device. Background Art

[0002] In the SMT (surface mount technology) patch process, the printed circuit board (PCB) is the basic carrier, and its surface cleanliness is crucial to the patch quality. However, during the PCB boarding process, due to improper operation and insufficient equipment cleanliness, foreign matter such as dust and debris is easily introduced, affecting the patch quality and product performance. In the existing technology, foreign matter is mainly prevented from entering through manual inspection and equipment cleaning, but there are problems such as low efficiency and easy missed inspection. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a method and device for detecting foreign matter on a circuit board on a board loading device, so as to improve the detection efficiency of foreign matter on the circuit board.

[0004] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0005] A first aspect of the present invention provides a method for detecting foreign matter on a circuit board on a board loading device, comprising:

[0006] Acquire sensor data for detecting a board-mounting state of a circuit board on a board-mounting device and image data of the circuit board;

[0007] Preprocessing the sensor data and the image data respectively to obtain preprocessed sensor data and preprocessed image data;

[0008] Performing a first detection on the circuit board according to the preprocessed sensor data to obtain a first detection result;

[0009] Perform a second inspection on the circuit board according to the first inspection result and the preprocessed image data to obtain a second inspection result;

[0010] A target foreign body detection result is obtained according to the first detection result and the second detection result.

[0011] Optionally, obtaining sensor data for detecting a board-mounting state of a circuit board on a board-mounting device includes:

[0012] Collecting optical sensor data of the circuit board using an optical sensor installed at a first preset position on the upper board device;

[0013] Collecting capacitance sensor data of the circuit board using a capacitance sensor installed at a second preset position on the upper board device;

[0014] Collecting infrared sensor data of the circuit board using an infrared sensor installed at a third preset position on the upper board device;

[0015] Sensor data of the circuit board is obtained according to the optical sensor data, the capacitive sensor data and the infrared sensor data.

[0016] Optionally, preprocessing the sensor data to obtain preprocessed sensor data includes:

[0017] Performing denoising processing on the sensor data to obtain denoised data;

[0018] Performing data standardization processing on the denoised data to obtain standardized data;

[0019] The standardized data is calibrated to obtain pre-processed sensor data.

[0020] Optionally, preprocessing the image data to obtain preprocessed image data includes:

[0021] Performing grayscale processing on the image data to obtain grayscale image data;

[0022] Performing filtering processing on the grayscale image data to obtain filtered image data;

[0023] Binarizing the filtered image data to obtain binary image data;

[0024] Perform image enhancement processing on the binary image data to obtain pre-processed image data.

[0025] Optionally, performing a first detection on the circuit board according to the preprocessed sensor data to obtain a first detection result includes:

[0026] Performing feature extraction on the preprocessed sensor data to obtain a plurality of feature information;

[0027] Performing feature fusion on the multiple feature information to obtain fused feature data;

[0028] A first detection is performed on the circuit board according to the fused feature data to obtain a first detection result.

[0029] Optionally, performing a second inspection on the circuit board according to the first inspection result and the preprocessed image data to obtain a second inspection result includes:

[0030] When the first detection result contains foreign matter information, segmenting the preprocessed image data to obtain segmented image data;

[0031] Performing format conversion on the segmented image data to obtain a feature vector;

[0032] Input the feature vector into a trained foreign body detection model to obtain a detection value; the foreign body detection model is obtained by training a preset network model using annotated historical circuit board image data;

[0033] A second detection result is obtained according to the detection value and a preset threshold.

[0034] Optionally, obtaining a target foreign body detection result according to the first detection result and the second detection result includes:

[0035] Determine a first feature of the foreign object and first positioning data of the foreign object according to the first detection result;

[0036] Determine a second feature of the foreign object and second positioning data of the foreign object according to the second detection result;

[0037] Matching the first feature of the foreign object with the second feature of the foreign object to obtain basic data of the foreign object;

[0038] Obtaining foreign object positioning data according to the first foreign object positioning data and the second foreign object positioning data;

[0039] A target foreign object detection result is obtained according to the foreign object basic data and the foreign object positioning data.

[0040] A second aspect of the present invention provides a foreign body detection device for a circuit board on a board loading device, comprising:

[0041] An acquisition module, used for acquiring sensor data for detecting the board state of the circuit board on the board loading device and image data of the circuit board;

[0042] A processing module is used to preprocess the sensor data and the image data respectively to obtain preprocessed sensor data and preprocessed image data; perform a first detection on the circuit board according to the preprocessed sensor data to obtain a first detection result; perform a second detection on the circuit board according to the first detection result and the preprocessed image data to obtain a second detection result; and obtain a target foreign body detection result according to the first detection result and the second detection result.

[0043] According to a third aspect of the present invention, a computing device is provided, comprising: a processor and a memory storing a computer program, wherein when the computer program is executed by the processor, the method according to the first aspect is executed.

[0044] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions, which, when executed on a computer, causes the computer to execute the method described in the first aspect.

[0045] The above solution of the present invention includes at least the following beneficial effects:

[0046] The above-mentioned scheme of the present invention obtains sensor data for detecting the upper board status of the circuit board on the upper board device and image data of the circuit board, and preprocesses the sensor data and the image data respectively, performs a first detection on the circuit board according to the preprocessed sensor data, and performs a second detection on the circuit board according to the first detection result of the first detection and the preprocessed image data to obtain a second detection result, and finally obtains the target foreign matter detection result according to the first detection result and the second detection result, thereby effectively improving the foreign matter detection efficiency and accuracy of the circuit board on the upper board device. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flow chart of a method for detecting foreign matter on a circuit board on a board loading device in an embodiment of the present invention;

[0048] Figure 2 is a schematic diagram of a docking port of an upper board machine in an embodiment of the present invention;

[0049] Figure 3 It is a structural schematic diagram of a foreign matter detection device for a circuit board on a board loading device in an embodiment of the present invention.

[0050] Explanation of the reference numerals: 11-board loading equipment, 12-board loading machine connection port, 13-chip mounting equipment, 14-circuit board. DETAILED DESCRIPTION

[0051] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0052] like Figure 1 As shown, an embodiment of the present invention provides a method for detecting foreign matter on a circuit board on a board loading device, comprising the following steps:

[0053] Step 101, acquiring sensor data for detecting a board-mounting state of a circuit board on a board-mounting device and image data of the circuit board;

[0054] Step 102, preprocessing the sensor data and the image data respectively to obtain preprocessed sensor data and preprocessed image data;

[0055] Step 103, performing a first detection on the circuit board according to the preprocessed sensor data to obtain a first detection result;

[0056] Step 104, performing a second inspection on the circuit board according to the first inspection result and the preprocessed image data to obtain a second inspection result;

[0057] Step 105: Obtain a target foreign body detection result according to the first detection result and the second detection result.

[0058] The method for detecting foreign objects on a circuit board on a board device proposed in an embodiment of the present invention obtains sensor data for detecting the board status of the circuit board on the board device and image data of the circuit board, and preprocesses the sensor data and the image data respectively, performs a first detection on the circuit board according to the preprocessed sensor data, and performs a second detection on the circuit board according to a first detection result of the first detection and the preprocessed image data to obtain a second detection result, and finally obtains a target foreign object detection result according to the first detection result and the second detection result, thereby effectively improving the efficiency and accuracy of foreign object detection on the circuit board on the board device.

[0059] In an optional embodiment of the present invention, the step 101 of acquiring sensor data for detecting the boarding state of the circuit board on the boarding device includes:

[0060] Step 1011, collecting optical sensor data of the circuit board using an optical sensor installed at a first preset position on the upper board device;

[0061] Specifically, the board loading device is a device that transports the circuit board from the storage position to the patch position. The optical sensor can determine whether there is dust or debris on the circuit board by detecting changes in light intensity. Therefore, it should be installed in a position where it can directly illuminate the surface of the circuit board, such as above or on the side of the circuit board, to ensure that the light can cover the entire area that needs to be detected. Therefore, the first preset position can be on the board loading device above or on the side of the circuit board.

[0062] Step 1012, collecting capacitance sensor data of the circuit board using a capacitance sensor installed at a second preset position on the upper board device;

[0063] Specifically, the capacitive sensor detects objects by measuring changes in capacitance. When dust or debris adheres to the circuit board, the capacitance value on the circuit board surface will change. The capacitive sensor should be installed at key locations on the circuit board, such as component-intensive areas and near interfaces. These areas are more susceptible to dust and debris. Therefore, the second preset location can be on the upper board device such as the component-intensive area of ​​the circuit board or near the interface.

[0064] Step 1013, collecting infrared sensor data of the circuit board using an infrared sensor installed at a third preset position on the upper board device;

[0065] Specifically, the infrared sensor determines the existence of an object by detecting the infrared radiation emitted by it. Dust and debris absorb and reflect infrared radiation, thereby changing the signal strength received by the sensor. The infrared sensor should be installed in a position where it can directly detect the surface of the circuit board while avoiding interference from other heat sources or light sources. Therefore, the third preset position can be on the upper board device without heat sources or light sources, such as the inside of the upper board device.

[0066] Step 1014: Obtain sensor data of the circuit board according to the optical sensor data, the capacitive sensor data, and the infrared sensor data.

[0067] Specifically, the sensor data of the circuit board may include at least one of optical sensor data, capacitive sensor data and infrared sensor data. The specific sensor type and sensor data may be installed according to actual needs and the structure of the on-board device.

[0068] In an optional embodiment of the present invention, the step 101 of acquiring image data for detecting a circuit board on a board loading device includes:

[0069] The image data of the circuit board on the upper board device is collected by using the collection device installed on the upper board device.

[0070] Specifically, the acquisition device (such as a high-resolution camera) is usually installed in a position where it can clearly capture the entire picture of the circuit board on the upper board device. This position needs to ensure that the distance between the camera and the circuit board is appropriate to capture enough details. At the same time, the camera should be perpendicular to the circuit board to reduce image distortion. Here, the acquisition device can be installed inside the upper board device, or fixed to the upper board device above or on the side of the circuit board through accessories such as brackets.

[0071] In an optional embodiment of the present invention, the sensor data is preprocessed in step 102 to obtain the preprocessed sensor data, including:

[0072] Step 1021, denoising the sensor data to obtain denoised data;

[0073] Specifically, since the sensor data may be affected by noise, filtering is required to eliminate the noise. Here, low-pass filters, high-pass filters, or band-pass filters can be used to eliminate noise in different frequency ranges. Median filtering, mean filtering, etc. can also be used to smooth the data to reduce the impact of noise. For example, for optical sensor data, a low-pass filter can be used to eliminate high-frequency noise to obtain more accurate light intensity data; for capacitive sensor data, a median filter can be used to eliminate capacitance fluctuations caused by uneven circuit board surfaces or uneven dust distribution.

[0074] Step 1022, performing data standardization processing on the denoised data to obtain standardized data;

[0075] Specifically, in order to make the data collected by different sensors or different time periods comparable, data standardization is required, and methods such as normalization and standard normal variable transformation can be used to standardize the data. For example, for infrared sensor data, the received infrared radiation intensity data can be normalized to make it fluctuate within a certain range, thereby facilitating subsequent data analysis and processing.

[0076] Step 1023, calibrate the standardized data to obtain pre-processed sensor data.

[0077] Specifically, the standardized data is calibrated to improve the accuracy of the data and the accuracy of subsequent detection based on the data. A known standard dust or debris sample can be used for calibration to determine the measurement range and accuracy of the sensor. For example, for optical sensor data, a light source with a known light intensity can be used for calibration to ensure that the sensor can accurately measure changes in light intensity.

[0078] In an optional embodiment of the present invention, preprocessing the image data in step 102 to obtain preprocessed image data includes:

[0079] Step 1024, performing grayscale processing on the image data to obtain grayscale image data;

[0080] Specifically, the collected color image data is converted into a grayscale image to reduce the amount of data and simplify subsequent processing steps.

[0081] Step 1025, filtering the grayscale image data to obtain filtered image data;

[0082] Specifically, filtering methods such as median filtering or mean filtering can be used to remove noise in the grayscale image and improve the image quality.

[0083] Step 1026, binarizing the filtered image data to obtain binary image data;

[0084] Specifically, the filtered image is converted into a binary image, that is, the pixels in the image are divided into two categories: foreground (such as components, dust, debris on a circuit board, etc.) and background, which helps to improve the accuracy of subsequent detection.

[0085] Step 1027: perform image enhancement processing on the binary image data to obtain pre-processed image data.

[0086] Specifically, at least one of erosion, expansion, opening operation, closing operation, etc. can be used to enhance the target features in the binary image, such as connecting broken edges, removing small isolated points, etc., so as to improve the subsequent processing efficiency.

[0087] In an optional embodiment of the present invention, step 103 includes:

[0088] Step 1031, extracting features from the preprocessed sensor data to obtain a plurality of feature information;

[0089] Specifically, the edge, texture and shape features of the circuit board can be extracted from the optical sensor data in the preprocessed sensor data; the capacitance change features can be extracted from the capacitance sensor data in the preprocessed sensor data; and the infrared radiation intensity features can be extracted from the infrared sensor data in the preprocessed sensor data. The multiple feature information can include image features, capacitance change features, and infrared radiation intensity features of the circuit board surface.

[0090] Here, the step of extracting image features of the circuit board surface from the optical sensor data in the preprocessed sensor data includes: identifying edge information in the optical sensor data, which helps to extract the contour and line features of the circuit board; based on the edge information, extracting texture features of the circuit board surface, such as contrast, energy, entropy, etc.; based on the texture features, extracting key points and feature descriptors on the circuit board, that is, shape features.

[0091] The step of extracting capacitance change features from capacitance sensor data in the preprocessed sensor data includes: extracting statistical features of capacitance change by calculating statistics such as mean, variance, maximum and minimum values ​​of capacitance sensor data; converting capacitance sensor data into frequency domain, and extracting frequency domain features of capacitance change, such as frequency components and power spectrum density, by spectrum analysis, Fourier transform and other methods. These features are helpful to identify the periodicity or randomness of capacitance change.

[0092] The step of extracting infrared radiation intensity features from the infrared sensor data in the preprocessed sensor data includes: extracting the infrared radiation intensity features of the circuit board by analyzing the brightness information of the infrared image in the infrared sensor data; and extracting the texture features of the infrared image by using texture analysis methods such as gray level co-occurrence matrix (GLCM).

[0093] Step 1032, performing feature fusion on the plurality of feature information to obtain fused feature data;

[0094] Specifically, by performing feature fusion on multiple feature information, more comprehensive circuit board surface information can be obtained. Here, multiple feature information can be fused according to Y=W1×X1+W2×X2+W3×X3 to obtain fused feature data. Among them, Y is the fused feature data, X1 is the feature information extracted from the optical sensor data in the preprocessed sensor data, W1 is the weight of X1, X2 is the feature information extracted from the capacitive sensor data in the preprocessed sensor data, W2 is the weight of X2, X3 is the feature information extracted from the infrared sensor data in the preprocessed sensor data, W3 is the weight of X3, and W1+W2+W3=1. Among them, the weight of each data can be preset according to the importance or relevance of the data, and set according to the actual situation to meet the actual needs. Here, the fused feature data includes features of multiple dimensions, such as image features, capacitive features, and infrared features.

[0095] Step 1033: Perform a first detection on the circuit board according to the fused feature data to obtain a first detection result.

[0096] Specifically, the fused feature data can be input into the first detection model to perform a first detection on the circuit board to obtain a model output result, and a first detection result is obtained according to the model output result. The first detection result includes information on whether there is a foreign object on the circuit board.

[0097] Here, the first detection model is obtained by training the first preset network model using the historical fusion feature data, and the training process includes:

[0098] Acquire historical fusion feature data, the historical fusion feature data including: fusion feature data on the circuit board with foreign matter and fusion feature data on the circuit board without foreign matter;

[0099] Divide the historical fusion feature data into a training set and a test set, usually with a ratio of 70% training set and 30% test set;

[0100] Initialize the model parameters of the first preset network model; including setting the number of trees to 100 or 200, the maximum depth of the tree to 10 or 20, the minimum number of samples required for splitting to 2 or 4, the minimum number of samples for leaf nodes to 2 or 4, the maximum number of features to 10 or 20, and the random seed can be set through the random_state parameter, etc.; the specific parameters can also be adjusted according to the actual situation;

[0101] Sampling with replacement is performed from the training set to build a tree. At each node of the tree, a subset of features is randomly selected and optimal partitioning is performed based on these features. The tree will continue to grow until a certain stopping condition is met (such as reaching the maximum depth, the number of samples in the node is less than the preset threshold, etc.).

[0102] All trained trees are integrated together to obtain the original first detection model;

[0103] The test set is used to evaluate and adjust the performance of the original first detection model to obtain the first detection model.

[0104] Here, the fused feature data is input into the first detection model to perform the first detection on the circuit board, and the model output results are obtained, including:

[0105] Input the fused feature data into the first detection model to obtain multiple prediction results; for example, the multiple prediction results may be: there is a foreign object on the circuit board, there is no foreign object on the circuit board, there is a foreign object on the circuit board, etc.;

[0106] The prediction result that appears most frequently among the multiple prediction results is used as the model output result. In this embodiment, "there is a foreign object on the circuit board" appears most frequently, so the model output result is that there is a foreign object on the circuit board.

[0107] Specifically, the fused feature data can also be matched with a preset foreign body feature library to identify the type and location of the foreign body. The first detection result can be information that the circuit board contains foreign matter, and can also include the type (such as dust or debris, etc.) and location of the foreign matter.

[0108] In an optional embodiment of the present invention, step 104 includes:

[0109] Step 1041, when the first detection result contains foreign matter information, segmenting the preprocessed image data to obtain segmented image data;

[0110] Specifically, when the first detection result is that there is a foreign object on the circuit board, further detection and determination is required based on the image data. The pre-processed image data can be segmented, that is, the circuit board image is segmented into multiple small blocks or areas. The purpose of segmentation is to divide the circuit board image into smaller, easier to process parts for subsequent analysis and location of foreign objects.

[0111] Step 1042, converting the format of the segmented image data to obtain a feature vector;

[0112] Specifically, features can be extracted from each segmented small block or area (i.e., segmented image data). These features can be pixel-based statistical features (such as grayscale mean, variance, texture features, etc.), shape features (such as area, perimeter, compactness, etc.), or more complex features (such as local binary patterns, HOG features (an image feature descriptor used for object detection in computer vision), etc.). The extracted features are converted into numerical feature vectors, which contain the key features in the segmented image data. The purpose of converting the segmented image data into feature vectors is to facilitate the subsequent foreign body detection model processing.

[0113] Step 1043, inputting the feature vector into a trained foreign body detection model to obtain a detection value; the foreign body detection model is obtained by training a preset network model using annotated historical circuit board image data;

[0114] Specifically, the training process of the foreign body detection model includes:

[0115] Collect historical circuit board image data and its corresponding classification labels (the presence of foreign objects such as dust and debris on the circuit board or the absence of foreign objects such as dust and debris on the circuit board) as training samples;

[0116] The training samples are preprocessed by data cleaning, feature selection and data standardization to improve data quality and make the defect recognition model obtained by training more effective. The preprocessed historical circuit board image data is divided into training samples and test samples.

[0117] Select kernel functions and hyperparameters. The kernel function is used to map data from the original space to a high-dimensional feature space so as to perform linear classification in the high-dimensional feature space and improve the processing efficiency of the model. The kernel function used in this embodiment is: ,in, is the input vector, is the center vector, for and The Euclidean distance between is a monotonic function, when keep away When the function value is very small, exp is an exponential function with the natural constant e as the base. The hyperparameters used in this embodiment are the regularization parameter C, the kernel function coefficient, etc., which are used to control the complexity of the model and prevent overfitting. A range is defined for the regularization parameter C and the kernel function coefficient respectively, and multiple candidate values ​​are randomly selected within this range to participate in the training of the subsequent model.

[0118] Determine the objective function: ,in, It is a regularization term, which makes the weight vector as small as possible, thereby controlling the complexity of the model. A small weight vector means that the model is less dependent on the training data, which may reduce the risk of overfitting. is the loss term, which measures how well the model fits the training data. C is the regularization parameter, which controls the trade-off between the regularization term and the loss term. Specifically, the larger the value of C, the more the model focuses on minimizing the training error (ie, the loss term), which may cause the model to perform well on the training data but may overfit. On the contrary, the smaller the value of C, the more the model focuses on reducing the norm of the weight vector (ie, the regularization term), which may cause the model to perform averagely on the training data but have stronger generalization ability. Therefore, by adjusting the value of C, the complexity and generalization ability of the preset network model can be controlled. In practical applications, the value of C is usually selected through methods such as cross-validation to find a model that performs well on both the training set and the test set. w is the weight vector (the coefficient that defines the decision boundary in the feature space), b is the bias term of the decision boundary, is a slack variable (used to handle samples that do not satisfy the hard interval constraint, that is, allowing some samples to be inside the interval or on the wrong side), n is the number of training samples, and C is the regularization parameter.

[0119] After the kernel function, hyperparameters and objective function of the preset network model are determined, the training samples are used to train it to obtain the trained preset network model; the test samples are input into the trained preset network model for performance evaluation, and the hyperparameters (i.e., regularization parameters C) and kernel function coefficients that make the trained preset network model have the best performance (or minimize the objective function) are selected. After these parameters or coefficients are determined, the optimal hyperplane and foreign body detection model can be determined.

[0120] Specifically, the feature vector is input into a trained foreign body detection model to obtain a detection value, including:

[0121] The feature vector is mapped to obtain mapping data; the feature vector is mapped to a higher-dimensional feature space through a kernel function of a foreign body detection model to find an optimal classification hyperplane.

[0122] According to the mapping data, a support vector is determined; in the high-dimensional space, the foreign body detection model finds the data point closest to the classification hyperplane, namely the support vector.

[0123] The support vector is maximized to obtain an optimal classification hyperplane; the foreign body detection model finds the optimal classification hyperplane by maximizing the distance (interval) from the support vector to the classification hyperplane;

[0124] According to the optimal classification hyperplane, the detection value is obtained. According to the distance between the feature vector and the optimal classification hyperplane, the detection value is obtained, that is, the detection value is equal to the distance between the feature vector and the optimal classification hyperplane (the formula is: ), where f(x) is the distance between the detection value or feature vector and the optimal classification hyperplane, x is the feature vector, b is the bias term, w is the weight vector, which has the same dimension as x, and T is the transpose operation).

[0125] Step 1044, obtaining a second detection result according to the detection value and a preset threshold.

[0126] Specifically, if the detection value (such as 0.8) is greater than a preset threshold (such as 0.5), it means that foreign matter such as dust and debris exists on the circuit board, and the second detection result may be that foreign matter exists on the circuit board.

[0127] Here, the feature points or markers in the image data can also be used in combination with image processing technology (such as edge detection, contour extraction, etc.) to determine the location of the foreign matter in the image. And the image data is matched with a preset foreign matter feature library to confirm the type of foreign matter (such as dust, debris, etc.). The second detection result can include whether the circuit board contains foreign matter, and the location of the foreign matter, etc.

[0128] In an optional embodiment of the present invention, step 105 includes:

[0129] Step 1051, determining a first feature of a foreign object and first positioning data of the foreign object according to the first detection result;

[0130] Specifically, the first detection result may include the type of the identified foreign object (ie, the first feature of the foreign object, such as dust or debris, etc.) and the position (first positioning data of the foreign object).

[0131] Step 1052, determining a second feature of the foreign object and second positioning data of the foreign object according to the second detection result;

[0132] Specifically, the second detection result may include the location of small pieces or areas of dust, foreign matter, etc. This can be achieved by recording the coordinates or index of each small piece or area, and then these locations can be marked on the original circuit board image data for visualization or further processing. The second feature of the foreign matter (such as dust, debris, etc.) and the second positioning data of the foreign matter (the location of the foreign matter in the image) are extracted from the second detection result.

[0133] Step 1053, matching the first feature of the foreign object with the second feature of the foreign object to obtain basic data of the foreign object;

[0134] Specifically, the first feature of the foreign object and the second feature of the foreign object are compared and verified. If there is a difference, calibration and adjustment are performed to improve the accuracy of the foreign object feature.

[0135] Step 1054, obtaining foreign object positioning data according to the first foreign object positioning data and the second foreign object positioning data;

[0136] Specifically, the first positioning data of the foreign object and the second positioning data of the foreign object are compared and verified. If there is a difference, calibration and adjustment are performed to improve the accuracy of foreign object positioning.

[0137] Step 1055: Obtain a target foreign object detection result according to the foreign object basic data and the foreign object positioning data.

[0138] Specifically, the target foreign body detection results may include the final determined foreign body basic data (i.e., foreign body type, such as dust, debris, etc.) and foreign body positioning data (the specific location of the foreign body, such as the location and coordinates of the foreign body in the image), which is convenient for subsequent processing.

[0139] In an optional embodiment of the present invention, the method further comprises:

[0140] Step 106, determining control parameters according to the target foreign body detection result;

[0141] Specifically, the coordinates of the foreign matter in the circuit board image can be extracted from the target foreign matter detection result and converted into control parameters that can be recognized by the cleaning equipment (robotic arm or air gun, etc.).

[0142] Step 107, controlling the removing device to remove foreign matter according to the control parameters.

[0143] Specifically, the control parameters can be input into a cleaning device such as a robot arm or an air gun, so that the robot arm or the air gun can remove the foreign matter from the circuit board according to the position of the foreign matter in the control parameters, thereby improving the work efficiency of subsequent processes. A specific embodiment of the method for detecting foreign matter on a circuit board on a board loading device proposed in an embodiment of the present invention is as follows:

[0144] Step 111, obtaining data;

[0145] The optical sensor installed on the upper board device is used to collect the optical sensor data of the circuit board, the capacitive sensor is used to collect the capacitive sensor data of the circuit board, the infrared sensor is used to collect the infrared sensor data of the circuit board, and the image acquisition device is used to collect the image data of the circuit board.

[0146] Step 112, data preprocessing;

[0147] The sensor data is subjected to denoising, data standardization, calibration and other processing to obtain preprocessed sensor data; the image data is subjected to grayscale processing, filtering processing, binarization processing and image enhancement processing to obtain preprocessed image data.

[0148] Step 113, performing a first foreign body detection on the circuit board;

[0149] The sensor data is fused and input into the first detection model to perform the first detection on the circuit board, and matched with the preset foreign body feature library to obtain the first detection result of whether there is a foreign body on the circuit board, the type and location of the foreign body.

[0150] Step 114, performing a second foreign matter detection on the circuit board;

[0151] When the first detection result is that there is a foreign object on the circuit board, a second foreign object detection is required based on the image data to improve the accuracy and reliability of the detection. After the image data is segmented, it is input into the trained foreign object detection model, and the output result of the model is used to determine whether there is a foreign object on the circuit board, and a second match is performed based on the preset foreign object feature library to obtain a second detection result of whether there is a foreign object on the circuit board, the type of foreign object, and the location of the foreign object.

[0152] Step 115, determining the target foreign body detection result.

[0153] According to the two test results and the comparison and verification of the two test results, the location and type of foreign matter can be obtained, and the control parameters can be determined accordingly to control the robot arm or air gun to remove foreign matter from the circuit board, thereby improving the safety and work efficiency of subsequent processes. The robot arm can be equipped with accessories such as nozzles to absorb and remove tiny foreign matter; the air gun is used to blow away surface dust and larger debris.

[0154] Step 116, performing foreign matter detection on the circuit board at the upper board machine interface.

[0155] like Figure 2As shown, after the foreign matter on the circuit board 14 is removed, the circuit board 14 will be transported by the board loading device 11 to the board loading machine docking port 12 connected to the input end of the subsequent patch equipment 13 (such as a patch machine, a reflow oven, etc.). At this time, the second optical sensor installed at the board loading machine docking port 12 is used to collect the real-time optical sensor data of the circuit board 14, the second capacitive sensor is used to collect the real-time capacitive sensor data of the circuit board 14, the second infrared sensor is used to collect the real-time infrared sensor data of the circuit board 14, and the second image acquisition device installed on the board loading machine docking port 12 is used to collect the real-time image data of the circuit board 14. According to these real-time data, the circuit board 14 is detected again (the detection method can be the same as that in the above embodiment). If there is no foreign matter, the circuit board is smoothly and accurately transferred from the docking port to the subsequent patch equipment 13 for patch operation, etc., by using belts, chains or rollers installed at the docking port. If there is foreign matter, the robot arm or air gun can be controlled again to remove the foreign matter from the circuit board 14, or the robot arm can be controlled to remove the circuit board 14 from the upper board machine docking port 12 for foreign matter removal or manual inspection.

[0156] By re-detecting foreign objects on the circuit board at the board loading machine interface, foreign objects on the circuit board are prevented from directly entering the subsequent patch equipment or printing equipment to cause damage to the steel mesh, thereby improving production safety. It should be noted that the patch equipment mentioned above can also be a printing equipment or other equipment, which can be determined according to the subsequent process.

[0157] In this embodiment, an alarm message may also be generated according to the target foreign body detection result to remind production or management personnel to check and handle the foreign body in a timely manner.

[0158] The foreign body detection method for a circuit board on a boarding device proposed in an embodiment of the present invention determines whether there are foreign bodies on the circuit board and the location of the foreign bodies by performing a first detection and a second foreign body detection on the circuit board, and then controls a mechanical arm or an air gun or other cleaning device to remove the foreign bodies from the circuit board, automatically detects and removes foreign bodies before the circuit board is boarded, ensures the cleanliness of the circuit board surface, improves the quality and efficiency of SMT patches, or the safety and production efficiency of other subsequent circuit board production processes. It is conducive to realizing the automatic detection and removal of foreign bodies on the circuit board, reducing manual intervention and labor intensity, and reducing labor costs.

[0159] like Figure 3 As shown, an embodiment of the present invention provides a foreign body detection device 200 for a circuit board on a board loading device, comprising:

[0160] An acquisition module 201 is used to acquire sensor data for detecting a board-mounting state of a circuit board on a board-mounting device and image data of the circuit board;

[0161] The processing module 202 is used to preprocess the sensor data and the image data respectively to obtain preprocessed sensor data and preprocessed image data; perform a first detection on the circuit board according to the preprocessed sensor data to obtain a first detection result; perform a second detection on the circuit board according to the first detection result and the preprocessed image data to obtain a second detection result; and obtain a target foreign body detection result according to the first detection result and the second detection result.

[0162] Optionally, obtaining sensor data for detecting a board-mounting state of a circuit board on a board-mounting device includes:

[0163] Collecting optical sensor data of the circuit board using an optical sensor installed at a first preset position on the upper board device;

[0164] Collecting capacitance sensor data of the circuit board using a capacitance sensor installed at a second preset position on the upper board device;

[0165] Collecting infrared sensor data of the circuit board using an infrared sensor installed at a third preset position on the upper board device;

[0166] Sensor data of the circuit board is obtained according to the optical sensor data, the capacitive sensor data and the infrared sensor data.

[0167] Optionally, preprocessing the sensor data to obtain preprocessed sensor data includes:

[0168] Performing denoising processing on the sensor data to obtain denoised data;

[0169] Performing data standardization processing on the denoised data to obtain standardized data;

[0170] The standardized data is calibrated to obtain pre-processed sensor data.

[0171] Optionally, preprocessing the image data to obtain preprocessed image data includes:

[0172] Performing grayscale processing on the image data to obtain grayscale image data;

[0173] Performing filtering processing on the grayscale image data to obtain filtered image data;

[0174] Binarizing the filtered image data to obtain binary image data;

[0175] Perform image enhancement processing on the binary image data to obtain pre-processed image data.

[0176] Optionally, performing a first detection on the circuit board according to the preprocessed sensor data to obtain a first detection result includes:

[0177] Performing feature extraction on the preprocessed sensor data to obtain a plurality of feature information;

[0178] Performing feature fusion on the multiple feature information to obtain fused feature data;

[0179] A first detection is performed on the circuit board according to the fused feature data to obtain a first detection result.

[0180] Optionally, performing a second inspection on the circuit board according to the first inspection result and the preprocessed image data to obtain a second inspection result includes:

[0181] When the first detection result contains foreign matter information, segmenting the preprocessed image data to obtain segmented image data;

[0182] Performing format conversion on the segmented image data to obtain a feature vector;

[0183] Input the feature vector into a trained foreign body detection model to obtain a detection value; the foreign body detection model is obtained by training a preset network model using annotated historical circuit board image data;

[0184] A second detection result is obtained according to the detection value and a preset threshold.

[0185] Optionally, obtaining a target foreign body detection result according to the first detection result and the second detection result includes:

[0186] Determine a first feature of the foreign object and first positioning data of the foreign object according to the first detection result;

[0187] Determine a second feature of the foreign object and second positioning data of the foreign object according to the second detection result;

[0188] Matching the first feature of the foreign object with the second feature of the foreign object to obtain basic data of the foreign object;

[0189] Obtaining foreign object positioning data according to the first foreign object positioning data and the second foreign object positioning data;

[0190] A target foreign object detection result is obtained according to the foreign object basic data and the foreign object positioning data.

[0191] The foreign object detection device for a circuit board on a board device proposed in an embodiment of the present invention obtains sensor data for detecting the board state of the circuit board on the board device and image data of the circuit board, and preprocesses the sensor data and the image data respectively, performs a first detection on the circuit board according to the preprocessed sensor data, and performs a second detection on the circuit board according to a first detection result of the first detection and the preprocessed image data to obtain a second detection result, and finally obtains a target foreign object detection result according to the first detection result and the second detection result, thereby effectively improving the foreign object detection efficiency and accuracy of the circuit board on the board device.

[0192] It should be noted that the device is a device corresponding to the above method, and all implementations in the above method embodiment are applicable to the embodiment of the device and can achieve the same technical effect. This embodiment will not be repeated.

[0193] The embodiment of the present invention further provides a computing device, comprising: a processor, and a memory storing a computer program, wherein when the computer program is executed by the processor, the method described in any one of the above embodiments is executed. All implementations in the above method embodiments are applicable to the embodiments of the device, and can achieve the same technical effects. They will not be described in detail in this embodiment.

[0194] The embodiment of the present invention further provides a computer-readable storage medium on which instructions are stored. When the instructions are executed on a computer, the computer executes the method as described in any one of the above embodiments. All implementations in the above method embodiments are applicable to the embodiments of the device and can achieve the same technical effects. They will not be described in detail in this embodiment.

[0195] It should be noted that in the apparatus and method of the present invention, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of the present invention. In addition, the steps of performing the above series of processes can naturally be performed in chronological order according to the order of description, but do not necessarily need to be performed in chronological order. Some steps can be performed in parallel, crosswise or independently of each other.

[0196] It should be noted that, in the above-mentioned embodiments, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include one..." do not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the implementation of the above-mentioned embodiments is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0197] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for detecting foreign matter on a circuit board on a board loading device, characterized in that: include: Acquire sensor data for detecting a board-mounting state of a circuit board on a board-mounting device and image data of the circuit board; Preprocessing the sensor data and the image data respectively to obtain preprocessed sensor data and preprocessed image data; Performing a first detection on the circuit board according to the preprocessed sensor data to obtain a first detection result; Perform a second inspection on the circuit board according to the first inspection result and the preprocessed image data to obtain a second inspection result; A target foreign body detection result is obtained according to the first detection result and the second detection result.

2. The method for detecting foreign matter on a circuit board on a board loading device according to claim 1, characterized in that: Acquire sensor data for detecting the board status of a circuit board on a board device, including: Collecting optical sensor data of the circuit board using an optical sensor installed at a first preset position on the upper board device; Collecting capacitance sensor data of the circuit board using a capacitance sensor installed at a second preset position on the upper board device; Collecting infrared sensor data of the circuit board using an infrared sensor installed at a third preset position on the upper board device; Sensor data of the circuit board is obtained according to the optical sensor data, the capacitive sensor data and the infrared sensor data.

3. The method for detecting foreign matter on a circuit board on a board loading device according to claim 1, characterized in that: Preprocessing the sensor data to obtain preprocessed sensor data includes: Performing denoising processing on the sensor data to obtain denoised data; Performing data standardization processing on the denoised data to obtain standardized data; The standardized data is calibrated to obtain pre-processed sensor data.

4. The method for detecting foreign matter on a circuit board on a board loading device according to claim 1, characterized in that: Preprocessing the image data to obtain preprocessed image data includes: Performing grayscale processing on the image data to obtain grayscale image data; Performing filtering processing on the grayscale image data to obtain filtered image data; Binarizing the filtered image data to obtain binary image data; Perform image enhancement processing on the binary image data to obtain pre-processed image data.

5. The method for detecting foreign matter on a circuit board on a board loading device according to claim 1, characterized in that: Performing a first detection on the circuit board according to the preprocessed sensor data to obtain a first detection result includes: Performing feature extraction on the preprocessed sensor data to obtain a plurality of feature information; Performing feature fusion on the multiple feature information to obtain fused feature data; A first detection is performed on the circuit board according to the fused feature data to obtain a first detection result.

6. The method for detecting foreign matter on a circuit board on a board loading device according to claim 1, characterized in that: Performing a second inspection on the circuit board according to the first inspection result and the preprocessed image data to obtain a second inspection result includes: When the first detection result contains foreign matter information, segmenting the preprocessed image data to obtain segmented image data; Performing format conversion on the segmented image data to obtain a feature vector; Input the feature vector into a trained foreign body detection model to obtain a detection value; the foreign body detection model is obtained by training a preset network model using annotated historical circuit board image data; A second detection result is obtained according to the detection value and a preset threshold.

7. The method for detecting foreign matter on a circuit board on a board loading device according to claim 1, characterized in that: Obtaining a target foreign body detection result according to the first detection result and the second detection result, including: Determine a first feature of the foreign object and first positioning data of the foreign object according to the first detection result; Determine a second feature of the foreign object and second positioning data of the foreign object according to the second detection result; Matching the first feature of the foreign object with the second feature of the foreign object to obtain basic data of the foreign object; Obtaining foreign object positioning data according to the first foreign object positioning data and the second foreign object positioning data; A target foreign object detection result is obtained according to the foreign object basic data and the foreign object positioning data.

8. A foreign body detection device for a circuit board on a board loading device, characterized in that: include: An acquisition module, used for acquiring sensor data for detecting the board state of the circuit board on the board loading device and image data of the circuit board; A processing module, used to preprocess the sensor data and the image data respectively to obtain preprocessed sensor data and preprocessed image data; Performing a first detection on the circuit board according to the preprocessed sensor data to obtain a first detection result; Perform a second inspection on the circuit board according to the first inspection result and the preprocessed image data to obtain a second inspection result; A target foreign body detection result is obtained according to the first detection result and the second detection result.

9. A computing device, characterized in that include: A processor and a memory storing a computer program, wherein when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is performed.

10. A computer-readable storage medium, characterized in that: Instructions are stored, and when the instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

Images