Positioning pin defect detection method, device, system and equipment and storage medium

Abstract

The invention provides a positioning pin defect detection method, device and system based on machine vision, electronic equipment and a storage medium, one scheme is used for processing equipment, the processing equipment comprises a positioning pin for fixing a PCB, and the method comprises the following steps: obtaining an appearance image of the positioning pin; positioning a target detection area in the appearance image, wherein the target detection area is an area where the actual position of the positioning pin in the appearance image is located; detecting the working state of the positioning pin in the target detection area to obtain a detection result; and when the detection result is that the positioning pin has defects, triggering a corresponding positioning pin defect alarm. The problem that the positioning precision is reduced in the subsequent PCB machining process due to the fact that the positioning pins are broken or inclined in the installation process is effectively solved, and the positioning precision in the PCB machining process is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine vision, and in particular to a positioning pin defect detection method, device, system, electronic device and storage medium based on machine vision. BACKGROUND

[0002] With the rapid construction of 5G communication infrastructure and the iterative upgrade of electronic equipment, the printed circuit board (PCB) industry has ushered in new opportunities for intelligent transformation, and automatic production has become an important means to improve productivity and reduce costs. In the process of fixing the PCB with positioning pins, the traditional manual positioning pin and adhesive fixing method has been replaced by an automatic PIN device to automatically sleeve the positioning pins, but the positioning pins used therein are four-hole openings, which are easily impacted during the fixing process of the PCB, resulting in breakage or skew, and thus causing the positioning accuracy to decrease during the PCB processing. SUMMARY

[0003] The present application provides a positioning pin defect detection method, device, system, electronic device and storage medium based on machine vision, aiming to solve the problem of breakage or skew of the positioning pin during installation, resulting in a decrease in positioning accuracy during the PCB processing.

[0004] In a first aspect, a positioning pin defect detection method based on machine vision is provided for a processing device, the processing device including a positioning pin for fixing a PCB, and the method including: obtaining an appearance image of the positioning pin; locating a target detection area in the appearance image, the target detection area being an area where the actual position of the positioning pin in the appearance image is located; detecting the working state of the positioning pin in the target detection area to obtain a detection result; when the detection result is a positioning pin defect, triggering a corresponding positioning pin defect alarm.

[0005] In a second aspect, a positioning pin defect detection device based on machine vision is provided for a processing device, the processing device including a positioning pin for fixing a PCB, and the device including: an acquisition module configured to acquire an appearance image of a positioning pin, the positioning pin being a pin with a blade structure; a locating module configured to locate a target detection area in the appearance image, the target detection area being an area where the actual position of the positioning pin in the appearance image is located; a detection module configured to detect the working state of the positioning pin in the target detection area to obtain a detection result; trigger a corresponding positioning pin defect alarm when the detection result is a positioning pin defect.

[0006] In a third aspect, a positioning pin defect detection system based on machine vision is provided. The system includes a processing device and an electronic device. The processing device includes an image acquisition device and a positioning pin for fixing a PCB. The electronic device is configured to: acquire an appearance image of the positioning pin captured by the image acquisition device; locate a target detection area in the appearance image, the target detection area being an area where the positioning pin is actually located in the appearance image; detect a working state of the positioning pin in the target detection area to obtain a detection result; trigger a corresponding positioning pin defect alarm when the detection result is a positioning pin defect.

[0007] In a fourth aspect, an electronic device is provided. The electronic device includes a processor and a memory. The memory is configured to store a computer program. The processor is configured to execute the program stored in the memory to implement the method steps of the first aspect.

[0008] In a fifth aspect, a computer readable storage medium is provided. The computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the method steps of the first aspect.

[0009] An advantage of one scheme provided in the present application is that: by acquiring an appearance image of a positioning pin, locating a target detection area corresponding to an actual position of the positioning pin in the appearance image, and then detecting a working state of the positioning pin in the target detection area, the positioning pin defect can be efficiently and accurately identified to obtain a detection result. Further, when the detection result is a positioning pin defect, a corresponding positioning pin defect alarm is triggered. In this way, after the PCB is positioned by the positioning pin, the positioning pin can be detected first. When the positioning pin fails, the subsequent processing of the PCB can be stopped in time, thereby effectively avoiding the problem that the positioning pin is broken or tilted during installation, resulting in a decrease in positioning accuracy during subsequent PCB processing, and improving the positioning accuracy during PCB processing. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 FIG. 1 is a schematic diagram of a processing device according to an embodiment of the present application; Figure 2 FIG. 2 is a three-dimensional schematic diagram of an automatic PIN sleeve device according to an embodiment of the present application; Figure 3 ​​is a three-dimensional schematic diagram of the positioning pin core rising in the automatic PIN device provided by an embodiment of the present application; Figure 4 is a flowchart of a positioning pin defect detection method based on machine vision provided by an embodiment of the present application; Figure 5 is a schematic diagram of a target circular ring provided by an embodiment of the present application; Figure 6 is a schematic diagram of a positioning pin not installed provided by an embodiment of the present application; Figure 7 is a schematic diagram of a positioning pin skew provided by an embodiment of the present application; Figure 8 is a schematic diagram of successful matching of key points provided by an embodiment of the present application; Figure 9 is a schematic diagram of failed matching of key points provided by an embodiment of the present application; Figure 10 is a schematic diagram of a positioning pin defect provided by an embodiment of the present application; Figure 11 is another schematic diagram of a positioning pin defect provided by an embodiment of the present application; Figure 12 is another schematic diagram of a positioning pin defect provided by an embodiment of the present application; Figure 13 is another schematic diagram of a positioning pin defect provided by an embodiment of the present application; Figure 14 is a structural diagram of a positioning pin defect detection device based on machine vision provided by the present application; Figure 15 is a structural diagram of a positioning pin defect detection system based on machine vision provided by the present application; Figure 16 is a structural diagram of an electronic device provided by the present application.

[0011] Among them, the reference signs are as follows: 401, processing equipment; 4011, image acquisition device; 4012, positioning pin; 4013, base plate; 4014, positioning pin core; 4015, processing main shaft; 600, target circular ring; 601, outer ring; 602, inner ring; 700, petal. DETAILED DESCRIPTION

[0012] In order to make the technical problems, technical solutions and beneficial effects solved by the present application clearer, the present application will be further described in detail below in combination with embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0013] Prior art Firstly, the detection of the existing positioning pin 4012 relies on manual visual inspection. Due to various reasons (for example, forgetting to check after installing the positioning pin 4012), it is difficult to find that the positioning pin 4012 is broken or tilted during the installation process in time, thereby causing the positioning accuracy to decrease in the subsequent PCB processing process.

[0014] Secondly, the existing defect detection of the PIN needle is directly detected on the collected PIN needle image, and the structure of the PIN needle and the positioning pin 4012 is different, so it cannot be directly applied to the defect detection of the positioning pin 4012 in the present application.

[0015] Term explanation In the present application, CCD (charge coupled device) refers to a charge coupled device, which is commonly used in optical imaging sensors in industrial image acquisition systems.

[0016] In the present application, PCA (Principal Component Analysis) refers to principal component analysis.

[0017] In the present application, GLCM (Gray-Level Co-occurrence Matrix) refers to a gray-level co-occurrence matrix.

[0018] In the present application, the positioning pin 4012 (slotted PIN) is a cylindrical shaft body provided with a slot along the axial direction, wherein the slot divides the cylindrical shaft body into a plurality of petals.

[0019] In the present application, the positioning pin core (PIN core) 4014 refers to a fixed positioning pin core, which can be controlled to move up and down by a mechanical method, and can expand the plurality of petals of the positioning pin 4012, i.e., expand the plurality of petals of the cylindrical shaft body, thereby achieving effective positioning and fixation of the PCB.

[0020] In the present application, HMI (‌Human Machine Interface) refers to a human-machine interface.

[0021] In the present application, TWO PIN refers to a method of fixing and positioning by two positioning pins by using a clamp in the processing process.

[0022] Structure explanation In the present application, please refer to Figures 1 to 3 The processing equipment 401 includes an automatic PIN fitting device, a processing spindle 4015, and an image acquisition device 4011. The processing spindle 4015 can clamp a tool to process the PCB. The image acquisition device 4011 is arranged on one side of the processing spindle 4015. The automatic PIN fitting device comprises a positioning pin 4012 and a base plate 4013, wherein the positioning pin 4012 is embedded in the base plate 4013; a target hole matched with the positioning pin 4012 is arranged on the PCB, that is, a target hole matched with the cylindrical shaft body is arranged on the PCB, so that when the positioning pin 4012 rises into the target hole of the PCB, the positioning pin core 4014 in the positioning pin 4012 rises (as shown in Figure 3 ), so as to separate the petals of the positioning pin 4012, that is, separate the petals formed by the cylindrical shaft body, thereby effectively positioning and fixing the PCB.

[0023] Preferably, the image acquisition device 4011 comprises an industrial camera (for example, a CCD camera) and a matched light source assembly, the industrial camera is used to capture the appearance image of the area where the positioning pin 4012 is located, and the light source matched with the light source assembly can realize high-contrast and high-resolution image acquisition effect.

[0024] Detection type description Since the positioning pin 4012 has many features, and the key features affecting whether the positioning pin 4012 has a fault may only include a few, but if each feature of the positioning pin 4012 is detected, the detection time will be longer, which will affect the subsequent processing efficiency, therefore, the present application adopts the following PCA analysis process, first determines the key features (i.e., position features and shape features) affecting whether the positioning pin 4012 has a fault, and then detects the key features of the positioning pin 4012, that is, it can ensure accurate detection of whether the positioning pin 4012 has a fault, and can effectively reduce the detection time, that is, based on the features of the positioning pin 4012 and the corresponding weights (i.e., the cumulative contribution rate of principal components), the features having greater influence on the result are selected, that is, the position features and the shape features, which specifically include the following steps: S101: Obtain sample images of a plurality of positioning pins 4012 and a plurality of corresponding features; S102: Based on the plurality of sample images and the plurality of corresponding features, a feature matrix is constructed.

[0025] As an example, it is assumed that there are sample images, features, represents the sample image, the feature, and constitutes a feature matrix : ; Wherein, each row corresponds to all features of a sample image, and each column corresponds to a sample image.

[0026] S103: Standardize the features corresponding to each sample image in the feature matrix to obtain the standardized feature matrix.

[0027] As an example, one can consider the feature matrix. Each feature of each sample image in the dataset is standardized, i.e., by the formula: Each feature is processed to obtain a standardized feature matrix. : ; in, , The first The mean and standard deviation of each feature.

[0028] S104: Calculate the covariance matrix using the standardized feature matrix.

[0029] As an example, let the standardized feature matrix be... middle, Indicates the first The first sample image One characteristic, Indicates the first The first sample image Features, traversal and ( , For each Execution formula: The covariance matrix is ​​obtained through calculation. : .

[0030] S105: Calculate the eigenvalues ​​and corresponding eigenvectors of the covariance matrix.

[0031] As an example, because the covariance matrix It is a positive semi-definite matrix, and its trace This indicates that the sum of the variances of the standardized eigenvalues ​​is Therefore, all the calculated eigenvalues ​​are non-negative, and they are sorted from largest to smallest as follows: The calculated eigenvalues ​​and corresponding eigenvectors are as follows: .

[0032] S106: Calculate the contribution rate corresponding to each feature value based on each feature value and its corresponding feature vector.

[0033] As an example, the first eigenvalues The contribution rate That is, it can be done according to the formula: The contribution rate of each eigenvalue is calculated one by one. This contribution rate represents the proportion of each eigenvalue's contribution to the total variance and reflects its importance in the eigenvalue system.

[0034] S107: Before calculation The cumulative contribution rate of each feature value is used to determine the position and shape features of the positioning pin 4012 based on the cumulative contribution rate and the preset contribution rate.

[0035] As an example, this can be achieved through the formula: Before calculation The cumulative contribution rate of each feature value is used to determine the positional and shape characteristics of the positioning pin 4012 based on the cumulative contribution rate and a preset contribution rate. Assuming a preset contribution rate of 80%, the physical characteristics associated with these feature values ​​are analyzed when the cumulative contribution rate exceeds the preset rate. The analysis reveals that the main influencing features of the positioning pin 4012 include positional and shape characteristics.

[0036] Through the above analysis, the key features that play a crucial role in identifying defects in locating pins were selected, allowing the detection to focus on important features. This reduces the amount of computation while ensuring the accuracy of defect identification. The reason is that the key features mentioned above have already characterized the key information of the locating pins, effectively avoiding interference from too many redundant features and improving detection efficiency.

[0037] Please refer to the following as well. Figures 4 to 13 This invention provides a detailed description of a machine vision-based method for detecting defects in positioning pins.

[0038] Example 1 Firstly, this application provides a machine vision-based method for detecting defects in positioning pins. Please refer to [reference needed]. Figure 4 The processing equipment 401 includes positioning pins 4012 for fixing a PCB. Taking an electronic device 500 as an example, the electronic device 500 is connected to the processing equipment 401 and to an image acquisition device 4011. The process includes the following steps: S201: Obtain the appearance image of the positioning pin 4012.

[0039] As an example, before the processing device 401 processes the PCB, the positioning pin 4012 can be embedded in the target hole of the PCB, and after the PCB is fixed by the petals, the image acquisition device 4011 can acquire the image of the positioning pin 4012. Specifically, since the image acquisition device 4011 is arranged on one side of the processing spindle 4015, the image of the positioning pin 4012 at the preset position can be acquired by controlling the movement of the processing spindle 4015 and the platform. For example, a CCD camera and a constant light source are used, when the positioning pin 4012 moves into the field of view of the CCD camera, the appearance image of the positioning pin 4012 is automatically captured, and then the image acquisition device 4011 transmits the appearance image of the positioning pin 4012 to the electronic device 500 (for example, an industrial computer, a computer device, and a notebook computer) through the network for subsequent processing, so as to obtain the appearance image of the positioning pin 4012.

[0040] S202: Locate the target detection area in the appearance image, and the target detection area is an area where the positioning pin 4012 is located in the appearance image.

[0041] As an example, after the appearance image of the positioning pin 4012 is obtained, the appearance image can be directly input into the pre-trained image recognition model to automatically locate the target detection area through the pre-trained image recognition model. Alternatively, the appearance image can be sequentially subjected to grayscale processing, histogram equalization processing, and Gaussian filter denoising processing to generate a preprocessed image, and the preprocessed image can be input into the pre-trained image recognition model to automatically locate the target detection area. Herein, no limitation is made. In this embodiment, the pre-trained image recognition model is used to quickly locate the target detection area, thereby improving the detection speed.

[0042] The pre-trained image recognition model can be trained in the following manner: collecting training images of the positioning pin 4012, labeling the area corresponding to the actual position of the positioning pin 4012 in each training image to form a training data set; inputting the training data set into the image recognition model, calculating the positioning error by a loss function and combining an optimizer for back propagation training, adjusting the parameters of the image recognition model until convergence, and obtaining the pre-trained image recognition model.

[0043] It should be noted that the above training method is only an example, and the image recognition model can also be trained by using a training data set and a test data set. The image recognition model of the present application can include a convolutional neural network model and other neural network models, and no limitation is made herein.

[0044] S203: Detect the working state of the positioning pin 4012 in the target detection area to obtain a detection result. S204: When the detection result is a positioning pin defect, a corresponding positioning pin defect alarm is triggered.

[0045] In this embodiment, the working state of the positioning pin 4012 in the target detection area can be detected by analyzing the position features and shape features of the positioning pin in the target detection area through an image processing algorithm. For example, the installation state of the positioning pin 4012, the skew state of the positioning pin 4012, and the shape state of the positioning pin 4012 can be detected to obtain a detection result.

[0046] In this embodiment, the positioning pin defect can include an uninstalled positioning pin, a skewed positioning pin, and a missing positioning pin leaf, etc.

[0047] As an example, when the detection result is a positioning pin defect, a corresponding positioning pin defect alarm is triggered and the processing equipment 401 is controlled to pause all work processes. For example, when the positioning pin defect is an uninstalled positioning pin, an uninstalled positioning pin alarm is triggered to remind the technician to perform secondary detection and processing on the positioning pin 4012, and the processing equipment 401 is controlled to pause all work processes, such as pausing subsequent drilling operations. In this way, when the positioning pin is uninstalled, skewed, or the positioning pin leaf is missing, timely alarm and stopping of subsequent drilling operations can effectively avoid the problem of reduced positioning accuracy in subsequent processing due to the fracture or skew of the positioning pin 4012 during installation. That is, the processing flow can be quickly cut off before the abnormal positioning pin 4012 causes positioning misalignment or processing errors, thereby fundamentally preventing scrap or reduced processing quality caused by positioning pin defects, improving positioning accuracy in the processing process, reducing the rate of defective products and rework costs (i.e., economic costs and time costs), and improving overall automation level and production efficiency. Further, if all detection results are normal, the processing flow is allowed to continue running.

[0048] In summary, the positioning pin defect detection method based on machine vision provided by the present application is used for the processing equipment 401, which includes a positioning pin 4012 for fixing a PCB. The method includes: obtaining an appearance image of the positioning pin 4012; positioning a target detection area in the appearance image, the target detection area being an area where the positioning pin 4012 is located in the appearance image; detecting the working state of the positioning pin 4012 in the target detection area to obtain a detection result; and when the detection result is a positioning pin defect, triggering a corresponding positioning pin defect alarm.

[0049] Technical effects compared with prior art: By acquiring the appearance image of the positioning pin 4012, locating the target detection area corresponding to the actual position of the positioning pin 4012 in the appearance image, and then detecting the working state of the positioning pin 4012 in the target detection area, the positioning pin 4012 can be efficiently and accurately identified for defects to obtain a detection result. Further, when the detection result is a positioning pin defect, the corresponding positioning pin defect alarm is triggered. In this way, after positioning by the positioning pin 4012, the positioning pin 4012 can be detected first, and when the positioning pin 4012 fails, the subsequent PCB processing is stopped in time, effectively avoiding the problem that the positioning accuracy is reduced in the subsequent PCB processing due to the fracture or skew of the positioning pin 4012 during installation, improving the positioning accuracy in the PCB processing process, and reducing the scrap rate (i.e., the rejection rate) and rework cost.

[0050] In an embodiment, i.e., in step S203, the detection result includes a position detection result and a shape detection result. That is, the working state of the positioning pin 4012 in the target detection area is detected to obtain the detection result, including the following steps: S231: Detect the position characteristics of the positioning pin 4012 in the target detection area to obtain a position characteristic detection result. The position characteristic detection result includes an installation state result of the positioning pin 4012 and a skew state result of the positioning pin 4012. S232: When the installation state result is that the positioning pin is installed and the skew state result is that the positioning pin is not skewed, detect the shape characteristics of the positioning pin in the target detection area to obtain a shape detection result.

[0051] In this embodiment, the detection result includes a position detection result and a shape detection result. The position detection result includes an installation state result of the positioning pin 4012 and a skew state result of the positioning pin 4012. The installation state result of the positioning pin 4012 includes positioning pin installation and positioning pin non-installation. The skew state result of the positioning pin 4012 includes positioning pin non-skew and positioning pin skew. The shape detection result includes positioning pin petal loss and positioning pin non-petal loss.

[0052] As an example, the position feature of the positioning pin 4012 in the target detection area can be detected first to obtain a position feature detection result, i.e., to obtain an installation state result of the positioning pin 4012 and a skew state result of the positioning pin 4012; then, the types of the installation state result and the skew state result are judged, i.e., only when the installation state result is that the positioning pin is installed and the skew state result is that the positioning pin is not skewed, the shape feature of the positioning pin 4012 in the target detection area is detected to obtain a shape detection result. In this way, when the position feature detection detects a defect of the positioning pin, i.e., detects that the positioning pin is not installed or the positioning pin is skewed, the subsequent detection process is stopped, i.e., invalid detection is skipped and a defect alarm is triggered and the machining instruction is stopped immediately, which improves the detection efficiency and effectively avoids subsequent invalid detection.

[0053] In an embodiment, i.e., in step S231, i.e., detecting the position feature of the positioning pin 4012 in the target detection area to obtain a position feature detection result, the following steps are included: S2311: determining a theoretical circular contour of the positioning pin 4012 in the target detection area; S2312: constructing a target circular ring 600 with the center of the theoretical circular contour as the center, the inner ring 602 of the target circular ring 600 being located inside the circle of the theoretical circular contour and the outer ring 601 being located outside the circle of the theoretical circular contour; S2313: detecting the white-black gradient change from the center of the target circular ring 600 outward to obtain a set of edge points; S2314: performing least squares circle fitting according to the set of edge points to obtain a fitting result; S2315: if the fitting result is fitting success, obtaining an installation state result that the positioning pin is installed; S2316: if the fitting result is fitting failure, obtaining an installation state result that the positioning pin is not installed.

[0054] In this embodiment, the theoretical circular contour of the positioning pin 4012 can be determined in the target detection area according to the preset center coordinates and radius coordinates of the positioning pin 4012. The theoretical circular contour represents the ideal geometric position of the positioning pin 4012.

[0055] As an example, refer to Figure 5 and Figure 6After the theoretical circular contour of the locating pin 4012 in the target detection region is determined, concentric inner and outer rings 601 can be drawn based on the center of the theoretical circular contour, thereby completing the construction of the target circular ring 600; the inner ring 602 is located within the theoretical circular contour, i.e., the radius of the inner ring 602 is smaller than the theoretical circular radius, and the outer ring 601 covers outside the theoretical circular contour, i.e., the radius of the outer ring 601 is greater than the theoretical circular radius, to define the edge search range; then, the center of the circular ring is taken as the starting point, the target detection region is scanned along each radial direction, the white-black gradient gradient change, i.e., the maximum gradient change position from light to dark or from dark to light, is detected, and is recorded in the edge point set; then, the edge point set is taken as the fitting data to perform least squares circle fitting to obtain a fitting result; if the fitting result is fitting success, e.g., a circle is fitted according to the fitting data, the installation state result of the locating pin installation is obtained; if the fitting result is fitting failure, e.g., a circle is not fitted according to the fitting data or cannot be fitted, the installation state result of the locating pin non-installation is obtained (as shown in Figure 6 Thus, the anti-noise capability and geometric fitting precision of the installation state detection of the locating pin 4012 can be effectively improved, the misjudgment caused by background interference or edge detection error is effectively avoided, and the automatic and high-precision discrimination of whether the locating pin 4012 is installed is realized.

[0056] In an embodiment, i.e., in step S2314, i.e., in the least squares circle fitting according to the edge point set to obtain a fitting result, the following steps are included: S2314A: removing outliers in the edge point set that deviate from the center by more than a preset threshold to obtain a modified edge point set; S2314B: performing least squares circle fitting according to the modified edge point set to obtain a fitting result.

[0057] As an example, after the edge point set is obtained, the distance of each point in the edge point set from the ring center can be calculated, points with a distance deviating from a preset threshold (e.g., a preset standard radius threshold) by too much are removed to obtain a modified edge point set, thereby eliminating the influence of interference points, making the modified edge point set closer to the true edge, and further making the fitting result obtained by performing least squares circle fitting according to the modified edge point set more consistent with the actual situation, effectively improving the precision and reliability of the locating pin 4012 related parameter detection, and reducing the misjudgment caused by edge detection error.

[0058] In an embodiment, please refer to Figure 7 After step S2315, i.e., if the fitting result is fitting success, the installation state result of the locating pin installation is obtained, the following steps are included: S2317: Obtain the fitting circle parameters of the fitting circle, which is obtained by performing least square circle fitting on the modified edge point set; S2318: Calculate the fitting error of the fitting circle parameters and the preset template circle parameters; S2319: If the fitting error is less than or equal to the preset fitting error, the positioning pin is obtained in the non-skewed skew state result; S23110: If the fitting error is greater than the preset fitting error, the positioning pin is obtained in the skewed skew state result.

[0059] In this embodiment, the fitting circle parameters include the fitting center and the fitting radius, and the preset template circle parameters are the parameters corresponding to the pre-constructed circle module, including the preset standard center and the preset standard radius.

[0060] As an example, after completing the installation state detection and confirming the installation of the positioning pin, the skew state detection of the positioning pin 4012 is entered, which specifically includes: calling the fitting result output in step S2315, that is, calling the fitting circle output in step S2315; then, extracting the fitting center (X1, Y1) and the fitting radius R1 of the fitting circle, and obtaining the preset standard center (X2, Y2) and the preset standard radius R2 of the preset template circle parameters; then, the fitting error of the fitting circle parameters and the preset template circle parameters is calculated, for example, the difference between the fitting center and the preset standard center is calculated, and / or the difference between the fitting radius R1 and the preset standard radius is calculated; when |X1-X2| or |Y1-Y2| is greater than the preset fitting error (for example, 0.05mm), and / or |R1-R2| exceeds the preset fitting error, the skew state result of the positioning pin is obtained in the skewed skew state result (as shown in Figure 7 ). Otherwise, the skew state result of the positioning pin is obtained in the non-skewed skew state result. In this way, by comparing the two core geometric parameters of the fitting center and the fitting radius, it can be quickly and quantitatively identified whether the positioning pin 4012 is skewed in the target hole, avoiding subjective judgment errors.

[0061] In an embodiment, that is, in step S232, the positioning pin 4012 is a cylindrical shaft body provided with a slot along the axial direction, wherein the slot divides the cylindrical shaft body into a plurality of petals, and the shape detection result includes petal loss and petal non-loss of the positioning pin; That is, the shape feature of the positioning pin 4012 in the target detection area is detected to obtain the shape detection result, which includes the following steps: S2321: Extract the petal contour key points of the positioning pin 4012 in the target detection area; S2322: Convert the petal contour key points into key point descriptors; S2323: Match the key point descriptors with the preset key point descriptors to obtain a key point matching result; S2324: If the key point matching result is a match success, a shape detection result of the valve piece without missing is obtained; S2325: If the key point matching result is a match failure, a shape detection result of the valve piece with missing is obtained.

[0062] In this embodiment, the valve piece 700 can include a first valve piece, a second valve piece, a third valve piece, and a fourth valve piece.

[0063] In this embodiment, the preset key point descriptor refers to a feature vector pre-extracted and stored based on the standard positioning pin 4012 (i.e., the valve piece without missing) for comparison with the valve piece of the real-time detected positioning pin 4012.

[0064] As an example, after the position feature detection of the positioning pin 4012 is completed and the position feature detection result is that the positioning pin is not defective, i.e., the positioning pin is installed and the positioning pin is not skewed, the shape feature detection of the positioning pin is entered, which specifically includes: first, the valve piece contour of the positioning pin 4012 in the target detection area can be scanned (for example, 360° full circumferential scanning is performed) to extract the valve piece contour key points of the positioning pin 4012. For example, taking the positioning pin 4012 with four valve pieces as an example, the valve pieces are distributed in a cross shape. When 360° full circumferential scanning is performed on the target detection area, the contour is scanned at an angle no matter the valve piece is rotated to any angle (such as 0°, 90°, 180°, 270° direction), which ensures that the four valve pieces of the positioning pin 4012 can be successfully extracted regardless of the angle at which they are located, and further extracts the valve piece contour key points corresponding to each valve piece, i.e., extracts the valve piece contour key points corresponding to the four valve pieces of the positioning pin 4012, for example, the first valve piece corresponds to the first valve piece contour key point; then, the valve piece contour key points are converted into corresponding key point descriptors, for example, the first valve piece contour key point is converted into the first valve piece contour key point descriptor; further, the first valve piece contour key point descriptor is matched with the corresponding preset key point descriptor in the four valve pieces in turn to obtain the corresponding key point matching result, and the matching process of the subsequent three valve pieces is completed in turn to obtain four key point matching results; if all four key point matching results are match successes (as shown in FIG. 11B), a shape detection result of the positioning pin valve piece without missing is obtained; if any of the four key point matching results is not a match success (i.e., a match failure), a shape detection result of the positioning pin valve piece with missing is obtained. In this way, accurate judgment of the valve piece is achieved, effectively eliminating the subjectivity of manual judgment, eliminating the risk of single valve piece abnormality being misjudged as normal, and improving the reliability of defect identification. Figure 8

[0065] ​Further, any one of the four key point matching results fails to match (i.e., fails to match), including: one key point matching result fails to match, two key point matching results fail to match, three key point matching results fail to match, or four key point matching results fail to match, wherein one key point matching result failing to match represents one missing lobe (as shown in Figure 10 ), two key point matching results failing to match represents two missing lobes (as shown in Figure 9 and Figure 11 ), three key point matching results failing to match represents three missing lobes (as shown in Figure 12 ), or four key point matching results failing to match represents four missing lobes (as shown in Figure 13 ). In this way, single or multiple lobe breakage or missing can be accurately identified, and missed detection caused by incomplete lobe contour under a single view is effectively avoided, and the comprehensiveness of detection is improved.

[0066] In an embodiment, that is, in step S202, that is, in locating the target detection region in the appearance image, the following steps are included: S2021, sequentially performing grayscale processing, histogram equalization processing, and Gaussian filter denoising processing on the appearance image to obtain a preprocessed image; S2022, according to the gray distribution mode of the preprocessed image, using a gray level co-occurrence matrix to locate the positioning pin region; S2023, applying an edge detection algorithm to the positioning pin region to locate the target detection region.

[0067] As another example, after obtaining the appearance image, first, the appearance image can be subjected to grayscale processing to convert the color image into a grayscale image; then, the grayscale image is subjected to histogram equalization processing to improve the image contrast, so that the positioning pin 4012 and the background gray level difference are more obvious, and then Gaussian filter denoising processing is performed to suppress high-frequency noise to obtain a preprocessed image; then, according to the gray distribution mode of the preprocessed image, a gray level co-occurrence matrix (GLCM) is used to preliminarily locate the positioning pin region and the background region. Subsequently, an edge detection algorithm (for example, a Canny edge detection algorithm) is applied to the positioning pin region to extract the edge contour; finally, the rectangular region that best matches the theoretical circular contour size is selected as the final target detection region. In this way, the image region where the positioning pin 4012 is located can be accurately and efficiently located.

[0068] In an embodiment, the number of positioning pins 4012 is related to the size of the positioning pin 4012.

[0069] As an example, after the detection of one positioning pin 4012 is completed, S101-S104 are repeatedly performed on each positioning pin 4012 for fixing the PCB in turn, so that all the positioning pins 4012 are detected one by one, and the subsequent PCB processing is performed only when each positioning pin 4012 on the PCB is normal, thereby further improving the positioning accuracy in the subsequent PCB processing process and reducing the scrap rate of the PCB.

[0070] Embodiment Two In a second aspect, the application also provides a positioning pin defect detection device 300 based on machine vision, which is used in a processing equipment 401, and the processing equipment 401 includes positioning pins 4012 for fixing a PCB. Please refer to Figure 14 The device includes: An acquisition module 301, configured to acquire an appearance image of the positioning pin 4012; A positioning module 302, configured to position a target detection area in the appearance image, the target detection area being an area where the actual position of the positioning pin 4012 in the appearance image is located; A detection module 303, configured to detect the working state of the positioning pin 4012 in the target detection area, and obtain a detection result; A triggering module 304, configured to trigger a corresponding positioning pin defect alarm when the detection result is a positioning pin defect.

[0071] In an embodiment, the detection result includes a position detection result and a shape detection result. The detection module 303 is specifically configured to: Detect the position feature of the positioning pin 4012 in the target detection area, and obtain a position feature detection result, the position feature detection result including an installation state result and a skew state result of the positioning pin 4012; When the installation state result is that the positioning pin is installed and the skew state result is that the positioning pin is not skewed, detect the shape feature of the positioning pin 4012 in the target detection area, and obtain a shape detection result.

[0072] In an embodiment, the detection module 303 is specifically further configured to: Determine a theoretical circular contour of the positioning pin 4012 in the target detection area; Construct a target circular ring 600 with the center of the theoretical circular contour as the center, the inner ring 602 of the target circular ring 600 being located inside the theoretical circular contour and the outer ring 601 being located outside the theoretical circular contour; Detect the white-black gradient change from the center of the target circular ring 600 outward, and obtain a set of edge points; Perform least square circle fitting according to the set of edge points, and obtain a fitting result; If the fitting result is a fitting success, an installation state result of the positioning pin installation is obtained; If the fitting result is a fitting failure, an installation state result of the positioning pin non-installation is obtained.

[0073] In an embodiment, the detection module 303 is specifically further configured to: remove outliers in the edge point set that deviate from the center by more than a preset threshold to obtain a modified edge point set; perform least squares circle fitting on the modified edge point set to obtain a fitting result.

[0074] In an embodiment, the detection module 303 is specifically further configured to: obtain a fitting circle parameter of the fitting circle, the fitting circle being obtained by performing least squares circle fitting on the modified edge point set; calculate a fitting error of the fitting circle parameter and a preset template circle parameter; if the fitting error is less than or equal to a preset fitting error, an oblique state result of the positioning pin non-oblique is obtained; if the fitting error is greater than the preset fitting error, an oblique state result of the positioning pin oblique is obtained.

[0075] In an embodiment, the shape detection result includes petal missing and petal non-missing of the positioning pin; The detection module 303 is specifically further configured to: extract petal contour key points of the positioning pin 4012 in the target detection region; convert the petal contour key points into key point descriptors; match the key point descriptors with preset key point descriptors to obtain a key point matching result; if the key point matching result is a matching success, a shape detection result of the petal non-missing is obtained; if the key point matching result is a matching failure, a shape detection result of the petal missing is obtained.

[0076] In an embodiment, the positioning module 302 is specifically configured to: perform, in sequence, a grayscale processing, a histogram equalization processing and a Gaussian filter denoising processing on the appearance image to obtain a preprocessed image; locate a positioning pin region according to a grayscale distribution mode of the preprocessed image by using a grayscale co-occurrence matrix; apply an edge detection algorithm to the positioning pin region to locate a target detection region.

[0077] It should be noted that the limitation on the positioning pin defect detection device based on machine vision can refer to the limitation on the aforementioned positioning pin defect detection method based on machine vision, and details are not repeated here to avoid repetition.

[0078] Thirdly, this application also provides a machine vision-based positioning pin defect detection system 400, please refer to... Figure 1 and Figure 15 The system includes processing equipment 401 and electronic equipment 500. Processing equipment 401 includes an image acquisition device 4011 and positioning pins 4012 for PCB fixing. Electronic equipment 500 is used for: Acquire an external image of the positioning pin 4012 captured by the image acquisition device 4011; Locate the target detection area in the appearance image. The target detection area is the area where the positioning pin 4012 is actually located in the appearance image. The working status of the positioning pin 4012 within the target detection area is detected, and the detection results are obtained. When the detection result indicates a defect in the locating pin, the corresponding locating pin defect alarm will be triggered.

[0079] In one embodiment, the detection results include position detection results and shape detection results; Electronic device 500 is also used for: The positional features of the positioning pin 4012 within the target detection area are detected to obtain the positional feature detection results, which include the installation status and tilt status of the positioning pin 4012. When the installation status result is that the positioning pin is installed, and the misalignment status result is that the positioning pin is not misaligned, the shape features of the positioning pin 4012 in the target detection area are detected to obtain the shape detection result.

[0080] In one embodiment, the electronic device 500 is further configured to: Determine the theoretical circular contour of the positioning pin 4012 within the target detection area; With the center of the theoretical circular contour as the center, construct the target ring 600. The inner ring 602 of the target ring 600 is located inside the circle of the theoretical circular contour, and the outer ring 601 is located outside the circle of the theoretical circular contour. The white-black gradient change is detected from the center of the target ring 600 outwards to obtain the set of edge points; The least-squares circle fitting is performed based on the set of edge points to obtain the fitting result; If the fitting result is successful, the installation status result of the positioning pin installation is obtained; If the fitting result is a failure, the installation state result of the positioning pin not being installed is obtained.

[0081] In one embodiment, the electronic device 500 is further configured to: Remove outliers from the edge point set whose deviation from the center exceeds a preset threshold to obtain the corrected edge point set; According to the modified edge point set, a least square circle fitting is performed to obtain a fitting result.

[0082] In an embodiment, the electronic device 500 is further configured to: obtain fitting circle parameters of the fitting circle, the fitting circle being obtained according to the modified edge point set by performing the least square circle fitting; calculate a fitting error between the fitting circle parameters and preset template circle parameters; if the fitting error is less than or equal to a preset fitting error, obtain the skew state result of the positioning pin being non-skewed; if the fitting error is greater than the preset fitting error, obtain the skew state result of the positioning pin being skewed.

[0083] In an embodiment, the shape detection result includes a missing petal of the positioning pin and a non-missing petal of the positioning pin. The electronic device 500 is further configured to: extract petal contour key points of the positioning pin 4012 in the target detection region; convert the petal contour key points into key point descriptors; match the key point descriptors with preset key point descriptors to obtain a key point matching result; if the key point matching result is a matching success, obtain the shape detection result of the non-missing petal; if the key point matching result is a matching failure, obtain the shape detection result of the missing petal.

[0084] In an embodiment, the electronic device 500 is further configured to: perform, in sequence, a grayscale processing, a histogram equalization processing and a Gaussian filter denoising processing on the appearance image to obtain a preprocessed image; according to a grayscale distribution mode of the preprocessed image, locate a positioning pin region by using a gray level co-occurrence matrix; apply an edge detection algorithm to the positioning pin region to locate a target detection region.

[0085] It should be noted that the definition of the positioning pin defect detection system based on machine vision can refer to the definition of the positioning pin defect detection method based on machine vision described above, and here will not be repeated.

[0086] In a fourth aspect, the present application further provides an electronic device 500, please refer to Figure 16 , comprising a processor 501 and a memory 502, wherein the memory 501 is used to store computer programs; the processor 502 is used to execute the programs stored in the memory 501 to realize the steps of the positioning pin defect detection method based on machine vision introduced in any embodiment of the present application.

[0087] In a fifth aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method for detecting pin defects based on machine vision according to any one of the embodiments of the present application.

[0088] In the present application, plural means two or more than two.

[0089] In the present application, the terms "first", "second", "third", "fourth" and the like (if any) are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence.

[0090] In the present application, the term "and / or" is only a description of the associated relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " in the present application generally represents an "or" relationship between the front and rear associated objects.

[0091] If not specifically stated, all steps of the present application can be performed in sequence or randomly. For example, the method comprises steps A and B, which means that the method can comprise steps A and B performed in sequence, or steps B and A performed in sequence. For example, it is mentioned that the method can further comprise step C, which means that step C can be added to the method in any order, for example, the method can comprise steps A, B and C, or steps A, C and B, or steps C, A and B, etc.

[0092] The above is only a preferred embodiment of the present application, and is not intended to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A machine vision based pin defect detection method, characterized by, A method for processing equipment comprising positioning pins for PCB fixation, the method comprising: acquiring an appearance image of the positioning pins; locating a target detection area in the appearance image, the target detection area being an area where an actual position of the positioning pins in the appearance image is located; detecting a working state of the positioning pins in the target detection area to obtain a detection result; when the detection result is a positioning pin defect, triggering a corresponding positioning pin defect alarm.

2. The method of claim 1, wherein, The detection result comprises a position detection result and a shape detection result; The detection of the working state of the positioning pins in the target detection area to obtain a detection result comprises: detecting a position feature of the positioning pins in the target detection area to obtain a position feature detection result, the position feature detection result comprising an installation state result and a skew state result of the positioning pins; when the installation state result is that the positioning pins are installed and the skew state result is that the positioning pins are not skewed, detecting a shape feature of the positioning pins in the target detection area to obtain the shape detection result.

3. The method of claim 2, wherein, The detection of the position feature of the positioning pins in the target detection area to obtain the position feature detection result comprises: determining a theoretical circular contour of the positioning pins in the target detection area; constructing a target annulus with the center of the theoretical circular contour as the center, the inner ring of the target annulus being located within the circle of the theoretical circular contour and the outer ring being located outside the circle of the theoretical circular contour; detecting white-black gradient changes from the center of the target annulus to obtain a set of edge points; performing least squares circle fitting according to the set of edge points to obtain a fitting result; if the fitting result is fitting success, obtaining an installation state result that the positioning pins are installed; if the fitting result is fitting failure, obtaining an installation state result that the positioning pins are not installed.

4. The method of claim 3, wherein, The least squares circle fitting according to the set of edge points to obtain a fitting result comprises: rejecting outliers in the set of edge points that deviate from the center by more than a preset threshold to obtain a modified set of edge points; performing least squares circle fitting according to the modified set of edge points to obtain the fitting result.

5. The method according to claim 3 or 4, characterized in that, After the installation state result that the positioning pins are installed is obtained if the fitting result is fitting success, the method further comprises: obtaining fitting circle parameters of a fitting circle, the fitting circle being obtained by least squares circle fitting according to the set of edge points; calculating a fitting error of the fitting circle parameters and preset template circle parameters; if the fitting error is less than or equal to a preset fitting error, obtaining a skew state result that the positioning pins are not skewed; if the fitting error is greater than the preset fitting error, obtaining a skew state result that the positioning pins are skewed.

6. The method of claim 2, wherein, The positioning pins are cylindrical shafts provided with slots in the axial direction, wherein the slots divide the cylindrical shafts into multiple petals, and the shape detection result comprises petal absence and petal absence of the positioning pins. The detection of the shape feature of the positioning pins in the target detection area to obtain the shape detection result comprises: extracting a petal profile key point of the positioning pin in the target detection region; converting the petal profile key point into a key point descriptor; matching the key point descriptor with a preset key point descriptor to obtain a key point matching result; if the key point matching result is a matching success, obtaining a shape detection result of a non-missing petal; if the key point matching result is a matching failure, obtaining a shape detection result of a missing petal.

7. The method of claim 1, wherein, The positioning of the target detection region in the appearance image comprises: performing, on the appearance image, a gray-scale processing, a histogram equalization processing and a Gaussian filter denoising processing in sequence to obtain a preprocessed image; positioning a positioning pin region according to a gray-scale distribution mode of the preprocessed image by using a gray-scale co-occurrence matrix; applying an edge detection algorithm to the positioning pin region to position the target detection region.

8. A machine vision based pin defect detection apparatus for locating defects, characterized by, The device is used for a processing equipment comprising a positioning pin for fixing a PCB, and comprises: an acquisition module configured to acquire an appearance image of the positioning pin; a positioning module configured to position a target detection region in the appearance image, the target detection region being a region where an actual position of the positioning pin in the appearance image is located; a detection module configured to detect a working state of the positioning pin in the target detection region to obtain a detection result; and a triggering module configured to trigger a corresponding positioning pin defect alarm when the detection result is a positioning pin defect.

9. A machine vision based pin defect detection system for locating defects, characterized by, The system comprises a processing equipment and an electronic device, the processing equipment comprising an image acquisition device and a positioning pin for fixing a PCB, and the electronic device is configured to: acquire an appearance image of the positioning pin collected by the image acquisition device; position a target detection region in the appearance image, the target detection region being a region where an actual position of the positioning pin in the appearance image is located; detect a working state of the positioning pin in the target detection region to obtain a detection result; and trigger a corresponding positioning pin defect alarm when the detection result is a positioning pin defect.

10. An electronic device, comprising: The system comprises a processing equipment and an electronic device, the processing equipment comprising an image acquisition device and a positioning pin for fixing a PCB, and the electronic device is configured to: acquire an appearance image of the positioning pin collected by the image acquisition device; position a target detection region in the appearance image, the target detection region being a region where an actual position of the positioning pin in the appearance image is located; 11. A computer readable storage medium, characterized in that, detect a working state of the positioning pin in the target detection region to obtain a detection result; and trigger a corresponding positioning pin defect alarm when the detection result is a positioning pin defect. The system comprises a processing equipment and an electronic device, the processing equipment comprising an image acquisition device and a positioning pin for fixing a PCB, and the electronic device is configured to: acquire an appearance image of the positioning pin collected by the image acquisition device; position a target detection region in the appearance image, the target detection region being a region where an actual position of the positioning pin in the appearance image is located; detect a working state of the positioning pin in the target detection region to obtain a detection result; and trigger a corresponding positioning pin defect alarm when the detection result is a positioning pin defect. The system comprises a processing equipment and an electronic device, the processing equipment comprising an image acquisition device and a positioning pin for fixing a PCB, and the electronic device is configured to: acquire an appearance image of the positioning pin collected by the image acquisition device; position a target detection region in the appearance image, the target detection region being a region where an actual position of the positioning pin in the appearance image is located; detect a working state of the positioning pin in the target detection region to obtain a detection result; and trigger a corresponding positioning pin defect alarm when the detection result is a positioning pin defect. The system comprises a processing equipment and an electronic device, the processing equipment comprising an image acquisition device and a positioning pin for fixing a PCB, and the electronic device is configured to: acquire an appearance image of the positioning pin collected by the image acquisition device; position a target detection region in the appearance image, the target detection region being a region where an actual position of the positioning pin in the appearance image is located; detect a working state of the positioning pin in the target detection region to obtain a detection result; and trigger a corresponding positioning pin defect alarm when the detection result is a positioning pin defect.

Images