PCB drilling quality real-time monitoring system and method based on machine vision

Through machine vision technology and deep learning, the feature vectors and image block feature vectors of PCB drilling images are extracted, combined with classifiers, real-time monitoring of PCB drilling quality is realized, solving the problems of poor consistency and accuracy of traditional manual detection, and improving the Test efficiency and accuracy.

CN119067925AInactive Publication Date: 2024-12-03珠海市卓然电子有限公司
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411085431.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-08
Publication Date
2024-12-03
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Traditional PCB quality control relies on manual visual inspection, which makes it difficult to ensure the consistency and accuracy of the inspection, and it is difficult to monitor the drilling quality in real time.

Method used

A real-time monitoring system for drilling quality based on machine vision is adopted to collect images through cameras, and feature vectors and image block feature vectors are extracted using deep learning technology. Combined with a classifier, we can determine whether there are appearance defects, and realize real-time monitoring.

Benefits of technology

Reduce the probability of missed inspection and missed inspection, promptly detect problems, ensure product quality, and improve the accuracy and efficiency of inspection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119067925A_ABST
    Figure CN119067925A_ABST
Patent Text Reader

Abstract

The invention relates to the field of quality monitoring, and particularly discloses a PCB drilling quality real-time monitoring system and method based on machine vision, and the method comprises the steps: firstly obtaining a PCB drilling monitoring image, then carrying out the feature extraction and correlation analysis of the image through a deep learning technology, and finally judging whether a to-be-detected PCB has appearance defects or not through a classifier. Therefore, the real-time monitoring of the PCB production process is realized, the probability of missing detection and false detection is reduced, problems can be found in time, measures are taken, and the product quality is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of quality monitoring, and more specifically, to a real-time monitoring system and method for PCB drilling quality based on machine vision. Background Art

[0002] With the rapid development of electronic technology, the quality control of printed circuit boards (PCBs), as an important part of electronic equipment, is particularly critical. Traditional PCB quality control methods mainly rely on manual visual inspection, but due to factors such as human eye fatigue and individual differences, it is difficult to ensure the consistency and accuracy of inspection. In recent years, the introduction of machine vision systems has brought revolutionary changes to PCB quality control. Machine vision systems are an advanced technology that automatically identifies and detects target objects by simulating human visual functions. It uses a camera to capture images and extracts feature information from images through image processing algorithms to achieve functions such as object positioning, identification and measurement. Machine vision systems are characterized by high efficiency, accuracy and stability, and are widely used in industrial automation, quality inspection, security monitoring and other fields.

[0003] PCBs may have a variety of appearance defects during the production process, such as scratches, stains, damage, etc. The machine vision system can automatically identify these defects through image acquisition and processing, and locate and classify the defects.

[0004] Therefore, a real-time monitoring system and method for PCB drilling quality based on machine vision is desired. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a real-time monitoring system and method for PCB drilling quality based on machine vision, which first obtains a PCB drilling monitoring image, then uses deep learning technology to perform feature extraction and association analysis, and finally uses a classifier to determine whether the PCB to be inspected has appearance defects, thereby achieving real-time monitoring of the PCB production process, reducing the probability of missed detection and false detection, so as to promptly discover problems and take measures to ensure product quality.

[0006] According to one aspect of the present application, a real-time monitoring system for PCB drilling quality based on machine vision is provided, which includes:

[0007] PCB drilling quality data acquisition module, used to obtain PCB drilling monitoring images collected by a camera;

[0008] A PCB drilling quality data extraction module is used to extract an optimized PCB drilling monitoring associated feature vector and a PCB drilling monitoring image block feature vector from the PCB drilling monitoring image collected by the camera;

[0009] The appearance defect judgment module is used to judge whether there is an appearance defect on the PCB to be inspected based on the optimized PCB drilling monitoring associated feature vector and the PCB drilling monitoring image block feature vector.

[0010] According to another aspect of the present application, a method for real-time monitoring of PCB drilling quality based on machine vision is provided, which includes:

[0011] Obtain PCB drilling monitoring images captured by the camera;

[0012] Extracting optimized PCB drilling monitoring associated feature vectors and PCB drilling monitoring image block feature vectors from the PCB drilling monitoring image captured by the camera;

[0013] Based on the optimized PCB drilling monitoring associated feature vector and the PCB drilling monitoring image block feature vector, it is determined whether the PCB to be inspected has appearance defects.

[0014] Compared with the prior art, the present application provides a real-time monitoring system and method for PCB drilling quality based on machine vision, which first obtains a PCB drilling monitoring image, then uses deep learning technology to perform feature extraction and association analysis, and finally uses a classifier to determine whether there are appearance defects on the PCB to be inspected, thereby realizing real-time monitoring of the PCB production process, reducing the probability of missed detection and false detection, so as to promptly discover problems and take measures to ensure product quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0016] Figure 1 It is a block diagram of a real-time monitoring system for PCB drilling quality based on machine vision according to an embodiment of the present application.

[0017] Figure 2 The present invention is a block diagram of a PCB drilling quality data extraction module in a real-time PCB drilling quality monitoring system based on machine vision according to an embodiment of the present application.

[0018] Figure 3 It is a block diagram of a PCB multi-channel feature extraction unit in a real-time monitoring system of PCB drilling quality based on machine vision according to an embodiment of the present application.

[0019] Figure 4The present invention is a flowchart of a method for real-time monitoring of PCB drilling quality based on machine vision according to an embodiment of the present application.

[0020] Figure 5 is a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0022] Figure 1 FIG. 1 is a block diagram of a real-time monitoring system for PCB drilling quality based on machine vision according to an embodiment of the present application. Figure 1 As shown, according to the real-time monitoring system 100 of PCB drilling quality based on machine vision according to the embodiment of the present application, it includes: a PCB drilling quality data acquisition module 110, which is used to acquire a PCB drilling monitoring image collected by a camera; a PCB drilling quality data extraction module 120, which is used to extract an optimized PCB drilling monitoring associated feature vector and a PCB drilling monitoring image block feature vector from the PCB drilling monitoring image collected by the camera; an appearance defect judgment module 130, which is used to judge whether there is an appearance defect in the PCB to be inspected based on the optimized PCB drilling monitoring associated feature vector and the PCB drilling monitoring image block feature vector.

[0023] In the above-mentioned real-time monitoring system 100 for PCB drilling quality based on machine vision, the PCB drilling quality data acquisition module 110 is used to acquire the PCB drilling monitoring image collected by the camera. It should be understood that with the rapid development of electronic technology, the quality control of printed circuit boards (PCBs) as an important part of electronic equipment has become extremely important. Traditional PCB quality control methods mainly rely on manual visual inspection, but due to factors such as human eye fatigue and individual differences, it is difficult to ensure the consistency and accuracy of detection. In recent years, the introduction of machine vision systems has brought revolutionary changes to PCB quality control. Machine vision systems are an advanced technology that automatically identifies and detects target objects by simulating human visual functions. It uses a camera to capture images, extracts feature information through image processing algorithms, and realizes functions such as object positioning, identification and measurement. Machine vision systems have the characteristics of high efficiency, accuracy, and stability, and are widely used in industrial automation, quality inspection, security monitoring and other fields. In the PCB production process, various appearance defects may occur, such as scratches, stains, and damage. In the technical solution of the present application, by acquiring the PCB drilling monitoring image captured by the camera and combining it with deep learning technology, it is judged whether the PCB to be inspected has appearance defects, so as to reduce the possibility of missed detection and false detection, discover problems in time and take measures to ensure product quality.

[0024] Specifically, as an optical sensor, the camera can convert optical signals into electrical signals and capture visual information of PCB drilling. In the PCB manufacturing process, drilling quality is crucial to the performance and reliability of the entire circuit board. Therefore, by collecting PCB drilling monitoring images through the camera, real-time monitoring of key parameters such as drilling shape, position, and size can be achieved. Through image processing algorithms, operations such as feature extraction and defect detection can be performed on PCB drilling images, thereby achieving evaluation and control of PCB quality. These algorithms can use edge detection, feature extraction, pattern recognition and other methods in image processing technology to perform quantitative and qualitative analysis of drilling in images. By continuously optimizing camera acquisition and image processing technology, the quality control level in the PCB manufacturing process can be further improved to ensure the improvement of product quality and production efficiency.

[0025] In the above-mentioned real-time monitoring system 100 for PCB drilling quality based on machine vision, the PCB drilling quality data extraction module 120 is used to extract optimized PCB drilling monitoring associated feature vectors and PCB drilling monitoring image block feature vectors from the PCB drilling monitoring image collected by the camera. It should be understood that extracting optimized PCB drilling monitoring associated feature vectors can capture key quality-related features in the PCB drilling image. These features may include information such as the diameter, roundness, position, and spacing of the drilling hole. By extracting and analyzing these features, the system can determine whether the drilling meets the specification requirements, thereby realizing the monitoring of PCB quality. The optimized feature vector can be obtained through feature engineering and data processing technology to ensure the accuracy and discrimination of the feature. The purpose of extracting the PCB drilling monitoring image block feature vector is to perform feature extraction and analysis at a finer granularity level. Dividing the PCB drilling image into small blocks and extracting the feature vector of each image block can more comprehensively understand the local features and detail information of the drilling image. This method helps to detect small defects or anomalies in drilling and improve the sensitivity and accuracy of the monitoring system. By extracting and optimizing the associated feature vectors for PCB drilling monitoring and the feature vectors for PCB drilling monitoring image blocks, the monitoring system can achieve comprehensive monitoring and evaluation of PCB quality. These feature vectors can be used as the input of the monitoring system, combined with machine learning algorithms for pattern recognition and anomaly detection, to achieve automated monitoring of PCB drilling quality. This feature vector-based approach not only improves the efficiency and accuracy of the monitoring system, but also provides important technical support for quality control in the PCB manufacturing process.

[0026] Figure 2 FIG. 1 is a block diagram of a PCB drilling quality data extraction module in a real-time PCB drilling quality monitoring system based on machine vision according to an embodiment of the present application. Figure 2 As shown, in a specific embodiment of the present application, the PCB drilling quality data extraction module 120 includes: a drilling monitoring image data preprocessing unit 121, used to perform data preprocessing on the PCB drilling monitoring image collected by the camera to obtain a PCB drilling monitoring multi-channel image; a PCB multi-channel feature extraction unit 122, used to perform multi-channel feature extraction on the PCB drilling monitoring multi-channel image to obtain the optimized PCB drilling monitoring associated feature vector; a drilling monitoring image block feature extraction unit 123, used to perform image block feature extraction on the PCB drilling monitoring image collected by the camera to obtain the PCB drilling monitoring image block feature vector.

[0027] It should be understood that data preprocessing includes image denoising, which removes noise in the image through filtering algorithms to improve the clarity and quality of the image. Secondly, the image is grayed to convert the color image into a grayscale image, simplifying the subsequent processing steps while retaining important grayscale information. Next, image enhancement can be performed to make the image clearer and easier to analyze by contrast enhancement, brightness adjustment and other methods. In addition, data preprocessing may also include image resizing to scale the image to an appropriate size. Edge detection can also be performed to highlight the edge information in the image, which is helpful for drilling location and shape analysis. Finally, it may be necessary to perform image binarization to convert the image into a black and white binary image to facilitate target segmentation and identification. Through these data preprocessing steps, more representative and distinguishable multi-channel images are extracted from the original PCB drilling monitoring image, laying a good foundation for subsequent image processing and analysis. The processed images can be better used for drilling monitoring and quality assessment, improve the accuracy and stability of the monitoring system, and ensure that the PCB drilling quality is effectively monitored and controlled.

[0028] Furthermore, multi-channel feature extraction is performed on the multi-channel image of PCB drilling monitoring to obtain an optimized PCB drilling monitoring associated feature vector. Here, multi-channel feature extraction refers to extracting key information representing image features from images of multiple channels so as to better identify, locate and classify defects of PCB drilling. Among them, multi-channel feature extraction can comprehensively utilize information from different channels to improve the diversity and richness of features. By extracting features from different channels, more comprehensive image information can be captured, which helps to distinguish different types of drilling and defects. Multi-channel feature extraction can enhance the characterization capability of PCB drilling and improve the accuracy and robustness of the system. By combining the feature information of multiple channels, the shape, size, position and other characteristics of the drilling can be better described, so as to more accurately identify and classify the drilling, so as to achieve more accurate monitoring and evaluation of PCB quality.

[0029] Furthermore, image block feature extraction refers to dividing the image into small blocks and then extracting representative features from each image block for better image analysis, recognition and classification. Here, image block feature extraction can help the system better capture local features in the image, thereby improving the accuracy of PCB drilling detection and analysis. By dividing the image into small blocks, the detailed information in the image can be observed more finely, which helps to find subtle changes and defects in drilling.

[0030] In a specific embodiment of the present application, the drilling monitoring image data preprocessing unit 121 includes: extracting a PCB drilling monitoring texture feature map from the PCB drilling monitoring image collected by the camera based on a gray level co-occurrence matrix; and cascading the PCB drilling monitoring image collected by the camera with the PCB drilling monitoring texture feature map to obtain the PCB drilling monitoring multi-channel image.

[0031] It should be understood that the grayscale co-occurrence matrix is ​​a statistical method for describing the texture features of an image. By analyzing the grayscale value relationship between pixels, the texture information of the image can be effectively extracted. Among them, the grayscale co-occurrence matrix can capture the grayscale distribution and texture information of different regions in the image. By analyzing the grayscale value co-occurrence relationship between pixel pairs, statistical information reflecting the texture features of the image, such as contrast, energy, inertia, etc., can be obtained, thereby more comprehensively describing the texture features of the PCB drilling image. The texture features extracted based on the grayscale co-occurrence matrix have strong distinguishing ability and robustness. Since the grayscale co-occurrence matrix can describe the spatial relationship and grayscale distribution between pixels, the extracted texture features can effectively distinguish different types of texture structures and accurately and reliably describe the texture features of PCB drilling. In addition, the texture features extracted by the grayscale co-occurrence matrix have strong anti-noise ability. By statistically analyzing the grayscale co-occurrence relationship between pixel pairs, the noise interference in the image can be suppressed to a certain extent, the stability and reliability of the texture features can be improved, and the texture information of the PCB drilling image can be more accurately reflected.

[0032] Furthermore, the cascaded image and texture feature map can fuse information at different levels to improve the description ability of PCB drilling. Considering that the image itself contains the appearance and shape information of the PCB drilling, and the texture feature map reflects the subtle texture characteristics of the PCB drilling. Cascading these two types of information can comprehensively consider the appearance, texture and structural characteristics of the PCB drilling, thereby more comprehensively describing the characteristics of the PCB drilling.

[0033] Figure 3 FIG. 1 is a block diagram of a PCB multi-channel feature extraction unit in a real-time monitoring system for PCB drilling quality based on machine vision according to an embodiment of the present application. Figure 3As shown, in a specific embodiment of the present application, the PCB multi-channel feature extraction unit 122 includes: a residual dual attention encoding subunit 1221, which is used to pass the PCB drilling monitoring multi-channel image through a residual dual attention mechanism model to obtain a PCB drilling monitoring enhanced multi-channel feature map; a global mean pooling subunit 1222, which is used to perform global mean pooling on each feature matrix along the channel dimension of the PCB drilling monitoring enhanced multi-channel feature map to obtain a PCB drilling monitoring enhanced feature vector; a PCB monitoring enhanced feature encoding subunit 1223, which is used to perform feature encoding on the PCB drilling monitoring enhanced feature vector to obtain the optimized PCB drilling monitoring associated feature vector.

[0034] It should be understood that residual connections can effectively transfer and retain image feature information, which helps to overcome the gradient vanishing and gradient exploding problems in deep neural network training. By introducing residual connections, the model can learn residual mapping, so as to better capture the subtle features and contextual information in the PCB drilling monitoring multi-channel image, and improve the feature representation ability. Among them, the dual attention mechanism can effectively learn the global and local feature relationships, which helps to improve the feature distinction ability and focus. By introducing the attention mechanism, the model can adaptively learn the importance of different positions and channels, so as to focus on the key features in the PCB drilling monitoring multi-channel image, and improve the feature representation and recognition accuracy. Specifically, the PCB drilling monitoring multi-channel image is passed through the spatial attention module of the residual dual attention mechanism model to obtain a spatial attention feature map; the PCB drilling monitoring multi-channel image is passed through the channel attention module of the residual dual attention mechanism model to obtain a channel attention feature map; the spatial attention feature map and the channel attention feature map are cascaded to obtain the PCB drilling monitoring enhanced multi-channel feature map.

[0035] Furthermore, global mean pooling can reduce the dimension of the feature map and reduce the computational complexity. By performing mean pooling on the feature matrix of each channel, the feature information of each channel can be integrated into a single value, thereby effectively reducing the dimension of the feature map, simplifying the subsequent feature processing and calculation process, and improving computational efficiency. In addition, global mean pooling can extract global feature information and enhance the characterization capability of features. By performing mean pooling on the entire feature map, the information of each channel can be fused and the global feature representation can be extracted, thereby more comprehensively describing the features of PCB drilling. This helps to capture the overall feature pattern of PCB drilling and improve the distinguishability and characterization capability of features.

[0036] Furthermore, feature encoding is performed on the enhanced feature vector for PCB drilling monitoring, aiming to further extract and enhance the representation ability of feature representation, so as to better describe the characteristics of PCB drilling and improve the performance and accuracy of the monitoring system. Among them, feature encoding can reduce the data dimension and improve the representation ability of features by learning effective feature representation. By encoding the enhanced feature vector for PCB drilling monitoring, the original feature space can be converted into a more discriminative and expressible feature space, which helps to improve the identification ability and classification accuracy of features. Moreover, feature encoding can help discover potential structures and patterns in data and improve the abstract expression ability of features. By encoding the enhanced feature vector for PCB drilling monitoring, high-order feature representations in the data can be learned, so as to better capture the complex characteristics and associated information of PCB drilling and improve the representation and recognition ability of features.

[0037] In a specific embodiment of the present application, the PCB monitoring enhanced feature encoding subunit 1223 includes: passing the PCB drilling monitoring enhanced feature vector through a one-dimensional convolution layer to obtain a PCB drilling monitoring associated feature vector; based on the PCB drilling monitoring image block feature vector, performing point-by-point feature depth correlation correction on the PCB drilling monitoring associated feature vector to obtain the optimized PCB drilling monitoring associated feature vector.

[0038] It should be understood that the one-dimensional convolution layer can effectively capture local patterns and spatial relationships in the feature sequence. By performing a one-dimensional convolution operation on the feature sequence, local features at different positions in the feature sequence can be identified and extracted, thereby capturing the spatial local correlation between features. This helps to better describe the feature sequence of PCB drilling and improve the representation and distinguishability of features. In addition, the one-dimensional convolution layer can introduce nonlinear transformations to enhance the nonlinear expression of features. By applying a nonlinear activation function to the feature sequence, the one-dimensional convolution layer can introduce nonlinear transformations, allowing the model to learn more complex feature representations, improve the distinguishability and generalization of features, and help improve the performance of the PCB drilling monitoring system. Specifically, each layer of the one-dimensional convolution layer is used to perform the following operations on the input data in the forward pass of the layer: convolution processing is performed on the input data based on the convolution kernel to generate a convolution feature map; mean pooling processing is performed on the convolution feature map to generate a pooling feature map; nonlinear activation is performed on the feature values ​​at each position in the pooling feature map to generate an activation feature map; wherein the output of the last layer of the one-dimensional convolution layer is a PCB drilling monitoring associated feature vector, the input from the second layer to the last layer of the one-dimensional convolution layer is the output of the previous layer, and the input of the one-dimensional convolution layer is the PCB drilling monitoring enhanced feature vector.

[0039] In a specific embodiment of the present application, based on the PCB drilling monitoring image block feature vector, the PCB drilling monitoring associated feature vector is corrected for feature depth correlation point by point to obtain the optimized PCB drilling monitoring associated feature vector, including: calculating the positional subtraction between the PCB drilling monitoring associated feature vector and the PCB drilling monitoring image block feature vector to obtain a phase difference feature vector; dividing the eigenvalues ​​of each position of the phase difference feature vector by two and then taking the absolute value, and then calculating the natural exponential function value to obtain an exponential feature vector; dividing the second hyperparameter by each eigenvalue of the exponential feature vector to obtain a second weighted coefficient vector; multiplying the second weighted coefficient vector by the PCB drilling monitoring image block feature vector by position to obtain a weighted PCB drilling monitoring image block feature vector; multiplying the first predetermined hyperparameter by the PCB drilling monitoring associated feature vector by position to obtain a weighted PCB drilling monitoring associated feature vector, and then adding the first predetermined hyperparameter to the weighted PCB drilling monitoring image block feature vector by position to obtain the optimized PCB drilling monitoring associated feature vector.

[0040] In particular, in the technical solution of the present application, it is considered that both the PCB drilling monitoring associated feature vector and the PCB drilling monitoring image block feature vector are intended to capture important features in the PCB drilling monitoring image. Since they are extracted from the same image data, they may contain some of the same information or features, resulting in duplication or redundancy. The PCB drilling monitoring associated feature vector and the PCB drilling monitoring image block feature vector may involve some common features or feature combinations. These features may affect the extraction of the two feature vectors at the same time, resulting in partial feature overlap or redundancy. The PCB drilling monitoring associated feature vector and the PCB drilling monitoring image block feature vector may be correlated in some aspects. For example, some image block features may be associated with the features of the overall monitoring image, resulting in duplication of some information in the two feature vectors. That is, there may be repeated or redundant parts in the PCB drilling monitoring associated feature vector and the PCB drilling monitoring image block feature vector. In order to improve the model performance and reduce redundant information, it can be considered to process these repeated or redundant parts in the feature fusion process to ensure that the final feature vector can better represent the features of the PCB drilling monitoring image, thereby accurately classifying whether there are defects on the surface to be detected. Based on this, in the technical solution of the present application, based on the PCB drilling monitoring image block feature vector, the PCB drilling monitoring associated feature vector is corrected for point-by-point feature depth correlation.

[0041] Specifically, based on the PCB drilling monitoring image block feature vector, the PCB drilling monitoring associated feature vector is corrected point by point for feature depth correlation to obtain an optimized PCB drilling monitoring associated feature vector, including: based on the PCB drilling monitoring image block feature vector, the PCB drilling monitoring associated feature vector is corrected point by point for feature depth correlation using the following optimization formula to obtain the optimized PCB drilling monitoring associated feature vector;

[0042]

[0043] in, represents the eigenvalue of the ith position of the PCB drilling monitoring associated eigenvector, represents the eigenvalue of the ith position of the PCB drilling monitoring image block feature vector, α and ∈ represent the first predetermined hyperparameter and the second predetermined hyperparameter, ⊙ represents multiplication by position, It means adding by position. Represents the eigenvalue of the i-th position of the optimized PCB drilling monitoring associated eigenvector.

[0044] In order to eliminate the repeated or redundant parts in the PCB drilling monitoring associated feature vector and the PCB drilling monitoring image block feature vector, so as to improve the generalization ability of the PCB drilling monitoring associated feature vector, in the technical solution of the present application, based on the PCB drilling monitoring image block feature vector, the PCB drilling monitoring associated feature vector is corrected for point-by-point feature depth correlation, which simulates the position scattering response depth of the PCB drilling monitoring associated feature vector and the PCB drilling monitoring image block feature vector in the class divergence space by calculating the gradient of the variance between the PCB drilling monitoring associated feature vector and the PCB drilling monitoring image block feature vector, and constructs a depth correlation compensation factor by using the gradient of the variance to perform position-by-position feature depth correlation compensation on the PCB drilling monitoring associated feature vector. In this way, the repeated or redundant parts in the PCB drilling monitoring associated feature vector and the PCB drilling monitoring image block feature vector are suppressed to improve the generalization ability of the PCB drilling monitoring associated feature vector.

[0045] In a specific embodiment of the present application, the drilling monitoring image block feature extraction unit 123 includes: passing the PCB drilling monitoring image through a convolutional neural network model with multi-feature extraction capability to obtain multiple PCB drilling monitoring image block feature maps; splicing the multiple PCB drilling monitoring image block feature maps to obtain a PCB drilling monitoring image block feature map; and performing global mean pooling on each feature matrix of the PCB drilling monitoring image block feature map along the channel dimension to obtain the PCB drilling monitoring image block feature vector.

[0046] It should be understood that obtaining multiple PCB drilling monitoring image block feature maps through a convolutional neural network model can improve the diversity and richness of features. Feature maps at different levels correspond to abstract features at different levels. These feature maps can capture information at different scales and semantic levels, providing richer feature representations for subsequent feature representation and classification tasks. This diversity of feature representation helps to improve the performance and generalization ability of the monitoring system. Specifically, the convolutional neural network model with multi-feature extraction capability includes: a first convolutional layer, a second convolutional layer parallel to the first convolutional layer, and a cascade layer connected to the first convolutional layer and the second convolutional layer, wherein the first convolutional layer uses a one-dimensional convolutional kernel with a first scale, and the second convolutional layer uses a one-dimensional convolutional kernel with a second scale.

[0047] Furthermore, stitching multiple feature maps can effectively integrate feature information at different scales, angles, and contexts. Each feature map may capture different aspects and feature information of the image. By stitching them together, this information can be comprehensively utilized to obtain a more comprehensive and rich feature representation, which helps to improve the description and recognition capabilities of PCB drilling. Among them, stitching multiple feature maps can also help the model better capture the relationship between global and local features. The information extracted by different feature maps may correspond to different spatial ranges and contexts. By stitching them together, the interaction and fusion between global and local features can be promoted, which helps to more comprehensively understand and describe the feature structure of PCB drilling and improve the performance and robustness of the monitoring system.

[0048] Furthermore, global mean pooling of each feature matrix of the PCB drilling monitoring image block feature map along the channel dimension can effectively reduce the feature dimension and convert the complex feature map into a concise feature vector. By performing mean pooling on the feature matrix of each channel, the feature information of each channel can be integrated and compressed to obtain a feature vector that globally describes the entire feature map. This dimensionality reduction method helps to reduce the redundancy of features and improve the stability and generalization ability of features.

[0049] In a specific embodiment of the present application, the appearance defect judgment module 130 is used to judge whether the PCB to be inspected has appearance defects based on the optimized PCB drilling monitoring associated feature vector and the PCB drilling monitoring image block feature vector. It should be understood that through the optimized PCB drilling monitoring associated feature vector and the PCB drilling monitoring image block feature vector, feature information at different levels and angles can be comprehensively utilized to improve the description and recognition capabilities of PCB drilling. Among them, the optimized PCB drilling monitoring associated feature vector can capture the global features and structural information of PCB drilling, while the PCB drilling monitoring image block feature vector can provide a more detailed local feature description. By combining these two feature vectors, a more comprehensive and accurate PCB drilling feature representation can be obtained, which helps to more accurately judge whether there are appearance defects.

[0050] In a specific embodiment of the present application, the appearance defect judgment module 130 includes: fusing the optimized PCB drilling monitoring associated feature vector with the PCB drilling monitoring image block feature vector to obtain a PCB drilling monitoring image classification feature vector; passing the PCB drilling monitoring image classification feature vector through a classifier to obtain a classification result, and the classification result is used to indicate whether the PCB to be inspected has an appearance defect.

[0051] It should be understood that the fusion optimization of PCB drilling monitoring associated feature vectors and PCB drilling monitoring image block feature vectors can optimize the feature representation and extraction process, and improve the feature discrimination and classification performance. By fusing feature information at different levels and angles, the appearance characteristics of PCB drilling can be better described, so that the classification model can more accurately distinguish different categories of PCB drilling images, and improve the classification accuracy and performance of the system.

[0052] Furthermore, by inputting the extracted PCB drilling monitoring image feature vector into the classifier, the learning ability and pattern recognition ability of the classifier can be used to classify different categories of PCB drilling. The classifier can automatically classify the PCB to be inspected according to the input feature vector and the pre-trained model to determine whether it has appearance defects. This machine learning-based method can realize the automatic identification of PCB drilling appearance defects based on the existing data and feature vectors, thereby improving the efficiency and accuracy of detection. The classification results can intuitively reflect the quality status of the PCB to be inspected, helping operators to promptly discover and deal with possible appearance defects. By using the classification results to indicate whether the PCB to be inspected has appearance defects, real-time monitoring and quality control of the PCB production process can be achieved, and existing problems can be discovered and eliminated in a timely manner, thereby improving product quality and production efficiency. This feedback mechanism based on classification results helps enterprises prevent and solve potential quality problems in advance, and ensure product quality and customer satisfaction.

[0053] In summary, the embodiment of the present application first obtains the PCB drilling monitoring image, then uses deep learning technology to perform feature extraction and association analysis, and finally uses a classifier to determine whether the PCB to be inspected has appearance defects, thereby realizing real-time monitoring of the PCB production process, reducing the probability of missed detection and false detection, so as to promptly discover problems and take measures to ensure product quality.

[0054] As described above, the real-time monitoring system 100 for PCB drilling quality based on machine vision according to the embodiment of the present application can be implemented in various terminal devices. In one example, the real-time monitoring system 100 for PCB drilling quality based on machine vision can be integrated into the terminal device as a software module and / or a hardware module. For example, the real-time monitoring system 100 for PCB drilling quality based on machine vision can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the real-time monitoring system 100 for PCB drilling quality based on machine vision can also be one of the many hardware modules of the terminal device.

[0055] Alternatively, in another example, the machine vision-based PCB drilling quality real-time monitoring system 100 and the terminal device may also be separate devices, and the machine vision-based PCB drilling quality real-time monitoring system 100 may be connected to the terminal device via a wired and / or wireless network, and transmit interactive information in accordance with an agreed data format.

[0056] Figure 4 FIG. 1 is a flow chart of a method for real-time monitoring of PCB drilling quality based on machine vision according to an embodiment of the present application. Figure 4 As shown, according to the real-time monitoring method of PCB drilling quality based on machine vision according to the embodiment of the present application, it includes: S110, acquiring a PCB drilling monitoring image collected by a camera; S120, extracting an optimized PCB drilling monitoring associated feature vector and a PCB drilling monitoring image block feature vector from the PCB drilling monitoring image collected by the camera; S130, judging whether the PCB to be inspected has appearance defects based on the optimized PCB drilling monitoring associated feature vector and the PCB drilling monitoring image block feature vector.

[0057] Here, those skilled in the art can understand that the specific operations of each step in the above-mentioned method for real-time monitoring of PCB drilling quality based on machine vision have been referred to above. Figures 1 to 3 The description of the real-time monitoring system for PCB drilling quality based on machine vision has been introduced in detail, and therefore, its repeated description will be omitted.

[0058] Below, reference Figure 5To describe an electronic device according to an embodiment of the present application.

[0059] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned real-time monitoring method of PCB drilling quality based on machine vision when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0060] See also Figure 5 , Figure 5 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0061] The processor 11 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application; the memory 12 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM).

[0062] The memory 12 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 12, and the processor 11 is called to execute the real-time monitoring method of PCB drilling quality based on machine vision in the embodiment of this application;

[0063] Input / output interface 13, used to implement information input and output;

[0064] The communication interface 14 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WI FI, Bluetooth, etc.);

[0065] A bus 15 that transmits information between the various components of the device (e.g., the processor 11, the memory 12, the input / output interface 13, and the communication interface 14);

[0066] The processor 11 , the memory 12 , the input / output interface 13 and the communication interface 14 are connected to each other in communication within the device via a bus 15 .

[0067] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium. The storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned real-time monitoring method of PCB drilling quality based on machine vision is implemented.

[0068] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0069] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0070] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0071] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0072] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0073] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0074] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0075] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0076] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0077] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method of each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-On lyMemory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs. The preferred embodiment of the embodiment of the present application is described above with reference to the accompanying drawings, and the scope of rights of the embodiment of the present application is not limited thereto. Any modifications, equivalent substitutions and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of rights of the embodiments of the present application.

Claims

1. A real-time monitoring system for PCB drilling quality based on machine vision, characterized in that: include: PCB drilling quality data acquisition module, used to obtain PCB drilling monitoring images collected by a camera; A PCB drilling quality data extraction module is used to extract an optimized PCB drilling monitoring associated feature vector and a PCB drilling monitoring image block feature vector from the PCB drilling monitoring image collected by the camera; The appearance defect judgment module is used to judge whether there is an appearance defect on the PCB to be inspected based on the optimized PCB drilling monitoring associated feature vector and the PCB drilling monitoring image block feature vector.

2. The real-time monitoring system for PCB drilling quality based on machine vision according to claim 1 is characterized in that: The PCB drilling quality data extraction module includes: A drilling monitoring image data preprocessing unit, used for performing data preprocessing on the PCB drilling monitoring image collected by the camera to obtain a PCB drilling monitoring multi-channel image; A PCB multi-channel feature extraction unit, used for performing multi-channel feature extraction on the PCB drilling monitoring multi-channel image to obtain the optimized PCB drilling monitoring associated feature vector; The drilling monitoring image block feature extraction unit is used to extract image block features from the PCB drilling monitoring image collected by the camera to obtain the PCB drilling monitoring image block feature vector.

3. The real-time monitoring system for PCB drilling quality based on machine vision according to claim 2 is characterized in that: The drilling monitoring image data preprocessing unit comprises: Extracting a PCB drilling monitoring texture feature map from the PCB drilling monitoring image acquired by the camera based on a gray level co-occurrence matrix; The PCB drilling monitoring image captured by the camera is cascaded with the PCB drilling monitoring texture feature map to obtain the PCB drilling monitoring multi-channel image.

4. The real-time monitoring system for PCB drilling quality based on machine vision according to claim 3 is characterized in that: The PCB multi-channel feature extraction unit comprises: A residual dual attention encoding subunit, used for passing the PCB drilling monitoring multi-channel image through a residual dual attention mechanism model to obtain a PCB drilling monitoring enhanced multi-channel feature map; A global mean pooling subunit, used for performing global mean pooling on each feature matrix along the channel dimension of the PCB drilling monitoring enhanced multi-channel feature map to obtain a PCB drilling monitoring enhanced feature vector; The PCB monitoring enhanced feature encoding subunit is used to perform feature encoding on the PCB drilling monitoring enhanced feature vector to obtain the optimized PCB drilling monitoring associated feature vector.

5. The real-time monitoring system for PCB drilling quality based on machine vision according to claim 4 is characterized in that: The PCB monitoring enhanced feature encoding subunit comprises: Passing the PCB drilling monitoring enhanced feature vector through a one-dimensional convolutional layer to obtain a PCB drilling monitoring associated feature vector; Based on the PCB drilling monitoring image block feature vector, the PCB drilling monitoring associated feature vector is corrected for point-by-point feature depth correlation to obtain the optimized PCB drilling monitoring associated feature vector.

6. The real-time monitoring system for PCB drilling quality based on machine vision according to claim 5 is characterized in that: Based on the PCB drilling monitoring image block feature vector, performing point-by-point feature depth correlation correction on the PCB drilling monitoring associated feature vector to obtain the optimized PCB drilling monitoring associated feature vector, including: Calculate the positional subtraction between the PCB drilling monitoring associated feature vector and the PCB drilling monitoring image block feature vector to obtain a phase difference feature vector; Dividing the eigenvalues ​​of each position of the phase difference eigenvector by two and taking the absolute value, and then calculating the natural exponential function value to obtain an exponential eigenvector; Dividing a second hyperparameter by each eigenvalue of the exponentially modified eigenvector to obtain a second weighting coefficient vector; Multiplying the second weighting coefficient vector by the PCB drilling monitoring image block feature vector by position to obtain a weighted PCB drilling monitoring image block feature vector; The first predetermined hyperparameter is multiplied by the PCB drilling monitoring associated feature vector by position to obtain a weighted PCB drilling monitoring associated feature vector, and then the weighted PCB drilling monitoring image block feature vector is added by position to obtain the optimized PCB drilling monitoring associated feature vector.

7. The real-time monitoring system for PCB drilling quality based on machine vision according to claim 6 is characterized in that: The drilling monitoring image block feature extraction unit comprises: The PCB drilling monitoring image is passed through a convolutional neural network model with multi-feature extraction capability to obtain a plurality of PCB drilling monitoring image block feature maps; splicing the multiple PCB drilling monitoring image block feature maps to obtain a PCB drilling monitoring image block feature map; Each feature matrix of the PCB drilling monitoring image block feature map along the channel dimension is globally mean pooled to obtain the PCB drilling monitoring image block feature vector.

8. The real-time monitoring system for PCB drilling quality based on machine vision according to claim 7 is characterized in that: The appearance defect judgment module includes: Fusion of the optimized PCB drilling monitoring associated feature vector and the PCB drilling monitoring image block feature vector to obtain a PCB drilling monitoring image classification feature vector; The PCB drilling monitoring image classification feature vector is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether the PCB to be inspected has appearance defects.

9. A real-time monitoring method for PCB drilling quality based on machine vision, characterized in that: include: Obtain PCB drilling monitoring images captured by the camera; Extracting optimized PCB drilling monitoring associated feature vectors and PCB drilling monitoring image block feature vectors from the PCB drilling monitoring image captured by the camera; Based on the optimized PCB drilling monitoring associated feature vector and the PCB drilling monitoring image block feature vector, it is determined whether the PCB to be inspected has appearance defects.

10. The method for real-time monitoring of PCB drilling quality based on machine vision according to claim 9, characterized in that: Extracting and optimizing the PCB drilling monitoring associated feature vector and the PCB drilling monitoring image block feature vector from the PCB drilling monitoring image collected by the camera includes: Performing data preprocessing on the PCB drilling monitoring image acquired by the camera to obtain a PCB drilling monitoring multi-channel image; Performing multi-channel feature extraction on the PCB drilling monitoring multi-channel image to obtain the optimized PCB drilling monitoring associated feature vector; Image block feature extraction is performed on the PCB drilling monitoring image captured by the camera to obtain the PCB drilling monitoring image block feature vector.

Citation Information

Cited By

  • Valve body tapping calibration method and system based on machine vision

    CN119658035A

  • A valve body tapping calibration method and system based on machine vision

    CN119658035B