PCB board defect category intelligent detection method, device, equipment and storage medium
Through intelligent detection methods and multi-level detection models, the defect types on the PCB board are identified, and the problems of difficulty and cost in the existing technology are solved, and fast and accurate defect identification and report generation are achieved. It is suitable for PCB boards of different specifications. .
Patent Information
- Application Number
- CN202410573325.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-10
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-05-10
AI Technical Summary
There are many types of PCB board defects, and different defect types are difficult to detect and distinguish. The existing detection methods are costly, not sensitive enough, or cannot detect defects in the welding process in real time.
Using intelligent detection methods, defect positioning and classification are obtained by obtaining the original image data of the PCB board, and multi-level detection models (including gradient hoisting machines, support vector machines and neural network models) are used to identify plug-in hole types and non-through holes containing metal types. The defects of the hole size error type, the hole collapse type, the hole wall type, the hole exposed copper type and the green oil hole type are defective, improving the accuracy and efficiency of detection.
It realizes the rapid and accurate identification of defect types on PCB boards, reduces detection costs, improves detection efficiency and accuracy, adapts to different types and specifications of PCB board inspections, reduces human intervention and resource consumption, and generates intuitive detection reports.
Smart Images

Figure CN118229669B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of electronic circuit technology, and in particular to a method, device, equipment and storage medium for intelligent detection of PCB board defect categories. Background Art
[0002] In the electronics manufacturing industry, PCB (printed circuit board) boards are one of the basic components of electronic devices, and their quality directly affects the performance and stability of the entire electronic device. However, during the manufacturing process of PCB boards, various defects will inevitably occur due to the influence of various factors. These defects may not only cause the performance of electronic devices to deteriorate, but may even cause equipment failure. Therefore, accurate classification and detection of PCB board defects is the key to ensuring product quality and production efficiency.
[0003] Among PCB board defects, hole defects are the most difficult to identify and distinguish. For the detection of PCB board defects, electrical performance testing, X-ray detection, thermal imaging detection and mechanical detection technology are usually used. Electrical performance testing mainly uses tools such as multimeters or oscilloscopes to check the conductivity between solder joints to ensure that the circuit connection after welding is normal. At the same time, the parameters of the circuit, such as resistance, capacitance, inductance, etc., can also be tested to ensure that it meets the design requirements. However, this method may not be sensitive enough for some subtle defects and is difficult to detect. X-ray detection is mainly used to observe the internal situation of welding. X-rays penetrate the solder layer to show the internal structure of the welded joint, thereby discovering potential welding defects. However, the cost of X-ray detection is high and there is a certain radiation hazard to the human body, so special care should be taken when using it. Thermal imaging detection observes the temperature changes of the PCB board during the heating process through a thermal imager to detect the heat distribution of the welded joint. This method can detect uneven heat distribution caused by poor welding, but it may also be affected by ambient temperature and other heat sources. Mechanical testing mainly judges the welding quality by testing the mechanical properties of the welded parts, such as tensile strength and yield strength. However, this method can only evaluate completed welds and cannot detect defects in the welding process in real time.
[0004] In summary, there are many types of PCB board defects, and different types of defects are difficult to detect and distinguish. Therefore, developing accurate and efficient PCB board defect detection technology and methods is of great significance to improving product quality and production efficiency. Summary of the invention
[0005] In view of this, the present invention provides a method, device, equipment and storage medium for intelligent detection of PCB board defect categories to solve the problem that there are many types of PCB board defects and different defect types are difficult to detect and distinguish.
[0006] In a first aspect, the present invention provides a method for intelligently detecting defect categories of a PCB board, the method comprising:
[0007] Obtaining original image data of the target PCB board to be inspected, locating defects in the original image data, and obtaining defect image data;
[0008] Input the defect image data into the trained first detection model to obtain the first detection result, which is one of the first defect type, the second defect type, the third defect type, the fourth defect type, the fifth defect type, the sixth defect type and the seventh defect type, wherein the first detection model is obtained by training the marked defect image data, the first defect type represents the plugging hole type defect; the second defect type represents the metal-containing type defect of the non-through hole; the third defect type represents the hole size error type defect; the fourth defect type represents the hole collapse type defect; the fifth defect type represents the hole wall roughness type defect; the sixth defect type represents the copper exposed type defect in the hole; the seventh defect type represents the green oil into the hole type defect;
[0009] If the first detection result is the first preset detection result, the defect image data and the first detection result are input into the trained second detection model to obtain the second detection result, the first preset detection result is the fifth defect type or the sixth defect type, and the second detection model is trained by the defect image data of the marked fifth defect type and the sixth defect type and the first detection result;
[0010] If the first detection result is the second preset detection result, the defect image data and the first detection result are input into a trained third detection model to obtain a second detection result, the first preset detection result is a third defect type or a fourth defect type, and the third detection model is trained by the defect image data of the marked third defect type and the fourth defect type and the first detection result;
[0011] If the first detection result is the third preset detection result, the first detection result is used as the second detection result, wherein the third preset detection result is the first defect type, the second defect type, or the seventh defect type;
[0012] The second test result is used as the final test result, and a test report is generated based on the final test result.
[0013] The intelligent detection method for PCB board defect categories provided by the present invention, firstly, by acquiring the original image data of the target PCB board to be detected and performing defect location, the defect position on the PCB board can be accurately and quickly located, which provides an important basis for subsequent defect type identification. This defect location method improves the detection efficiency, reduces the need for human intervention, and thus reduces the detection cost. Secondly, the present invention uses a trained first detection model to classify the defect image data, and can identify seven different defect types. This classification method not only improves the accuracy of detection, but also can cover various defect situations that may occur on the PCB board. At the same time, the first detection model is obtained by training with marked defect image data, which ensures the generalization ability and robustness of the model, so that it can adapt to the detection of PCB boards of different types and specifications. Then, for specific defect types, the present invention also designs a second detection model and a third detection model. When the first detection result is the fifth defect type or the sixth defect type, the second detection model will be further used for detailed identification; when the first detection result is the third defect type or the fourth defect type, the third detection model will be used for further judgment. This hierarchical detection method can more accurately identify specific types of defects, and improve the accuracy and reliability of detection. In addition, when the first detection result is the first defect type, the second defect type or the seventh defect type, the first detection result is directly used as the second detection result, which avoids unnecessary calculation and resource consumption and improves the detection efficiency. Finally, a test report is generated based on the final test result, so that the test result can be intuitively presented to relevant personnel, which is convenient for them to perform subsequent analysis and processing. This automated report generation method reduces manual operation and improves work efficiency. In summary, the PCB board defect category intelligent detection method of the present invention has shown significant technical effects in terms of detection efficiency, accuracy, reliability and ease of use, which is of great significance for improving the production quality of PCB boards and reducing production costs.
[0014] In an optional implementation, original image data of a target PCB board to be inspected is obtained, and defect positioning is performed on the original image data to obtain defect image data, including:
[0015] Obtain the original image data of the target PCB board to be inspected;
[0016] Preprocessing the original image data to obtain preprocessed original image data;
[0017] The edge detection method is used to perform edge detection on the preprocessed original image data to obtain the hole edge data;
[0018] Perform morphological operations on the hole edge data to obtain hole edge refinement data;
[0019] The region growing technology is used to segment the hole edge refinement data into regions to obtain defect image data of each defect region.
[0020] In an optional implementation, preprocessing the original image data to obtain preprocessed original image data includes:
[0021] Performing grayscale processing on the original image data to obtain first original image data;
[0022] Performing filtering processing on the first original image data based on a preset frequency band to obtain second original image data;
[0023] Performing contrast adjustment on the second original image data to obtain third original image data;
[0024] The third original image data is converted into a binary image based on a preset threshold value to obtain preprocessed original image data.
[0025] In an optional implementation, the first detection model is a gradient boosting machine model, and the defect image data is input into the trained first detection model to obtain the first detection result, including:
[0026] Extract features from defect image data to obtain geometric features, texture features and color features of the defect image data. The geometric features include the area, perimeter and circularity of each defect area. The texture features include the gray level co-occurrence matrix and the local binary pattern histogram. The color features include the color histogram and the color moment.
[0027] Iterative processing is performed based on geometric features, texture features and color features using the first detection model;
[0028] When the preset condition is reached, the iteration is stopped and the classification result is output as the first detection result.
[0029] In an optional implementation, the second detection model is a support vector machine model. If the first detection result is the first preset detection result, the defect image data and the first detection result are input into the trained second detection model to obtain the second detection result, including:
[0030] Acquire texture features and color features of defect image data;
[0031] Performing feature encoding on the first detection result to obtain a first detection result feature;
[0032] Splicing the texture feature, the color feature and the first detection result feature to obtain a first splicing feature;
[0033] Calculate a fifth defect type score and a sixth defect type score based on the first splicing feature using the second detection model, wherein the fifth defect type score is a score of the target PCB board to be detected belonging to the fifth defect type, and the sixth defect type score is a score of the target PCB board to be detected belonging to the sixth defect type;
[0034] The second detection model is used to make a classification decision based on the fifth defect type score and the sixth defect type score to obtain a second detection result.
[0035] In an optional implementation, the third detection model is a neural network model. If the first detection result is the second preset detection result, the defect image data and the first detection result are input into the trained third detection model to obtain the second detection result, including:
[0036] Acquire texture features and geometric features of defect image data;
[0037] Performing feature encoding on the first detection result to obtain a first detection result feature;
[0038] Splicing the texture feature, the geometric feature and the first detection result feature to obtain a second splicing feature;
[0039] Using the hidden layer of the third detection model, a nonlinear transformation is performed on the second splicing feature to obtain a third splicing feature;
[0040] The third defect type probability value and the fourth defect type probability value are calculated using the classifier of the third detection model, wherein the third defect type probability value is the probability value that the target PCB board to be detected belongs to the third defect type, and the fourth defect type score is the probability value that the target PCB board to be detected belongs to the fourth defect type;
[0041] The third detection model is used to make a classification decision based on the third defect type possibility value and the fourth defect type possibility value to obtain a third detection result.
[0042] In an optional implementation, taking the second test result as the final test result, and generating a test report based on the final test result, including:
[0043] When the second inspection result is the third defect type or the fifth defect type, a rework inspection report is generated, and the rework inspection report includes a final inspection result and defect image data;
[0044] When the second inspection result is the first defect type, the second defect type, the fourth defect type, the sixth defect type or the seventh defect type, a scrap inspection report is generated, and the scrap inspection report includes the final inspection result.
[0045] In a second aspect, the present invention provides a PCB board defect category intelligent detection device, the device comprising:
[0046] The defect location module is used to obtain the original image data of the target PCB board to be inspected, locate the defects on the original image data, and obtain the defect image data;
[0047] A first detection module is used to input defect image data into a trained first detection model to obtain a first detection result, where the first detection result is one of a first defect type, a second defect type, a third defect type, a fourth defect type, a fifth defect type, a sixth defect type, and a seventh defect type, wherein the first detection model is obtained by training with labeled defect image data, the first defect type characterizes a plugging hole defect; the second defect type characterizes a non-through hole containing metal defect; the third defect type characterizes an aperture size error defect; the fourth defect type characterizes a collapsed hole defect; the fifth defect type characterizes a hole wall roughness defect; the sixth defect type characterizes a copper exposed hole defect; and the seventh defect type characterizes a green oil entering the hole defect;
[0048] a second detection module, for inputting the defect image data and the first detection result into a trained second detection model to obtain a second detection result if the first detection result is a first preset detection result, the first preset detection result is a fifth defect type or a sixth defect type, and the second detection model is trained by the defect image data of the fifth defect type and the sixth defect type that are marked and the first detection result;
[0049] A third detection module, for inputting the defect image data and the first detection result into a trained third detection model to obtain a second detection result if the first detection result is a second preset detection result, the first preset detection result is a third defect type or a fourth defect type, and the third detection model is trained by the defect image data of the marked third defect type and fourth defect type and the first detection result;
[0050] a fourth detection module, configured to use the first detection result as the second detection result if the first detection result is a third preset detection result, wherein the third preset detection result is the first defect type, the second defect type, or the seventh defect type;
[0051] The report generating module is used to use the second detection result as the final detection result and generate a detection report based on the final detection result.
[0052] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the intelligent detection method for PCB board defect categories of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.
[0053] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the intelligent detection method for PCB board defect categories of the above-mentioned first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0055] Figure 1 1 is a flow chart of a method for intelligently detecting defect categories of a PCB board according to an embodiment of the present invention;
[0056] Figure 2 is a flow chart of another PCB board defect category intelligent detection method according to an embodiment of the present invention;
[0057] Figure 3 is a structural block diagram of a PCB board defect category intelligent detection device according to an embodiment of the present invention;
[0058] Figure 4 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0059] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0060] In the electronics manufacturing industry, PCB (printed circuit board) boards are one of the basic components of electronic devices, and their quality directly affects the performance and stability of the entire electronic device. However, during the manufacturing process of PCB boards, various defects will inevitably occur due to the influence of various factors. These defects may not only cause the performance of electronic devices to deteriorate, but may even cause equipment failure. Therefore, accurate classification and detection of PCB board defects is the key to ensuring product quality and production efficiency.
[0061] Among PCB board defects, hole defects are the most difficult to identify and distinguish. For the detection of PCB board defects, electrical performance testing, X-ray detection, thermal imaging detection and mechanical detection technology are usually used. Electrical performance testing mainly uses tools such as multimeters or oscilloscopes to check the conductivity between solder joints to ensure that the circuit connection after welding is normal. At the same time, the parameters of the circuit, such as resistance, capacitance, inductance, etc., can also be tested to ensure that it meets the design requirements. However, this method may not be sensitive enough for some subtle defects and is difficult to detect. X-ray detection is mainly used to observe the internal situation of welding. X-rays penetrate the solder layer to show the internal structure of the welded joint, thereby discovering potential welding defects. However, the cost of X-ray detection is high and there is a certain radiation hazard to the human body, so special care should be taken when using it. Thermal imaging detection observes the temperature changes of the PCB board during the heating process through a thermal imager to detect the heat distribution of the welded joint. This method can detect uneven heat distribution caused by poor welding, but it may also be affected by ambient temperature and other heat sources. Mechanical testing mainly judges the welding quality by testing the mechanical properties of the welded parts, such as tensile strength and yield strength. However, this method can only evaluate completed welds and cannot detect defects in the welding process in real time.
[0062] In summary, there are many types of PCB board defects, and different types of defects are difficult to detect and distinguish. Therefore, developing accurate and efficient PCB board defect detection technology and methods is of great significance to improving product quality and production efficiency.
[0063] According to an embodiment of the present invention, an embodiment of a method for intelligently detecting defect categories of a PCB board is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0064] In this embodiment, a method for intelligently detecting defect categories of a PCB board is provided. Figure 1 FIG. 1 is a flow chart of a method for intelligently detecting PCB board defect categories according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:
[0065] Step S101, obtaining original image data of a target PCB board to be inspected, and performing defect location on the original image data to obtain defect image data.
[0066] Specifically, use a high-resolution industrial camera or scanner to shoot or scan the target PCB board to ensure that the image is clear and detailed, and save the acquired image data in an appropriate format on a computer or server for subsequent processing. Perform necessary preprocessing operations on the original image to improve image quality and improve the accuracy of defect detection. Combine traditional machine learning methods or deep learning methods for defect location. These methods can automatically extract features from the image and identify potential defect locations. According to the results of defect location, the area containing the defect is cropped from the original image to obtain defect image data.
[0067] Step S102, input the defect image data into the trained first detection model to obtain the first detection result, the first detection result is one of the first defect type, the second defect type, the third defect type, the fourth defect type, the fifth defect type, the sixth defect type and the seventh defect type, wherein the first detection model is obtained by training with the marked defect image data, the first defect type characterizes the plugging hole type defect; the second defect type characterizes the non-through hole containing metal type defect; the third defect type characterizes the aperture size error type defect; the fourth defect type characterizes the collapsed hole type defect; the fifth defect type characterizes the rough hole wall type defect; the sixth defect type characterizes the copper exposed in the hole type defect; the seventh defect type characterizes the green oil into the hole type defect.
[0068] Specifically, the plugging type defect is a common type, which is mainly caused by the failure of ink or other materials to completely fill the vias during the manufacturing process, resulting in voids inside the vias. This defect may affect the electrical connection performance of the circuit board. The non-through hole metal-containing type defect refers to the presence of metal residues inside the vias that have not been completely removed, which may be caused by improper drilling or etching processes. This defect may cause circuit short circuits or unstable electrical performance. The hole size error type defect refers to the diameter of the via not meeting the design requirements, which may be caused by insufficient precision or improper operation of the drilling equipment. This defect may affect the installation of components on the circuit board and the reliability of electrical connections. The hole collapse type defect refers to the cracking or falling off of the edge of the via, which may be caused by poor board quality or improper drilling parameter settings. The hole collapse not only affects the appearance quality of the circuit board, but also may damage the integrity of the electrical connection. There is a certain difficulty in distinguishing between the hole wall roughness type defect and the hole exposed copper type defect. The hole wall roughness refers to the uneven surface of the inner wall of the via, while the hole exposed copper refers to the copper layer inside the via being exposed. Both defects may lead to the degradation of the electrical performance of the circuit board, but their causes and manifestations are different. Rough hole walls may be caused by improper etching process or problems with the board itself, while exposed copper in the hole may be caused by the failure of solder mask ink to completely cover the copper layer. In addition, it is difficult to distinguish between incorrect aperture size and broken holes. In actual inspection, if the aperture change exceeds the allowable range and is accompanied by cracking or falling off of the edge, it is necessary to carefully analyze which defect it is. The aperture size error may be just a simple size deviation, while the broken hole involves the destruction of the integrity of the hole structure. Inputting the defect image data into the trained first detection model to obtain the first detection result is a key link, involving the application of the model and the extraction of the results. Ensure that the first detection model has been fully trained and has achieved the expected detection accuracy and performance, use a large amount of labeled defect image data to train the model, and optimize its performance by adjusting the parameters and structure of the model. Load the trained first detection model into a suitable computing environment, such as a deep learning framework, for subsequent reasoning and detection. Before the defect image data is input into the first detection model, a preprocessing operation is required to ensure that the format and range of the input data are consistent with the data used in model training. Preprocessing operations include image resizing, normalization, data enhancement, etc. to improve the generalization ability and detection accuracy of the model. The preprocessed defect image data is passed as input to the first detection model, the image data is converted into a format acceptable to the model, and it is input into the input layer of the model. After receiving the defect image data, the first detection model performs forward propagation calculations, extracts features from the image, and classifies and locates the defects based on the learned knowledge. The first detection model outputs one or more detection results, which may include information such as the type, location, and size of the defect. The first detection result is extracted from the output of the model.
[0069] Step S103, if the first detection result is the first preset detection result, the defect image data and the first detection result are input into the trained second detection model to obtain the second detection result. The first preset detection result is the fifth defect type or the sixth defect type. The second detection model is trained by the marked defect image data of the fifth defect type and the sixth defect type and the first detection result.
[0070] Specifically, check the result output by the first detection model to confirm whether it is the first preset detection result. In this example, the first preset detection result refers to a rough hole wall defect or a copper exposed hole defect. If the first detection result is indeed the fifth or sixth defect type, then prepare to input the defect image data and the first detection result into the second detection model. The second detection model is a model specially trained and optimized for the fifth and sixth defect types. It uses the marked defect image data of the fifth defect type and the sixth defect type, as well as the output result of the first detection model as training data to train the model. Before inputting the second detection model, the defect image data and the first detection result are preprocessed to ensure that the data format, resolution and range meet the input requirements of the second detection model. For the first detection result, encode it into a specific format or vector so that it can be used as one of the inputs of the second detection model. The preprocessed defect image data and the first detection result are input into the second detection model together. Here, the first detection result is used as an additional input of the model to provide preliminary judgment information about the defect type, helping the second detection model to identify defects more accurately. After receiving the input data, the second detection model performs internal calculations and uses the learned features and rules to perform more detailed classification or identification of defects. The output of the model is the second detection result, which is a further distinction between the fifth and sixth defect types. The second detection result is extracted from the output of the second detection model, including information such as the specific type, size, and location of the defect. In this way, the first detection model and the second detection model form a cascaded detection system, with the first model responsible for preliminary classification and the second model performing more in-depth analysis and identification of specific types of defects. This structure helps to improve the efficiency and accuracy of the entire detection system.
[0071] Step S104: If the first detection result is the second preset detection result, the defect image data and the first detection result are input into a trained third detection model to obtain a second detection result. The first preset detection result is a third defect type or a fourth defect type. The third detection model is trained by defect image data of the marked third defect type and fourth defect type and the first detection result.
[0072] Specifically, the first detection model will output one or more detection results, which include information such as the type and location of the defect, and confirm whether the third defect type or the fourth defect type is included. If the first detection result contains the third defect type or the fourth defect type, ensure that the third detection model has been loaded into the memory and is in an available state. Since different models have different requirements for the input format and range of data, the defect image data and the first detection result are preprocessed. For the defect image data, resize, color space conversion, normalization and other operations are performed to ensure that the format and range of the image data meet the input requirements of the third detection model. For the first detection result, since it is information in text or numerical form, it is encoded into a vector or matrix form so that it can be used as one of the inputs of the third detection model.
[0073] The preprocessed defect image data and the first detection result are used as input data and passed to the third detection model through a programming interface (such as an API call) or a data pipeline (such as a data stream). During the transmission process, it is necessary to ensure the integrity and correctness of the data to avoid data loss or format errors. After receiving the input data, the third detection model performs internal forward propagation calculations to identify or classify defects more finely based on the learned features and rules. The third detection model outputs one or more second detection results, including detailed information such as the specific type, location, and size of the defect. Ensure that the model's reasoning process proceeds normally and obtains the output results in a timely manner, extract the second detection results from the output of the third detection model, and convert them into a readable form.
[0074] Step S105: if the first detection result is the third preset detection result, the first detection result is used as the second detection result, wherein the third preset detection result is the first defect type, the second defect type or the seventh defect type.
[0075] Specifically, after the first detection model completes reasoning, the output result thereof is checked to confirm whether it contains the third preset detection result. In this example, the third preset detection result includes the first defect type, the second defect type, or the seventh defect type. Once the first detection result is confirmed to be any one of the third preset detection results, it can be directly regarded as the second detection result without further processing or analysis by other models.
[0076] Step S106: taking the second detection result as the final detection result, and generating a detection report based on the final detection result.
[0077] Specifically, after a series of detection steps, a second detection result is obtained, and the second detection result is initially judged by the first detection model and further confirmed by the second detection model, or the first detection result is directly used as the second detection result, so it has high accuracy and reliability. When generating a test report, first create a report template, and then fill the final test result into the corresponding position in the template.
[0078] The intelligent detection method for PCB board defect categories provided by the present invention, firstly, by acquiring the original image data of the target PCB board to be detected and performing defect location, the defect position on the PCB board can be accurately and quickly located, which provides an important basis for subsequent defect type identification. This defect location method improves the detection efficiency, reduces the need for human intervention, and thus reduces the detection cost. Secondly, the present invention uses a trained first detection model to classify the defect image data, and can identify seven different defect types. This classification method not only improves the accuracy of detection, but also can cover various defect situations that may occur on the PCB board. At the same time, the first detection model is obtained by training with marked defect image data, which ensures the generalization ability and robustness of the model, so that it can adapt to the detection of PCB boards of different types and specifications. Then, for specific defect types, the present invention also designs a second detection model and a third detection model. When the first detection result is the fifth defect type or the sixth defect type, the second detection model will be further used for detailed identification; when the first detection result is the third defect type or the fourth defect type, the third detection model will be used for further judgment. This hierarchical detection method can more accurately identify specific types of defects, and improve the accuracy and reliability of detection. In addition, when the first detection result is the first defect type, the second defect type or the seventh defect type, the first detection result is directly used as the second detection result, which avoids unnecessary calculation and resource consumption and improves the detection efficiency. Finally, a test report is generated based on the final test result, so that the test result can be intuitively presented to relevant personnel, which is convenient for them to perform subsequent analysis and processing. This automated report generation method reduces manual operation and improves work efficiency. In summary, the PCB board defect category intelligent detection method of the present invention has shown significant technical effects in terms of detection efficiency, accuracy, reliability and ease of use, which is of great significance for improving the production quality of PCB boards and reducing production costs.
[0079] In this embodiment, a method for intelligently detecting defect categories of a PCB board is provided. Figure 2 FIG. 1 is a flow chart of a method for intelligently detecting PCB board defect categories according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:
[0080] Step S201, obtaining original image data of a target PCB board to be inspected, and performing defect location on the original image data to obtain defect image data.
[0081] Specifically, the above step S201 includes:
[0082] Step S2011, obtaining original image data of the target PCB board to be inspected.
[0083] Specifically, use a suitable image acquisition device (such as an industrial camera) to photograph the target PCB board to ensure that the image is clear, unblurred, and includes all areas that need to be inspected, and save the captured image data in a digital format for subsequent computer processing.
[0084] Step S2012, preprocessing the original image data to obtain preprocessed original image data.
[0085] Specifically, the above step S2012 includes:
[0086] Step a1, performing grayscale processing on the original image data to obtain first original image data.
[0087] Specifically, grayscale processing is the process of converting a color image into a grayscale image. In a grayscale image, the brightness of each pixel is represented by a grayscale value, which usually varies between 0 (black) and 255 (white). Through grayscale processing, the color information in the image is eliminated and only the brightness information is retained, which helps to simplify subsequent processing steps. Convert each pixel of the image from the RGB color space to the grayscale space. This is usually achieved by calculating the average value of the three components of R, G, and B of the pixel. Save the converted grayscale image as the first original image data.
[0088] Step a2: filtering the first original image data based on a preset frequency band to obtain second original image data.
[0089] Specifically, filtering is a common method used in image processing to remove noise or enhance specific frequency components. The preset frequency band refers to the frequency range that you want to keep or remove. For example, a low-pass filter can remove high-frequency noise, while a high-pass filter can emphasize edges and details. Select a suitable filter, such as a Gaussian filter, a median filter, or a Fourier transform filter. Apply the filter to the first original image data and filter based on the preset frequency band. Save the filtered image as the second original image data.
[0090] Step a3: performing contrast adjustment on the second original image data to obtain third original image data.
[0091] Specifically, contrast adjustment aims to enhance the contrast of an image and make the details in the image clearer. By contrast adjustment, the visual effect of the image is improved and subsequent image analysis tasks are facilitated. A contrast histogram of the second original image data is calculated. According to the histogram distribution and a preset contrast adjustment such as a contrast stretching coefficient, the contrast of the image is adjusted, and the contrast-adjusted image is saved as the third original image data.
[0092] Step a4: performing binary image conversion on the third original image data based on a preset threshold value to obtain pre-processed original image data.
[0093] Specifically, binary image conversion is the process of converting a grayscale image into a binary image. In a binary image, each pixel has only two possible values, usually 0 and 255, corresponding to black and white. This conversion helps to simplify the image and highlight key information. Select a suitable threshold value, which is determined based on the characteristics of the image and the processing requirements. Compare each pixel value of the third original image data with the threshold value, and if the pixel value is greater than the threshold value, set it to white or 255, otherwise set it to black or 0. Save the converted binary image as the preprocessed original image data.
[0094] Step S2013, using an edge detection method to perform edge detection on the preprocessed original image data to obtain hole edge data.
[0095] Specifically, select a suitable edge detection algorithm (such as Canny edge detection, Sobel edge detection, etc.) to perform edge detection on the preprocessed image. The edge detection algorithm will identify the location where the pixel value in the image changes dramatically, that is, the edge, thereby obtaining image data containing hole edge information.
[0096] Step S2014, performing morphological operations on the hole edge data to obtain hole edge refinement data.
[0097] Specifically, morphological operation is a shape-based processing technique that is often used to remove noise, separate and connect different parts in an image. Morphological operations (such as corrosion, dilation, opening, closing, etc.) are applied to the hole edge data to remove burrs on the edges, fill small holes or disconnect narrow connections, thereby obtaining clearer and more accurate hole edge refinement data.
[0098] Step S2015, using region growing technology to perform region segmentation on the hole edge refinement data to obtain defect image data of each defect region.
[0099] Specifically, region growing technology is a method for image segmentation based on pixel similarity. According to the hole edge refinement data, one or more seed points are selected as the starting point of growth. Growth criteria are set, such as pixel value difference threshold, connectivity of adjacent pixels, etc. Starting from the seed point, adjacent pixels that meet the growth criteria are gradually added to the same region until no more pixels that meet the conditions can be found. Repeat the above process until all pixels are assigned to the corresponding regions, thereby obtaining the defect image data of each defect region.
[0100] Step S202, input the defect image data into the trained first detection model to obtain the first detection result, the first detection result is one of the first defect type, the second defect type, the third defect type, the fourth defect type, the fifth defect type, the sixth defect type and the seventh defect type, wherein the first detection model is obtained by training with the marked defect image data, the first defect type characterizes the plugging hole type defect; the second defect type characterizes the non-through hole containing metal type defect; the third defect type characterizes the aperture size error type defect; the fourth defect type characterizes the collapsed hole type defect; the fifth defect type characterizes the hole wall roughness type defect; the sixth defect type characterizes the copper exposed in the hole type defect; the seventh defect type characterizes the green oil into the hole type defect.
[0101] Specifically, the above step S202 includes:
[0102] Step S2021, perform feature extraction on the defect image data to obtain the geometric features, texture features and color features of the defect image data. The geometric features include the area, perimeter and circularity of each defect area, the texture features include the gray level co-occurrence matrix and the local binary pattern histogram, and the color features include the color histogram and color moment.
[0103] Specifically, feature extraction is the process of extracting meaningful information from image data, which can describe the properties and characteristics of the target or defect in the image. In PCB board defect detection, common features include geometric features, texture features and color features. In geometric features, area represents the number of pixels occupied by the defect area. Perimeter represents the boundary length of the defect area. Circularity describes the degree of proximity of the defect shape to the circle, which can be calculated by perimeter and area. In texture features, the gray level co-occurrence matrix describes the texture characteristics of the image by analyzing the gray level relationship between pixel pairs in different directions and distances in the image. The local binary pattern histogram (LBP) is an algorithm used to describe the local texture features of the image. It generates texture features by comparing the gray value relationship between the central pixel and its neighboring pixels. In color features, the color histogram counts the frequency of occurrence of different color levels in the image, which can reflect the color distribution of the image. Color moments describe the distribution of colors by calculating statistics such as the mean and variance of colors in the image.
[0104] Step S2022: using the first detection model to perform iterative processing based on geometric features, texture features and color features.
[0105] Specifically, the extracted geometric features, texture features, and color features are used as inputs to the trained gradient boosting model. The first detection model iterates based on these features and optimizes its classification performance through multiple iterations. In each iteration, the first detection model learns a new decision tree and makes adjustments based on previous errors to gradually improve the accuracy of classification.
[0106] Step S2023, when the preset condition is met, stop the iteration and output the classification result as the first detection result.
[0107] Specifically, the iteration process will continue until the preset stop condition is met. These conditions may include reaching the maximum number of iterations, the model performance no longer significantly improving in multiple consecutive iterations, etc. When the stop condition is met, the iteration process will terminate, and the model will output the current classification result as the first detection result. These results include the category (i.e. defect type) to which each defect area belongs and the corresponding confidence score.
[0108] Step S203: If the first detection result is the first preset detection result, the defect image data and the first detection result are input into the trained second detection model to obtain the second detection result. The first preset detection result is the fifth defect type or the sixth defect type. The second detection model is trained by the marked defect image data of the fifth defect type and the sixth defect type and the first detection result.
[0109] Specifically, the above step S203 includes:
[0110] Step S2031, acquiring texture features and color features of defect image data.
[0111] Specifically, on the PCB board, the reason why texture features and color features can distinguish between rough hole wall defects and exposed copper defects in the hole is that these two features provide detailed descriptions of the surface structure and color information of the defects, respectively. First, texture features can accurately describe the microstructure and arrangement of the surface of the object. For rough hole wall defects, the surface is usually uneven, with obvious bumps, scratches or granular structures. These subtle texture changes can be effectively captured and extracted as an important basis for distinguishing rough hole wall defects from other types of defects. Secondly, color features can reflect the color information of the surface of the object. In the exposed copper defect in the hole, due to the exposure of the copper layer, the color of the defect area is usually significantly different from the surrounding area. By obtaining color features, these color changes can be quickly identified, thereby accurately locating the exposed copper defect in the hole. Combining texture features with color features can further improve the accuracy and efficiency of defect recognition. Because rough hole wall defects and exposed copper defects in the hole usually show different characteristic patterns in texture and color, the comprehensive use of these two features can more accurately distinguish these two types of defects.
[0112] Step S2032, feature encoding is performed on the first detection result to obtain the first detection result feature.
[0113] Specifically, the first detection result is the classification result output by the first detection model (such as the gradient boosting machine model). In order to use these results as input features of the second detection model, feature encoding is required. Feature encoding is the process of converting the classification results into numerical form to facilitate subsequent mathematical calculations and model processing. The specific encoding method can be selected according to the actual situation. For example, the classification results can be converted into one-hot encoding or label encoding.
[0114] Step S2033, concatenating the texture feature, the color feature and the first detection result feature to obtain a first concatenated feature.
[0115] Specifically, the texture feature, the color feature and the first detection result feature are concatenated to form a comprehensive feature vector, namely the first concatenated feature. The concatenation operation is to connect these features numerically to form a longer feature vector so that the second detection model can simultaneously use these features to classify defects.
[0116] Step S2034, using the second detection model to calculate the fifth defect type score and the sixth defect type score based on the first splicing feature, the fifth defect type score is the score of the target PCB board to be detected belonging to the fifth defect type, and the sixth defect type score is the score of the target PCB board to be detected belonging to the sixth defect type.
[0117] Specifically, the second detection model is a pre-trained model for calculating scores of different defect types based on input features. Here, the second detection model receives the first splicing feature as input and outputs a score for the fifth defect type and a score for the sixth defect type. The two scores represent the probability or confidence that the target PCB board to be inspected belongs to the fifth defect type and the sixth defect type, respectively.
[0118] Step S2035: Use the second detection model to make a classification decision based on the fifth defect type score and the sixth defect type score to obtain a second detection result.
[0119] Specifically, based on the fifth defect type score and the sixth defect type score, the second detection model will make a classification decision to determine the defect type to which the target PCB board to be detected belongs, and by comparing the scores of different defect types, select the defect type with the highest score or that meets a certain threshold condition as the final classification result. The final classification result is the second detection result.
[0120] Step S204: If the first detection result is the second preset detection result, the defect image data and the first detection result are input into a trained third detection model to obtain a second detection result. The first preset detection result is a third defect type or a fourth defect type. The third detection model is trained by defect image data of the marked third defect type and fourth defect type and the first detection result.
[0121] Specifically, the above step S204 includes:
[0122] Step S2041, acquiring texture features and geometric features of defect image data.
[0123] Specifically, on the PCB board, the texture features and geometric features of the defect image data can distinguish between the aperture size error type defect and the hole collapse type defect for the following reasons: First, the texture features can reflect the surface structure information of the defect area. For the aperture size error type defect, although the aperture size does not match the standard size, the surface structure of the hole wall may not change significantly. However, for the hole collapse type defect, due to the collapse or fracture of the hole wall material, its surface structure usually changes significantly, such as cracks, fragments or uneven surfaces. These changes in texture features help to distinguish between these two types of defects. Secondly, the geometric features describe the spatial information of the defect, such as the shape, size and position. The main feature of the aperture size error type defect is that its aperture size does not match the expected size. This size difference can be accurately measured and identified by geometric features. The hole collapse type defect may be manifested as deformation, collapse or complete destruction of the hole. These changes in geometric shape can also be captured and analyzed by geometric features. The comprehensive use of texture features and geometric features can provide a more comprehensive understanding of the nature and characteristics of the defects, thereby more accurately distinguishing the aperture size error type defect and the hole collapse type defect. These two features complement each other. The texture features provide surface structure information of the defect area, while the geometric features provide shape and size information of the defect, together forming a complete basis for defect identification.
[0124] Step S2042, feature encoding is performed on the first detection result to obtain the first detection result feature.
[0125] Specifically, the first detection result is the classification result output by the first detection model. In order to use these classification results as input features of the third detection model, feature encoding is required. Feature encoding converts the classification results into numerical form to facilitate subsequent mathematical calculations and classification processing by the neural network model.
[0126] Step S2043, concatenating the texture feature, the geometric feature and the first detection result feature to obtain a second concatenated feature.
[0127] Specifically, the extracted texture features, geometric features and the first detection result features are spliced to form a comprehensive feature vector, namely the second spliced feature. The purpose of this is to fuse information from different sources together to provide a more comprehensive input for the neural network model, thereby improving the accuracy of classification.
[0128] Step S2044: Obtain a third splicing feature by performing a nonlinear transformation on the second splicing feature using the hidden layer of the third detection model.
[0129] Specifically, the hidden layer of the neural network model is responsible for performing nonlinear transformations on the input data to learn the complex patterns and relationships of the data. Here, the hidden layer performs nonlinear transformations on the second concatenated features to extract deeper feature information to form the third concatenated features. This step helps the model capture complex structures and patterns in the data, thereby improving classification performance.
[0130] Step S2045, using the classifier of the third detection model to calculate the third defect type possibility value and the fourth defect type possibility value, the third defect type possibility value is the possibility value that the target PCB board to be detected belongs to the third defect type, and the fourth defect type score is the possibility value that the target PCB board to be detected belongs to the fourth defect type.
[0131] Specifically, the classifier of the third detection model calculates the probability values of the target PCB board to be detected belonging to the third defect type and the fourth defect type based on the third splicing feature output by the hidden layer. These probability values represent the probability or confidence that the PCB board belongs to different defect types.
[0132] Step S2046: Use the third detection model to make a classification decision based on the third defect type possibility value and the fourth defect type possibility value to obtain a third detection result.
[0133] Specifically, based on the calculated probability values of the third defect type and the fourth defect type, the third detection model makes a classification decision to determine the defect type to which the target PCB board to be detected belongs, and by comparing the probability values of different defect types, selects the defect type with the highest probability as the final classification result.
[0134] Step S205: If the first detection result is the third preset detection result, the first detection result is used as the second detection result, wherein the third preset detection result is the first defect type, the second defect type or the seventh defect type. Figure 1 Step S105 of the illustrated embodiment will not be described in detail here.
[0135] Step S206: taking the second test result as the final test result, and generating a test report based on the final test result.
[0136] Specifically, the above step S206 includes:
[0137] Step S2061: when the second inspection result is the third defect type or the fifth defect type, a rework inspection report is generated, and the rework inspection report includes the final inspection result and defect image data.
[0138] Specifically, mechanical grinding or mechanical extrusion methods can be used to try to repair the aperture size for aperture size errors, and chemical treatment methods, such as acidic or alkaline solutions, can be used to dissolve irregular or rough parts of the aperture wall surface for aperture size errors. For the third defect type (aperture size error defect), the report should record in detail the deviation between the actual aperture size and the standard size, as well as the specific defect conditions shown by the defect image data. For the fifth defect type (aperture wall roughness defect), the report should indicate the degree of aperture wall roughness and possible causes.
[0139] Step S2062, when the second inspection result is the first defect type, the second defect type, the fourth defect type, the sixth defect type or the seventh defect type, a scrap inspection report is generated, and the scrap inspection report includes the final inspection result.
[0140] Specifically, unlike rework inspection reports, scrap inspection reports usually do not contain defect image data. This is because for PCB boards that need to be scrapped, the possibility of repair is low, and further analysis of defect image data may not be a necessary or priority task. Instead, the focus is on quickly and accurately identifying PCB boards with serious defects and taking appropriate treatment measures to prevent them from entering the subsequent production process or market.
[0141] The intelligent detection method for PCB board defect categories provided by the present invention, firstly, by acquiring the original image data of the target PCB board to be detected and performing defect location, the defect position on the PCB board can be accurately and quickly located, which provides an important basis for subsequent defect type identification. This defect location method improves the detection efficiency, reduces the need for human intervention, and thus reduces the detection cost. Secondly, the present invention uses a trained first detection model to classify the defect image data, and can identify seven different defect types. This classification method not only improves the accuracy of detection, but also can cover various defect situations that may occur on the PCB board. At the same time, the first detection model is obtained by training with marked defect image data, which ensures the generalization ability and robustness of the model, so that it can adapt to the detection of PCB boards of different types and specifications. Then, for specific defect types, the present invention also designs a second detection model and a third detection model. When the first detection result is the fifth defect type or the sixth defect type, the second detection model will be further used for detailed identification; when the first detection result is the third defect type or the fourth defect type, the third detection model will be used for further judgment. This hierarchical detection method can more accurately identify specific types of defects, and improve the accuracy and reliability of detection. In addition, when the first detection result is the first defect type, the second defect type or the seventh defect type, the first detection result is directly used as the second detection result, which avoids unnecessary calculation and resource consumption and improves the detection efficiency. Finally, a test report is generated based on the final test result, so that the test result can be intuitively presented to relevant personnel, which is convenient for them to perform subsequent analysis and processing. This automated report generation method reduces manual operation and improves work efficiency. In summary, the PCB board defect category intelligent detection method of the present invention has shown significant technical effects in terms of detection efficiency, accuracy, reliability and ease of use, which is of great significance for improving the production quality of PCB boards and reducing production costs.
[0142] In this embodiment, a PCB board defect category intelligent detection device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0143] This embodiment provides a PCB board defect category intelligent detection device, such as Figure 3 As shown, including:
[0144] The defect location module 301 is used to obtain the original image data of the target PCB board to be inspected, and perform defect location on the original image data to obtain defect image data.
[0145] The first detection module 302 is used to input the defect image data into the trained first detection model to obtain the first detection result, which is one of the first defect type, the second defect type, the third defect type, the fourth defect type, the fifth defect type, the sixth defect type and the seventh defect type, wherein the first detection model is obtained by training the marked defect image data, the first defect type represents the plugging hole type defect: the second defect type represents the non-through hole containing metal type defect: the third defect type represents the hole size error type defect, the fourth defect type represents the collapse hole type defect, the fifth defect type represents the hole wall roughness type defect, the sixth defect type represents the hole exposed copper type defect. The seventh defect type represents the green oil into the hole type defect.
[0146] The second detection module 303 is used to input the defect image data and the first detection result into a trained second detection model to obtain a second detection result if the first detection result is a first preset detection result. The first preset detection result is the fifth defect type or the sixth defect type. The second detection model is trained by the defect image data of the marked fifth defect type and the sixth defect type and the first detection result.
[0147] The third detection module 304 is used to input the defect image data and the first detection result into a trained third detection model to obtain a second detection result if the first detection result is the second preset detection result. The first preset detection result is the third defect type or the fourth defect type. The third detection model is trained by the defect image data of the marked third defect type and the fourth defect type and the first detection result.
[0148] The fourth detection module 305 is configured to use the first detection result as the second detection result if the first detection result is a third preset detection result, wherein the third preset detection result is the first defect type, the second defect type or the seventh defect type.
[0149] The report generating module 306 is used to use the second detection result as the final detection result and generate a detection report based on the final detection result.
[0150] The intelligent detection device for PCB board defect categories provided by the present invention can, firstly, accurately and quickly locate the defect position on the PCB board by acquiring the original image data of the target PCB board to be detected and performing defect location, thereby providing an important basis for subsequent defect type identification. This defect location method improves the detection efficiency, reduces the need for human intervention, and thus reduces the detection cost. Secondly, the present invention uses a trained first detection model to classify the defect image data, and can identify seven different defect types. This classification method not only improves the accuracy of detection, but also can cover various defect situations that may occur on the PCB board. At the same time, the first detection model is obtained by training with marked defect image data, which ensures the generalization ability and robustness of the model, so that it can adapt to the detection of PCB boards of different types and specifications. Then, for specific defect types, the present invention also designs a second detection model and a third detection model. When the first detection result is the fifth defect type or the sixth defect type, the second detection model will be further used for detailed identification; when the first detection result is the third defect type or the fourth defect type, the third detection model will be used for further judgment. This hierarchical detection method can more accurately identify specific types of defects, thereby improving the accuracy and reliability of detection. In addition, when the first detection result is the first defect type, the second defect type or the seventh defect type, the first detection result is directly used as the second detection result, which avoids unnecessary calculation and resource consumption and improves the detection efficiency. Finally, a test report is generated based on the final test result, so that the test result can be intuitively presented to relevant personnel, which is convenient for them to perform subsequent analysis and processing. This automated report generation method reduces manual operation and improves work efficiency. In summary, the PCB board defect category intelligent detection method of the present invention has shown significant technical effects in terms of detection efficiency, accuracy, reliability and ease of use, which is of great significance for improving the production quality of PCB boards and reducing production costs.
[0151] In some optional implementations, the defect location module 301 includes:
[0152] The original image data acquisition unit is used to acquire the original image data of the target PCB board to be inspected.
[0153] The preprocessing unit is used to preprocess the original image data to obtain preprocessed original image data.
[0154] The edge detection unit is used to perform edge detection on the preprocessed original image data using an edge detection method to obtain hole edge data.
[0155] The morphological operation unit is used to perform morphological operations on the hole edge data to obtain hole edge refinement data.
[0156] The region segmentation unit is used to perform region segmentation on the hole edge refinement data using the region growing technology to obtain defect image data of each defect region.
[0157] In some optional embodiments, the pre-processing unit includes:
[0158] The grayscale processing subunit is used to perform grayscale processing on the original image data to obtain first original image data.
[0159] The filtering processing subunit is used to perform filtering processing on the first original image data based on a preset frequency band to obtain second original image data.
[0160] The contrast adjustment subunit is used to perform contrast adjustment on the second original image data to obtain third original image data.
[0161] The binary image conversion subunit is used to perform binary image conversion on the third original image data based on a preset threshold value to obtain pre-processed original image data.
[0162] In some optional implementations, the first detection module 302 includes:
[0163] The feature extraction unit is used to extract features from the defect image data to obtain the geometric features, texture features and color features of the defect image data. The geometric features include the area, perimeter and circularity of each defect area, the texture features include the gray level co-occurrence matrix and the local binary pattern histogram, and the color features include the color histogram and color moment.
[0164] The iterative unit is used to perform iterative processing based on geometric features, texture features and color features using the first detection model.
[0165] The iteration stopping unit is used to stop the iteration when a preset condition is reached and output the classification result as the first detection result.
[0166] In some optional implementations, the second detection module 303 includes:
[0167] The first feature acquisition unit is used to acquire texture features and color features of defect image data.
[0168] The first feature encoding unit is used to perform feature encoding on the first detection result to obtain a first detection result feature.
[0169] The first feature splicing unit is used to splice the texture feature, the color feature and the first detection result feature to obtain a first splicing feature.
[0170] The first calculation unit is used to calculate a fifth defect type score and a sixth defect type score based on the first splicing feature using the second detection model, wherein the fifth defect type score is a score of the target PCB board to be detected belonging to the fifth defect type, and the sixth defect type score is a score of the target PCB board to be detected belonging to the sixth defect type.
[0171] The first classification decision unit is used to make a classification decision based on the fifth defect type score and the sixth defect type score by using the second detection model to obtain a second detection result.
[0172] In some optional implementations, the third detection module 304 includes:
[0173] The second feature acquisition unit is used to acquire texture features and geometric features of the defect image data.
[0174] The second feature encoding unit is used to perform feature encoding on the first detection result to obtain the first detection result feature.
[0175] The second feature splicing unit is used to splice the texture feature, the geometric feature and the first detection result feature to obtain a second splicing feature.
[0176] The nonlinear transformation unit is used to obtain a third splicing feature by performing a nonlinear transformation on the second splicing feature using the hidden layer of the third detection model.
[0177] The second calculation unit is used to calculate a third defect type possibility value and a fourth defect type possibility value by using a classifier of a third detection model, wherein the third defect type possibility value is a possibility value of the target PCB board to be detected belonging to the third defect type, and the fourth defect type score is a possibility value of the target PCB board to be detected belonging to the fourth defect type.
[0178] The second classification decision unit is used to make a classification decision based on the third defect type possibility value and the fourth defect type possibility value by using the third detection model to obtain a third detection result.
[0179] In some optional implementations, the report generation module 306 includes:
[0180] The rework inspection unit is used to generate a rework inspection report when the second inspection result is the third defect type or the fifth defect type, and the rework inspection report includes a final inspection result and defect image data.
[0181] The scrap detection unit is used to generate a scrap detection report when the second detection result is the first defect type, the second defect type, the fourth defect type, the sixth defect type or the seventh defect type, and the scrap detection report includes the final detection result.
[0182] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.
[0183] The PCB board defect category intelligent detection device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.
[0184] The embodiment of the present invention also provides a computer device having the above Figure 3 The PCB board defect category intelligent detection device shown.
[0185] See also Figure 4 , Figure 4 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 4 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 10 is taken as an example.
[0186] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.
[0187] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.
[0188] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0189] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.
[0190] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.
[0191] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.
[0192] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.
Claims
1. A PCB board defect category intelligent detection method, characterized in that: The method comprises: Acquire original image data of the target PCB board to be inspected, perform defect location on the original image data, and obtain defect image data; Input the defect image data into a trained first detection model to obtain a first detection result, wherein the first detection result is one of a first defect type, a second defect type, a third defect type, a fourth defect type, a fifth defect type, a sixth defect type, and a seventh defect type, wherein the first detection model is obtained by training with labeled defect image data, the first defect type characterizes a plugging hole defect; the second defect type characterizes a non-through hole containing metal defect; the third defect type characterizes an aperture size error defect; the fourth defect type characterizes a collapsed hole defect; the fifth defect type characterizes a hole wall roughness defect; the sixth defect type characterizes a copper exposed hole defect; and the seventh defect type characterizes a green oil into hole defect; If the first detection result is a first preset detection result, inputting the defect image data and the first detection result into a trained second detection model to obtain a second detection result, the first preset detection result is a fifth defect type or a sixth defect type, and the second detection model is trained by the defect image data of the fifth defect type and the sixth defect type that are marked and the first detection result; If the first detection result is a second preset detection result, inputting the defect image data and the first detection result into a trained third detection model to obtain a second detection result, the first preset detection result is a third defect type or a fourth defect type, and the third detection model is trained by defect image data of the marked third defect type and fourth defect type and the first detection result; If the first detection result is a third preset detection result, taking the first detection result as the second detection result, wherein the third preset detection result is the first defect type, the second defect type, or the seventh defect type; The second detection result is used as the final detection result, and a detection report is generated based on the final detection result.
2. The method according to claim 1, characterized in that The method of acquiring original image data of a target PCB board to be inspected and locating defects on the original image data to obtain defect image data includes: Obtain the original image data of the target PCB board to be inspected; Preprocessing the original image data to obtain preprocessed original image data; Performing edge detection on the preprocessed original image data using an edge detection method to obtain hole edge data; Performing morphological operations on the hole edge data to obtain hole edge refinement data; The hole edge refinement data is segmented by region using region growing technology to obtain defect image data of each defect region.
3. The method according to claim 2, characterized in that The preprocessing of the original image data to obtain the preprocessed original image data includes: Performing grayscale processing on the original image data to obtain first original image data; Performing filtering processing on the first original image data based on a preset frequency band to obtain second original image data; performing contrast adjustment on the second original image data to obtain third original image data; The third original image data is converted into a binary image based on a preset threshold value to obtain preprocessed original image data.
4. The method according to claim 3, characterized in that The first detection model is a gradient boosting machine model, and the step of inputting the defect image data into the trained first detection model to obtain a first detection result includes: Extracting features from the defect image data to obtain geometric features, texture features, and color features of the defect image data, wherein the geometric features include the area, perimeter, and circularity of each defect region, the texture features include a gray-level co-occurrence matrix and a local binary pattern histogram, and the color features include a color histogram and a color moment; Using the first detection model to perform iterative processing based on the geometric features, texture features and color features; When the preset condition is reached, the iteration is stopped and the classification result is output as the first detection result.
5. The method according to claim 4, characterized in that The second detection model is a support vector machine model, and if the first detection result is a first preset detection result, the defect image data and the first detection result are input into a trained second detection model to obtain a second detection result, including: Acquire texture features and color features of the defect image data; Performing feature encoding on the first detection result to obtain a first detection result feature; Splicing the texture feature, the color feature and the first detection result feature to obtain a first splicing feature; Calculating a fifth defect type score and a sixth defect type score based on the first splicing feature using the second detection model, wherein the fifth defect type score is a score for the target PCB board to be detected belonging to the fifth defect type, and the sixth defect type score is a score for the target PCB board to be detected belonging to the sixth defect type; The second detection model is used to make a classification decision based on the fifth defect type score and the sixth defect type score to obtain a second detection result.
6. The method according to claim 5, characterized in that The third detection model is a neural network model, and if the first detection result is the second preset detection result, the defect image data and the first detection result are input into the trained third detection model to obtain the second detection result, including: Acquire texture features and geometric features of the defect image data; Performing feature encoding on the first detection result to obtain a first detection result feature; Splicing the texture feature, the geometric feature and the first detection result feature to obtain a second splicing feature; Obtain a third splicing feature by performing a nonlinear transformation on the second splicing feature using a hidden layer of the third detection model; Calculating a third defect type possibility value and a fourth defect type possibility value using the classifier of the third detection model, wherein the third defect type possibility value is a possibility value of the target PCB board to be detected belonging to the third defect type, and the fourth defect type score is a possibility value of the target PCB board to be detected belonging to the fourth defect type; The third detection model is used to perform classification decisions based on the third defect type possibility value and the fourth defect type possibility value to obtain a third detection result.
7. The method according to claim 6, characterized in that The step of taking the second test result as the final test result and generating a test report based on the final test result includes: When the second inspection result is the third defect type or the fifth defect type, generating a rework inspection report, the rework inspection report including the final inspection result and defect image data; When the second inspection result is the first defect type, the second defect type, the fourth defect type, the sixth defect type or the seventh defect type, a scrap inspection report is generated, and the scrap inspection report includes a final inspection result.
8. An intelligent detection device for PCB board defect categories, characterized in that: The device comprises: A defect location module is used to obtain original image data of the target PCB board to be inspected, perform defect location on the original image data, and obtain defect image data; A first detection module is used to input the defect image data into a trained first detection model to obtain a first detection result, wherein the first detection result is one of a first defect type, a second defect type, a third defect type, a fourth defect type, a fifth defect type, a sixth defect type and a seventh defect type, wherein the first detection model is obtained by training with labeled defect image data, the first defect type represents a plugging hole defect; the second defect type represents a non-through hole containing metal defect; the third defect type represents an aperture size error defect; the fourth defect type represents a collapsed hole defect; the fifth defect type represents a hole wall roughness defect; the sixth defect type represents a copper exposed inside the hole defect; the seventh defect type represents a green oil into the hole defect; a second detection module, configured to input the defect image data and the first detection result into a trained second detection model to obtain a second detection result if the first detection result is a first preset detection result, the first preset detection result is a fifth defect type or a sixth defect type, and the second detection model is trained by the defect image data of the fifth defect type and the sixth defect type that are marked and the first detection result; a third detection module, configured to input the defect image data and the first detection result into a trained third detection model to obtain a second detection result if the first detection result is a second preset detection result, the first preset detection result is a third defect type or a fourth defect type, and the third detection model is trained by defect image data of the marked third defect type and fourth defect type and the first detection result; a fourth detection module, configured to use the first detection result as a second detection result if the first detection result is a third preset detection result, wherein the third preset detection result is the first defect type, the second defect type, or the seventh defect type; The report generating module is used to use the second detection result as the final detection result and generate a detection report based on the final detection result.
9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 7 by executing the computer instructions.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.
