Printed circuit board welding spot failure mechanism prediction system and method based on artificial intelligence

Abstract

The invention provides a printed circuit board welding spot failure mechanism prediction system and method based on artificial intelligence, and the system comprises a PCB welding region detection module which is used for capturing an RGB image of a PCB welding region through a camera, and detecting the PCB welding region based on a deep learning model; the detection module of the thermal imaging camera is used for detecting the PCB failure mechanism prediction module under high current; and the PCB failure mechanism prediction module is used for carrying out analysis according to the PCB defect classification model, obtaining the heat temperature of a PCB welding area, and further reasoning and predicting a failure mechanism. The method has the advantages that implementation is easy, all PCB welding spots on a production line can be quickly and accurately checked, the method is more suitable for large-scale production, data can be quickly analyzed, correction measures are taken before large-scale product production, manufacturers can recognize potential problems, and higher flexibility and expandability are achieved in the production process.

Description

Technical Field

[0001] The invention belongs to the field of computer technology and relates to a printed circuit board solder joint failure mechanism prediction system and method based on artificial intelligence. Background Art

[0002] PCB has become a key component in electronic products, providing necessary mechanical support for various circuit components. PCB has high neutrality and reliability, which is beneficial to daily production, design and maintenance. Mechanical automation and product efficiency have made significant progress due to their use. In view of these advantages, PCB has been widely used in various fields such as computer, automobile and aviation industries.

[0003] However, due to human error, machine failure and other reasons, some defects will appear in the PCB production process. These defects, such as short circuit, open circuit, rat bite, etc., will affect the remaining service life of the product, threaten the life of the user, and cause economic losses. In recent years, two trends have emerged in the PCB electronics industry. The first is lead-free soldering in electronic packaging and assembly, and the second is the miniaturization of electronic products. Both trends have brought new challenges to electronic materials and manufacturing. Soldering has become a very complex process due to many factors (such as non-wettability, dewetting, wrong solder mask design, warpage effect and cracks) that affect the final quality of the product solder joints. There are also many reasons for the formation of defects and imperfections during the soldering process. Some of the main reasons include improperly designed technical process settings, improper use of materials, design errors and environmental impacts.

[0004] The evaluation of solder joint reliability is crucial in the field of electronic packaging. The life of a PCB depends largely on the durability of its solder joints. The vulnerability of PCB solder joints to the generation and propagation of cracks and fractures poses a significant risk to the overall performance of electronic products, especially under harsh conditions such as high temperature, low temperature and strong vibration. The presence of any voids and cracks may cause damage to the pins of electronic components, thereby compromising the reliability of their solder joints and, in some cases, significantly shortening the service life of the PCB. According to a literature review, approximately 60% of electronic product failures can be attributed to solder joint failures. With the recent developments in the PCB industry, especially in the advancement of high precision, high density and high reliability, the requirements for improving the reliability of PCB manufacturing processes are increasing. Therefore, the reliability of PCB solder joints must be thoroughly studied and analyzed to meet these growing demands.

[0005] Traditional manual quality inspection consumes a lot of human resources for problems such as PCB soldering defects, and there is a trend to replace manual inspection with machine automated inspection. Using manual features for diagnosis has been shown to improve diagnostic performance. Diagnosis is sometimes referred to as a classification problem due to the complexity of identifying failure modes and / or causes, pinpointing defect types, and defining degradation levels. Similarly, manually designing a good set of features is a time-consuming, problem-specific, and non-scalable operation. Therefore, there is a growing demand for software that can automatically discover properties related to anomaly detection, diagnosis, and prediction. In recent years, the booming development of deep learning has created opportunities for the application of deep learning models in PCB defect detection based on visual appearance.

[0006] In addition, the electronics manufacturing industry often uses automated optical inspection (AOI) systems to capture PCB images and evaluate the quality of their solder joints through image feature extraction, processing, and analysis. Although AOI systems can effectively test solder joints on high-density boards and small electronic components, they may not be suitable for detecting and inspecting defects in PCB soldering due to the diversity of soldering processes and materials used. Therefore, a data-driven AI model with deep learning algorithms is needed in combination with AOI to inspect and detect defects in component PCB soldering.

[0007] Furthermore, the failure mechanism of solder joints can refer to the process by which a system, component, or material fails to perform its intended function. The assembly of PCBs is often subject to multiple temperature changes, which can exacerbate the effects on the materials. Therefore, PCB materials must be able to withstand multiple exposures to high temperature environments. Currently, SAC305 is the main lead-free solder widely used in PCB manufacturing due to its high performance and low cost. However, the reliability of solder joints remains an important area of ​​research in the field of electronic packaging. The service life of PCB boards depends largely on the durability of solder joints, which are often considered to be the weak link of electronic products. FR-4 is a commonly used material in PCB manufacturing. It is a composite material made of glass fiber woven cloth with a flame-retardant epoxy resin adhesive. However, this material is not capable of withstanding high temperatures in lead-free soldering processes.

[0008] When exposed to such high temperatures, especially when the temperature is very close to the glass transition temperature (Tg) of the PCB, the material properties of the PCB will change. The glass transition temperature (Tg) is a key standard parameter of the substrate because it determines the temperature at which the resin matrix changes from a glassy, ​​brittle state to a soft, elastic state. The glass transition temperature of the substrate determines the upper limit at which the resin matrix breaks down and delamination occurs. Therefore, it is not the highest operating temperature, but the temperature that the material can withstand for a relatively short period of time.

[0009] The Tg of a printed circuit board (PCB) can vary depending on the materials used to make the PCB. PCBs can be made from a variety of materials. Different materials have different Tg values. For example, FR-4 typically has a Tg of about 130-140°C. It is very important to consider the Tg of the materials used in your PCB design to ensure that it can withstand the expected operating temperature range. The Tg of a PCB will affect the coefficient of thermal expansion (CTE) of the solder joints of electrical or electronic components mounted on the same PCB. The CTE of a material is a measure of how much a material expands or contracts as temperature changes.

[0010] When the temperature of a PCB changes above its Tg, the material becomes more flexible and its CTE increases. However, the CTE of the electrical or electronic components mounted on the PCB remains relatively constant or changes only slightly with temperature. As a result, the thermal expansion and contraction of the two materials become mismatched, causing stresses on the solder joints that connect the components to the PCB. Initially, the increased flexibility of the PCB allows it to absorb some of the stresses created by the mismatched CTE. However, as the temperature rises further above the Tg, the PCB becomes increasingly softer and begins to deform under the stress. This deformation can cause the solder joints to crack or break, resulting in component connection failure. The failure mechanism is essentially a failure in which the solder joint expands and contracts due to temperature changes, ultimately leading to weakening and failure of the solder joint. Failure may not occur immediately, but may occur over time as the PCB is subjected to thermal stresses during normal operation. To prevent this failure mechanism, one existing approach is to design the PCB with a Tg that is appropriate for the expected operating temperature range and ensure proper thermal management to minimize thermal stresses on the solder joints. In addition, a second approach is to select solders with higher ductility, which also helps reduce the likelihood of solder joint failure due to CTE mismatch. However, these methods are time-consuming and ineffective in identifying solder joint failure mechanisms. In addition, previous works (such as those by other researchers) focus more on PCB solder joint failure detection models using only visual inspection systems (such as RGB images). According to the literature review, there are few effective methods for developing PCB solder joint failure prediction models.

[0011] Predictive maintenance is the condition monitoring of machinery using smart sensors and other Internet of Things (IoT) technologies. Data acquired from condition monitoring provides information and predictions about the health status of the equipment. Therefore, early identification of defects and failures helps maintenance operators to take timely actions, thereby reducing the frequency of machine failures. It provides comprehensive proactive warnings and downtime predictions in advance, so service teams can choose what to prioritize or plan to replace manufacturing components in advance in a proactive manner, rather than responding to downtime. Traditional experimental-based methods of detecting solder failure mechanisms can be a time-consuming and costly process that may only be able to inspect a limited number of PCB solder joints during the manufacturing process.

[0012] Therefore, a system and method is needed to prevent significant degradation of machines and their systems. Reducing the degree of degradation can prevent the spread of other failures and defects. A reduction in equipment failures can increase production because maintenance personnel and managers are free to focus on important maintenance tasks. Maintenance costs include employee costs and maintenance department management expenses, as well as the spare parts and tools required. Therefore, by reducing the severity of damage through predictive maintenance, maintenance costs will be reduced.

[0013] The technology of Chinese patent document number CN113409250 A discloses a solder joint detection method based on a convolutional neural network model, including first collecting a PCB solder joint data set, preprocessing the data, and then marking and storing the data; establishing a neural network model based on computer vision; finally, using the solder joint data training set to train the established neural network model, and using the solder joint data test set to test the established model. The method improves the YOLOv3 network structure, detects solder joint targets through five feature detection layers of different scales, and improves the detection effect of the target detection network on small-scale targets; the loss function of the convolutional neural network consists of four parts, and the results can be optimized in different aspects by using multiple loss functions for constraints, so as to ensure that the model has high accuracy; the improved convolutional neural network model can achieve real-time detection while ensuring accuracy, meeting the actual production needs of the factory. However, the method is only a solder joint defect detection method based on artificial intelligence, and cannot provide early prediction of the failure mechanism of solder joints in the manufacturing process. Therefore, more advanced and automated systems and methods are needed to improve the detection and prevention of welding defects.

[0014] The technology of Chinese patent document number CN114372949 A discloses a PCB surface defect detection method based on an improved YOLOv5 algorithm. The method comprises the following steps: preprocessing the original PCB data set, establishing the network structure of the YOLOv5 algorithm; determining the YOLOv5 network loss function and performance evaluation index according to GioU; the method improves the neck of the network structure, adds adaptive feature fusion (ASFF), makes full use of features of different scales, and enhances the small target detection performance. The method of improving the final prediction bounding box; adopting target box weighted fusion (WBF) instead of non-maximum suppression (NMS) as the method of selecting the final predicted bounding box; according to the improved YOLOv5 algorithm network structure, the network structure is trained using the idea of ​​transfer learning; the sample data of the PCB surface defect to be detected is input into the PCB surface defect detection model trained based on the improved YOLOv5 algorithm; and the location and category information of the PCB surface defect to be detected is output. The invention realizes high efficiency and high precision of PCB surface defect detection through instance detection. Although the method provides efficient and accurate detection accuracy, it can only detect solder joint defects. Therefore, there is still a need for a solder joint failure mechanism prediction system and method.

[0015] The technology of Chinese patent document number CN115906573A discloses a PCB service life analysis method based on reliability analysis, including the following steps: according to the expected use environment and design information of the circuit board card, obtain the main faults of the circuit board card; build a CAD simulation model according to the design file of the circuit board card; obtain the natural frequency of the circuit board card, and perform simple harmonic vibration test and random vibration test on the CAD simulation model under different working conditions according to the natural frequency to obtain fault data; perform thermal analysis to obtain the failure data of the circuit board under different simulated working conditions within the expected life cycle; perform solder joint fatigue analysis and electroplating perforation fatigue analysis to obtain the solder joint fatigue failure data and electroplating perforation fatigue failure data of the circuit board card; obtain the expected life of the circuit board card according to the fault information vector, fault data, failure data, solder joint fatigue failure data and electroplating perforation fatigue failure data of the circuit board card. This improves the accuracy of fault diagnosis of the circuit board card and reduces the fault frequency of the circuit board card during use. However, the method is based on CAD simulation and thermal analysis for fault mode diagnosis and remaining service life prediction of PCB. Therefore, the method does not provide defect detection of PCB solder joints. Summary of the invention

[0016] The purpose of the present invention is to overcome the above-mentioned shortcomings and provide a printed circuit board solder joint failure mechanism prediction system and method based on artificial intelligence, which can solve various problems caused by the inability to effectively detect and predict solder joints in the prior art.

[0017] In order to achieve the above-mentioned purpose, the technical solution adopted by the present invention is: a printed circuit board solder joint failure mechanism prediction system based on artificial intelligence, characterized in that it includes:

[0018] PCB soldering area detection module, including a PCB defect classification model, which is used to capture RGB images of the PCB soldering area through a camera and detect the PCB soldering area based on a deep learning model;

[0019] The detection module of the thermal imaging camera is used to detect the PCB failure mechanism prediction module under high current;

[0020] The PCB failure mechanism prediction module is used to analyze according to the PCB defect classification model, obtain the thermal temperature of the PCB welding area, and obtain the inference result to predict the failure mechanism solder joint.

[0021] Furthermore, the PCB welding area detection module includes a camera-based PCB welding area detection model.

[0022] Furthermore, the PCB welding area detection model adopts the YOLOv4 model, and the PCB defect classification model applies the ResNet50 network structure.

[0023] Furthermore, the PCB defect classification model is a pre-trained model.

[0024] Furthermore, the PCB welding area detection module includes: a PCB board, a thermal imaging camera, an electronic load and a DC power supply, which are connected to each other.

[0025] Furthermore, the PCB soldering area detection module is configured to perform RGB image annotation and predict and monitor temperature changes of PCB solder joints to classify defects.

[0026] Furthermore, the PCB soldering area detection module is further configured to conduct experiments to provide information for training the PCB failure mechanism prediction module.

[0027] Another object of the present invention is to provide a method for predicting the failure mechanism of solder joints of printed circuit boards based on artificial intelligence, characterized in that it includes:

[0028] Capture RGB images of the PCB soldering area using a thermal imaging camera;

[0029] Detect PCB soldering areas based on deep learning models and extract PCB defect classification models;

[0030] According to the PCB failure mechanism prediction module, analysis is performed to obtain the thermal temperature of the PCB welding area, and the reasoning result is obtained to predict the failure mechanism solder point.

[0031] Furthermore, the PCB soldering area is detected based on the deep learning model, and the PCB defect classification model is extracted, including:

[0032] Use cameras positioned at different angles and distances to capture images of the PCB soldering area;

[0033] Collect image datasets of PCB soldering areas containing various soldering defects;

[0034] Data annotation of the images by labeling bounding boxes around the weld area to show detected defects;

[0035] Augmenting data by generating new variations of images;

[0036] Input the image into the YOLOv4 model of the PCB solder area detection model;

[0037] Apply the ResNet50 network structure to the PCB defect classification model to classify images into different types of defects;

[0038] The inference result of the predicted label is output, showing the type of defect present in the image of the PCB soldering area.

[0039] Furthermore, the ResNet50 network structure is applied to the PCB defect classification model, including: training on an image dataset with labels showing defects in PCB solder joint images.

[0040] Further, the data annotation of the image by marking a bounding box around the welding area to show that a defect has been detected includes: generating an alarm or notification in real time when a defect is detected.

[0041] Further, the PCB defect classification model is analyzed to obtain the thermal temperature of the PCB soldering area, and the reasoning result is obtained to predict the fault mechanism solder joint, including:

[0042] Configure the experimental PCB circuit by connecting it to a power supply and an electronic load;

[0043] Set up thermal imaging cameras;

[0044] Apply stress loading to the experimental PCB circuit;

[0045] Capture thermal and RGB images to detect failure mechanisms by predicting and classifying thermal temperatures in PCB soldering areas.

[0046] Furthermore, thermal images and RGB images are collected to detect failure mechanisms by predicting and classifying the thermal temperature of the PCB soldering area, including: continuously monitoring the temperature changes in the PCB soldering area to detect any potential defects.

[0047] Further, the thermal temperature of the PCB soldering area is predicted and classified to detect the failure mechanism, including: when the temperature of the soldering area reaches 125° C., classifying the soldering area as defective and thermally failed.

[0048] Furthermore, analysis is performed based on the PCB failure mechanism prediction module, including: training the module through deep learning.

[0049] Further, training is performed on an image dataset with labels showing defects in PCB solder joint images, including: using a small number of PCB soldering images with known defect classifications during the training process.

[0050] Further, training is performed by a deep learning model, including: using a thermal image dataset collected during testing and stress loading of the PCB during the training process.

[0051] Further, training is performed through a deep learning model, including:

[0052] Input the PCB defect classification model using the ResNet50 network structure as the pre-trained model;

[0053] The failure mechanism prediction module is trained using the RGB images and corresponding temperature labels obtained from the failure mechanism detection.

[0054] Furthermore, the PCB defect classification model using the ResNet50 network structure as a pre-trained model is input, including:

[0055] Fine-tune the ResNet50 network structure on the PCB defect classification model;

[0056] The weight integration is achieved by taking the weighted average of the pre-trained ResNet50 network structure and the fine-tuned PCB defect classification model;

[0057] Fine-tuning detection on the failure mechanism dataset using average weights.

[0058] Further, according to the PCB failure mechanism prediction module, analysis is performed to obtain the thermal temperature of the PCB welding area, and an inference result is obtained to predict that the failure mechanism solder joint is based on the thermal temperature of the PCB welding area and the known defect classification.

[0059] Further, the PCB failure mechanism prediction module is analyzed to obtain the thermal temperature of the PCB welding area, and the reasoning result is obtained to predict the failure mechanism solder joint, including: updating the reasoning result and outputting feedback.

[0060] Compared with the prior art, the present invention has the following advantages:

[0061] It is simple to implement and has multiple advantages in terms of accuracy, reduced steps, non-destructive, time saving, mass production, cost-effectiveness, automation, and greater flexibility to test all PCB samples with solder joints, etc. The AI-based model is a non-destructive process because it only requires a camera to monitor the solder joints and an AI model to monitor the solder joints to predict whether the solder exceeds Tg during the manufacturing process. The present system and method can quickly and accurately inspect all PCB solder joints on the production line, allowing manufacturers to identify potential problems and take corrective measures before any damage occurs. AI-based models are more suitable for large-scale production because they can quickly and accurately analyze data, so that corrective measures can be taken before large-scale product production. This AI-based model is a cost-effective solution because it does not require the expensive equipment and materials required by traditional experimental-based inspection methods. This makes it a more accessible solution for manufacturers of all sizes, allowing for greater flexibility and scalability in the production process. AI-based models can automate the process of identifying potential defects, thereby reducing the need for manual inspection and analysis, which can improve efficiency, reliability, and consistency. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The features of the present invention will be more readily understood and appreciated when the following detailed description is read in conjunction with the accompanying drawings of preferred embodiments of the present invention, in which:

[0063] Figure 1 A schematic diagram of a PCB solder joint defect detection module based on AI of the present invention is shown;

[0064] Figure 2 A schematic diagram of the present invention depicting possible failure mechanisms in solder joints due to temperature variation and CTE mismatch is shown;

[0065] Figure 3 A schematic diagram of a PCB failure mechanism test module under high current based on a thermal imaging camera of the present invention is shown;

[0066] Figure 4 The experimental device for detecting the failure mechanism of PCB welding in the present invention is shown;

[0067] Figure 5 An example of testing a PCB board with different soldering defects is shown;

[0068] Figure 6 The experimental results of the present invention for detecting the PCB welding failure mechanism are shown;

[0069] Figure 7 The training process of the PCB failure mechanism prediction module in the present invention is shown;

[0070] Figure 8The overall schematic diagram of PCB solder joint defect detection and failure prediction of the present invention is shown;

[0071] Fig. 9 The weight sets of ResNet50 derived from two pre-trained ResNet50 are shown;

[0072] Fig.10 The reasoning process of PCB failure mechanism prediction in the present invention is shown. DETAILED DESCRIPTION

[0073] As required, specific embodiments of the present invention are disclosed herein. However, it should be understood that the disclosed embodiments are only examples of the present invention, which can be implemented in a variety of different forms. Therefore, the specific structural and functional details disclosed herein should not be interpreted as limitations, but only as the basis of the claims. It should be understood that the drawings and their detailed description are not used to limit the present invention to the specific forms disclosed herein. On the contrary, the present invention covers all modifications, equivalents and alternatives that fall within the scope defined by the claims. As used throughout this application, the word "may" means optional (i.e., it means possible), rather than mandatory (i.e., it means must). Similarly, the words "include" and "comprise" are meant to include but are not limited to. In addition, unless otherwise mentioned, the word "one" means "at least one" and the word "multiple" means one or more. When using abbreviations or technical terms, these refer to the generally accepted meanings known in the art.

[0074] The present invention discloses a printed circuit board (PCB) solder joint failure mechanism prediction system 1000 based on artificial intelligence (AI), comprising: a camera-based PCB soldering area detection module (1002), comprising a PCB defect classification model (104); a thermal imaging camera-based detection module (302), used to detect PCB soldering failure mechanisms under high current; and a PCB failure mechanism prediction module (1004), wherein the camera-based PCB soldering area detection module (1002) and the PCB failure mechanism prediction module (1004) use a deep learning model to identify PCB solder joints, classify defects and predict failure mechanisms.

[0075] In a preferred embodiment of the present invention, the camera-based PCB soldering area detection module (1002) further includes a PCB soldering area detection model (102). The PCB soldering area detection model (102) uses a YOLOv4 model, and the PCB defect classification model (104) uses ResNet50.

[0076] In a preferred embodiment of the present invention, the PCB defect classification model (104) is a pre-trained model.

[0077] In a preferred embodiment of the present invention, the thermal imaging camera-based detection module (302) includes a PCB board, a thermal imaging camera (402), an electronic load, and a direct current (DC) power supply. The thermal imaging camera-based detection module (302) is configured to perform RGB image annotation and predict and monitor temperature changes of PCB solder joints to classify defects.

[0078] In a preferred embodiment of the present invention, the thermal imaging camera-based detection module (302) is further configured to conduct experiments to provide information for training the PCB failure mechanism prediction module (1004).

[0079] The present invention also discloses a method for predicting the welding failure mechanism of a printed circuit board (PCB) based on artificial intelligence (AI), comprising the following steps: capturing RGB images; detecting the PCB welding area with a deep learning model; extracting the PCB welding area and feeding it to a PCB failure mechanism prediction module (1004) for predicting thermal temperature; and obtaining inference results to detect and predict the failure mechanism solder joint. PCB failure mechanism prediction module

[0080] In a preferred embodiment of the present invention, the detection of PCB welding area includes the following steps: using a camera positioned at different angles and distances to capture images of the PCB; collecting image datasets of PCB welding areas containing various welding defects; annotating the images by marking bounding boxes around the welding areas to show defects; enhancing the data by generating new changes in the images; inputting the images into the YOLOv4 model of the PCB welding area detection model (102); applying ResNet50 to the PCB defect classification model (104) to classify the images into different types of defects; and outputting the inference results of the predicted labels, showing the types of defects in the PCB welding area images. The further application of ResNet50 to the PCB defect classification model (104) includes the step of training the model on an image dataset with labels showing defects in the PCB solder joint images.

[0081] In a preferred embodiment of the present invention, the method further comprises the step of generating a real-time alarm or notification when a defect is detected in a solder joint of the printed circuit board.

[0082] In a preferred embodiment of the present invention, extracting the PCB soldering area and feeding it to the PCB failure mechanism prediction module (1004) includes the following steps: setting up the experimental PCB circuit by connecting it to a power supply and an electronic load; setting up a thermal imaging camera; applying a stress load to the circuit; acquiring thermal images and RGB images; and detecting the failure mechanism by predicting and classifying the thermal temperature of the PCB soldering area. The method also includes the step of continuously monitoring the temperature changes of the PCB soldering area to detect any potential defects. The prediction, classification and monitoring of the thermal temperature changes also include the step of classifying the soldering area as a defect and a thermal failure when the temperature of the soldering area reaches 125°C.

[0083] In a preferred embodiment of the present invention, the method further comprises: wherein the method further comprises the step of training the PCB failure mechanism prediction module (1004). wherein the training of the deep learning model further comprises the step of using a small number of PCB welding images of known defect classifications during the training process. wherein the training of the PCB failure mechanism prediction module (1004) further comprises using a thermal image dataset collected during testing and stress loading of the PCB during the training process.

[0084] In a preferred embodiment of the present invention, the method comprises the following steps: inputting a PCB defect classification model (104) using ResNet50 as a pre-trained model; and training a PCB failure mechanism prediction module (1004) using RGB images and corresponding temperature labels obtained from fault mechanism detection.

[0085] In a preferred embodiment of the present invention, the method further comprises the following steps: fine-tuning ResNet50 on the PCB defect classification model; achieving weight integration by weighted averaging the weights of the pre-trained ResNet50 and the fine-tuned PCB defect classification model (104); and using the average weight to fine-tune the detection of the fault mechanism dataset.

[0086] In a preferred embodiment of the present invention, the prediction and classification of failure mechanisms are based on the thermal temperature of the PCB soldering area and known defect classifications.

[0087] In a preferred embodiment of the present invention, the artificial intelligence-based printed circuit board (PCB) solder joint failure mechanism prediction method further includes the step of simultaneously updating the output feedback of the PCB failure mechanism prediction module (1004).

[0088] Example Embodiments

[0089] The camera-based PCB soldering area detection module is specifically a schematic diagram of an AI-based PCB solder point defect detection module as shown below: Figure 1As shown. The AI-based PCB solder joint defect detection model includes an RGB image capture system, PCB soldering image data collection, data annotation, data enhancement, a PCB soldering area detection module (102), a PCB defect classification model (104) and inference results. The process starts with an image capture system that collects different soldering images. The images of the PCB are captured using a camera positioned to capture PCB images from different angles and distances. At least one camera can be mounted on a bracket or an automatic arm to facilitate movement and positioning. Then, a large number of image datasets are collected, which contain PCB soldering areas with various soldering defects (such as bridging, cold solder joints, and insufficient solder). Images from the large dataset are marked with bounding boxes around the soldering areas to indicate which PCB soldering areas are good or defective during data annotation. The module increases the size of the dataset by applying transformations through data enhancement, thereby further generating new variants of the image. For example, the transformations include but are not limited to rotation, flipping, and scaling.

[0090] The image is input to a YOLOv4 model that has been trained on a large dataset of labeled images in a PCB soldering area detection module (102). The trained YOLOv4 model is fine-tuned on the soldering area dataset to improve performance by adjusting the model's weights for soldering area detection. After the model is trained, it can be applied to new images of the PCB to automatically detect soldering areas. The image of the PCB soldering area is further classified into different types of defects by applying ResNet50 to a PCB defect classification model (104). The model can be trained on an image dataset with labels indicating the type of defects present in the soldering area. Therefore, the output of the PCB defect classification model (104) is a predicted label indicating the type of soldering defect present in the input image of the PCB soldering area. The model takes a soldering area image as input and outputs a specific label corresponding to the soldering defect classification. The output of the process is a list of defects and their locations on the PCB. The process is efficient and accurate and can significantly reduce the time and cost of manual PCB inspection. It is widely used in manufacturing and quality control processes and has the potential to improve the reliability and performance of electronic products. Therefore, PCB soldering defects can be detected early on the production line.

[0091] The thermal temperature failure mechanism of solder joints caused by high current use is a common problem in electronic devices. When the device experiences high current, the heat generated causes the temperature of the solder joint to increase. If the temperature exceeds the glass transition temperature (Tg) of the material, the solder joint will soften and lose rigidity, causing the solder joint to crack or fail. This may lead to loss of electrical contact and potential malfunction or failure of the device. Therefore, it is critical to detect the temperature of the solder joint under high current use to ensure that the solder joint does not exceed its Tg and remains within the expected operating temperature range. An experimentally based solder failure mechanism has been developed in the present invention for early detection of PCB solder failure mechanisms by allowing manufacturers to identify potential problems and take corrective measures before any damage occurs. In order to ensure the reliability and safety of electronic devices, sampling strategies need to be implemented when testing solder joints. These strategies include simple random sampling, stratified random sampling, cluster sampling, and systematic sampling. By using these methods, representative samples of solder joints can be tested to detect potential problems and improve the overall reliability of electronic devices. Figure 2 A schematic depicting possible failure mechanisms in solder joints due to temperature variation and CTE mismatch (e.g., Case 1: Plated Through Hole Soldering, Case 2: Surface Mount Technology Soldering, etc.) resulting from CTE mismatch between component substrate and PCB is shown. First, when the temperature of the PCB changes above its Tg, the material becomes more flexible and its CTE increases. Second, the thermal expansion and contraction of the two materials become mismatched, causing stresses in the solder joints that connect the component to the PCB. Ultimately, the solder joint may fail.

[0092] like Figure 3 As shown, a detection module (302) based on a thermal imaging camera is disclosed for detecting PCB welding faults under high current. First, a PCB circuit is set up and a DC power supply and an electronic load are connected. Then an image capture system, in particular a thermal imaging camera, is set up. Then the position of the thermal camera is designed and positioned on the PCB assembly and around the PCB solder joints. Then, a stress load of a large current of 40A, 50A, and 55A is applied to the circuit every 20 minutes, and the change of the current over time is recorded. The mechanism further collects thermal images and RGB images. Subsequently, the temperature changes on the thermal images of different welding positions are checked and recorded. The data obtained is input into the fault detection mechanism for further analysis to classify the defects.

[0093] After the PCB completes the welding process with the required electronic components on the production line, a reliability test is performed to evaluate whether the heat generated by the solder joint exceeds the measured temperature of about 130°C under high current conditions above 40A. The temperature limit of 130°C is to avoid the solder joint from exceeding the glass transition temperature (Tg), which is an important parameter for identifying the PCB welding failure mechanism. The grading value set by the present invention, 125°C for PCB welding failure mechanism detection is about 5°C lower than the Tg (130°C) value of the PCB. This difference is necessary to promote the subsequent work of early detection of welding defects based on artificial intelligence and prediction of solder joint failures based on artificial intelligence. Finally, a failure mechanism detection is performed to predict and classify the thermal temperature of the PCB welding area. Therefore, the thermal failure mechanism used in the present invention is simply defined as follows: if the recorded temperature does not reach 125°C, the welding area is classified as normal. Otherwise, the welding area is considered to have a thermal failure and is classified as defective.

[0094] Figure 4 Figure 2 shows the experimental setup used to detect PCB soldering failure mechanisms. Figure 4 As shown in Figure 1, a thermal imaging camera, a high current DC power supply and an electronic load are used to detect the failure mechanisms of different soldering defects. For example, eight PCB boards were tested, two of which were normal and the other six had different soldering defects, such as Figure 5 In the experiment, each PCB board is alternately kept at a DC current of 40A, 50A, and 55A for 20 minutes, and then the maximum thermal temperature of the corresponding solder joint is measured using a thermal imaging camera (402).

[0095] PCB boards are control circuit boards used in high-power electric tools such as sanders and electric drills. PCB samples either have no soldering defects or have soldering defects at the interconnection points. The thermal imaging camera (402) is a measuring device such as Figure 4 As shown, the temperature distribution on the surface of an object can be captured without contacting the object. A thermal imaging camera (402) is used to record the thermal changes and profiles of different solder joints. In addition, the use of electronic loads has also become a common practice for testing power supplies. Therefore, electronic loads are used in various sizes and applications, from low power consumption to high power consumption. They provide resistance to draw current from a DC power supply, so that devices such as batteries, solar cells, electronic components and portable chargers can be tested. The DC power supply provides a DC voltage to power the device under test (such as a circuit board or electronic product). It is correspondingly used to provide 40A to 60A of current to the circuit of the PCB sample.

[0096] Figure 6The experimental results of detecting PCB welding failure mechanism are shown, which can effectively detect the early failure mechanism of solder joints. Therefore, the experimental-based thermal imaging camera of the present invention provides early detection of PCB welding failure mechanism under high current usage conditions. The detection of PCB welding failure mechanism uses the above-mentioned deep learning model to improve the quality and reliability of PCB by identifying and solving potential manufacturing defects, diagnosing problems with existing PCBs, predicting potential failure mechanisms, and providing feedback for product design and improvement. In addition, RGB image annotation is performed to train the PCB failure mechanism prediction module (1004).

[0097] A detection module (302) based on a thermal imaging camera is used for PCB soldering failure mechanisms, so that the solder joints of the circuit passing through the PCB sample can have different current levels. The temperature difference between the areas around the solder joints of normal samples and defective samples can be checked. The temperature difference between the areas around the solder joints of normal samples and defective samples can be checked. Software can be used to capture temperature, current and voltage changes in solder joint images. The present invention uses a feature recognition system to distinguish the differences between the solder joint images of normal samples and defective samples in order to detect specific failure mechanisms early. Therefore, the present invention can find an image-based solder joint reliability detection model.

[0098] The present invention further discloses a camera-based capture system with an AI model for predicting failure mechanisms in solder joints. The system uses computer vision techniques to capture images of solder joints and analyze their appearance to identify potential defects or areas of concern. The AI ​​model is trained on a dataset of solder joint images with known defects and failures, enabling it to learn patterns and features that indicate potential problems. By analyzing the appearance of solder joints in real time, the AI ​​model can provide early warnings of potential failures, enabling proactive maintenance and repairs.

[0099] The ResNet50 soldering defect classification model is used as a pre-trained model to predict PCB failure mechanisms. The RGB images obtained from the fault mechanism detection and the corresponding temperature labels (i.e., the highest temperature recorded by the label) are used to train the fault mechanism prediction module. Figure 7 The training process of the PCB failure mechanism prediction module (1004) is shown. The PCB failure mechanism detection system provides temperature labels for solder joint images as additional input features for training the failure mechanism prediction module. The temperature labels can be obtained through thermal imaging or other temperature sensing technologies and can provide valuable information about the thermal behavior of the solder joints. However, applying the temperature labels is optional because the model can learn to detect and predict potential failure mechanisms based only on the RGB image input.

[0100] Limited data of RGB images with thermal information are collected and applied to the soldering failure mechanism detection system to fine-tune the parameters of ResNet to perform PCB failure mechanism prediction. Figure 8 An overall schematic diagram of PCB solder joint defect detection and failure prediction of the present invention is shown.

[0101] In addition, the present invention can improve the performance of PCB failure mechanism prediction by applying a pre-trained PCB defect detection model for transfer learning, which significantly reduces the amount of data and training time required. This is because the features and patterns learned by the pre-trained model are suitable for detecting PCB welding defects and predicting welding failure mechanisms. Due to the extremely small number of training images, a special fine-tuning method is required. The present invention tested two training methods. The first method is to pre-train the model using a modified version of weight space integration for fine-tuning (WISE-FT). In WISE-FT, it can maintain the accuracy and robustness of the pre-trained model even in the case of distribution changes. However, the pre-trained model in the present invention uses ImageNet pre-trained ResNet and fine-tunes it on the PCB defect classification task to obtain a model with the same domain as the target domain. PCB defect classification is trained on images of the same type of printed circuit boards with the same top view. Enhanced modeling capabilities for PCB image feature extraction. Afterwards, the weights of the ImageNet pre-trained ResNet and the fine-tuned PCB defect classification model are integrated by averaging the weights. The average weights are then used to robustly fine-tune the PCB failure mechanism dataset. Fig. 9 The weights of ResNet50 derived from two pre-trained ResNet50s are shown, which makes it robust to fine-tuning.

[0102] Table 1

[0103]

[0104] Table 1 shows the confusion matrix of the PCB failure mechanism prediction module (1004) using the modified WISE-FT model. The evaluation indicators of the failure mechanism prediction indicate that the AI ​​model performs well overall. The evaluation indicators of the failure mechanism prediction indicate that the AI ​​model performs well overall. The F1 score is 0.9091, indicating that the accuracy of predicting the failure mechanism is high. The sensitivity of 1.0 indicates that the AI ​​model correctly identified all failure mechanism instances, indicating that its ability to detect failures is strong. The precision of 0.8333 means that among the predicted failure mechanisms, 83.33% are actually true positives, which indicates a fairly good level of accuracy. The accuracy of 0.8571 indicates that the AI ​​model correctly classified 85.71% of all instances, regardless of their positive or negative nature. Although the Matthews Correlation Coefficient (MCC) is 0.645, indicating that there is a moderate correlation between the predicted classification and the actual observations, the overall performance of the AI ​​model is quite good based on other indicators. Fig.10 The reasoning process of the PCB failure mechanism prediction module (1004) is shown. In the reasoning process of failure mechanism prediction, only RGB images are needed to predict the potential failure mechanisms of PCB solder joints. The trained model can use RGB images as input and use the information learned during training to predict potential failure mechanisms. After capturing the RGB images, an AI-based PCB soldering area detection module (102) is utilized, in which the machine learning model can accurately identify and locate the solder joint areas on the PCB. Once the solder joint areas of the PCB are identified and located, these areas are extracted and fed to the PCB failure mechanism prediction module (1004) for thermal temperature prediction. Subsequently, inference results are obtained to detect and predict the failure mechanisms of solder joints. By identifying the above-mentioned thermal temperatures early, manufacturers can take corrective measures to improve the quality of the PCB, thereby improving the overall quality and reliability of the product. Therefore, the PCB failure mechanism prediction module (1004) in the present invention can help manufacturers identify and resolve potential solder thermal failures before they occur, thereby enhancing the overall reliability and performance of their products.

[0105] The AI-based PCB solder joint failure mechanism prediction system of the present invention has multiple advantages over experimental-based PCB solder joint failure mechanism detection technology in terms of accuracy, reduced steps, non-destructiveness, time saving, mass production, cost-effectiveness, automation, and greater flexibility to test all PCB samples with solder joints, etc. The AI-based model is a non-destructive process because it only requires a camera to monitor the solder joints and an AI model to monitor the solder joints to predict whether the solder exceeds Tg during the manufacturing process. The present system and method can quickly and accurately inspect all PCB solder joints on the production line, allowing manufacturers to identify potential problems and take corrective measures before any damage occurs. AI-based models are more suitable for large-scale production because they can quickly and accurately analyze data, so that corrective measures can be taken before large-scale product production. This AI-based model is a cost-effective solution because it does not require the expensive equipment and materials required by traditional experimental-based detection methods. This makes it a more accessible solution for manufacturers of all sizes, allowing for greater flexibility and scalability in the production process. AI-based models can automate the process of identifying potential defects, thereby reducing the need for manual inspection and analysis, which can improve efficiency, reliability and consistency.

[0106] The above explanation of the present invention is not limited to the aforementioned embodiments and drawings, and it is obvious to those skilled in the art that various substitutions, modifications and changes may be made without departing from the scope of the present invention.

Claims

1. A printed circuit board solder joint failure mechanism prediction system based on artificial intelligence, characterized in that: include: PCB soldering area detection module, including a PCB defect classification model, which is used to capture RGB images of the PCB soldering area through a camera and detect the PCB soldering area based on a deep learning model; The detection module of the thermal imaging camera is used to detect the PCB failure mechanism prediction module under high current; The PCB failure mechanism prediction module is used to analyze according to the PCB defect classification model, obtain the thermal temperature of the PCB welding area, and obtain the inference result to predict the failure mechanism solder joint.

2. The artificial intelligence-based printed circuit board solder joint failure mechanism prediction system according to claim 1 is characterized in that: The PCB welding area detection module includes a camera-based PCB welding area detection model.

3. The artificial intelligence-based printed circuit board solder joint failure mechanism prediction system according to claim 2 is characterized in that: The PCB welding area detection model adopts the YOLOv4 model, and the PCB defect classification model applies the ResNet50 network structure.

4. The artificial intelligence-based printed circuit board solder joint failure mechanism prediction system according to claim 3 is characterized in that: The PCB defect classification model is a pre-trained model.

5. The artificial intelligence-based printed circuit board solder joint failure mechanism prediction system according to claim 1 is characterized in that: The PCB welding area detection module includes: a PCB board, a thermal imaging camera, an electronic load and a DC power supply, which are connected to each other.

6. The artificial intelligence-based printed circuit board solder joint failure mechanism prediction system according to claim 5, characterized in that: The PCB soldering area detection module is configured to perform RGB image annotation and predict and monitor temperature changes of PCB solder joints to classify defects.

7. The artificial intelligence-based printed circuit board solder joint failure mechanism prediction system according to claim 6, characterized in that: The PCB soldering area detection module is further configured to conduct experiments to provide information for training the PCB failure mechanism prediction module.

8. A method for predicting the failure mechanism of solder joints in printed circuit boards based on artificial intelligence, characterized in that: include: Capture RGB images of the PCB soldering area through a camera; Detect PCB soldering areas based on deep learning models and extract PCB defect classification models; According to the PCB failure mechanism prediction module, the thermal temperature of the PCB welding area is obtained, and the inference results are obtained to predict the failure mechanism solder joint.

9. The artificial intelligence-based printed circuit board solder joint failure mechanism prediction method according to claim 8, characterized in that: Detect PCB soldering areas based on deep learning models and extract PCB defect classification models, including: Use cameras positioned at different angles and distances to capture images of the PCB soldering area; Collect image datasets of PCB soldering areas containing various soldering defects; Data annotation of the images by labeling bounding boxes around the weld area to show detected defects; Augmenting data by generating new variations of images; Input the image into the YOLOv4 model of the PCB solder area detection model; Apply the ResNet50 network structure to the PCB defect classification model to classify images into different types of defects; The inference result of the predicted label is output, showing the type of defect present in the image of the PCB soldering area.

10. The artificial intelligence-based printed circuit board solder joint failure mechanism prediction method according to claim 9, characterized in that: The ResNet50 network structure is applied to the PCB defect classification model, including: training on an image dataset with labels showing defects in PCB solder joint images.

11. The artificial intelligence-based printed circuit board solder joint failure mechanism prediction method according to claim 9, characterized in that: The image is annotated with data by marking a bounding box around the weld area to indicate that a defect has been detected, including: generating an alarm or notification in real time when a defect is detected.

12. The artificial intelligence-based printed circuit board solder joint failure mechanism prediction method according to claim 8, characterized in that: According to the PCB defect classification model, the thermal temperature of the PCB soldering area is obtained, and the reasoning result is obtained to predict the failure mechanism solder joint, including: Configure the experimental PCB circuit by connecting it to a power supply and an electronic load; Set up thermal imaging cameras; Apply stress loading to the experimental PCB circuit; Capture thermal and RGB images to detect failure mechanisms by predicting and classifying thermal temperatures in PCB soldering areas.

13. The artificial intelligence-based printed circuit board solder joint failure mechanism prediction method according to claim 12, characterized in that: Capture thermal and RGB images to detect failure mechanisms by predicting and classifying thermal temperatures in PCB soldering areas, including: Continuously monitor temperature changes in PCB soldering areas to detect any potential defects.

14. The artificial intelligence-based printed circuit board solder joint failure mechanism prediction method according to claim 12 or 13, characterized in that: Predict and classify thermal temperatures of PCB soldering areas to detect failure mechanisms, including classifying soldering areas as defective and thermally failed when the temperature of the soldering area reaches 125°C.

15. The artificial intelligence-based printed circuit board solder joint failure mechanism prediction method according to claim 8, characterized in that: Performing analysis based on the PCB failure mechanism prediction module includes: training the module through deep learning.

16. The artificial intelligence-based printed circuit board solder joint failure mechanism prediction method according to claim 10, characterized in that: Trained on a dataset of images with labels showing defects in PCB solder joint images, including: A small number of PCB solder images with known defect classifications used during training.

17. The artificial intelligence-based printed circuit board solder joint failure mechanism prediction method according to claim 15, characterized in that: Training is performed through a deep learning model, including using a thermal image dataset collected during testing and stress loading of the PCB during the training process.

18. The artificial intelligence-based printed circuit board solder joint failure mechanism prediction method according to claim 15, characterized in that: Training through deep learning models, including: Input the PCB defect classification model using the ResNet50 network structure as the pre-trained model; The RGB images and corresponding temperature labels obtained from the failure mechanism detection are used to train the PCB failure mechanism prediction module.

19. The artificial intelligence-based printed circuit board solder joint failure mechanism prediction method according to claim 18, characterized in that: Input the PCB defect classification model using the ResNet50 network structure as the pre-trained model, including: Fine-tune the ResNet50 network structure on the PCB defect classification model; The weight integration is achieved by taking the weighted average of the pre-trained ResNet50 network structure and the fine-tuned PCB defect classification model; Fine-tuning detection on the failure mechanism dataset using average weights.

20. The artificial intelligence-based printed circuit board solder joint failure mechanism prediction method according to claim 8 or 12, characterized in that: According to the PCB failure mechanism prediction module, the thermal temperature of the PCB soldering area is obtained and the inference result is obtained to predict the failure mechanism of the solder joint based on the thermal temperature of the PCB soldering area and the known defect classification.

21. The artificial intelligence-based printed circuit board solder joint failure mechanism prediction method according to claim 8, characterized in that: According to the PCB failure mechanism prediction module, analysis is performed to obtain the thermal temperature of the PCB welding area, and the reasoning result is obtained to predict the failure mechanism solder point, including: updating the reasoning result and outputting feedback.

Images