Method and device for detecting welding quality of element and electronic equipment

Abstract

The invention provides an element welding quality detection method and device and electronic equipment, and mainly relates to the technical field of electronic manufacturing. The element welding quality detection method comprises the steps that the plate type of a target mainboard is recognized according to configuration data of the target mainboard; wherein the target mainboard is a mainboard which is detected by the AOI equipment and is suspected to be abnormal in element welding; acquiring a detection model and detection threshold data according to the board type of the target mainboard; and inputting the image data of the target mainboard into the detection model, so that the detection model performs element welding detection on the image data according to the detection threshold data, and outputs a welding detection result. According to the method, the element welding quality is automatically and intelligently detected based on the detection model, the mainboard with the suspected element abnormality detected by the AOI equipment is efficiently and accurately re-judged, the workload of manual re-judgment is reduced, the cost and time of manual re-judgment are reduced, and the detection efficiency of the element welding quality is improved.

Description

Technical Field

[0001] This application relates to the field of electronic manufacturing technology, and in particular to a method, apparatus and electronic device for detecting the welding quality of components. Background Technology

[0002] Surface Mount Technology (SMT) is widely used in the manufacturing process of electronic motherboards. SMT involves mounting electronic components onto the motherboard and soldering them using soldering materials to achieve electrical connections and mechanical fixation between the components and the motherboard. In SMT production lines, the inspection of component soldering quality is crucial, as soldering defects can affect the performance and reliability of the motherboard. Related technologies utilize Automatic Optical Inspection (AOI) equipment to inspect soldered motherboards and identify potential anomalies such as missing solder or poor solder joints. However, AOI equipment is prone to misjudgments or missed detections due to limitations in optical imaging quality, component surface reflectivity, the complexity of solder joint appearance, and detection algorithms. When AOI equipment detects a motherboard with suspected soldering anomalies, manual review is usually required to further confirm the accuracy of the inspection results. However, manual review is time-consuming and labor-intensive, reducing the efficiency of component soldering quality inspection. Summary of the Invention

[0003] This application provides a method, apparatus, and electronic device for detecting component soldering quality. Based on a detection model, it achieves automated and intelligent detection of component soldering quality, and efficiently and accurately re-judges motherboards suspected of component abnormalities detected by AOI equipment, thereby improving the detection efficiency of component soldering quality and at least partially solving the above-mentioned technical problems.

[0004] According to a first aspect of this application, a method for detecting component soldering quality is provided. The method includes: identifying the board type of the target motherboard based on configuration data of the target motherboard; wherein the target motherboard is a motherboard suspected of having component soldering abnormalities detected by an AOI device; acquiring a detection model and detection threshold data based on the board type of the target motherboard; inputting image data of the target motherboard into the detection model, so that the detection model performs component soldering detection on the image data according to the detection threshold data, and outputs a soldering detection result; wherein the soldering detection result is used to indicate the soldering abnormality state of the target motherboard.

[0005] Optionally, obtaining the detection model and detection threshold data based on the board type of the target motherboard includes: loading the detection model matching the board type of the target motherboard from a model database; wherein the model database includes model parameters of the detection model matching at least one of the board types; and reading the detection threshold data associated with the board type of the target motherboard from a knowledge database; wherein the knowledge database includes the detection threshold data associated with at least one of the board types.

[0006] Optionally, the welding inspection results include at least one of the following: component welding position data, component welding anomaly type, and component welding anomaly confidence level.

[0007] Optionally, the detection model includes a feature extraction module, a feature fusion module, and a welding detection module; the step of inputting the image data of the target motherboard into the detection model, so that the detection model performs component welding detection on the image data according to the detection threshold data and outputs the welding detection result, includes: inputting the image data of the target motherboard into the feature extraction module, so that the feature extraction module extracts component features from the image data; inputting the component features into the feature fusion module, so that the feature fusion module enhances the component features to obtain fused features; inputting the fused features into the welding detection module, so that the welding detection module performs component welding detection on the fused features according to the detection threshold data and outputs the welding detection result.

[0008] Optionally, the feature extraction module includes a solder joint attention unit, and the component features include small-sized features extracted by the solder joint attention unit.

[0009] Optionally, the detection model further includes a feature separation module; the method further includes: inputting the image data of the sample motherboard into the feature extraction module, so that the feature extraction module extracts the component features from the image data; inputting the component features into the feature separation module, so that the feature separation module separates the board type features from the component features according to a knowledge database; wherein, the knowledge database includes component types associated with at least one of the board types; inputting the component features into the feature fusion module, so that the feature fusion module enhances the component features to obtain the fused features; inputting the fused features into the welding detection module, so that the welding detection module performs component welding detection on the fused features according to the knowledge database and outputs the welding detection result; wherein, the knowledge database further includes detection threshold data associated with at least one of the board types; determining the training loss value of the detection model according to the board type features of the sample motherboard, the welding detection result, and the welding annotation data; adjusting the model parameters of the detection model to make the training loss value converge.

[0010] Optionally, determining the training loss value of the detection model based on the board type features of the sample motherboard, the welding inspection results, and the welding annotation data includes: determining the base loss value and false alarm loss value of the detection model based on the welding inspection results and the welding annotation data of the sample motherboard; determining anomaly frequency weight values ​​based on the welding inspection results, the welding annotation data, and the knowledge database of the sample motherboard, and determining the anomaly frequency loss value of the detection model in combination with the anomaly frequency weight values; wherein, the knowledge database includes the occurrence frequency of at least one component welding anomaly type; determining the board type difference penalty loss value based on the cosine similarity between the board type features of the sample motherboard; and determining the training loss value of the detection model based on the base loss value, the false alarm loss value, the anomaly frequency loss value, and the board type difference penalty loss value.

[0011] Optionally, the method further includes: updating the knowledge database based on the target motherboard; and updating the model parameters of the detection model based on the target motherboard and the updated knowledge database.

[0012] According to a second aspect of this application, a component soldering quality detection device is provided, the device comprising: a board type identification module, configured to identify the board type of a target motherboard based on configuration data of the target motherboard; wherein the target motherboard is a motherboard suspected of having component soldering abnormalities; a data acquisition module, configured to acquire a detection model and detection threshold data based on the board type of the target motherboard; and a quality detection module, configured to input image data of the target motherboard into the detection model, so that the detection model performs component soldering detection on the image data based on the detection threshold data, and outputs a soldering detection result; wherein the soldering detection result is used to indicate the soldering abnormality state of the target motherboard.

[0013] According to a third aspect of this application, an electronic device is provided, comprising: a memory having a computer program or instructions stored thereon; and a processor for executing the computer program or instructions in the memory to implement the component welding quality detection method as described above.

[0014] In summary, the technical solution provided in this application, after an AOI device detects a target motherboard with suspected component soldering abnormalities, identifies the board type of the target motherboard and obtains corresponding detection models and detection threshold data accordingly. The detection model then performs component soldering detection on the image data of the target motherboard based on the detection threshold data to obtain soldering detection results. This application implements an automated and intelligent method for detecting component soldering quality based on a detection model. It efficiently and accurately re-judges motherboards with suspected component abnormalities detected by AOI devices, effectively removing a large number of motherboards falsely reported by AOI devices as having component soldering abnormalities, reducing the workload of manual re-judgment, lowering the cost and time of manual re-judgment, and improving the efficiency of component soldering quality detection. Furthermore, the component soldering quality detection in this application uses a detection model that matches the board type of the target motherboard, enabling more targeted and accurate detection based on the board type, thus improving the accuracy of component soldering quality detection.

[0015] Other features and advantages of this application will be described in detail in the following detailed description section. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] To gain a more complete understanding of this application and its beneficial effects, the following description will be provided in conjunction with the accompanying drawings, wherein the same reference numerals in the following description denote the same parts.

[0018] Figure 1 This is a flowchart of a method for detecting the welding quality of components provided in an embodiment of this application;

[0019] Figure 2 This is a schematic diagram of a detection model provided in an embodiment of this application;

[0020] Figure 3 This is a schematic diagram of a solder joint attention unit provided in an embodiment of this application;

[0021] Figure 4 This is a schematic diagram of a false alarm suppression branch provided in an embodiment of this application;

[0022] Figure 5 This is a schematic diagram of a feature separation module provided in an embodiment of this application;

[0023] Figure 6 This is a schematic diagram of a hierarchical training strategy provided in an embodiment of this application;

[0024] Figure 7 This is a schematic diagram of a data layering logic provided in an embodiment of this application;

[0025] Figure 8 This is a schematic diagram illustrating the construction of a knowledge database according to an embodiment of this application;

[0026] Figure 9 This is a schematic diagram of a method for detecting the welding quality of components provided in an embodiment of this application;

[0027] Figure 10 This is a block diagram of a component welding quality detection device provided in an embodiment of this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the protection scope of this application.

[0029] Please see Figure 1 , Figure 1 This is a flowchart illustrating a method for detecting the welding quality of components according to an embodiment of this application. Figure 1 As shown, the method for detecting the welding quality of this component may include the following steps:

[0030] Step S100: Identify the board type of the target motherboard based on its configuration data; wherein, the target motherboard is the motherboard with suspected component soldering abnormalities detected by the AOI equipment;

[0031] Step S200: Based on the board type of the target motherboard, obtain the detection model and detection threshold data;

[0032] Step S300: Input the image data of the target motherboard into the detection model so that the detection model can perform component welding detection on the image data according to the detection threshold data and output the welding detection result.

[0033] This application embodiment is applied to AOI equipment inspection, mainly used to automatically and intelligently re-evaluate motherboards with potential solder joint defects such as missing components or cold solder joints based on the detection model when the AOI equipment identifies such defects. This reduces the cost and time of manual re-evaluation and improves the inspection efficiency of component soldering quality. Therefore, the above steps S100 to S300 can be executed when the AOI equipment detects a target motherboard, which is a motherboard with suspected component soldering defects detected by the AOI equipment. Component soldering defects include, but are not limited to, defects such as missing components, cold solder joints, component misalignment, and solder short circuits.

[0034] In step S100, the board type of the target motherboard is identified based on its configuration data. The configuration data indicates relevant information about the target motherboard and can be directly printed on it, such as as PCB (Printed Circuit Board) silkscreen markings; or it can be a file generated during the production phase of the target motherboard, such as as a CAM (Computer Aided Manufacturing) file. Based on the configuration data, the board type of the target motherboard can be identified. The board type includes, but is not limited to, power boards, decoder boards, and TCON (Timing Control) boards; the target motherboard can be any of these board types.

[0035] In step S200, based on the board type of the target motherboard, a detection model and detection threshold data for inspecting the component soldering quality of the target motherboard are obtained. The detection model can be implemented as an artificial intelligence (AI) model, such as YOLOv8 or RCNN (Region-based Convolutional Neural Networks). The detection threshold data can be threshold data that the detection model needs to refer to when inspecting the component soldering of the target motherboard, such as component diameter and its error range. In some embodiments, a model database and a knowledge database can be set up. The model database includes model parameters of detection models matched with at least one board type, and the knowledge database includes detection threshold data associated with at least one board type. Therefore, step S200 can include: loading a detection model matching the board type of the target motherboard from the model database; and reading the detection threshold data associated with the board type of the target motherboard from the knowledge database. Of course, the model database can also include the model structure of the detection model, and can load model parameters and model structure at the same time to realize the loading of the detection model; the knowledge database can also include component types, data preprocessing methods, process parameters, etc., for use in model training and other processes.

[0036] In step S300, the image data of the target motherboard is input into the detection model. The detection model performs component welding detection on the image data based on detection threshold data to obtain welding detection results. The welding detection results indicate the welding abnormality status of the target motherboard. In some embodiments, the welding detection results include at least one of the following: component welding position data, component welding abnormality type, and component welding abnormality confidence level; wherein, the component welding position data indicates the position of the component with welding abnormality in the target motherboard, the component welding abnormality type indicates the type of welding abnormality (e.g., cold solder joint, missing component, misalignment, etc.), and the component welding abnormality confidence level indicates the probability that the detected welding abnormality is a true abnormality. The detection model can extract component features from the image data of the target motherboard, process the component features based on detection threshold data, and output welding detection results, realizing automatic and intelligent re-judgment of component welding quality.

[0037] In summary, the technical solution provided in this application, after an AOI device detects a target motherboard with suspected component soldering abnormalities, identifies the board type of the target motherboard and obtains corresponding detection models and detection threshold data accordingly. The detection model then performs component soldering detection on the image data of the target motherboard based on the detection threshold data to obtain soldering detection results. This application implements an automated and intelligent method for detecting component soldering quality based on a detection model. It efficiently and accurately re-judges motherboards with suspected component abnormalities detected by AOI devices, effectively removing a large number of motherboards falsely reported by AOI devices as having component soldering abnormalities, reducing the workload of manual re-judgment, lowering the cost and time of manual re-judgment, and improving the efficiency of component soldering quality detection. Furthermore, the component soldering quality detection in this application uses a detection model that matches the board type of the target motherboard, enabling more targeted and accurate detection based on the board type, thus improving the accuracy of component soldering quality detection.

[0038] In some embodiments, to improve the accuracy of welding inspection results, clearer image data can be input into the inspection model. Therefore, the above-mentioned method for detecting component welding quality can further include: acquiring image data of the target motherboard; preprocessing the image data to input the preprocessed image data into the inspection model. Specifically, when the AOI device detects a target motherboard suspected of having component welding abnormalities, it automatically captures a high-definition image of the corresponding area (e.g., resolution ≥ 0.02 mm / pixel); a high-resolution industrial camera (≥ 5 MP) paired with a ring light source can be used to acquire image data to ensure clear imaging of solder joints and components; the processing platform in this embodiment supports the GigE Vision protocol to achieve millisecond-level image transmission with the AOI device.

[0039] Data preprocessing can optimize the quality of acquired image data. In some embodiments, data preprocessing includes at least one of the following: noise suppression processing, local enhancement processing, and normalization processing. Noise suppression processing can employ asymmetric Gaussian filtering to preserve high-frequency details of solder joints while suppressing background noise (such as PCB texture interference); local enhancement processing can adaptively stretch the contrast only on the solder joint area (solder joints, component leads); normalization processing can unify image pixel values ​​to the range [0, 1] to improve model stability. The specific methods of data preprocessing and the parameters used, such as the contrast enhancement intensity of electrolytic capacitor solder rings, can be obtained from a knowledge database.

[0040] Please see Figure 2 , Figure 2 This is a schematic diagram of a detection model provided in an embodiment of this application. Figure 2As shown, in some embodiments, the detection model described above may include a feature extraction module 210, a feature fusion module 220, and a welding detection module 230. Based on this, step S300 may include: inputting image data of the target motherboard into the feature extraction module 210, so that the feature extraction module 210 extracts component features from the image data; inputting the component features into the feature fusion module 220, so that the feature fusion module 220 enhances the component features to obtain fused features; and inputting the fused features into the welding detection module 230, so that the welding detection module 230 performs component welding detection on the fused features according to the detection threshold data and outputs the welding detection result.

[0041] Since the detection model obtained in this embodiment matches the board type of the target motherboard, the component features extracted by the feature extraction module 210 from the image data of the target motherboard can include not only some general features, but also features specific to the board type of the target motherboard. This makes the component features more accurate, thereby further improving the accuracy of the welding detection results. To enhance feature extraction of small targets (such as 0402 resistance solder joints), such as... Figure 2 As shown, in some embodiments, the feature extraction module 210 includes a solder joint attention unit 211, which may be located at the end of the feature extraction module 210, so that the component features include small-sized features extracted by the solder joint attention unit. The solder joint attention unit 211 may be based on a channel attention mechanism, such as... Figure 3 As shown.

[0042] The welding inspection module 230 can output component welding position data, component welding anomaly type, and component welding anomaly confidence level. For example... Figure 2 As shown, the welding detection module 230 can include three branches: a bounding box loss branch, a classification loss branch, and a false alarm suppression branch. The bounding box loss branch outputs component welding position data, the classification loss branch outputs component welding anomaly types, and the false alarm suppression branch outputs welding anomaly confidence levels. The false alarm suppression branch can be implemented based on a binary classification sub-network, such as... Figure 4 As shown. It should be understood that, Figure 2 The welding inspection module 230 includes three branches. Each branch includes a bounding box loss branch, a classification loss branch, and a false alarm suppression branch. Different branches can process features of different sizes and dimensions to output corresponding detection results, thereby improving the accuracy of the detection results.

[0043] In order for the detection model to learn the differences in features between different plate types, such as Figure 2As shown, in some embodiments, the detection model further includes a feature separation module 240, which is used to output vectors of different board types during the training phase. Based on this, the training method for the detection model, or the aforementioned method for detecting component welding quality, further includes: inputting image data of the sample motherboard into a feature extraction module 210, so that the feature extraction module 210 extracts component features from the image data; inputting the component features into a feature separation module 240, so that the feature separation module 240 separates board type features from the component features according to a knowledge database; inputting the component features into a feature fusion module 220, so that the feature fusion module 220 enhances the component features to obtain fused features; inputting the fused features into a welding detection module 230, so that the welding detection module 230 performs component welding detection on the fused features according to a knowledge database and outputs welding detection results; determining the training loss value of the detection model based on the board type features of the sample motherboard, the welding detection results, and the welding annotation data; adjusting the model parameters of the detection model to make the training loss value converge. When the training loss value converges, the detection model that has completed training is obtained.

[0044] The sample motherboard is used to train the detection model. This sample motherboard can be a motherboard from the AOI device's historical false alarms or missed detections. Multiple sample motherboards can be used, and their types can be varied, thereby improving the robustness and generalization of the detection model. Each sample motherboard can be annotated to obtain welding annotation data. This welding annotation data includes, but is not limited to: basic annotations such as component bounding boxes, component types, and welding anomaly types, as well as board type, false alarm labels, and frequently false alarm components.

[0045] The knowledge database includes component types associated with at least one board type. The feature separation module 240 can separate board type features, i.e., features unique to the board type of the sample board, from the component features of the sample board based on the knowledge database. For example... Figure 5 As shown, the feature separation module 240 can be connected to the feature extraction module 210 to separate the board shape features from the component features output by the feature extraction module 210.

[0046] In some embodiments, determining the training loss value of the detection model based on the board shape features, welding inspection results, and welding annotation data of the sample motherboard includes: determining the base loss value and false alarm loss value of the detection model based on the welding inspection results and welding annotation data of the sample motherboard; determining the anomaly frequency weight value based on the welding inspection results, welding annotation data, and knowledge database of the sample motherboard, and determining the anomaly frequency loss value of the detection model in combination with the anomaly frequency weight value; determining the board shape difference penalty loss value based on the cosine similarity between the board shape features of the sample motherboard; and determining the training loss value of the detection model based on the base loss value, false alarm loss value, anomaly frequency loss value, and board shape difference penalty loss value.

[0047] Typically, the construction of the loss function considers a base loss value and a false positive loss value based on the output of the detection model. These base loss values ​​and false positive loss values ​​are determined by comparing the welding inspection results with the welding annotation data. The base loss value is used to indicate the differences between bounding boxes (i.e.,...). Figure 2 The differences between the component weld position data output by the middle bounding box loss branch and the bounding boxes annotated in the weld annotation data, as well as the differences between component weld anomaly types (i.e., Figure 2 The difference between the component welding anomaly type output by the classification loss branch and the anomaly type in the welding annotation data, and the false positive loss value is used to indicate the difference between the confidence levels of component welding anomalies (i.e., Figure 2 (Binary classification loss with false alarm suppression branch).

[0048] In constructing the loss function, this application also considers the feature differences between board types and the frequency of occurrence of component welding anomaly types to guide the detection model in learning the difference features between different board types and prioritizing the optimization of key defect detection. The knowledge database includes the frequency of occurrence of at least one component welding anomaly type. Based on the knowledge database, anomaly frequency weight values ​​can be determined. Then, the difference between the welding detection results and the welding annotation results is weighted according to the anomaly frequency weight values ​​to obtain the anomaly frequency loss value. Furthermore, the board type difference penalty loss value can be calculated based on the cosine similarity between the board type features of different sample motherboards. Through contrastive learning, the distance between different board type features is forced to be greater than or equal to a certain threshold to ensure feature distribution differences.

[0049] For example, the training loss value of the detection model can be calculated using the following formula 1.

[0050] Formula 1: L = L det +αL freq +βL type +L false

[0051] Among them, L det Based on the loss value, L freq L represents the abnormal frequency loss value. type L represents the penalty loss value for plate shape difference. false The false positive loss value is α, and the weight values ​​are β.

[0052] For example, the abnormal frequency loss value can be calculated using the following formula 2.

[0053] Formula 2:

[0054] Where N is the number of sample motherboards, c i The actual anomaly type for the labeled component soldering abnormality. To detect the predicted anomaly type of component welding anomalies output by the model, w i This represents the abnormal frequency weight value. The abnormal frequency weight value can be determined by the high-frequency abnormality types in the knowledge database; for example, the abnormal frequency weight value w for a power board solder joint defect. i The anomaly frequency weight value w for other anomaly types is 2.0. i It is 1.0.

[0055] For example, the plate shape difference penalty loss value can be calculated using the following formula 3.

[0056] Formula 3:

[0057] Where M is the number of plate types, φ j Let j be the plate shape characteristic, and cos(φ) j φ k Let be the cosine similarity between the plate type features of plate type j and plate type k, and γ be the threshold. By forcing the distance between different plate type features to be greater than or equal to γ ​​through contrastive learning, the difference in feature distribution can be ensured. The value of γ can be 0.7 or 0.8, etc., and this embodiment does not limit this.

[0058] In some embodiments, a hierarchical training strategy can be adopted for training the detection model to improve its robustness. For example... Figure 6 As shown in the embodiments of this application, the training of the detection model can be divided into three stages. In the first stage, general training is performed, which inputs multiple sample motherboards of a full range of mixed board types. Based on the board type difference characteristics (such as the thermal deformation law of power boards) in the knowledge database and the feature separation module in the detection model, the detection model is forced to learn the feature distinction between different board types. In the second stage, specific optimization is performed. For each board type, corresponding sample motherboards are constructed to enhance the detection model's ability to detect the component soldering quality of that board type. For example, for power boards, thermal deformation simulation samples can be generated according to the thermal stress soldering rules in the knowledge database to enhance the detection model's ability to detect high-temperature defects. For example, for decoding boards, sample motherboards can be labeled and detection threshold data adjusted according to the correlation between resistance and false alarm rate in the knowledge database. In the third stage, incremental learning is performed. Sample motherboards can be added periodically (e.g., daily) according to the rules of the knowledge database (such as board type, component type, soldering anomaly type, etc.), and the detection head parameters of the corresponding board type can be updated to achieve dynamic adaptation of the detection model.

[0059] Based on a hierarchical training strategy, the target motherboard in the application phase of the detection model can be used for incremental learning in the third phase. Therefore, in some embodiments, the method further includes: updating the knowledge database based on the target motherboard; and updating the model parameters of the detection model based on the target motherboard and the updated knowledge database. This may involve updating only the detection head parameters of the detection model, such as... Figure 2 The parameters of the welding inspection module 230 shown are as follows.

[0060] In some embodiments, the aforementioned knowledge database can be constructed based on a data layering system. The data layering logic can be as follows: Figure 7 As shown, a three-level knowledge database can be constructed based on board type, component type, and welding anomaly type. Fields in the knowledge database include, but are not limited to: board type (example value: power board; purpose: selection of trigger detection model), component type (example value: electrolytic capacitor_220μF_diameter10mm; purpose: matching component features and detection threshold data), high-frequency welding anomaly type (example value: bottom cold solder joint; purpose: calculation of training loss value), associated process parameters (example value: reflow soldering peak temperature 245℃; purpose: process feedback optimization), and historical false alarm rate (example value: 18%; purpose: dynamic adjustment of detection threshold).

[0061] like Figure 8 As shown, in practical applications, the latest (e.g., within the last 3 months) AOI equipment false alarm / missed inspection records can be collected from the production line, and the data can be recorded according to board type, component type, welding anomaly type, etc., and high-frequency welding anomaly types can be statistically analyzed. In addition, process parameters (such as furnace temperature profile, pick-and-place machine number, etc.) in MES (Manufacturing Execution System) can be linked for the correlation analysis of defects and processes.

[0062] Please see Figure 9 , Figure 9 This is a schematic diagram of a method for detecting the welding quality of components provided in an embodiment of this application. Figure 9 As shown, after the AOI equipment detects a target motherboard with suspected component soldering abnormalities, the board type of the target motherboard, such as a power board or a decoder board, is identified through the PCB silkscreen code or CAM file. Based on the board type, a trained detection model is invoked; for example, a thermal deformation optimization model is loaded for a power board. Then, detection parameters associated with the target motherboard's board type are read from the knowledge database; for example, electrolytic capacitor parameters are read for a power board. The detection model then processes the image data of the target motherboard according to the detection parameters to output soldering detection results. Furthermore, the knowledge database can be updated based on the target motherboard and its related data, such as image data and soldering detection results, to facilitate incremental learning of the detection model.

[0063] After the detection model outputs the welding inspection results, the embodiments of this application can also visualize the results and feed them back to the production system. For example, the location (e.g., a red box) and type of component welding anomalies (e.g., cold solder joints, misalignment) can be marked in the image data of the target motherboard, and the confidence level of the component welding anomalies (e.g., 0 to 1) can also be displayed. When feeding back to the production system, closed-loop process feedback can also be performed. For example, when the cold solder joint rate of electrolytic capacitors on the power board is greater than 10%, an automatic push of furnace temperature optimization suggestions (e.g., increasing the peak temperature by 5°C) is sent to the MES system; when the offset rate of the 0402 resistor on the decoder board is greater than 8%, a chip mounter nozzle calibration command is triggered. In addition, all inspection data can be stored, and the welding inspection results can be associated with board type, component type and specifications, process parameters, etc., to support quality traceability.

[0064] Based on the same inventive concept, this application also provides a component soldering quality detection device for implementing the component soldering quality detection method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations of one or more component soldering quality detection device embodiments provided below can be found in the limitations of the component soldering quality detection method described above, and will not be repeated here.

[0065] Please see Figure 10 , Figure 10 This is a block diagram of a component welding quality detection device provided in an embodiment of this application. Figure 10 As shown, the component welding quality detection device 1000 may include a board type recognition module 1010, a data acquisition module 1020, and a quality detection module 1030.

[0066] The board type identification module 1010 is used to identify the board type of the target motherboard based on the configuration data of the target motherboard; wherein the target motherboard is a motherboard suspected of having abnormal component soldering.

[0067] The data acquisition module 1020 is used to acquire detection model and detection threshold data according to the board type of the target motherboard;

[0068] The quality inspection module 1030 is used to input the image data of the target motherboard into the inspection model, so that the inspection model performs component welding inspection on the image data according to the inspection threshold data and outputs welding inspection results; wherein, the welding inspection results are used to indicate the welding abnormality state of the target motherboard.

[0069] In some embodiments, the data acquisition module 1020 is further configured to: load the detection model matching the board type of the target motherboard from the model database; and read the detection threshold data associated with the board type of the target motherboard from the knowledge database. The model database includes model parameters of the detection models matching at least one of the board types; the knowledge database includes the detection threshold data associated with at least one of the board types.

[0070] In some embodiments, the welding inspection results include at least one of the following: component welding location data, component welding anomaly type, and component welding anomaly confidence level.

[0071] In some embodiments, the detection model includes a feature extraction module, a feature fusion module, and a welding detection module; the quality detection module 1030 is further configured to: input the image data of the target motherboard into the feature extraction module, so that the feature extraction module extracts component features from the image data; input the component features into the feature fusion module, so that the feature fusion module enhances the component features to obtain fused features; input the fused features into the welding detection module, so that the welding detection module performs component welding detection on the fused features according to the detection threshold data, and outputs the welding detection result.

[0072] In some embodiments, the feature extraction module includes a solder joint attention unit, and the component features include small-sized features extracted by the solder joint attention unit.

[0073] In some embodiments, the detection model further includes a feature separation module; the component welding quality detection device 1000 further includes a training module, configured to: input the image data of the sample motherboard into the feature extraction module, so that the feature extraction module extracts the component features from the image data; input the component features into the feature separation module, so that the feature separation module separates board type features from the component features according to a knowledge database; input the component features into the feature fusion module, so that the feature fusion module enhances the component features to obtain the fused features; input the fused features into the welding detection module, so that the welding detection module performs component welding detection on the fused features according to the knowledge database and outputs the welding detection result; determine the training loss value of the detection model according to the board type features of the sample motherboard, the welding detection result, and the welding annotation data; and adjust the model parameters of the detection model to make the training loss value converge. The knowledge database includes component types associated with at least one of the board types; the knowledge database also includes detection threshold data associated with at least one of the board types.

[0074] In some embodiments, the training module is further configured to: determine the base loss value and false alarm loss value of the detection model based on the welding inspection results and welding annotation data of the sample motherboard; determine anomaly frequency weight values ​​based on the welding inspection results, welding annotation data, and knowledge database of the sample motherboard, and determine the anomaly frequency loss value of the detection model in combination with the anomaly frequency weight values; determine the board shape difference penalty loss value based on the cosine similarity between the board shape features of the sample motherboard; and determine the training loss value of the detection model based on the base loss value, the false alarm loss value, the anomaly frequency loss value, and the board shape difference penalty loss value. The knowledge database includes the occurrence frequency of at least one type of component welding anomaly.

[0075] In some embodiments, the training module is further configured to: update the knowledge database according to the target motherboard; and update the model parameters of the detection model according to the target motherboard and the updated knowledge database.

[0076] This application embodiment achieves the above-mentioned component welding quality detection method through the coordinated operation of various modules in the component welding quality detection device 1000. The component welding quality detection device 1000 has all the beneficial effects of the above-mentioned component welding quality detection method, which will not be repeated here.

[0077] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program or instructions thereon, which, when executed by a processor, implements the steps of the above-described method for detecting the welding quality of components. This computer-readable storage medium possesses all the beneficial effects of the above-described method for detecting the welding quality of components, which will not be elaborated upon here.

[0078] Accordingly, this application also provides a computer program product, including a computer program or instructions. When the computer program or instructions are executed by a processor, they implement the above-mentioned method for detecting the welding quality of components and have all the beneficial effects of the above-mentioned method for detecting the welding quality of components. This application will not elaborate further here.

[0079] Accordingly, this application also provides an electronic device, including: a memory and a processor, wherein the memory stores a computer program or instructions; the processor executes the computer program or instructions in the memory to implement the steps of the above-described method for detecting the welding quality of components. This electronic device possesses all the beneficial effects of the above-described method for detecting the welding quality of components, which will not be elaborated upon here.

[0080] Computer-readable storage media can be, for example, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof, without specific limitation herein. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0081] In some embodiments of this application, a computer-readable storage medium may be any tangible medium that contains or stores a program that may be used or combined with an instruction execution system, apparatus, or device.

[0082] The aforementioned computer-readable storage medium may be included in the aforementioned electronic device, or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable storage medium carries one or more programs that, when executed by the electronic device, cause the electronic device to:

[0083] Based on the configuration data of the target motherboard, the board type of the target motherboard is identified; wherein, the target motherboard is a motherboard suspected of having abnormal component soldering detected by AOI equipment;

[0084] Based on the board type of the target motherboard, obtain the detection model and detection threshold data;

[0085] The image data of the target motherboard is input into the detection model, so that the detection model performs component soldering detection on the image data according to the detection threshold data and outputs the soldering detection result; wherein, the soldering detection result is used to indicate the soldering abnormality of the target motherboard.

[0086] Computer program code for performing operations of some embodiments of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network (including a Local Area Network (LAN) or a Wide Area Network (WAN)), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0087] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function.

[0088] It should also be noted that in some alternative implementations, the functions marked in the box may occur in a different order than those marked in the attached figures.

[0089] For example, two consecutively represented blocks can actually be executed in substantially parallel order, and sometimes they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, as well as combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified functions or operations, or using a combination of dedicated hardware and computer instructions.

[0090] The units described in some embodiments of this application can be implemented in software or in hardware. The described units can also be located in a processor.

[0091] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), Application Specific Standard Parts (ASSP), System on Chip (SOC), Complex Programmable Logic Device (CPLD), and so on.

[0092] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0093] In the above embodiments, the descriptions of each embodiment have their own emphasis. Parts not described in detail in a certain embodiment can be referred to in the relevant descriptions of other embodiments. The embodiments, implementation methods, and related technical features of this application can be combined and substituted with each other without conflict. The above are merely preferred embodiments of this application and are not intended to limit this application in any way. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of this application without departing from the scope of the technical solution of this application shall still fall within the scope of the technical solution of this application.

Claims

1. A method for detecting the soldering quality of components, characterized in that, The method includes: Based on the configuration data of the target motherboard, the board type of the target motherboard is identified; wherein, the target motherboard is a motherboard suspected of having abnormal component soldering detected by AOI equipment; Based on the board type of the target motherboard, obtain the detection model and detection threshold data; The image data of the target motherboard is input into the detection model, so that the detection model performs component soldering detection on the image data according to the detection threshold data and outputs the soldering detection result; wherein, the soldering detection result is used to indicate the soldering abnormality of the target motherboard.

2. The method according to claim 1, characterized in that, The step of obtaining the detection model and detection threshold data based on the board type of the target motherboard includes: From the model database, the detection model that matches the board type of the target motherboard is loaded; wherein, the model database includes model parameters of the detection model that matches at least one of the board types; From a knowledge database, the detection threshold data associated with the board type of the target motherboard is read; wherein the knowledge database includes the detection threshold data associated with at least one of the board types.

3. The method according to claim 1, characterized in that, The welding inspection results include at least one of the following: component welding position data, component welding anomaly type, and component welding anomaly confidence level.

4. The method according to claim 1, characterized in that, The detection model includes a feature extraction module, a feature fusion module, and a welding detection module; The step of inputting the image data of the target motherboard into the detection model, so that the detection model performs component soldering detection on the image data according to the detection threshold data, and outputs the soldering detection result, includes: The image data of the target motherboard is input into the feature extraction module so that the feature extraction module extracts component features from the image data; The component features are input into the feature fusion module, so that the feature fusion module enhances the component features to obtain fused features; The fusion feature is input into the welding detection module, so that the welding detection module performs component welding detection on the fusion feature according to the detection threshold data and outputs the welding detection result.

5. The method according to claim 4, characterized in that, The feature extraction module includes a solder joint attention unit, and the component features include small-sized features extracted by the solder joint attention unit.

6. The method according to claim 4, characterized in that, The detection model also includes a feature separation module; The method further includes: The image data of the sample motherboard is input into the feature extraction module so that the feature extraction module extracts the component features from the image data; The component features are input into the feature separation module, so that the feature separation module separates the board type features from the component features according to the knowledge database; wherein, the knowledge database includes component types associated with at least one of the board types; The component features are input into the feature fusion module, so that the feature fusion module enhances the component features to obtain the fused features; The fusion feature is input into the welding detection module, so that the welding detection module performs component welding detection on the fusion feature according to the knowledge database and outputs the welding detection result; wherein, the knowledge database also includes detection threshold data associated with at least one of the plate types; Based on the board shape characteristics of the sample motherboard, the welding inspection results, and the welding annotation data, the training loss value of the detection model is determined; Adjust the model parameters of the detection model to make the training loss value converge.

7. The method according to claim 6, characterized in that, The step of determining the training loss value of the detection model based on the board type characteristics of the sample motherboard, the welding detection results, and the welding annotation data includes: Based on the welding inspection results and welding annotation data of the sample motherboard, determine the basic loss value and false alarm loss value of the detection model; Based on the welding inspection results of the sample motherboard, the welding annotation data, and the knowledge database, anomaly frequency weight values ​​are determined, and anomaly frequency loss values ​​of the detection model are determined in combination with the anomaly frequency weight values; wherein, the knowledge database includes the occurrence frequency of at least one type of component welding anomaly. The board shape difference penalty loss value is determined based on the cosine similarity between the board shape features of the sample motherboards; The training loss value of the detection model is determined based on the base loss value, the false positive loss value, the anomaly frequency loss value, and the plate shape difference penalty loss value.

8. The method according to claim 6, characterized in that, The method further includes: Update the knowledge database according to the target motherboard; The model parameters of the detection model are updated based on the target motherboard and the updated knowledge database.

9. A device for detecting the welding quality of components, characterized in that, The device includes: A board type identification module is used to identify the board type of the target motherboard based on the configuration data of the target motherboard; wherein the target motherboard is a motherboard suspected of having abnormal component soldering. The data acquisition module is used to acquire detection model and detection threshold data based on the board type of the target motherboard; The quality inspection module is used to input the image data of the target motherboard into the inspection model, so that the inspection model performs component soldering inspection on the image data according to the inspection threshold data and outputs the soldering inspection result; wherein, the soldering inspection result is used to indicate the soldering abnormality of the target motherboard.

10. An electronic device, characterized in that, include: A memory on which computer programs or instructions are stored; A processor for executing the computer program or instructions in the memory to implement the component welding quality detection method as described in any one of claims 1 to 8.

Images