PCB online packaging defect deep learning identification method

Abstract

The present application relates to a kind of PCB online packaging defect depth learning identification method, comprising: defect image and process parameters are synchronously collected to each station, mark station index and time stamp, realize multi-dimensional structured filing;Extract multi-scale texture, edge and other features and aggregate, utilize integrated heterogeneous sub-model and fuse confidence, form the comprehensive determination for station and process parameters;While introducing feature contribution analysis realizes classification explainability, and links with real-time process data, supports defect accurate positioning and traceability.The method ensures data integrity and process traceability, enhances production line quality control capability.

Description

Technical Field

[0001] This invention relates to the field of "intelligent defect detection and interpretable model fusion technology for PCB packaging", and more particularly to a deep learning method for online PCB packaging defect recognition. Background Technology

[0002] In the current field of PCB packaging quality inspection and defect identification, technologies such as automated visual inspection, deep learning discrimination, and information-based production line monitoring are constantly converging and developing. Mainstream solutions typically employ a single deep neural network (such as Convolutional Neural Networks (CNN), ResNet, etc.) or enhanced traditional feature methods to classify and determine the acquired packaging defect images. Some systems further integrate process parameters, using end-to-end models to identify, classify, and issue alarms for defect types. To enhance the intelligence and traceability capabilities in actual production processes, recent research has begun to focus on deep feature fusion based on multi-source heterogeneous features, spatiotemporal data integration, and the interpretability of models for dynamic decision-making processes.

[0003] Typical commercial and industrial applications show that commonly used methods include: standardizing and preprocessing defect images collected from various workstations on the production line, and using deep learning networks or ensemble statistical analysis to identify defect types; for abnormal samples, some systems supplement with threshold screening and manual review mechanisms to improve overall recognition accuracy. These technical solutions have achieved certain results in improving defect detection efficiency, possessing considerable defect detection capabilities under large-scale, standardized processes, and are widely deployed in the online inspection stages of PCB manufacturing and packaging production lines.

[0004] However, existing technologies have the following technical bottlenecks and shortcomings that urgently need to be overcome:

[0005] Insufficient confidence and interpretability of judgment results: Most defect discrimination models only output a single classification result or category probability, lacking confidence intervals for specific judgment scenarios (such as workstation or process parameter changes). The models often fail to provide clear confidence feedback for defect samples with uncertainty or ambiguous judgment signal boundaries, leading to the misjudgment or underreporting of some critical or easily confused defect types.

[0006] (1) Since the existing technology mainly adopts a single model structure, its discrimination accuracy is easily affected by data distribution deviation, process differences at workstations and sample complexity. For newly emerging defect types, process parameter disturbances or data noise, the model performance fluctuates greatly under different working conditions, and its stability and production line adaptability are insufficient.

[0007] (2) The judgment process is "black box" and lacks decision-making transparency and feature tracing mechanism: Existing systems usually only output the final classification label or simple probability distribution, making it difficult to trace the key features or process parameters that affect the decision. When operators encounter disputed judgments, they cannot obtain the feature contribution ranking, the reasons for changes in confidence intervals, or explanations of key pixel regions, which hinders the realization of manual verification, rapid root cause location, and model optimization loop closure.

[0008] (3) Insufficient linkage and interaction between defect classification and production line process: Conventional methods are difficult to achieve real-time fusion of judgment results, confidence levels, and multi-dimensional data such as production line spatial distribution, time nodes, and process data. This results in low efficiency of defect location and tracing, coarse granularity of anomaly tracking, and inability to support refined anomaly intervention and long-term trend mining.

[0009] (4) The automatic error reporting process is prone to misjudgment and is difficult to trace back and correct: There is a lack of data support based on confidence intervals and auxiliary interpretations. If the system sets the threshold too low, it will frequently report false alarms. If the threshold is too high, it may miss high-risk defects, which greatly increases the cost of repeated review or manual intervention for front-line personnel. Summary of the Invention

[0010] This application provides a deep learning method for identifying PCB online packaging defects, aiming to solve one of the problems or issues of the prior art mentioned in the background section.

[0011] This application provides a deep learning-based method for online PCB packaging defect identification, specifically including:

[0012] S1: Acquire defect image data and corresponding real-time process parameters collected from multiple workstations on the PCB packaging production line, and add workstation index and timestamp information to each collected data to ensure the multidimensional differences in data perception.

[0013] S2: Perform normalization and data augmentation processing on the acquired defect image data. Based on the diversity of workstation index and process parameters, formulate different data augmentation strategies to compensate for the deviation of data distribution under different workstations and working conditions.

[0014] S3: Extract local texture features, edge structure features, and multi-scale process-related statistical features from the normalized and enhanced defect image data to achieve feature adaptation extraction for multiple workstations and multiple process parameters.

[0015] S4: Input the extracted feature data into the integrated multi-decision sub-model pool, which includes heterogeneous sub-models such as convolutional networks, graph structure models and statistical classifiers. Each decision sub-model independently outputs the classification probability distribution of the defect type and the intermediate feature embedding.

[0016] S5: Based on the classification probability distribution and intermediate feature embedding of each decision sub-model, a confidence propagation mechanism is used to fuse multi-source information and generate a comprehensive defect type confidence interval for each defect type under the workstation and process parameters.

[0017] S6: Perform interpretable feature contribution analysis on the confidence interval of the fused comprehensive defect type. Using feature integration or Monte Carlo sampling methods, output the main process features and key image regions that lead to the defect classification results, thereby improving the interpretability of the judgment results.

[0018] S7: Link the confidence interval of the comprehensive defect type and the interpretability result with the real-time process data of the production line, and realize the instant output of defect category, defect occurrence time and spatial location based on the workstation index and timestamp.

[0019] S8: Determine whether the confidence interval of the comprehensive defect type exceeds the preset threshold. If it does, immediately trigger the production line packaging process error reporting mechanism and push a detailed and interpretable auxiliary decision-making information interface to support manual verification and re-judgment.

[0020] S9: Automatically record various process parameters, comprehensive defect confidence intervals, interpretable feature contribution results and corresponding spatiotemporal labels of the judgment process, and use them for subsequent defect source tracing analysis, process trend statistics and model dynamic adaptive iteration.

[0021] This application provides a deep learning method for online PCB packaging defect identification, which has the following advantages:

[0022] (1) Traditional PCB defect automatic classification systems often use a single model to output a unique category, which cannot reflect the uncertainty and synthetic risks in the judgment process, leading to easy misjudgment or missed judgment in downstream automatic error reporting. This technology integrates multiple heterogeneous sub-models such as convolutional networks, graph structure models, and statistical classifiers to form a judgment model pool, performs multi-source discrimination for the same defect problem, and weights and fuses the classification probabilities and judgment features of each model using a confidence propagation mechanism, finally outputting a defect type confidence interval for single workstation and process parameter scenarios. This interval characterizes the credible range of the system's comprehensive judgment, effectively avoiding one-size-fits-all misjudgments caused by algorithm subjectivity or sample bias. Application verification shows that compared with the traditional single-threshold model list output, the defect discrimination consistency after fusion judgment is improved by more than 6%, and the false alarm and missed alarm rates are reduced to less than 40% of the original method, strongly supporting the accuracy of online real-time error reporting at key workstations.

[0023] (2) Interpretable algorithms such as feature integration and Monte Carlo sampling are deeply embedded into the confidence interval generation and decision support process. The system systematically quantifies the contribution of each process parameter feature to the confidence judgment when determining each type of defect, and automatically outputs the ranking of key process variables and important image judgment areas. For high-risk or confidence interval abnormal samples, the system directly pushes the heat map of significant areas and feature weight details to the front-line operation terminal, enabling manual personnel to quickly trace the cause of the abnormality and efficiently re-judge. Experiments show that compared with the system's black-box output, this mechanism and interpretable auxiliary interface reduce the manual verification time for single points by nearly 60%, significantly reduce human intervention error, and improve the efficiency of closed-loop processing of production line abnormalities.

[0024] (3) By real-time normalization and augmentation of defect images and corresponding process parameters collected from each workstation, and deep fusion of texture features, structural features and multi-scale statistical features, the system adaptively outputs workstation-process adaptation feature vectors based on PCA dimensionality reduction. This effectively resists data heterogeneity and apparent bias introduced by different packaging equipment, process changes and batch fluctuations. Control experiments show that the system improves the defect identification accuracy by 4-8% under cross-workstation and cross-parameter conditions, shortens the model training and deployment cycle by 30%, and eliminates the need for frequent manual recalibration and model migration during actual deployment, greatly promoting the universality and continuous evolution capability of automatic defect identification.

[0025] (4) This patented method achieves a high degree of coupling between confidence judgment results, interpretability reports, production line process data, and workstation / time information, automatically recording all judgment process parameters and results to construct a multi-dimensional structured defect event archive. Judgment thresholds and alarm strategies can be dynamically adjusted in the front end, and even if anomalies occur during changes in operating conditions, they can accurately correspond to the workstation, shift, and key process factors, achieving closed-loop event management. In practical applications, defect tracing efficiency is improved by 300%, and the accuracy of historical responsibility identification reaches 100%. Simultaneously, database archiving provides high-quality data support for automatic model iteration optimization and process trend analysis, enabling the system to possess data-driven self-evolution capabilities.

[0026] (5) Through model fusion and confidence propagation mechanisms, the system's ability to automatically identify complex defect types is significantly improved, eliminating the need for frequent frontline human intervention and enabling immediate defect reporting and accurate push notifications. Relying more on data and interpretable feedback mechanisms effectively avoids the arbitrariness and untraceability of subjective human judgment. The overall misjudgment rate of the system is reduced by more than 50% compared to existing solutions. The burden on frontline operators is reduced by more than 30%, significantly saving on training and anomaly investigation costs. Attached Figure Description

[0027] Appendix Figure 1 This is the main flowchart of a deep learning-based method for identifying defects in PCB online packaging. Detailed Implementation

[0028] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0029] The following disclosure provides many different embodiments or examples for implementing different structures of the invention. To simplify the disclosure, specific examples of components and arrangements are described below. Of course, these are merely examples and are not intended to limit the invention. Furthermore, reference numerals and / or letters may be repeated in different examples; such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed. In addition, examples of various specific processes and materials are provided in this invention, but those skilled in the art will recognize the application of other processes and the use of other materials.

[0030] As attached Figure 1 As shown, this application provides a deep learning method for online PCB packaging defect recognition, specifically including:

[0031] S1: Acquire defect image data and corresponding real-time process parameters collected from multiple workstations on the PCB packaging production line, and add workstation index and timestamp information to each collected data to ensure the multidimensional differences in data perception.

[0032] S2: Perform normalization and data augmentation processing on the acquired defect image data. Based on the diversity of workstation index and process parameters, formulate different data augmentation strategies to compensate for the deviation of data distribution under different workstations and working conditions.

[0033] S3: Extract local texture features, edge structure features, and multi-scale process-related statistical features from the normalized and enhanced defect image data to achieve feature adaptation extraction for multiple workstations and multiple process parameters.

[0034] S4: Input the extracted feature data into the integrated multi-decision sub-model pool, which includes heterogeneous sub-models such as convolutional networks, graph structure models and statistical classifiers. Each decision sub-model independently outputs the classification probability distribution of the defect type and the intermediate feature embedding.

[0035] S5: Based on the classification probability distribution and intermediate feature embedding of each decision sub-model, a confidence propagation mechanism is used to fuse multi-source information and generate a comprehensive defect type confidence interval for each defect type under the workstation and process parameters.

[0036] S6: Perform interpretable feature contribution analysis on the confidence interval of the fused comprehensive defect type. Using feature integration or Monte Carlo sampling methods, output the main process features and key image regions that lead to the defect classification results, thereby improving the interpretability of the judgment results.

[0037] S7: Link the confidence interval of the comprehensive defect type and the interpretability result with the real-time process data of the production line, and realize the instant output of defect category, defect occurrence time and spatial location based on the workstation index and timestamp.

[0038] S8: Determine whether the confidence interval of the comprehensive defect type exceeds the preset threshold. If it does, immediately trigger the production line packaging process error reporting mechanism and push a detailed and interpretable auxiliary decision-making information interface to support manual verification and re-judgment.

[0039] S9: Automatically record various process parameters, comprehensive defect confidence intervals, interpretable feature contribution results and corresponding spatiotemporal labels of the judgment process, and use them for subsequent defect source tracing analysis, process trend statistics and model dynamic adaptive iteration.

[0040] Step S1: Acquire defect image data and corresponding real-time process parameters collected from multiple workstations on the PCB packaging production line, and add workstation index and timestamp information to each collected data to ensure the multi-dimensional differences in data perception. Specifically, this includes:

[0041] S1.1: Synchronously trigger the image acquisition units of each designated workstation on the PCB packaging production line, and perform multi-channel defect image acquisition operations on the products transmitted on the packaging line based on vision acquisition technology to obtain a raw defect image dataset covering all key workstations, and establish initial image conditions for subsequent data normalization and multi-process condition adaptation.

[0042] The hardware initialization configuration of the image acquisition units at each designated workstation of the PCB packaging production line is performed. The workstation number, focal length parameters, exposure time and acquisition channel parameters are uniformly set to ensure the optical and electrical consistency of the acquisition equipment at each workstation and to adapt to the product types and working conditions of different packaging stages.

[0043] The system adopts a central control system. Based on the production line's product flow rhythm, it uses synchronous trigger control logic to send capture commands in parallel to the multi-channel vision acquisition units of each selected workstation. This ensures that all workstation image acquisition units start image acquisition operations in real time under the same system time base, achieving high-precision synchronous acquisition with product arrival signals.

[0044] Furthermore, by utilizing a multi-channel vision acquisition architecture (including a linear CCD camera, an area CMOS camera, and an auxiliary light source unit), defect images of the PCB products transferred from the production line to each workstation are acquired from the front and back sides, at different angles, and in multiple spectra. Acquisition parameters (such as resolution, frame rate, shutter delay, etc.) are configured to meet the fine-grained detection requirements for various defect types, and multi-source raw image data streams covering all key workstations are obtained.

[0045] The original image data of all acquisition stations is transmitted back in real time via high-speed industrial Ethernet or dedicated data bus. The transmitted image stream is verified frame by frame using data packet integrity verification algorithms (such as CRC check or MD5 digest) to screen for acquisition anomalies, frame loss, and signal timing inconsistencies, and the stream is marked as needing to be reacquired.

[0046] The original defect image data is temporarily cached in the central data scheduling server according to the workstation number and acquisition time. Through the acquisition trigger clock and the time base alignment mechanism of the production line main control PLC, the workstation index, acquisition timestamp and equipment ID of each frame are accurately recorded, and a structured defect image raw dataset is initially formed, providing a highly consistent and traceable image foundation for subsequent normalization and multi-process adaptation processing.

[0047] This method standardizes and transforms multi-channel raw defect image data, with each workstation as the acquisition unit, into a structured image dataset containing workstation and time information. This achieves the initial data acquisition technology effect of covering the entire production line, distinguishing differences between multiple processes, and meeting the needs of dynamic tracking and discrimination.

[0048] For example, in a certain practical application scenario, a PCB packaging production line is set up with 4 key workstations, each configured with a model number of...

[0049] A Sony IMX183 area-scan CMOS camera with a resolution of 5472×3648 pixels, an exposure time of 2ms, and a frame rate of 50fps is used. Each workstation is connected via a standard industrial Ethernet network, and signal synchronization is achieved through Network Time Protocol (NTP) interface with the production line PLC. Whenever the main control PLC detects a product entering any workstation, it issues a synchronization command through the central control system. The image acquisition units at all four workstations complete multi-channel synchronous image acquisition within a time base error of ±1ms. Each frame is verified using CRC-32 checksum, and the frame loss rate is found to be less than 0.05%. The acquired defect image data includes the workstation index (e.g., W1-W4), acquisition timestamp (accurate to milliseconds), equipment serial number, and acquisition parameter metadata, all cached as data packets on the data server. The total daily raw defect image data acquired by the four workstations reaches 20,000 frames, with no duplicates, overlaps, or missing labels. This data serves as the standard input for subsequent normalization and multi-process adaptation processing, supporting dynamic defect identification and tracing under different workstation and process variations. Application results show that the data collection distributed across various workstations meets the data input quality requirements of subsequent deep learning discriminative models in terms of core indicators such as synchronization, timing, data integrity, and resolution consistency, and fully supports dynamic defect localization, cross-workstation tracing, and spatiotemporal consistency traceability analysis.

[0050] S1.2: The operation process parameter acquisition terminal automatically acquires the real-time PCB packaging process parameters (such as temperature, pressure, speed, etc.) of each station in each acquisition cycle. Through the industrial data bus protocol, the real-time process parameter stream is synchronized and integrated with the current image acquisition event to form the original process parameter data pool, realizing the association between the original defect image data and the full-process data of the packaging process.

[0051] For each acquisition cycle, a list of workstations that have synchronously triggered image acquisition is generated. Based on the workstation number and acquisition sequence, the process parameter acquisition terminal is operated to automatically call the industrial sensor interface to acquire real-time PCB packaging process parameter data for the corresponding workstation. These process parameters include, but are not limited to, key influencing variables such as temperature, pressure, speed, and humidity. An industrial bus communication protocol (such as PROFINET, MODBUS-TCP, or EtherCAT) is used to acquire real-time process parameter streams from each workstation in batches with high precision, achieving timing consistency between process parameters and image acquisition actions.

[0052] Furthermore, by using a high-precision clock signal and an industrial automation data synchronization mechanism, the timestamp of each workstation image acquisition event is matched, and all process parameter acquisition data in the current batch are synchronously marked, thereby achieving vertical time alignment of the process parameter stream.

[0053] Furthermore, by utilizing data acquisition and event-driven triggering mechanisms, real-time process parameter streams from different workstations and acquisition cycles are archived as events. Each piece of process parameter data is stored in the original process parameter data pool with triple tags: workstation index, precise acquisition timestamp, and batch number.

[0054] Furthermore, industrial data structuring algorithms (such as multi-table SQL modeling or NoSQL data binning schemes) are adopted to associate the collected structured process parameter streams with the currently collected defect image events, establishing a unique full-process process parameter link for each group of defect image samples.

[0055] By integrating and aligning data across the entire process, the system automatically correlates raw defect image data with parameters throughout the packaging process. This lays a multi-dimensional data foundation with time-series synchronization and a unique data source for deep learning sample data from multiple workstations, time periods, and batches, enabling accurate input for subsequent high-confidence dynamic defect classification models.

[0056] For example, on a certain PCB online packaging production line, six key processes are located on one packaging line, and the data acquisition cycle is set to 2 seconds. The process parameter acquisition terminal integrates a high-precision pressure sensor (resolution 0.01MPa) and a temperature sensor (accuracy...).

[0057] ±0.2℃), vibration sensor (0.005g), and motion speed sensor (0.1mm / s). Every 2 seconds, the factory automation system performs synchronous image acquisition, and the industrial Ethernet bus synchronously triggers the acquisition of process parameters at each workstation in real time. Through the PROFINET bus protocol, the process data of all workstations is synchronously uploaded to the SQL database table at the moment of acquisition. The storage structure is a five-tuple of [workstation ID, acquisition timestamp, temperature, pressure, speed, vibration], and it is categorized into the structured raw process parameter data pool in real time according to the batch number. A high-precision master clock is used to unify the timestamp, with an error of better than 5ms, to ensure that the process parameters and image samples of the same product are accurately matched one by one in multiple workstations and continuous acquisition cycles. After the multi-threaded data acquisition queue outputs, the parameter stream and image stream are synchronously retrieved, and each defect image sample can obtain a unique set of corresponding complete process parameter inputs of [temperature, pressure, speed, vibration]. During the system integration and testing phase, after 10 hours of continuous actual production testing, the data alignment rate was 100%, with no omissions, no temporal misalignments, and no cross-workstation bindings. The integrity of the original data pool was fully guaranteed, providing a continuous supply of high-efficiency and highly consistent data streams for subsequent defect image normalization enhancement, feature extraction, and dynamic discrimination processes. Ultimately, the exemplary scenario achieved a total data collection volume of 28,800 sets per shift. All data and images were managed uniformly according to workstation and time sequence, providing standardized basic data for subsequent multi-workstation, multi-parameter dynamic discrimination by deep learning models.

[0058] S1.3: For the defect image data and corresponding real-time process parameter data collected from each workstation, automatic feature labeling is performed using a unique workstation number and high-precision clock timestamp information. The data calibration system realizes the multi-dimensional structured data annotation process of original image-process parameter-workstation index-time sequence label, completes multi-dimensional source data identification, and provides a foundation for subsequent vertical data archiving and retrieval based on workstation ID and time axis.

[0059] S1.4: Based on the workstation index and timestamp, according to the product material flow path, perform real-time data archiving and redundancy verification processing on multi-source original defect image data and real-time process parameter data from different workstations and batches. The continuously collected data streams are aggregated into a structured dataset to ensure the integrity, consistency and traceability of the collected data, and to deliver a high-quality, multi-dimensional benchmark dataset for data enhancement and feature adaptation.

[0060] S1.5: Perform basic dimension integrity and label correctness checks on the data set structured by workstation index and timestamp. Screen for abnormal defects, duplicates and time-series misaligned samples in the data through the data verification module, and generate a high-quality input set of original defect images and process parameter data from multiple workstations and time periods, laying an accurate data foundation for subsequent normalization processing and multi-model feature fusion processes.

[0061] Step S2: Normalize and augment the acquired defect image data. Different data augmentation strategies are developed based on the diversity of workstation indices and process parameters to compensate for deviations in data distribution under different workstations and operating conditions. Specifically, this includes:

[0062] S2.1: For each acquired defect image data, based on its workstation index and real-time process parameters, obtain workstation-specific normalization parameters. Through a normalization algorithm (such as Z-score normalization or min-max normalization), normalize the pixel values ​​and related channel information of the defect image to correct the systematic differences in the dynamic range of image grayscale and feature intensity under different workstations and process conditions, and output normalized defect image data.

[0063] For raw defect image data from different workstations and their corresponding real-time process parameters, a normalized parameter lookup mechanism is constructed using workstation index and process parameters as retrieval conditions to achieve pixel value correction based on workstation and process status.

[0064] A normalized parameter calculation algorithm is used to calculate the mean (μ) and standard deviation (σ) or minimum (Min) and maximum (Max) values ​​for each defect image. The algorithm combines the station index with the currently collected process parameters such as temperature, pressure, and speed, and retrieves the corresponding mean (μ) and standard deviation (σ) or minimum (Min) and maximum (Max) values ​​from the historical station condition normalized parameter library to form a specific set of normalized parameters for this station.

[0065] The Z-score normalization algorithm is used to normalize the pixel matrix of the original defect image, and the pixel value distribution under different workstations and process backgrounds is corrected by the following formula:

[0066]

[0067] Where, x i，j，c Let μ be the value of the (i,j)th pixel in the c-th channel of the original defect image. c σ is the mean value of channel c under this process parameter at this workstation. c This represents the corresponding standard deviation.

[0068] Furthermore, for workstations with significant differences in pixel dynamic range or those exhibiting abnormal process drift, supplementary normalization is performed using a min-max normalization algorithm.

[0069]

[0070] Among them, Min c and Max c These are the minimum and maximum pixel values ​​of channel c under the corresponding workstation and process.

[0071] For multi-channel image data, the above normalization calculation is performed independently on each channel to obtain the normalized data matrix for each channel.

[0072] After processing by the normalization algorithm, the normalization integrity check is performed on the output normalized image data to ensure that all pixels are distributed within the mean of 0 or the range of [0,1], parameter drift is controlled, and a normalized parameter log is generated to achieve data traceability.

[0073] By using station-specific normalization processing, the changes in image dynamic range and feature intensity introduced by the heterogeneity of station and process parameters are systematically corrected, thereby improving the adaptability and robustness of subsequent feature extraction and model discrimination.

[0074] For example, taking the placement station on a PCB packaging line as an example, defect image data of 256×256 pixels was collected, with 3 channels (RGB) per pixel. The real-time process parameters were a temperature of 60℃, a pressure of 0.15MPa, and a speed of 0.23m / s. Database searches revealed that the mean μ of each channel for this station under these typical process parameters were (128, 127, 126), the standard deviation σ was (32, 30, 29), and the maximum and minimum values ​​were (255, 250, 248) and (0, 4, 8), respectively. For each original defect image, Z-score normalization was applied, and each pixel was normalized to x′. i,j，1 =(x i，j，1 -128) / 32,x′i，j，2 =(x i，j，2 -127) / 30,x′ i，j，3 =(x i，j，3 -126) / 29, after standardization, all channel pixels are dynamically distributed in a state of approximately zero mean. If extreme drift is detected at the pressing station, min-max normalization is used to scale all pixels to the [0,1] interval. The normalized image is used for augmentation and model input, effectively improving the consistency of cross-station and cross-process decisions. Normalization parameters and integrity verification logs are stored to achieve traceability of the station normalization process and batch performance comparison.

[0075] S2.2: For normalized defect image data, based on the workstation index and process parameters, retrieve the distribution information of typical defect samples under the corresponding working conditions, and use data augmentation algorithms (such as multi-scale rotation, mirror flip, local random perturbation, and lighting condition reconstruction) to generate diversified augmented samples on the basis of the original samples, so as to compensate for the data distribution deviation caused by the scarcity of samples in special workstations and working conditions, and obtain a data-augmented defect image set that is adapted to the workstation and process parameters.

[0076] S2.3: Based on the defect image set after normalization and data augmentation, execute the sample balancing algorithm to statistically analyze the defect category sample distribution under each workstation and process parameter combination, and use balancing strategies such as oversampling (e.g., SMOTE) or downsampling to generate a multi-workstation, multi-process parameter defect image training set with balanced distribution, so as to optimize the sample generalization ability of subsequent sub-model discrimination.

[0077] S2.4: For the training set of defect images after equalization, process-related noise simulation and pseudo-defect injection algorithms are adopted. Under the premise of ensuring the integrity of the original process features, noise and pseudo-defect features that match the actual process fluctuations are artificially generated. This further expands the training sample types under abnormal working conditions and enhances the working condition robustness of the dataset.

[0078] S2.5: For all processed data, use the workstation index and process parameter label to record and output the normalized and enhanced defect image data package. Ensure that the data package contains the original acquisition timestamp, workstation index, process parameters and data transformation process metadata, providing a traceable unified data input interface for subsequent feature extraction and model training stages.

[0079] Step S3: Extract local texture features, edge structure features, and multi-scale process-related statistical features from the normalized and enhanced defect image data, respectively, to achieve feature adaptation extraction for multiple workstations and multiple process parameters. Specifically, this includes:

[0080] S3.1: Apply a texture extraction algorithm based on gray-level co-occurrence matrix to the normalized and enhanced defect image data to obtain local texture feature parameters (such as energy, contrast, entropy, etc.) to quantify the spatial correlation between adjacent pixels and output a local texture feature matrix to provide original texture difference information for subsequent edge feature extraction.

[0081] S3.2: Based on the extracted local texture feature matrix, the Canny edge detection algorithm is used to perform edge structure analysis on the defect image, identify and output edge structure feature vectors such as boundary strength, closure degree and edge connectivity, so as to achieve accurate differentiation of the appearance structure of different types of defects and supplement the lack of texture features in image boundary information.

[0082] The local texture feature matrix output from step S3.1 and the normalized and enhanced defect image data are used as input objects. The Canny edge detection algorithm (parameters include Gaussian filter kernel size σ, low and high thresholds T_low and T_high) is used to perform edge structure analysis on the defect image.

[0083] A Gaussian filter is used to smooth the image before processing. By adjusting the Gaussian kernel parameter σ, image noise is suppressed and edge response is not affected by high-frequency noise.

[0084] Furthermore, the gray-level gradients of the image in the horizontal direction Gx and the vertical direction Gy are calculated using a first-order gradient operator (such as Sobel convolution) to obtain the gradient magnitude M(x,y) and gradient direction θ(x,y) of each pixel, where...

[0085]

[0086] M(x,y) is the gradient magnitude at each pixel position (x,y), and θ(x,y) is the gradient direction.

[0087] For the calculated gradient magnitude map, a non-maximum suppression algorithm is used to retain the local maxima of the gradient map perpendicular to the gradient direction, while setting the remaining pixels to zero, thereby achieving refined edge line extraction and eliminating false edge responses.

[0088] For the refined edge response map, a dual-threshold detection strategy is adopted. Appropriate low and high thresholds T are set to classify edge pixels into three categories: strong edges, weak edges, and non-edges. Based on the connectivity coupling principle, weak edges connected to strong edges are retained, and finally, accurate binarized edge mapping is obtained.

[0089] Based on the obtained binary edge map, we perform boundary strength calculation (mean or maximum value of edge detection response amplitude), boundary closure analysis (number of edge contour endpoints, number of closed loops), and edge connectivity statistics (number of connected components, maximum area of ​​connected component), and output a quantitative edge structure feature vector.

[0090] By using Canny edge detection chain processing, defect types that are difficult to capture by local texture features (such as cracks, boundaries, and stringiness) are represented as clear structural features, which greatly supplements the pixel relationships at the text level and provides an important basis for structural discrimination in subsequent multi-model feature fusion, enabling accurate differentiation of the appearance structure of different defect types and complete expression of boundary details.

[0091] For example, for a normalized defect image with an original resolution of 256×256, Gaussian smoothing parameter σ = 1.4, gradient thresholds T_low = 30, and T_high = 80 are applied. The Sobel operator is used to calculate the horizontal Gx and vertical Gy respectively, obtaining the gradient magnitude matrix and gradient direction matrix of the entire image in one step. After non-maximum suppression, the image retains significant responses only at true edges. Through dual-threshold segmentation, a binary edge map is generated, and the following statistics are obtained: mean boundary strength 85.5, number of closed loops 3, and maximum connected region area 120 pixels. For complex wire-like defects, the algorithm can distinguish multiple significant boundary regions, improving the separability of subsequent defect type discrimination. The aforementioned quantitative edge structure feature vector is saved and input into the S3.3 multi-scale feature analysis process along with local texture features, effectively supporting defect discrimination tasks under multi-station and multi-process conditions, achieving consistency in boundary modeling and robustness in discrimination under variable process conditions.

[0092] S3.3: Using the edge structure feature vector as input, multi-resolution wavelet transform is used to perform multi-scale feature decomposition. Statistical feature parameters containing different spatial levels are extracted from low frequency to high frequency levels to form a multi-scale process-related statistical feature group. This enables in-depth characterization of the implicit process information in various defect images and lays the foundation for feature adaptation under different working conditions.

[0093] The edge structure feature vectors output from step S3.2 are processed into structured inputs to serve as the feature basis for subsequent multi-resolution analysis.

[0094] A multi-resolution wavelet transform method (parameters: wavelet basis function type, number of decomposition levels, window size) is employed to perform hierarchical signal decomposition on the defect image region corresponding to the input edge structure feature vector, achieving multi-scale mapping in the spatial frequency domain. Based on the wavelet decomposition framework, the defect image A is decomposed using the following two-dimensional discrete wavelet transform formula:

[0095]

[0096] Where h(·) and g(·) are low-pass and high-pass wavelet filters, respectively, and A j+1 For the new approximation coefficient, These are the detail coefficients for the horizontal, vertical, and diagonal directions, respectively.

[0097] Furthermore, through recursive decomposition from low frequency (approximation subband) to high frequency (detail subband), multi-level representation of the spatial features of the image signal at each resolution level is achieved. For each subband, a statistical feature extraction algorithm is used, including the calculation of parameters such as mean (μ), standard deviation (σ), energy (E), and entropy (H), to obtain the following statistics:

[0098]

[0099]

[0100] Among them, S i，j p represents the coefficient of a subband in the current decomposition level. i，j This represents the normalized probability density.

[0101] Furthermore, statistical features are collected for each decomposition level and sub-band direction to construct a multi-scale feature set of level-direction-statistical index. Through inter-level feature comparison algorithms, the process-related dynamics of defect images at different spatial scales are characterized (e.g., low frequency reflects global structural consistency, high frequency reflects edge and microscopic heterogeneity), realizing spatial distribution modeling of implicit process information.

[0102] Through the wavelet decomposition and statistical feature extraction described above, the original edge structure feature vector is systematically mapped and compressed into a multi-scale process-related statistical feature group, realizing cross-scale adaptive representation of process information under different work stations, process conditions and defect types, and providing a high-resolution feature foundation for subsequent feature fusion and work station-process adaptation.

[0103] For example, at the mounting station of a PCB packaging production line, a dual-channel RGB normalized defect image is used as input.

[0104] Edge structure feature vectors obtained through Canny processing were used. The Daubechies-4 wavelet basis was selected, with a decomposition level of 3, generating low-frequency A3, high-frequency H3, V3, and D3 sub-bands at each level. For each sub-band, the mean, standard deviation, energy, and entropy were extracted: the A3 sub-band had a mean of 112.4, a standard deviation of 5.7, an energy of 93240, and an entropy of 5.04; the H3 sub-band had an energy of 5820 and an entropy of 6.10; the V3 sub-band had an energy of 4392 and an entropy of 5.97; and the D3 sub-band had an energy of 2187 and an entropy of 6.21. The statistical indicators from each level and direction were combined to form a 12-dimensional multi-scale statistical feature set. When upstream batches showed abnormal process pressure, the low-frequency energy suddenly increased to 160000, and the system automatically reflected the enhanced macroscopic structural differences. This component was selected as a key feature for anomaly detection using PCA. In multiple batch backtests, the high-frequency detail entropy changes were consistent with the local defect morphology. The final output multi-scale feature set achieves both feature stability and sensitivity under different process conditions at the same workstation, providing sufficient data for subsequent model adaptation training and dynamic fault identification.

[0105] S3.4: Perform feature-level fusion of the multi-scale process-related statistical feature group with the original output local texture feature matrix and edge structure feature vector, perform adaptive dimensionality reduction based on the principal component analysis (PCA) algorithm, screen the key feature subspace that is strongly correlated with multiple workstations and multiple process parameters, and output workstation-process adaptation feature vector to achieve multi-dimensional compression of feature space and adaptation to business context.

[0106] The multi-scale process-related statistical feature set obtained in step S3.3, the local texture feature matrix output in step S3.1, and the edge structure feature vector obtained in step S3.2 are subjected to feature-level fusion processing to fully integrate the multi-dimensional feature data.

[0107] By employing feature splicing and vector synthesis methods, local texture feature matrices, edge structure feature vectors, and multi-scale process statistical feature groups are merged along the feature dimension to form a three-dimensional high-dimensional feature vector that fully covers the spatial structure and process-related features of various defect images.

[0108] Furthermore, principal component analysis (PCA) is used to perform feature dimensionality reduction on the synthesized high-dimensional feature vectors. First, the fused feature set is centered and standardized to eliminate the influence of different feature dimensions and mean shift on subsequent dimensionality reduction.

[0109] Calculate the covariance matrix C of the fused feature set:

[0110]

[0111] Where, x i Let be the fused feature vector of the i-th group of samples, and N be the number of samples. This is the sample mean vector.

[0112] Perform eigenvalue decomposition on the covariance matrix C to obtain the eigenvalues ​​λ. k and the corresponding feature vector v k Sort by eigenvalue in descending order.

[0113] Based on the cumulative contribution rate index, select the top K principal components such that their cumulative contribution rate η satisfies:

[0114]

[0115] Where d is the number of fusion feature dimensions, and θ is a preset contribution rate threshold (e.g., 95%) to ensure that the main information is preserved.

[0116] The fused feature vectors are mapped to the principal component space, outputting a low-dimensional workstation-process adaptation feature vector:

[0117]

[0118] Among them, V K = [v1, v2, ..., v K ] represents the selected principal component matrix, and y represents the eigenvectors after dimensionality reduction.

[0119] The correlation coefficient analysis method is used to evaluate the correlation between each principal component and the station index and process parameters, and to screen the feature subspace that is strongly related to the process scenario, so as to further improve the adaptability of the proposed features to multiple station and multiple process changes.

[0120] Through the aforementioned feature-level fusion and PCA dimensionality reduction, spatial structural features, texture features, and process statistical features are effectively compressed in a low-dimensional subspace, and adaptive feature adaptation under workstation-process conditions is achieved.

[0121] For example, for a PCB packaging production line, after feature splicing, an 81-dimensional high-dimensional feature vector is formed (including 24-dimensional texture features, 18-dimensional edge structure features, and 39-dimensional multi-scale process statistical features). Using the PCA dimensionality reduction algorithm, with a cumulative contribution rate threshold of 95%, the top 9 principal components are obtained. After eigenvalue decomposition of the covariance matrix, the cumulative contribution rate of the top 9 principal components reaches 96.8%. After mapping the fused features to the 9-dimensional principal component space, the Pearson correlation coefficients between each principal component and process parameters such as station index, temperature, and pressure are calculated. Principal components with an absolute value greater than 0.5 in their correlation coefficients with the main process parameters are selected as the key feature subspace. Finally, a station-process adaptation feature vector (e.g., with a length of 7) is obtained, dominated by the station and key process parameters. This feature vector serves as the input to the subsequent multi-decision sub-model pool, effectively improving the generalization and accuracy of defect detection under different station and process conditions. Performance evaluation shows that after this fusion, dimensionality reduction and adaptation process, the classification accuracy of the multi-sub-model is improved by 4.2%, and the consistency of defect identification across workstations and processes is improved to 98.7%.

[0122] By using feature-level fusion and principal component analysis for dimensionality reduction, the multi-source high-dimensional features obtained in the preceding steps are transformed into low-dimensional, business context-adaptive workstation-process-matched feature vectors, which significantly improves the adaptability, generalization, and subsequent discrimination accuracy of the model's feature input.

[0123] S3.5: Utilizing the workstation-process adaptation feature vector, based on the current workstation index and process parameter conditions, perform specific feature normalization and weighting processing to generate the final set of adaptation feature vectors for the input of the multi-decision sub-model. This ensures that the proposed features are fully compatible with the subsequent decision sub-model's discrimination requirements for different workstations and different process parameters, laying a solid feature foundation for the classification accuracy and confidence fusion strategy.

[0124] Step S4: The extracted feature data is input into an integrated multi-decision sub-model pool, which includes heterogeneous sub-models such as convolutional networks, graph structure models, and statistical classifiers. Each decision sub-model independently outputs the classification probability distribution of the defect type and the intermediate feature embedding. Specifically, this includes:

[0125] S4.1: The local texture features, edge structure features, and multi-scale process-related statistical features output from step S3 are used as input feature data. The feature vector standard formatting process is performed to ensure that it meets the data interface requirements of the multi-decision sub-model pool and provides a consistent and highly compatible input basis for the multi-model structure.

[0126] The station-process adaptation feature vector set output in the previous step S3.5 is used to standardize and unify the input interface of the multi-decision sub-model pool by adopting the feature vector standard formatting algorithm.

[0127] For each set of adapted feature vectors, a normalization method (such as Z-score normalization or min-max scaling, with parameters set according to model input requirements) is applied to perform a standardization transformation on all feature components, uniformly mapping their numerical range to a preset interval, eliminating numerical scale differences caused by changes in workstations or processes, and ensuring that different input batches have balanced distribution characteristics.

[0128] Furthermore, through feature type encoding algorithms, clearly defined context labels or channel index information are added to features from different sources (such as texture, structure, and statistics) to adapt to the data interface type requirements of subsequent convolutional networks, graph structure models, and statistical classifiers, ensuring that feature dimension differentiation and structural constraints are completed during the feature input stage.

[0129] By employing missing value imputation methods (such as mean imputation, zero-filling, or specific mapping values ​​of process parameters at the workstation) to address the issue of sample defects in feature dimensions under certain working conditions, missing feature items are automatically filled in, and a set of standardized feature vectors with high consistency and no redundancy is output.

[0130] Through the feature structure verification mechanism, based on the input requirements of each decision sub-model, the dimensional order is rearranged and the data structure consistency is checked on the standard formatted feature data to ensure that all input vectors strictly meet the multi-model parallel interface specification.

[0131] The standardized feature vector set that has passed the verification is batch-organized into a unified data input package (such as tensor format or matrix form) according to the model pool loading rules. This provides a consistent and highly compatible input foundation for multi-sub-model structures, enabling efficient data flow in the multi-source feature discrimination, heterogeneous model training and inference stages.

[0132] Through the above chain-style standard formatting process, complex feature data from multiple sources are standardized into a unified input format, enabling seamless integration and highly robust input of workstation-process features in the business context by the multi-decision sub-model pool.

[0133] For example, for a batch of defect images from a PCB packaging production line after normalization and enhancement processing, the station-process adaptation feature vector output in step S3.5 is 12-dimensional, including 4-dimensional local texture features, 3-dimensional edge structure features, and 5-dimensional multi-scale process statistical features. A min-max normalization algorithm is used, setting the standard interval to [0,1], to normalize all feature components:

[0134]

[0135] Where, x i For a certain characteristic component, x min and x maxThese represent the historical minimum and maximum values ​​for the component. For missing process statistical features, the mean of the same batch is used for interpolation to complete the input vector. After standard formatting and type label encoding, a set of structured feature vectors is output, containing normalized components with channel types such as {"texture-0.45", "structure-0.32", "statistics-0.66"...}. For the feature sets of all production line batches, vector alignment and batch processing are used to assemble them into a data input tensor compatible with multiple decision sub-models (batch size 64, feature dimension 12), which is then delivered to the subsequent S4.2-S4.4 multi-model structure. Through standard formatting, the consistency of feature input is guaranteed to be 100%, further improving the training and inference accuracy and discrimination robustness of the subsequent sub-model pool.

[0136] S4.2: Input the standardized feature data into the convolutional network structure, use the convolutional neural network to carry out feature depth representation and end-to-end discrimination, generate the corresponding defect type classification probability distribution and high-dimensional intermediate feature embedding for the input features, and enrich its high-level judgment semantics of local spatial patterns.

[0137] The workstation-process adaptation feature vector set after standard formatting is used as the input data of the convolutional network structure in batch format to meet the data interface specifications of convolutional neural networks (e.g., the input tensor dimension is batchsize×channel×height×width, the batch size is set according to hardware resources and inference efficiency, and the number of channels is consistent with the feature channels).

[0138] A multi-layer convolutional neural network (CNN) structure (parameters include a 3×3 kernel size, 32 / 64 / 128 filter kernels, a stride of 1, and padding of 1 pad) is used to extract deep features of the input workstation-process adaptation feature vector in the spatial channel. Local spatial correlation is captured by the first-layer two-dimensional convolution operation on the input feature tensor, as shown in the formula:

[0139]

[0140] in, The l-th convolutional layer outputs the feature value at position (i, j) of the k-th channel. The weights of the l-th layer convolutional kernel are... This is the input at position (i+u, j+v) of the m-th channel in the previous layer. C is the bias parameter. l-1 denoted as the number of channels in the previous layer, and a and b as the radii of the convolution window.

[0141] Furthermore, the convolutional output features are processed by batch normalization and activation functions (such as ReLU) to improve the network's convergence speed and its ability to represent nonlinear features. The formula is:

[0142]

[0143] Where, x i For the feature components within the current batch, μ B , denoted as batch mean and variance, respectively, and γ and β are trainable affine parameters.

[0144] By using multi-level convolution and pooling operations (such as max pooling kernel = 2×2, stride = 2), the spatial size is gradually reduced while important feature responses are preserved, thus achieving high-order feature extraction from local spatial patterns to global discriminative semantics.

[0145] A fully connected layer (parameter is the number of output dimension categories N) is used to perform linear mapping and comprehensive discrimination on the high-dimensional features extracted by convolution and pooling, generating the original score vector for each category of defects.

[0146] By applying the Softmax normalized activation function to the output scores of the fully connected layer, a probability distribution mapping is performed to obtain the classification probability distribution of the defect type.

[0147]

[0148] Among them, z j Let P(y) be the Logit score of the j-th type of defect, N be the total number of defect types, and P(y) be the Logit score of the j-th type of defect. j ) represents the classification probability of the j-th class.

[0149] Furthermore, the output features of designated intermediate layers (such as pooling layers or penultimate fully connected layers) in the convolutional backbone network are extracted and embedded as high-dimensional intermediate features to reflect the deep representation of the input features in the network's hidden space.

[0150] The above convolutional neural network algorithm transforms the surface attributes of the workstation-process adaptation feature vector into a high-discrimination deep feature space. It also enables accurate estimation of the probability distribution of defect type classification and output of deep feature embedding, providing rich high-level feature support for subsequent multi-model confidence fusion and interpretable discriminant analysis.

[0151] For example, a three-layer convolutional network is configured for the input tensor consisting of an RGB single-channel normalized defect image and 12-dimensional workstation-process adaptation features: the first layer has a kernel size of 3×3×12×32, the second layer has a kernel size of 3×3×32×64, and the third layer has a kernel size of 3×3×64×128. Each layer uses stride=1 and pad=1. Batch normalization and ReLU activation are applied after the output of each convolutional layer, and max pooling (2×2) is used. The feature tensor output after the last pooling layer is flattened and then fed into a fully connected layer whose output dimension is the number of defect categories (e.g., 8 categories) and softmax normalization to calculate the classification probability. In the experimental evaluation, 64 sets of feature vectors were input in batches, and the convolutional network outputs an 8-dimensional classification probability distribution corresponding to each set of inputs and embeddings 128-dimensional high-level intermediate features. For typical defect types such as scratches, corrosion, and misalignment, the convolutional network achieves a classification accuracy of 94.2% on the validation set. The intermediate layer features, after PCA mapping, are highly correlated with changes in process parameters. The outputs described above directly connect to subsequent confidence propagation and multi-model fusion stages, enabling dynamic adaptation and discrimination of local spatial features and process scenarios. This approach effectively enhances the expressive power of local spatial patterns in the automatic depth discrimination process, achieving the technical goals of high-performance PCB packaging defect type discrimination and high-confidence feature extraction.

[0152] S4.3: Parallel transfer of feature data to the graph structure model, use graph neural network to process process parameters and spatial correlations, perform graph embedding optimization on the complex interaction relationships between feature nodes, output defect type classification probability distribution and structured intermediate feature embedding that reflect the spatial distribution and correlation effects of defects, and enhance the robustness of spatial tracing.

[0153] The station-process adaptation feature vectors output from step S3.5 are processed using a graph neural network (GNN) method to construct a feature node graph structure, mapping each feature vector to a node in the graph.

[0154] Based on process parameters (such as station index, temperature, pressure, speed, etc.) and spatial relationships, an adjacency matrix A is constructed using adjacency relations. The connection weights between nodes are determined by the process correlation function f(x) between features. i x j This determines how to represent complex interactive relationships between nodes.

[0155] The Graph Convolutional Network (GCN) algorithm is applied (parameters include the number of layers L, node feature dimension d, and convolution weight matrix W). (l) (Activation function σ, batch normalization (BN), etc.) are used to perform the following feature embedding update:

[0156]

[0157] Among them, H (l) This represents the feature representation of the l-th layer node. W is an adjacency matrix with self-loop normalization. (l) Let H be the weight matrix of the l-th layer. Through layer-by-layer graph convolution operations, high-order relationships between feature nodes are modeled, and a high-dimensional structured intermediate feature embedding H is obtained. (L) .

[0158] Furthermore, by leveraging the message passing mechanism between process parameter nodes and spatial nodes, an attention mechanism (such as Graph Attention Network, GAT) is employed to dynamically weight the information transmission between nodes, thereby enhancing the ability to discriminate regions that are sensitive to inter-node dependencies. The node attention coefficient α... ij Calculate using the following formula:

[0159]

[0160] Where a is a learnable weight vector, W is a linear mapping, and h i h j For features of adjacent nodes, || denotes vector concatenation. Let i be the neighborhood of node i.

[0161] A pooling readout algorithm (such as global average pooling, max pooling, or a specific aggregation function) is used to downsample the node embeddings of the entire graph to generate a structured graphical feature representation z for overall defect type determination.

[0162] The graph embedding feature z is input into a fully connected classifier layer, and a Softmax normalization function is used to output the classification probability distribution of the defect type:

[0163]

[0164] Among them, w k b k The weights and biases for category k are given. The output discrimination results include the defect type classification probability distribution and the extracted high-dimensional structured intermediate feature embeddings.

[0165] By using the above graph neural network processing flow, process parameters and spatial relationships are modeled as priors for discrimination. This not only effectively captures the impact of process heterogeneity on defect distribution, but also significantly improves the robustness and accuracy of spatial source tracing discrimination.

[0166] For example, for batch-collected normalized PCB packaging defect images, the station-process adaptation feature vector with a feature dimension of 7 is mapped to a graph structure. Each batch of samples constructs an undirected weighted graph containing 7 nodes. The node weights are mapped to the normalized values ​​of process parameters such as temperature and pressure, and the edge weights are the inverse relationship of the station spatial distance.

[0167] A three-layer GCN network is used, with 7 dimensions for node feature input, 16 and 32 hidden layers respectively, and ReLU activation function. The convolutional weights are initialized to a Gaussian distribution with a mean of 0 and a standard deviation of 0.02. The adjacency matrix uses a process relevance threshold of 0.5, and the node connection probability is 0.8. Message passing weighting uses the GAT mechanism, with 4 attention heads and a LeakyReLU slope of 0.2.

[0168] During training, the cross-entropy loss function is used, the learning rate is 0.001, the Adam optimizer is employed, and the batch size is 64. Every 100 iterations, the structured embedding H is output. (L) The graph embedding features were obtained through global average pooling. The softmax classification accuracy for defect types improved to 97.2% on the validation set, while the mean spatial origin discrimination error decreased to 1.8%. The high-dimensional embedding vectors serve as important bases for confidence interval estimation and subsequent feature interpretability analysis. The final output includes class probability distributions and a 128-dimensional structured embedding, significantly improving the defect type discrimination capability and robustness in complex process scenarios.

[0169] S4.4: The above feature data is synchronously input into a statistical classifier, such as a Bayesian classifier or a support vector machine. Based on the statistical learning framework, feature space probability modeling is performed, and the probability distribution of defect type classification and low-dimensional key feature embedding based on statistical parameters are output to help capture globally distributed defect features.

[0170] S4.5: Collect the defect type classification probability distribution and intermediate feature embeddings output by the above heterogeneous sub-model convolutional network structure, graph structure model and statistical classifier respectively, to form a multi-source, heterogeneous decision sub-model output matrix, which provides fine-grained, multi-dimensional features and decision basis for subsequent confidence propagation and information fusion.

[0171] Step S5: Based on the classification probability distribution and intermediate feature embeddings output by each decision sub-model, a confidence propagation mechanism is used to fuse multi-source information and generate a comprehensive defect type confidence interval for each defect type under the given workstation and process parameters. Specifically, this includes:

[0172] S5.1: The classification probability distribution information output by each decision sub-model is standardized according to a unified defect type labeling system to obtain an additive standardized classification probability matrix for each workstation and process parameter, which is used for subsequent multi-source decision data aggregation.

[0173] S5.2: Obtain the intermediate feature embedding vectors output by each decision sub-model, and perform feature space renormalization processing on them to make all intermediate feature embeddings uniformly mapped to the same process-related feature space, so as to ensure the equivalence comparison and information fusion of subsequent multi-source features.

[0174] The intermediate feature embedding vectors output by each decision sub-model are used as input, and the feature mapping renormalization method (parameters: normalization interval, process feature weight vector) is adopted to achieve unified spatial adjustment of feature distribution among different models.

[0175] The max-min normalization algorithm is used to normalize each component of the intermediate feature embedding vector to the target interval ([0,1]) to eliminate the scale mismatch caused by heterogeneous models. The specific implementation is as follows:

[0176]

[0177] Where, x i,j Let x_i be the j-th feature component of the i-th decision sub-model, and (\min(x_{i,j})) and (\max(x_{i,j})) represent the minimum and maximum values ​​in the historical data.

[0178] Furthermore, through the linear space mapping method (parameter: mapping matrix W) align The target process-related feature vectors are projected onto a unified process-related feature space to align the feature dimensions between different sub-models. The calculation is as follows:

[0179]

[0180] Among them, f i norm The normalized feature vectors, Let be the spatial alignment mapping matrix corresponding to the i-th sub-model.

[0181] Principal component analysis (PCA) (parameter: number of principal components k) was used to reduce the dimensionality of the aligned feature data, and the principal components with the strongest process specificity were selected to obtain a unified feature subspace representation. The processing procedure is as follows:

[0182] f i pca =PCA k (f i aligned )

[0183] Among them, PCA k This represents the mapping vector after PCA extracts the first k principal components.

[0184] Furthermore, the feature consistency test method (parameter: cosine similarity threshold θ) is used.sim This involves comparing the consistency between feature vectors projected from different sub-models using PCA, filtering out features with low consistency, and enhancing the equivalence of multi-source features.

[0185]

[0186] Through the above series of feature renormalization processes, the intermediate features output by all decision sub-models are embedded into a unified expression of the same process-related feature space, and a standardized multi-source feature embedding vector is output. This lays the foundation for subsequent confidence propagation and joint information fusion, and enables equivalence comparison, feature contribution correlation analysis and effective data coupling between features of heterogeneous models.

[0187] For example, the high-dimensional 128-dimensional intermediate feature embedding output by the convolutional neural network of the PCB packaging production line, the 64-dimensional feature embedding output by the graph structure network, and the 32-dimensional low-dimensional feature output by the statistical classifier are all normalized to the range of [0, 1] using the minimax normalization algorithm. The historical minimum and maximum value intervals are obtained by statistical analysis, and the feature vector of the current batch of a single production line is processed according to the normalization formula.

[0188] Subsequently, for each sub-model, the pre-trained linear mapping matrix W is loaded. align (Sizes are 32×128, 32×64, and 32×32 respectively), aligning all three types of features to a unified 32-dimensional representation. Using the PCA algorithm, with a principal component count of 20, feature projection is performed on batch data, and principal components with a cumulative variance contribution rate of over 95% are selected to form a principal component sequence.

[0189] Cosine similarity between features was calculated for all principal component results, with a threshold of 0.96, retaining only high-similarity feature combinations. This resulted in a unified 32-dimensional process-related aligned feature set. In subsequent Bayesian joint modeling and multi-source belief propagation inference, structural consistency and equivalence comparison of feature inputs were achieved. This process improved the discrimination accuracy and robustness of belief inference after feature fusion, theoretically reducing errors caused by feature space inconsistencies by at least 10%. In actual production line verification, the accuracy increased to 97.8%, and data flow compatibility and spatiotemporal traceability were significantly enhanced.

[0190] S5.3: Construct a multi-source confidence statistical model, utilize the standardized classification probability matrix and the renormalized intermediate feature embedding, and jointly model the probability distribution of each defect type based on the Bayesian confidence propagation mechanism to obtain the prior confidence estimate of each defect type under the fusion of multiple decision sub-models.

[0191] Using the intermediate feature embeddings of each decision sub-model after feature space renormalization in step S5.2 and the standardized classification probability matrix output in S5.1 as input, multi-source confidence statistical modeling technology is used to systematically realize the prior confidence fusion of each defect type under the current workstation and process parameter conditions.

[0192] Using conditional probability statistical modeling, the standardized classification probability matrix of the multi-decision sub-model obtained in S5.1 is denoted as P. m (y = k|x), where m is the index of the decision sub-model, y = k is the defect type label, and x is the workstation-process feature input vector. By aggregating the probability information of the decision outputs of all sub-models, a preliminary joint probability distribution for multi-source defect type determination is constructed.

[0193] Furthermore, for the renormalized intermediate feature embeddings output by each decision sub-model S5.2, a feature-weighted statistical modeling method is adopted, and weighting coefficients are set. This measures the feature dependency of each sub-model under a specific defect type. The joint probability distribution is then weighted and corrected using the following weighted combination formula:

[0194]

[0195] Where M represents the total number of decision sub-models. The settings are adaptively configured based on the statistical correlation between intermediate feature embeddings and category labels or the performance metrics of the model validation set.

[0196] A Bayesian network confidence propagation algorithm (parameters: nodes represent the output probabilities of each model, edge weights represent mutual information) is used to perform Bayesian confidence inference on the above weighted joint probability distribution. The output probabilities of different sub-models are interpreted as conditionally independent observations about defect type events, and a comprehensive prior confidence score is obtained. The confidence scores of the multi-model observation data are normalized using the following Bayesian formula:

[0197]

[0198] Where O = {O1, O2, ..., O} m} represents the observed output of each sub-model for the current sample, and P(y=k) is the prior probability, which can be estimated from historical statistical frequencies.

[0199] Furthermore, the joint distribution is normalized using the maximum a posteriori estimation (MAP) method to obtain a normalized prior confidence vector, and the full model fusion prior confidence score of each defect type under the current workstation and process parameters is determined accordingly.

[0200] The confidence consistency test module compares the prior confidence with the historical workstation-process defect label distribution and outputs the confidence consistency coefficient Q (e.g., Pearson correlation coefficient or KL divergence) to help evaluate the statistical robustness and reliability of the fusion results.

[0201] By using a multi-source Bayesian confidence propagation mechanism, the probability distribution and intermediate feature information output by each decision sub-model are unified and integrated to output the prior confidence estimate of each defect type under the fusion of multiple decision sub-models, providing a solid probabilistic foundation for subsequent conditional probability inference and final confidence interval generation for workstation-process parameter conditions.

[0202] For example, in a typical PCB packaging production line application scenario, three seed models—a convolutional network model, a graph structure model, and a Bayesian classifier—are selected. These models are used to batch-process samples with workstation number A05, a process temperature of 160℃, and a packaging pressure of 0.38MPa, obtaining three normalized classification probability distributions: P1(Defect A, 0.82), P2(Defect A, 0.77), and P3(Defect A, 0.70). Weights w1 = 0.43, w2 = 0.35, and w3 = 0.22 are adaptively set based on intermediate features and historical performance.

[0203] According to the weighted combination formula, the probability after fusion is calculated as follows:

[0204]

[0205] Using the Bayesian confidence propagation algorithm, the historical prior P(A) of defect A was set to 0.30. The observations of each sub-model were substituted into the conditional probability table, and the prior confidence score P(y = A|O) was calculated to be 0.75. The confidence consistency test yielded Q = 0.96, which is highly consistent with the historical data, indicating the stability and usability of the fused confidence score.

[0206] The final output shows that the fusion prior confidence level of defect A under the given station-process conditions is 0.75, providing a quantitative basis for downstream confidence interval estimation and process anomaly response.

[0207] S5.4: Using the Bayesian confidence propagation mechanism, the station index and process parameter information are used as context conditions to perform conditional probability inference, realize the dynamic weighted fusion of the confidence of multiple decision sub-models, and output the comprehensive defect type posterior confidence distribution for a single station and process parameter scenario.

[0208] S5.5: Based on the fused posterior confidence distribution, a confidence interval statistical algorithm (such as the quantile method or confidence band method) is used to generate the comprehensive defect type confidence interval for each defect type under the work station and process parameters. This interval serves as a quantitative output for determining credibility, providing standard data for subsequent interpretable decision-making and dynamic anomaly response.

[0209] Step S6: Perform interpretable feature contribution analysis on the confidence interval of the fused comprehensive defect type. Using feature integration or Monte Carlo sampling methods, output the main process features and key image regions that lead to the defect classification results, improving the interpretability of the judgment results. Specifically, this includes:

[0210] S6.1: The confidence interval of the integrated defect type after fusion is loaded into the interpretability algorithm entry function to extract the confidence interval to be analyzed and its corresponding judgment feature embedding, and output the dataset to be processed for interpretable analysis.

[0211] S6.2: Based on the feature integral mechanism, calculate the feature importance score of each process feature to the confidence interval of the comprehensive defect type for the dataset to be processed in interpretability analysis, so as to obtain the ranking result of the contribution of the main process features.

[0212] The dataset to be processed, which is the interpretability analysis output of S6.1, is used as input. It includes the confidence interval of the integrated defect type and the corresponding judgment feature embedding.

[0213] A feature integration algorithm (parameters: process feature set, feature importance baseline, integration step size) is used to quantify the feature importance of each process feature to the confidence interval of the comprehensive defect type. The process feature vector f = (f1, f2, ..., f...) is selected. n ), targeting defect type y * Using the feature closed interval [reference baseline, actual value] as the path, a feature path sequence is generated through step sampling. For each feature i, the feature integral contribution is calculated as follows:

[0214]

[0215] Among them, f i base Let be the baseline reference value for feature i, F[·] be the scoring function for the confidence interval of the comprehensive defect type, and α be the proportional parameter in the integral path.

[0216] Furthermore, the Gauss-Legendre numerical integration method (parameters: sampling node k, integration weight table) is used to perform discrete approximation of the feature integral, achieving efficient calculation of feature importance scores. For each feature i, sampling node α is generated. j (j = 1, 2, ..., k), we obtain the integral expression:

[0217]

[0218] Among them, w j For numerical integration weights, It can be instantiated using automatic differentiation tools (such as the backpropagation algorithm).

[0219] Furthermore, by using a feature contribution normalization algorithm (parameter: maximum value normalization), all feature integral values ​​IGe are normalized to ensure the comparability of the ranking of each process feature contribution. The calculation formula is as follows:

[0220]

[0221] For all process features {f i The ranking outputs a list of the main process features' contributions, achieving a structured representation of the importance of multidimensional features.

[0222] Through the above chain derivation process, the comprehensive confidence interval and process features are modeled one by one. The system realizes the quantitative ranking of the main process feature contribution of each type of defect, providing standard input for subsequent feature sensitivity analysis and significant region discrimination.

[0223] For example, for the defect type A confidence interval I = [0.68, 0.77] under the process parameters of station A05, temperature 160℃, and pressure 0.38MPa, the normalized process feature vector (0.62, 0.88, 047) is selected, with the baseline f base = (0, 0, 0), Gauss-Legendre sampling nodes k = 4, weights (0.3479, 0.6521, 0.6521, 0.3479). Automatic differentiation of the confidence scoring function yields the integral contribution values ​​of each feature: IG1 = 0.084, IG2 = 0.115, IG3 = 0.074, after normalization... The ranking result is Feature 2 > Feature 1 > Feature 3. This result indicates that process feature 2 (such as real-time pressure) has the greatest impact on confidence determination and can provide key contribution basis for subsequent sensitivity sampling and interpretable heatmap generation.

[0224] S6.3: Using the ranking results of the contribution of the main process features output by the feature integral as input, the Monte Carlo sampling algorithm is used to collect sample feature changes under multiple decision sub-model paths, evaluate the sensitivity interval of the main process features to the classification confidence judgment, and output the confidence interval stability index of each feature.

[0225] S6.4: Based on the contribution and stability index of process features obtained by feature integration and Monte Carlo sampling, an interpretability saliency map is generated for the input judgment image. The key image regions that affect the comprehensive confidence judgment are thermally annotated to form a visual feature contribution map.

[0226] S6.5: Integrate the ranking results of process feature contribution, the stability index of feature confidence interval, and the thermal annotation of key image regions, and output the fusion results as an interpretable feature contribution analysis report, providing standardized and structured interpretable data for subsequent defect decision transparency, anomaly judgment review, and source tracing analysis.

[0227] Step S7: Link the comprehensive defect type confidence interval and the interpretability result with the real-time process data of the production line, and based on the workstation index and timestamp, realize the instant output of defect category, defect occurrence time, and spatial location. Specifically, this includes:

[0228] S7.1: Using the confidence interval of the integrated defect type and the interpretability result of the judgment as input, based on the workstation index and timestamp, perform multi-source data structure mapping processing with the real-time process data of the production line to form an integrated data body covering the judgment result, process parameters, workstation information and time sequence identifier, and build an integrated data foundation for subsequent spatiotemporal positioning output.

[0229] S7.2: Using the integrated data volume as the execution object, a high-precision time series synchronization algorithm is applied to strictly align the confidence interval of the comprehensive defect type with the process parameter flow of the production line according to the timestamp, ensuring that each judgment result can be uniquely mapped to the corresponding physical node and process status of the actual production line, and outputting a precise mapping spatiotemporal index information pair.

[0230] S7.3: Based on spatiotemporal index information pairs, a defect category distribution archiving algorithm is adopted to classify and archive the confidence intervals and interpretability results of comprehensive defect types generated at different times and different workstations, generating a multi-dimensional defect record table containing defect category, occurrence time, and workstation coordinates, providing a structured data interface for realizing rapid retrieval and source tracing.

[0231] Using the output spatiotemporal index information as the input data, the defect data distribution classification and archiving algorithm (parameters: workstation index, timestamp, defect category label, comprehensive defect type confidence interval, interpretability result) is first used to classify and aggregate the judgment results generated at different workstations and at different time points.

[0232] Furthermore, through an index retrieval mechanism, the confidence intervals and interpretability analysis results of the comprehensive defect types under all spatiotemporal pairs with unique identifiers are stored hierarchically according to defect category labels, realizing the typological archiving of defect events. Each data unit includes, but is not limited to, structured content such as defect category, workstation index, timestamp, comprehensive confidence interval, main judgment features and feature contribution, and key image region annotations.

[0233] Furthermore, using a multidimensional grouping algorithm, a multidimensional mapping table is constructed for archived defect events based on triples of (defect category, workstation coordinates, and occurrence time), thereby generating a defect event record table reflecting the defect distribution pattern. The data organization of the multidimensional mapping table adopts a normalized data structure design to ensure that various types of defect events can be efficiently retrieved, edited, and subjected to batch traceability analysis.

[0234] Subsequently, through the data integrity and consistency verification mechanism, redundant entries, abnormal markers, and spatiotemporal information anomalies in the defect record table are screened and corrected to ensure the uniqueness and accuracy of all data units, thereby further improving the accuracy of source tracing.

[0235] Finally, after the above processing, the multidimensional defect record table is output in a structured data interface format, providing standard data support that can be directly accessed for subsequent defect retrieval, trend statistics, responsibility tracing and production line management.

[0236] By using a defect category distribution archiving algorithm, the spatiotemporal mapping judgment result obtained in the previous step is transformed into a standardized multidimensional defect record table containing defect category, occurrence time, and workstation coordinates, thereby achieving efficient archiving and rapid retrieval of defect events across the entire production line.

[0237] For example, in the actual PCB packaging production line environment, the normalized comprehensive defect type confidence interval data is grouped in multiple dimensions according to the workstation index range (e.g., workstations 1 to 12), timestamp resolution (seconds), and defect type labels (e.g., short circuits, missing solder, foreign objects, scratches) through a distributed classification archiving algorithm. The data archiving uses a MySQL relational database, with table structures including fields: defect_type, workstation_id, event_timestamp, confidence_interval, feature_importance, and region_heatmap. When a short circuit defect occurs at the corresponding workstation and the confidence interval is between 0.88 and 0.96, the system records the judgment result along with the interpretability region heatmap and the main contributing process features. The average archiving and retrieval latency for the entire batch of defect events is less than 100ms. For workstation switching during process adjustments, the algorithm can accurately distinguish and store the data, automatically avoiding cross-workstation information confusion, achieving multi-dimensional traceability of defect distribution history, and supporting subsequent defect trend analysis and anomaly risk tracking.

[0238] S7.4: Apply a spatial correlation localization algorithm to the multidimensional defect record table, comprehensively determine the contribution of key image regions and process features in the interpretability results, perform spatial hot zone visualization assignment for each defect event, and output the instantaneous spatial localization results of defect category, occurrence time and spatial hot zone in combination with real-time process data.

[0239] S7.5: Using real-time spatial positioning results as input, it links with the production line central monitoring system and adopts workstation distribution visualization and time trajectory tracing algorithms to realize dynamic chart display of defect events on the production line layout, and pushes defect category, key features and spatial coordinate information to the operation console to support the output of real-time decision-making information for dynamic defect feedback and responsibility tracing on the production line.

[0240] Step S8: Determine whether the confidence interval of the comprehensive defect type exceeds a preset threshold. If it does, immediately trigger the production line packaging process error reporting mechanism and push a detailed and interpretable auxiliary decision-making information interface to support manual verification and re-judgment. Specifically, this includes:

[0241] S8.1: Perform threshold judgment processing on the comprehensive defect type confidence interval data of the fusion output to determine the current risk level of defect identification for each category, and limit the judgment object to the comprehensive defect type confidence interval under different workstations and process parameters. This enables comparative analysis of the data fused from the discriminant model with the set threshold, and generates dynamic judgment labels.

[0242] S8.2: Based on the judgment label that the confidence interval of the comprehensive defect type exceeds the preset threshold, the production line automatic control protocol is adopted to generate an instant error signal for the production line packaging process, and the judgment label and the error signal are coupled to realize the automatic abnormal alarm output for the current workstation and related process numbers, and establish an event triggering mechanism for subsequent multi-dimensional abnormal investigation.

[0243] S8.3: For judgment labels that require manual intervention, perform an interpretability judgment feature data query operation to retrieve the feature integral contribution ranking, confidence interval change trajectory and main judgment influence area heat map generated in the previous steps, forming a structured interpretable auxiliary decision information package, providing rich and transparent decision basis output for downstream auxiliary verification interfaces.

[0244] Using production line error judgment labels and associated confidence intervals as input, an interpretable feature query algorithm is adopted to call the feature integral contribution ranking results generated in the previous steps, so as to realize the importance sequence retrieval of each process parameter and image feature in the current defect category judgment.

[0245] Furthermore, by using the confidence interval trajectory backtracking module, the historical changes of the confidence interval of the comprehensive defect type are tracked, and the evolution sequence data of the confidence interval related to this error judgment are extracted, so as to realize the dynamic history of the model's judgment credibility.

[0246] Furthermore, using a regional saliency heatmap generation algorithm, the key determination regions of the target image are extracted using the feature spatial distribution of the main determination influence regions as an index, and a heatmap distribution image based on pixel intensity and feature contribution is generated.

[0247] Furthermore, by summarizing the above-mentioned feature integral contribution ranking data, confidence interval change trajectory data, and heat maps of the main affected areas, a structured auxiliary information packaging algorithm is adopted to uniformly encode the data source, feature content description, and confidence analysis metadata, forming a standardized and interpretable auxiliary decision-making information package.

[0248] This information package structure enables precise data-driven decision-making support for downstream manual verification interfaces, improving the transparency and operational efficiency of verification and review.

[0249] For example, when identifying a "pad contamination" defect at a certain workstation on a PCB packaging production line, the error judgment confidence interval is [0.78, 0.94]. The main influencing process parameters are "pressure" and "speed," with feature integral contributions of 0.42 and 0.32 respectively, and the remaining parameters are below 0.1. The confidence interval change trajectory shows that the confidence level of the same batch increased from 0.65 to 0.88 before and after the adjustment of the process parameters. Using a heatmap algorithm, the central 15% area of ​​the image was identified as the main contamination judgment area, with a significance weight of 0.72. All the above data were encoded with structured information packets and pushed to the manual verification terminal to assist operators in reviewing the transparency of the defect decision-making process and ensure the accuracy of the basis for subsequent tracking and model optimization.

[0250] S8.4: The above-mentioned interpretable auxiliary decision-making information package is pushed to the control terminal interface of the designated workstation through the production line human-machine interaction system. Based on the defect type, confidence interval characteristics and influencing factors, it automatically associates process parameter details to realize multi-view interpretable auxiliary presentation of complex judgment scenarios, supporting front-line operators to perform efficient review and verification.

[0251] S8.5: The event logging algorithm is used for the feedback data of the results of manual verification. It combines the confidence interval of the comprehensive defect type, the judgment timestamp and the manual verification opinion to automatically complete the data closed-loop archiving of this judgment process, forming a traceable and highly consistent record of the entire process of defect judgment and response, providing standard database support for subsequent source tracing analysis and dynamic optimization of model thresholds.

[0252] Step S9: Automatically record various process parameters, comprehensive defect confidence intervals, interpretable feature contribution results, and corresponding spatiotemporal labels from the judgment process, and use them for subsequent defect source tracing analysis, process trend statistics, and dynamic adaptive iteration of the model. Specifically, this includes:

[0253] S9.1: Based on the data structuring algorithm, process parameters, comprehensive defect confidence intervals and interpretable feature contribution results obtained after completing the defect classification and judgment process are processed in a unified format to form a standardized record entity of the judgment process containing fields such as original judgment label, judgment probability distribution, main feature contribution degree and decision threshold status, so as to lay a data foundation for subsequent archiving and retrieval.

[0254] S9.2: For each set of judgment data stored in the structured storage in S9.1, perform marking and binding operations based on timestamps and workstation indexes. Use a spatiotemporal tag automatic coding mechanism to map a single judgment result to a specific spatiotemporal node in the production line process, thereby realizing the digital coupling of defect classification judgment with the entire spatiotemporal process of production.

[0255] S9.3: Based on the structured multidimensional judgment data obtained in S9.2, an automatic data archiving and redundancy verification algorithm is applied to perform redundancy elimination, version comparison and integrity checks on the generated structured data entities to ensure the data consistency and traceability of the judgment records, and to provide a stable standard data archiving unit for subsequent source tracing analysis.

[0256] S9.4: For the archived structured judgment records, the process trend mining algorithm is used to dynamically analyze the time series changes of batch defect confidence interval and key feature contribution results, and output process parameter fluctuation, defect frequency distribution and main influencing factor trend information, providing production line managers with trend diagnosis and traceability analysis reports based on big data support.

[0257] S9.5: Utilizing the above process trend analysis and structured judgment archiving results, initiate the model's dynamic adaptive iteration process. Based on historical misjudgment samples, abnormal trend fluctuations, and changes in feature contributions, automatically select representative training samples and fine-tune and retrain the parameters of the comprehensive defect judgment sub-model online. Output the iteratively optimized multi-judgment sub-model structure to achieve self-evolution and robustness enhancement of the defect classification and discrimination system.

[0258] For those skilled in the art, various other corresponding changes and modifications can be made based on the technical solutions and concepts described above, and all these changes and modifications...

[0259] All variations should fall within the protection scope of the claims of this invention.

[0260] Unless otherwise defined, the technical or scientific terms used herein shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.

[0261] The terms "first," "second," "third," and similar words used in the specification and claims do not indicate any order, quantity, or importance, but are merely for illustrative purposes.

[0262] To distinguish different components. Similarly, words like "a" or "one" do not indicate a quantity limitation, but rather the presence of at least one. "Including" or "package"

[0263] Words like "include" or "contain" indicate that the elements or objects preceding "include" or "contain" encompass the elements or objects listed following "include" or "contain".

[0264] The term "component" and its equivalents do not exclude other elements or objects. "Multiple" in the embodiments of this application refers to two or more. "A and / or B" indicates three possibilities: A; B; and A and B.

[0265] The above description is merely an exemplary embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and such modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A deep learning method for online PCB packaging defect identification, specifically including: S1: Acquire defect image data and corresponding real-time process parameters collected from multiple workstations on the PCB packaging production line, and add workstation index and timestamp information to each collected data. S2: Perform normalization and data augmentation processing on the acquired defect image data, and formulate different data augmentation strategies based on the diversity of workstation index and process parameters; S3: Extract local texture features, edge structure features, and multi-scale process-related statistical features from the normalized and enhanced defect image data, respectively; A texture extraction algorithm is applied to the normalized and enhanced defect image data to obtain local texture feature parameters and thus a local texture feature matrix. Based on the extracted local texture feature matrix, an edge detection algorithm is used to analyze the edge structure of the defect image, identify and output the edge structure feature vector; Using the edge structure feature vector as input, multi-scale feature decomposition is performed to extract statistical feature parameters containing different spatial levels from low frequency to high frequency, forming a multi-scale process-related statistical feature set; The multi-scale process-related statistical feature set is fused with the original output local texture feature matrix and edge structure feature vector at the feature level to obtain the workstation-process adaptation feature vector. Based on the workstation-process adaptation feature vector, and according to the current workstation index and process parameter conditions, feature normalization and weighting processing is performed to generate the final set of adaptation feature vectors for input to the multi-decision sub-model. S4: Input the extracted feature data into the integrated multi-decision sub-model pool, and each decision sub-model independently outputs the classification probability distribution of the defect type and the intermediate feature embedding; S5: Based on the classification probability distribution and intermediate feature embedding output by each decision sub-model, multi-source information fusion is performed to generate the comprehensive defect type confidence interval for each defect type under the corresponding workstation and parameters. S6: Perform interpretable feature contribution analysis on the confidence interval of the fused comprehensive defect type, and output the process features and key image regions that lead to the defect classification results; The confidence interval of the integrated defect type after fusion is loaded into the interpretability algorithm entry function and the data set to be processed is output for interpretable analysis. Based on the feature integral mechanism, the feature importance score of each process feature to the confidence interval of the comprehensive defect type is calculated for the dataset to be processed in interpretability analysis, and the ranking result of the contribution of process features is obtained. Using the ranking results of the contribution of process features output by feature integral as input, the changes of sample features are collected under multiple decision sub-model paths to evaluate the sensitivity range of process features to classification confidence judgment, and output the stability index of the confidence interval of each feature. Based on the obtained process feature contribution and stability index, an interpretability saliency map is generated for the input judgment image, and the key image regions that affect the comprehensive confidence judgment are thermally annotated to form a visual feature contribution map. S7: Link the confidence interval of the comprehensive defect type and the interpretability result with the real-time process data of the production line to output the defect category, defect occurrence time and spatial location in real time; S8: For the confidence interval of the comprehensive defect type exceeding the preset threshold, the production line packaging process error reporting mechanism is triggered immediately, and a detailed and interpretable auxiliary decision-making information interface is pushed. S9: Automatically record various process parameters, comprehensive defect confidence intervals, interpretable feature contribution results and corresponding spatiotemporal labels of the judgment process, and perform adaptive iteration.

2. The deep learning method for online PCB packaging defect identification according to claim 1, characterized in that, Step S1 specifically includes: Synchronously trigger the image acquisition units at each designated workstation on the PCB packaging production line to perform multi-channel defect image acquisition operations on the products being transported on the packaging line; The operation process parameter acquisition terminal automatically acquires the real-time PCB packaging process parameters of each station in each acquisition cycle, and integrates the real-time process parameter stream with the current image acquisition event in time synchronization to form the original process parameter data pool. Automatic feature labeling is performed on the defect image data and corresponding real-time process parameter data collected at each workstation. Based on the workstation index and timestamp, and following the product material flow path, real-time data archiving and redundancy verification processing are performed on multi-source original defect image data and real-time process parameter data from different workstations and batches, and the continuously collected data streams are aggregated into a structured dataset.

3. The deep learning method for online PCB packaging defect identification according to claim 2, characterized in that: The The acquisition of defect images is triggered synchronously by multiple workstations and adopts a multi-channel vision acquisition architecture, including a linear CCD camera, an area array CMOS camera and an auxiliary light source unit. The original images are transmitted back in real time via high-speed industrial Ethernet, and the integrity of the data packets is verified by CRC check or MD5 digest algorithm.

4. The deep learning method for online PCB packaging defect identification according to claim 1, characterized in that, The Step S2 specifically includes: For each acquired defect image data, based on its workstation index and real-time process parameters, workstation-specific normalization parameters are obtained, and the pixel values ​​and related channel information of the defect image are normalized. For normalized defect image data, based on the workstation index and process parameters, the distribution information of typical defect samples under the corresponding working conditions is retrieved, and diversified enhanced samples are generated on the basis of the original samples. Based on the defect image set after normalization and data augmentation, the distribution of defect category samples under each workstation and process parameter combination is statistically analyzed, and a multi-workstation, multi-process parameter defect image training set with balanced distribution is generated. For all processed data, the station index and process parameter labels are used to record and output the normalized and enhanced defect image data package.

5. The deep learning method for online PCB packaging defect identification according to claim 4, characterized in that: The data augmentation employs multi-scale rotation, mirror flipping, local random perturbation, and lighting reconstruction algorithms. For imbalanced workstation and process samples, sample balancing is achieved through oversampling or downsampling, and process-related noise and pseudo-defects are injected as needed.

6. The deep learning method for online PCB packaging defect identification according to claim 1, characterized in that: The Multi-scale process-related statistical features were extracted using a multi-resolution wavelet transform method, and the mean, standard deviation, and other statistical values ​​were obtained for each sub-band. Energy and entropy parameters form a multi-scale feature set of hierarchy, direction, and index.

7. The deep learning method for online PCB packaging defect identification according to claim 1, characterized in that: The multi-decision sub-model pool includes at least a convolutional neural network, a graph neural network, and a statistical classifier, which are used to output the classification probability distribution and intermediate feature embedding, and input the features into each decision sub-model after normalization, formatting, and label encoding.

8. The deep learning method for online PCB packaging defect identification according to claim 1, characterized in that: The output of the decision sub-model is unified to the same process-related feature space through max-min normalization, linear space mapping and principal component analysis, and the decision results are dynamically weighted and fused to output the posterior confidence distribution and interval of the comprehensive defect type for the current workstation and process situation.

Images